The Data Organization Spectrum
Read more about the framework for data organization, as published in The Data Administration Newsletter.
When trying to build out a data organization, it is important to consider how people will be interacting with data. Will many people have access to the data or only a few? How will this affect the way that workers answer questions with the data? How will it affect the way data is governed at the company?
It is important that these questions be answered deliberately when setting a new direction for your data culture. Without proper consideration of the data organization’s structure, you can end up with poor data culture, resulting in duplicated analyses, inconsistent metrics, overloaded analysts, among many other issues.
In this article, I introduce and define Centralized, Democratized, and Federated organizational structures and provide a few of the benefits and challenges of each. These organizational structures exist on a spectrum between Centralized and Democratized, as seen in the diagram below. For ease of understanding concepts and definitions, I will first write about Centralized, followed by Democratized, and finally Federated.
Centralized
A Centralized data organization is characterized by all data governance and analysis occurring on a single team (which I call the “Central Data Team”). The Central Data Team is the keeper of data and associated knowledge. They are experts on the metrics, the data tools, and how to use the data responsibly. I have seen centralization work in small companies where there are very few people trained in data analysis. Those on the Central Data Team have high expertise on both the best data practices and the company’s specific data.
Benefits
It is easy to promote responsible use of data among a Central Data Team. They were all likely hired specifically to do a data role, which means that they are interested in developing or already have data skills. They are likely comfortable with data and take an interest in data governance and responsibility, as well as have more bandwidth and training to conduct complex analyses. Additionally, as a part of the same team, they are likely to communicate frequently. This encourages shared documentation platforms, a similar understanding of metrics, and a common vocabulary. People are likely working on similar projects, so they are able to collaborate and share knowledge across the team.
Challenges
While Central Data Teams are experts on data, but not on the questions that the data needs to answer. They need to be brought up to speed on the business issues, understand how stakeholders will interact with the data, and recognize the expected goals and actions from the information. The other major drawback of a Central Data Team is that they are expected to service and prioritize data needs across the entire organization. Does the analysis for the marketing campaign need to be completed before the Sales Results report? Many competing requests often result in a long backlog, meaning that Central Data Teams are less responsive and agile than their stakeholders would hope.
Mitigation
One strategy to mitigate these challenges is to increase the Central Data Team’s business knowledge, ensuring that they are up to date on the problems, goals, and actions of the business. Have them meet with people in various parts of the business to have a better understanding of jobs across the organization. The Central Data Team benefits from being informed about business context in two major ways. First, they are able to quickly be caught up on any problem they are asked to solve. The up-front investment in business understanding allows them to move more quickly. Second, increased business understanding leads to increased user empathy; Data Teams will be better able to customize dashboards and analyses to stakeholder needs.
Democratized
A Democratized data organization is characterized by getting data into the hands of those who are closest to the problem. In its purest form, this means allowing everyone to have access to the data and use it to research and solve the issues closest to them. Democratized data organizations also have Central Data Teams, but such teams are usually small and in charge of data governance and culture: getting the data into a usable format, moderating data conversations, and being available for questions. When I worked for a small company, we had a Democratized data organization. The Central Data Team worked on ETL, data governance, and education, but all analyses were done by people across the organization. Non-technical people were able to access data through Business Intelligence tools such as Tableau and Power BI, and were empowered to answer their own business questions with data.
Benefits
Challenges
Mitigation
Federated
A Federated data organization is characterized by allowing people in a variety of departments to have access to the data, but also installing enablers across the organization who ensure that everyone is using it responsibly. Federated data organization can come in many different forms but can be generally categorized into two groups: Centralized Partners or Democratized Representatives.
Federated – Centralized Partners
A Federated data organization that uses Centralized Partners has a Central Data Team, with each team member specializing in a specific business unit or issue area. For example, one team member is specialized in Sales, another in Marketing, and yet another in Operations. This approach usually means that there is one full-time person (or a small group) in charge of the data for an entire department. I have worked for a large organization that employed Centralized Partners effectively: an 8-10 person Central Data Team that assigns one or two people servicing each of the major departments.
Benefits
The benefits of a Federated organization with Centralized Partners is that it takes the strengths of centralization—understanding of data, strong governance, and knowledge sharing—while allowing individual members of the Data Team to immerse themselves in their business unit specialty. For example, a Data Team member specialized in marketing can attend Marketing meetings, be involved in marketing discussions, and be familiar with the members of the marketing team. They are able to do this more thoroughly than those in a purely Centralized data organization because they are able to focus on their specific partners.
Challenges
Federated organizations with Centralized Partners have similar issues to Centralized data organizations. As the data support is only a small group of people for each department, they cannot be involved with every meeting and every issue in their partner business unit. It is likely that they will have to get up to speed with specific business questions each time they need to perform an analysis. Additionally, the Central Data Team member must prioritize needs from across the department, as several people may ask for analyses at once. These challenges are less pronounced, however, since members of a Centralized Partners structure are able to be more focused when learning about and providing analyses to the business.
Mitigation
The more integrated one can be into both the Central Data Team and on their partner business units, the easier it is for them to quickly answer questions. Being included in meetings that are not specifically data-centric, but are related to data questions, is a helpful way of providing context to a data person without needing to re-explain the issues. Additionally, because those doing data analysis are few in number, creating a list of ranked priorities to discuss with the partner team is particularly helpful for a small group to prioritize their data requests. This way, the priority list is clear and any added or changed tasks can be weighed against other assignments.
Federated – Democratized Representatives
A Federated structure with Democratized Representatives employs a small Central Data Team to ensure that data is set up with sensible guardrails and overarching governance. In addition, the Central Data Team frequently meets with representatives to provide proper data responsibility guidelines. Data Representatives, who are a part of stakeholder teams, then enforce the practices from the Centralized Data Team such as good data hygiene, common data vocabulary, and data responsibility amongst their team.
In a 200-person organization where I worked, we transitioned from a purely Democratized data organization to Democratized Representatives structure because it was becoming increasingly difficult to ensure that the data was being used responsibly. It was an effective way to allow people to continue to answer their business questions but ensure some oversight on new analyses.
Benefits
The benefits of a Democratized Representatives structure are that many people still can have access to the data, but there is someone who is close to their function that can guide best practices in a business-specific way. It means that Centralized Data Team can be small, but still disseminate the information to everyone who is using data via the representatives. This data organization also fosters a strong culture around data, allowing those in the Democratized Representative position to take a leadership role in data governance of their departments while still ensuring that they are abiding by the Central Data Team’s guidelines.
Challenges
Similar to a purely Democratized structure, many of those who will be interacting with the data will not be doing it as regularly as people on a Centralized Data Team would. This means that they are not as entrenched in the data governance guidelines, vocabulary, and nuances that someone who works with data every day might be. This may result in misaligned metrics and analyses and long sleuthing missions to uncover the data errors.
Mitigation
As with all Democratized data organizations, the most important thing is to embed people in the data culture. This means following data governance guidelines, ensuring that people are using the correct vocabulary, and talking about data at all levels of the organization. A Democratized Representatives structure makes this easier than a purely Democratized structure because there are more people with an interest in creating robust data practices. They are better able to enforce strong data culture with their teams in a way that makes business sense.
An important part of the Data Representative’s role is helping others improve analyses and follow best practices; they should not be doing only analyses, but also be expected to spend time implementing data governance in their departments. For more information on mitigating challenges in Democratized data organizations, refer to my article Data Culture Issues and How to Fix Them.
So Which Organization is Best?
There is no definitively “correct” way to structure your data organization. In recent years, an increasing number of business-user friendly business intelligence tools has resulted in a rise in Data Democratization, as people without data management backgrounds can access the data more easily and answer their own business questions. However, many companies still have Centralized data organizations, preferring to leave their data in the hands of experienced professionals, even if they are not as close to the business questions. Characteristics of the respective data organization structures are presented in Table 1.
I have found that most companies’ data organizations fall somewhere in the middle of the spectrum, either growing their Centralized Data Teams to a point where each person takes on a business specialty or appointing specialized Representatives within Democratized data organizations. The Federated systems temper a lot of the major challenges that occur from strictly Democratized or Centralized organizations, allowing them to take on some of the benefits of the other.
Understand that your business may not stick with the same data organization for its entire lifetime. As a company grows and changes, so should its data organization. In this article, my goal has been to outline benefits and challenges for some of the major options along the data organization spectrum; in the future I will discuss when and how to make decisions about evolving the data organizations.
Regardless of where on the data organization spectrum your company falls, it is essential to balance the needs and responsibility of data governance and business agility and the ability to properly answer business questions. The data organization you choose should be dependent on the balance your business requires.