Why Collaboration in Data Modeling is Critical to Business Success
The Covid-19 pandemic has reminded us that daily life is full of interdependencies. The data models and logic for tracking the progression of the pandemic, understanding its spread in the population, predicting the future course of the pandemic, and evaluating potential policy options have proven flawed. This resulted in everything from ineffective containment measures has flawed containment policies has confusing messaging.
The problem was not the data but how it was interpreted and contextualized. Businesses face the same issues.
In 2022, for example, Equifax provided to millions of individuals with incorrect credit scores when they applied for credit cards, mortgages and car loans. The consequences of this technical problem led to a class action. Industry sources say that poor data modeling played a role in the calculation error. Values such as “number of inquiries within 1 month” or “age of oldest trade line” were potentially incorrect for some transactions.
No business wants such results, so prioritizing proper data modeling is imperative.
What is data modeling and why does it fail?
Fundamentally, data modeling involves organizing data in a structured way to improve accessibility and usability for a range of applications and analyses. This involves putting data objects, their relationships, and the rules that govern them into visual form.
Because it calls upon diverse expertise, data modeling is rarely a one-off, solo effort. This requires ongoing collaboration among diverse stakeholders with diverse perspectives, including data engineers, data analysts, and business users.
When different stakeholders share their knowledge and perspectives, the result is a more accurate and comprehensive data model that better reflects the needs of the organization, minimizes communication issues, and ensures that everyone is aligned with the data structure. data and intended use.
Unfortunately, many organizations fail to bring together diverse expertise for a comprehensive understanding of data and its requirements. Teamwork can be difficult, as business people often lack a deep understanding of data modeling and analysis, while data experts are less familiar with business terminology.
Additionally, many business users view data modeling as something strictly technical and dependent on extensive coding skills.
Semantic sprawl exacerbates the problem.
Another barrier to effective teamwork is the deadline-driven nature of reporting.
Businessmen usually depend on data engineers to code timely data models every reporting cycle to deliver reports on time. Different data models and hard-coded business logic are stored in various semantic layers distributed across business intelligence (BI) and visualization tools or within the data platforms themselves, causing “semantic sprawl.”
Semantic sprawl is problematic because semantic meanings (business definitions) quickly become static and rigid, making it difficult for centralized architecture teams to meet the domain-specific needs of different work groups. Scattered code becomes unmanageable and inconsistent as it scales, causing delays and dependencies that hinder data-driven decision-making.
Start with a solid foundation.
When it comes to something as strategically important as data, it helps to take a step back and start with a solid, cohesive foundation. The first step is to implement a universal semantic layer where data models are built and maintained.
A universal semantic layer creates a common language within an organization, which is the starting point for a “single source of truth” for data and the efficient flow of data within the organization. It resides outside of the BI and analytics tools and data source, but any BI or analytics system can access it to ensure accuracy and allow users to continue using their favorite tools.
Deploying a universal semantic layer is not without challenges. Organizations often deal with complex, unstructured or poorly documented data sources, as well as inaccurate or inconsistent data. Users may be reluctant to move from their usual methods of accessing data to a new semantic layer. Businesses also need to find a way to balance the centralization and consistency of a universal semantic layer with the perceived flexibility of current processes.
As with any change, a universal semantic layer requires training and effective change management.
Drive collaboration through visual data modeling.
While there are several factors to consider when choosing a universal semantic layer, an important prerequisite is to support collaboration across teams, from data engineering to data analysts and business users . The perception that data modeling is solely an engineering function has become obsolete with modern self-service BI platforms.
People across the organization are already modeling data. Now is the time to unify these efforts in one place. A shared visual workspace makes it easy for everyone to see the data structure so non-technical people can participate in the modeling process. Engineers can choose to work in code or no-code mode, and non-technical team members can actively engage without writing code.
In addition to visual workspaces, collaboration involves including representatives from different business areas, facilitating open communication, and establishing clear ownership and governance processes to ensure alignment of all stakeholders, with a centralized data team often acting as the primary facilitator of the collaboration process.
A dedicated data team acts as a central platform to coordinate collaboration, manage data governance, and ensure consistency across the organization. Data architects, business analysts, data engineers, and subject matter experts from different departments should all play a role within these teams.
Once the team is assembled, it is important to hold regular meetings to discuss data model design, feedback, and updates. Likewise, use collaborative data modeling tools with version control to track changes and facilitate feedback, and be sure to clearly document data definitions and business rules to avoid ambiguity.
A final word
Ultimately, deeper data models produced by diverse perspectives lead to better business decisions and outcomes. Workflows are streamlined because data models can be created and improved more quickly, and non-technical users are no longer dependent on developers to execute changes.
While it takes some work, the consistency and accuracy provided by a universal semantic layer with collaborative, visual data modeling can pay off in better strategic decisions and a substantial reduction in business risk.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Am I eligible?