Today, having data is not enough to increase the competitiveness of an organization, but it is necessary that the data are integrated in one place to be able to take full advantage of them. To achieve this, the organization must carry out an ETL process.
At Bismart we carry out projects with ETL processes in many different fields.
One of the projects we have developed is for a leading company in its sector that has a complex series of systems and technologies to respond to business needs, such as different ERP, CRM, DWH... In this organisation, heterogeneity created a trend towards data silos. Among other consequences, data could become duplicated and sometimes inconsistent, incomplete or inaccurate. In addition, data integration between these systems was really complex and costly for IT departments.
The problems with management in the government of data with which they had to deal with were:
- Dispersed data: because data is on different systems, it is very likely that they do not share a common understanding of the data and are fragmented across the organization.
- Inconsistent data: because the organization cannot implement its systems at the same time and because data is created in different departments and users, it is not easy to make the data integrate with other systems, which can cause inconsistencies. Due to the lack of common identification it is difficult to analyze customer behavior stored in multiple systems and take advantage of the opportunity to cross-sell or increase sales.
- Inaccurate data state: It is not easy to capture the state of data over time or in a given state if these are not common between systems.
- Data-centric ownership: Most business applications do not provide a way to allow companies to impose ownership rules on data through applications, identifying business domains such as product, customer, etc. Anyone with access to the application can modify any data. This makes audits and follow-ups difficult.
- There is no business process: most systems do not allow companies to meet their lifecycles and processes for data governance. This means for the company, by industry standards, regulatory framework or other requirements, that changes in data require a subsequent approval process or quality control. This cannot be done due to the rigidity of the system.
- Slow response to changing business needs: It is difficult to react quickly to changing business needs of companies when it affects multiple systems. This is not only error-prone, it is also costly in effort and time. Among others, because there is no defined mechanism for the automatic distribution of modified data among all affected systems.
- Multiple data formats: companies manage large volumes of data, in different formats and from various sources. Since there is no single, centralized standard for data integration, companies have to dedicate a large amount of resources to the conversion and processing of data and then distribute this data to all the systems that require it.
- Mergers and acquisitions: mergers and acquisitions happen frequently, and that means that companies, with all their systems, have to merge into a single logical entity. These transitions add more data inconsistency and more disparate systems.
- Regulatory compliance: when a company has to be audited externally, this can be a very costly operation. Without a centralized system, it is difficult to collect and combine reports from multiple systems for auditing and compliance purposes.
The solution created by Bismart
The project that Bismart developed for this organization was a solution that facilitated a single view of master data throughout the organization and interoperability between the systems involved through proper management of master data.
Master Data Management (MDM) is a process of collecting enterprise data from multiple sources or systems, applying standard rules and business processes, building a single view of the data, and finally distributing this 'gold' version of the data to the various enterprise systems and therefore making it available to all consumers securely.
An MDM platform is different from a DataWarehouse platform. The purpose of the DataWarehouse platform is to facilitate the analysis of information from historical data, data from transactional management systems, departmental applications, external sources... as well as from an MDM system. An MDM platform reconciles data from several systems to provide a single view of master data and mainly for operational processes.
The information contained in the Master Data in an MDM is by definition much broader than in a DWH, since in the DWH only the required data is stored for later analysis.
With this solution, the company was able to resolve the dispersion of the data, by having all of them in one place. In addition, the implementation of the solution facilitated the improvement of efficiency, since the possibility of being able to access data from all sources and the compatibility of the various systems made management such as life cycles for data governance or audits faster.
In addition, the user can now enter or manipulate the data, as it has a direct action interface over the MDS data. The business indicators introduced by MDS allow the generation of a system of process and business notifications. Therefore, the solution offers more control and monitoring and more transparency.
Another advantage of the MDM solution was the reduction in the number of data errors. Thanks to the reconciliation of data from various systems and this reduction, the company was able to increase its speed of response to business needs. In addition, in the face of the possibility of mergers and acquisitions the company is much better prepared and can guarantee that these actions will not generate more data inconsistency.