Overview of Enterprise Data Integration

Home

Ask a Question

Data Islands are all around Us
They often become critical only when major commercial organisations try to integrate.
LloydsTSB has, so far, been unable to combine the Lloyds and TSB back-office systems
resulting in that customers of each bank are unable to access the same banking services, 
and promised cost savings have not yet materialized.
A possible explanation for this kind of predicament can, perhaps, be found in the low 
priority given to integrating IT systems when organizations merge or acquire another company. 
In fact, a typical M&A; top 10 to-do list cites the first three priorities as being financial and 
legal incorporation, human resources and physical plant congregation, and product and service consolidation. 
The issue of IT integration usually comes a poor tenth, demonstrating that M&A; game plans fail to recognize 
that there can be no product and service consolidation without IT integration, and often the effort required 
to integrate disparate IT architectures and a myriad of legacy systems can be huge.

The Difference Between Success and Failure
With large proportions of company revenues being spent on IT, it's no surprise that one of the economies of
scale most organizations hope to achieve after a merger is a greater efficiency in their data processing.
On the surface, this would seem simple enough, as most companies have some applications such as general ledger,
payroll, etc. in common.
Yet, it is precisely this area of application and data consolidation that can prove to be the difference between
success and failure for M&A; projects.
Even seemingly similar applications can prove extremely challenging to successfully integrate.
Forrester Research estimates that building, maintaining and supporting application integration accounts for 30 percent
of the average corporate IT budget even without absorbing the costs to perform the data consolidation required after a
merger or acquisition.
To maximize the potential return on investment of mergers and acquisitions, the necessary integration between the two
companies must be achieved quickly, minimizing the disruption to customers.
Economy of Scale or Diversification?
There are generally two drivers for M&As;: economy of scale and diversification, each of which brings its own set of IT
challenges.
Where two companies offer similar services to similar customers, the key driver for M&A; is often the desire to achieve greater economies of scale – that is, to sell to more people more efficiently. One of the key problems is getting a consolidated view of the customer.
For example, Mr. Jones might have both a life insurance and auto insurance policy with the same insurer, each with a different policy number. Because the customer does not renew or alter his life insurance policy on a regular basis, the insurer's records show a previous address for the customer – which is different to the auto insurance policy which was only acquired four months ago. Under the company's existing database structure, there is no easy way to identify that it is the same Mr. Jones that has both the auto and life insurance policies with the company.
Consider that Mr. Jones might also have a household-contents policy with the second of the merging organizations, it is easy to see how difficult it can be to get a consistent, corporate view of the business and its customers.
Mr. Jones, seeing that all three of his policies are now with the same insurer, believes he should be able to inquire about each of them in a single call. But, unless the insurer has successfully tackled the issue of data integration, this may not be the case.
The Challenge of Diversification
Where the two organizations offer different, if not entirely unrelated, services to a wider range of customers, the anticipated benefit of the M&A; is more likely to be revenue diversification – i.e., selling a wider range of services to more customers. In this scenario, the two companies can hope to leverage one another's penetration in certain markets while also enjoying the benefits of a consolidated infrastructure.
As in the previous example, a major challenge remains the integration of different IT systems. Here also, a major issue can be the fact that, unlike companies which sell the same services to similar customers, the business applications of the two organizations are likely to be completely different – not just in the way data is stored and accessed but in the very information recorded. Often the systems will hold very different data in different ways.
A common element to either of these scenarios is the customer. After all, without the customer, you simply don't have a business. Through all this data and systems integration, it is imperative to maintain good customer relationships – which means customer data must be accurate and available.
Issues in Data Consolidation
When a company undergoes a merger or acquisition, it can take one of three possible IT approaches:-
1) Allow the IT organizations to continue independently for a period of time, using a data warehouse to create a corporate view of the enterprise.
2) Migrate to one of the organization's applications.
3) Migrate to a best-of-breed configuration of applications (for example, one company's life insurance system and another's auto insurance system).
The first two options are more likely to be adopted by companies seeking economies of scale, while the third option is more practical for companies looking for diversification. Regardless of which of the methods is chosen, inevitably there will be an amount of data and application integration to be performed. In every case, efficient data integration is key to success, whether this be in cost-effectively loading and refreshing a data warehouse, migrating one organization’s data to the applications of another or a combination of migrating data and creating new interfaces between applications.
Bridging the Technology Gap
When confronted with two different applications that suddenly need to work together, organizations are faced with the need to implement bridging interfaces. Traditionally, the most popular method of creating interfaces between disparate IT systems has been for teams of software developers to manually write them.
With increasingly large and complex migrations involving multiple data sources, however, hand coding can be a risky, lengthy and potentially expensive option. It also tends to be an inflexible approach when it comes to accommodating rapid change and does not allow for the extensive reuse of code. Forrester Research estimated that more than $100 billion was spent by companies hand coding interfaces in 1999.
Cost is not the only issue either.
According to research by the Standish Group, around 30 percent of data migration projects fail making integration a highly risky business. Finding an effective, reliable solution is an absolute must.
There is, however, an alternative to manual coding. Data migration tools can dramatically cut the man-hours involved with creating hub-and-spoke interfaces and can reduce the likelihood of human error. Most data migration tools rely on meta data to help bridge disparate applications.
What is Meta Data?
The ability to merge with or acquire another company and integrate their data with your own, delivering almost instant competitive advantage – is often no more than a pipe dream to many IT managers. But, for those IT managers with the foresight to adopt an effective meta data strategy, this can represent a challenge they are happy to deliver against swiftly and efficiently. A major problem faced by many companies looking to consolidate data after a merger or acquisition arises because few organizations really understand the state of the data in their operational systems.
Essentially, meta data can be described as data about data. It is a way of understanding an organization's data that not only helps guarantee the integrity of data but ensures that an organization can quickly determine what must be modified when changes in business process require changes to the systems that handle business-critical data.
If an organization uses products that produce descriptions of what users have done, this information (meta data) can help an organization build a history, or audit trail, of its data by tracking changes made during a period of time. In terms of your data history, it tells you where you've come from, where you've been, when you got there, what you did on the way and what would happen if something changed.
Without meta data, the audit trail of changes to data is either easily lost or incomplete
meaning that, when the time comes to integrate, a huge number of man-hours must be spent
relearning how the data elements were stored, formatted, used and related to other data elements.
Meta data comes into its own when companies need to integrate disparate applications as is often the case with mergers and acquisitions.
Armed with the right information about an organization's data, changes to applications and
the migration of data to new applications can be achieved much faster and with far less
pain than would otherwise be the case.
Building a Meta Data Strategy
Regardless of whether an organization is actively involved in current M&A; activities, building a meta data strategy is crucial to future success. IT systems are constantly evolving, and the added complication of e- commerce is escalating the need data integration management.
Pursuing a meta data strategy involves building a system of record, a meta data repository which defines data elements,
their attributes and their interrelationships so the knowledge acquired in implementing one project can be reused in others.
If an organization required that the products they use automatically capture and exchange a meta data audit trail, the task of building this system of record would be greatly simplified.
There are two types of software products in particular that are essential to a successful data integration project.
1) A meta data management tool can help automate impact analysis, identifying and highlighting the parts of an application
that are affected by changing requirements.
2) a data integration tool can generate the new integration code necessary to create the required interfaces to load and bridge applications.
By building a meta data strategy and collecting meta data, companies can get a competitive advantage on competitors because they can then integrate data and applications for future mergers and acquisitions much faster, cheaper and more effectively.