Data Integration Blog -

Whether the focus of an enterprise is on “big data” or not, analytics-driven companies are faced with processing many new data sources from inside and outside the firewall. These data sources may be complex structures — or the complexities may come from other aspects such as data trading protocols and message transports. Depending on the kind of analytics being used for a particular purpose, strategic decisions must be made about data quality, such as requiring “perfection” or choosing “good enough”.

Overall enterprise data processing has increased in complexity since the origins of data warehouse solutions for business intelligence purposes. And key data for certain analytics frequently no longer lives in the enterprise data warehouse – nor should it, in many cases. But data management and data integration solutions still have to be able to handle these disparate sources to prepare them for analytics processes, increasingly in real-time or near real-time.

With disparate uses for analytics growing and the time pressures of on-demand and right-time processes for data integration and analytics, it has become imperative for many enterprises to process many different kinds of data faster just to be able to compete in today’s accelerated markets. To handle the diversity of analytics, integration and data quality needs, enterprises may need to depend on an ecosystem of integration and analytics solutions to match the right processes and technologies to the job to be done. The issue facing enterprises will then be choosing how to build out the ecosystem while ensuring interoperability and ease of management.

The caveat as always is this: you have to make sure that the data is trustworthy and useful within the constraints and goals for distinct analytics processes. For example: “good enough” data works in certain situations but cannot be allowed to populate across all enterprise BI and analytics functions. Business users who are data subject matter experts (SMEs) are needed to review and monitor data usage and quality in systems and processes — to make sure that bad data and the wrong data aren’t polluting decision-making processes.