Data Integration Blog -

I just wrote a book about SOA and cloud computing, and how the two are converging. The reaction to the book that took me aback the most is the fact that few out there understand the differences between SOA and cloud computing, and the role of data integration with all of this. Perhaps it’s time we cleared things up.

SOA is something you do, not something you buy. It’s an architectural pattern derived from more traditional enterprise architecture. SOA, very simply put, is about the decomposition of IT assets and building them back up as sets of services. Using processes or composites as the configuration layers, these services can be configured, and more importantly, reconfigured, into business solutions.

We do SOA for a few reasons. First and foremost is agility. Considering that we can put volatility into a configuration layer, we can alter core business processes, indeed, core business systems, with much less effort than traditional approaches. Second, and related to agility, is reuse. While we’ve been looking to move to reuse for years, we needed the ability to address core systems as sets of services, which are really sets of functional application behavior that can be leveraged by any application or process that can leverage a services standard, such as Web services.

The final side benefit of SOA, and more to the topic of this blog, is the fact that, with the advent of cloud computing, services can now exist inside or outside of the enterprise. Indeed, most cloud services, such as storage and middleware services, are consumed via APIs or services. Thus the use of cloud computing is almost always service oriented, and furthermore, requires data integration as a core component. You see the intersection coming together.

Data integration is a core component of SOA. Indeed, you can’t have a sound SOA without a sound data integration strategy and technology. Data integration provides the mechanisms to exchange information between very dissimilar platforms, accounting for the differences between source and target systems through deep semantic management and mitigation layers. However, data integration stands alone as a concept. Thus, you can leverage data integration regardless of whether or not you’re moving to a SOA. The concepts of data integration and SOA are such that SOA requires a data integration strategy, and technology. But, data integration does not require SOA.

Thus, you need data integration technology to allow for the free flow of information between systems in support of a SOA. SOA is the way to address an architecture as sets of services. However, in order for those services to leverage data, which most do, there has to be an underlying data integration model, plan, and enabling technology. You can think of SOA as the way to design a car, where data integration is the engine that makes it actually move forward.

So, back to cloud computing. Cloud computing is a mere platform option, or new and hopefully more cost effective place to run your applications and store data. When leveraging cloud computing, the core data integration mechanisms become even more important. This is logical considering that you typically do not own or control the systems in the cloud, and you’ll always have to sync data that resides within multiple clouds with data contained within your enterprise. To that end, you’re dealing with very different platforms that need to exchange information, as well as typically deeper requirements around service and data governance, security, and integrity issues associated with both the temporary and on-going use of cloud-based systems.

A much easier way to look at this is to consider SOA as the foundation of what is done to define and build the architecture. Cloud computing should be considered as a set of architectural options where data and processing may reside, if it fits the needs of the architecture. We currently have IaaS, SaaS, and PaaS as macro cloud computing architectural options. Now, figure that those options will expand in the future.

Core to all of this is data integration. In my travels, and dealing with hundreds of problem domains, it’s clear that a data integration strategy and enabling technology are critical success factors for all of this. I would call it systemic to all architectures, including SOA, and critical for all platform options, especially cloud computing-based systems. Thus, it’s something that is always required no matter which path you take.