MDM & SOA - Layer, Repurpose or Replace?

An Architect Friend sent me this extended architecture question...

I recently joined a company that provides business consulting (via many MBAs) services related to sales and marketing.  Most clients are large pharma companies.

In addition to consultants, there are business process outsourcing teams (offshore) that do operations (like incentive plan management, report distribution, etc.)  There is also BI/reporting group that creates BI/DW solutions (custom ones using template approach) for large clients.  Plus there is an Software Development group (SD).

Over the years the Software Development group of the company created various (10+) browser-based (.NET/SQL) point-solutions/tools to help consultants (and eventually some head-quarters users) perform specific tasks. For example:

- Designing sales territories and managing the alignment of reps to territories
- Custom ETL-based tools to perform incentive calculation
- Some Salesforce-like platform for creating custom form-based apps
 
The applications are architected as single-tenant – with some deployment tricks to be able to deploy an “instance per client” on the web servers. The databases are isolated per client/instance.  The tools are sold as if they are part of an integrated suite, but they aren’t natively integrated and require custom integration.

There is a custom grown ETL-like tool for interconnecting the tools to each other (but not standard connections since the data models are all “flexible” and not well defined) plus Informatica and Boomi to get data from clients.  Some clients use one tool, some use 2, some use 3, etc.  Some tools are used directly by the client, but most are used by the consulting teams on behalf of the client.
 
Lately, there is desire to make it all “integrated” across the company (SD + BI + all else).  Two main themes are emerging (even prior to me joining): “common data model” and “SOA”.  There is also the question of letting existing applications function as-is and developing new ones on a more proper architecture versus trying to evolve the existing apps.
 
However, the understanding of how this applies to an Enterprise looking inward on its own systems and trying to align them, versus Independent Software Vendor (ISV) looking to build software for other Enterprises did not yet sink in… and concepts are being confused…
 
The tension between a standardized productized software versus customized (consulting company) software solution is not yet resolved.

I wanted to ask if you had experience in environments were an ISV was trying to define the enterprise architecture of their solutions for customers versus their own internal architecture.

Are there any case-studies or resources you could point me to get some reference architecture examples?

I usually do not like “next gen” approaches, but I am not seeing much potential in evolution of existing assets into an integrated state (they have a lot of “baggage” and features that were there but don’t play nicely with “integrated” world-view).

=============

Here's my answer:

I wanted to ask if you had experience in environments were an ISV was trying to define the enterprise architecture of their solutions for customers versus their own internal architecture.

-        No, though I have built integration competency centers and projects that were providing service environments across very large scale enterprises of disparate divisions.

Are there any case-studies or resources you could point me to get some reference architecture examples?

-        Not that I know of.  I'm not much of a fan of such studies, mostly because the requirements and details are always highly complex, and those details directly affect the approaches taken.  Studies and reference architectures provide a nice high level structure – but the more you try to keep to them in the details the less effective they are (as they are mismatched to the exact situation).  I use bits of Togaf-9 from opengroup.org, bits of CBDI from Everware http://www.everware-cbdi.com/, and various tidbits picked up from Zapthink (though every few years they discuss the benefits of yet another framework).

I usually do not like “next gen” approaches, but I am not seeing much potential in evolution of existing assets into an integrated state (they have a lot of “baggage” and features that were there but don’t play nicely with “integrated” world-view).

-        It's a pretty standard problem: how to balance between what is, how it can be extended / expanded / reused, and what should be replaced / redeveloped / moved up to a new generation of technology, pattern and features.

- The problem you describe sounds like it crosses between SOA / integration and MDM (master data management).  Sometimes a SOA façade can provide an MDM operational model, with composite services doing multi-system queries, combining or rationalizing the result, and presenting single meaningful "views".  In other cases it's the SOA abilities enabling MDM to do it's job, which often involves signification bi-directional synchronization.

- The MDM tools tend to be heavy, and the business and systems analysis work (which system wins when data is in conflict, for example) is a major portion of the success or failure.

- That said, IF you are only trying to get views of the data, I am hearing reports of good success with some of the easier BigData tools (such as MongoDB).  Success meaning they are able to develop and deploy meaningful business results in months, whereas MDM and big integration SOA projects almost always take over a year.