Is SOA a Bad Idea?

In consulting at various companies, I'm seeing a lot of "SOA failure". Oh, the technology works and the tools are capable. Further, their developers are using a variety of SOA(ish) techniques. But their formal SOA programs, their Integration Centers, are failures. (I define failure as an unhappy department, senior management that is unhappy with the results, a general feeling of failure, unable to demonstrate and most likely without ROI, and an impression that Integration is in-the-way and provides unsufficient benefit versus the effort to engage the process.)

These projecs or departments have almost always started bottom up, they are an IT department solution to pinpoint (narrow) IT problems. They've never really mapped out how they're going to get from here to "there", and have defined "there" as only an IT solution without connecting to concrete business goals.

First a basic question, do you want to be “there”? The easy answer is yes, integration is getting more complex and must be managed, while at the same time demand for systems agility is increasing. Like the change from procedural oriented programming to object oriented programming, the benefits are real and the future of IT will be “there”.

But a major mistake being made by many IT organizations in moving to Service Oriented Architecture is moving for purely technology reasons rather than business reasons…

Example: A regional utility company called me in as a SOA and Integration consultant. We spent a day reviewing the SOA model and benefits, issues such as reliability and security of the new model, and discussing how an organization begins making such a transition. At the end of the meeting they said, “great, that was very helpful and enlightening. Now we know we don’t want to do that.”

I was a bit taken aback. I tried to explain that like the transition from procedural to object oriented programming, the SOA model was going to be part of the future. Just like procedural programming was left in the past and exclusively in legacy systems… At which point they stopped me and said “but we still do procedural programming.” Huh? “We’re still doing new development in procedural methods, languages and environments.”

“See,” they explained, “we need absolute systems reliability. We know procedural technologies. They are thoroughly understood, all operational aspects well known. We know exactly what we’re getting and how it’s going to work. Even if other technologies offer a variety of benefits, our primary concern is reliability, supportability, stability.”

In another ten years they’ll be doing SOA. But today their business concerns require they NOT do SOA. Business and IT management concerns of environmental complexity, systems and business process agility, and gaining futuristic high speed full systems development advantages through applications assembly and BPM (Business Process Management) capabilities are great reasons to begin moving towards SOA. “It’s a bunch of really neat technologies” is not.

Programmers Arrested

It's rather unusual to hear of programmers, non-hackers, to be arrested for programming. But today's news announces the arrest of the programmers who build the applications that operated Madoff's financial empire of cards. Apparently the programmers are not being arrested for creating something illegal, but rather for "knowing" that their systems weren't complete (and therefore could not complete the financial trading operations)...

They were accused of knowing that the computer programs they developed in 2003 and 2004 contained fraudulent information used in U.S. and European regulatory reviews.

The SEC said O'Hara and Perez (the arrestees) knew that the "House 17" computer was missing functioning programs needed for actual securities trading.

In August or September 2006, they cashed out hundreds of thousands of dollars in their personal BLMIS accounts before meeting with Madoff and telling him they would no longer lie for him, the FBI and the SEC said.

The charges against the pair carry maximum prison sentences of 30 years and millions of dollars in fines. (Whole article here.)

As always, the news and the authorities position the situation in it's worst light. I certainly only know the details I read. But a couple of salaried programmers who had retirement accounts of a couple of hundred thousand dollars (a very normal figure after 15 years at a financial services company) and received a "huge" bonus of $60,000 (also a very normal yearly figure for top IT people working at financial houses) seem to be an odd target.

Is every IT employee involved in any financial system or business system development supposed to question the business process if what they're being told to develop doesn't seem complete or thorough enough? Would you, as a programmer or software architect, challenge a company Vice President making millions every year (versus your regular salary) when he says "this is what I need, go build it" if it seemed incomplete (and you pointed that out)? Would you quit over it?

Troubling.

Towards SOA (I)

(Part 1)

Given an opportunity to build a new a set of IT systems from scratch, only the latest techniques, technologies and architectural approaches would be used. In the real world, practically every IT organization is faced with legacy applications, a nice euphemism for the old software that’s running most companies. Anywhere from 5 to 30 years old (or more), it’s running on old (style) infrastructures, operating systems, programming languages, etc.

It incorporates years of business logic development. The capabilities (and failings) are well known and well understood. And you probably can’t afford to replace it.

Not only do IT organizations face this problem, IT vendors face a similar challenge. Even the best known ERP and CRM systems are composites of well known functionality with bits of the new tacked on over time.

In all of these cases, decomposing from the big box application model to a service oriented business function model frequently means completely replacing or rewriting systems. And this is simply not a viable option for most people and companies.

So how to you get from here to there? That will be addressed in part 2.