Two weeks ago, Zapthink published a fluff piece Batch Processing with SOA (by Ronald Schmelzer).The article says just two simple things:
1 - You can make batch process controls into callable services.
2 - On-demand batches ("workload automation") are the future of batch processing.
The article ignores what are the key topics of a batch SOA conversation, namely:
A - Can web services be utlized in the context of a batch process (or workflow automation or whatever you want to call it)?
B - Can an ESB provide an enviroment for a batch process, or a significant portion thereof?
In other words, is there a convergence between batch processes and SOA integration tools (web services, ESB, or even governance)? Can these respective spaces leverage one another, or are they fundementally incompatible?
This issue has come up in the context of two clients within the past 2 weeks. The first question usually asked is WHY? Why would you consider an ESB in the context of batch processing?
There's a few answers to this question:
1. The first is one of the nominal goals of SOA, reuse. Making connections is a big part of service development. Depending on the environment and technologies involved, making a connection to a given system can be a major effort in and of itself. Security considerations, gateways, protocol translations and adapters required, just making contact with the right people and receiving the right approvals. The actual labor (programming or configuring) required isn't usually that time consuming, but finding the right people to get in touch with, going through the security approval processes, getting adapters installed or gateways configured, and coordinating it all, that is time consuming.
So, you've got a service means you have a valid working connection to a given system, with a valid working transaction - business function - service. Some batch process comes along that has a new requirement for your service, why not use it and save all that setup time? (And the related value of having one code source - function basis, real reuse.)
2. The second is ESB tools are very good at data transformations - in particular at the programming of data transformations. Visual drag-and-link development and integrated converter functions make the type of data format transformations often involved in batch processes a good fit. Further reinforcing this thought is that the ESB usually has a trained group of developers just waiting to create those transformations.
(As opposed to batch developers, which are usually doing this effort as part of their primary project - which is not exlusively batch work - and therefore are less likely to have such specialized tools or, if having them - be less skilled in their use. If they have tools, it's ETL tools [Extract Transform Load] which are very good at what they do BUT tend to be significantly more expensive than ESB tools, and because of their limited use they're harder to justify.)
These reasons, and perhaps just the general Integration Compentency Centers tend to be thought of when connecting things - regardless of the processing model - are dragging many an ESB and/or integration team into implementing batch processes in the context of the Enterprise Service Bus (ESB). This, however, is usually a mistake.
Why will be covered in my next article.