Making the Cloud, Microservices, and Enterprise IT work!

(Continuing the discussion of Batch Processing and SOA.) This article focuses on USING services (consuming a web service) as part of a batch oriented process. As mentioned in Part 1 , there's a strong tendency to want to consume (web) services as part of a newly developed batch process - usually due to reuse (other reasons discussed in the Part 1). It's pretty natural to say "well, services are reusable, and here we have a process that needs to use them. So, use them." However, there's a natural architecture incompatibility that has to be address. Web services are, by nature, remote. Batch process are oriented around local processing of large quantities of data/transactions, by marshalling large amounts of local resources to maximize the throughout. Here's an example to explain better... In the ancient days of programming, say before 1990 or so, the majority of systems were built using local files. Relational databases didn't exist yet, or were just...

Two weeks ago, Zapthink published a fluff piece Batch Processing with SOA (by Ronald Schmelzer). The article says just two simple things: 1 - You can make batch process controls into callable services. 2 - On-demand batches ("workload automation") are the future of batch processing. The article ignores what are the key topics of a batch SOA conversation, namely: A - Can web services be utlized in the context of a batch process (or workflow automation or whatever you want to call it)? B - Can an ESB provide an enviroment for a batch process, or a significant portion thereof? In other words, is there a convergence between batch processes and SOA integration tools (web services, ESB, or even governance)? Can these respective spaces leverage one another, or are they fundementally incompatible? This issue has come up in the context of two clients within the past 2 weeks. The first question usually asked is WHY? Why would you consider an ESB in the context of batch processing? ...