Making the Cloud, Microservices, and Enterprise IT work!

Web service security is a tricky business.  EVERY service exposed by any service provider, be it .Net, Java, the Mainframe, or any other provider needs to be secured.  Certainly if it’s exposing sensitive data (say customer data), allowing activation of a business process, and most especially if it’s involving a financial transaction. But how do you do it?  While every vendor and (almost) every technology announces compatibility with every web service security buzzword (WS-Security, SAML, X.509, etc.), they don’t describe how to actually make use of all this security data attached to the web service request. I’ve had recent discussions with IBM, Oracle, and Software AG (as leading SOA middleware tool providers) on this exact topic and the results are disappointing. The architecture model for this says that to provide SOA security I should use the tools as a SOA security layer, allowing my services to go about their business and the security tools to grab and pr...

I sat with my client on a major service orientation project as IBM presented their updates to CICS.  My client is a few versions behind (could actually find no compelling reason to bother to upgrade) on their mainframe and wanted to review if IBM’s added capabilities would offer any abilities the project could utilize. IBM’s CICS support for web services is well known by now.  Their CICS updates keep the web service support up to the latest WSDL – SOAP – and WS-* standards.  If you need or want to expose COBOL programs as a web service (and this can be valuable to allow older applications to be partially utilized as transaction engines in their area of expertise) IBM’s keeping the capabilities in line. IBM has also continued to expand a large variety of internal CICS capabilities.  This was a major time warp for me as CICS command line functions, com areas and linkage sections, and various transaction batch and script coding abilities are so dated.  As an...

Loriane Lawson over at IT Business Edge has a penchant for touching upon particularly interesting integration problems.  This week she asks … It's interesting: Writing custom code for data integration is nearly universally frowned upon by experts – and yet, I'm seeing a lot of discussion about creating your own metadata solutions for support with integration. My question is: If you're getting away from hand-coding, why would you want to delve into customized metadata solutions? The article focuses upon some of the technical issues and technical approach to such a discussion.  I’d like to focus on the business issues, and how those should be directing the technical approach (but aren’t). Every application involved in integration brings along it’s database (or general data) model and it’s internal data object model.  These models were developed to meet the functional goals of the particular application.  For example, the “customer” representation in t...

Loraine Lawson over at the IT Business Edge Integration blog discusses Microsoft’s entry into the MDM space in her most recent post .  She notes that: “I was also intrigued to learn that the whole thing is API-based. Why does that matter? As Hayler explains, it allows ISVs to build apps on top of MDS. That's a good thing, since the description of MDS sounds pretty bare-bones at this point. The idea seems to be that ISVs will be able to build on better user interfaces and create support for things like version comparison, which, oddly, it does not provide out of the box.” I was surprised at this bit of thinking.  It’s always nice when a vendor product provides ways of extending it’s functionality.  It’s occasionally useful for enterprise IT shops, and often useful for niche vendor’s or ISV’s looking for opportunities to extend a major vendor’s product(s).  But that’s some old school thinking. Since the early days of COM (Microsoft’s Common Object Model),...

  The Datapower has an internal structure of components that can inherit or be reused, depending on their place in the inheritance chain.  Here’s the secret internal architecture of the DataPower:   And here’s the Datapower’s capabilities in a nutshell:   Multi-Protocol Gateway: (superset of XML Firewall) Transformations – Any-to-any transformation engine: MPGW can parse and transform arbitrary binary, flat text, and XML messages, including EDI, COBOL Copybook, ISO 8583, CSV, ASN.1, and ebXML. Transport Bridging – protocols such as HTTP, HTTPS, MQ, SSL, IMS Connect, FTP, and more Message-level Security - Messages can be filtered, validated, encrypted, and signed, helping to provide more secure enablement of high-value applications. Supported technologies include WS-Security, WS-Trust, SAML, and LDAP. Logging - logging and audit trail, including non-repudiation support Web Service Proxy: Schema Validation Policy Application SLA Monitori...

Encryption Every web service call should be SSL encrypted (HTTP/S). Similarly, any web service operation or file transfer sent via FTP should utilize Secure-FTP (S/FTP). Basic level security requires that the receiving source have a valid signed certificate, but it does not require dual-side certificates or any validation of the certificate beyond it being a valid source on a valid root chain. The objective of this requirement is encryption, not authentication. Now encryption will frequently be argued against.  “It’s high overhead, slows things down too much.” and “Setting up security on every server is time consuming.” First, we must layer our security.  By exposing services we’re transmitting internal application data outside the security perimeter of the application!  This data must be protected from view, and dropping a network sniffer onto a developer’s workstation or a development or test server is a trivial exercise.  So even if there’s som...

The first and foremost feature of an IBM DataPower is as a security device.  However, most organizations turn their Datapower over to their security team and ignore it afterwards.  The security team(s) generally use it as a perimeter security device – as a firewall and filter for exposing SOA services out to the Internet (or via VPN connections, as who can trust a vendor’s network anymore).  It works in this capacity very well but is far more capable than just this narrow role.   With SOA breaking down the outer perimeter of our internal applications, security must now be layered and extended to EVERY exposed service or interface.  There’s two general approaches to providing this: The agent based model, where an agent is installed upon every server / application / application container and controls access to each service.  The other is an agentless model, where each web service is routed through a control point – in this case the Datapower, and the...

I had a very interesting meeting with a client a few weeks back. (For those who don't read regularly, I do high level IT consulting.) This client has been doing SOA for a few years. Actually though, they're doing SOI - Service Oriented Integration - not SOA, Service Oriented Architecture. Meaning they're creating and/or exposing lots of services from existing big box applications, and using web service technology to connect apps together. After a couple of years of SOI they have several hundred exposed services and application interconnections. Their architects have spotted that some services have much higher reuse patterns than others and are trying to apply architecture standards such as standard entities and interface patterns to every new service - to up the reuse, decrease the maintenance, and move towards SOA architecture goals and ROI. This is a very natural step along the SOA maturity path. Along comes a change in senior business management. They bring a new...

When Anne Thomas Manes (of the Burton Group) famously declared in January of 2009 that "SOA is dead", everyone rushed around to understand what she meant. Being that a year later she's still giving presentations on SOA Governance and other SOA topics, clearly she didn't mean that SOA was a failed technology. (There are plenty of IT technologies that come along with much hype but never quite translate into practical usage patterns or benefits for Enterprise IT, and therefore fade away as quickly as they arrived.) Today when I'm talking with IT organizations the majority are doing some level of SOA. So clearly SOA has moved along the adoption curve. The innovators struggled with it but got and touted their early advantage. The early adopters picked it up and integrated it into their enterprise IT model. We're clearly past even the early majority and a good way into the late majority. The late majority are organizations that 2 years ago weren't consider...

Every IT system has an internal data and object model. This model is matched with a relational model that turns into the supporting database. When systems interface, bridging the data & object model from one system to another is a significant effort. Signifcant effort means up to 50% of the integration effort! The historical method is for the exposing system to slightly simplify and expose it’s model, and the receiving system to create significant manual code to transform from the exposed model to it’s internal model. Over the past ten years or so, a large number of industry IT consortiums have been working to create industry data+transaction standards. These standards are designed to be highly interoperable, covering each business process and data objects for the particular industry. They appear rather complex (they are rather complex) as they’re designed to cover all aspects of an industry object with necessary flexibility. However, literally years of thought have gone into co...

For some years I've been dealing with IBM SOA oriented products. About 3 years ago I had a chance to be in an IBM center and discuss their EBS product strategy with some top IBM SOA experts. As IBM had (and now has even more) products in the space, I was trying to make sense of where to position which product that was being pitched to our large enterprise IT. At the end of the conversation I was not successful. Recently I was speaking to a top MDM expert about Oracle's product strategy in the MDM space. I was commenting on Oracle's "product", for which I had recently received a vendor pitch. He responded that Oracle has 5 products competing in the MDM space (and primary MDM tools). Today I'm trying to produce an architecture model for a medium sized IT shop that purchased IBM DataPower to include within their existing SOA model (and fit with existing tools). In my search I came across this slide from IBM... I see... one product provides fast connectivity...

Governance (service governance, design time governance, run time governance) is not going away. There are some who are under the impression that as the tools become more sophisticated and become integrated into the environments, and as SOA progresses into a realistically deployable cloud computing model, that governance becomes part of the background operation. Todd Biske makes a very convincing argument otherwise ... The ...problem with (this, that governance is going away) statement is the notion that design-time governance is only concerned with service design and development. That’s simply not true. (There are) three timeframes of governance: pre-project, project, and run-time. There’s a lot more that goes on before run-time than design, and these activities still need to be governed. It is true that if you’re leveraging an external provider, you don’t have any need to govern the development practices... Todd has a lot more to say on the matter worth reading. But the point is, e...

Integration is tough. Traditional IT applications are spending as much as 40 percent of their budget on integration. As the environment complexity increases as well as the number of connections per system, that number may increase to 60 percent. Why? At the basic level every system has it’s internal data model and logical model. Every interface has to bridge and convert those models (for the interfaced elements) across two systems. Then there’s the practical aspects – matching connectivity technologies (or bridging them), matching security patterns, simply determining appropriate error handling, human contacts, etc. SOA has standardized the interface technologies and provided a wealth of tools to bridge the issues where standards don’t match. Most organizations are using these tools today, whether intentionally or because the programmers are using recent development tools that use SOA interface technologies by default (the more likely situation). Interfacing significantly easier natura...

Twenty five years ago my boss handed me the primary billing program and described a series of new features needed. The program was about 4 years old and had been worked on by 5 different programmers. It had an original design model, but between all the modifications, bug fixes, patches and quick new features thrown in, the original design pattern was impossible to discern. Any pattern was impossible to discern. It had become, to quote what’s titled the most common architecture pattern of today, ‘a big ball of mud’. After studying the program for several days, I informed my boss the program was untouchable. The effort to make anything more than a minor adjustment carried such a risk, as the impact could only be guessed at, that it was easier and less risky to rewrite it from scratch. If they had considered the future impact, they never would have let a key program degenerate that way. They would have invested the extra effort to maintain it’s design, document it property, and consider ...