Microsoft extends webservices, IMS Enterprise and LIP support
The extended support consists of a raft of new, education specific features built on the Microsoft BizTalk 2004 platform. The lot is known as the Enterprise Services Framework (ESF), and essentially aims to be the traffic cop in the heterogenous, multi-platform environment of a typical FE or HE network. It can take information in wide variety of forms, and push it out again in standard and not so standard formats.
Health warning: I've not tested the ESF myself, especially because that makes most sense in a real-life Managed Learning Environment (MLE). In the end, the value of a solution such as EST on BizTalk lies mostly in its ability to take the peculiar set of systems of a particular institution and stitching them together. Nonetheless, the concepts and the approach are well worth a further look.
The ESF is essentially a bag of functions that, taken together, allow systems to be integrated using standard data formats such as IMS Enterprise 1.1 and Learner Information Profile (LIP) 1.0. Other supported standards include HRXML (Human Resources XML) and XBRL (eXtensible Business Reporting Language), but there are tools to handle almost any other suitable XML format.
The basic function of the ESF, and the Source Data Manager that sits alongside of it, is simply data synchronisation. i.e. making sure that all systems that need data such as learner information, group data or student records get the latest version from the right source, on time and automatically. Except that synchronisation is seldom that simple.
How it's done
To achieve it, the ESF is built around a hub, with a hub manager, a hub services framework, and an agent framework around it. The hub provides the basic messaging infrastructure. The hub manager polices which applications are allowed to provide or consume particular kinds of data and it makes sure that the messages actually get delivered. The hub manager also provides a management user interface that allows system administrators to configure the traffic flows, as well as the data formats of the messages that are coming in and going out.
The hub services framework provides the message interfaces from the agents to the hub, and the agent framework provides the all important bridges between the systems out on the network and the whole ESF system. Anything outside of the hub and its service interfaces doesn't need to be a Microsoft .Net application, though it will probably help when developing agents.
So in a typical case, one might have a Student Record System (SRS) that needs to talk to a Virtual Learning Environment (VLE). If the SRS can output data in IMS Enterprise, it will hand over the flat XML file to its publishing agent, who sends it to the hub services manager using Web Services Reliable Messaging (WSRM). The hub receives the message, looks which system it came from, determines which channel (i.e. synchronisation connection) it is part of, and in what format the message should be. With security credentials of both the publishing SRS and the receiving VLE checked and the message format itself validated, it can then pass the message on to the receiving agent at the VLE.
The wider picture
There's three main aspects to note about this set-up. First, it is largely built around WSRM. Second, it is a hub and spokes design that mixes standard based with proprietary technology. Third, it has a pretty flexible agent layer. It is number three that coud alleviate some of the drawbacks of numbers one and two.
WSRM is one of many asynchronous reliable messaging specs for web services. Other major vendors are pushing different reliable messaging specs, which means that a lot of people have focussed on the more mature, but less sophisticated synchronous web services such as those profiled by the WS-I consortium.
The new IMS Enterprise Services spec, for example, is such a synchronous web service. It assumes a direct request - response type conversation between a service provider and a service consumer. Which is fine, until a request doesn't get a response because the Enterprise Services provider is overloaded, or the network is playing up, for example. In such cases, the consuming system could hang, or just not be sure exactly what kind of data it is getting, or whether it is complete, or in the right order etc. Reliable messaging specs take care of that by allowing rather more sophisticated conversations between service providers and consumers that keeps everyone in the loop.
The set-up can be made even more reliable, and arguably more convenient, by setting up a hub (such as the ESF on BizTalk) that takes care of the message flow. Rather than making sure that each system on the network is behaving, you can administer and control the traffic in one convenient location. In the case of the ESF, that also allows you to tap into things like Microsoft's Active Directory for single sign on functionality, and Microsoft's SQL server for persistence and other goodies.
All of which is great, but could find you relying on a single vendor for the very core of the institutional network. Some may find that an acceptable trade-off, others may prefer not to put so many eggs in one basket.
Fortunately, there might be a way to spread such a political risk, and that is to make maximal use of the ESF's agent layer. It shouldn't be too difficult to develop agents that take relatively simple webservices such as IMS Enterprise Services on the system end, and more sophisticated WSRM on the hub end. Especially since such an agent would have to be written only once per synchronous service spec. That way, Microsoft's ESF hub becomes nice to have, but not essential. With a little work, stuff can be made to work without a hub, or with a different hub.
Also, Sun and Microsoft are chairing a group within IMS that is defining, among other things, a reliable messaging specification that can be used with other future IMS specifications. In other words, the present IMS Enterprise Services spec is likely to gain a beefed up, asynchronous reliable messaging version that is supported by all major vendors.
But that's not here yet. For now, the ESF looks like a potentially very useful solution, particularly for those shops that already make use of various Microsoft networking platforms and technologies.
The CETIS Enterprise SIG regularly discusses enterprise integration issues, both on a mailing list and face to face. Any real-life experiences with the ESF on BizTalk kit would be much appreciated.
The ESF is available for testing by getting in touch with your friendly neighbourhood Microsoft Business Development Manager, Technical Evangelist or Partner Engagement Manager.