Conformance programmes gather momentum
For as long as we've been talking about standards and specifications the issue of conformance has kept cropping up. The lack of any sort of trusted agencies bestowing the Official Stamp of Approval on e-learning products brought us to a state where the claims of vendors regarding standards compliance are treated as highly suspicious - and rightly so. However, this year we've seen several strides forward, and some hints of better times to come.
The problem with conformance is the variety of possible interpretations of what such a statement means. For example, the IMS Meta-data specification contains only optional fields (other than its root element), so statements about conforming to the specification are rather meaningless.
What are needed to make conformance work are, on the one hand, a well-defined profile for interoperability that insists on minimum acceptable datasets and capabilities, and on the other a trusted agent that can test against those profiles and award certificates.
Part of the problem is technical, partly it is one of establishing appropriate legal frameworks, and partly there is an economic issue: certification testing costs money!
Back in April the Schools Interoperability Framework (SIF), the US K-12 specification body, announced the launch of its Compliance Program. SIF provides a testing kit that vendors can develop against, and licensed the certification of compliance to the Open Group, one of several well-known international product certification organisations.
Since then, the SIF Compliant certification has been awarded to over a dozen products, and each product's conformance statement can be examined via the Open Group website's SIF Certification page.
Getting a certificate costs vendors $35,000 (or a much more reasonable $3,500 if they are already a SIF member), plus $5,000 ($1,150) for an annual renewal or a recertification for a new version of the same product.
But it isn't just SIF. ADL have also initiated their conformance program for SCORM 1.2, and last month Click2Learn's Aspen learning management system was awarded SCORM 1.2 Certification. ADL have licensed the Academic Co-Lab in Wisconsin, and the US Navy's Undersea Warfare Centre in the state of Washington as its certification agents.
Unlike SIF, which concentrates on system conformance, ADL covers both SCORM-certified products and content. The ADL website has a set of SCORM-conformant content available on its website.
Both ADL and SIF are very much centred on US sectors (government/military and K-12, respectively), however, there is momentum behind efforts elsewhere - the UK, Europe, and globally - to establish meaningful conformance.
The first step is to create the appropriate profiles suitable for certification - this has already been underway for some time now in the UK, with CETIS working to develop UK-specific application profiles for IEEE LOM, IMS Enterprise and IMS LIP. CurriculumOnline and UFI have also been working on profiles for their sectors, and the government is producing e-GIF - the e-Government Interoperability Framework.
Elsewhere, Singapore has its SingCore profile, with similar efforts underway in other countries around the globe.
The next step towards conformance is to formalise those profiles and develop the frameworks needed, including the technical, legal, and economic aspects required for a formal test programme.
Supporting the move towards developing national conformance programmes is the IMS International Conformance Programme. The ICP is developing guidelines for developing profiles suitable for conformance testing, and intends to help communities working on conformance testing facilities to share tools and test criteria.
However, the Holy Grail of conformance is at the end of a long and winding road.
Building test kits takes time - and is an expensive business in its own right. ADL has been refining its test kit (and reference implementation) for quite some time; SIF were relatively lucky in having a specification that is inherently amenable to testing (and SIF has a highly capable developer donating his efforts).
Legal and economic issues have to be dealt with to make testing viable - someone has to be willing to take legal responsibility for certification, and be remunerated for their risk. Someone needs to pay for testing and certification - usually the creator of the product or content - and the costs need to be sufficiently attractive to make certification achievable and desirable for a broad range of products and content.
The final piece of the puzzle is the customer. When profiles, test kits and certification agencies become available, we have to ensure that our procurement procedures take cognizance of that fact. If we don't demand certification of conformance to standards in products we buy, we aren't going to get it! Having a certificate of conformance on the product has to mean something in the procurement process, or the cost to vendors can't be justified.
What this all means for open-source solutions, and home-grown content, is still up for debate. The availability of free test kits for profiles - something both SIF and ADL have provided - would enable developers and academics to "self-certify" their products, giving a reasonable indication of conformance, but without the legal liability and cost of certification. When procuring an open-source system, the onus would be on the customer to verify conformance tests, and take on the associated risks when connecting with other systems.
So will the Official Seal of Approval for standards-compliance starts appearing on products in the UK? I think its safe to say yes, it will - eventually.