Sections

1 Purposes
2 Information
3 Architectures
4 Mapping
5 Transforming
6 Transferring
7 Feedback
Workshop Presentations

Our links

PDP survey
LIP Baseline Pack
UK Dev LIP

External Links

DPA 1998
IMS LIP

5 Transforming the information between the database and standard representations

There is a choice of two fundamental strategies at this stage. Either the specifically required software can be built, which can be labour-intensive (and therefore costly), or some specalist tools and services can be bought in, which may take less time but cost more up front. To make the right decision, one needs to understand the specifics of the the transformation challenge for one's own particular system, as the best decision could be very different dependent on the parameters. Most of the space here is used to discuss the former option, as the latter naturally involves fewer considerations.

Comparing the standard representation with that of the system

Once the mapping has been done, the way is open to creating, by hand, some example XML files which will validate against the specification schemas, representing the kind of information that is agreed as appropriate for transfer between the interoperating systems. It may be a good idea to do this in any case, for use as test data. The next questions are, how does one create these files automatically, starting with whatever form the information is held in naturally, and conversely, how does one take information in this standard XML representation and transform it so that it can be used in the target system? This are the questions of transformation.

Transformation basics

To get a grip on the question of transformation, one needs to know in detail about the information storage on the system in question. Firstly, what kind of operating system is it? Windows or Linux or Solaris or what? Then, is it a relational database or some other form of storage? If it is a relational database, then of what type? Oracle, common in HEIs? Free software, such as MySQL? And further, what is the structure of the database? What are the tables, and fields within those tables? Relational databases can be built in very different ways, with different levels of abstraction.

And then, what facilities for export and import are provided within the software, or the database? This can be a major problem. Some software vendors give very little access to the underlying data in their system. Or it can make life very easy, if the software used already provides export and import in some XML format, whatever format that is.

Approaches to exporting information

If we are dealing with an interactive, web-based system, there are probably some tools already in use to take information from the database to display in the user interface. These range from the very common Perl and PHP, through Java servlets, Active Server Pages, right the way to more wide ranging systems such as Coldfusion and Zope. Thus, there may be a relatively straightforward way of generating appropriate XML from a database.

A sensible approach here is to look for a relatively simple way of doing the job, as there will undoubtedly be many ways of doing it in a more complex way.

If the system provides some kind of XML output, however unlike the standard XML representation, another approach would be to export the information in some way that is allowed straightforwardly by the software, and then to transform it using XSLT. XSLT is not rocket science, and there are tutorials on the Web as well as in books. This needs just the time of someone who either has, or is prepared to gain, expertise in XML and XSLT technologies.

The experience of LUSID

LUSID is written in Java, using servlets, but employs an XML model for page construction. The initial output in the trials was done using a variant of the normal page output procedure.

Coverage

There is no need to try to cover all the specification during export of information. In general, it is generally recognised that PDP in particular is quite diverse, and that therefore the information that will be gathered within different system will itself differ. What is important is to ensure that sufficient information is gathered to satisfy the destination system.

Approaches to importing information

Importing information is generally more challenging than exporting it. Not only does the incoming XML have to be parsed, but all the necessary information has to be confirmed as present. If all is OK, the most challenging part is to feed the data into the database.

The experience of LUSID

The approach piloted was to take the standard XML, and transform it using XSLT into a different form of XML. This second form was designed to be interpreted in Java and converted to instructions to input into the database directly. Thus the main complexity is dealt with using technology independent of platform.

The experience of ePARS

A relevant document is given in here.

Buying solutions

If it all looks very difficult, another approach is to buy and use a specialist tool for the job.

One company participating in JISC's work in this area is ETL Solutions Ltd., and they have a Transformation Manager product which is specifically designed for this purpose.

If readers could inform the page maintainer of other solutions, they will be included here.

Testing input and output

During the building of the transformation, whether it is done with in-house software or purchased, it will be necessary to test the results of the transformation to ensure that it is being done effectively.

Testing syntax and conformity

For output of information, it is possible to take a generated file and validate it against the schemas of the standard representation. The current UK Development LIP schemas are found here within the UK Developmental LIP site, and to use these schemas, the initial element of the learnerinformation XML should be something like:

<learnerinformation xmlns="http://www.imsproject.org/xsd/ims_lip_rootv1p0" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.imsproject.org/xsd/ims_lip_rootv1p0
    http://www.cetis.ac.uk/profiles/uklip/schemas/uk_lip_rootv1p0.xsd">

so that the schemas are found in the proper place. If this is done, loading the XML file into any XML tool that validates against schemas should determine whether or not the file is, firstly, well-formed, and secondly, conformant to the schemas.

Beyond the schemas, the other thing that must be checked is that the vocabularies of permitted type values are drawn from, and mean the same as, the vocabularies defined for use with the standard representations. In the case of UK Developmental LIP, these vocabularies are held within in this separate folder. At the time of writing, their format is being changed.

The intention is to allow automatic checking that the type values are permitted in the vocabularies, though software to do this has not yet been developed.

Semantic testing for usefulness

To take testing beyond syntax and conformance, one really has to get an outputting and an inputting system working together. The principle then is to trace through individual parts of the learner information from source to destination, comparing what the information meant in the context of the source system with what it appears to mean in the context of the destination system.

However, this is unlikely to catch all problems that might arise, given that the system users may not use the system for the purposes they were designed for.

If you are a members of one of the JISC MLEs for Lifelong Learning projects, we are here to support you. For general and non-technical matters, please contact the Centre for Recording Achievement. For technical questions you may contact Simon Grant directly.

Please also send suggestions for additions or improvements to this page or site to Simon Grant.

Revised 2004-01-23 14:32:29