Exam in Hull, results from Glasgow
The TOIA (Technologies for Online Interoperable Assessment) project has successfully run a formative maths exam that took place in Hull on a TOIA server in Glasgow. The hosted service for the QTI compliant system is designed to make it easier to use for those (UK) institutions that don't have the resources or inclination to run a TOIA server of their own.
The assessment that the 59 engineering students from Hull took is the first in a pilot of a proposed hosting service from the TOIA team at Strathclyde. According to Prof. Keith Attenborough, who organised the exam, the students appeared to like answering the questions online, and the fact that they could get their marks immediately after finishing the exam.
From an educator's perspective, the professor thought that "there was a significant learning curve in respect of creating and publishing assessments and registering students as users but this has paid off in respect of substantial time saving and flexibility."
What brought Strathclyde and Hull together in this venture is the HELM (Helping Engineers Learn Mathematics) project, which is busy authoring thousands of math questions for use in undergraduate Engineering courses. Through working with this assessment bank, the TOIA project manager, Jalshan Sabir, realised that offering a centrally hosted solution would best serve the immediate needs of the community, and help the transition from application specific assessment bank to those based on standards- IMS Question and Test Interoperability (QTI).
Apart from Hull, Oxford has also signed up to the hosted service, with more expected until the pilot finishes in August 2005. Until that time, the hosted service is free of charge to members of the UK Further and Higher Education (FE, HE) community.
The wider significance
The hosted TOIA service is an early indication of a move away from functionality locked in a single box in a single institution, to a rather more flexible service oriented approach. In this case, an institution does not have to have its own questions in its own assessment bank, running through its own assessment management and rendering service to the PC lab down the hall in order to do Computer Assisted Assessment. Instead, the questions can be either home grown or brought in from existing banks, and everything else but the PC lab outsourced to people with the resource and inclination to look after all that stuff.
In the near future, it should be possible to disaggregate that functionality further. The standardised, QTI question format already means that the questions themselves can already be independent from where they are authored and stored, and which system plays them. This could be extended further by decoupling the assessment management from a rendering service such as APIS.
The upshot of all this componentising is that people can choose what system to use to author question items, what system to store them, what system manages the test, and what system takes care of the rendering, and where each of these applications live.
That kind of flexibility is more practical than it may, at first, look. Though the TOIA application, for example, fits most uses most of the time, it can't fit all situations perfectly. The server software, for example, runs on Microsoft's Internet Information Server (IIS) only, which may be inconvenient and/or expensive for those shops that rely on Solaris or Linux infrastructures. That kind of situation is now addressed by the hosting service.
Another situation would be where institutions have decided to discourage the use of Microsoft Internet Explorer (as happened recently at Penn State University), which is required by the present version of TOIA. The ability to use a different rendering service would solve that problem, and also make it easier to use the same questions without the overhead of a full assessment management system; inside a VLE, for example.
Conversely, if an institution requires extremely robust security for exceptionally high stake assessments, some other assessment management solution could be slotted into the setup.
For right now, though, TOIA already offers two important steps; the hosted service, and the ability to handle the standardised question format on which any further progress relies.
The TOIA press release about the online Hull exam.
The TOIA hosting service pilot page.
The homepage of the HELM project.