Notes from the Joint Accessibility and Assessment SIG Meeting

Back to SIG Meetings Index.

Click on topic (5 topics) to move to section:
Introduction, Welcome and Notices.
Accessible e-Assessment: What We Need to Think About - Sharon Perry, CETIS.
Accessible e-Assessment: Issues and Experiences - John Kleeman, Questionmark.
Awarding Bodies and Accessible e-Assessment and the Accessible e-Assessment Checklist Game - Simon Ball, TechDis.
Future Accessibility Work - Andy Heath, Axelrod Access For All.

The Joint Accessibility and Assessment SIG Meeting was held at the University of Wales Bangor, on 27th June 2006.

Introduction, Welcome and Notices.

The meeting was held in the lovely surroundings of the Oswalds' Conference Centre at the University of Wales Bangor. Thanks to all the staff at Oswalds for a smooth and pleasant day.

CAA Conference 2006 - Myles Danson, Director of the annual Computer Assisted Assessment Conference at the University of Loughborough, reminded participants that this year's conference would be held on 4th and 5th July. This year will be the biggest to date, with over 150 delegates indicative of a broad range of stakeholders. Proceedings for this and previous years are available free online from http://www.caaconference.co.uk, with all papers being double blind peer reviewed.

Changes to CETIS - Paul Hollins, CETIS Business Manager, outlined some forthcoming changes to CETIS. The existing CETIS project was successful in tendering to run the new JISC Educational Technology and Interoperability Standards and Specifications service for five years from 1 August 2006, with a review after three years. The new JISC-CETIS service will be known as the Centre for Educational Technology and Interoperability Standards, and will cover some of the pragmatic challenges of interoperability such as retrospective interoperability and closed systems.

Most of the existing SIGs (special interest groups) will continue but their activities will be reviewed, with more joint events reflecting joining up in standards work.

JISC will be investing a considerable amount in development work over the next few years and there will be a major role in that for CETIS with closer integration between JISC development activities and the SIGs, although the SIGs will retain their independence. For example, the SIGs may be able to award small scale funding to develop ideas in specific areas, report writing and initial preparations.

A new reader, Dai Griffiths, will be starting with CETIS in Bolton in August, and recruitment is underway for a Deputy Director, expected to be in place by September. There has also been some internal reorganisation within CETIS to aid communication and ensure that community input continues to drive the SIGs.

Back to Top of Page

Accessible e-Assessment: What We Need to Think About
Accessible e-Assessment: What We Need to Think About (HTML Format);
Accessible e-Assessment: What We Need to Think About (PowerPoint Format - 72Kb);
by Sharon Perry, CETIS.

Sharon introduced some of the issues teachers and developers should be aware of when trying to make (e)assessments accessible. These include:

• Technical: such as font size, background colour, effectiveness and usability of assistive technologies and hardware, compliance with appropriate standards and specifications.
• Pedagogic: including learning outcomes, test objectives and test validity.
• Strategic: such as departmental or institutional policies, legislation, interoperability and results handling.
• Practical: such as consultation with students, physical environment, time allowed and security.

Back to Top of Page

Accessible e-Assessment: Issues and Experiences
Accessible e-Assessment: Issues and Experiences (HTML Format);
Accessible e-Assessment: Issues and Experiences (PowerPoint Format - 1.65Mb);
by John Kleeman, Questionmark.

John gave participants an insight into the issues around making software accessible faced by a major commercial developer. QuestionMark has been in existence for over eighteen years, developing from a DOS based system to Windows and Macintosh versions and now web based assessment, and the company has a strong commitment to quality e-assessment. John's presentation led into a discussion session around the issues raised.

Accessibility for disabilities, rather than location or time, is not often high up customers' lists of priorities. In commerce, it is often the survival of the fittest, and those who provide what customers want will survive. However, it is rare that accessibility features are requested - but why is this? Legislation has had a significant impact, particularly in the US as a result of Section 508 which requires that software be accessible from the start. Legislation in the UK also needs to tackle this issue head on as there is no requirement for developers to adhere to particular standards. There is however a lack of clarity about what needs to be done, with WCAG v2 being subject to debate about whether it is 'right' or not, combined with varying regional requirements. WCAG is also difficult to implement.

Where an issue is of particular importance to a customer they may 'sponsor' its development. The ability to override time limits for individual candidates within a cohort (rather than setting up a separate assessment for them, a significant management issue) is the only accessibility related example of this, but as a very recent development it may be indicative that accessibility is becoming and increasing priority.

QuestionMark also offers the ability to construct and print paper-based assessments which can then be scanned using Principia to produce comma separated files which can then be merged with results from the same assessment taken online for automatic marking and grading. There are also means of navigating through versions relevant to individual learners. Paper-based delivery and scanning is used not only for accessibility reasons but also for practical reasons, such as when 250 candidates are sitting an exam and there are not 250 computers available to them.

From the commercial perspective, users need to make the decision to make assessment more accessible and press developers for this functionality; at the moment users want more 'bells and whistles'. A good taxonomy of disabilities and how they can be accommodated is also required. There is excellent information available, such as on the TechDis website, but people need to access this.

Where an assessment system is embedded in a platform such as WebCT, the software needs to know how to talk to the underlying architecture to obtain accessibility information.

As accessibility can be seen as an emotive issue, there is no competition between vendors on the basis of accessibility. This means that it is not very clear to developers what the issues actually are, and that all vendors are at the same level. Customer surveys never show accessibility as a high priority, but is this because of poor communication or because they're using workarounds? Pressure on vendors needs to come from purchasing organisations, yet purchasers are often very far removed from practitioners who are picking up issues and often having to spend considerable amounts to meet students' needs. The accessibility of the authoring software itself is also a low priority for buyers.

There is also a danger of looking at technical elements within a question, such as making a Flash animation accessible, rather than accessibility within questions themselves. Work flow and quality assurance processes should be incorporated into the development process, and can even be automated when using Section 508 which takes a checklist approach to accessibility evaluation. Constructivist approaches, where students develop learning and assessment materials for their peers, also tend not to take account of accessibility requirements.

These issues aren't isolated to assessment but are relevant to delivering learning in general. Validity is a big issue: do accommodations affect validity? The relationships between pedagogy and technology, and online and offline, are major issues.

Back to Top of Page

Awarding Bodies and Accessible e-Assessment and the Accessible e-Assessment Checklist Game
Awarding Bodies and Accessible e-Assessment and the Accessible e-Assessment Checklist Game (HTML Format);
Awarding Bodies and Accessible e-Assessment and the Accessible e-Assessment Checklist Game (PowerPoint Format - 415Kb);
by Simon Ball, TechDis.

Simon delivered his presentation using remote presentation software. After a few initial teething troubles, the system worked extremely well and provided an excellent example of the possibility for synchronous remote participation.

TechDis have done considerable work with awarding bodies, which will be fed up to the DfES for implementation within FE. They have no specific plans at this time for HE, which is hugely fragmented.

As part of his presentation, Simon asked participants to undertake several exercises to simulate the experience of individuals with accessibility needs. These vividly illustrated the challenges faced both by learners and by developers in attempting to increase the accessibility of resources.

Simon also pointed out a range of accommodations which can make e-assessments more accessible. These include placing radio buttons or tick boxes for multiple choice or multiple response questions after their associated distracter rather than before, which significantly increases accessibility and navigability. This is not possible in some delivery systems, which is why good, widely adopted guidelines are necessary.

Back to Top of Page

Future Accessibility Work
Future Accessibility Work (HTML Format);
Future Accessibility Work (PowerPoint Format - 1.79Mb);
by Andy Heath, Axelrod Access For All.

Andy's presentation included a standards update and discussion of blended accessibility requirements.

WCAG v2.0 is almost here. It does not cover cognitive disabilities. v1 had many problems yet represented a significant step forward. The documentation is a mess but that is only to be expected and it is gradually being improved.

WAI Dynamic Web Accessibility such as AJAX provides accessible Javascript. This is not addressed in WCAG v1.0.

The IMS Access For All meta-data standard is being developed into the new ISO Individualised Adaptability and Accessibility standard and due for release in September 2006.

Digital resource descriptions combined with personal needs profiles can produce matched content: content located, transformed, augmented or adopted to match personal needs. This work is based on the concept of distributed resources being brought together to produce personalised resources, being able to relate media with parts of other media. This is related to work being undertaken by the IMS Common Cartridge and Rich Media working groups, together with IMS Content Packaging v1.2 and Digital Rights Management work. Content Packaging is related because of the solution it has adopted for dealing with varying resources, solving many use cases not just within accessibility but also for language, bandwidth, etc. The most appropriate resources are selected at run-time.

The IEEE LTSC RAMlet (resource aggregation model for learning and educational technology) has produced ontologies which enable automatic transformations.

The IMS Rich Media group may tackle the difficult issue of digital rights management. Issues relevant to this include screen reader access to .pdfs, and the development of trusted tools which are able to access the underlying code to allow screen readers to function.

The SC36 working group 7 is looking at blended accessibility solutions in the context of Access For All.

The emphasis is on a move away from modal thinking towards services architecture.