skip to main page content CETIS: Click here to return to the homepage
the centre for educational technology interoperability standards

skip over the long navigation bar
Press centre

Inside Cetis
what is Cetis?
Contact us
Cetis staff
Jobs at CETIS


XML: Click here to get the news as an RSS XML file XML: Click here to get the news as an Atom XML file iCAL: Click here to get the events as an iCalendar file

what are learning technology standards?
who's involved?
who's doing what?

CETIS Groups
what are cetis groups?
what difference can they make?
Assessment SIG
Educational Content SIG
Enterprise SIG
Metadata SIG
Life Long Learning Group
Portfolio SIG
Accessibility Group
Pedagogy Forum
Developer's forum

Accessibility (310)
Assessment (74)
Content (283)
Metadata (195)
Pedagogy (34)
Profile (138)
Tools (197)
For Developers (569)
For Educators (344)
For Managers (339)
For Members (584)
SCORM (118)
AICC (18)
CEN (34)
DCMI (36)
EML (47)
IEEE (79)
IMS (302)
ISO (21)
OAI (24)
OKI (20)
W3C (37)

print this article (opens in new window) view printer-friendly version (opens in new window)

Repositories open up to web crawlers

A new gateway service allows web crawlers the programs that index web pages for search engine sites like Google to access metadata in repositories that implement the Open Archives Initiative (OAI) protocol.

Many libraries and databases are still a closed book as far as the web is concerned, with search engines unable to peer inside and index their contents. Using the new DP9 gateway, a repository that uses the OAI protocol can be indexed by web crawling software, enabling material held in the repository to show up in web searches.

DP9 does this by providing a persistent URL for repository records, and converting this to an OAI query against the appropriate repository when the URL is requested. For example:

This URL is converted by the DP9 gateway into an OAI query to retrieve the Dublin Core metadata for this record, which is then presented to the user or web crawler as an HTML page.

Gateway services like DP9 provide one way in which existing repositories can become interoperable with the web; similar gateway services are being proposed within the IMS Global Learning Consortium's Digital Repositories group to allow applications to search and retrieve resources from both Learning Object repositories and existing library systems.

This presents a technical challenge as many libraries use the z39.50 standard for searching, whereas other repositories may use SQL, XmlQuery or other languages.

Gateway services provide a method of allowing applications to access a wide range of resources in different repositories without their developers or their users needing to understand multiple query languages and communication protocols.

The Open Archives Initiative has its roots in an effort to enhance access to e-print archives as a means of increasing the availability of scholarly communication. The organisation also develops and promotes interoperability specifications, such as the Metadata Harvesting Protocol.

For more information on the DP9 gateway, visit the DP9 website.

Related items:


No responses have been posted

Creative Commons License This work is licensed under a Creative Commons License.

syndication |publisher's statement |contact us |privacy policy

 go to start of page content