Keywords: Bid, Automated Assessment

Test Bank Interoperability: defining and designing a prototype tool for test bank interoperability.

Background

This proposal comes as a result of the convergence of two fields of work initiated in the Interactive Learning Centre and the Multimedia Research Group at the University of Southampton. Work is currently proposed (Davis & White 1999) to develop a test bank of peer reviewed questions to be used as a resource for automated assessment in electrical and electronic engineering (E3AN). Alongside the issue of question content, there is an issue of question format. The Instructional Management System (IMS) Project hosted by EDUCAUSE in the US and represented in the UK by the JISC funded IMS Project hosted jointly by Bangor and the Open University is concerned with establishing standard formats for information exchange through the use of descriptive meta data. A range of different question engines are in use in UK Higher Education and there is a need to determine an appropriate data format for question banks which will enable interoperability of the banks between different test engines.

Investigation to be carried out

The task is to research existing standards for representation of Computer Automated Assessment Questions (principally multiple choice questions) and to produce tools (where necessary) for interchange between different standard formats.

Role of the student/methods employed

i) research current activity in the area of test question interoperability ( 3weeks).

ii) define a format for interoperable test question bank data which incorporates relevant IMS standards ( in parallel with i above).

iii) design implement and test a working tool which takes question bank data from the most commonly used engines and transforms it to a standard format which can be imported into a standard question bank. (3 weeks).

iv) design implement and test a working tool which takes standard question bank data and transforms it to formats which can be imported by the most commonly used question engines. ( 3weeks).

v) produce brief interim and final project progress reports ( 1 week).

Objectives to be achieved

i) produce a survey report based on the research which identifies the most commonly used question systems, the question formats which they support, and the file import and export facilities provided by the systems.

ii) produce a software tool which takes questions exported from the most commonly used test engines and renders the data into a standard test bank format incorporating IMS headers.

iii) produce a software tool which takes questions from the standards test bank format and renders the data into a format which can be imported by the most commonly used question engines.

iv) produce a summary report which reflects on the research and development process, and points to areas of further activity.

The supervisor appreciate that the implementation of tools that would deal with import and export of all types of questions to all existing packages is an enormous task. The important outcomes of the project are the data standard (which will be represented in XML) and a framework for transferring questions into and out of this standard. It may well be that the student will only be able to deal with one question type and one question format. The coding will be done in Java so that it will be usable across heterogeneous platforms.

Supervision Arrangements

The first week of the project will be spent with the student and supervisor and E3AN project team members working together breaking the research task down into a series of discrete activities with clear targets and milestones.

Thereafter there will be weekly progress meetings, although the student will be required to brief the supervisor on a daily basis, indicating progress made over the previous day, objectives for the coming day, and any modifications to the plan are being proposed in light of the progress to date.

Other Sources of Funds

The project will be supported by the Faculty of Engineering and Applied Science via the involvement of the faculty learning and teaching co-ordinator (Su White) who is also a member of the E3AN project. The student will be accommodated in the Multimedia Research Group in the Department of Electronics and Computer Science and will be provided with all the necessary facilities including computers.

No external facilities are required although it will be necessary for the student to travel to establish working relations with the National Computer Assisted Assessment Centre, and with one of the prominent assessment projects.

This proposal is for a useful and interesting piece of work which fits well within an existing programme of research within the Engineering Faculty, and the Multimedia Research Group. The objective is to provide the student with a real task which will give them authentic experience of the various components of a research career. Ideally the student will incorporate the learning over the summer period into their work of their third year project, and then forward for postgraduate study.

References

saw@ecs.soton.ac.uk

15 February 2000