AIMS: Academic Information Management System
Final Report
December 1999
Author: Gareth Hughes
Investigators: Professor Wendy Hall, Professor Tony Hey, Dr Les Carr
Multimedia Research Group
Department of Electronics and Computer Science
University of Southampton
1: Table Of Contents
2: Executive Overview
This project has set out to develop a Web based document management system tailored to the needs of an academic setting and to apply open linking research techniques to enhance the content of the material. It is designed for administrative personnel to contribute day to day administrative documents directly from their Web browser. The server automatically generates Web pages from the content of documents. The system is now in use within the author's department, under trial at a number of other locations and freely available for use by members of HE in the UK. The main report describes the project and system. There is a detailed report of the research aspects of the project in Appendix 1 "The Application of Open Linking". The report is complemented by a manual covering usage, administration and installation which is available for download from the project Web site, see "URLs for Project Reports" on page 17.
The system is a Lotus Notes application that runs on a Windows NT based Lotus Domino server. Contributors create documents such as minutes or newsletters as normal using their usual applications. When a document is complete the creator submits the file with a minimal amount of information to the AIMS server via a simple Web form. The AIMS server understands a variety of formats and imports them into the internal Lotus Notes database format. From there the server can generate the Web pages on demand. Seconds after submission the content of a Word document will be available on the Web without any manual creation of HTML. The server does not store HTML but generates it on demand creating pages designed using sophisticated development tools. Contributors do not need to be able to create Web pages or have to manage the navigation between documents in the system.
Facilities include a full text search engine, version control, sophisticated security measures and automatic creation of matching Adobe Acrobat files. The system also acts as a form of backup by storing the original file. A variety of tools have been developed to manage the system and to aid contributors in managing their documents.
Due to the nature of the design it is not desirable to create normal static HTML links between documents in the system. A variety of experiments have been performed to find optimal ways to enhance the documents stored in AIMS with relevant and dynamic links. Work carried out within the author's research group has been utilised and extended in order to deliver documents with links contained and controlled by an external linking system. See Appendix 1 "The Application of Open Linking" for details of a variety of methods and experiments of utilising dynamic linking.
The system has been in use within ECS at the University of Southampton for nearly a year. A number of trials have also been carried out elsewhere during that time. This report includes the results of formal testing carried out within the department.
AIMS is now ready to be released. After reading this report interested parties are welcome to browse the running system on the project server and to contact the author in order to proceed with a trial of the system.
3: Acknowledgements
Gareth Hughes, Multimedia Research Group, Zepler Building, Department of Electronics and Computer Science, University of Southampton.
Email: gvh@ecs.soton.ac.uk. AIMS Home Page: http://aims.ecs.soton.ac.uk/
This is funded by JISC, project number 462.
The project was managed by Professor Wendy Hall, Director of the Multimedia Research Group and Professor Tony Hey, Dean of the Faculty of Engineering and Applied Science. User testing was done by Dr. Gary Wills of ECS. Advisors to the project are Dr. Les Carr and Dr. Tom Heywood of ECS and Dr. Carole Goble of the Department of Computer Science, University of Manchester.
The author wishes to thank the patience of his users at Southampton and the persistence of those wishing to trial the system. He is also grateful for all the help from fellow Notes developers and the actual writers of Notes on the Domino forum at Notes.Net without which he would have been doomed.
4: Background
The ECS department has had a need for an electronic document management solution for some time. Almost no administrative information was available online. There was no policy for keeping documents and no electronic archive. Documents such as minutes were produced by secretaries, printed and sometimes the files lost. In a number of cases the electronic files were deleted when creating the next set of minutes. Because none of the administrative staff had any knowledge of creating Web documents none of the information could be found online. The project brief was to create a system that would enable the authors of the documents to directly contribute to the Web site without needing a large upheaval in working practice or the intervention of a Web site editor. The system generates the pages on the fly reducing the enormous link maintence overhead of a traditional Web site.
This project did not set out to create a paperless environment, documents are still produced to be printed first and submitted to AIMS second. As the project has progressed many more people are questioning why this is still done indicating a slow culture shift. The system has been designed to allow people to work as they always have done using the tools that they wish to use. No standards have been forced upon people and there have been no working practice rules enforced because the author was in no position to do so. In an industrial environment these types of working practices are the norm whereas in academia it is almost impossible to tell people how to work. The design of AIMS reflects this ethos. For instance the site expects the minimum of information about each document in order not to discourage users from using the submission form. The site has also been designed for access by any Web browser rather than just the most modern. Features such as frames, Javascript, Java applets and advanced HTML tags are not used. Special areas are provided for access by PDA's and text based Web browsers.
5: System Highlights
Lotus Notes was chosen for the project because it is an application development environment. It is not a document management system and it is not an email system. It is a complex and highly programmable starting point for organisations to create applications that reflect the way the company is organised and works. Because it is this powerful and complex a large investment is required to create the bespoke applications that are its speciality. This is why a lot of people do not understand it and major corporations use and benefit from it most. Earlier project reports have more complex explanations of Notes, see "References" on page 16.
The basic design of the AIMS system is an automatically generated document database accessible from any Web browser. This is implemented as a complex Lotus Notes database and hosted by a Lotus Domino Web server. Submission to AIMS is achieved by a person submitting a file produced by a word processor accompanied by a few details about the document. The AIMS application imports that file into the database and the Domino Web server renders the content of the file as a Web page automatically. The main deliverable is a Lotus Notes database template file which is used to create AIMS databases on a Lotus Notes server. The AIMS Web site currently hosts 6 databases for various departments of the University and all are maintained through the use of the template system.
The documents in the AIMS system are listed in the Document Store. This lists the current versions of all the documents in the system by the Category of the documents. The Category is something that the contributor chooses when submitting a document. Older versions of documents can be accessed from the Document Archive listing.
To submit a new document the contributor fills in a form. The person selects the actual file to put into the system using the Browse button. They then add a Title, Subtitle, Version number and choose a Category. Once the file type is chosen the form is submitted and the server does the rest. The contributor does not need to create a HTML file.
Once a document is in the system it is possible for the contributor to alter the details of that document via an online form. This is useful for correcting spelling errors in fields, changing the security settings of the document or changing the category of a document.
To update a document in the system the contributor opens that document and clicks on the link "Submit a New Version". This will open the document submission form and fill in many of the fields for them. The person simply needs to select the new file and submit the form. The system will add in the new version of the document and provide navigation links between the different versions.
The displayed document is not suitable for printing and may not be an accurate rendering of the original file. To solve this problem AIMS creates an Adobe Acrobat file version of all Word and WordPerfect documents in the system. This is an automated process that is handled by the server overnight.
Other interface features include a full text search engine tailored to the system.
6: Planning to deploy an AIMS system
6.1: Technical
The AIMS system runs on a Lotus Domino 5 server running on a Windows NT server. Almost any modern PC is currenty capable of running the application though a large amount of memory, at least 128Mb, is recommended. This is covered in detail in the manual, see "URLs for Project Reports" on page 17.
6.2: Managing the Running Service
Once an AIMS system is running there are two distinct areas of work required to look after it. The first is technical. The administrator needs a certain amount of Lotus Domino knowledge. Many tools are provided with AIMS to maintain the documents and the database but using them effectively will take understanding. The code is written in a language very similar to Visual Basic so it is not a great learning curve. The server administration tools shipped with Lotus Domino are powerful and well documented.
The second, vital, role is editorial. This role is crucial in keeping the site up to date and relevant. As well as expecting all secretaries and administrators to actively submit their work to the server a person or persons will be needed to ensure that other documents are in the system. That person needs to know what documents exist and where to obtain them electronically to ensure they are put online. In ECS the major regular submission of documents follows the meetings of the Departmental Board. The minutes, agenda and accompanying papers are submitted to AIMS. Some of the papers are by academic staff and the system works most efficiently if one person ensures that all documents are submitted. This may require that person to ask others to submit them or to gather the files and do it them self. That person will also have to respond to requests to find and submit certain documents. The author has recently added access to 58 forms that can be downloaded and printed from AIMS. This gives all staff the ability to print their own travel expense claim form rather than ask a secretary for one. This has generated requests for a lot more forms to be put into the system and this is going to require considerable knowledge of the workings of the University administration system in order to identify them.
6.3: Training
Persons wishing to read documents on AIMS Web site do not require training as the site is simple to navigate. The training of staff who will contribute documents is vital. The secretaries in ECS were trained to use AIMS in a short session of less than an hour. The training was done in small groups of around 4 people with a programme of hands-on activities. They were shown how to navigate the system, use the major features of the system and then guided through submitting documents. They then edited the details of that document and submitted new versions. As well as the large manual the author has also created a double sided reminder sheet on how to submit a document and how to edit the document details. This is in the back of the manual and can be obtained from the project Web site. The secretaries quickly understood and appreciated the system. In many cases they did not start using the system at that time and so have needed a quick reminder lesson at the time when they first started using the system properly.
6.4: Lessons from the trial by University of Southampton administration staff
In April 1999 the University Information Strategy Committee recommended a six month trial of the AIMS system to store the minutes and associated papers of a number of committees of the central administration. The Committee was set up to create a long term strategy for an intranet and electronic document management system across the University. The trial consisted of the author hosting a separate database on the ECS server into which select staff, mainly committee secretaries, would submit documents. The author worked with MICS, Management Information Computer Services, to discuss the requirements of the trial. There was particular emphasis on restricting access in a highly complex, fine grained manner. Each committee would be a category in AIMS with sub-categories. Only the secretary of that committee would be able to submit documents to that category. Some secretaries were responsible for more than one committee and some access controls needed to be for more than one person. The result was a many to many relationship between contributors and their set of allowed categories. The solution was difficult and complex to write but was completed on time.
The trial was halted due to concerns by a number of senior University personnel over the implications of putting minutes in the public domain for the first time. The Working party was instructed to stop until the matter could be raised at an even higher level and a decision made. This has effectively stopped the trial for the foreseeable future.
The lesson to be learnt from this is that an intranet or document management solution is not to be entered into lightly. There are many implications to consider including the legal issues of releasing the information onto the Web and the technical issue of how to control restricted information. The work required to make AIMS comply with the needs of MICS also highlighted that AIMS is not suitable for every organisation. The larger the body wanting to move into this world the larger the shift and the greater the cost. The AIMS application is a good system for a departmental sized organisation but it is not designed for a campus wide solution. For a system that large to work the accounts system must be integrated. All system accounts, hierarchical groups and email should be integrated into one system so that applications such as document management and work flow are easier to maintain. With that complexity comes a variety of costs; the cost of training, cost of maintenance and the main cost of severely altering the work practices of a large number of people in a coordinated manner. At that scale the cost of fully adopting Lotus Notes becomes worthwhile which is why it is used by some of the worlds largest corporations.
6.5: Other Trials
The system was also trialled at the Language Centre of the School of Modern Languages. It was set up on a separate server for storing a variety of documents produced by academics. This was the first time that AIMS had been installed on another machine other than the dedicated server in ECS. The considerable difficulties experienced led to a large amount of rewriting of the system to make it installable. When trialled at the Geography Department, Bristol University the AIMS system installed correctly but there was trouble with installing and configuring the external applications. The reliance on Microsoft Word, Adobe Acrobat and Zmerge can have its disadvantages, especially when new versions come out. It has been extremely difficult to write detailed instructions on how to install and configure the complete system. This has only been possible by the use of VMware. VMware is software that runs multiple virtual computers on a single PC, at the same time, without partitioning or rebooting. AIMS has been installed on a virtual Windows NT machine many times to test the instructions and trial research code without breaking the existing server.
6.6: Gaining Acceptance
To deploy AIMS in ECS has required a large amount of effort in a number of areas. The first has been that the requirement to make it a success must come from the top of the organisation. It must be adopted by the Department and must be made policy in order to force people to use it. This has been coupled with demonstrations to both academic staff and administrative staff as well as other publicity techniques such as newsletter articles and global emails. This process has been repeated continually in order to keep the system at the forefront of people's minds. The system requires acceptance from a large number of people. There also needs to be acceptance from technical support staff to take on the system and work to make it a success. This is not so easy because such people are usually highly biased about what technologies they will use. Often there is extreme bias towards or against certain platforms, operating systems and applications. The system still runs the risk of being abandoned simply because it uses Microsoft Windows.
On the editorial side there is a difficult start up period to overcome in which a lot of work has to be done to fill the system with documents. This `boot-strap' problem has required the author to spend a fair proportion of his time being the AIMS editor, not the AIMS programmer. The list of categories that will be displayed needs to be carefully designed by people who understand the structure of the department and what documents are available. Tools are provided to make it easy to create and delete categories. It is also possible to re-categorise large numbers of documents quickly.
7: Licencing Costs
The licensing model for Lotus Notes has changed since the project began. When Notes was chosen for this task, user accounts were free for users contributing to a Notes application using a Web browser (and not using Lotus Notes client or email). In the summer of 1998 Lotus changed the pricing structure. Every user of a Notes Web site who has an account on the Notes server and is an employee of the organisation must purchase a Client Access Licence (CAL). Users who are not members of the company do not need a licence. In other words Lotus are now charging for intranet style usage. Readers of this report are directed to the FAQ section of the following Web site. http://www.lotus.com.sg/world/sg_ppadv.nsf/.
A site deploying AIMS only needs to buy a server licence, designer licence, a number of client licences (CAL) and the other pieces of software listed in the manual. The total cost of software and licenses being of the order of £1000. At the time of writing these are the education prices for Lotus Notes: Server Licence £526.00, Server CD £16.00, Designer Licence £134.00, Desktop Licence £22.00. The School of Modern Languages recently set up its own Lotus Notes server to run AIMS, they purchased 50 CAL licences costing £400 or £8 per person. The licences and prices were from Pugh Computers. http://www.pugh.co.uk/
8: User Testing and Evaluation
A report [Wills et al 1999] has been written on evaluating the acceptance of the AIMS system by the secretaries of ECS. The evaluation focused on the subjective opinion of the people and measured the time taken to perform the major functions of the system. The report presents the methodology and describes the rationale behind the approach used. The following report summary has been contributed by the report's author. In this section references to `users' means staff who are `contributers' to the Web site and not just `readers' of the Web pages.
The evaluation was conducted in two stages [Wills et al 1999], first a contextual review [Preece 94] of the working practices of the secretarial staff was undertaken. The second stage, was a usability study of the system, which was conducted in two parts. A structured expert review using discounted usability engineering [Nielsen 89, Nielsen 94] with an additional principle of 'Provide Navigational Aids' to ensure that the hypertext components of the system are reviewed [Wills 97]. The second half of the usability study was to conduct a time trial with users completing questionnaires. The questionnaires were used to measure the user subjective opinion of the system [Hirst 95] and their acceptance to use the system [Davis 93, Davis 96], an additional criteria of Navigation was added to measure the users opinion on how well they could move around and through the information space.
The contextual review showed that currently the main method of archiving is in the paper format, making it difficult for academic staff to obtain back copies of meetings to check on discussion or discussions made at previous meetings. The length of time required to photocopy, collate minutes, put them into envelopes, stick the address labels on (which have previously been printed) then walk them to each group, takes on average between six to eight hours.
The average time to complete the tasks is shown in Table 1. The time taken to complete the tasks was consistent throughout the user group. The obvious saving of time comes from the ability to publish the set of minutes extremely quickly, when compared to the current method. The ability to disseminate information quickly would be a significant cost benefit to the department. However, this will only be an advantage if there is a paradigm shift in that an academic member of staff will need to read the minutes online, or as last resorts print them off themselves to read them.
The general comments from the users showed that they were interested in the idea of a central repository for administrative information. This was reflected in the high score from the questionnaires given to the Impression Category in Table 2, and the 'intention to use' category from the Technology Acceptance Model criteria Table 3. All the users replied that they would recommend the system to their colleagues, and that the majority (87%) could navigate the information space with ease.
The scores were normalised by dividing the score by the number of respondents and the number of questions they answered. The maximum normalised score is +1 indicating a very strongly agreed to -1 indicating a very strongly disagreed to all positively phrased questions (and visa-versa on all the negatively phrased questions). A score of 0.5 indicates that all the users agree with a positive statement, i.e. They agreed with 'The AIMS system is one that I want to use on a regular basis.' The questionnaire also showed that the scores given were not affected by the user's access to the internet from home, or the user's preference for using the internet.
9: Managing Security and Accounts
The security of the system and how it integrates with existing systems has been the most intransigent problem during the lifetime of the project. Even now there is no simple set of answers but at the time of writing it is thought that a solution has been found by using LDAP. See below. Investigations will continue in the new millenium.
The security of Lotus Notes is highly configurable allowing for a complex model of security to be created as required. The current model is to give people who wish to contribute documents into the system an account on the AIMS server. Only people who can log on can submit documents. The AIMS site is only accessible from the staff section of the ECS Web site which is secure and only accessible to staff. Staff can then follow the link into AIMS. People do not need to log on to the AIMS Domino server to read documents. Contributers can mark a document private using the details editor and this document can then only be viewed by other people with accounts. More complex models can implemented such as defining groups of people with access to certain categories of documents.
The Notes server cannot share account information with a Unix (NIS) based accounts system. Hence the need for an actual account on the Notes server. Accounts can be shared with Windows accounts through the Windows NT User Manager making it a simpler task to install AIMS into an environment where accounts are Windows based. This is not the case in the author's Department.
Account information on the Notes server needs to be maintained. This is not a difficult task and can be managed over the Web. Account holders can even perform simple maintenance tasks on their own account such as changing passwords. It is still an overhead that needs to be planned for.
The required scenario is for the AIMS database for the Department to be accessible by members of ECS only and certain types of documents to be only available to staff or senior staff. The system would then use Secure Sockets (HTTPS) for all browser access and anonymous access would be blocked totally. All readers would have to log on to access the system. This could be achieved already but each person in ECS would have to have an account on the AIMS server. The access groups would also have to be set up. This duplicates an existing working accounts system and would be completely out of the question. At the moment certain sensitive documents are not placed in AIMS.
Notes 5, released in March 1999, has the ability for a developer to add custom authentication libraries into the Web server component of the Notes server. This would allow people to authenticate using their Unix (NIS) id and password. This library would need to be written using the low level and highly complicated C API. This was too large and complex a task to undertake for this project as it was not obvious that the approach would be successful or that the code could be reused elsewhere.
Notes 5 supports LDAP (Lightweight Directory Access Protocol) which has emerged as a standard for accessing and managing directory information. This protocol allows differing systems to exchange information about people in an organisation such as email addresses and hierarchical information. For a long time it was not thought relevant to solving the authentication problems in AIMS. Investigation of what LDAP does within Lotus Domino has revealed that it can be used to authenticate people accessing a Domino server from a Web browser via a third party LDAP server. If the Department deploys an LDAP server for the existing Unix account information then the Domino AIMS server can be set to authenticate with it. This will hopefully solve the authentication problems and make the accounts on AIMS manageable.
10: Milestones
11: The Future of AIMS
The system is now ready to be released. Although the author will not be officially working on the project he will be working in ECS for some time to come and is keen to make sure the system works and is used.
The ECS Department has agreed to take over the running and maintenance of AIMS, so that it will continue to grow as a resource in the Department and the software will be maintained and developed as necessary for the foreseeable future. It will now be sent out with the user manual to a number of organisations who have expressed an interest in evaluating the system including :
School of Modern Languages, University of Southampton
Faculty of Engineering and Applied Sciences, University of Southampton
Department of Geography, University of Bristol
Surrey University Library (for storing past examination papers for the whole campus with the possibility of storing theses)
The University of the Third Millennium project, www.u3m.org
School of Information Management, Faculty of IT, University of Brighton
We are actively seeking a commercial partner with whom we could commercialise the software. One strong possibility is through our spin-off company Multicosm Ltd, with whom we are currently in discussion to try to identify a suitable partner, possible one of the US companies through whom Multicosm distribute Microcosm
12: Dissemination Activities
Dissemination activities have taken the form of publicising the online system and encouraging people to browse the Web site to try the system for themselves. To this end dissemination activities have targeted administrators and managers at HE institutions rather than Web site programmers. A two page project flyer was distributed at the Heads of Administration Conference at the Chilworth Research Park, Southampton in April 1998. This flyer is available from the project home page.
In September 1998 a one day interactive workshop was organised with invitations targeted at administrators. The invitations generated enough interest for the workshop to be over-subscribed. The workshop largely consisted of hands on sessions using AIMS with all the facilities of the system being used simultaneously by approximately a dozen people.
The invitations sent out have resulted in subsequent enquiries. Interested parties are given an account on a demonstration AIMS database and invited to try the system for themselves with their own documents. Support is given by email and via the online discussion site.
The author attended the 2nd ALT Workshop on Assisting Collaborative Education, Lancaster in October 1998. The theme of the workshop was to examine the use of systems such as Lotus Notes for collaborative teaching and learning. An impromptu live demonstration of AIMS was given which led to a full presentation at the third ALT ACE workshop on 13th and 14th January 1999 in Birmingham.
A paper was presented at the Webnet 99, World Conference on the WWW and Internet, Honolulu, Hawaii.
13: References
[Carr et al 1995] Carr, L., De Roure, D., Hall, W. and Hill, G. (1995). The Distributed Link Service: A Tool for Publishers, Authors and Readers. The Web Journal 1, 1, 647-656, OReilly and Associates.
[Carr et al 1998] Carr, L., De Roure, D., Hall, W. and Davis, H. (1998). Implementing an Open Link Service for the World Wide Web. The World Wide Web Journal 1, 2, 1998, OReilly and Associates.
[Carr et al 1998a] Carr, L., Hall, W., Hitchcock, S. (1998). Link Services or Link Agents ? Ninth ACM Conference on Hypertext and Hypermedia.
[Davis 93] Davis FD. User acceptance of information technology: system characteristics, user perceptions and behavioural impacts. International Journal of Man Machine Studies (1993) Vol. 38 pp475-487.
[Davis 96] Davis FD. A critical assessment of potential measurement biases in the technology acceptance mode: three experiments. International Journal of Human Computer Studies (1996) Vol. 45, pp19-45.
[Fountain et al 1990] Fountain A., Hall W., Heath I. and Davis H. (1990). Microcosm: an Open Model With Dynamic Linking. Hypertext: Concepts, Systems and Applications (Proceedings of the European Conference on Hypertext, INRIA, France, November 1990) 298 - 311.
[Hill et al 1995] Hill, G., Hall, W., DeRoure, D. and Carr, L. (1995). Applying Open Hypertext Principles to the WWW. International Workshop on Hypermedia Design, Montpellier, France, 1-2 June 1995.
[Hirst 95] Hirst SJ, Corthouts J. Hyperlib Deliverable 5.1: Evaluation of the Hyperlib Interfaces. Hyperlib Electronic Document Store University of Antwerp 1995 available at http://143.169.20.1/MAN/WP51/root.html
[Hitchcock et al 1997] Hitchcock, S., Carr, L., Harris, S., Hey, J. (1997). Citation linking: improving access to online journals. In Proceedings of Second ACM Conference on Digital Libraries, Philadelphia, pp.115-122.
[Hitchcock et al 1998] Hitchcock, S., Carr, L., Hall, W., Harris, S., Probets, S., Evans, D. and Brailsford, D. (1998). Linking electronic journals: Lessons from the Open Journal project. D-Lib Magazine, December 1998.
[Hughes 1999] Hughes, G. V., (1999). AIMS - Academic Information Management. The Active Web, a British HCI Group Day Conference, Staffordshire University, 20 January 1999. ISBN 1897898 45 2. http://www.visualize.uk.com/conf/activeweb/proceed/contents.asp.
[Hughes et al 1999a] Hughes, G. V., Hall, W., Carr, L., Hey, A. J. G., (1999). Academic Information Management: an Open Linking Approach. In proceedings of Webnet 99, World Conference on the WWW and Internet, Honolulu, Hawaii. October 25 - 30 1999.
[Nielsen 89] Nielsen J. Usability Engineering at a Discount. In: Designing and using Human-Computer Interfaces and Knowledge Based Systems. Editors, Salvengy G & Smith MJ. Elsvier1989, pp394-401.
[Nielsen 94] Nielsen J. Enhancing the Explanatory Power of Usability Heuristics. Factors in Computing Systems CHI '94. Boston, Massachusetts, USA, April 24-28, 1994, p152-158.
[Preece 94] Preece J, Rogers Y, Sharp H, Benyon D, Holland S, Carey T. Human-Computer Interaction. Addison-Wesley 1994.[Wills et al 1999]
[Wills et al 1999] Wills, G. B., Hughes G. V., Hall, W., (1999) Evaluation of AIMS-Academic Information Management System. Multimedia Research Group Technical report No M99-5 November 1999 ISBN-085432700-2.
[Wills 97] Wills G.B, Heath I, Crowder R.M, Hall W. Evaluation of a User Interface Developed for Industrial Applications. University of Southampton Technical report No M97-4 ISBN-0854326499.
13.1: URLs for Project Reports
AIMS Home Page. http://www.docs.ecs.soton.ac.uk/
AIMS Status Reports. http://www.docs.ecs.soton.ac.uk/aimsdb.nsf/docs/779
AIMS User Manual. http://www.docs.ecs.soton.ac.uk/aimsdb.nsf/docs/770
AIMS Reminder Sheet. http://www.docs.ecs.soton.ac.uk/aimsdb.nsf/docs/840
During the lifetime of the project work has been carried out to integrate the system with an external linking service in a variety of ways. Some of the methods attempted are documented here along with conclusions drawn, see "References" on page 16.
The consequence of the AIMS design is that documents do not contain embedded links in the way that a normal Web site would. The initial research goal of the project was to provide useful links on this content in a way that is easy to maintain. This is achieved by making use of an open hypermedia link service developed within the group. Over time the service has been implemented in two flavours though both work along similar principles. The first is the Distributed Links Service (DLS) [Carr et al 1995] [Carr et al 1998] [Carr et al 1998a ] [Hill et al 1995] developed within the group mainly funded by the Open Journals Project [Hitchcock et al 1997] [Hitchcock et al 1998]. The commercial version, Webcosm, was developed by Multicosm Ltd, a company originally founded to exploit the Microcosm [Fountain et al 1990] system. Each brings differing advantages for the AIMS project. Webcosm has a larger feature count and will be used when the linking service is made available to users of AIMS whereas the DLS source is controlled by the group and the system can be altered at a low level for experimental purposes.
The link service is implemented as a Web proxy. The user configures their browser to use the link service as their Web proxy. When the browser requests a page it will ask the link service for the page. The link service will retrieve the page from the server (AIMS) and annotate the HTML with extra links before passing the enhanced Web page back to the user's browser. Links are stored on the link server in link databases or linkbases. An entry in a linkbase consists of the word or phrase to make an anchor, the destination URL and a description of the link. When the link service is in use any occurrence of the word or phrase in pages the user views will become a link to the destination document even though this link is not in the original Web page. This is a generic or glossary link as first implemented in the Microcosm hypermedia system. An example of an entry in a linkbase would be to link the acronym of a research group in the department to the home page of the research group. A link of type local restricts the anchor to only occur in a single document.
The main linkbases in use with AIMS are a set of links for all undergraduate courses to the relevant course page, a large linkbase linking user id's and user names to home pages and a linkbase with a variety of acronyms and other useful words or phrases. The key advantage of using the system is ease of maintenance. If the home page of our research group changes then the link can be updated in one simple action by altering the entry in the appropriate linkbase. The first goal of using the link service with AIMS is to author linkbases of official terms and definitions for the department with effort going towards automating link creation at every opportunity.
A more advanced application of the linking service is to try and make more use of the structure of the minutes stored in AIMS. In the diagram below the Distributed Links Service has been modified in an experiment to link the items and actions in minutes together. The minutes of the Department Board of Electronics and Computer Science contain item numbers of the form B.n where n is a number. Frequently items in minutes will be following on from Actions and Items in previous editions of the minutes. The Distributed Links Service has been used to link together these item numbers enabling readers to follow the trails of issues back through time.
The problem with this application was that it only applied to a few documents in the system matching a specific pattern. Even then it was not obvious how to use the system and assumptions made in the code about the Web navigation patterns of users were not maintainable. For pattern matching algorithms to be more successful, users must be trained to be accurate in their use of the numbering schema. Users submitting documents could be constrained in what they can enter in the fields or more fields could be added to describe the document in more detail. This will quickly discourage users, especially if they lack confidence and are concerned that their categorisation would be considered 'incorrect'.
Webcosm ships with a Java applet link editor. A more preferable solution would be to create links whilst viewing the documents. Microsoft Internet Explorer 4 allows developers to add items to the context menu allowing the creation of a simple authoring tool. The user opens the destination and selects the appropriate word in the text to use as the anchor. An additional entry on the context (or right click) menu 'Create Generic Link' uses Javascript to create a filled in form for submission directly to the Webcosm server.
This experimental application has enough functionality to allow a person to directly author links between documents. The links will also have a high probability of still being relevant as new versions of documents are added. If the destination document is updated then the link will point to the new version of the document. If a document containing an anchor is updated then the link will survive if the same word or word-pair still occurs in the content of the new version. Whether these links are still relevant to the content is a more complicated issue.
The next step has been to try a process of pre-compilation of links. Instead of placing links on the page on the fly use a system to add the links into the documents in the system in a batch or compilation style process. An experimental system was devised to `pull' the body of AIMS documents through the link service which adds the appropriate links. This HTML representation is stored in the database and served to readers who are unaware that the main portion of the document is not actually being generated on demand. Though this has the obvious disadvantage that links are not being created on the fly the system now looks and feels like a normal Web site. This system was difficult to maintain and created many problems inside the database. It was discontinued in favour of a more elegant solution.
Webcosm is implemented as a Web proxy component and a link server engine. The linkserver answers requests on a port from remote clients. Hence it is possible to write programs that communicate directly with the link server. Within the AIMS application, programs have been written that run every time a document is requested from the server. When a document is opened by a browser a Java application asks the link server for the links for that particular document by passing the link service the text from the body of the document. The link service acts in just the same manner as it would if being used as normal as part of the proxy. For each word in the document that would have become a link it returns the HTML it would have placed into the document. The Java application collects and processes these multiple links. The application sorts and removes duplicates before parsing the HTML into link objects. These are then used to form a block of HTML code. The agent can then embed this into a field in the document. Once processing has completed the document can then be rendered and delivered by the Domino server. The links appear in the margin or at the bottom of the page as a list of recommended sites for the user to view. Though the links are not being embedded in the main flow of the document the links are being generated on demand and in context depending on the users chosen linkbases.
A refinement of this has been to combine the two approaches. The design has been modified to compile links for each document overnight using this approach. The database configuration document (the Global Profile Document) will contain a time stamp saying when the last time that the Webcosm linkbases were updated. Each night the system will compile links into new documents and if Webcosm has been updated re-compile links for all documents. The system also needs modification to cope with very large documents. Only a certain amount of text can be passed to the linkserver at a time so the Java application needs to make a number of requests with successive portions of text to the linkservice.
The limitations of generic links soon become apparent. Too many links will overwhelm the reader and be a distraction. This problem has been addressed by work done for the Elib funded Open Journals project by modifying the DLS. The DLS has a concept of link priority schemes, different links can be given a priority by their author and the system can colour the links accordingly or even not display certain priority levels of links. Using pattern matching to create links in text is shown by the example of linking Board Minutes to be a promising way to find more link anchors with less explicit link authoring. Another application of such techniques would be to help with recognising names of people within documents though such techniques are much harder to implement. Pattern matching algorithms to solve this problem would need to be complicated compromising the performance of the service and slowing the access times of documents. There needs to be an alternative approach to the proxy synchronously processing each page. See [Carr et al 1998a]. The proxy could farm out the processing of documents to agents that would each examine the document and try to provide the user with useful information and links. These results would need to be delivered to the browser separately from the document causing a number of implementation problems.
An important implementation issue is how to create useful links for the documents. Some linkbases are automatically generated by extracting data from other online systems such as the staff database. This is useful but is limited to specific topics. Another method could be to require contributors to supply potential keywords and metadata for a document. This places unwanted demands on people and would be very difficult to implement. A more interesting method is to use some system that can understand the content of the documents and generate keywords automatically. Such a system, Refindment, has been developed by Multicosm and is being tested with AIMS.
Here are some pictures of older versions of AIMS to demonstrate how the system has changed over time.
Above are pictures of Version 1 (left) and a later copy of Version 2 of AIMS. The substantial redesign between the two included the designing of the complex set of HTML tables to form the navigation areas. These were re-written again for Version 3 when Lotus Domino 5 made it easier to achieve the same effect. Version 3 looks very similar to Version 4, the major differences are to the internals in order to make it easier to install AIMS.
An investigation was carried out at the start of the project to find alternatives to using Lotus Domino. At that point in time there were a few document management systems for the Web but all had the key weakness that they just stored files and did nothing with the content. They all assume that the user is a Windows user and can download and open the documents on their computer. Lotus Notes stood out because it is not a document management product but a complex application development environment in which a document management system could be created. The project's research objective of using external linking techniques to enhance the content was a primary reason for being awarded funding so working with document content and being able to write programs to extend the system would be of primary importance. Hence the others were rejected in favour of the unique Lotus Notes.
In June 1999 an investigation was carried out to find out if the AIMS functionality could be recreated using some system other than Lotus Notes. Microsoft development products were investigated to see if each of the major features could be written. It was found to be possible using a combination of Internet Information Server 4 and Active Server Pages (asp files) created using Visual Basic 6. An OCX programming component for Visual Basic was found that could perform the crucial file format conversion. The proposed system would be quite different in that it would store HTML files on the hard disk and use some form of external database to manage them. This would require a considerable management overhead in looking after and understanding approximately a dozen interlinked Microsoft products.
Near the end of the investigation a product called Microsoft SiteServer was discovered that does most of what AIMS does. It is primarily a system for creating E-Commerce Web sites but also for making Intranets. The downside of the system is that it is primarily aimed at Windows users and does not deal with the contents of files but just expects users to be able to read them locally. Hence it was rejected for the same reasons as the others were at the start of the project.