Evaluation

Evaluation covers a series of activities exploring and informing the success of a third party's outputs, often with end user consultation. Aspects include:

• Analysis, user testing
• Information Architecture testing, website evaluation
• Expert review, workshop evaluation

Example: Information Architecture testing

Information Architecture testing for the new JISC Collections website

Background

Netskills was commissioned by JISC Communications and Marketing to perform Information Architecture (IA) testing to inform the development of a new web site for JISC Collections. This document, prepared for Dicky Maidment-Otlet (JISC Communications and Marketing) and the JISC Collections team, describes the test exercises carried out and the results analysis.

Information Architecture Testing

A "wireframe" diagram was provided upon which to base the test exercise. The exercises described were designed to test the information entities and labelling from which the structure for a web site could be derived; not to test the actual site itself.

Card sorting was used for IA testing, a method that is user-focussed and generates an overall structure for the information being tested. The process involves sorting a series of cards, each labelled with an information entity, into groups that make sense to the participants undertaking the exercise. Results from a card sorting exercise can provide indications for web site structure and navigation.

Before the IA testing event with library and University staff, a pilot card sorting exercise was first carried out with a group of three Netskills staff with backgrounds in librarianship, information skills and web site structure and design. Results from this session clarified the effectiveness of the card sorting technique and led to minor alterations being made for the testing event.

Testing event

14 participants, from five different institutions, attended the JISC Collections IA testing event. The participants were divided into two teams of 3, each containing specialist library staff, and two teams of 4, containing a mixture of library, academic and web staff.

Analysis

The sorted cards were analysed looking at the categories in which it was placed by the four teams. Significant agreement scores (≥50%) were used to compile the JISC Collections Information Grouping Diagram.