the database, a further 20.8% were due to typographical errors or spelling mistakes

35

. The majority of 

other errors were due to problems with the system such as author searches for titles, and misuse of 

controlled vocabulary. The error rates were high, as high as 46% of searches, so this is relatively 

severe, even though the systems themselves were popular. Common problems included low use of 

advanced features or poor understanding of how to use the systems.  

Part of this difficulty in using IR systems is that different types of knowledge are required for different 

tasks. Borgman, [Borg96], for example, identified three types of knowledge necessary in information 

seeking: 

i. 

conceptual knowledge of the information retrieval process. This is knowledge necessary to 

translate an information need into a searchable query. 

ii. 

semantic knowledge of how to implement a query in a given system. Once the user has 

established what concepts and terms are to be used to form a query these elements must be 

converted into an appropriate query for the system. This requires knowledge of how and when to 

use the system features. 

iii.  technical knowledge. This covers basic computing skills and the knowledge of the query 

language. 

A user's lack of knowledge may not only hinder search effectiveness but may also require the user to 

interact ineffectively with the system. This problem also relates to the earlier discussion on interactive 

query expansion: the presentation of what the system is trying to achieve is important for effective 

interaction with the system. 

7.3 Feedback, term selection and relevance assessments 

The success of RF depends largely on two components: the user's evidence as to what constitutes 

relevant material and the quality of the RF algorithm. In this paper we have concentrated mainly on the 

latter component   the RF algorithms themselves. We have briefly discussed some of the factors, such 

as the interface, which can have an affect on the former component   the relevance information given 

by the user.  

The information given by the user is vitally important in helping the RF algorithm make good query 

modification decisions. In this section we shall outline some of the studies that have examined how 

users give relevance information. In particular we shall concentrate on what types of feedback users 

employ, section 7.3.1, how user's choose query terms, including terms chosen during feedback, section 

7.3.2, and the factors that affect a user's relevance assessments, section 7.3.3. These sections are 

intended to highlight aspects of the users' interaction that can affect the quality of information given to 

a RF system. 

7.3.1 Types of feedback 

Spink [Spi97] looked at the various types of feedback in mediated

36

 Boolean information seeking 

sessions. Based on her study of 40 searches, she proposed a classification of five types of feedback. 

These are not all types of relevance feedback; they also include query modification actions that are 

intended to modify the search in some other way. Her classification of feedback types is: 

i. 

content relevance feedback. In all the searches studied the user and intermediary used the 

content of documents to make relevance judgements. The judgements could be either negative or 

positive. This is the second most common type of feedback and was the only type of feedback where 

the users' judgements were more common than the intermediaries' judgements. Based on content 

relevance feedback searchers could modify their query and re search. 

ii. 

term relevance feedback. This was the identification of new search terms by the user or 

intermediary from the relevant material. This is equivalent to the common notion of RF discussed in 

35

This may be a particular problem for Boolean systems in which one mispelt query term can result in an empty 

result set. 

36

 Mediated searches are those in which a professional searcher, such as a librarian, aids a searcher in formulating 

queries. 

 42