(and paper 

(func appears in (sing SIGIR93)) 

(all author (func affiliation (sing IEI CNR))) 

(c some deals with modal_logic))  

Figure 8: Terminological representation of a concept regarding modal_logic 

or broaden the query by omitting one of the conditions, e.g. to retrieve all documents about logic 

written by a member of IEI CNR, irrespective of where the paper was published. This would be a 

matching on only some of the components of our concept, as shown in Figure 9. 

(and paper 

(all author (func affiliation (sing IEI CNR))) 

(c some deals with logic))  

Figure 9: Terminological representation of a concept  

ii. personaliation of concepts. In addition to modifying the content of the query we could incorporate 

personalised thesaural knowledge. In the example, the term logic need not refer to a single term but 

could refer to a class of terms, e.g. modal_logic,  conceptual_graphs,  cumulative_logic, etc. This 

knowledge can be used as default values in retrieval but we could tailor this information to individual 

users based on feedback information. That is, the system automatically learns important synonymous 

concepts for individual users. 

iii. uncertainty modelling. Logical concepts and rules reflecting thesaural knowledge are often 

associated with uncertainty values such as probabilities to reflect the importance of concepts or strength 

of relationship between concepts. These values can be changed in a similar fashion to the vector space 

or probabilistic models to reflect important concepts in a search or the strength of association between 

concepts. Based on the example concept in Figure 8, for example, we could change the query to treat 

the author's affiliation as more important than the topic of the paper. 

iii. rule modification or refinement. In this case, the information given by analysing the relevant 

documents is not only used to expand the query as in traditional feedback but is also used to modify the 

rules of the system. Examples of this approach include systems to select rules for retrieving documents, 

e.g. [DBM97] and the use of abductive logic to create new rules for retrieving documents, [Mull98]. 

2.3 Presentation of retrieved documents 

A lengthy discussion of interfaces to IR systems will not be given at this point. Unless otherwise stated 

we shall assume that retrieved documents are presented either as a list (best match) or set (exact 

match). Hearst [Hea99] discusses the wide range of graphical and visualisation techniques that have 

been suggested for IR systems. Interfaces designed specifically for RF will be discussed in more detail 

in section 6. 

2.4 Evaluation of retrieval systems and relevance feedback 

We will now discuss the evaluation of IR systems and RF. The most common evaluation tool for IR 

systems is a test collection. This is a set of documents, a set of queries and a list of which documents 

are considered relevant for each query. The list of documents assessed as being relevant for each query 

  the relevance assessments   is usually not gathered from real life search data. Rather test collections 

are usually constructed within a laboratory setting. Currently the foremost example of test collection 

construction is to be found within the TREC (Text REtrieval Conference) initiative, [VH96].  

Test collections are primarily used for comparative evaluation: comparing the performance of two 

systems, or two versions of the same system on the same set of queries. Two standard evaluation 

measures are commonly used with test collections: precision and recall. Recall is measured as the ratio 

of relevant documents retrieved to the number of relevant documents in the collection. Precision is the 

ratio of relevant documents retrieved to the number of documents retrieved. In a best match, or ranking 

model, recall and precision figures can be calculated at various points in the document ranking to give 

an indication of performance at different levels of retrieval. Typically this would be done at 10% recall, 

 15