LBSC 796/INFM 718R
Information Retrieval Systems
Spring 2011
Information Retrieval Softeare

Available Text Retrieval Software

The Big Three

Lucene

A freely available Java IR system, probably the easiest system to get up and running, and the most easily modified. The SOLR extensions to Lucene are well worth getting too.

Indri

Indri is optimized for efficiency, and thus is a good choice if you have a large collection and a single processor. It is built on top of the Lemur toolkit for building language modeling systems for information retrieval.

Ivory

An information retrieval system for the Hadoop MapReduce framework. This is a good choice is you have a very large collection and at least a modest size server cluster. You can buy time from Amazon Web Services if you don't have your own cluster.

The Others

Cheshire 3

Freely available research software implementing a logistic regression model from the University of California at Berkeley. Gettig it working may require some facility with Z39.50.

Glimpse

Freely available software from the University of Arizona that is designed for efficient indexing (at some cost in retrieval efficiency). Glimpse is not configured for TREC-style evaluations, so that would take some extra work.

InQuery

Commercial software based on inference networks that has a very flexible query language. We have a research and teaching license for this system from the University of Massachusetts, and still use it occassionally. InQuery includes an X-Windows interface and it is configured to run TREC-style evaluations, but the source code is not available.

IRF

A Java toolkit for building IR systems for small applications. The strength of IRF is that the object oriented framework greatly simplifies tasks that require working wiht the source code. Bt because Java is designed for platform independence rather than efficiency, the size of the collections that can be handled is quite limited.

MG

Research software from RMIT that is designed to maximize storage efficiency on very large collectons. It is available under the GNU public license. We installed this once several years ago and it wasn't too difficult. Click here to download the tarfile.

PRISE

Public domain vector space research software developed at NIST We regularly use this system for TDT evaluations. PRISE includes a very nice Z39.50 interface, but it takes some facility with that stangard to get the interactive part running. PRISE is configured to run TREC-style evaluations and the source code is available.

SMART

A vector space research software that is freely available by FTP from Cornell University. We have extensive experience using SMART. SMART includes only a VT-100 interface, but it is configured to run TREC-style evaluations and the source code is available.

Terrier

An information retrieval system from the University of Glasgow that is optimized for efficiency. Terrier implements the divergence from randomness framework for ranked retrieval.

Xapian

An open source IR system that is designed ot run under Linux. Xiapan is a descendent of Omseek, which itself is a decendent of Open Muscat. Xiapan is designed to handle several Western European languages, and thus might be a good choice if you want to work with languages other than English.

Zettair

Zettair is optimized for both efficiency and modifiability. It therefore occupies a part of the design space between Lucene and Indri.