Cross-Language Retrieval
LBSC 796/INFM 718R
Douglas W. Oard
Session 12: November 26, 2007
Agenda
•	Questions

•	Overview

•	Cross-Language Search

•	User Interaction
User Needs Assessment
•	Who are the potential users? 

•	What goals do we seek to support? 

•	What language skills must we accommodate?
Global Internet Users
Global Internet Users
Most Widely-Spoken Languages
Global Trade
Who needs Cross-Language Search?
•	When users can read several languages
–	Eliminate multiple queries
–	Query in most fluent language

•	Monolingual users can also benefit
–	If translations can be provided
–	If it suffices to know that a document exists
–	If text captions are used to search for images
The Problem Space
•	Retrospective search
–	Web search
–	Specialized services (medicine, law, patents) 
–	Help desks
•	Real-time filtering
–	Email spam
–	Web parental control
–	News personalization
•	Real-time interaction
–	Instant messaging
–	Chat rooms
–	Teleconferences
A Little (Confusing) Vocabulary
•	Multilingual document
–	Document containing more than one language
•	Multilingual collection
–	Collection of documents in different languages
•	Multilingual system
–	Can retrieve from a multilingual collection
•	Cross-language system
–	Query in one language finds document in another
•	Translingual system
–	Queries can find documents in any language
The Information Retrieval Cycle

Early Work
•	1964 International Road Research
–	Multilingual thesauri
•	1970 SMART
–	Dictionary-based free-text cross-language retrieval
•	1978 ISO Standard 5964 (revised 1985)
–	Guidelines for developing multilingual thesauri
•	1990 Latent Semantic Indexing
–	Corpus-based free-text translingual retrieval
Multilingual Thesauri
•	Build a cross-cultural knowledge structure
–	Cultural differences influence indexing choices
•	Use language-independent descriptors
–	Matched to language-specific lead-in vocabulary
•	Three construction techniques
–	Build it from scratch
–	Translate an existing thesaurus
–	Merge monolingual thesauri

Free Text CLIR
•	What to translate?
–	Queries or documents

•	Where to get translation knowledge?
–	Dictionary or corpus

•	How to use it?
The Search Process
Translingual Retrieval Architecture
Evidence for Language Identification
•	Metadata
–	Included in HTTP and HTML

•	Word-scale features
–	Which dictionary gets the most hits?

•	Subword features
–	Character n-gram statistics
Query-Language IR
Example: Modular use of MT
•	Select a single query language

•	Translate every document into that language

•	Perform monolingual retrieval

Is Machine Translation Enough?
Document-Language IR
Query vs. Document Translation
•	Query translation
–	Efficient for short queries (not relevance feedback)
–	Limited context for ambiguous query terms
•	Document translation
–	Rapid support for interactive selection
–	Need only be done once (if query language is same)
•	Merged query and document translation
–	Can produce better effectiveness than either alone
Interlingual Retrieval

Generalized Vector Space Model
•	“Term space” of each language is different
–	Document links define a common “document space”

•	Describe documents based on the corpus
–	Vector of similarities to each corpus document

•	Compute cosine similarity in document space

•	Very effective in a within-domain evaluation
Latent Semantic Indexing
•	Cosine similarity captures noise with signal
–	Term choice variation and word sense ambiguity

•	Signal-preserving dimensionality reduction
–	Conflates terms with similar usage patterns
•	Reduces term choice effect, even across languages

•	Computationally expensive

What’s a “Term?”
•	Granularity of a “term” depends on the task
–	Long for translation, more fine-grained for retrieval
•	Phrases improve translation two ways
–	Less ambiguous than single words
–	Idiomatic expressions translate as a single concept
•	Three ways to identify phrases
–	Semantic          (e.g., appears in a dictionary)
–	Syntactic          (e.g., parse as a noun phrase)
–	Co-occurrence (appear together unexpectedly often)
Learning to Translate
•	Lexicons
–	Phrase books, bilingual dictionaries, …

•	Large text collections
–	Translations (“parallel”)
–	Similar topics (“comparable”)

•	Similarity
–	Similar pronunciation

•	People
Types of Lexical Resources
•	Ontology
–	Organization of knowledge
•	Thesaurus
–	Ontology specialized to support search
•	Dictionary
–	Rich word list, designed for use by people
•	Lexicon
–	Rich word list, designed for use by a machine
•	Bilingual term list
–	Pairs of translation-equivalent terms
Dictionary-Based Query Translation 
Original query:	El Nino and infectious diseases

Term selection:	“El Nino” infectious diseases

Term translation:	
				(Dictionary coverage: “El Nino” is not found)

Translation selection:

Query formulation:
Structure: 
Four-Stage Backoff
•	Tralex might contain stems, surface forms, or some combination of the two.
Results
Results Detail

The Short Query Challenge

“Structured Queries”
•	Weight of term a in a document i depends on:
–	TF(a,i): Frequency of term a in document i
–	DF(a):   How many documents term a occurs in

•	Build pseudo-terms from alternate translations
–	TF (syn(a,b),i) = TF(a,i)+TF(b,i)
–	DF (syn(a,b) = |{docs with a}U{docs with b}|

•	Downweight terms with any common translation
–	Particularly effective for long queries
Computing Weights
•	Unbalanced:
–	Overweights query terms that have many translations

•	Balanced (#sum): 
–	Sensitive to rare translations

•	Pirkola (#syn):
–	Deemphasizes query terms with any common translation

Measuring Coverage Effects
35 Bilingual Term Lists
•	Chinese (193, 111)
•	German (103, 97, 89, 6)
•	Hungarian (63)
•	Japanese (54)
•	Spanish (35, 21, 7)
•	Russian (32)
•	Italian (28, 13, 5)
•	French (20, 17, 3)
•	Esperanto (17)
•	Swedish (10)
•	Dutch (10)
•	Norwegian (6)
•	Portuguese (6)
•	Greek (5)
•	Afrikaans (4)
•	Danish (4)
•	Icelandic (3)
•	Finnish (3)
•	Latin (2)
•	Welsh (1)
•	Indonesian (1) 
•	Old English (1)
•	Swahili (1)
•	Eskimo (1)
Size Effect
Out-of-Vocabulary Distribution
Measuring Named Entity Effect

Types of Bilingual Corpora
•	Parallel corpora: translation-equivalent pairs
–	Document pairs
–	Sentence pairs  
–	Term pairs      

•	Comparable corpora: topically related
–	Collection pairs
–	Document pairs
Exploiting Parallel Corpora

•	Automatic acquisition of translation lexicons

•	Statistical machine translation

•	Corpus-guided translation selection

•	Document-linked techniques

Some Modern Rosetta Stones
•	News:
–	DE-News (German-English)
–	Hong-Kong News, Xinhua News (Chinese-English)
•	Government:
–	Canadian Hansards (French-English)
–	Europarl (Danish, Dutch, English, Finnish, French, German, Greek, Italian, Portugese, Spanish, Swedish)
–	UN Treaties (Russian, English, Arabic, …)
•	Religion
–	Bible, Koran, Book of Mormon
Parallel Corpus
•	Example from DE-News (8/1/1996)

Word-Level Alignment
A Translation Model
•	From word-aligned bilingual text, we induce a translation model

•	Example:

Using Multiple Translations
•	Weighted Structured Query Translation
–	Takes advantage of multiple translations and translation probabilities
•	TF and DF of query term e are computed using TF and DF of its translations:

Evaluating Corpus-Based Techniques
•	Within-domain evaluation (upper bound)
–	Partition a bilingual corpus into training and test
–	Use the training part to tune the system
–	Generate relevance judgments for evaluation part

•	Cross-domain evaluation (fair)
–	Use existing corpora and evaluation collections
–	No good metric for degree of domain shift
Ranked Retrieval Effectiveness
Exploiting Comparable Corpora
•	Blind relevance feedback
–	Existing CLIR technique + collection-linked corpus

•	Lexicon enrichment
–	Existing lexicon + collection-linked corpus

•	Dual-space techniques
–	Document-linked corpus
Bilingual Query Expansion
Query Expansion Effect
Blind Relevance Feedback
•	Augment a representation with related terms
–	Find related documents, extract distinguishing terms

•	Multiple opportunities:
–	Before doc translation:	Enrich the vocabulary
–	After doc translation:	Mitigate translation errors 
–	Before query translation:	Improve the query
–	After query translation:	Mitigate translation errors

•	Short queries get the most dramatic improvement
Indexing Time: Doc Translation
  
Post-Translation “Document Expansion”
Why Document Expansion Works
•	Story-length objects provide useful context

•	Ranked retrieval finds signal amid the noise

•	Selective terms discriminate among documents
–	Enrich index with low DF terms from top documents

•	Similar strategies work well in other applications
–	CLIR query translation
–	Monolingual spoken document retrieval
Lexicon Enrichment
Lexicon Enrichment
•	Use a bilingual lexicon to align “context regions”
–	Regions with high coincidence of known translations

•	Pair unknown terms with unmatched terms
–	Unknown:	language A, not in the lexicon
–	Unmatched:	language B, not covered by translation 

•	Treat the most surprising pairs as new translations

Cognate Matching
•	Dictionary coverage is inherently limited
–	Translation of proper names
–	Translation of newly coined terms
–	Translation of unfamiliar technical terms

•	Strategy: model derivational translation
–	Orthography-based
–	Pronunciation-based
Matching Orthographic Cognates
•	Retain untranslatable words unchanged
–	Often works well between European languages

•	Rule-based systems
–	Even off-the-shelf spelling correction can help!

•	Character-level statistical MT
–	Trained using a set of representative cognates
Matching Phonetic Cognates
•	Forward transliteration
–	Generate all potential transliterations

•	Reverse transliteration
–	Guess source string(s) that produced a transliteration

•	Match in phonetic space
Leveraging Cognates
Cross-Language “Retrieval”
Interactive Translingual Search
Selection
•	Goal: Provide information to support decisions

•	May not require very good translations
–	e.g., Term-by-term title translation

•	People can “read past” some ambiguity
–	May help to display a few alternative translations
Language-Specific Selection
Translingual Selection
Merging Ranked Lists
•	Types of Evidence
–	Rank
–	Score

•	Evidence Combination
–	Weighted round robin
–	Score combination

•	Parameter tuning
–	Condition-based
–	Query-based
Examination Interface
•	Two goals
–	Refine document delivery decisions
–	 Support vocabulary discovery for query refinement

•	Rapid translation is essential
–	Document translation retrieval strategies are a good fit
–	Focused on-the-fly translation may be a viable alternative
Uh oh…
Translation for Assessment
    Indonesian City of Bali in October last year in the bomb blast in the case of imam accused India of the sea on Monday began to be averted. The attack on getting and its plan to make the charges and decide if it were found guilty, he death sentence of May. Indonesia of the police said that the imam sea bomb blasts in his hand claim to be accepted. A night Club and time in the bomb blast in more than 200 people were killed and several injured were in which most foreign nationals. …
MT in a Month

Experiment Design
Maryland Experiments
•	MT is almost always better
–	Significant overall and for narrow topics alone (one-tailed t-test, p<0.05)

•	F measure is less insightful for narrow topics
–	Always near 0 or 1
iCLEF 2002 Evaluation
Better Mental Process Models
Delivery
•	Use may require high-quality translation
–	Machine translation quality is often rough

•	Route to best translator based on:
–	Acceptable delay
–	Required quality (language and technical skills)
–	Cost
Where Things Stand
•	Ranked retrieval works well across languages
–	Bonus: easily extended to text classification
–	Caveat: mostly demonstrated on news stories

•	Machine translation is okay for niche markets
–	Keep an eye on this: accuracy is improving fast

•	Building explainable systems seems possible
Recap: Finding What You Can’t Read
•	Three key challenges
–	Segmentation, coverage, evidence combination

•	Segmentation objectives differ
–	Translation: 	Favor precision over coverage
–	Retrieval:	Balance precision and recall

•	Multiple coverage enhancement techniques
–	Expansion, backoff translation, cognate matching

•	Translating evidence beats translating weights
Research Opportunities