Information Filtering
LBSC 796/INFM 718R
Douglas W. Oard
Session 10, November 12, 2007
Agenda
•	Questions

•	Observable Behavior

•	Information filtering
Information Access Problems
Information Filtering
•	An abstract problem in which:
–	The information need is stable
•	Characterized by a “profile”
–	A stream of documents is arriving
•	Each must either be presented to the user or not
•	Introduced by Luhn in 1958
–	As “Selective Dissemination of Information”
•	Named “Filtering” by Denning in 1975
Information Filtering
Standing Queries
•	Use any information retrieval system
–	Boolean, vector space, probabilistic, …

•	Have the user specify a “standing query”
–	This will be the profile

•	Limit the standing query by date
–	Each use, show what arrived since the last use
What’s Wrong With That?
•	Unnecessary indexing overhead
–	Indexing only speeds up retrospective searches
•	Every profile is treated separately
–	The same work might be done repeatedly
•	Forming effective queries by hand is hard
–	The computer might be able to help
•	It is OK for text, but what about audio, video, …
–	Are words the only possible basis for filtering?
Stream Search: “Fast Data Finder”
•	Boolean filtering using custom hardware
–	Up to 10,000 documents per second (in 1996!)
•	Words pass through a pipeline architecture
–	Each element looks for one word 
Profile Indexing (SIFT)
•	Build an inverted file of profiles
–	Postings are profiles that contain each term

•	RAM can hold 5 million profiles/GB
–	And several machines could run in parallel

•	Both Boolean and vector space matching
–	User-selected threshold for each ranked profile
•	Hand-tuned on a web page using today’s news
Profile Indexing Limitations
•	Privacy
–	Central profile registry, associated with known users

•	Usability
–	Manual profile creation is time consuming
•	May not be kept up to date
–	Threshold values vary by topic and lack “meaning”
Adaptive Content-Based Filtering
Latent Semantic Indexing
Content-Based Filtering Challenges
•	IDF estimation
–	Unseen profile terms would have infinite IDF!
–	Incremental updates, side collection 

•	Interaction design
–	Score threshold, batch updates

•	Evaluation
–	Residual measures
Machine Learning for User Modeling
•	All learning systems share two problems
–	They need some basis for making predictions
•	This is called an “inductive bias”
–	They must balance adaptation with generalization
Machine Learning Techniques
•	Hill climbing (Rocchio)
•	Instance-based learning (kNN)
•	Rule induction
•	Statistical classification
•	Regression
•	Neural networks
•	Genetic algorithms
Rule Induction
•	Automatically derived Boolean profiles
–	(Hopefully) effective and easily explained
•	Specificity from the “perfect query”
–	AND terms in a document, OR the documents
•	Generality from a bias favoring short profiles
–	e.g., penalize rules with more Boolean operators
–	Balanced by rewards for precision, recall, …
Statistical Classification (e.g., SVM)
•	Represent documents as vectors
–	Usual approach based on TF, IDF, Length
•	Build a statistical models of rel and non-rel
–	e.g., (mixture of) Gaussian distributions
•	Find a surface separating the distributions
–	e.g., a hyperplane
•	Rank documents by distance from that surface
Training Strategies
•	Overtraining can hurt performance
–	Performance on training data rises and plateaus
–	Performance on new data rises, then falls

•	One strategy is to learn less each time
–	But it is hard to guess the right learning rate

•	Usual approach: Split the training set
–	Training, DevTest for finding “new data” peak
Collaborative Filtering
The Cold Start Problem
•	Social filtering will not work in isolation
–	Without ratings, no recommendations
–	Without recommendations, we rate nothing

•	An initial recommendation strategy is needed
–	Popularity
–	Stereotypes
–	Content-based
Implicit Feedback
•	Observe user behavior to infer a set of ratings
–	Examine (reading time, scrolling behavior, …)
–	Retain (bookmark, save, save & annotate, print, …)
–	Refer to (reply, forward, include link, cut & paste, …)
•	Some measurements are directly useful
–	e.g., use reading time to predict reading time
•	Others require some inference
–	Should you treat cut & paste as an endorsement?

Recommending w/Implicit Feedback
Spam Filtering
•	Adversarial IR
–	Targeting, probing, spam traps, adaptation cycle
•	Compression-based techniques
•	Blacklists and whitelists
–	Members-only mailing lists, zombies
•	Identity authentication
–	Sender ID, DKIM, key management