Evidence from Behavior
LBSC 796/INFM 719R
Douglas W. Oard
Session 7, October 22, 2007
Picture of Relevance Feedback
Rocchio Formula
Rocchio Example
Using Positive Information
Using Negative Information
Rating-Based Recommendation
•	Use ratings as to describe objects
–	Personal recommendations, peer review, …

•	Beyond topicality:
–	Accuracy, coherence, depth, novelty, style, …

•	Has been applied to many modalities
–	Books, Usenet news, movies, music, jokes, beer, …
Motivations to Provide Ratings
•	Self-interest
–	Use the ratings to improve system’s user model

•	Economic benefit
–	If a market for ratings is created

•	Altruism
Explicit Feedback: Assumptions
•	A1: User has sufficient knowledge for a reasonable initial query

•	A2: Selected examples are representative

•	A3: The user will try giving feedback

•	A4: The user will keep giving feedback
A1: Good Initial Query?
•	Two problems:
–	User may not have sufficient initial knowledge
–	Few or no relevant documents may be retrieved

•	Examples:
–	Misspellings (Brittany Speers)
–	Cross-language information retrieval
–	Vocabulary mismatch (e.g., cosmonaut/astronaut)
A2: Representative Examples?
•	There may be several clusters of relevant documents

•	Examples:
–	Burma/Myanmar
–	Contradictory government policies
–	Opinions
A3: Will People Use It?
•	Efficiency
–	Longer queries require more processing time

•	Understandability
–	Harder to see why subsequent documents retrieved

•	Risk
–	Users are reluctant to provide negative feedback
A4: Self-Interest Decreases!
Solving the Cost vs. Value Problem
•	Maximize the value
–	Provide for continuous user model adaptation

•	Minimize the costs
–	Use implicit feedback rather than explicit ratings
–	Minimize privacy concerns through encryption
–	Build an efficient scalable architecture
–	Limit the scope to noncompetitive activities
Solution: Reduce the Marginal Cost

Estimating Authority from Links
Anchor Text Indexing
•	Sometimes yields unexpected results: 
–	Google ranked Microsoft #1 for “evil empire”
Collecting Click Streams
•	Browsing histories are easily captured
–	Make all links initially point to a central site
•	Encode the desired URL as a parameter
–	Build a time-annotated transition graph for each user
•	Cookies identify users (when they use the same machine)
–	Redirect the browser to the desired page

•	Reading time is correlated with interest
–	Can be used to build individual profiles
–	Used to target advertising by doubleclick.com

More Complete Observations
•	User selects an article
–	Interpretation: Summary was interesting
•	User quickly prints the article
–	Interpretation: They want to read it
•	User selects a second article
–	Interpretation: another interesting summary
•	User scrolls around in the article
–	Interpretation: Parts with high dwell time and/or repeated revisits are interesting
•	User stops scrolling for an extended period
–	Interpretation: User was interrupted

Recommending w/Implicit Feedback
Critical Issues
•	Protecting privacy
–	What absolute assurances can we provide?
–	How can we make remaining risks understood?

•	Scalable rating servers
–	Is a fully distributed architecture practical?

•	Non-cooperative users
–	How can the effect of spamming be limited?
Gaining Access to Observations
•	Observe public behavior
–	Hypertext linking, publication, citing, …

•	Policy protection
–	EU: Privacy laws
–	US: Privacy policies + FTC enforcement

•	Statistical assurance of privacy
–	Distributed architecture
–	Model and mitigate privacy risks
Search Engine
Query Logs
Search Engine Query Logs
http://hannu.biz/aolsearch/
AOL User 4417749