WWW 2007 / Poster Paper

Topic: Search

Understanding Web Search via a Learning Paradigm
Bernard J. Jansen, Brian Smith, Danielle L. Booth
College of Information Sciences and Technology The Pennsylvania State University University Park, PA, 16801, USA

jjansen@acm.org, bsmith@ist.psu.edu, dlb5000@psu.edu ABSTRACT
Investigating whether one can view Web searching as a learning process, we examined the searching characteristics of 41 participants engaged in 246 searching tasks. We classified the searching tasks according an updated version of Bloom's taxonomy, a six level categorization of cognitive learning. Results show that Applying takes the most searching effort as measured by queries per session and specific topics searched per sessions. The lower level categories of Remembering and Understanding exhibit searching characteristics similar to the higher order learning of Evaluating and Creating. It appears that searchers rely primarily on their internal knowledge for Evaluating and Creating, using searching primarily as fact checking and verification. Implications are that the commonly held notion that Web searchers have simple information needs may not be correct. We discuss the implications for Web searching, including designing interfaces to support exploration. even if learning explicitly manifests itself in the searching process. Bloom's Taxonomy may be a method for investigating Web search as a learning process. Bloom's Taxonomy is a primary classification of learning in the cognitive domain [2]. An updated version, Anderson and Krathwohl's Taxonomy, redefined Bloom's original classification [1]. Anderson and Krathwohl's Taxonomy is a six-tiered model for classifying learning according to cognitive levels of complexity.

3. RESEARCH QUESTIONS
Our research question is: Is a learning paradigm effective for analyzing information searching? In order to analyze this question, we used Anderson and Krathwohl's re-design of Bloom's taxonomy of learning in the cognitive domain to develop searching tasks for each of the six taxonomy's categories. We then analyzed the exhibited searching characteristics to detect if there were differences in exhibited searching behavior.

Categories and Subject Descriptors
H.3.3 [1] Information Search and Retrieval ­ Search process.

4. METHODS
We devised searching scenarios for each of the six levels of Anderson and Krathwohl's Taxonomy, with each scenario correlated to one of the classifications. The scenarios were pilot tested twice before we used them in the laboratory study. The six classifications with definitions [1, p. 67-68] and searching scenarios are shown in Table 1. Over the course of a week, 41 participants engaged in a laboratory study. Each participant engaged in six searching scenarios and instructed to address the scenarios, verify their answers. Each participant had access to an individual computer with Internet access. All user interactions with the computer were logged using a non-inclusive logging software package. We analyzed participant interactions in accordance with standard characteristics of information searching using transaction log analysis as the methodological approach.

General Terms
Experimentation, Human Factors

Keywords
Information searching, Bloom's Taxonomy

1. INTRODUCTION
A common paradigm for analyzing Web searching is problem solving or decision making. Much information and Web searching research is intertwined with this view of searching as a decision making process. However, there is little empirical evidence supporting the relationship between information searching and problem solving [3]. We therefore explored other possible paradigms in which to view Web searching namely as a learning activity. In the following, we present a brief literature review, research questions, preliminary results, and implications for future Web searching systems. We conclude with our upcoming research aims and directions.

5. RESULTS
We investigated whether or not there would be a significant difference in (1) the number of queries and (2) the number of topics per session among the classifications in Anderson and Krathwohl's Taxonomy. A topic is the information focus of one or more queries. A searching session may have several topics. Using a one-way ANOVA, our results indicate that there is a significant difference among the groups (F(5) = 4.175, p < 0.01) in terms of queries per session. Applying was significantly different then the classifications of Remembering, Understanding, Analyzing, and Evaluating (p < 0.05). Applying was not significantly different in number of queries per session than Creating.

2. REVIEW OF LITERATURE
There is information searching literature that refers to an on-going learning process while a person is engaged in information searching. However, there has been little research into how or

Copyright is held by the author/owner(s). WWW 2007, May 8--12, 2007, Banff, Alberta, Canada. ACM 978-1-59593-654-7/07/0005

1207


WWW 2007 / Poster Paper

Topic: Search

Classification Remembering Understanding Applying Analyzing Evaluating Creating

Table 1. Anderson and Krathwohl's Taxonomy with Searching Scenarios Definition Scenario Retrieving, recognizing, and recalling relevant knowledge List 5 movies directed by Steven Spielberg. from long-term memory Constructing meaning from oral, written, and graphic Give a brief plot summary of the TV show, messages through interpreting, exemplifying, classifying, Veronica Mars. summarizing, inferring, comparing, and explaining Carrying out or using a procedure through executing, or What are some possible characteristics of a person implementing who would enjoy trip-hop music? Breaking material into constituent parts, determining how A certain television show contains intense violence the parts relate to one another and to an overall structure or and coarse language. Which rating should it purpose through differentiating, organizing, and attributing receive? Making judgments based on criteria and standards through Create a list of pros and cons for the new iPod checking and critiquing Shuffle. Based off of this, would you purchase it (assuming you had the money)? Why or why not? Putting elements together to form a coherent or functional Which do you think will have better overall sales -whole; reorganizing elements into a new pattern or the XBox 360, the Nintendo Wii, or the Playstation 3? Why? structure through generating, planning, or producing
3.5 3 2.5 Mean Value 2 1.5 1.05 1 0.5 0 Remembering Unders t anding Apply ing Analy z ing Evaluating Creat ing Tot al Ande rson and Krathwohl's Taxonomy Topic s Per Session Queries Per Session 1 1.05 0.95 2. 98 2.64 2.33 2. 14 1.77 1. 65 1.66 1.44 1. 33 1. 17

Figure 1. Queries per Session and Topics Per Session. Concerning topics per session, an ANOVA analysis indicate that there is a significant difference among the groups (F(5) = 6.869, p < 0.01). Applying was significantly different then the classifications of Remembering, Understanding, Analyzing, and Evaluating (p < 0.05). Applying was not significantly different in number of topics per session than Creating.

7. ACKNOWLEDGMENTS
The Air Force Office of Scientific Research funded portions of this research.

8. REFERENCES
[1] Anderson, L. W. and D. A. Krathwohl, A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman, 2001. [2] Bloom, B. S. and D. R. Krathwohl, Taxonomy of Educational Objectives: The Classification of Educational Goals, by a Committee of College and University Examiners. Handbook 1: Cognitive Domain. New York: Longmans, 1956. [3] Jansen, B. J. and M. D. McNeese, "Evaluating the Effectiveness of and Patterns of Interactions with Automated Searching Assistance," Journal of the American Society for Information Science and Technology, 56, pp. 1480-1503, 2005.

6. CONCLUSION
Searching as learning is an appropriate paradigm to view Web searching. The implications of this study are that commonly held notions of Web searchers having simple information needs may be incorrect. Simple searching expressions may be supportive of higher-level information needs at the Evaluating and Creating levels of learning. With this view, the goal of systems design shifts to one of not closing this uncertainty too quickly but instead accommodating exploring and formulating. We consider this an exploratory study to investigate whether learning is an appropriate paradigm to study Web searching. Along with further analysis of the existing date, we aim to conduct larger scale studies from to focus on required system design changes.

1208