An Exploratory Web Log Study of Multitasking
Nikolai (Nick) Buzikashvili
Human-Computer Interaction laboratory, institute of system analysis, Russian Academy of Science 9, 60-Letiya Oktyabrya, Moscow 117312, Russia +7 495-135-5357

buzik@cs.isa.ru ABSTRACT
The Web search multitasking study based on automatic task session detection procedure is described. The results of the study: 1) multitasking is very rare, 2) it usually covers only 2 task sessions, 3) it is frequently formed into a temporal inclusion of an interrupting task session into the interrupted session, 4) the quantitative characteristics of multitasking greatly differ from the characteristics of sequential execution of one and several tasks. A searcher minimizes task switching costs: he avoids multitasking and while multitasking he uses cheapest manner of task switching. session executed the same task; 4) We distinct between sequential execution of several task sessions when the tasks are executed one-by-one and a parallel execution when a searcher switches between unfinished tasks (multitasking). 5) A maximum number of unfinished tasks during a temporal session is called a session width. If the width of a temporal session is bigger than 1 this session is referred to as "wide". Data Used. To detect real-life multitasking search we use log samples of 2 search engines: the Russian Yandex (2005, a week sample, 175,000 clients) and the Excite (2001, a day sample, 305,000 clients). The latter data are used to yield the results comparable with both Yandex results yielded by the same tool and the results previously elaborated on the same log. On the preprocessing stage, the clients, who are likely to be local area networks, were eliminated. To segment a time series of a client's transactions into temporal sessions we use 15min cut-off.

Categories and Subject Descriptors
H.3.3 [Information Search and Retrieval]: Search process.

General Terms
Algorithms, Experimentation, Human Factors.

Keywords
Multitasking, Web log analysis.

2. QUESTIONS AND HYPOTHESES
(a) How frequent are parallel task sessions? How are temporal sessions distributed over a session width? (b) Patterns of task session switching. For unfinished task sessions it is less likely that a transaction of one unfinished session be followed by a transaction of another session. In particular, transactions of parallel task sessions or remaining parts of these sessions are not randomly ordered over time. (c) Patterns of task session resumption. How is an interrupted task session resumed in multitasking -- by a new query or by viewing the results of the interrupted query? Does this manner differ from a manner of a "continuation" of a sequential session? (d) Do quantitative characteristics of multitasking sessions (length, duration, query length, etc.) significantly differ from the characteristics of the sessions executed one-by-one?

1. INTRODUCTION
Multitasking is one of the processes mostly actively investigated by cognitive science (e.g. [3]). While multitasking considered previously (driving, dispatching, etc) is mostly externally triggered multitasking, the Web search is an area of free choice of tasks execution. Multitasking is an expensive manner. So the conclusion "multitasking information searching is a common behavior" [5] seems surprising. However, [4,5] consider all cases of different tasks execution as `multitasking' and make no distinction between multitasking and sequential search, in which a searcher executes tasks one-by-one: he starts and ends one task, then starts and ends next task. During multitasking a user executes tasks, interrupting one and returning to the interrupted task later. A true multitasking search is rare and cannot be investigated manually [4,5]. To study it we elaborated an automatic procedure of task session detection. Then the results were checked manually. Terms Used. In this paper, we use the notions: 1) a temporal session as a sequence of the single client's transactions with the search engine cut from previous and successive sessions by certain cut-off interval; 2) a task as a search of the same or of similar objects; 3) a task session as all transactions of a temporal

3. METHOD
To study multitasking Web search we use 2-stage approach. The first stage includes an automatic tasks and multitasking detection. These results are manually verified at the second stage. A similarity between any queries may be detected indirectly, as a similarity between sets of documents relevant to the queries. A similarity between queries of the same user submitted in the same session may be detected also directly. We follow the former approach and our first-stage procedure 1) fills a matrix of pairwise orthographic similarity of queries; 2) makes a transitive closure of this relation and considers any connected component as a single

Copyright is held by the author/owner(s). SIGIR'06, August 6-11, 2006, Seattle, Washington, USA. ACM 1-59593-369-7/06/0008.

623


task; 3) detects wide temporal sessions and stores them. While other ways to detect queries similarity exist [1,2], the direct approach is simple and effective for multitasking detection. To detect similarity the procedure converts a source queries into low letters, deletes blanks and punctuators (e.g. query <JozephLui +Gay-Lussac> is glued into gjozephluigaylussac) and finds high correlated substrings in each pair of these strings. If a pair of substrings is found, the procedure checks (1) if the substring of at least one string doesn't intersect with an auxiliary part of speech in the original query and (2) if both substrings don't intersect with inflective endings (for the inflective Russian language). If yes, the original queries are marked as similar, otherwise the next pair of correlated substrings is checked. This approach doesn't cover synonymy. However, the logs present a lot of misprints, variations and non-vocabulary words rather than synonyms. The procedure covers these problems. During an active search users frequently describe in different queries different sides of the same object (queries are "compatible" rather than similar): The procedure detects doesn't detect compatibility. Besides, since the procedure doesn't recognize branches of the same task it may not detect parallel execution of these branches. Table 1. Sequential and parallel tasks in 2 search engines logs Yandex, 2005 Excite, 2001 Type and number 1-task >1 task wide 1-task >1 task wide of tempor.sessions 166508 34138 2069 296491 40750 4046 Tasks/temp.session 1 2.19 2.30 1 2.21 2.35 Task continuation 26.9 53.1 52.4 26.8 52.4 52.8 by new query (%) Unique queries / 1.3 1.2 1.5 1.3 1.2 1.5 task session Unique queries / 1.3 2.6 3.4 1.3 2.6 3.5 temporal session Transactions/query 1.6 1.5 2.0 1.7 1.6 2.0 Terms / query Duration (min) of 2-query session of 3-query session 2.6 1.7 5.6 9.0 2.8 6.9 11.0 2.3 13.6 14.3 2.5 1.2 4.5 7.4 2.7 5.7 8.9 2.0 11.5 11.8

characteristics of all types of temporal sessions are very different: not only multitasking differs from both types of sequential execution but also execution of 1 task differs from execution of more tasks. The "queries/task", "transactions/query" and "terms/ query" characteristics show an interesting non-monotonicity over different types of sessions: sequentially executed task sessions are shorter than task sessions of both other types but queries of these sessions are longer. This fact cannot be completely explained by task complexity and searcher's estimates of it.

5. CONCLUSION
The procedure reveals great differences between each 3 types of sessions. However, are these differences real differences of real task sessions rather than procedural artifacts? Also, are automatically discovered multitasking properties real properties? The procedure doesn't always detect queries similarity and doesn't detect compatibility. As a result, the number of detected task sessions turns out to be bigger and the width of temporal sessions is overestimated. However, we can use this systematic bias as an advantage: to check the results of automatic multitasking extraction we should manually verify only those temporal sessions, which were detected as wide. 200 wide temporal sessions were manually verified. The results are: 1) the automatic procedure overestimates the number of wide sessions near twice; 2) an automatic detection of sessions whose width is bigger than two is an artifact and may be explained by the active search: when a user searches actively he frequently reformulates queries of the same task in new terms; 3) in other respects manually approved multitasking shows the same dependencies as automatically detected multitasking. Thus, multitasking is very uncommon behavior (1%). However, the final question is "Does multitasking search exist?" and the final diagnosis may be "non-existing". Indeed, (1) 1% of multitasking sessions may be the artifact resulted from nonexclusion of local area networks; (2) multitasking usually is more expensive than sequential execution of the same tasks. A costmotivated user may select multitasking when it is less expensive (he submits a query not to forget the task) or when he estimates these costs as negligible.

4. RESULTS OF AUTOMATIC ANALYSIS
The analysis of automatically detected multitasking answers research questions. Searchers follow the principle of least effort and select the cheapest tactics in any situation; (a) more than 98% of temporal sessions are sessions of sequential execution; (b) during multitasking a searcher uses only 2 tasks and (c) he frequently executes them in an "enveloped manner": he interrupts one task session, starts and completes the second task session and returns to the unfinished task. (d) We consider separately 3 types of temporal sessions 1) sessions which include only one task ("1 task" column in Table 1); and 2 types of sessions containing more than 1 task and which, therefore, may be executed either 2) sequentially one-by-one (">1 task" column); or 3) in parallel (wide sessions). Table 1 shows quantitative characteristics of the search. First of all, logs of both search engines show surprisingly similar characteristics. Next, the

6. REFERENCES
[1] Jung , J.J., Jo, G-S. Semantic outlier analysis for sessionizing Web logs. 1st European Web Mining Forum. (2004) [2] He, D., Goker, A., Harper, D. Combining evidence for automatic Web session identification. Information Processing & Management, 38 (2002) 727-742 [3] Monsell S. Task switching. Trends in Cognitive Sciences, 7(3) (2003), 134-140 [4] Spink, A., Jansen, B.J., Park M., Pedersen J. Multitasking during Web search sessions. Information Processing & Management, 42(1) (2006) 264-275. [5] Spink, A., Ozmutlu, H.C., Ozmutlu, S. Multitasking information seeking and searching processes. JASIST, 53(8) (2002) 639-652.

624