SIGIR 2007 Proceedings

Demonstration

Mobile Interface of the MEMORIA Project
Interactive Multimedia Group, CITI and DI/FCT, New University of Lisbon Multimedia and Machine Learning Group, Instituto Superior de Engenharia de Lisboa Quinta da Torre, 2825 Monte da Caparica, Por tugal
1

Ricardo Dias1 , Rui Jesus1,2 , Rute Frias1 , Nuno Correia1

2

rjfd@di.fct.unl.pt

ABSTRACT
This pro ject develops tools to manage personal memories that include a multimedia retrieval system and user interfaces for different devices. This paper and demonstration presents the mobile interface which allows browsing, retrieving, and taking pictures that are automatically annotated with GPS data and audio information. The multimedia retrieval system uses multimodal information: visual content, GPS metadata and audio information. The interface was evaluated in a cultural heritage site. Categories and Sub ject Descriptors: H.3.3 [Information Storage and Retrieval]:Information Search and Retrieval; H.5.2 [Information Interfaces and Presentation]:User Interfaces. General Terms: Algorithms, Design, Human Factors. Keywords: Mobile User Interfaces, Multimedia Information Retrieval, Personal Memories. Figure 1: User interface for mobile devices (PDAs) evaluated in a cultural heritage site. The visitors can share their pictures with other excursionists and use the retrieval system to guide the visit.

1. INTRODUCTION
People are collecting a huge amount of media information related to their personal experiences and they have difficulties in organizing these agglomerations. In effect, the lack of structure in multimedia collections can result in misplaced images or inability to appreciate recollections. If possible, collecting and retrieving personal memories should not mean time-consuming file organization but a pleasing experience. The Memoria pro ject develops a new set of tools for storing and retrieving personal digital memories without any kind of user organization. It comprises a multimedia retrieval system, and several user interfaces. This paper describes the mobile version, which runs on a PDA. With the mobile interface users can access the system features ubiquitously, share their pictures instantly and on location. The proposed mobile interface allows browsing, retrieving of photos and taking pictures that are automatically annotated with GPS data and audio information. The multimedia retrieval system that uses visual content and context metadata (GPS data and audio information) is triggered by the user interface by means of queries composed by the user. These queries can be defined by concepts [1], location information, audio information and visual content or also by a composition of the previous ones. The queries results are visualized in the user interface. This application was
Copyright is held by the author/owner(s). SIGIR'07, July 23­27, 2007, Amsterdam, The Netherlands. ACM 978-1-59593-597-7/07/0007.

2. USER INTERFACE
This interface contains three main system functions: picture capture, photo retrieval from a server and results display, which are all constantly available from the main screen. Other features however can be personalized to overcome the small screen available and improve the visual experience. The visualization is personalized by a pull down menu and contextual controls that turn on and off layers with results (thumbnails grids), paths, maps, preview images and slide shows. The picture capture feature starts a camera interface that allows shooting pictures and in the future it will also allow to record small movie clips. The design of the user interface was done to facilitate image retrieval with filters. Queries are defined using drag and drop operations. The user can drag many visual items into the query box, (lower right corner), in order to compose different queries. Besides thumbnails, images and concepts, the user can drag directions, map sections or time periods. With this application and interface visitors of a cultural heritage site can preview the locations they intend to visit, discover direction in case they get lost, find photos of the surroundings captured by other visitors as well as record digital memories of the trip.

3. REFERENCES
[1] R. Jesus, Abrantes A., Correia N., "Photo Retrieval from Personal Memories using Generic Concepts", Advances in Multimedia Information Processing ­ PCM 2006, Springer LNCS, vol.4261, pp.633­640, 20.

904