Introduction
The project will investigate theoretical
and technical aspects in the design of full-scale mobile guides
in different application domains. The project will propose an
integrated framework for the development of context-aware mobile guides, characterized
by:
- a standard and shared format for the representation of data
allowing interoperability between different devices and applications
- ability to derive information on the state and behaviour of the
user from common sensors data (e.g., GPS data)
- automatic generation of natural language presentations
- device-dependent generation and adaptation of the user interface
- usability-centered
input/output modalities based on visual interfaces and multimedia and
multimodal techniques
- multi-user features (mobile communication, community)
Three applications for three different domains (tourism,
culture and sport) will be developed to prove the effectiveness and
generality of the proposed solution.
Short presentation
Partners
Università di Udine, Luca Chittaro (Coordinatore)
Università di Torino, Ilaria Torre
Università di Pisa, Maria Simi
Goals of our research unit
The research activity of the Pisa unit will
focus on two essential aspects of the integrated framework for
context-aware mobile guides that will be jointly designed by the three research
units: the first one concerns the support of user cooperation that will be
deployed in collaborative learning solutions during group visits; the second one
concerns the implementation of a module for the generation of adaptive natural
language comments.
The solutions developed will be experimentally evaluated
with proper user studies. In addition, the application domain of museum guides
will be considered as a benchmark for experimentation of the proposed solutions
and a prototype guide will be developed for a specific museum (the Marble Museum
of Carrara).
This research will be performed in collaboration by
researchers of the University of Pisa and researchers of the group of
human-computer interaction of ISTI-CNR (main investigator Fabio Paternò).
The university researchers have competences in the fields of knowledge
representation, implementation of Web applications and services, search engines,
natural language analysis and generation. Of special relevance for this project
is a research activity performed in collaboration with the Accademia Navale di
Livorno (Esuli, Cisternino, Pacini, Simi, 2002), where a system was developed
for teaching about sail racing and regattas rules. The system features a
component for the analysis of dynamic scenarios (an expert system in sail rules
and strategies), a tridimensional visualization of the evolving scenarios and
automatic generation of comments in real-time. The system is able to adjust the
audio comments to the level of competence of the student and to the 3D
visualization times.
The ISTI researchers bring to the project
complementary competences in the fields of human-conputer interaction and
usability evaluation of man-machine interfaces.
In the field of
investigation of the project, they have experiences in the automatic generation
of adaptive comments for virtual guides in Web based museums (Marucci, Paternò,
2002). In this previous work, comments are generated dynamically according to
the user profile and to the information accesses effectively performed during
the navigation. These comments are created by enriching standard descriptions
with additional information of various types (introductive, comparison,
difference, summary, curiosity) according to a style which is typical of human
guides. In another project they developed a supporting software for a handheld
device (Ciavarella, Paternò, 2004) able to provide multimedial information and
services which depend on the room where the user is located. The result is a
palm-top guide, now available to the visitors of the Marble Museum in Carrara,
which is able to detect in which room the visitor is located by means of
infrared devices. This guide now offers predefined audio comments, as well as
text, graphics, images and videos.
The proposal is to extend this
previous experience along two directions:
- Dynamic adaptive comment
generation, taking into account users movements and their accesses to
information; along the lines of real time comments for regattas scenarios, a new
dimension could be the visit style and the time available which imposes
constraints on the length of the audio comment.
- Support for multi-user
cooperative games, in order to make the learning experience more effective and
enjoyable, by introducing the possibility of group cooperation.
Thanks to
the existing cooperation with the Marble Museum in Carrara, it will be possible
to experiment the results of the project within the museum, and to verify their
effectiveness with real visitors.
References
Esuli A., A.
Cisternino, G. Pacini, M. Simi, Adaptive Real Time Comment Generation for Sail
Racing Scenarios, Adaptive Hypermedia and Adaptive Web-Based Systems, Second
International Conference, AH 2002, vol. LNCS 2347, pp. 392-396, Malaga, Spain
2002.
Ciavarella C., F.Paternò, The design of a handheld, location-aware
guide for indoor environments, Personal and Ubiquitous Computing, Vol.8 N.2,
pp.82-81, Sprinter Verlag, May 2004.
Marucci L., F. Paternò, “Design and
evaluation of an adaptive virtual guide for Web applications”, Universal Access
in the Information Society, Springer Verlag, Vol.1, N.3, June 2002,
pp.163-175.
First year
The unit of Pisa will participate with the
other units to the definition of the requirements for the shared architecture of
mobile guides, and will implement in particular the modules dedicated to
multi-user functionalities and for community/group support.
During the
first year of the project, the Pisa unit will study the part of the software
infrastructure concerned with multi-user management for supporting collaborative
group visits.
Pisa will study in particular the domain of museum/cultural
guides and the requirements for a mobile guide in such a context. In order to
guarantee the generality of the solutions with respect to different kinds of
applications, they will acquire requisites from the other research units of
Torino and Udine, which investigate tourism and sport related applications
respectively.
In order to support cooperative learning by means of
interactions between visitors, it will be necessary to study the software
architecture and the infrastructure for communication and interaction among
participants, devise the best techniques and state-of-the-art technologies for
localization and wireless communication, investigate how to deliver intuitive
interfaces on mobile devices, and the support for context-awareness. Regarding
the user localization, we will adopt solutions characterised by a good balance
between installation cost and localization precision.
This task will be
performed in collaboration with the other units; it will be a specific task of
the Pisa unit to devise how to integrate in the software architecture detection
and communication of the activities performed by other users, coordination
aspects and the support for group cooperation, which are not usually required by
mobile guides for individual use.
For supporting cooperation, the
software infrastructure will have to include the following aspects:
-
distance communication by means of mobile devices mediated by a server;
-
management of data related to the accesses to information (art works, artists,
...) and to specific activities (for instance game related taks) performed by
users, as well as data related to user localization;
- the generation of
multimedia interfaces specialized according to the user and the device, which in
general could differ in the technology used;
- support for group
cooperation;
- the way to exploit guide related information in the context of
collaborative activities.
Moreover, during the first year of the
project, the Pisa unit will explore in particular the application of the
multi-user support component in the context of museum visits. They will
investigate the most suitable educational cooperative games to support the
museum experience and will devise application scenarios for the guides in
collaborative educational games within the museum. Two classes of games will be
considered: those where cooperation is explicit and those where cooperation is
implicit and is carried on by means of information concerning individual
activities performed by other users. In the former kind of activities users are
requested to share and discuss what they have learned during the visit and are
called to cooperate in the solution of a shared game. As an alternative, one can
think of different forms of implicit cooperation where users are invited to
explore the museum individually in order to solve distinct parts of a shared
game, such as a puzzle. Differently from what happens in the Nottingham project,
where the collaboration among visitors takes place only when they are in front
of interactive displays, the collaboration will be supported by mobile devices
during the all visit, or until visitors agree to participate in the game and
have a common goal.
Still in relation to museum guides, the different
content categories to be provided by the mobile guide and the different
dimensions in adaptivity will be analyzed and the requisites for the natural
language comment generation module for this application domain defined.
Finally, the unit of Pisa will study and define the specifications for a
museum mobile guide with cultural contents and multi-user edutainment
functionalities.
The application field will be the Marble Museum in Carrara,
a multi-disciplinary museum containing a number of sections, related in various
ways to the marble (sculpture, roman archeology, working techniques, crafts.
...). This material has been used in many artworks famous worldwide and for this
reason the museum has a wide foreign audience. It is located a few kilometres
away from the marble quarries, which are often the destination of tourist
visits.
The final goal is to create a guide supporting groups of visitors
(some of them could be inside the museum, some at the quarries site) in a way
that makes their experience stimulating and interesting.
Expected results
at the end of the first year
- Software module of the architecture to
support communication and cooperation among users by means of mobile
devices.
- Document on the implementation of the software module to support
communication and cooperation among users.
- Analysis and requirement
specification document for the implementation of a museum guide with natural
language adaptive comments, and support for multi-user visits and group games.
Second year
During the second year of the project the unit of
Pisa will study and take care of implementing the part of the framework
concerning the generation of adaptive comments, used to provide natural
languages commentaries on mobile devices, also in audio form.
The general
idea is to develop a methodology and software tools to deploy virtual guides
which can act very much like real world guides, by using similar techniques in
order to engage visitors and make their visit interesting and enjoyable. The
virtual guide application will be available in addition to the application
supporting access to information related to the artworks and related aspects, in
such a way as to add additional information depending on the user behaviour and
profile. The comments will be mainly provided through the vocal channel in order
to avoid interfering the real museum fruition mainly performed through the
visual channel.
In order to support a dynamic comment generation
functionality reusable in different application contexts, the generation
software will rely on a interoperable knowledge representation layer (which is
the task of Torino to produce); moreover it will be necessary to clearly
separate, in the comment generation strategy, the context-invariant part, which
could be reused in different applications, from the variable part.
The first
step towards defining and implementing the software module for comment
generation will be extending to the other application domains the analysis about
the categories of information which are being used effectively in the context of
museum guides (introductory information, comparison information, summary ... )
as well as the dimensions in adaptivity, necessary for flexible comment
generation.
This will be done with generality in mind, by considering the
result of parallel analysis carried on in cooperation with the other units in
their respective application domains and with the goal of devising the basic
discourse structures characterizing this kind of communication and their main
variants, as well as the rules for their composition.
This process will
result in the definition of an XML based language for describing contents so as
to render the interface generation system independent of the
application.
Next, a strategy of dynamic comment generation will be defined
and a module for adaptive and dynamic comment generation implemented, as general
as possible or at least easily adaptable to different context of use.
In
this second year, the unit of Pisa will also implement a prototype of mobile
museum guide with cultural contents and multi-user edutainment functionalities.
The technologies developed inside the project will be tested on the field,
by extending the existing experimental setting already in place at the Marble
Museum in Carrara. The new guide will extend the functionalities of the previous
one by introducing adaptive comments, generated dynamically in real time, and
will include exploration experiments by means of collaborative games to be
played by visitors.
On-field experiments will allow evaluating the
effectiveness of the adopted solutions with real visitors. The evaluation will
be performed with logging tools that provide quantitative information regarding
visit time, games performed, and the tasks accomplished. In addition, we will
ask users to fill in questionnaires, which can be useful to gather information
related to their satisfaction and appreciation of the museum
experience.
Expected results at the end of second year
- Software
module for the dynamic generation of natural language comments
- Engineered
prototype of a guide for the Marble Museum in Carrara, with edutainment
functionalities and dynamic comment generation.
- Report on the comment
generation module and guidelines for contents editing.
- Report on usability
evaluation of the guide prototype, on the basis of users tests with visitors of
the museum.