Embodied conversational agents - let's specify and evaluate them!

16 July, 2002

Bologna, Italy

in conjunction with

The First International Joint Conference on

Autonomous Agents & Multi-Agent Systems


In recent years an increasing number of R&D projects have taken an interest in the deployment of so-called embodied conversational characters as avatars representing users in virtual meeting spaces, as part of the user-interface or, in some cases, even as the ultimate interface metaphor to information and computer-based services in a diverse range of potential application areas, including E-learning, E-Commerce, entertainment, and multi-user telecommunication. Across applications and projects the audio-visual appearance of characters vary widely in the amount of embodiment (e.g. face-only / full-body / avatar), the style of rendering (e.g., cartoon / photo realistic, 2D / 3D, synthetic / natural voice), and the sophistication of animation (e.g., still images / cartoon / realistic). There is also a great variation in terms of a character’s internal or "mental" abilities and skills – among other things, a character’s ability to engage in a conversation with humans or possibly other synthetic creatures, and whether or not it possesses further human-like qualities, such as personality and emotions.

Moreover while both presentational and internal aspects of a character may vary widely from one project to another, a central issue that must be addressed by all of them concerns the language and representation format which, metaphorically speaking, bridges between a character’s mind and body. Technically speaking, such a language is part of the interface between modules which are responsible for determining an agent’s behaviour on the one hand, and player technologies which are responsible for animation rendering and speech synthesis. Several attempts have been made to provide a standard for mark-up languages for characters. The VHML [] initiative deserves mentioning here as it subsumes some of the earlier attempts. When looking at ongoing character projects both in academia and industries, however, each project seems to define its own special-purpose languages and representation formats to specify expressive character behaviour. Unfortunately, proprietary solutions do little to foster synergies of results among research groups and may severely delay a broader up-take and exploitation of research results.

Agreeing on standard formats is important for sharing work, but another crucial component is evaluation. We need to be able to understand how well the goals of a system are being achieved, both in terms of the architecture, e.g. effectiveness of special-purpose languages, and in terms of application, e.g. how well an agent accomplishes a certain interaction with a human. By comparing results, we can make informed choices about what to incorporate, extend and rethink as a community of researchers. However, evaluation and comparison have been very hard because this is a relatively new field that both lacks evaluation tools, an established set of evaluation criteria, and empirical tests carried out to learn about the benefits of embodied agents on different aspects such as entertainment, mental load, system efficiency. To further complicate matters, the field employs a great diversity of paradigms, application domains and techniques.

The purpose of this full-day workshop is to bring together researchers and developers of embodied conversational characters to exchange ideas and experiences on:

  1. representation formats and specification languages for expressive behaviour:
  2. We are especially interested in the dependency between representation formats on the one hand, and the chosen theories of personality, affect, as well as verbal and non-verbal communication on the other hand. Last but not least we want to gain insight in how these disparate approaches and corresponding technologies are welded together. During the workshop we will identify and formulate requirements that a potential standard should provide. A further goal in time is to turn this specification into an international standard so that the whole community of researchers and developers of embodied conversational characters may benefit from it.

  3. the basis for evaluation of embodied agent prototypes:

We would like to concentrate on the added value of the use of embodied agents. After having seen many embodied agents in the past years, this time we wish to ask questions like: What are the characteristics and qualities of a specific agent? Are they really useful for the given application? What are the dimensions to evaluate a synthetic agent? What is the relationship of 'learning factors', 'pleasure factors' and 'efficiency factors'? What should a synthetic agent be like, depending on the intended usage and users?



  1. representation formats and specification languages for expressive behaviour
  2. Issues to be addressed:

    1. Evaluation frameworks, methods and results

    Issues to be addressed:



    The workshop will feature a mix of presentations, demos, and discussion rounds. The presentations could cover one of the several themes of the workshop and should be oriented toward:



    In case of space limitations, priority will be given to presenters and authors. Paper length should be 4-8 pages long (using 11pt, single space, all margins of 2cm) and should be accompanied as much as possible with an animation or URL’s showing multimedia content (actual systems, screenshots, animations, etc) describing the work presented. Every paper submitted will be reviewed by at least 2 reviewers from the program committee.

    Submissions should be emailed to Catherine Pelachaud ( in PS or PDF format by April 10th, 2002. If email is not possible, please sent two copies of your paper to (though, email is much preferred):

    Catherine Pelachaud
    University of Rome "La Sapienza"
    Computer Science Department
    via Salaria, 113
    00198 Rome Italy

    Note: Workshop participants will be required to register for the First International Joint Conference on Autonomous Agents & Multi-Agent Systems main conference.



    April 10 - Deadline for paper Submission
    April 30 - Notification of Acceptance / Rejection
    May 27 - Deadline for camera-ready paper
    July 16 - Workshop dates
    Post-workshop activities including publication of the workshop outcome
    in an appropriate channel.

    Andrew Marriott, Curtin University of Technology, Australia
    Catherine Pelachaud, University of Rome, Italy
    Thomas Rist, DFKI, Germany
    Zsofia Ruttkay, CWI, The Netherlands
    Hannes Vilhjalmsson, MIT Media Lab, USA
    Elisabeth Andre (G) 
    Norman Badler (USA)  
    Simon Beard (AU) 
    Nadja de Carolis (I)  
    Kees van Deemter (UK)  
    David House (S) 
    Katherine Isbister (USA) 
    Mitsuru Ishizuka (J)
    Khristina Höök, (S)
    Emiel Krahmer (NL)  
    Fabio Lavagetto (I) 
    Brigitte Krenn (AT)
    Nadia Magnenat-Thalmann (CH)  
    Stacy Marsella (USA)
    Shigeo Morishima  (J)
    Sharon Oviatt (USA)
    Ana Paiva (P)
    Helmut Prendinger (J) 
    Jeff Rickel (USA) 
    Candace Sidner (USA)
    Alistair Suttcliffe (GB)
    Daniel Thalmann (CH)