Workshop Plans

The plans for the workshop are as follows:

Pre-Workshop

Participants will be recruited from those who have worked or published on this topic and those who participated in previous relevant events, some of which are listed below. We anticipate a balanced mix of participants from academia and industry and from work in different sub-genres. Our call identifies thematic interests of the organizers, but the workshop content will be shaped by the position papers and study results that are submitted. Submissions will be clustered into tentative themes to form the basis of breakout groups. Position papers will be shared in advance and not presented at the workshop.

Themes

Our interests and those of the submitters will be used to identify a few significant areas in which to address challenges and consider opportunities for future research. Below are examples of possible themes, only a few of which can be explored in a one-day workshop.

  • Assistant/agent goals: open-ended dialog (e.g. ELIZA), narrow task focus (customer service), broad quick-response (Siri): How do we design for efficiency and engagement in different contexts?
  • Assistant/agent capabilities: How can users learn about limitations, capabilities, upgrades?
  • Usage scenarios: How do design assumptions shift when an agent operates behind organizational firewalls or supports patients and doctors in hospitals?
  • Usage and engagement: How do we measure success when use can be regular or occasional, short-duration or long?
  • User perceptions: How are assistants understood and experienced by users?
  • User-chatbot behavior: How is conversational behavior affected when talking to machines?
  • Skills: When should designers add a skill to an existing agent or create a new bot?
  • Context: How much context—conversation history or software application—can and should bots make use of? What issues arise when crossing boundaries, for example between home and work?
  • Platforms: Standalone apps, embodied in boxes or robots, or software platforms (SMS, email, Messenger…) What is effective for what purposes?
  • Modalities: When should agents support voice, text, images, video, menus as input or output?
  • Personality: When is personality beneficial? How should gender be selected?
  • Ethics and trust: How do we design responsible experiences? Manage bias? Should agents reflect societal norms or embody aspirational goals?
  • Bots in social networks: What changes when chatbots join networks of humans and bots?
  • Humans as partners: When should agents escalate requests to humans?
  • Training and assessment: How should training data be collected and curated? How can agents best be tested prior to general release?

Workshop Activities

Time Activity
8:30 – 900 Arrival, poster set up and viewing
9:00 – 9:15 Welcome and introduction to the goals and schedule
9:15 – 10.00 Opening discussion of themes. Participant introductions
10.00 – 10.45 Pre-determined groups form, adjust, start discussion
10.45 – 11.00 Coffee break and poster viewing
11.00 – 11.45 Groups resume discussion
11.45 – 12.30 Plenary: report out by groups, discuss afternoon
12.30 – 13.45 Lunch
13.45 – 15.15 Group or new group discussions
15.15 – 15.30 Coffee break and poster viewing
15.30 – 16.45 Plenary: report out and look for syntheses
16.45 – 17.00 Future plans and wrap up

This is a tentative plan. After the welcome, the workshop will have a group discussion of the pre-arranged themes and possible cross-cutting issues during which each participant with a comment will first introduce themselves. Before lunch, the morning breakout sessions will report. We will plan the afternoon, deciding whether to continue with the same groups, form new ones, or have some group members switch to share insights. The final 90 minutes will focus on conclusions, possible syntheses or divergences as we consider what a comprehensive view of conversational user interfaces might look like, and assess what would be helpful moving forward, both for the workshop output and for the field.

Post-Workshop

This workshop follows a handful of CHI SIGs, panels, courses, and paper sessions on related topics, as well as similar discussions in related fields. The workshop will pull some of this work together and get the word out. The timing is appropriate as other events are being lined up to take on some of these issues. Interactions, Communication of the ACM, a special issue of a journal, a guideline for practitioners, or a CHI 2020 paper are possible products we could pursue.