role
user researcher | methods writer
skills
personas, storyboarding, speed dating, affinity diagramming
tools
sketch, procreate, user interviews, post-it notes
More and more families are beginning to use agents, such as Google Home and Amazon Alexa. As technology advances, we aim to explore the boundaries of multi-user interactions with these agents, even ones that have not been created yet, in home settings. In a four month project, we ask many questions about the usage of these agents with families around pittsburgh to gain greater insight on perspectives of social agents. We are not looking to design the "perfect" agent, but rather understand attitudes and learn about what users consider acceptable behavior. Others can then use this research to gather information and also reflect on how we should be implementing these devices in the future.
The study was published in CHI 2019.
Who are the Users?
We first want to understand which type of user we are looking to understand through this research. We decided to choose families as our main group, as they compose a large portion of home agent owners.
We created several personas to get a better sense of our users, as well as establish an inclusive mindset since all families are different.
Find our personas here.
Ideation
Our team of seven designers began our ideation process by brainstorming questions, topics, and possible contextual boundaries in physical space, social roles, and environments. We generated about a hundred or so ideas using several design methods: custom generative card games which used card decks with various topics relating to the agents in the home to generate rapid ideas; New Metaphors, a method which employs the use of metaphors to reflect on abstract ideas. During this brainstorming process, we did not bound or restrict the capabilities of social agents. Based on our brainstormed questions and topics, we began writing one-line scenarios that involved agents in home settings.
As we went through the ideation process, we used affinity diagrams to organize the concepts and draw out themes in a structured and physical way. For example: Should an agent be allowed to access a spouse’s calendar without their permission? Should a child be able to track his father’s location through the agent?
Defining Themes of Exploration
Through affinity diagramming, we honed in on five main themes:
1. Proactivity of the agent — the agent initiates some action without the user’s asking
2. Permissions — an agent controls whether access is given or an action is taken
3. Computer-Like Behavior — the agent behaves more algorithmically and computationally
4. Deception — an agent purposefully deceives one user at the request of another user
5. Social DDD (Dull, Dirty, Dangerous) Work — the agent takes on a greater role in the social interactions in a user’s life
Scenarios & Storyboards
After determining themes, we wrote out lengthier scenarios of a few sentences while working out issues of fidelity and topic selection. We then created storyboards using prototyping tools such as Sketch and Illustrator to add a visual element to complement the scenarios. While all members of the team created storyboards, we used similar drawing styles, color, and formatting for consistency. Our intention was to select about twenty scenarios that would be neither too positive or negative, and would still provoke participants on the topic of interest. We did not want to skew participants’ opinions in one direction or the other and thus, avoided extremely dramatic contexts.
We ended up with about twenty scenarios, each dealing with a social agent handling a new situation for a family. These were chosen because they allowed us to investigate the role of agents in different homes, in interesting and non-trivial contexts. Participants are shown scenarios one by one and for each enactment, we asked several questions based on their initial responses.
Interviews
We interviewed about twenty famiies, each with at least one child over twelve. During the weeks that we were interviewing, we made, removed, and modified scenarios to better fit our study. Some were too negatively skewed, and others highlighted responses that were unhelpful to our study of the boundaries within multi-user interactions with agents in home settings. Because we were ocnducting qualitative, not quantitative research, we were able to be flexible with the storyboards we presented.
During the actual interview, we often went to the family's home to establish a more realistic environment that matched the storyboards. We explained the purpose of our study and presented each storyboard, asking broad questions to spark discussion within the family. For instance, we asked about how the behavior of the agent made the family feel and how it affected the family's relationship with the agent. We avoided the use of leading questions and focused instead on finding the deeper feelings for the agent as a whole.
“How does this agent’s behavior impact the family dynamic or the family’s relationship with the agent?”
Analysis
After transcribing each interview, our analysis focused on highlighting the important quotes that were said that reveal some interesting insight on how we view agents in the home. We then used affinity diagramming to draw out main themes that we found from various different individuals.
This pattern-seeking was a challenging, yet rewarding task. It required our team to push past the more "obvious" opinions such as not wanting the agent to give data access to any individual who asked to more deeper themes such as the tension between parents and the agent on the subject of their children or the interference of the agent into personal relationships.
Currently, the paper is being published in CHI 2019.