Exploring CONTEXT and INTENTION in human-AI interaction. AI agents, social games.
In collaboration with: Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, India.
Stakeholder: Dr. Ravi Kiran Sarvdevabhatla (Associate Professor); Computer vision and applied machine learning.
human-ai
collaboration
Prologue
Intention and context play a vital role in human-AI interactions. This case study explores how context and parts of speech affects AI perception and a change in outcomes.
How does transparency improves trustworthiness in AI?
How do you balance cognitive load with engaging gameplay?
What are the ethics of Human-AI collaboration?
Through iterative refinement, we navigated ethical dilemmas, technical hurdles, and behavioral differences. Scroll down for summary to uncover the insights at a glance.
Summary
PROBLEM
How does the presence of AI affects collaboration in social settings, when people are geographically separated?
SOLUTION
We developed an online social game, DePiction. It enabled us to study behavioral patterns between humans and AI, focusing on collaboration and communication.
RESULTS
It was interesting to find out that AI was able to depict nouns through sketches but struggled to portray verbs which need more contextual understanding.
MY IMPACT
400%
Boosted user retention by 400%.
100+
Enabled 100+ concurrent sessions to optimize telemetry.
40%
Reduced design turn around time by 40%.
KEY DECISIONS
The major decisions I took centered around understanding the system (AI) complexity, enabling me to apply the right design principles.
MY ROLE
Human-AI interaction design, design system, user experience design, and front-end development.
MY TEAM
1 HCAI practitioner (me), 3 Software Developers, 1 Researcher, 1 Supervisor
social cooperative game
1 The Game: Pictionary Reimagined
DePiction is a social game. But for the study it was restricted between two players. This small change helped us in understanding the collaboration between people when they are geographically separated. Additionally, gameplay helped in extracting information when AI came into picture.
understanding the user
2 User Demographics
We had a large pool of participants spanning from a wide age group. Although most of the users were male and right-handed (as sketching was involved, dexterity was recorded), a large group helped in recording exhaustive quantitative data.
479
Participants
328
328 male and
98 female
388
388 right-handed
38 left-handed
19-60
Diverse age range
Identified problems
3 Understanding Design Requirements
Design Systems
DePiction did NOT had a design system to maintain standardization and responsiveness.
RESPONSIVE UI Design
Our users belonged to a wide age group having different preferences when it comes to engaging in gameplay, which motivated me to design a responsive gameplay experience.
design systems
4 Building Design Systems
The design system was initiated through small tokens, which were later merged to build complex components, used for building interactive tokens and pages. The process is detailed below-
Atoms
These atoms include basic HTML like forms, labels, inputs, buttons and others that cannot be broken down any further without ceasing to be functional.
Pencil
Button
NEXT
False Alarm
Molecules
UI elements functioning together as a unit. For example, a form label, search input, and button can join together to create a search form molecule.
Pencil
Eraser
Highlighter
Guesser on right track
Guesser on wrong track
Correct
Organisms
Complex UI components composed of group and molecules and/or atoms and/or other organisms.
Templates
Templates are page-level objects that place components into a layout and articulate the design’s underlying content structure.
Pages
Pages are specific instances of templates that show what a UI looks like with real representative content in place.
breaking down complexity
5 AI Design Challenges
Designing for Probabilistic AI
4 possible outputs
system OUTPUTS
Fixed outputs - Hit, Miss, False Alarm, Correct Rejection
200 words in corpus
SYSTEM CAPABILITY
Current database is fixed, therefore, probabilistic.
To design for probabilistic AI, I took two things into consideration: system outputs (number of outputs) and system capability (corpus length). This allowed me to design for all the scenarios.
ethics, perception, and transparency
6 Ethics, AI Inclusion
Challenge: AI Ethics, Human-AI Perception
In the initial iterations, the design did not acknowledge the fact that the user may be playing with an AI (acting as a human) unbeknownst to them, raising some ethical questions.
Solution: Designing for Transparency
I prototyped few design ideas, ensuring the application conveys the possibility of being paired with an AI in-game. I designed to convey possibility and not certainty, leaving the decision to the user.
Outcomes: What I Designed
AI Inclusion
Designing for AI ethics, ensuring user autonomy.
I implemented a loading screen message, one of the user said: “I didn’t know that AI was involved in the game”. Highlighting the importance of transparency and user expectations.
improvements implemented through redesign
7 Solving the pain points
Due to a strict timeline, I also incorporated heuristic analysis, enabling me to solve pain points without user testing. Navigation, color palette, and legibility were revisited and redesigned to adhere to WCAG guidelines and Neilson’s heuristic principles–reducing noise through CTAs.
Navigation
Color Palette
Legibility and Readability
Reduced Cognitive Load
Before
After
Challenge: Long list of instructions overlooked by users!
First iteration was only focused on identifying proof of concept and didn’t follow sound design principles leading to quick fixes and instruction-driven interactions rather than intuitive ones.
58%
I revisited the instructions screens and through common patterns, reduced repetitive screens, from 19 to 8 (58%), reducing the boring tasks.
redesign
8 Final Designs, DePiction
Onboarding
The onboarding process include sign up and login as part of data collection. User names being stored as pseudonyms to maintain privacy.
Loading screens and instructions
I designed the loading screens to improve trust among users in the use of AI, and optimized instructions to ease the learning curve.
Gameplay (drawing and guessing)
Finally, I designed the drawing and guessing screens, which captured user reactions and interaction patterns. Highlighting differences between human and AI interactions.
AI perception
9 Nuances of Human-AI Interactions
Context
Verbs and adverbs can be tricky to draw for AI agents, without context. Diving; Diving in the ocean or into a pool? Where should the person be positioned? Complicating it further.
Intention
Human players have intention, whether to win or just have fun. AI being devoid of this has the same style of play regardless of prior game results, leading to a monotonous gaming experience.
Human-Human vs Human-AI Interactions
Humans interacted more with the subjective markers, like thumbs up and thumbs down from the tool bar. AI tends to convey its messages only through sketches lacking context and intention.
development
10 Results
Concurrent session augmentation
I achieved over 100 concurrent gaming sessions, which was a result of guiding the user with CTA, designing trustworthy probabilistic AI, and solving the pain points, expediting data collection.
AI agents in social context
We identified how parts-of-speech affect human-AI conversations when visibility is limited (limited to text conversations). Nonetheless, our training datasets will enable the development of better AI agents.
learnings
Epilogue
AI Agents
I learned how AI agents can be optimized for specifics. Enabling with multimodal interactions improves the user experience with regards to context and intents.
Ethics in AI
I realized that how users may perceive AI is essential, raising questions on AI ethics. In the controlled experiment users were not made aware of the use of AI right away, but in general settings should they be?
Trustworthiness in AI
It is vital to clearly communicate AI capabilities with the user which ensures the building of right expectations from the application, leading to an increased trust.
