Leveraging Emotional Design for Improved Interaction
Enhancing User retention in Business Messaging App: A Research Proposal
Yarden Ben Shushan | UX Designer
Shay Mohar | UX Designer
Project Overview
This project was part of the "Quantitative UX Research" course's final assignment. The goal was to showcase the ability to design and present a thorough research plan, applying causal research principles using descriptive statistics from a B2B/SaaS product dataset, and demonstrating expertise in user experience research.
The Product
The product is an instant messaging application designed for business use, similar to Slack. Its goal is to facilitate communication and collaboration among team members.
After conducting a thorough analysis of the product's dataset using the Mixpanel platform, we derived several key insights that guided the subsequent phases of our research.
Key Performance Indicator (KPI): Improving User Retention
Preliminary Data Insights
Most Common Action: Sending messages.
Average Number of Messages Sent Per User: 1.2 messages in the last 30 days.
Drop-Off Rate (from Tutorial Completion to Message Sending): 85.63%.
Median Time to Send First Message: 24.8 days.
Identified Issues
01 Discrepancy Identified: Between common action and actual message sent per user.
02 Extended Time to First Message: Users take a long median time to send their first message.
03 High Drop-Off Rate: Significant decline in user engagement post-tutorial completion.
Theoretical Basis – Emotional Design by Donald Norman
Visceral: Users form a first impression based on the interface's appearance within 30 seconds. A positive impression increases the likelihood of continued use.
Behavioral: During use, users subconsciously evaluate how well the design helps achieve their goals. If the interface matches their mental model, they find it easy to use and are likely to continue.
Reflective: Users consciously evaluate the interface's performance and benefits. Satisfaction leads to continued use, emotional connection, and recommendations.
From this data, we infer that the tutorial process does not meet user goals as it causes cognitive overload by explaining less common actions (e.g., sending messages). According to the second stage of emotional design theory, users assess whether the interface helps achieve their goal (sending messages) easily. Currently, the tutorial is lengthy and unhelpful.
Research Question & Hypothesis
Question: How does the tutorial procedure impact users' first message-sending behavior?
Hypothesis: Making the tutorial optional will improve retention and increase the frequency of message sending.
Variables & Measurement
Independent Variable
Mandatory vs. Optional Tutorial
Dependent Variable
Nominal Definition: Rate of Message Sending
Operational Definition: Average of 3 messages sent per user per month
Target Audience & Sample Size
Target Audience
New users who are about to start the tutorial
Sample Size
The study comprises two groups, each with approximately 49 subjects, drawn from an average of 98.7 new users in the last 30 days.
Breakdown by Usage Platforms
38.83%
61.17%
Desktop (Windows, OSX & Linux)
Mobile (Android & IOS)
Breakdown by Tutorial Completion Time
61.03%
38.97%
Users who completed the tutorial between 30-70 minutes
Users who completed the tutorial in less than 30 minutes
Research Methodology
Duration: 30-day A/B testing period
Tools: Utilizing Google Optimize for A/B testing
Aids: Mixpanel for data analysis and insights, and Emotional Design theory as the basis.
Research Mapping
Use: Natural use - users interact with product without researcher instructions
Study Type: Causal study focusing on the impact of making the tutorial optional.
Axis: Do - Say & Qualitative - Quantitative
DO
Qualitative
Quantitative
SAY
Purpose of the Study
Increase message sending through the interface
Key Focus
Changing tutorial procedure to improve message sending rates
Approach
Summative assessment to evaluate tutorial impact on message sending
Variant A
Variant B
Conclusion
We believe that the A/B testing we proposed, based on the theoretical framework presented, will demonstrate that making the tutorial optional rather than mandatory will lead to an increase in retention. This change will result in users sending more messages within the interface, thereby improving overall engagement.