AI Augmented Writing
Action
Outcome
Business Requirement
The Apple App store "app of the year" has millions of active users. App can recognize and convert handwriting into text. However, note taking can be a time-consuming and stressful experience.
The business wanted to help users take better and quicker notes by introducing AI-assisted features. Some of the success criteria and KPIs were:
Increased customer acquisition : Users will perceive it as a differentiator
Task efficiency : Help improve users' note taking efficiency
Reduced cognitive errors : Help users avoid errors and articulate their thoughts
Feature adoption : Users will want to use it for various use cases
I conducted exploratory and evaluative user research to understand the needs, expectations, perceptions and challenges of using AI assisted note taking features and provided a comprehensive report with evidence-backed insights and actionable recommendations.
Requirements Gathering
Research Framework
Research Plan
Usability Tasks
Research Analysis
Research Report
The Apple App store "app of the year" has millions of active users. App can recognize and convert handwriting into text. However, note taking can be a time-consuming and stressful experience.
The business wanted to help users take better and quicker notes by introducing AI-assisted features. Some of the success criteria and KPIs were:
Increased customer acquisition : Users will perceive it as a differentiator
Task efficiency : Help improve users' note taking efficiency
Reduced cognitive errors : Help users avoid errors and articulate their thoughts
Feature adoption : Users will want to use it for various use cases
I conducted exploratory and evaluative user research to understand the needs, expectations, perceptions and challenges of using AI assisted note taking features and provided a comprehensive report with evidence-backed insights and actionable recommendations.
Hypotheses
Cognitive Load
With Auto-complete users could take notes efficiently by completing words automatically which could reduce cognitive load on working memory and free up attention
Hypotheses
Augment Language
Using AI to augment note-taking process could be valuable for students when they want to use complex or long words, particularly for those having limited language fluency
Hypotheses
Control & Flexibility
Users may not be comfortable with AI completing the words without their approval. There needs to be a fine balance between automation and enabling user intention
Based on the research objectives, I conducted literature review and used mixed methods research to validate the hypotheses across identified user segments. I formulized a research plan that involved using task analysis as a method where users would perform the following three tasks simulating three distinct situations:
Copying : writing without much thought or auditory input
Dictation (class lectures) : writing based on auditory input + self thoughts
Reflection : writing based on self reflection
Artifact Review - Students' Past Notes
Task Analysis - Comparison & Preference
Task Analysis - Video Lecture Notes & Summary
Self Rated Responses
Semi Structured Interview
Users would "Copy" thrice, once each with a control and two variants (randomized to reduce the recency bias). We used three different variants for comparison and randomized the order to reduce the recency bias.
Variant A : Activate Ghostwriting and Suggestions
Variant B : Activate suggestions without Ghostwriting
Variant C : Activate Ghostwriting, requires long press for suggestions
"I don't think that my handwriting changed a lot, but it is not guessing it correctly".
~ student
After the interviews were over, I analyzed transcribed and tagged the interviews. I assigned codes / tags to findings.I further organized the tags into clusters and generated themes out of emerging patterns and relationships. These themes were then synthesized into insights with supporting videos and quote from the participant.
I further compiled all the insights into a report with clear evidence that either validated or invalidated the initial hypotheses.
Preference
Users largely preferred the variant B, the one where suggestions are automatically activated on drawing a line and there is no ghostwriting or preview. They felt it was faster and easier to work, was comparatively more responsive* and required one less step to look at Ghostwriting.
Those who preferred the variant A with Ghostwriting, felt that it is easier and faster as they could quickly double tap and commit a suggestion. This would work if prediction accuracy is high. However, suggestions were not always accurate and that could work against this option.
Behavior
Users had to think of number of letters to write and try several times before they could get the words right suggestions which made them believe that writing itself is faster than using Auto-complete. e.g. for one participant, writing “Il” gave “India” as suggestion and even after trying several times, he could not get the right word
Those who preferred the variant A with Ghostwriting, felt that it is easier and faster as they could quickly double tap and commit a suggestion. This would work if prediction accuracy is high. However, suggestions were not always accurate and that could work against this option.
Helpfulness (Task: Video lecture)
Helpfulness (Task: Summary)
Accuracy (Auto-complete suggestions)
Trustworthiness (Auto-complete suggestions)
Attitude
Users believe auto-complete could be useful for writing words that are long or difficult to spell and could offer alternative word suggestions which could help them improve their vocabulary and learn as they get better with words.
However, Auto-complete is difficult to trust for taking notes in high stimuli environments requiring attention and decision making such as during lectures notes as it creates high cognitive load and students fear they might miss important points.
Iterate : considering insights and suggestions from the study
Improve : accuracy of suggestions by improving handwriting detection
Contextualize : suggestions based on text before to improve accuracy
Reframe : problem statement to impact learning process meaningfully