Can Zhao



Toki is a meeting assistant that automates the note-taking process and transcribes conversations into digestible data.

Click above to watch our product video!



Timeline: 20 weeks
My role: Interaction Design, Information Architecture, UX Research, Prototyping
Tools: Sketch, InVision, Flinto, Photoshop, Omnigraffl

Design Challenge


Team Members

Can Zhao
Xinbei Hu
Corey Brown

How might we manage complexities of modern business meetings by capturing important moments for multi-disciplinary attendees?

Overall Process

complete process.png

This is my visualization of the whole process. Sometimes activities overlap within a period of time. 


Problem Setting


Rethink the future of modern business communications.


Attendees from various disciplines contribute diverse perspectives to modern business meetings. By talking to software engineers, product/project managers, and designers in technology companies (view research process), we summarized the following pain points in multi-disciplinary meetings: 

1. Difficult to cross knowledge gaps
knowledge gap.png

In multidisciplinary meetings, attendees find it difficult to distill and engage useful details of another discipline due to knowledge gaps of different domains.


"It makes the meeting less productive to get into very detailed domain-specific topics when meeting with people from different backgrounds."

- Participant 8

2. Hard to keep track of conversations

Attendees might forget the context of meetings during conversations and thus lose track of conversations.


"It's really useful if people can provide some context like, information that tells me, how we are, where we are now. A lot of times what we're trying to do is getting from one place to another. If you don't understand how you got there then you're missing the context. "

- Participant 1

3. Cumbersome to remember everything

A mismatch between attendees’ behavior and expectations results in a vicious circle of seeking help after meetings. 


"Sometimes people just don't retain everything that happened during meetings and just ask you questions...I guess it's hard anyways. I always send out notes with references after meetings, but of course, nobody reads these emails these days."

- Participant 9


The Opportunity

"In reality, it would be impossible to confirm each other’s memories and understanding of words during meetings despite the gaps of various memory capacity and understanding of things…The thing that makes meeting powerful is to create a shared memory."

- Kevin M. Hoffman, Author of Meeting Design


Our Solution

Toki encapsulates important decisions for meeting attendees by creating a shared timeline that everyone can refer to during or after meetings.



Product Features

How It Works

Toki can be deployed on smart speaker systems.

It utilizes speech-to-text technologies to parse conversations during meetings. Toki marks important moments as decisions by capturing trigger words and phrases in conversations such as :”can you do…”, “follow-up”, “due date” etc.

During-Meeting Features

Mark Decisions/Agreements on the Timeline

Listen and parse conversations

Generate a “moment” when a decision is verbally confirmed
Augment Note-taking with Relevant Transcripts

Suggest previous transcripts that are relevant to what the user is typing

Help user record the context when taking notes
Provide Private and Public Notes Options

Empower users to share their thoughts with attendees

Post-Meeting Features

Provide Context of Decisions and Notes

Document all the important information of decisions

Save snippets of transcripts around each important moment

Visually represent possible connections between notes and decisions on the timeline
Accommodate Second Thoughts about Decisions

Allow users to "push back" on decisions when necessary

Provide Screen Recordings of Attendees’ Shared Assets

Screen recording when a laptop is plugged to a projector

Cater to need to share various types of documents
Allow Users to Filter Moments by Name

Let users quickly locate moments when there are too many on the timeline

Contextualize users' private notes by locations on the timeline

Research Process

At the very beginning, we tried to understand the scope of our design space - meetings in modern companies.  We explored this problem space with competitive assessments, surveys, secondary research, semi-structured expert and user interviews.

Competitive Assessment

In order to better understand our problem area, we conducted a competitive analysis of 8 applications that either feature working environment or focus on meeting experience. Asana, SharePoint, Confluence, LessMeeting, Voicera, Quip,, and Basecamp were analyzed.
We examined them in terms of their features, information architecture, and user interface.  

We map their functions onto a 2x2 matrix: the Individual vs. Collaborative axis distinguishes the concepts focused on personal management versus collective contributions; the Multi-Function vs. Meeting-Specific axis distinguishes the concepts focused on contextualizing meetings with multiple functions verses meetings specifically.

The blank space in the lower-right suggests our opportunity space and direction: We should to design a tool that is both collaborative and meeting-specific.


We distributed the survey via email and social networks to those who are currently working or have working experience in the United States. We collected and analyzed the results of the survey as responses are submitted. The survey also functioned as a screening to recruit users in the following interviews.


Information of the 204 survey respondents

Our survey results (based on 200+ responses) show that multidisciplinary meetings occupy at least 20% of all the meetings respondents attended last month. In response to the most common challenges of meetings, 43.3% chose “keeping meeting focused and on track” and 38.3% chose “coming to decisions”.

Expert & User Interviews

affinity diagram.png
5 experts and 12 users

Our semistructured interviews consist of 2 parts, expert and user interviews:

1. Expert: We first remotely interview the author of Meeting Design, Kevin Hoffman. Then we went to Microsoft Envisioning Center and focus-group interviewed:
Jessica Cohen (meeting metrics)
Brian Stucker (Envisioning Center)
Charlie Chung (Outlook and Exchange)
Caitlin Hart (Outlook Calendar)

2. User: We recruited and interviewed 12 users from California, Washington, Minneapolis, Chicago, Georgia, New York, and Pennsylvania

7 insights and 34 themes

We used affinity diagramming to organize the interview data. The blue notes represent themes of findings; yellow notes represent evidence from users; red notes represent insights.

I distilled these insights into 3 problems as shown in Problem Setting


Synthetic Model


As the following model illustrates, we formed 3 personas that represent different types of users in our interviews. 


Click to Play the Video


Design Principles


Promote individual awareness of context.

The solution should provide context during the meeting to help attendees to be aware whether they are off topics or not, and help them to stay on track.

The solution should inform the attendees of the current state of the meeting.

If we layer each attendee’s engagement levels, the red highlighted areas represent moments with high variation of engagement.

1. When Keith is passionate about some detailed domain-specific topics, Sean’s engagement level drops down.

2. Immediately following the meeting, Diana sends out all the notes, yet still receives questions. Her engagement level keeps at a certain level because she’s the only resource people can turn to. 

 This model informed us the direction of our design path.


Ideation and Evaluation


Testing with purpose

We had 3 round of testings.
For lofi (paper) testing, we aimed to narrow down our concepts; for mid-fi testings, we aimed to improve usability and iterate on the core features of our product. For hi-fi testings, we aimed to focus on usability problems. 
During this stage, I recruited most of the participants, made half of the prototypes, and wrote all the testing scripts and evaluation reports.

Paper prototype Testing

108 Concepts → 5 Concepts → 1 Concept

We narrowed down from 108 concepts to 5 concepts based on the principles of viability, desirability, and feasibility, and made paper prototypes for the 5 concepts (I made 2 of them). After testing the paper prototypes with 4 designers and 1 engineer who work in tech companies, we reached 3 major insights

1. Self-documentation can distribute accountability to each individual and thus can be useful in the future.
2. While auto-transcription of meeting conversations can help attendees trace context, reading long transcripts is distracting during meetings.
3. Whiteboarding and other forms of assets sharing (rather than slides) are common especially at the early stage of a product cycle.

Mid-fidelity prototype Testing

Based on the results of the 1st testing, we revised our concept to "Interactive timeline," and used InVision to build our mid-fi prototype.  

We tested the prototype with 2 designers and 3 product managers (age 25~50).

During the testing, we were able to perceive how our target users would interact with the prototype and their insights on the concept of an interactive timeline. Below are a list of key findings that include usability problems, needs, and concerns.

During-meeting Findings

1. Three out of five intended to type notes under “Notes” section and ignored the button "Mark Notes."
2. Three out of five thought it would be too much work to click, highlight, and make notes during meetings.
3. Four out of five participants and didn’t realize the differences between “Mark Moment” and “Mark Notes.
4. Three out of five participants did not notice the collapsing button when requested to collapse the timeline.
Post-meeting Findings

1. Three out of five had trouble returning to the timeline to click notes. They’d prefer to click the timeline and see notes and details of the moments on the same page.
2. Three out of five had concerns about overcrowded content on the timeline. They wanted to filter for specific person or certain time range.
3. Two participants thought the notes were others’ notes because they’re under different names.
4. Two participants found it acceptable to have access to complete transcriptions of meetings. Three participants had privacy concerns. Only seeing moments of transcriptions instead of the complete transcriptions in “moments” might reduce such concerns.

All the participants thought providing context during or after meetings would help target users retain useful information. We decided to move forward with this feature and and iterate based on the findings above. 


High-fidelity prototype Testing

Testing Process

We recruited 2 engineers, 1 designer, and 1 PM, and asked them to interact with the prototype in a coffeeshop near their working places. The testing consisted of 4 parts:

1. We briefly explained what toki does and how it works for users.

2. We made up a working scenario and asked the user to imagine themselves in that situation.

3. We explained what (fictionally) happened during and after the meeting, and asked the user to interact with the prototype accordingly.

4. We asked for users' general feedback and some follow-up questions.

Main Findings

1. The timeline lacks clear visual cues for users to recognize “moments” and “public notes.”

2. Users cannot tell whether the avatar photo represents a task owner or an assigner of the task at a glance.

3. Users’ perceptions of color labeling vary across roles and companies.


Final High-Fidelity Prototype


User Flow

This user flow illustrates a typical user journey of generating moments, adding notes, and viewing each item on the timeline during or after meetings. Moments are pieces of transcripts around tasks, decisions, or actionable items.


I created the architecture (user flow and system map) for the prototype.


Visual Styling


My teammate Xinbei finalized the styling. 


Final Thoughts



In this project, the most valuable thing I learned is to triangulate from all the data - not just from different methods of research, but also from the larger picture, to synthesize the feedback from users, our sponsors, and our mentors. While user testing is incredibly valuable, as a UX designer and researcher, I had to figure out what the users really meant and wanted. Sometimes they don't need a faster horse; what they really need is a car. When our sponsors’ opinions diverged from our mentors' opinions, I learned to evaluate both, figure out the rationale I identified with, and persuade both sides of the path I chose to follow.  In the end, although all feedback is valuable, rather than following blindly, our team had to decide how to design a solution that takes all perspectives and data into account and produces a useful product.

Future Directions

If given more time, I would definitely work on the on-boarding for toki. Currently, toki is based on the assumption that all companies have some log-in system that can be deployed to recognize each meeting attendee’s voice. However, how and when should toki collect their voice? How can I design such data input without causing privacy concerns? Another important part of the on-boarding is error-correction. Ideally, toki can gain more accuracy by learning from a pattern of positive examples. This interactive machine learning model can be successful if correcting toki is easy enough for people to be willing to perform it during meetings. Otherwise, users won't bother to use it.