*The details specific to the case study have been generalized due to the project’s protected information.
Our team at RS21 was asked to help our client create accessible, practical documents from a library of reports. While there were several core stages in solving this problem, this case study focuses on the UX experience process, specifically a two-day workshop we conducted with our clients.
The library we were working with contained dozens of reports and each report had a unique focus. The information in the reports was isolated both physically and digitally. If the reader wanted to find ALL the information on a specific topic, they would need to read ALL the reports touching on that topic.
A large library of documents made finding all relevant information on a topic problematic.
Each report was also highly contextual. It would not contain information about or reference another report that covered the same topic. For example, Report A, about apples in Alabama, would not cross-reference information in Report B, which is about apples in Washington.
[Also of note is that this project was a pilot to explore how NPL (natural language processing) could be used to find insights, themes, and hidden patterns within the overall library. That’s a whole other case study but it’s relevant to understand the benefit of having AI doing the heavy lift of parsing the reports described below.]
An analyst would need to read through pages and pages of documents.
The next step was to understand what information our end users needed. These needs would affect the content and format of the final outputs.
Our client was based in Washington D.C. while the RS21 office is Albuquerque, NM. To maximize resources, we interviewed the project’s key personnel via conference call to build context and develop a deeper understanding of the problem and the solution we were building.
UX tool: Individual interviews
The product we designed for the client would impact, and be used by, many partners spread across a wide cross-section of departments. Because of this, we decided that the best way to gain input, identify user needs, and prioritize features was to hold a workshop to bring the stakeholders together.
We held the 6-hour workshop split between two consecutive days. The attendees were subject matter experts from diverse departments. The first day, we introduced the project and explained its goals. Our team, which consisted of the project manager, a consultant, and myself as UX lead, then began a series of consensus-building and decision-making exercises.
Workshops allow a diverse group of stakeholders to spend time focusing on a specific problem.
After introductions, the first exercise was a questionnaire. We chose this exercise as a warm up to get our clients thinking about the problems at hand. We also used the individual ideas reported on the survey to validate ideas expressed by the group. Finally, the questionnaire would give us a baseline for past actions; the answers would tell us what, why, and how our clients had been interacting with the library of documents until this point.
UX tool: Questionnaire
One of our participants wrote this on his questionnaire. A little humor goes a long way toward keeping workshops upbeat and productive.
In the next exercise, we used intention models to organize our group’s thinking about outcomes. The intention model is not a “will do this / won’t do that” continuum. Instead, it sheds light on where priorities lie.
UX tool: Intention Model
Continuums are not either/or choices. A choice on the right does not mean the left side will be ignored. Intention models clarify the direction of design decisions.
Credit: The Understanding Group
After the intention model exercise, we continued the discussion by posting a series of slides with specific questions on screen. We asked these questions for two reasons. First, these were opened-ended questions – until this point, we’d be asking questions that needed a short, definitive answer. Second, these questions solicited ideas about change. The answers would provide frameworks on which to build new processes.
UX tool: Group Interview
The second day, we shifted from information gathering to brainstorming.
The first activity we did was to show what the group they had said the day before. After the first day, our team condensed and streamlined the data we had gathered. We put that information up on screen and asked the group to offer corrections and changes.
We did this activity for two reasons. The first was to confirm that what our team heard was correct and the next actions problem-solving actions would be valid. The second reason was to re-engage our workshop members so they could confidently contribute to the brainstorming activities.
UX tool: Research Validation
For the next activity, we used a classic UX method: card sorting. We asked our group to write on each sticky note a need or want and to generate as many sticky notes as possible.
The group was a little unsure about this activity. In general, they worked with concrete information and very structured protocols. Asking for blue sky answers was unusual for them.
After generating their sticky notes, the group put them all up on a white board. They were then asked to sort the cards into relational categories – putting like with like.
The choice to use the card sorting method allowed our participants to experience several things. The first was that writing the notes alone allowed them to contribute ideas without self-censorship in a group discussion. The second part was that they saw that good ideas can come from anywhere. The third, which was a product of the grouping exercise, was that they could see where the commonalities existed as well as where there might be holes or gaps in thinking.
UX tool: Card sorting
The final activity was to use dot voting to identify the most important ideas or categories that appeared during card sorting. We gave each group member a set of sticky dots and asked them to vote for an idea by placing one dot a sticky note. In the end, we had some very clear winners that identified product design priorities.
While card sorting in and of itself is very useful at macro scale, dot voting uses specificity to reveal priorities at a more granular scale.
UX tool: Dot voting
Group members put these sticky notes in the same category. The blue note received the most sticky dots.
In brief, the UX research heavily informed the final product. (The other major influence being the NPL outcomes.) In fact, one of the sticky notes that received the most dots (see image above) maps very highly to the information structure and content derived from the overall library.
On the whole, the research plan was successful in identifying needs and guiding our client to consensus. There were some hiccups…
On the first day one of our client’s leaders stated that he did not like the uncertainty of the information we were generating. We reassured him that our job as researchers was to be comfortable with that uncertainty because our goal was not to decide on a solution until we worked through the research process.
Lesson: It’s important to state that this research is nonlinear. The path to a solution may be unclear and sometimes it feels as though there is no progress happening.
One of the issues was social hierarchy influencing who spoke or answered questions. Some voices were louder than others.
Lesson: Find ways for participants to have a voice in a more equitable way. Ask people who have not spoken up for their insights.
There was dissension in the group because of truly diverse, disparate set of needs; it was clear that our product would only really work for some people.
Lesson: While there was little we could do in this case because of project scope and definition, we were able to ensure everyone had the opportunity to give input and feedback.
As a facilitator, I learned that I needed to be more clear about the “what” of the process. I suffered from a knowledge bias because I knew how the activity would unfold. I thought certain activities were more self-evident than they were and I caused confusion.
Lesson: Don’t assume everyone knows why and how to do an exercise.