Duration: 3-week design sprint

Tools: Sketch App, InVision, Adobe Illustrator

QUICK FACTS

↓ THE DETAILS ↓

To serve Washington Business Week (WBW), our team contracted with Unloop to provide our UX expertise and create, from the ground-up, a student-focused experience for WBW’s learning simulation previously called “BizSim” or “EDGE” to facilitate learning among students and camp advisors at WBW’s business week summer camp. Note that the product was later rebranded to “WILE” (Washington’s Innovative Learning Environment) just before launch, so the terms “EDGE” and “WILE” may be used interchangeably in this case study.

Our direct client (Unloop) possessed the software development skills but not the UX expertise to craft an engaging student experience for WBW’s summer camp program. At the time Unloop and WBW approached our team, we learned the interface had only been used by simulation facilitators, but never by the students. Our team applied UX design practices to shape the student experience for the WILE simulation.

Digital learning tools can enhance student learning — if they are designed well.

Clumsy digital learning tools don’t actually support student learning. If anything, they could be a hindrance. Clunky technological tools might not always provide proper feedback loops with the right information, or the information is there but not presented in a way that the user can easily process. In the case of the WILE simulation, both students and facilitators had issues understanding the digital learning tool because of these very challenges.

What exactly were the issues with using the software? Here’s what users had to say:

We created an interactive user interface that gives the students the information they need to confidently make decisions in the learning simulation. While student teams collaborate face-to-face in the classroom, reviewing metrics and business case decisions would be done through this interface on mobile or tablet, with one point-person on each team inputting responses.

Our Process:

(1) Research — (2) Define — (3) Design & Test — (4) Deliver & Present

GOALS

The research I conducted had the goal to explore the highlights, pain points, and behaviors with users of the current learning simulation.

METHODS

I selected semi-structured interviews as the primary method for data collection among users. In creating the 7-item question guide for interviews, there were specific topics to be covered but it also allowed for a natural trajectory in conversation to happen, so it was neither formal nor informal. In moderated phone interviews, I planned to ask each user about how long they have been a part of the learning simulation, highlights, challenges, frustrations, and recommendations for the UX design team to consider for the student experience.

USER RECRUITMENT

During our client kick-off meeting, I learned there were many layers involved to actually reaching both current and end users for research and testing. Who we were actually serving was the client’s partner WBW and their high school students, not Unloop’s main beneficiaries (inmates at Washington State prisons enrolled in web development training).

Challenges: Unloop had not anticipated the need for a list of 20 or more users for research and testing until we had our kick-off meeting. I had not forecasted time and resources to assist with recruitment. In consulting with our lead instructor, I was advised to have the clients to recruit as it would not be my main responsibility. Though they did not have a list prepared, our point person from Unloop made a concerted effort to get me in touch with the right contacts at WBW. However, the long communication chain between client to partner to users had also caused significant delays.

Decisions Points: Once Unloop had directly connected me with WBW’s Executive Director about my recruitment request, I kindly offered to the Executive Director to conduct the outreach on my own, with their permission to take the task off their plate. I was specific about recruiting adults (18+) because in the interest of time, obtaining parental consent and conducting user interviews with minors would be another barrier in the project timeline, which was already behind at that point.

IN-DEPTH INTERVIEWS

I created a 7-item question guide and interviewed 10 participants with 8 them being current users of WILE (facilitators), 1 student, and 1 stakeholder from WBW executive leadership. Eight of the participants interviewed by phone and two respondents submitted answers in writing.

In response to the question guide, each interviewee shared how long they have been involved with WILE, highlights, challenges, frustrations, and recommendations for the UX team to consider.

concepT DEVELOPMENT

As UX researcher, I sought to understand the current system, business needs, and users’ attitudes and behaviors related to the WILE simulation. Defining the simulation’s fundamental elements, potential task flows, and added features were necessary for the project to move forward. I initiated and led conceptual sketching that allowed the team and client to visualize and confirm how the simulation would look on a basic level.

We agreed to design for responsive web as the hope was that when the WILE simulation is scaled, it can work on any device (e.g., mobile, desktop, tablet, etc). Our focus was on the student experience with their dashboard and “mini-cases,” which threw curve balls at students during the simulation as they made business decisions.

SYNTHESIzing user data

To define the main problem for the design direction, I began affinity mapping, pulling key quotes and grouping them for a thematic analysis. I invited my teammates to review the themes for congruency in patterns.

This analysis uncovered these main user frustrations and pain points:

• Challenging to understand the metrics and what they mean

• Software is outdated and the input process is inefficient, causing delays in progress

• Not adapted to the students’ context (using smartphones or tablets)

• Facilitators have difficulties explaining the simulation to students due to lack of clarity

• Lack of engagement from both students and facilitators because of poor clarity

DESIGN STUDIO WORKSHOP: PROBLEM STATEMENT

Challenges: When arriving at the problem statement, our team had difficulties reaching consensus on the identification of the main user issue. I encouraged a design studio on a problem statement and hypothesis based on the latest affinity map.

Decisions Points: Through the design studio, we fused together aspects of each team member’s problem statement and hypothesis.

Using the research findings and coming to an agreement on the problem statement and solution, we set the direction for our design to create a visually engaging, interactive user interface that gives the students the information they need to confidently make decisions in the learning simulation. Our interaction designer began sketching preliminary wireframes while also incorporating feedback from the client.

MEET JAZMINE, A SUMMER CAMP STUDENT

Using this problem statement to ground the design process, I created a user persona to embody the insights we obtained about WBW’s user base.

While creating Jazmine, my goal was to define a model user who is excited to embark on the WILE simulation during the week-long WBW program. We used Jazmine to show how a user with specific needs and priorities could work through the newly designed simulation as a student with no prior business knowledge.

USER TASKS & TEST PROTOCOL

To get my team ready for user testing on low fidelity wireframes, I identified the user tasks and developed a script to be applied at each round. I also developed data collection templates for them to insert both qualitative (user feedback) and quantitative data (responses to the System Usability Survey (SUS)). I had the interaction designer and project manager carry out user testing to minimize any bias from me as the UX researcher.

LOW FIDELITY WIREFRAMES

The team’s interaction designer had conducted three rounds of user testing (3 users, then 5 and 5 respectively) and reiterated the designs for user feedback.

In taking the last round of testing, I incorporated the user responses into mockups with greater fidelity and began applying elements of the style guide.

COLOR PALETTE & TYPOGRAPHY

The client and its partners did not have any specific style guide to follow and emphasized that it was “blank slate.” However, given that WILE as a product belonged to WBW, we wanted to ensure that the proposed style guide would align with their current selections for color and typeface pairings.

Now putting on my Visual Designer role, I started by reviewing WBW’s current website to assess their main colors and observed they emphasize a saturated red and a smoky off-white:

For typography, I took into account the users’ context (logging into the simulation via mobile website and reading large chunks of text from “mini-cases”) and the product name (an acronym with many sharp corners, “EDGE” at the time). For the title, I sought a typeface for the title that was a bit softer or informal. For the body typeface, I sought out a font that was reminiscent of a school setting since the simulation would be used in educational program.

Arial Rounded MT Bold is a gentler, more informal version of its Arial counterpart. It only appears in the title of the simulation when the user logs in.

Verdana has a heavier bold weight than most other fonts that are considered screen-friendly. Even at small sizes, Verdana is a very legible font and also invites a notion of an educational setting. It appears throughout the simulation aside from the title of the simulation.

HIGH FIDELITY WIREFRAMES

Using Sketch App for Mac, I imported the latest low-fi mockups from our interaction designer and began applying the style guide elements.

With additional research, I gained a rough understanding of how business reports are presented in graphs and aimed to bring a match between what users may see in the real world (based on NNG heuristics for usability). For example, real world data visualization would include trend line graphs or candle-stick graphs:

USABILITY METRICS

As we relied on the System Usability Survey (SUS) scores, I crunched the numbers available from the first round of low-fi prototype to the single round of user testing we did on the hi-fi mockups. These are the average scores at each respective phase (3 users initially and 3 again in the latest):

Challenges: Although we made a 17-point difference, our team readily recognized that we scored fairly low even in the latest iteration. One disclaimer is that the users we tested on were not completely representative of the end users; the closest we had to complete representation was going to Seattle Central College and testing with 18 and 19 year olds who recently graduated high school. Had we tested users who would likely enroll in WBW or have enrolled, I would predict that the SUS average scores may look different.

Decision Points: In moving forward with the project, recruiting end-users representing the target population of students would be more valuable in aligning the product’s efficacy with WBW students. There is also the possibility of incorporating other metrics such as the the amount of time it takes to complete the user tasks.

The web development team at Unloop was our primary audience as we had been regularly updating them in the design process. In centering Jazmine’s experience in the story, we were able to illustrate to our client how she would engage with the simulation. Below is a user journey that our interaction designer created based off our generative and evaluative research insights:

Our latest version of the hi-fi prototype demo was met with enthusiasm and positive reception. In rounding out this project, we handed to the developers at Unloop the slide deck we presented as well as the hi-fi wireframes in a Sketch file per their request. They had shared these files with WBW on the design team’s behalf. Based on Unloop’s reports back to our design team, WBW was extremely pleased with the latest prototype.

Watch a video demo of the EDGE Learning Simulation here:

As reported by our client WBW, since having designed the WILE digital learning experience in fall 2019, it has withstood the COVID-19 pandemic. Of the 23 states that offered a business week camp in the previous summer of 2019, Washington was the only state that successfully delivered a business week virtual camp in summer 2020, reaching 265 students across the US, Poland, and the Republic of Georgia.

As written in WBW’s 2020 annual report, the students of that year were faced with an incredibly challenging period. High school seniors lost their last season playing sports, prom, graduation ceremonies, and so on. Freshmen were faced with an extremely difficult transition into a new world that is crucial in their development. All students lost a vital connection, not only to their peers, but to their teachers and mentors. [The program] received many thank you letters from students indicating the impact the virtual summer program was for them emotionally and academically. …If this year has taught us anything, it’s that the next generation of leaders is impressive, adaptable, and incredibly resilient.”

LESSONS

About the Process

• Tech can’t replace everything in the classroom. In the process of interviewing users, fewer than half emphasized that they wanted to avoid digitizing the simulation and wanted students to have more face-to-face interaction with each other.

  What I’d do differently: Get a better idea of the context and “ecosystem” that the user is in. I would concept sketch how things would work outside the simulation, such as multi-player access; a protocol for a single point-person designated on each team for data entry; and team discussions documented with pen and paper first, then entering responses with the learning simulation’s interface.

  
  
  

• Make sure to have enough representative end-users in the evaluative research. There were some flaws in this phase, given our challenges with user recruitment for testing and design iterations. It took some time for both the design team and its clients/partners to get on the same page for expectations, especially with the demographic of the end-user group (14-18 years old). As the project was significantly delayed due to the research phase getting off to a slow start, we did not want to compromise the timeline further. While testing on adult users moved us along, the disadvantage is that the users we tested on had very little context of what the simulation meant even with explanation.

  What I’d do differently: Clarify expectations with the client and its partners about the user recruitment process. Knowing that the end-users would most likely be minors, the design team would need support from the client and its partners to obtain written consent for evaluation and testing with their parents/legal guardians.

  
  
  

• Quality assurance on usability testing process is key. We ran into challenges gathering quantitative data during usability testing. It is likely that implementation of the test script and data collection templates was not consistent.

  What I’d do differently: Conduct a pilot demo for the team with the first one or two users. This way, the team will have a better understanding about what they need to cover and collect during testing, and it remains consistent throughout each round of usability testing.

About Me as a UX Researcher

• A functional, well-coordinated process is reflected in the quality of product. I learned a lot about collective ownership for a product’s outcome, not just having a sense of your own piece (i.e. research) in the process while other aspects of the development life cycle may be falling to the wayside. It is important for me to not minimize issues that could blow up later in the cycle; speak up!

  Even at the risk over over-stepping bounds and egos, I have learned to be vocal with the intent to reinforce congruency and synergy within a team, so we can address conflicts right away, and what emerges from development is a well thought-out product.

• Continue learning about user research approaches, including practicing my own awareness as a researcher. As a UX researcher who has spent more time practicing qualitative research, I want to continue applying reflexive thinking to combat my preconceptions, assumptions, and biases about how research could be done.

Reflexive thinking adds a level of integrity and rigor to the research, removing any room for bias and preconception. As a UX professional with an inclusive design mindset, I aim to build in extra time before and throughout the research process to check in on my own thinking and how I approach interactions. This way, I can continue identifying any assumptions and limitations of the research we conducted, seeing myself as an “instrument” of the inquiry process, too.

ABOUT THE LONG-TERM IMPACTS

As I write this in 2024, I am truly overjoyed to know that the evolution of BizSim into WILE has withstood the test of time, particularly as the camp industry crumbled in the face of the COVID-19 pandemic. As a designer, I am excited to learn that it has gone through additional micro-iterations based on student feedback. I hope that this kind of sustainable design that can reach hundreds of students globally is a testament to my chops as both a design researcher and a visual designer who strives to actualize universal design principles in all she does.

>>Next project: Piroshky Piroshky eCommerce on Desktop Web