Overview
Problem: OpenSAP, owner of an online learning platform, was looking for a way to increase student engagement, reduce the workload for moderation staff, and resolve usability issues.
Solution: As part of an academic project, a team of my classmates and I worked with OpenSAP to complete usability research related to their goals. We conducted interviews, a competitive evaluation, a survey, a heuristic evaluation, and usability tests to determine potential solutions. Some of our recommendations to OpenSAP were to introduce a tiered pricing system with course moderation as a premium feature, redesign the platform's navigation structure, and strengthen their brand in comparison to competitors.
Duration: Four months
Role: Researcher on a team of four
Gathering Requirements
Determining goals and challenges
During our first meeting with OpenSAP, we conducted a stakeholder interview to determine their goals for research. During this conversation, we learned that their goals included the following:
- Increase engagement for students using the platform,
- Decrease workload for already stretched-thin moderation staff,
- Uncover usability issues within the system and determine a course of action.
To better understand moderators' workload and OpenSAP-specific challenges, my team conducted four hour-long semi-structured interviews with online course moderators. From these interviews, we learned that OpenSAP employees moderate courses as a side job in addition to their full-time responsibilities. Moderators struggle to devote as much time as they would like to providing personalized feedback, despite this being one of the most fulfilling parts of the job.
Competitive Evaluation
How are others resolving the same problems?
Once we had learned more about the moderation process and associated pain points, my team completed a competitive evaluation. We compared OpenSAP’s platform to other popular systems, such as LinkedIn Learning, Coursera, Udemy, and design bootcamps to gauge the type of content offered, the target audience, cost, and level of moderation.
Analysis of competitors led us to the following findings:
- Most learning platforms cater to a defined target audience, as indicated by clear marketing messages,
- When it is offered, expert moderation usually comes with a hefty price tag.
Based on our analysis of competitors, we provided some initial recommendations to OpenSAP:
- Consider narrowing their target audience,
- Strengthen their brand to be more competitive.
OpenSAP had such a broad target audience (global learners) that marketing messages lacked direction in comparison to competitors. Additionally, the copy that users see when logging into the platform did not emphasize that all courses were moderated, which was one of the major differentiators for the platform. Most importantly, we found that competitors charge money for the level of moderation that OpenSAP provides for free. Collecting a fee for this service would provide more resources to pay moderators and allow them to focus on students who are engaged enough to pay for the service.
Survey
What are users looking for?
Related to our client’s goal of increasing user engagement, my team designed a survey to determine the types of content and level of moderator engagement that learners prefer when taking an online class. Our participant pool included 28 individuals who had participated in an online learning experience within the past two years.
Survey results:
- Video content is seen as the most beneficial feature within online learning platforms,
- Of the features that involved moderators, the ability to chat with the course instructor about questions was seen as the most beneficial to learning,
- Course moderation is beneficial to learning outcomes, but online learners do not expect expert moderation in free courses.
Mid-point recommendations:
- Shift resources toward developing high-quality video content and allocate moderator resources toward implementing chat with instructor features,
- Implement a tiered pricing structure within the OpenSAP platform that provides expert moderator feedback as a premium feature.
Heuristic Evaluation and Usability Study
Troubleshooting usability issues
Using Nielsen’s “10 Heuristics” article as a guideline, we conducted a heuristic evaluation of the platform, examining issues such as help documentation, navigation, and more. The most pressing usability issues we found were related to navigation, and we structured the tasks for our usability test based on this finding.
Finally, we conducted usability tests with 5 individuals who met the same criteria as our survey group. We analyzed these usability tests for variables such as time on task and number of mistakes.
Findings:
- Online courses require unique structures and layouts to guide students through the course,
- Tutorials can help students feel supported as they begin a course and could lead to increased engagement.
Final recommendations:
- Redesign the layout of the course, especially navigation,
- Include tools for new students to orient them to the course.
Good UX doesn't happen in a vacuum
What I learned
My team originally expected to help OpenSAP reach their goals by providing recommendations for the platform's interface and workflow. Though some of our final recommendations were focused on these things, we discovered that OpenSAP's goals would most likely be reached through rebranding and adjustments to their pricing strategy.
My team found that we were wearing multiple hats throughout this process to give strategic suggestions in addition to our design-related ones. It was helpful that my team's backgrounds varied from marketing to product management, and the need for this expertise during our research emphasized the importance of interdepartmental collaboration within a business context.