The Proposal and Solution

This project explores how Achieve, a primarily higher-ed courseware platform, can help students be aware of tasks they need to complete and information that is meaningful to them to increase engagement and ultimately succeed in a course.

The solution consisted of a medium-fidelity prototype with new and customizable communication features within Achieve, such as the ability to create reminders and view notifications.

Takeaways

Understand and communicate expectations.

Deciphering how to best communicate through project documentation and narrowing down scope and was essential, especially with the limited time of the internship. The project sheet served as a reference for those who were not familiar with the project but also was a guide for myself to look back on and update throughout the process.

Get feedback early on and make time for iterating.

I'm grateful that I brought my concepts to design critiques on a weekly basis, but I feel like I benefitted from the more thoughtful feedback from my one-on-one conversations. As the end was looming, I realized I needed more time for iteration- not that my designs were lacking something particular, but there was more to explore.

Get to know what others are up to. 

This was by far my favorite part of the experience, despite my introversion. I set up countless intro chats with a variety of people- from project managers and directors on other teams to design contractors to the inspiring CEO. It was fascinating and valuable learning about each employee's diverse personal and professional backgrounds, as well as their current and past projects. Each conversation led to me to someone else. I found past similar projects, which was helpful for my own, and intriguing studies and initiatives, which lead me to my current role on the Learning Science & Research team.

Communications Audit

Prior to thinking about designing for specific use cases and user tasks, I worked with another intern under the guidance of a UX design manager to delve head-first into Achieve and conduct a brief audit of the communication capabilities within the platform.

Here's what we got up to that first week:

• Exploring communication within Achieve and conducting an audit based on current features.

• Conducting generative research and finding relevant past research.

• Pondering the various ways instructors, students, and Macmillan Learning can communicate with each other.

• Focusing in on a specific relationship and recommending solutions based on recurring concerns.

We experienced the courseware tool as instructors and students. The types of assignments and tools differed based on the discipline, for example, Writing Tools in Achieve for English, so it was important to also have access to classes from different disciplines. 

Some access was a limited and I realized later we probably should have had proper product training, which I eventually got around to on my own.

Initial Student Interviews

In the second week of this audit, we chatted with five current college students to not only better understand their experiences with courseware and communication, but also visualize their complex user ecosystem.

Our primary goal was to understand the value of communication for students in general to inform how to focus and enhance our communication solutions to better meet the needs of students.

Here' are some questions we asked: 

• Why or why not do you reach out to your professor(s)?

• What do you like and dislike about the current method your professor(s) communicate with you?

• What specific tools have you found helpful for communicating with peers? What about collaboration?

Despite the availability bias and a nebulous research goal in mind, we found several key insights from these interviews, which validated some past research like student journey maps and competitive analyses we stumbled upon. 

Findings & Recommendations

After some empathy mapping, too-broad HMWs, and messy storyboarding, we came to a couple of findings and recommendations:

Students don't reach out because they feel as though they may be behind or they are expected to know what to do.  →  A low-stakes chat option between the instructor and student as well as between students to support of a sense of belonging.

Personalized and actionable feedback is important, especially if they aren't learning in person or are in a larger class.  →  Make feedback more accessible for both instructors to give and students to receive. Reporting & Insights can be more actionable (allowing instructors share insights, filter students based on skillsets, and provide supplemental resources to certain groups of students).

Having everything in one places helps students stay on track.  →  Explore LMS integration and how we can embed a notification framework to create a seamless, and ultimately helpful, experience.

A consistent need for instructor announcements, assignment updates, accessing grades and feedback, and study resources throughout a semester.  →  A functioning mobile app.

Admittedly, some of these are a bit broad and unrealistic to take on, but this was meant to start a conversation with other designers and get thinking about more specific concepts. 

The Project Sheet

Even before receiving a proposal related to communication, we learned about project documentation. Once we received our proposals, we began to build our respective Project Sheets to help narrow down and define the scope of our main projects.  

Here is a summary of what the Project Sheet covered:

• The Objective

  • Explore how Achieve can help students be aware of tasks they need to complete and information that is meaningful to them to increase engagement and ultimately succeed in a course.

  • This problem area would include exploring what types of assignment notifications are critical, how students want to receive the notifications, and how students (and instructors) want to manage these notifications and reminders. 

  • Assignment Reminders & Notifications are in the top 5 features requested year-to-date when looking at ProductBoard insights.

• Users & Use Cases

  • students

    • specifically students not integrated within an LMS must log in to Achieve to view upcoming assignments and their calendar

    • students integrated into an LMS (76%) receive in platform notifications about upcoming assignments from Achieve  

      • How can we ensure that these students aren’t notified twice? Does this have to be something they have to go manually switch on/off?

  • instructors

    • specifically instructors that “set and forget”

    • instructors that are integrated and not

• Research Gaps

  • time constraints

  • feasibility

  • encapsulating personal preferences

• Assumptions & Areas to Avoid

  • not everyone wants more communication (another place where they receive notifications)

  • communications should be helpful and actionable

• Milestones & Deliverables

  • research

    • secondary research- past projects, related research

    • stakeholder interviews

    • 30 minute student interviews (moderated) OR student surveys

    • research synthesis

  • and eventually, design-wise:

    • wireframes

    • prototype

    • usability testing plan and synthesis

    • tech and accessibility validation 

    • a project framework-type deck

• Measuring Outcomes 

  • after initial research:

    • a better idea of the scope

    • a specific use case we should design for

    • actionable research insights to inform design decisions 

  • eventually:

• reduction of late assignments and extension requests

• adoption of notification features

• interaction (such as manual customization or dismissal) of notifications 

• Appendix

  • similiar initiative & projects

  • Productboard insights

  • competitor analysis 

  • Monday timeline

• Lessons Learned & Next Steps

  • formulating a reserch plan for a survey with a co-designing section

  • narrowing the scope to system > student, considering both integrated and not

  • focusing on customizable reminders for students, while considering how to add a framework that does not impede the current experience

Research Planning

A survey where students explained their reasoning and with a co-design section provided an indirect way to collect attitudinal as well as some behavioral data.

With some fine-tuning of the goals and questions from a research mentor and other senior researchers and validation from my product manager, we launched a survey with 20 students in UserTesting. Here are the primary reasearch questions:

1. What are the current pain points, challenges, or frustrations experienced by users in managing their learning progress and staying on track within the online courseware platform? Without an LMS?

2. How would users like to be aware of tasks that they need to complete? How do they like or not like the way their upcoming tasks are communicated? Within Achieve? Within an LMS?

3. What information, in terms of communications from system to student, is meaningful to users? Within Achieve?

4. How do users prefer to receive notifications and reminders (e.g., email, in-app push notifications, SMS), and what content and timing would be most helpful to them in terms of improving engagement and task completion rates?

Research Synthesis

Once I did some more note taking, data visualization, and affinity mapping, I found that students want notifications to ultimately be helpful, subtle, and all in one place.

Here's what was most surprising:

• There was not a whole lot of want for text messages.

• No one complained about too many notifications, they just needed to be in the right place (and at the right time).

These things could have been clarified or asked more in depth:

• Making the differentiation between text and mobile notifications clearer. 

• Asking about the current usage and customization of LMS notifications, for those who use it.

After consolidating themes, reviewing good examples of notifications mentioned by students, and utilizing the co-design results of the survey I was more than ready to finally jump into ideating.

Initial Concepts

These concepts were quickly jotted down following the secondary research as I was still planning the survey. 

In the first concept, I imagined creating a companion-type app that reminded students of what they needed to do via a chatbot. It would also have addressed some of the recommendations we made in the audit, but the scope was a bit much for the time we had.

Synthesizing Concepts

Before any wireframing or lo-fis, I did a brain dump of what notifications and a dropdown could entail. I also explored iconography. I denoted the concepts I was leaning towards in yellow. They included:

• a bell icon (surprise, surprise) with a badge

• icons for each type of notification for scannability

• settings accessible through the dropdown

• basic information hierarchy 

I moved back and forth between sketching and putting ideas together on paper and in Whimsical. 

I thought about the tone and copy from what I learned from my research, but also considered brand guidelines some of Macmillan Learning's values. I was still thinking about an AI-helper functionality, but didn't get around to exploring it. 

For the settings page, I went straight into visualizing it digitally. It helped to keep those previous designs from Sapling accessible as I moved forward with mine. 

I redesigned the Achieve settings without thinking too much about the specifics, as they were currently on a separate page from Achieve (and narrow). 

I primarily experimented with how the notification frequency and alerts/reminders would be set. 

Usability Testing

Upon the third iteration or so, I felt confident enough to test out the design. We tested with 10 students through user testing and gave them 2 tasks:

1. Using the prototype, show the steps you would take to customize your notification and reminder settings. Edit the settings so that notifications are sent to your email and LMS. Then, add reminders to notify you when an upcoming assignment, quiz, or exam is due as well as when a due date has been changed.

   • 100% reported task completion

   • users stated the overall the task was easy (10%)  or very easy (90%)

2. After setting up your reminders, you come back to the course homepage in a couple weeks to view recent notifications. Find where your notifications would be and view past notifications.  

   • 100% reported task completion

   • users stated the overall the task was easy (30%)  or very easy (70%)

Here's what users liked:

• an intuitive and customizable design

• highlights for high priority items

• time estimations

• Once you know where it is, it's easy.

There were also some valuable recommendations:

• making the dropdown scrollable

• denoting past/read notifications clearly

• a filter/sort option

• the bell icon could be more noticeable

• save changes can be automatic

It was important to watch each user interact with the prototype; Some didn't actually complete the task, despite them answering that they successfully completed the task.

Here's what users liked:

• an intuitive flow and customizable options

• highlights for high priority items

• time estimations

• Once you know where it is, it's easy.

There were also some valuable recommendations:

• making the dropdown scrollable, like it initially was

• denoting past/read notifications clearly

• a filter/sort option to highlight priority tasks and announcements

• the bell icon could be more noticeable as some didn't notice the subtle animation

• save changes can be automatic as some failed to scroll back up to complete the task

It was important to watch each user interact with the prototype and talk through their thinking; Some didn't actually complete the task, despite them answering that they successfully completed the task.

Iterating the Notification Dropdown

Here's how the notification dropdown changed after feedback:

• • added a button to view past notifications

  • did away with the scrollbar and the unnecessary label for settings

  • differentiated each notification with subtle coloring

  • settled on a mix of buttons and links, depending on the action 

  • system updates were removed, as it made more sense that they appeared as toasts or where in the bottom left of the homepage where the current help icon is

After usability testing, I considered a larger view of the notifications dropdown that could be expanded and minimized. However, this felt like too much of a disruption and almost a separate page, which I had to remind myself was the least preferred among students in the survey.

Iterating the Settings

Here's how the notification settings changed after feedback:

• made in-line editing (of the email) more in line with standard practices

• adding a quick way to turn off all notifications for external delivery/reset settings

• more detailed, consistent, and clear copy

• automatic saving, rather than having to scroll back up to save settings

• using toggles for delivery methods and checkboxes for the types of notifications

• creating collapsable areas with customization directly underneath each notification type

The Final Prototype

After just 10 days of iterating, I found a stopping place.

Ship it.

Tech Validation

We got the chance to show our prototypes to several engineers and have a discussion of what creating it would realistically entail and require. Here's what we discussed:

• It's not impossible (nice!).

• Big rock, not boulder- The design/ask would need to broken up into chunks, such as starting with a phase which allowed for just customizing email notifications. 

• 4 areas (central servicing, IAM, gradebook, and courseware) would need to work together, which might prove challenging.

Accessibility Validation

My mentor designer mentioned Accessibility Office Hours early on. I attended a couple out of curiosity and eventually brought my own prototype for feedback. Here's what I learned:

• It's best to avoid temporary banners or toasts, especially if they're necessary to the student's experience.

• Be sure to have adequate labels for each icon, section, toggle, checkbox, and other assets.

• I may need to consider how to better denote high priority items besides a yellow outline.

Future Design Considerations

• Rethink and iterate upon the designs to increase clarity and decrease cognitive load, especially as users first get into Achieve.

• Make it obvious that users are not changing their actual email address, just the one where they'll be receiving notifications.

• Sketch what would reminders look like as emails, texts, and to the LMS.

• Brainstorm an accessible visual confirmation that a reminder has been set ( such as a microanimation or visible confirmation message).

Areas for Exploration

• Consider and conduct additional research with more diverse users and for specific use cases, such as neurodivergent students, specifically those with ADHD.

• Understand the instructors role and control with notifications.

• Dig into what is actually possible in terms of LMS integration.

Other Takeaways & Final Thoughts

• The Proposal - Was this idea groundbreaking or particularly aesthetics and creative? Nah. Was it intriguing to explore student (and instructor) ecosystems and come up with potential new features? Definitely.

• Research - I admittedly spent too long on research synthesis (below), even though I was ready to start sketching. I most definitely could have utilized some kind of AI tool to synthesize and analyze student comments and survey results. 

• Presenting - I was nervous presenting this case study to stakeholders, mentors, and other senior management because I thought it wouldn't be captivating enough. Taking a full week to plan and practice presenting was worth it.

• I ultimately (just barely) met my goals of wanting to at least test my prototype. I was also traveling around the UK and Italy during the project, so there was some brainstorming done in the air; It was definitely an eventful summer.

Nice. You made it to the end!

If you have any questions or would like to see anything more in depth, such as the project sheet, the research plan and results, or any design work, please don't hesitate to reach out!