Choreo

Choreo is a platform that allows a diverse range of users to produce sounds and visual effects from body movements to share with the community

This is a group project for a UX design challenge in summer 2020.

User Research

Knowing that we wanted to create something that combines AR/VR and music, the first thing we wanted to understand is the AR/VR gaming experience for hearing and sight-impaired users, their feelings about music and art, and their habits when playing games and enjoying music.

User Interviews

Initially, I interviewed 4 users who are aural/sight-impaired and mostly within Gen-Z and Millennial demographics.

Here are some questions that I asked:

• Have you played any AR/VR games? If so, how do you enjoy them? What aspects do you like/dislike about them?

• Tell me about your experience with music?

• How do you feel about creating music or other forms of art?

I got some really insightful responses:

• "What excites me in VR and AR games are really the sound effects, touch feedback, and other details that make things 'real'."

• "People assume that I wouldn't enjoy music because I can't hear well but I love putting my hands over the speaker and feeling the bass and beats in songs."

• "I have always wanted to play around with music but I just don't know enough to actually make something."

• “I don’t want a game to be accessible just for the sake of accessibility; I want it to be enjoyable and catered to every type of user, not just people like me.”

Since we are designing for users who won’t be able to enjoy VR/AR visually or aurally, we need to conceptualize an AR experience that’s more than just visual overlays. This means that we will have to figure out what elements and technology we need to include to make our product a multi-layered experience.

User Persona

After knowing more about the people that we are designing the project for, I made a user persona that reflects the problems shared by our stakeholders.

The problem

Based on our user persona and all the information we’ve collected, we went back to our problem statement and finalized our hypothesis.

Competitive Analysis

To see what’s missing from current apps, we compared Choreo (our product) to the two most popular video-sharing platforms on the market: TikTok and Instagram Reels.

User Flow

To structuralize our solution, we created a user flow that identifies the user actions, touchpoints, emotions, and pain points that users may face at each different step. We also addressed the pain points with specific design plans.

Feature Prioritization

We held a brainstorming meeting and planned out what features need to be included in our solution using the MoSCoW method. We dropped the “won’t have” category since we all agreed that future product features are not our current focus.

Final Solution

Library: a creative vault for creative souls

Recording Page: just have fun! We’ll figure out the hard stuff

Choreo allows users to create sounds and visual effects with body movements. For sounds, each movement will trigger a note and ultimately create sounds that will never go out of tune. To quickly explain, each song will start with a note (randomly or determined by the user), and the algorithm will figure out an appropriate note that goes after it*. If you already picked a track to work with, the algorithm will create harmonizing notes accordingly. For visual mode, Choreo will generate effects that the user has picked previously following the user’s body movements. Users can also control the instrument and settings by just pointing at it on the camera.

*This feature is inspired by the instruments in Animal Crossing: New Horizons, videos showing how it works.

Haptic feedback & spatial audio: an immersive experience

Profile Page: Share your love with the community

Last but not least, you can check out other users’ visuals, videos, and tracks and post your own contents and share them with the community on Choreo. Don’t worry, if you just want a good time without all that social media pressure, you can still play without recording anything!

Next Steps & Reflections

I am really proud of what I and my group members were able to put out in a span of 3 weeks while all being in different locations and time zones. If I come back to this project in the future, I would love to spend more time designing the UIs and other visual elements and do some quantitative research on how accessibility has been implemented in mobile apps and games.

With this project, I realized that for users with disabilities, having an app or a game specifically designed for them should not mean that this game is only playable for people with disabilities; it sounds like common sense but most of the time they just want to be treated equally. As a UX designer, this means that I need to place accessibility at the center of my design choices and build products and features that will utilize the “limitations” posed on accessible designs.