Masterpeace - VR Stroke Rehabilitation (2023)
Experience Designer & Programmer, Storyteller, Researcher, Composer, & Modeler.
The Paintstroke Productions Team: Belize Kosoko, Diego Garcia Rojas, Colin Horvath
An Immersive Virtual Reality Experience created with Unity + DevOps. Assets created in-engine, and external programs such as RoughAnimator, MuseScore, and OpenBrush.
Masterpeace wears many hats, but it is a virtual rehabilitation VR experience aimed towards helping patients who have suffered strokes of all kinds. This program can assist patients recovering long-term, to patients that just suffered a stroke and is right out of urgent care. In fact, that is the most important time of all to give this experience a try. (Keep reading below to find out why!)
Masterpeace Stills
Area 1: Plaza Paradise
Research, Purpose, and Understanding the Problem.
To create an experience that is set out to change the world, we need as a team to understand the concept of what we’re doing and why we need to do it. There are lots of people in our lives that have had strokes. I recently had a family member who is very close to me encounter such an experience, and it was a very scary time. With limited knowledge on the subject at the time, I had no idea what would come of it as a result. After talking with my family member about it, a lot of the rehabilitation is extremely rustic in our current time, and this family member mentioned they didn’t want to go back, as it was incredibly long, tiresome, and boring. The methods of stroke rehabilitation can be broken into three pieces, as described by Dr. Ross, Md:
Speech Therapy | Physical Therapy | Occupational Therapy
These methods are practiced separately with patients in order to reform to their pre-stroke selves. With today’s methods, stroke rehabilitation provides the following, as explained by Northwestern University:
10% of patients recover almost completely.
25% of patients recover with only minor impairments.
40% of patients experience moderate-to-severe impairments that require special care.
10% of patients require long-term care.
15% of patients die shortly after.
My team and I broke down these statistics and we studied what happens at therapy rehabilitation centers. And it turns out — VR does all of what is offered in centers today — and most importantly, it makes rehabilitation a beautiful and interesting experience. But, creating this experience would not have the goal of serving as an individual product, that could potentially remove jobs from therapists — in fact, this experience is far from it. This application is built from the ground up for therapists to look at statistics, and monitor how the patient is improving.
And that leads to the reason I want to do this project — the purpose of creating this in the first place. One of my biggest goals is to be able to give this to my family member, to have them try it out and tell me how effective it is, and if it would be helpful to them. They still struggle to walk but are very slowly improving and I’m hopeful this could benefit the way my family member lives life.
I have an incredibly strong feeling we as a team can revolutionize the way strokes are managed. It’s a treacherous journey to make an application like this, however. Only one application for stroke patients has been made in history, and the reasoning became very evident as we continued to research. Strokes have a big variation of symptoms — but my team and I think we’ve found the recipe.
The Marvelous Journey of Masterpeace
The Plaza, Surveys, and a Foundation of VR
————— Week 2-3
This one idea kept coming back — the idea of having a central area the player of our experience starts in. This would be called the Plaza. A place for the user to start in, and since there is a lot of stimulation going on immediately when a stroke patient enters a headset, we want there to be a relaxing, yet encouraging environment, geared primarily toward the current health situation of the patient. One idea we had was to have three portals, each of which represent a different painting. We’ve decided Starry Night, Dogs Playing Poker, and last but not least, The Bedroom. Our teammate Belize sent out a phenomenal survey, getting approximately 70 responses about which painting they would like to see as an immersive environment. And out of about 10 different paintings, Starry Night and Dogs Playing Poker had wide acceptance in the survey. During this time, I was establishing a way to view our project in the headset, along with a brand new way of using the player in Unity as a game controller, so we can apply things like gravity to the user and other effects much more easily.
Planning Out Each Experience’s Mechanics
————— Week 3-4
We discussed and finalized today how we want each painting’s experience to work. Essentially, the paints work as portals — and as you walk toward the painting, it will become a whole new world around you (with assets exclusively created with OpenBrush.) We also found the roles of each painting and how we will apply it to our mission of creating a stroke rehabilitation immersive experience:
Starry Night: A Puzzle Game with the accent of Physical Therapy (They stretch both arms to reach the puzzle pieces from far away.)
Dogs Playing Poker: A speech callback game. A dog passes you a card at a table and tells you to read it out loud. If done correctly, a dog gets a dog treat and wags his tail. This has the accent of Speech Therapy.
The Huge L (But not all the way)
————— Week 4
As we began set dressing with Unity’s Built-In render engine, our project corrupted. This was because we had an underscore in our project’s name (which was a challenge.) But, in a way it was a blessing in disguise. I took what I learned from a previous project, Shroom Slayer, and prepared to build our VR game in a new project using Unity’s Universal Render Pipeline (URP.) This is a fantastic pipeline for VR because it gives you absolutely stellar graphics, with being easily adaptable to any device’s rendering capabilities. From here, we built our tech stack around using the URP, and are continuing with only having to slightly alter our pipeline for asset creation. During this new renaissance of creation for our project, I spent a lot of time doing set dressing to our plaza area, along with a coded locomotion system for a demo we would present to Dr. Benton as a team.
First Public Demo, Open Floor, and Feedback
————— Week 5
We presented our project to Dr. Benton, with some scratch audio. (I was designing the soundtrack at this time, so we had placeholder music that portrayed the mood goal of the original piece I was working on.) Through this we got some incredible feedback, and since we presented to a large group of people, our project got into a big open discussion about our project’s scope. Some thought that we needed to hold back on our design philosophy, while others were arguing for a more narrow scope. Through this, we took all of the critiques very deeply and ingrained it into our next big build that will address a lot of the issues concerned. Most critiques are for the long run, rather than our current position at this time, as we were still perfectly on track to achieving our goals at this time. The stretch goals were still a distance away, but they were within reach.
The Very Beginning
————— Week 1 (Out of 10 Weeks Total)
In the beginning, our team had pieces of ideas, with a clear determination for making something that hasn’t been done before. This took a lot of conceptual ideation, but, most of all, we wanted to get research on certain problems in the medical industry, since we were set on doing something in the medical field with virtual reality. I was very fortunate to speak to Dr. Ross about some possible issues in the medical industry that were ongoing, and showed her a previous project I’ve worked on called Shroom Slayer VR. The immersion quickly got her attention and suggested the idea of doing speech therapy for patients with strokes and alzheimer’s. I brought what she said to the team and they loved it equally as much as I did. It was awesome.
Growing Research, Ideas, and Initial Insights
————— Week 1-3
We began tailoring our research to stroke patients. We as a team spoke with Dr. Benton on the idea of doing a VR experience for stroke patients. We kept in mind everything she said, as she has worked with VR in the past in the medical field, and understands these concepts very well. During this time, we were also trying to think of ways to improve the therapy experience through all means of stroke rehabilitation. We decided to expand our goals and make them extremely big, fitting to a tight schedule to get the best product we can out there. 3 separate experiences. One experience for speech therapy, one experience for physical therapy, and one experience for occupational therapy. From our initial ideation stage in the very beginning of our team’s origins, we took some older ideas and tailored them to our current projects’ goals, such as famous paintings. We discussed different options to do for these 3 separate experiences, and we collected survey data asking people what they would like to see in a virtual environment.
Intent with Design
————— Week 3
With the new ideas of implementing mechanics, we wanted to have purpose backed with research to define our principles we wanted to include in our experience. The three paintings, we established, would represent each type of therapy: Speech Therapy — with the intent of improving speaking using a mic. Physical Therapy — with the intent of removing temporary paralysis. Occupational Therapy — intended for the patient to do everyday tasks. We spent a good amount of time trying to figure the best way to apply those principles to each painting, for these paintings we’ve selected are vastly different from each other. We also discussed the idea of adding a train to the lobby with the intent of having the patient go through all the experiences in sequential order for those recently getting out of urgent care and need immediate brain stimulation.
The Bedroom: A morning routine game. You have a list of some things to do, and the game heavily guides you to do each one. At the end you get a sticky note at the door that says, “Great Job! :)” And when you leave the door, you enter back into Paradise Plaza.
Paradise Plaza: The place that connects the experiences together. While it is important for the patient to play the brain stimulation games, this can be an overwhelming task when one puts their mind to these games that continue to get more challenging over time. Paradise Plaza is meant to be a place for the patient to relax, and reconnect with nature. As each minigame is played, parts of the experience get added to Paradise Plaza. Paradise Plaza starts out as a not-so-paradise plaza at first, but as you play each of the minigames, elements get added to the plaza world for the patient to relax to. Starry Night adds water to the Plaza. Dogs Playing Poker adds animals (especially dogs!) to the scene, and The Bedroom adds foliage and plants to the scene. This encapsulates a goal for the patient.
Finalizing the Narrative and Story
————— Week 4-5
We discussed this as a team, and the story, without a doubt, is one of the most important things I hold near and dear about any project, as I believe it is what makes life move on, and learn, and grow. The story of Masterpeace begins like this:
“The Patient begins as a sailor on a little boat. The area is foggy around you and then you wash up on dry land. Here, you find yourself at an ancient art museum, but all the paintings are blank, and everything is crooked. In the middle of the museum, lies a mural with a paintbrush in the middle. The Patient is encouraged to pick it up, and when grabbed, the paintings come to life. The patient must complete the puzzles to bring back peace and nature to this once beautiful landscape. Solve puzzles, take breaks when needed, and rebuild the museum to its former beauty!”
Lighting, Portals, and Paint!
————— Week 6-10
There were 3 systems we dedicated our entire focus to — the game’s lighting, the portal system, and streamlining our process to enable the creation of assets to the Universal Render Pipeline with OpenBrush, the software we were dreaming of using to create native assets in our Unity experience.
In-Depth Finalized Presentation
Our finalized presentation for the scope of the initial project timeline! The length of this project however continues past the scope due to our team’s long-term commitment to making the world a better place, by providing tools for therapists to use in aid of treatments that might not be as accessible to everyone in need. We hope that with Masterpeace VR, everyone who has had a stroke has the help they need to recover from such a severe situation.