a visual storyteller utilizing design, comics and illustration.
Keep in touch and get exclusive comics in my email newsletter.

Interaction Protoyping Lab Final Assignment

The brief for this project was very open-ended; we needed to incorporate kinetic typography using the word “Design” into an interactive experience of some kind.

Cardboard Dreams

Since I’ve been very excited about Google Cardboard for the past few months, I hoped to build something simple that could be viewed in that VR Context. I had never used Unity, built anything in 3D nor done any coding beyond HTML and CSS. However, I knew that Cardboard Camera allows users to take panoramic photos of their environment with native cellphone cameras.  I hoped to overlay an animation on top of a panorama that would follow the user’s eye sight but be delayed just slightly.  See the rough prototype video below.

From there I began thinking about what image I’d like to use for the background, and what typography would be relevant for that. I have always loved Mid-Century modern furnishing, and my grandmother’s living room is a perfect example of that.

With my background figured out, I began researching contemporary font faces that would feel appropriate in the space. My love for 1950’s brush lettering and typography goes back to my Dispatch from the Desert zine series.

Vector Illustration and Sprite Sheets

After doing some sketches and research, I drew a quirky font in Illustrator.

Then I pulled these vector letters into After Effects and animated the “catch up” motion that I wanted. Next. I researched how to make a sprite sheet, which is a way to turn frames of an animation into separate PNGs so they are easy to import into Unity. I researched and taught myself how to export an animation as a PNG from After Effects and then use Texture Packer to make a sprite.

Sprite sheet

 

Building in Unity

Simultaneously I began working in Unity to develop my idea.  I wrapped the panoramic photo as a texture around a sphere, and installed the Cardboard SDK. I then set a camera inside the sphere, so when clicking the Play button I saw this:

 

 

 

 

I had the reticle and the stereo vision working. My thought was that then I’d import the sprite sheet and add the animation to the scene, and write some Javascript to trigger it in response to reticle movement.

Whoops

Unfortunately, I didn’t realize that creating a VR experience means that everything you do needs to be in 3D, therefore a 2D sprite sheet won’t work in this context.  Also, I knew zero (0) Javascript, which meant I didn’t know where to start on writing the code to make the interaction work. I turned to a Design School tutor who is very experienced with Js, but she wasn’t familiar with the language necessary to code for Unity.  She pointed me towards a Lynda.com tutorial for Unity geared for Javascript developers.  I used this, and the W3 Schools section on Javascript to give myself a really basic intro.

Pivot, Part 1

Realizing now that I’d spent many, many hours down the road on a concept that was proving infeasible, I needed to pivot.  Through the tutorial video I realized I could make 3-D bubbles and use the OnMouseDown() function to make them pop when you clicked them.  With some help from my Industrial Designer boyfriend, I turned my 2D vector letters into 3D .obj files which could stand on their own in Unity.

Then I created bubble spheres to go around each letter and turned them into prefabs.

 

Once the interaction was working, I then spent another 8 or so hours trying to build and run Unity for Google Cardboard. This was the most frustrating part of the entire project because I downloaded all the components necessary (i.e., the Android SDK and the Java SDK) and put the file paths for those kits into Unity, but Unity would claim it couldn’t find them.  I tried this same process on two different computers, spent hours fruitlessly googling and reading the Unity forums and documentation, with no luck.  Clearly, I’m not an expert here but I’d say I’ve encountered a bug in the software.

Pivot, Part 2

It was time to switch directions yet again. Several of my classmates were working with Hololens, so I thought maybe I’d have more luck (and more help) with that.  Unfortunately after downloading Visual Studio and trying to build the game on my PC I was not able to build it without errors.  At this point, I was out of time on the assignment, and felt that the valuable learning part of the project had been fulfilled. I focused on making a documentation video in order to complete the assignment.

If I were able to make the project work on the Hololens, here is what it would look like.