Unity: Where to start if you come from filmmaking and are new to interactive development and storytelling. Part 1

December 14, 2020

Todd: I’m a UX design engineer here at Fair Worlds, and I’ve been using Unity for five years. I started in 2015. It was largely because I was hearing so much about the potential of VR, but it seemed daunting right from the get go. I was going to these indie game dev groups, because I was interested in game development in general and started to hear about Unity. I looked at the interface and there were so many different windows and it was really confusing. Right now it’s much more approachable than it was back even five or six years ago.

I went to school in user experience design, at the University of Washington and in that process, got interested in accessibility work. VR had just jumped off the map when the Oculus rift and  the Vive came out. So I got to do some capstone projects that were really exciting. One was with the (at that time top secret) Valve Index controllers.

They called them knuckles controllers back then, but we did a lot of civic experiments of what it felt like to be able to hold the controllers. Instead of tapping the button to throw something we tried different things to simulate, say, what it feels like to grab a fireball? To see if there was any sort of immersion breaking when that happens. After that,I got involved with Fair Worlds pretty soon after that. It’s been a great journey, just taking what I learned in those years in university, with Unity and exploring that, and then kind of now bringing it into sort of agency life.

Erik: You said Unity was daunting when you jumped in. What was your entry point? Were there a series of tutorials you looked at? Were you going to the meetups and the jams? What was that experience like?

Todd: Do you guys remember the game Myst? (everyone enthusiastically agrees) My point of entry was HyperCard if you remember that. It was the 1990s so my point of entry was you that have stacks and then you’ve got cards, right? So every screen would be a card.

The syntax, the coding language, was very English.The classic Hello World example is like put quote “Hello World” unquote into a field. So instead of you could actually read it like English and kind of understand it. Whereas if you look at C-sharp or something like that, it’s much harder for people to get up to speed.

Gates: Well, just a little background about myself. I’m a filmmaker and Erik and I had a company together a while back called Super Alright. It was similar to FairWorlds although it wasn’t interactive. It was more, video and some motion graphics. I stayed in film whereas Eric went more in the graphics, UX, and interactive field. I kind of moved more into filmmaking, but I’ve always been passionate about games. Chris is my brother. He’s very passionate about games. And then between he and Erik tickling my brain about what’s happening in that world and then the pandemic shutting down the film industry, my interest in it became kind of two-fold. The idea of maybe trying to pivot to game design as visual storytelling. Tthe medium is exciting and the possibilities got me thinking about that world. I‘ve been learning more about virtual cinema and the power of unity for filmmaking itself. It looks like engines are going to be replacing the Mayas and the Cinema 4DS of the world, or at least augmenting or supplementing them. It’s tricky for me because I don’t have an interactive background. I know a little bit of 3D, not much. I guess my whole thing is that I feel a little too behind the curve and a little too old to dive in and become a hardcore dev, but I want to be able to speak the language and I want to know what the possibilities are.

Chris: Quick aside for you there though, Gates. If you’re feeling like you’re a little too old school, if it’s any consolation, I know the creative director for Overwatch, that super huge Blizzard title. His very first game directing was Overwatch and he entered game design as an industry when he was like 33.

Mike: I’m still 1000 years older than that so it really doesn’t help (everyone laughs knowingly)

Todd: The industry is so new that everybody’s kind of getting up to speed on the new tools. I know a few friends who are 3D artists and they don’t use the traditional tools. They don’t use Blender or Maya. Instead they instead use Tilt Brush or use VR only tools.

Gates: You can model like that with TiltBrush?

Todd: Yeah, you can model, you can do Google blocks. You can use Oculus Medium. There’s four or five different 3D modeling tools that are VR specific. By doing that I think that it’s fun to play with your own limitations. I know in game design there’s a guy who created a game that was basically just using cubes. It’s using whatever kind of storytelling, and starting off with just cubes or spheres and going from there.

Everything

Sean: You’re thinking of David O’Reilly. He’s a digital artist, but he made a few games that are really interesting. One’s called Everything. You can start on a macro level, a cellular level and you go through to the universe, or you can start as a universe and you can decrease down to the cellular level. It's a different take on perspective.

I’m Sean. I have a small, litigation support firm, that does legal support, but we use a lot of A.I. and technology to crunch the data, gather public data sets and make recommendations to attorneys based on the predictions of judges, rulings, or the likely likelihood that they’re going to rule on one motion versus another, depending on what. I had a small startup called, wrap God.AI with a company called Second Brain here in town. And this was two or three years ago. We built a rap engine, basically that would cumulate to the style of various rappers. So we trained the machine’s learning engine on all the different hip hop artists and lyrics that we could find and we went to Techstars incubator. Then we went to Dubai to do a big tech conference there, which was cool. And then it didn’t cap and all the monies were dried up and everyone else kind of caught up to us. Now there’s tons of artificial intelligence writing tools that are coming out, getting the market that we were sort of pursuing two years ago, and GPT3.

I’ve been looking at non-linear storytelling where you can just immerse yourself. For instance there is a meditation app called TRIPP.

It claims that it’s Artificial Intelligence. Every iteration is different. So every day there’s different sorts of visuals and there’s a different process. The unique thing about it is that it listens to your voice and your breath. When you exhale there’s little sparks and embers that emerge from where your mouth would be and when you inhale they breathe in. The graphic element of breathing in and out really sort of paces and sets your breathing, which is cool. So the intersection between the biological processes and VR or AI is pretty interesting because it paces you through a meditation.

Todd: There is an experience called SoundSelf. It’s a meditative experience where you lay down on your back and put on a VR headset and then as you breathe in and then breathe out, it moves you forward in the experience. In this experience also supposed to be humming, so you hum. And the visuals will change. So it’s very kind of psychedelic. but kind of meditative experience.

Gates: Is there some sort of advice you would give for someone starting out? Is it better to burn through a bunch of tutorials or is just learning code better? I would be curious to hear what you guys have to say about starting out and common pitfalls and the things you wish people had told you that you didn’t know?

Todd: What I would suggest is looking at specific tutorials and failing fast. Maybe starting off with small chunks. Instead of going for the super ambitious idea, I would suggest going for something, that’s going be playable within a week. Kind of work up to that bigger idea, but start off small. The nice thing about Unity is they’ve got some built in tutorials.


What’s cool about the tutorials is that they have these little modules. So whatever interests you, you can add on to that particular one. For example they have a Mario Kart one that you can play around with or they have a first person shooter version, and then whatever interests you can kind of explore.. You can start off with the full project and all of its files. And then the nice thing is you can sort of sum things up, be running immediately and you can kind of kit bash. You can bring in your own 3d assets. You can cut up what it’s doing.

Gates: You can actually use that base as a springboard for your own projects instead of starting from scratch?

Todd: Totally. That way, you don’t have to start from a blank canvas, trying to figure out, say, that there is an empty Skybox. If you don’t know what to do you can start with a template and then kind of learn from how they’re doing it. I think that’s probably the best approach, because if you start completely from fresh, there’s so much onboarding that you have to learn.

It doesn't end here. Stay tuned for part two of our roundtable with filmmakers and interactive developers!