June 23, 2020

Spatial Audio and other WWDC 2020 reveals

I miss going to the movies. It’s been 163 days since I was in a theater. 163 days…..That is the longest I’ve gone in my entire life without seeing a movie in a theater.

It’s basic but we all know a movie theater is absolutely the best place to see ANY movie.

The big dumb tent-pole flicks with explosions.

The comedies with the cuss words and foreign films with the words I have to read.

The art films that I really like and sometimes pretend to understand even though I didn’t.

And it doesn’t matter what kind of theater it is.

The dainty little 2 screen art house that sells wine.

The mega-plex that you have to put a down payment on a dump truck of popcorn and a water tower of Coke Zero (trying to cut my sugars).

And our beloved Alamo Drafthouse.

I would go any time, any day, any movie, but I really loved going on opening weekends when it’s packed with people like me.

I’m just reminiscing here.

The lights go down in a theater filled with strangers. The projector lights the screen. There’s a crowd shift of collective energy you can feel between the trailers and when the main feature begins. It’s a collective “ alright let’s go, we’re ready”. We’re together as a crowd and individually we feed off and add to the group with chuckles, gasps, laughs, groans, tears and every other appropriate (and sometimes not appropriate) response. It’s more than the sounds of people though. There is an energy of being in a crowd. On paper we’re just watching another movie on a really big TV but it’s not the same is it? You can buy a really big TV with a really impressive sound system for your home but it really isn’t the same is it?

There has got to be a poet who can describe the human spirit of gathering better than me but Dylan Thomas sadly passed away before 2 Fast 2 Furious came out.

Many of us here at Fair Worlds come from every aspect of film backgrounds. Back when we were in the office together (seems like a different life) it wasn’t too hard to get sidetracked talking about what, when, where, and with whom we’d seen a film that moved us in any direction.

For those of us in the Austin team we live in a great spot for cinema. Our office is right next to the Alamo Drafthouse office.

We have friends and family over there. It hurts to see them hurt.


It’s bleak overall. AMC has “substantial doubt” it can remain in business. They are the world’s biggest theater chain with over 11,000 screens across 15 countries.

According to the survey, 40 percent of the country’s cinemas could permanently shut down due to the financial hit they’ve incurred as a result of COVID-19, Variety reports. This would result in the loss of 5,000 venues and nearly 28,000 screens. China accounted for nearly 22 percent of the global box office last year; such a significant loss of theaters would produce a damaging financial ripple effect throughout Hollywood.

All that is scary for cinefiles and movie geeks but let’s not forget how resilient the crowd gathering can be. Theater owners have gotten panicky and many an op-ed about the death of cinema have been written over the years. Television, video cassette, DVD and streaming services have all been written up in their respective times as being the nail in the coffin for movie goers.

Yet people keep coming out and the kids still wanna have fun. Certainly there will always be a communal aspect that is difficult if not impossible to replace.

That being said there are itches that need to be scratched and a lot of us feel like we’ve been rolling in poison oak. And XR has taken a few cracks on what this could look like in the future.

I don’t think there were really that many people in the theater for Genisys

I don’t actually have enough friends with VR headsets and times that would match up to pull off a Bigscreen VR hang, although it sounds ideal. It works on the model of a theater with showtimes actual specific times. There is a lobby that looks like a theater lobby where you can hang out and I assume you spend too much on virtual Raisinettes. The theaters themselves are customizable. The only drawback is see is the content. It’s nothing new, nothing old, nothing weird, nothing classic. Just kind of oddly in the middle. As of this writing the selection feels like a United Airlines flight 3 years ago. Top Gun and old Michael Bay type pictures along with some anime. I suppose those are good choices to watch in VR for visual purposes but I would guess that 95% of the time people are spending their money in the theater they are there for something they haven’t seen before.

If your friends don’t have access to headsets there are some more low-fi options.

Every Thursday night a few of my friends have set up a makeshift movie hangout. There are a bunch of ways to do this. Netflix Party works but Netflix is more of background channel than something to appointment watch. Amazon Watch Party won’t allow you to swear in the chat. We’ve tried multiple ways to get this going and so far this is the best version of how we have set our nights up.

We all chat in Facebook messenger about what we wanna watch that week (except for the one guy who wisely isn’t on Facebook). One guy sets up a zoom link to us all in messenger except emailing it to our friend who (again wisely) isn’t on Facebook. From there we catch up on Zoom for a bit and then another fella usually sets a group stream we all log into on TwoSeven.xyz. I had no idea you could end a web address with “xyz” but I also don’t know a lot of other things too. TwoSeven is a free service that’s you watch videos together. It’s Patreon funded and free but if it’s a pay account (Netflix, Hulu, Disney) every member will have to have their own account. That limits some things. Now there is a video chat function on TwoSeven but it’s not very good. So we have the TwoSeven on one monitor, Zoom on the other.

It’s kind of unwieldy and due to the internet and the fragility of both zoom and Two Seven it seems like one or the other is always freezing up. Again, it’s better than other things we have tried. Also for some reason the load screen on TwoSeven is ALWAYS the start screen for a music video of Disturbed covering Sounds of Silence. I mean isn’t life confusing enough right now?

Makes you wonder how good The Graduate could’ve been

Not quite me and the crew hitting the theater and then grabbing a drink after but it is what it is. Without trying to replace being a physical place with a group of people what is the next best option? We are in the infancy of trying to bridge the gap but now is the time for innovation.

Or reverse innovation.

Drive In’s are also having a resurgence for the first time since the 1980’s. They were a juggernaut through the 50’s into the 70’s before urban sprawl, VHS, mandated daylight savings time and finally digital projection chipped away at them. What was once antiquated combo’d with current technology may be the perfect compromise.You can get out of the house while staying safe in a hermetically sealed pod (your car). All the ticketing and concessions can be done online. Even the audio signal can run over WiFi instead of an AM channel. Currently we have an excess of event space parking lots with no events happening and digital projection and stage rigging has made it really easy to make pop ups.

Maybe I’m just looking to save money by stuffing myself in trunk of a KIA but I would really like to watch Tenet on a giant screen with a crowd. The floorboards of my car are easily as dirty as those of a movie theater and microwave popcorn doesn’t make my stomach hurt near enough. Mostly I just want to get out of the house and feel a little sense of a the “before times.” I can’t think of anything I wouldn’t go to see right now…

Well….almost anything.


To date, the history of “Smart” or AR Glasses has had two modes — excitement and disappointment.

We have read about the promise of smart glasses for the last decade or so. One would think that these game changers would have…changed the game? So where are we with these devices? Exactly which game are we trying to change?

Image for post

It’s important at the start to delineate between a professional tool and a day to day device. We are not talking about smart glasses for industry, as those are here and are practical for a certain subset of industries and jobs. The HoloLens 2 being the current leader in our opinion, but there are more on the way for that niche. But these are multi thousand dollar devices that are not intended to be worn like normal glasses. Let’s keep this discussion to a hypothetical consumer product. One that is worn from morning to night — Consumer Smart Glasses.

Image for post

I think we can all agree that once perfected consumer smart glasses are a no-brainer. A wearable heads up display that layers on information as we go through our daily lives would fulfill all our cyberpunk dreams. Visual 3D navigation, real time translations and training are just the tip of the iceberg for what is possible. We haven’t yet dreamed up the use cases that will be “killer apps” for the future.

But we are ready for them? Hopefully they’ll look cool like Tony Stark’s spectacles instead of some 90’s Oakley’s but either way let’s go.

Tony Stark AR Glasses compared to Oakleys

It’s not like we’re demanding Neuromancer cyberpunk silver eyeballs right now. This isn’t Star Trek transporter technology. It seems like every few months we get an article like this one that lets us feel like we are behind the scenes of a big merger. Yes, indeed Google acquires smart glasses maker North but they’re not making them and not shipping them. So what’s the deal?

Between science fact and fiction it’s hard to tell who first started legitimately probing around into the smart glasses game. We can only assume that it had to do with the disappointment of finding out that Arnold’s sunglasses were not in fact producing the Terminator-vision as seen by the titular killing machine. Either way the idea has been around for awhile with the most public attempt in 2012, with Google Glass.

Image for post

Google Glass may be forgotten by the majority of the general public but it’s spectacular industry failure may be still causing clenched teeth by would-be developers. It can not be understated how quickly the words Google and Glass went from evolutionary tech leap to punchline.

In 2012 Time Magazine named Google Glass as one of the best inventions of the year. The forward fashion gadget got it’s own 12 page spread in Vogue. It was everywhere in the media. People were excited… until they actually tried it. It was plagued with bugs, had no support, an awful battery life and let’s face it: it looked dorky.

How dorky? How about, “give me your lunch money” dorky. Multiple reports of people getting assaulted for wearing Google Glass surfaced. The public was not cool with the idea of weird looking expensive monocles that are recording their every move. It became late night monologue fodder and had it’s moment on SNL a’la Fred Armisen’s character Randall Meeks.

Joking aside, there were also practical security concerns. Some of these concerns seem a bit quaint now. This is right before the ubiquity of everyone recording everything with their phones. It was 2013 and somehow the idea of a crowd pulling out their phones in unison to record anything between life’s most mundane and dramatic moments wasn’t an accepted norm yet. Hitting record on your glasses vs. holding up your phone to record hardly seems like a real difference in 2020. In 2013 however, movie theaters were worried about bootlegs and banned Google Glass thus saving a compromised pirated viewing of Divergent. Casinos were worried about Ocean’s 11’s style heists by nerd versions of George Clooney and had them banned as well.

They just made people mad. Some of it was a class issue. In San Francisco the general public saw Google Glass as a $1500 totem of how Silicon Valley was rapidly pricing them out of their city. They quickly became a punchline as evidenced with the blog White Men Wearing Google Glass. Initally it was simply a collection of photos mocking the tech monocles. The site still exists in name but has shifted to different subject matter seeing as any photographer trying to snap a pic of someone wearing Google Glass in the wild is more difficult then getting a shot of Big Foot.

Sure some other attempts at smart glasses have been made. Many. But reviews always seem to include the phrase “awkward first step” or “designed by people who’ve never worn glasses before”. Snapchat Spectacles exist but having the word “Snapchat” in the title certainly scared off a large demographic. I don’t know any consumer who has ever worn a pair despite constantly hearing about these world changers. I’m starting to get the feeling that these are like the hoverboards promised to me when Back To The Future 2 came out in 1989.

Image for post

We also would be remiss not to mention Magic Leap. They were ahead of the curve in terms of getting an AR headset to consumers. Unfortunately, their original price tag of $2,295 far exceeded consumers ability and willingness to pay for that. Earlier this year, Magic Leap laid off a large portion of their workforce and have pivoted towards the enterprise market. They cite COVID-19 as the main reason, but it should be mentioned that their low consumer sales definitely played a role.
As we mentioned before — this idea of everyday wear is really critical for adoption of AR headsets. Are you willing to be tethered to a puck that you carry around your belt buckle? How does this feel to wear for extended periods of time? Are you willing to pay extra for prescription lenses? We are seeing headset makers having to make compromises to pack a lot of sensors without making the device uncomfortable (off-balanced).

Image for post

Here is what’s going on now.

At the end of June, Google announced that it had acquired North. North being the company that made the Focals smart glasses. The Canadian based company was planning on Focals 2.0 being the next release but since the Google acquisition those are not shipping. Consumers that pre-bought a pair are getting a full refund. The Focals app is going down and once again the “you can almost touch it” future of smart glasses feels yanked away from us.

What’s the plan? It’s all speculation now but obviously the guess is a Google assistant powered pair of smart glasses is in the works. Something that could work with all the existing Android and Google devices. How far away we are from that is an unanswered question.

With equal amounts of quiet to speculate and gossip on lots of of people are looking towards Apple. Of course there have been rumors. The rumors that Apple was going to at least tease some smart glasses this summer were starting feel really credible. Alas, we didn’t get any word about Apple Glasses from WWDC 2020 like we were hoping. There was talk that the lenses would launch this year but, really, what has gone as planned this year? Rumors are now posing to 2022 as a launch date with others even speculating 2023. We do know that Tim Cook is a fan of AR and that Apple acquired Akonia Holographics in 2018 so fingers crossed this isn’t wishful thinking.

We can at least make a healthy guess without getting into tech that Apple is heavily invested in design. There is no chance they jumped into the Smart Glasses game without planning on being the antithesis style wise to the fashion disaster of Google Glass. As long as they don’t look like a giant cheese grater I feel reasonably assured I won’t hate the way they look.

Image for post

Tech wise we are in the dark. Lots of patents have been bought and leaked. A new Apple patent shows how the lenses might be auto adjustable saving buying prescription lenses. It’s called “Pupil Expansion.” It generally relates to optical reflective devices, and more specifically to expanding light beams within optical reflective devices.

It’s clear that the big boys are all invested. Facebook, Google and Apple all have 3rd parties they are working with on both the tech and the frames front. Throw Ray-Ban in the mix and take out a 2nd mortgage.

In order to get anywhere these companies have some work cut out for them. Our phones are in our pockets half the time (OK, a quarter of the time). Glasses obviously would be visible 100% of the time and the way they look, for better or worse, say a lot about the person who wears them. Fashion will factor in to it’s success in a much larger way than any other technology has had to deal with. Look at Beats by Dre and what they did for the headphone industry. Hell, many argue that the sound quality is lesser with Beats, but the fashion aspect has made them worth billions. That’s without bringing any significant new technology to the headphone game.

That’s not to say that fashion alone is going to declare the “emperor of smart spectacles.” Whoever is going to be on top is going to have to nail both fashion and function. Headphones are meant to take us away from reality whereas smart glasses need to accentuate and sharpen our surroundings. If these devices distract or impede vision they are headed to the tech junk drawer. If they make you look stupid they aren’t even going to get tried on. It’s a fine line, and a tough job figuring out how to pull off both. Marketing will be a big part. What celebrity or athlete is going to be able to actually pull off making these things look cool. My guess is either Wallace Shawn or The Rock.

Time will tell, but hopefully not too much time. We could all use a new gadget to fixate on right now. Fortunately I’ve been told that they finally figured out hover boards and mine should show up any day now.

AirPods

Leading off with what got us the most excited: Apple’s new features to their AirPods.

The big news is that the AirPods Pro are getting spatial audio. The new feature will offer 3D, surround sound style audio to replicate a movie theater experience and will constantly calibrate based on the position of your head to whatever device you’re using. Your head movement will be tracked allowing you to “look” at the sound you are hearing. This opens up a whole new ball game of ideas and innovation in developing audio for VR and AR experiences.

Also announced is a nice time saving feature of automatic switching. If you’re like me you are in a constant state of pairing, unpairing in order to switch between phone, computer and ipad. The new switching feature will recognize what device you are actively using and automatically switch over.

Tim Cook kicked off the Apple Worldwide Developers Conference 2020 keynote by addressing racism, inequality and injustice. Apple has created the Racial Equity and Justice Initiative with a commitment of 100 million dollars as well as a developer and entrepreneur program for black developers.

From there we got some sneak peaks of a number of changes coming to every platform. Lots of new features, some bigger than others. Here are the ones that made an impression

IOS 14

The key innovation seems to be an effort into grouping and organizing apps (something I could indeed use some help with).

The App Library as they are calling is at the end of your home screen pages. It has a streamlined search feature that sits on top and alphabetizes the apps.

Apps are also grouped into sub categories based on frequency of use, recently added and customizable curations

Image for post

Widgets

Widgets have always been a little sidelined with iOS but now seem to be getting some fresh attention and already there is some online side eying from android users . Widgets are now moveable onto any screen and resizable. A smartstack widget will allow you to scroll through all your widgets or set timers to appear at different times ofday based on their uses. (I know I’ll be reminding myself to meditate)

Multiple open media

Picture in picture so you can keep watching media while scrolling through other apps is now resizable and a blessing for the ADD multitasker.

Translation

Siri translation sounds innovative and helpful for communicating in different languages. It’s no babel fish in your ear but it seems super helpful in trying to communicate conversationally with someone who speaks a different language

Texting

Pinning texts in messaging will be helpful for those of us who constantly refer to a text with specific information (In my case a key code so I’m not locked out).

Group texts are getting some extra little tweaks like specific replies or those of us who get confused as to whom is talking about what from earlier in the thread.

Maps and Travel

Maps is adding new countries and a cycling feature to help you navigate cities on bicycles. The 2021 BMW 5 series will be the first car to allow the iPhone to be used as a car key to both start and unlock your car. You can share your key information with others by sending information to their phones.

Privacy

Privacy in where your data goes and who is collecting was emphasized. Maps will have a feature giving you the option of leaving approximate location as well as specific pin drops. Apps will have to ask if they can track you. 3rd party apps will be forced to record what data they are collecting on you and you will be able to see that information.

Ipads

Ipads are getting a much needed search as well some sidebars and the widget effect. The big iPad push seems to be the scribble function to be used in conjunction with the iPad pencil. Writing to text on every app.

AirPods

The AirPods big new feature is spatial audio giving an immersive audio experience. This could be great for VR .

Big Sur and Apple Silicon

Big Sur is the name of the new OS for Mac.

Not too much was unveiled with Big Sur other than many design tweaks and layout placement. Again, widgets are getting love from Apple.

Big Sur was really a smaller reveal to the fact that it was designed to work with Apple Silicon. Apple Silicon is an in-house chip designed to replace the previously used intel. Developers are working on converting to support. Microsoft Office and Adobe Creative Cloud will all be ready to work with the rollout of Apple Silicon. According to Cook the first Mac with Apple Silicon is scheduled to ship by the end of the year and expect a transition of 2 years.

RIP, Oculus Go. We’ll miss you. Sort of.

Some ideas seem so good that in the second that you hear them, you knowthat they will be “The Next Big Thing.” These are ideas you are convincedwill change the world.

And then a couple years later… they are no longer available.

We can now add the Oculus Go to that list.

On that same list of great-but-defunct ideas is the Circarama, which preceded the Oculus Go by about 65 years. Both inventions are now relics of the past, and both were bold attempts at new ways to present stories.

Last week, the team at Fair Worlds discussed the Oculus Go’s recent discontinuation and its similarities to the fate of Circarama theaters.

Before we dive into this, let’s start at the beginning.


In response to the shiny, new, three camera, film format of the Cinerama, Walt Disney, in the parlance of our times, had a “hold my beer” moment. (Fun fact: Walt actually drank Scotch Mists.)

Adding eight more cameras to Cinerama’s three, Ol’ Walt revamped the format with the premiere of his Circarama in 1955, at Disneyland.

The Circarama was mounted on top of a car and used footage from 11 16mm cameras. To view the Circarama’s first film, A Tour Of The West, audiences stood in the center of a round room and watched as the film was displayed on 11 movie screens that wrapped around the perimeter. Viewers had to constantly turn around to figure out where to focus their attention. Some had to hold onto poles in the auditorium to center their equilibrium in order to keep from getting nauseous. (Does this sound familiar?)

Eventually, the Circarama became Circle-Vision 360 and continued to show nature and travel films at Disneyland until ’97. Outside of the park however, it had no utility, and all proclamations that it was the next big thing fell flat. Do you have a Circle-Vision 360 theater in your town? Case in point.

The answer to why larger adoption never occurred mirrors some adoption failures with the Oculus Go.

Both innovations put the user in the center of a story, and surrounds them with 360 degrees of visuals — and more importantly, filmed visuals. Even the camera systems are almost identical.*

https://variety.com/2019/digital/news/google-jump-shutting-down-1203219306/

In technical terms, both are what we call 3DoF (three degrees of freedom). This means that the user has rotational camera control and can look all around them, but they cannot walk into the image or interact with it. With 6DoF, the user has complete control. This is best demonstrated with the following Gif: 3DoF is above, 6DoF is below.

SOURCE: http://www.outpostvfx.com/blog/2016/12/19/vfx-for-vr-projects-and-why-you-are-not-prepared-for-it-part-1

The Go is a 3DoF experience, and I have always referred to a 3DoF experience as you being an observer. With 6DoF, you get to be a participant. That is the thing with VR - it’s the participatory experience that makes it incredible.
— Ross Safronoff , XR Developer at Fair Worlds

Despite the happy-go-lucky marketing, Oculus Go users should be seated for safety purposes. Even as a device for just watching VR and traditional films (watching stereoscopic 3D movies is an underrated use case for VR), the Oculus Go has a risk of inducing nausea. In fact - a huge factor in the quality of the experience is determined by what one is sitting on.

Imagine sitting on a couch and trying to watch a 360 video. If that video truly utilizes all 360 degrees, it will be physically uncomfortable to turn all the way around. Even if you are in a swivel chair - which is the best option for 360 videos - you will still have micro head movements forwards and backwards. Eventually, this could cause nausea due to the disconnection with your eyes and inner ear.

The Oculus Go didn’t die because of advancements in immersive technology, it died because of stagnation in chair technology.
— Brad Parrett, Creative Director at Fair Worlds

The issue of needing a specific chair to properly enjoy a 360 movie was made apparent when we unveiled a piece that our Creative Director, Erik Horn, produced for the NYTimes a few years ago. He proudly placed a Samsung Gear VR (another R.I.P.) on his mother, who was sitting on a couch. She quickly complained that she could not see what was behind her…and took the headset off.

We learned from that lesson when we created The Monarch Effect in collaboration with the Environmental Defense Fund (EDF). We opted to use the woefully under appreciated VR180 platform from Google. Creatively, this is more akin to the largest IMAX 3D movie you have ever seen, and has some minor interactive capabilities. It currently is still on the Oculus Go storefront for free and we hope to bring it to the Quest in 2021.

Though it is being retired to the trophy case of obsolete VR Headsets (where there are a few), we have to applaud the Oculus Go. The little grey headset did a great job in setting a precedent for what consumers were going to want and need. It introduced a new audience to the concept of VR, and like a good showman, left them wanting more. While it lacked technical prowess and more immersive, full body experiences, it brought a lightweight comfort-ability and made waves as a wireless headset.

And maybe most importantly of all, it signaled the end of the Google Cardboard era, as it did not require a user to insert their phone into a frustrating “contraption.” (Though the idea of the cardboard was, and is, brilliant, the execution was maddening)

What’s interesting in the history of the Oculus Go is that it was an R&D project. A lot of women were involved in it, and the straps came from bra straps… to make it feel comfortable. [The Oculus Go] feels more comfortable (to me) than the Oculus Quest when worn for long periods of time.
— Todd Little, Lead UX Design Engineer at Fair Worlds

Moving into the future, you can see the Go’s influence on the Oculus Quest (our current headset of choice) design-wise, and you’ll surely see its positive aspects emulated in future headsets to come. Ultimately, the Oculus Go will have a large place on the technological timeline of immersive storytelling.

So — we started with standing in line at a theme park in order to be amazed (and slightly nauseous) by a futuristic 360 degree film experience, and years later were able to experience a similarly awe-inspiring (and slightly nauseating) experience in our very own homes. Similar concept, same result.

We joke around as a company, but there are going to be bumps on the road to new immersive reality. In a few years, we’ll probably be reveling in the antiquated notion that our headsets were larger than sunglasses and required controllers at all.

In our current, socially distant isolation, we still would not mind going to a theme park at some point again…sigh. Even if it has to be in yet anotheroutdated format, Cinerama.

*We realize that the Oculus Go featured more content than just 360 and 180 degree videos, but for the sake of this comparison, we focused on them as they were the predominant use case. We love(d) experiences like “Virtual Virtual Reality” from Tender Claws and the output of the Google Spotlight Stories, but that is for a different post.