It Takes a Village

The W3C VR Workshop team. See us in full 360.

After all this time, VR really is the Web’s Next Big Thing. A two-day workshop brought the Web’s best and brightest together to define the future of the medium.

TL;DR: The web community’s next big push on browser technology puts VR and AR front and center. The Immersive Web is going to happen, and sooner than we think. It will take vision, hard work, compromise, and lots of really smart people.

The New Normal

I’m not really sure how it happened, but within two years WebVR — the technology that connects web browsers to virtual reality hardware — has gone from a wacky idea to the new normal.

WebVR was hatched a few years back by WebGL creator Vlad Vukićević of Mozilla, collaborating with Google Chrome hotshot Brandon Jones, and publicly demoed for the first time at our San Francisco WebGL meetup in the summer of 2014. The demos were crude, and the tracking sucked, but it was a start, and we were excited by the possibilities.

In the year that followed, WebVR was greeted with blank stares by the VR hipster elite and outright derision by industry players. We heard familiar arguments: bad performance, because JavaScript; too much bureaucracy with standards groups; and anyway, why would you want to do something like that? To be fair, folks had a lot on their plate, and most people didn’t have the mental bandwidth think about something as abstract and forward-looking as WebVR.

But over the past year, something changed. Seed-stage investors started asking me about WebVR, wanting to know the lay of the land. VentureBeat sang its praises and touted the benefits. Microsoft announced support for WebVR in Edge and contributed to the spec to make it suitable for AR. And in a penny-drop moment, a team at Oculus approached me with their plan to get into the WebVR game in a big way. This led to WebVR being featured in the keynote at Oculus Connect 3, with a demo by my fledgling startup and WebVR prominently placed in the Oculus Developer Portal.

It appears that WebVR has arrived. In a previous post I went into some detail about why I think the timing is now. But that only covers the why. How we are going get there has been by no means clear, till now.

The Final Frontier

While all this WebVR goodness was brewing, a group from the World Wide Web Consortium (W3C), the people that brought you the Web, organized a Workshop on Web & Virtual Reality. The workshop, hosted by Samsung in San Jose, CA on October 19th and 20th, brought together browser makers, content creators, application developers, and technical experts to discuss enabling technologies and standards for an open VR infrastructure.

The very existence of this meeting showed that the powers-that-be in W3C understand the importance of VR as the new frontier of web development. That’s a big win, in and of itself. But the reality of the event went beyond that. The technical quality of the presentations, the concrete plans shared by product vendors, and the positive energy and spirit of collaboration showed how seriously the industry is taking this initiative. In my view, this workshop was another watershed, in a watershed month for VR that included Daydream View, PlayStation VR and Oculus Connect 3.

The W3C Workshop covered broad ground and when it could, went deep. After two days of lightning talks, panel sessions and breakouts, my ears were fairly bleeding from the information overload. I am sure the organizers will post full meeting notes soon. (See the schedule page for the detailed list of speakers and topics.) In the meantime, here are some highlights.

  • Sean White keynote. An old friend from the VRML days, now VP of Technology Strategy at Mozilla, Sean White delivered a homey keynote that hearkened back to early VR and the collaboration that built the web, setting the tone for the rest of the workshop.
  • WebVR API Update and Browser Support. There are more tweaks coming in the WebVR APIs, leading to a likely 1.2 version before broad adoption. Google, Mozilla, Samsung and Oculus shared concrete plans and expected ship dates for desktop and mobile VR browsers.
  • Lightning talks. A barrage of 5-minute lightning talks covered UI design, accessibility, 360 video formats and metadata, immersive audio, multi-user networking, and declarative VR languages and 3D file formats.
  • Breakout sessions. We split the group into sessions on various topics, including high performance VR implementations, hyperlinking in VR, and extending the browser DOM into 3D.

My fingerprints can be seen on a lot of the Workshop’s content, but I take particular pride in one development that I hadn’t even planned for. glTF, the new file format standard for 3D scenes, continues to build steam, with a groundswell of endorsement from industry partners over the last several months. glTF figured prominently in many discussions over the two days. Folks even floated the idea of glTF as a built-in format that browsers could natively read and display (analogous to JPEG images for 2D), with immediate application as 3D favicons, hyperlink transition animations, and built-in layer graphics, e.g. for splash screens and heads-up displays. Whoa. Mind blown.

Rip Van VRML

As fun a time as this was for me, it was at times surreal. A generation of brilliant technologists had locked themselves in meeting rooms to design the Metaverse, along the way rehashing ideas we explored two decades before, such as 3D scene representation, VR interface design, and shared presence. In earnestness and with great enthusiasm, the kids at this workshop were reinventing wheels left and right. But how could they know? Many of them were in middle school the first time around… if they were even born yet.

A modern day Rip Van Winkle, I had fallen asleep during a 1996 VRML meeting, and woke up twenty years later in the same room. Then I began to realize that things were different. People were holding these little computers in their hands. The Internet was fast and, apparently, connected everybody on the planet. And you could fit VR equipment in your backpack! Most of all, the people leading the discussions weren’t crazy futurists on the fringe; they were sane futurists working at mainstream companies.

It was 2016, and the real world was ready for the virtual one. While most of the problems are the same as twenty years ago, now we’re looking at them through a post-Facebook, post-Pokemon Go lens, and building on technology that is thousands of times more powerful.

A Community Effort

The W3C Workshop explored a vast landscape of technologies and standards, interface design and best practices, and tools and frameworks. While it was a great kickoff, it was just that: a kickoff. There will be much hard work going forward.

We already have a head start, and some success under our belt. WebVR is maturing and really working now, with 90FPS head tracking and support in many browsers. glTF is a ratified standard from Khronos, with steadily growing support. Much of what we discussed at the workshop simply extends existing standards like HTML Video and Web Audio. So we’re not tackling any of this from a standing start, or from an ivory tower. The output from the workshop will be brought back to working groups at W3C and Khronos, or new groups will form to tackle new pieces of the problem.

That, generally, is how the process will unfold. But it’s not just about process; it’s about people. People were the key to the success of this workshop. The organizers, Dominique Hazael-Massieux, Anssi Kostiainen, and Chris Van Wiemeersch, worked tirelessly to put on a first-class event, extremely well-run with top-notch content. The familiar names of WebGL lore — Josh Carpenter, Brandon Jones, Michael Blix and Laszlo Gombos — have been joined by new ones, like Justin Rogers and Amber Roy of Oculus, Nell Waliczek and Adalberto Foresti of Microsoft, Ada Rose of Samsung, Kevin Ngo and the A-Frame team from Mozilla, and Shannon Norrell and other energetic community-builders. There were numerous positive contributions and, given the headiness of the subject matter, the mood remained light throughout the proceedings. There was a real spirit of cooperation and hope for the future.

If we can bring a fraction of this energy to bear in the coming months, we will make great progress. The movement is growing. We have enough people on this, with big brains, pure motivations and a shared vision. And that’s good… because it takes a village to build a Metaverse.

Advertisements

Third Time’s the Charm

How VR and the Web Have Finally Converged — In My Lifetime

Twenty five years after the first consumer VR crash, virtual reality is poised to upend human-computer interaction, the Internet has disrupted every facet of life as we knew it back then, and the two are now on a collision course. The Metaverse that we’ve all been dreaming about for decades — that shared vision of everybody connected and communicating in a web of virtual reality— is upon us.

Twice before, the industry attempted to consummate this chemical wedding… and twice now, somebody ran from the altar. Well, this time, the bride and groom are pure of heart and truly ready. The confluence of cheap VR hardware, accessible 3D development, and ubiquitous networking has set the stage for an explosion of VR content, delivered over the World Wide Web.

Why has it taken so long? And why is it happening now? I’ll tell you. But first, a little history.

The Best of Intentions

The notion of combining VR and the Web is hardly new. Tim Berners-Lee put out a request for proposals way back at the first-ever World Wide Web conference in 1994. Mark Pesce and I answered the call; the result was our problem child, VRML.

VRML was designed as a universal language for authoring 3D applications on the web, the first piece of a technology stack intended to bring Neal Stephenson’s vision of the Metaverse to life. After we made 3D rendering universal, we figured we would tackle multi-player networking and then, finally, when a new generation of VR hardware was ready, we’d connect it all together — with the assumption that these other pieces were right around the corner.

VRML was built on then-state-of-the-art tech: an open, scalable 3D infrastructure allowing anyone to create and share, with the burgeoning Internet as its backbone. We created VRML out in the open, didn’t patent anything, and gave everything away in the hope of starting an explosion of 3D creation online.

VRML captured the imagination of the fledgling Web industry. Software leaders Netscape, Microsoft and Adobe hooked up with hardware titans like Silicon Graphics, Sun, Intel, Apple and IBM to build our collective 3D future. Though a few had knives under the table, most of the big guys did their best to cooperate on standards. We created killer demo showcases. The hype train, powered by SGI’s marketing machine, kicked into high gear. Startup fortunes were made.

But there was a problem: we didn’t have a market. The processing power and bandwidth required for quality 3D weren’t in the average home. Most people with PCs didn’t even have a Web browser yet. VRML was a noble experiment, conceived with the best of intentions, that ultimately came up short. Wrong place; wrong time.

Worlds in Collision

In the 2000’s, virtual worlds like Second Life promised us the Metaverse all over again via real-time, user-generated 3D social environments, running on a new generation of cheap high-performance PCs. Second Life was, for its time, a damn good experience, much better than anything ever built in VRML.

By 2007, Second Life was at the center of its own hype bubble, hitting the cover of Business Week with the promise of a new way to play and communicate, and even make money by selling each other virtual stuff. SL gave rise to well-funded copycat startups, including one that I founded. But by 2010, most of the virtual worlds companies from that period were gone. Second Life was the sole exception, having built a solid business, though not a large one by Internet standards.

So why did the category fail?

Partly, it was because of the lack of scalability inherent in such a closed system. While VRML approached the Metaverse bottoms-up, via an open infrastructure and industry cooperation, these virtual worlds systems went at it top-down, delivering highly structured and stylized experiences, via AOL-style walled garden networks. The products provided powerful authoring for users to create their own content — but each company owned its own full stack, from client to tool to server. Without an open ecosystem it is really hard to achieve Web scale, and it’s all on the shoulders of one company to deliver continual value and growth.

Second Life and its ilk may have also floundered because open-ended worlds are inherently limited in what they have to offer compared to their achievement-oriented, MMOG cousins like Warcraft and League of Legends. It takes a lot of commitment to build a Second Life, and for many, it’s apparently not worth it, because there isn’t a big reward at the end.

But mostly, I think virtual world growth stalled because it got sideswiped by something bigger. Social networks provided 80 percent of the bang for way less effort, and ran everywhere, without needing to install custom software. Why go to the trouble of buying a gaming PC, installing a fat software package, and learning how to build 3D worlds, when you can instead sign into Farmville with a click using your Facebook ID, and grow virtual soybeans on your shiny silver Macbook?

Move along; this is not the Metaverse you’re looking for.

An Idea Whose Time Has Come

Half a decade after the virtual worlds bubble burst, everything has changed. Today’s smartphones have way more 3D power than the workstations that originally ran VRML; everybody is connected on fast networks; and affordable consumer VR hardware is blowing up. If now isn’t Metaverse time, I don’t know when is.

A significant development got buried in the noise around the resurgence of consumer VR. 3D is on the Web to stay, with the advent of WebGL. WebGL makes it possible to deliver hardware-accelerated 3D in a browser, programmed in JavaScript and accessible to everyone — with no additional download. WebGL runs on all desktop computers and modern smartphones. At 3 billion seats and counting, it’s ubiquitous.

But thus far, with a few exceptions WebGL has been an optional add-on to commercial sites. In the end, the results are still rendered on a flat desktop or mobile screen — granted, with more speed and sizzle, but still part of a 2D experience. Well, with a stereoscopic VR or AR display, that’s not an option: you must render in 3D. So if you want to create a Web-based application for VR, you really have no choice but to use WebGL.

Now, if you’re reading this, I am just going to assume that I don’t need to convince you why people want to create VR for the Web. To me, the idea that there won’t be VR applications built on Web tech, based on Web content, well… that’s just absurd. It’s just as absurd as someone in 2007 predicting that smartphones wouldn’t deliver Web content, and mobile apps wouldn’t someday be mostly based on HTML. Well, in 2016 they do, and they are.

Market factors will force the industry’s hand on this. The desire for cheaper, easier ways to produce, deploy and deliver VR is there: not everyone can master a game engine, and labor through the deployment and maintenance process that comes with app packaging and app store distribution. And for consumers, the long tail of applications demands an open system without the friction of app store discovery, download and installation. The makers of VR hardware used the mobile app store model as the starting point to the get the industry kick-started, but surely this is just a transient stage on the way to a fully connected Metaverse.

The technology underpinnings are now in place. In addition to WebGL, we have WebVR, a new set of VR browser APIs in development since the ink was drying on the Facebook/Oculus acquisition. We also have glTF, the new portable 3D file format that is like JPEG, but for 3D. Add to these myriad JavaScript libraries for creating VR, and the Electron framework for building native apps in HTML5, and the sky is the limit. These pieces are the kindling for a wildfire; all it’s going to take are a few simple tools and killer apps to set it off.

Which tools and killer apps? We don’t know… and we don’t care. The Metaverse may have been imagined in fiction as the product of a singular vision, a Grand Architect of the Universe mapping out how, when, why and where people will be interacting socially in VR. But that’s not how it will actually get built. The Metaverse is going to be a messy, out-of-control affair, with multiple entry points, and a face and shape that we can’t yet imagine. What we do know is that it will be comprised of a billion plus people using VR systems connected via the Web. That’s all we know. And that is enough.

Don’t take my word for any of this. Google and Mozilla are leading the way by implementing VR-enabled browsers, but you will be hearing shortly from others that are going to spearhead the effort with market applications, enabling platforms and distribution networks. The dynamics I have described here aren’t just based on two decades of my own hit-or-miss insights; they are rooted in real market pain points, developer desires, and stated strategy from big industry players.

It’s finally here.

I guess third time really is the charm.

The VR Headset Nobody is Building

 

Let’s just put this out there right now: the future of VR is mobile. Morpheus, Vive and Rift will make a big splash in the living room, but for sheer numbers we are going to see maybe 10x the number of mobile VR headsets in the next few years. I’m not here to debate the point. It’s just the way I think it’s going to play out. If you don’t agree, then nothing to see here, feel free to move on.

Still with me? Then riddle me this: why isn’t anybody building the “iPod Touch of VR?” That is, a separate, dedicated, fully contained VR device capable of running apps, games and videos, retailing for $300–500. It seems to me that this is what the world needs, and I have to wonder out loud why it doesn’t exist.

The Cardboard VR approach, where you can put your existing phone into a cheap housing, seems SO sensible. The total cost of ownership is next to nothing, as low as twenty bucks, and you don’t need to get a new phone. The cardboard box will likely be succeeded by evolutionary advances like the plastic Wearality Sky. It’s more durable, has a wider field of view and maybe most importantly, it folds nicely and fits in your pocket. Gizmos like this cost a bit more, but they’re still under a hundred bucks, and therefore could become a wildly popular phone accessory and, as advertised, be the entry point to VR for most consumers.

Problem is, with Cardboard VR, quality is all over the map. Only the highest-resolution phones provide a decent experience. And until Google and Apple unclog the awful refresh rate of the built-in accelerometer, even the best Cardboard VR will start making you motion sick after a couple of minutes. That development looks like it’s a year or so out. So for the moment, Cardboard is still Street VR: the stuff of parties, promotions and live event giveaways. And a nice stocking-stuffer.

Then there’s Gear VR. I love Gear VR. It’s the closest thing I have seen to realizing the true promise of VR, and I believe I’ve tried them all. Gear VR is not at the fidelity level of the Vive or the Rift CV, but so what? It’s lightweight and comfortable, the apps are plentiful and cheap, and the experiences are good enough to keep me in there for an hour or more. Oh and look ma: no wires.

Gear VR has shown the path to usable, mobile VR. However, it only works with two phones. These are phones that, before I wanted to play with Gear VR, I didn’t own. Of course I went out and bought them, but I don’t think most people are going to buy a different phone just to do VR… especially at the price point. I do think that Note 4 and S6 owners will find the Gear VR to be a nifty accessory, maybe even a must-have in the next few years. And perhaps prospective Samsung phone buyers will view Gear as part of a Samsung ecosystem and yet another reason to make the purchase.

What I would buy, and what nobody seems to be building, is a dedicated VR appliance: the display, headset and brain are all in one unit that’s basically a phone without the 4G. It can’t make calls, but it can do WiFi to download everything. It has sensible, ergonomic input accessories bundled with it. Under the hood, it’s probably just Android or iOS (or Windows 10 mobile, maybe? hint hint) with a VR shell. And it just works. I would lay out a cool $500 for this product. I’m sure many, many other early adopters would. Then, a few years out, legions more would take the plunge when the price drops to $300.

Note to phone manufacturers: this wouldn’t replace phones, or cannibalize sales in any way. In fact, app developers could design products that use the VR headset and the phone together, with the phone as a controller or input device. It’s sort of like the way watches and phones are starting to work together now. So this could sell more phones, in addition to selling another VR device. Oh and by the way, I don’t want my primary phone to be stuck in a VR headset. I want it to make calls, and all the other stuff it already does. So I’d rather have another device anyway. I’m sure that I’m not alone in this.

So what I’m saying is, phone people: this is a new product you can sell.

So why isn’t anybody building it?

The Domino Effect

Finally, you have broader considerations that might follow what you would call the “falling domino” principle. You have a row of dominoes set up, you knock over the first one, and what will happen to the last one is the certainty that it will go over very quickly…

— Dwight D. Eisenhower, April 7, 1954, on the rise of communism

It’s taken a while, but it looks like the final domino is about to fall. The global onslaught of WebGL was already unstoppable – once Microsoft got on board with IE 11 last year – but now it’s official, at least on the desktop. Today at WWDC, Apple announced that WebGL will turned on by default in Safari on the upcoming Mac OS X 10.10, code-named ‘Yosemite’.

Now, for the 5+% of web users who browse this way, it’s good news. I would assume that this takes desktop WebGL adoption to near 100% – blacklisted cards and ancient desktop hardware being the exceptions.

But of course, the $64B question on everybody’s minds is: what about mobile Safari? That’s what most people care about. Well, Apple didn’t say anything at WWDC, but the site HTML5 Test, a browser capability testing site, reports that WebGL is running on iOS 8.0! If you’ve got the beta installed, please go try it out and confirm this. I haven’t done yet. Also here is an independent confirmation by Jay Moretti, tweeted by AlteredQualia.

So you could have a beginning of a disintegration that would have the most profound influences...

 

The Tipping Point

2014 is the year WebGL tips. Do we need more proof than:

  1. Microsoft is on board, with WebGL in both desktop and mobile Internet Explorer.
  2. Amazon built WebGL into the Silk Browser and WebKit for Fire OS. At $229, the 7″ Kindle Fire HDX is probably the best multimedia device deal on the planet… thanks in part to WebGL.
  3. Sony built the whole PS4 user interface out of WebGL. 4.2M seats in one whack… and growing.
  4. The NORAD Tracks Santa site saw 48.8% WebGL success across all browsers & platforms for 20M visitors, an increase of 146% over 2012.
  5. And next week, we’ll be talking WebGL at one of the best-ever-attended SFHTML5 meetups. We broke the group’s record for wait list (over 500)!
First they ignore you, then they laugh at you, then they fight you – then 3D is everywhere.

From the Front Lines, February 2013

WebGL continues to gain in popularity. New fun showcases are coming out all the time, and we are starting to see experimentation from big media names like Disney with its Find Your Way to OZ movie promo, and The New York Times showing a visualization of last year’s record hang-gliding flight. Google also recently released mobile WebGL support in the latest beta of Chrome for Android, and Blackberry continues to lead with its most-conformant HTML mobile implementation that has included WebGL for over a year.

With big brands, great content and growing mobile support, 2013 is already shaping up to be a great year. Is this the year Microsoft relents, and Apple uncorks WebGL for iOS? Here’s hoping.

Graphics Considered Harmful

OK now I’m getting pissed off. This security crap just won’t go away.

Gregg Tavares, Chrome developer,  discovered that Silverlight has the same security vulnerability as WebGL. Benoit Jacob, WebGL guru at Mozilla, even filed a bug with Microsoft about it. Cool… the cat is out of the bag. But I wouldn’t hold my breath waiting for Microsoft to change its position.

Turns out that Unity, Molehill, etc. also are likely to have similar security problems. News flash: graphics hurt. But no pain, no gain. And yet, the FUD continues to fly.

Jon Peddie said it succinctly: any graphics technology is going to have these issues, and the industry needs to manage the situation as developers inevitably adopt 3D on the web. (Yes: I said inevitably. Suck on that.) Regardless, he calls the question: should we kill this thing before it grows? I know he is being rhetorical but the scary thing is, the blogosphere is easily swayed and someone needs to fight the FUD. Bully, Jon! Keep ’em flying.

The truth is, all 3D applications come with elevated security risk because they tap into lower-level hardware. “But this is on the web,” you say, the implication being that it makes the situation inherently more dangerous. Well, these days, most 3D games run on a network so don’t go crying that this is different. It’s close enough to the same that it’s the same: bits on screen powered by random code susceptible to tampering by nefarious agents. We’ve been fighting this fight for years; we know how to do it. The real deal is, consumers either want rich experiences– and by rich, I mean real 3D not bullshit 3D– or they don’t. And if they do, the market will power through any obstacles in its path.

Web 3D is inevitable. (There, I said it again.)  The history of media is the history of a rise in production value. Production value has nowhere else to go… but 3D.

Kill WebGL? You might as well kill graphics. You’re welcome to try.