It Takes a Village

The W3C VR Workshop team. See us in full 360.

After all this time, VR really is the Web’s Next Big Thing. A two-day workshop brought the Web’s best and brightest together to define the future of the medium.

TL;DR: The web community’s next big push on browser technology puts VR and AR front and center. The Immersive Web is going to happen, and sooner than we think. It will take vision, hard work, compromise, and lots of really smart people.

The New Normal

I’m not really sure how it happened, but within two years WebVR — the technology that connects web browsers to virtual reality hardware — has gone from a wacky idea to the new normal.

WebVR was hatched a few years back by WebGL creator Vlad Vukićević of Mozilla, collaborating with Google Chrome hotshot Brandon Jones, and publicly demoed for the first time at our San Francisco WebGL meetup in the summer of 2014. The demos were crude, and the tracking sucked, but it was a start, and we were excited by the possibilities.

In the year that followed, WebVR was greeted with blank stares by the VR hipster elite and outright derision by industry players. We heard familiar arguments: bad performance, because JavaScript; too much bureaucracy with standards groups; and anyway, why would you want to do something like that? To be fair, folks had a lot on their plate, and most people didn’t have the mental bandwidth think about something as abstract and forward-looking as WebVR.

But over the past year, something changed. Seed-stage investors started asking me about WebVR, wanting to know the lay of the land. VentureBeat sang its praises and touted the benefits. Microsoft announced support for WebVR in Edge and contributed to the spec to make it suitable for AR. And in a penny-drop moment, a team at Oculus approached me with their plan to get into the WebVR game in a big way. This led to WebVR being featured in the keynote at Oculus Connect 3, with a demo by my fledgling startup and WebVR prominently placed in the Oculus Developer Portal.

It appears that WebVR has arrived. In a previous post I went into some detail about why I think the timing is now. But that only covers the why. How we are going get there has been by no means clear, till now.

The Final Frontier

While all this WebVR goodness was brewing, a group from the World Wide Web Consortium (W3C), the people that brought you the Web, organized a Workshop on Web & Virtual Reality. The workshop, hosted by Samsung in San Jose, CA on October 19th and 20th, brought together browser makers, content creators, application developers, and technical experts to discuss enabling technologies and standards for an open VR infrastructure.

The very existence of this meeting showed that the powers-that-be in W3C understand the importance of VR as the new frontier of web development. That’s a big win, in and of itself. But the reality of the event went beyond that. The technical quality of the presentations, the concrete plans shared by product vendors, and the positive energy and spirit of collaboration showed how seriously the industry is taking this initiative. In my view, this workshop was another watershed, in a watershed month for VR that included Daydream View, PlayStation VR and Oculus Connect 3.

The W3C Workshop covered broad ground and when it could, went deep. After two days of lightning talks, panel sessions and breakouts, my ears were fairly bleeding from the information overload. I am sure the organizers will post full meeting notes soon. (See the schedule page for the detailed list of speakers and topics.) In the meantime, here are some highlights.

  • Sean White keynote. An old friend from the VRML days, now VP of Technology Strategy at Mozilla, Sean White delivered a homey keynote that hearkened back to early VR and the collaboration that built the web, setting the tone for the rest of the workshop.
  • WebVR API Update and Browser Support. There are more tweaks coming in the WebVR APIs, leading to a likely 1.2 version before broad adoption. Google, Mozilla, Samsung and Oculus shared concrete plans and expected ship dates for desktop and mobile VR browsers.
  • Lightning talks. A barrage of 5-minute lightning talks covered UI design, accessibility, 360 video formats and metadata, immersive audio, multi-user networking, and declarative VR languages and 3D file formats.
  • Breakout sessions. We split the group into sessions on various topics, including high performance VR implementations, hyperlinking in VR, and extending the browser DOM into 3D.

My fingerprints can be seen on a lot of the Workshop’s content, but I take particular pride in one development that I hadn’t even planned for. glTF, the new file format standard for 3D scenes, continues to build steam, with a groundswell of endorsement from industry partners over the last several months. glTF figured prominently in many discussions over the two days. Folks even floated the idea of glTF as a built-in format that browsers could natively read and display (analogous to JPEG images for 2D), with immediate application as 3D favicons, hyperlink transition animations, and built-in layer graphics, e.g. for splash screens and heads-up displays. Whoa. Mind blown.

Rip Van VRML

As fun a time as this was for me, it was at times surreal. A generation of brilliant technologists had locked themselves in meeting rooms to design the Metaverse, along the way rehashing ideas we explored two decades before, such as 3D scene representation, VR interface design, and shared presence. In earnestness and with great enthusiasm, the kids at this workshop were reinventing wheels left and right. But how could they know? Many of them were in middle school the first time around… if they were even born yet.

A modern day Rip Van Winkle, I had fallen asleep during a 1996 VRML meeting, and woke up twenty years later in the same room. Then I began to realize that things were different. People were holding these little computers in their hands. The Internet was fast and, apparently, connected everybody on the planet. And you could fit VR equipment in your backpack! Most of all, the people leading the discussions weren’t crazy futurists on the fringe; they were sane futurists working at mainstream companies.

It was 2016, and the real world was ready for the virtual one. While most of the problems are the same as twenty years ago, now we’re looking at them through a post-Facebook, post-Pokemon Go lens, and building on technology that is thousands of times more powerful.

A Community Effort

The W3C Workshop explored a vast landscape of technologies and standards, interface design and best practices, and tools and frameworks. While it was a great kickoff, it was just that: a kickoff. There will be much hard work going forward.

We already have a head start, and some success under our belt. WebVR is maturing and really working now, with 90FPS head tracking and support in many browsers. glTF is a ratified standard from Khronos, with steadily growing support. Much of what we discussed at the workshop simply extends existing standards like HTML Video and Web Audio. So we’re not tackling any of this from a standing start, or from an ivory tower. The output from the workshop will be brought back to working groups at W3C and Khronos, or new groups will form to tackle new pieces of the problem.

That, generally, is how the process will unfold. But it’s not just about process; it’s about people. People were the key to the success of this workshop. The organizers, Dominique Hazael-Massieux, Anssi Kostiainen, and Chris Van Wiemeersch, worked tirelessly to put on a first-class event, extremely well-run with top-notch content. The familiar names of WebGL lore — Josh Carpenter, Brandon Jones, Michael Blix and Laszlo Gombos — have been joined by new ones, like Justin Rogers and Amber Roy of Oculus, Nell Waliczek and Adalberto Foresti of Microsoft, Ada Rose of Samsung, Kevin Ngo and the A-Frame team from Mozilla, and Shannon Norrell and other energetic community-builders. There were numerous positive contributions and, given the headiness of the subject matter, the mood remained light throughout the proceedings. There was a real spirit of cooperation and hope for the future.

If we can bring a fraction of this energy to bear in the coming months, we will make great progress. The movement is growing. We have enough people on this, with big brains, pure motivations and a shared vision. And that’s good… because it takes a village to build a Metaverse.

Advertisements

Third Time’s the Charm

How VR and the Web Have Finally Converged — In My Lifetime

Twenty five years after the first consumer VR crash, virtual reality is poised to upend human-computer interaction, the Internet has disrupted every facet of life as we knew it back then, and the two are now on a collision course. The Metaverse that we’ve all been dreaming about for decades — that shared vision of everybody connected and communicating in a web of virtual reality— is upon us.

Twice before, the industry attempted to consummate this chemical wedding… and twice now, somebody ran from the altar. Well, this time, the bride and groom are pure of heart and truly ready. The confluence of cheap VR hardware, accessible 3D development, and ubiquitous networking has set the stage for an explosion of VR content, delivered over the World Wide Web.

Why has it taken so long? And why is it happening now? I’ll tell you. But first, a little history.

The Best of Intentions

The notion of combining VR and the Web is hardly new. Tim Berners-Lee put out a request for proposals way back at the first-ever World Wide Web conference in 1994. Mark Pesce and I answered the call; the result was our problem child, VRML.

VRML was designed as a universal language for authoring 3D applications on the web, the first piece of a technology stack intended to bring Neal Stephenson’s vision of the Metaverse to life. After we made 3D rendering universal, we figured we would tackle multi-player networking and then, finally, when a new generation of VR hardware was ready, we’d connect it all together — with the assumption that these other pieces were right around the corner.

VRML was built on then-state-of-the-art tech: an open, scalable 3D infrastructure allowing anyone to create and share, with the burgeoning Internet as its backbone. We created VRML out in the open, didn’t patent anything, and gave everything away in the hope of starting an explosion of 3D creation online.

VRML captured the imagination of the fledgling Web industry. Software leaders Netscape, Microsoft and Adobe hooked up with hardware titans like Silicon Graphics, Sun, Intel, Apple and IBM to build our collective 3D future. Though a few had knives under the table, most of the big guys did their best to cooperate on standards. We created killer demo showcases. The hype train, powered by SGI’s marketing machine, kicked into high gear. Startup fortunes were made.

But there was a problem: we didn’t have a market. The processing power and bandwidth required for quality 3D weren’t in the average home. Most people with PCs didn’t even have a Web browser yet. VRML was a noble experiment, conceived with the best of intentions, that ultimately came up short. Wrong place; wrong time.

Worlds in Collision

In the 2000’s, virtual worlds like Second Life promised us the Metaverse all over again via real-time, user-generated 3D social environments, running on a new generation of cheap high-performance PCs. Second Life was, for its time, a damn good experience, much better than anything ever built in VRML.

By 2007, Second Life was at the center of its own hype bubble, hitting the cover of Business Week with the promise of a new way to play and communicate, and even make money by selling each other virtual stuff. SL gave rise to well-funded copycat startups, including one that I founded. But by 2010, most of the virtual worlds companies from that period were gone. Second Life was the sole exception, having built a solid business, though not a large one by Internet standards.

So why did the category fail?

Partly, it was because of the lack of scalability inherent in such a closed system. While VRML approached the Metaverse bottoms-up, via an open infrastructure and industry cooperation, these virtual worlds systems went at it top-down, delivering highly structured and stylized experiences, via AOL-style walled garden networks. The products provided powerful authoring for users to create their own content — but each company owned its own full stack, from client to tool to server. Without an open ecosystem it is really hard to achieve Web scale, and it’s all on the shoulders of one company to deliver continual value and growth.

Second Life and its ilk may have also floundered because open-ended worlds are inherently limited in what they have to offer compared to their achievement-oriented, MMOG cousins like Warcraft and League of Legends. It takes a lot of commitment to build a Second Life, and for many, it’s apparently not worth it, because there isn’t a big reward at the end.

But mostly, I think virtual world growth stalled because it got sideswiped by something bigger. Social networks provided 80 percent of the bang for way less effort, and ran everywhere, without needing to install custom software. Why go to the trouble of buying a gaming PC, installing a fat software package, and learning how to build 3D worlds, when you can instead sign into Farmville with a click using your Facebook ID, and grow virtual soybeans on your shiny silver Macbook?

Move along; this is not the Metaverse you’re looking for.

An Idea Whose Time Has Come

Half a decade after the virtual worlds bubble burst, everything has changed. Today’s smartphones have way more 3D power than the workstations that originally ran VRML; everybody is connected on fast networks; and affordable consumer VR hardware is blowing up. If now isn’t Metaverse time, I don’t know when is.

A significant development got buried in the noise around the resurgence of consumer VR. 3D is on the Web to stay, with the advent of WebGL. WebGL makes it possible to deliver hardware-accelerated 3D in a browser, programmed in JavaScript and accessible to everyone — with no additional download. WebGL runs on all desktop computers and modern smartphones. At 3 billion seats and counting, it’s ubiquitous.

But thus far, with a few exceptions WebGL has been an optional add-on to commercial sites. In the end, the results are still rendered on a flat desktop or mobile screen — granted, with more speed and sizzle, but still part of a 2D experience. Well, with a stereoscopic VR or AR display, that’s not an option: you must render in 3D. So if you want to create a Web-based application for VR, you really have no choice but to use WebGL.

Now, if you’re reading this, I am just going to assume that I don’t need to convince you why people want to create VR for the Web. To me, the idea that there won’t be VR applications built on Web tech, based on Web content, well… that’s just absurd. It’s just as absurd as someone in 2007 predicting that smartphones wouldn’t deliver Web content, and mobile apps wouldn’t someday be mostly based on HTML. Well, in 2016 they do, and they are.

Market factors will force the industry’s hand on this. The desire for cheaper, easier ways to produce, deploy and deliver VR is there: not everyone can master a game engine, and labor through the deployment and maintenance process that comes with app packaging and app store distribution. And for consumers, the long tail of applications demands an open system without the friction of app store discovery, download and installation. The makers of VR hardware used the mobile app store model as the starting point to the get the industry kick-started, but surely this is just a transient stage on the way to a fully connected Metaverse.

The technology underpinnings are now in place. In addition to WebGL, we have WebVR, a new set of VR browser APIs in development since the ink was drying on the Facebook/Oculus acquisition. We also have glTF, the new portable 3D file format that is like JPEG, but for 3D. Add to these myriad JavaScript libraries for creating VR, and the Electron framework for building native apps in HTML5, and the sky is the limit. These pieces are the kindling for a wildfire; all it’s going to take are a few simple tools and killer apps to set it off.

Which tools and killer apps? We don’t know… and we don’t care. The Metaverse may have been imagined in fiction as the product of a singular vision, a Grand Architect of the Universe mapping out how, when, why and where people will be interacting socially in VR. But that’s not how it will actually get built. The Metaverse is going to be a messy, out-of-control affair, with multiple entry points, and a face and shape that we can’t yet imagine. What we do know is that it will be comprised of a billion plus people using VR systems connected via the Web. That’s all we know. And that is enough.

Don’t take my word for any of this. Google and Mozilla are leading the way by implementing VR-enabled browsers, but you will be hearing shortly from others that are going to spearhead the effort with market applications, enabling platforms and distribution networks. The dynamics I have described here aren’t just based on two decades of my own hit-or-miss insights; they are rooted in real market pain points, developer desires, and stated strategy from big industry players.

It’s finally here.

I guess third time really is the charm.

Virtually Anywhere

OR, how the Web will eat everything in its path – again.

Now that WebGL is truly everywhere, the close-knit, doggedly persistent and technically masterful group of folks who made it happen over the last several years can take a collective bow. From Vlad  Vukićević, WebGL’s creator, to the countless engineers and working group members on browser teams who created a great spec and world-class implementations, to Ricardo Cabello Miguel, aka Mr.doob for the amazing Three.js, to us camp followers and blogging faithful: congrats, felicidades, and kudos on a job well done!

But we can barely pause to enjoy the moment, because there’s a new game afoot: Virtual Reality. From the Oculus kickstarter to the Facebook acquisition, this surreal ride has already turned the industry on its ear. New products will be forged, new markets will appear out of nowhere, and new fortunes will surely be made.

And a new war begins.

Ultimately, hardware like the Oculus Rift will become a commodity. That’s just the way of it. In the long term, the big winners will be the applications and content that run on VR hardware. And those applications will need software.

But what software? Whose software?

Do a web search for “Oculus Rift demos.” You will find many portal sites featuring amazing experiences. All of the demos are native code applications for PC or Mac – solitary, wall-garden experiences; massive downloads; not integrated with the Internet. Earlier this month, I was privileged to speak at the first-ever Silicon Valley Virtual Reality Conference. The demos on the show floor were also all native “silo” applications. Given my purview, I naturally asked several of the startup founders  at the conference about their plans to use WebGL and other web technology. The reactions ranged from blank stares to outright truculence: it’ll never work, it’s too slow, and why would I want to do that? It’s a familiar tune I’ve been hearing for years.

I imagine we’ll hear it for a while longer. In the rush to snag millions in venture money while it’s popping fresh, developers have flocked to proprietary tools, closed systems, and native apps. It’s hard to blame people. This is the easy way, the obvious way. But we don’t do these things because they are easy; we do these things because they are hard.

The hard way, the right way, the way that will win in the long haul, is to build virtual reality on the Web. Using HTML5, WebGL and CSS3, we can create VR experiences that run virtually anywhere, instantly accessible, with no downloads. Integrated. Connected. Social. Mashable. Hackable. Shareable. You know: the Web. Yes, it is early. And we’ll need some extra support in the browser itself to make it work. But that’s coming.

Last night I presented early demos of WebGL applications running in Oculus VR using various browser extension hacks. It’s not great yet, it’s still very early, but it’s promising. I also gave a great talk – but Mozilla’s Josh Carpenter (@joshcarpenter) stole the show. Josh threw down about how in 5 years’ time we won’t be navigating flat pages any more, that the whole interface to information will be in 3D, experienced through virtual reality display technology and new interface paradigms. An inspiring few minutes (dude, you had me at “psychedelic”)!

Josh’s manifesto wasn’t just idle speculation. He’s already working on it. In fact, today Mozilla shared that they are working on building Oculus Rift support directly into the Firefox nightly builds. Carpenter and Vukićević gave a live talk on Air Mozilla that outlines their plans. Josh actually dropped the bomb at last night’s meetup, and then was joined by Brandon Jones of Google, who is also playing with doing the same in Chrome. There are no time frame commitments yet, but knowing these teams, it won’t be long. (Odds are they’ll probably have great support for the devices long before Oculus or anybody else actually ships a commercial headset.)

While these developments roll out, I imagine we are going to continue to see the bulk of VR development on closed platforms using proprietary tools, and active resistance, nay-saying and political posturing. For a while, it will be a war, resembling the fracas over HTML5 on mobile. But ultimately, the Web will win VR as it did mobile, and this, too, will pass.

The language of Metaverse will be JavaScript. The platform will be the browser. To paraphrase Vukićević : the Web is the Metaverse – just with a 2D interface. Well, it’s time for those walls to start tumbling down. It’s going to be amazing to watch.

In the meantime, native app devs: go forth and build silos. Create your mind-blowing Oculus demos. Stick them in front of investors – and have your checkbook ready to sign before their eyes uncross. Godspeed, I say!

But word to the wise: learn JavaScript and HTML5 too. You’re gonna need it.

 

Steam Engine Time

It’s been happening slowly till now, but in 2012 I think we’ll see an acceleration of new game engines being developed for WebGL. Like the old saying goes, it’s Steam Engine Time for WebGL.

As many of you known I’ve been working on my own little engine that could, Skybox, but it’s totally experimental and it’s only me (I lost my partner in crime, Don Olmstead, when he took a job at Sony). I have no idea where this project will, go if anywhere. We’ve got the best of intentions with design, but as the old saying goes…

So that’s why I was so tickled that the folks at Mozilla Labs today announced an early version of their Gladius engine. It’s designed with gaming best-practices in mind, part of a larger Mozilla effort called Paladin to push forward on gaming technologies for the open web. Huzzah! I encourage you to go check it out and play the demo games they’ve built.

I think this is going to be great, and I have a feeling this will be the first of many.