Third Time’s the Charm

How VR and the Web Have Finally Converged — In My Lifetime

Twenty five years after the first consumer VR crash, virtual reality is poised to upend human-computer interaction, the Internet has disrupted every facet of life as we knew it back then, and the two are now on a collision course. The Metaverse that we’ve all been dreaming about for decades — that shared vision of everybody connected and communicating in a web of virtual reality— is upon us.

Twice before, the industry attempted to consummate this chemical wedding… and twice now, somebody ran from the altar. Well, this time, the bride and groom are pure of heart and truly ready. The confluence of cheap VR hardware, accessible 3D development, and ubiquitous networking has set the stage for an explosion of VR content, delivered over the World Wide Web.

Why has it taken so long? And why is it happening now? I’ll tell you. But first, a little history.

The Best of Intentions

The notion of combining VR and the Web is hardly new. Tim Berners-Lee put out a request for proposals way back at the first-ever World Wide Web conference in 1994. Mark Pesce and I answered the call; the result was our problem child, VRML.

VRML was designed as a universal language for authoring 3D applications on the web, the first piece of a technology stack intended to bring Neal Stephenson’s vision of the Metaverse to life. After we made 3D rendering universal, we figured we would tackle multi-player networking and then, finally, when a new generation of VR hardware was ready, we’d connect it all together — with the assumption that these other pieces were right around the corner.

VRML was built on then-state-of-the-art tech: an open, scalable 3D infrastructure allowing anyone to create and share, with the burgeoning Internet as its backbone. We created VRML out in the open, didn’t patent anything, and gave everything away in the hope of starting an explosion of 3D creation online.

VRML captured the imagination of the fledgling Web industry. Software leaders Netscape, Microsoft and Adobe hooked up with hardware titans like Silicon Graphics, Sun, Intel, Apple and IBM to build our collective 3D future. Though a few had knives under the table, most of the big guys did their best to cooperate on standards. We created killer demo showcases. The hype train, powered by SGI’s marketing machine, kicked into high gear. Startup fortunes were made.

But there was a problem: we didn’t have a market. The processing power and bandwidth required for quality 3D weren’t in the average home. Most people with PCs didn’t even have a Web browser yet. VRML was a noble experiment, conceived with the best of intentions, that ultimately came up short. Wrong place; wrong time.

Worlds in Collision

In the 2000’s, virtual worlds like Second Life promised us the Metaverse all over again via real-time, user-generated 3D social environments, running on a new generation of cheap high-performance PCs. Second Life was, for its time, a damn good experience, much better than anything ever built in VRML.

By 2007, Second Life was at the center of its own hype bubble, hitting the cover of Business Week with the promise of a new way to play and communicate, and even make money by selling each other virtual stuff. SL gave rise to well-funded copycat startups, including one that I founded. But by 2010, most of the virtual worlds companies from that period were gone. Second Life was the sole exception, having built a solid business, though not a large one by Internet standards.

So why did the category fail?

Partly, it was because of the lack of scalability inherent in such a closed system. While VRML approached the Metaverse bottoms-up, via an open infrastructure and industry cooperation, these virtual worlds systems went at it top-down, delivering highly structured and stylized experiences, via AOL-style walled garden networks. The products provided powerful authoring for users to create their own content — but each company owned its own full stack, from client to tool to server. Without an open ecosystem it is really hard to achieve Web scale, and it’s all on the shoulders of one company to deliver continual value and growth.

Second Life and its ilk may have also floundered because open-ended worlds are inherently limited in what they have to offer compared to their achievement-oriented, MMOG cousins like Warcraft and League of Legends. It takes a lot of commitment to build a Second Life, and for many, it’s apparently not worth it, because there isn’t a big reward at the end.

But mostly, I think virtual world growth stalled because it got sideswiped by something bigger. Social networks provided 80 percent of the bang for way less effort, and ran everywhere, without needing to install custom software. Why go to the trouble of buying a gaming PC, installing a fat software package, and learning how to build 3D worlds, when you can instead sign into Farmville with a click using your Facebook ID, and grow virtual soybeans on your shiny silver Macbook?

Move along; this is not the Metaverse you’re looking for.

An Idea Whose Time Has Come

Half a decade after the virtual worlds bubble burst, everything has changed. Today’s smartphones have way more 3D power than the workstations that originally ran VRML; everybody is connected on fast networks; and affordable consumer VR hardware is blowing up. If now isn’t Metaverse time, I don’t know when is.

A significant development got buried in the noise around the resurgence of consumer VR. 3D is on the Web to stay, with the advent of WebGL. WebGL makes it possible to deliver hardware-accelerated 3D in a browser, programmed in JavaScript and accessible to everyone — with no additional download. WebGL runs on all desktop computers and modern smartphones. At 3 billion seats and counting, it’s ubiquitous.

But thus far, with a few exceptions WebGL has been an optional add-on to commercial sites. In the end, the results are still rendered on a flat desktop or mobile screen — granted, with more speed and sizzle, but still part of a 2D experience. Well, with a stereoscopic VR or AR display, that’s not an option: you must render in 3D. So if you want to create a Web-based application for VR, you really have no choice but to use WebGL.

Now, if you’re reading this, I am just going to assume that I don’t need to convince you why people want to create VR for the Web. To me, the idea that there won’t be VR applications built on Web tech, based on Web content, well… that’s just absurd. It’s just as absurd as someone in 2007 predicting that smartphones wouldn’t deliver Web content, and mobile apps wouldn’t someday be mostly based on HTML. Well, in 2016 they do, and they are.

Market factors will force the industry’s hand on this. The desire for cheaper, easier ways to produce, deploy and deliver VR is there: not everyone can master a game engine, and labor through the deployment and maintenance process that comes with app packaging and app store distribution. And for consumers, the long tail of applications demands an open system without the friction of app store discovery, download and installation. The makers of VR hardware used the mobile app store model as the starting point to the get the industry kick-started, but surely this is just a transient stage on the way to a fully connected Metaverse.

The technology underpinnings are now in place. In addition to WebGL, we have WebVR, a new set of VR browser APIs in development since the ink was drying on the Facebook/Oculus acquisition. We also have glTF, the new portable 3D file format that is like JPEG, but for 3D. Add to these myriad JavaScript libraries for creating VR, and the Electron framework for building native apps in HTML5, and the sky is the limit. These pieces are the kindling for a wildfire; all it’s going to take are a few simple tools and killer apps to set it off.

Which tools and killer apps? We don’t know… and we don’t care. The Metaverse may have been imagined in fiction as the product of a singular vision, a Grand Architect of the Universe mapping out how, when, why and where people will be interacting socially in VR. But that’s not how it will actually get built. The Metaverse is going to be a messy, out-of-control affair, with multiple entry points, and a face and shape that we can’t yet imagine. What we do know is that it will be comprised of a billion plus people using VR systems connected via the Web. That’s all we know. And that is enough.

Don’t take my word for any of this. Google and Mozilla are leading the way by implementing VR-enabled browsers, but you will be hearing shortly from others that are going to spearhead the effort with market applications, enabling platforms and distribution networks. The dynamics I have described here aren’t just based on two decades of my own hit-or-miss insights; they are rooted in real market pain points, developer desires, and stated strategy from big industry players.

It’s finally here.

I guess third time really is the charm.

Virtually Anywhere

OR, how the Web will eat everything in its path – again.

Now that WebGL is truly everywhere, the close-knit, doggedly persistent and technically masterful group of folks who made it happen over the last several years can take a collective bow. From Vlad  Vukićević, WebGL’s creator, to the countless engineers and working group members on browser teams who created a great spec and world-class implementations, to Ricardo Cabello Miguel, aka Mr.doob for the amazing Three.js, to us camp followers and blogging faithful: congrats, felicidades, and kudos on a job well done!

But we can barely pause to enjoy the moment, because there’s a new game afoot: Virtual Reality. From the Oculus kickstarter to the Facebook acquisition, this surreal ride has already turned the industry on its ear. New products will be forged, new markets will appear out of nowhere, and new fortunes will surely be made.

And a new war begins.

Ultimately, hardware like the Oculus Rift will become a commodity. That’s just the way of it. In the long term, the big winners will be the applications and content that run on VR hardware. And those applications will need software.

But what software? Whose software?

Do a web search for “Oculus Rift demos.” You will find many portal sites featuring amazing experiences. All of the demos are native code applications for PC or Mac – solitary, wall-garden experiences; massive downloads; not integrated with the Internet. Earlier this month, I was privileged to speak at the first-ever Silicon Valley Virtual Reality Conference. The demos on the show floor were also all native “silo” applications. Given my purview, I naturally asked several of the startup founders  at the conference about their plans to use WebGL and other web technology. The reactions ranged from blank stares to outright truculence: it’ll never work, it’s too slow, and why would I want to do that? It’s a familiar tune I’ve been hearing for years.

I imagine we’ll hear it for a while longer. In the rush to snag millions in venture money while it’s popping fresh, developers have flocked to proprietary tools, closed systems, and native apps. It’s hard to blame people. This is the easy way, the obvious way. But we don’t do these things because they are easy; we do these things because they are hard.

The hard way, the right way, the way that will win in the long haul, is to build virtual reality on the Web. Using HTML5, WebGL and CSS3, we can create VR experiences that run virtually anywhere, instantly accessible, with no downloads. Integrated. Connected. Social. Mashable. Hackable. Shareable. You know: the Web. Yes, it is early. And we’ll need some extra support in the browser itself to make it work. But that’s coming.

Last night I presented early demos of WebGL applications running in Oculus VR using various browser extension hacks. It’s not great yet, it’s still very early, but it’s promising. I also gave a great talk – but Mozilla’s Josh Carpenter (@joshcarpenter) stole the show. Josh threw down about how in 5 years’ time we won’t be navigating flat pages any more, that the whole interface to information will be in 3D, experienced through virtual reality display technology and new interface paradigms. An inspiring few minutes (dude, you had me at “psychedelic”)!

Josh’s manifesto wasn’t just idle speculation. He’s already working on it. In fact, today Mozilla shared that they are working on building Oculus Rift support directly into the Firefox nightly builds. Carpenter and Vukićević gave a live talk on Air Mozilla that outlines their plans. Josh actually dropped the bomb at last night’s meetup, and then was joined by Brandon Jones of Google, who is also playing with doing the same in Chrome. There are no time frame commitments yet, but knowing these teams, it won’t be long. (Odds are they’ll probably have great support for the devices long before Oculus or anybody else actually ships a commercial headset.)

While these developments roll out, I imagine we are going to continue to see the bulk of VR development on closed platforms using proprietary tools, and active resistance, nay-saying and political posturing. For a while, it will be a war, resembling the fracas over HTML5 on mobile. But ultimately, the Web will win VR as it did mobile, and this, too, will pass.

The language of Metaverse will be JavaScript. The platform will be the browser. To paraphrase Vukićević : the Web is the Metaverse – just with a 2D interface. Well, it’s time for those walls to start tumbling down. It’s going to be amazing to watch.

In the meantime, native app devs: go forth and build silos. Create your mind-blowing Oculus demos. Stick them in front of investors – and have your checkbook ready to sign before their eyes uncross. Godspeed, I say!

But word to the wise: learn JavaScript and HTML5 too. You’re gonna need it.


Alias Ugly

WebGL just got beat hard with the ugly stick. As reported by Ken Russell of the Chrome team, there was a graphics driver issue on “some Mac OS hardware” that caused a nasty bug which corrupted memory. The culprit turned out to be antialiased rendering, that great technique for smoothing out jaggy lines along edges by blending adjacent pixels. The Chrome team decided to fix the bug by turning antialiasing off on all offending machines until Apple deals with a couple of bugs in their NVIDIA drivers. This sounds safe enough, and I can’t blame them. Better safe than sorry, especially when it comes to trashing computer memory.

Except that it turns out that when citing “some Mac OS hardware” Ken meant: “The affected machines are those with NVIDIA and Intel GPUs.” Um, that’s pretty much any Macbook. Including mine. Suddenly one day some awesome WebGL demos I wrote got visibly less awesome after Chrome auto-updated. Then I checked the same demos in Firefox and it turns out they followed suit and recently pushed a similar fix. The net result is that several of my previously beautiful projects now look shite. I am seriously bummed.

I could investigate one of several techniques for doing my own antialiased rendering via writing custom GLSL shaders. But that’s some R&D and work load I hadn’t been budgeting for. What would be much better is if Apple fixes this situation ASAP. Dean Jackson from Apple is on the case (thanks for the email replies Dino!) and I hope they fix it soon! If you’re having issues like I am, maybe add your voice to the chorus. The email archive for this thread contains info on the already-filed bug reports and an address for filing new bugs with Apple.

Just a heads-up for everyone who may have wondered why their WebGL got ugly overnight… meantime I am seriously considering (shudder) buying a new Windows laptop to show off my stuff. Somebody stop me.

From the Front Lines, August 2012

A year and a half ago I presented WebGL for the first time to the crowd at Dorkbot San Francisco. Full of excitement that the browser makers finally, really shipped it, I spun the big vision and showed the most beautiful demos I could get my hands on. Back then it was Aki Rodic’s Chrysaora jellyfish forest, the  Zygote visible human simulation, and Brandon Jones’ insane port of Id Software’s RAGE. Still crowd-pleasers, all.

This year it’s different. Things have moved from buzz to reality, and the reality is good. Last week I gave the Dorkbot crew an update from the front lines. While the demos aren’t as exciting as a year ago, they are more real. They’re about visualization, mapping, 3D printing, terrain generation, new devices and just creating fun projects– in other words, the stuff of web development. The overriding theme is that WebGL is here to stay, it’s appearing in real applications, on new devices, in money-making products and books (nudge nudge as in by yours truly). These are wonderful developments… but frankly I am just relieved that some skittish product manager at Google or Apple didn’t yank WebGL support sometime in the past year. Paranoia? Perhaps. But after watching 3D appear on the web in fits and starts for fifteen years, indulge me.

Apologies for the lame HTML page graphics, it’s been a busy summer and I had to put it together on the spot.

Shades of Chrome

A new blog posting by Microsoft Security Research and Defense, with the combative title “WebGL Considered Harmful,” has reignited the WebGL security debate. Recent moves by the Khronos Group and browser makers supporting WebGL (that is, pretty much everybody but Microsoft) had quelled the noise for about a month. Now, with Redmond weighing in on the negative side of the issue, we can expect the rhetoric around security to heat up again. That much seems clear. What seems less clear is the motivation: does Microsoft have genuine security concerns, or is this a tactical smokescreen to mask a nefarious strategy? Put another way: is Microsoft up to their old API tricks? Conspiring minds want to know.

A paranoid observer might wonder if Microsoft sees WebGL as a threat to its Silverlight product– I don’t see how they couldn’t– and is just using the security issue as an excuse to stonewall WebGL. And while it may just be paranoia, indulge me for a moment, because there is an aspect of history repeating here.

Web 3D old-timers may recall Microsoft’s failed attempt to derail VRML, originally known as ActiveVRML, subsequently renamed to Chrome (no relation to Google’s browser) and again renamed to Chromeffects, back in 1998. At the time, VRML was gaining serious traction and solid vendor support. Meanwhile, a cohort of Sun graphics transplants, with the help of the DirectX team and IE teams, lobbied the Web3D Consortium to consider ActiveVRML as an alternative standard. After much debate, the ActiveVRML initiative was resoundingly defeated within the Consortium, delivering MS a shocking (if not truly surprising) blow. The net result: Microsoft picked up their ball and went home. Chromeffects was released as its own competing thing within IE, and Microsoft’s built-in VRML support (supplied by yours truly btw) was mothballed.

Ultimately, Chromeffects was no more of a commercial hit than VRML, and within a few years of its introduction it withered. But it never died: Chromeffects was eventually resurrected as– you guessed it– Silverlight. The code may be unrecognizable from the original ActiveVRML but many of the key concepts remain, such as 2D/3D media integration, authoring with built-in XML tags, and hardly any actual 3D rendering. Silverlight is a conceptual, if not genetic, descendant of ActiveVRML. Sure, this whole line of thinking might be a stretch… but I’m just sayin’.

Of course, 1998 was a very different time: the OpenGL/DirectX war was in full swing simultaneous with the IE/Netscape war, and Microsoft was in the dominant position with application developers. So while they lost the ActiveVRML battle, they won the two big wars handily, largely because of their leverage with developers. Cut to 2011, however, and we see a very different picture. IE is no longer the market share leader browser (accordingly to many of today’s stat counters) and MS is fighting a battle for developer mindshare on several fronts. The strong-arm tactics they formerly employed with developers aren’t nearly as effective as they were back then. So this conspiracy theory might come up short, unless you consider another very effective tactic from the MS playbook: sow Fear, Uncertainty and Doubt. In the VRML days, performance was the boogeyman with which competitors generated FUD. Today, security might be the hobgoblin of choice.

For what it’s worth, this time around I think we can take MS at face value about this security thing. I really don’t believe they are being that crafty. The security issue is real and should be of concern within the industry. However, the tone and title of the MS blog post, plus alarmist reporting by CNet and ZDNet on the topic don’t help. Particularly disturbing is the poll-within-the-article on ZDNet: the article is titled “Microsoft is right to label WebGL ‘harmful'”, and within the article there is a poll asking, “Do you think WebGL is ‘harmful’?” And guess what: over 50% of the respondents do. Now there’s a shocker. Framing anyone?

Despite the scary going around, I am optimistic that cooler heads will prevail. The coolest head in this debate so far, ironically, rests on the shoulders of Microsoft employee Avi Bar-Zeev. Writing in his personal blog he advocates a balanced look at the situation and argues that Microsoft ultimately needs to support WebGL because web developers will come to demand it. Avi has hit the crux of the matter: Microsoft’s success or failure rests largely on, as it always has, its application developers. My prediction is that developers aren’t going to be scared away from WebGL based on FUD, intentional or unintentional. It’s just too cool, and it’s already supported in the other browsers. So, yeah, MS will come around… it’s just a matter of time.

WebGL Dev Camp – Winning!

Ignoring our AV issues, today’s WebGL Camp has been really exciting. Great talks, great demos, great discussion.

Remi Arnaud presented a proposal for a RESTful API. This is an especially interesting area of research and discussion. Not sure about how the business will line up around this but I am sure the capabilities are required and can be used in various open framework settings.

Things feel different from the old VRML days… more mature, more sober in tone, but the pixels look 1000x better. Through the lenses of my patented Parisiometer, the future is looking pretty bright.

I’d say we’re WINNING. 🙂