“There is a different way to represent 3D graphics than how everything else does it these days.” So begins one of the most interesting presentations of last month’s San Francisco conference week.

PocketGamer.biz sits down with voxel streaming company Atomontage in a rented office a couple of blocks from GDC. Co-founders Daniel Tabar, president, and Branislav Siles, CEO, are keen to show us how their Virtual Matter tech performs on a mobile phone.

Virtual Matter is a new approach to representing 3D graphics using vanishingly small voxels (3D pixels) instead of polygons, allowing for highly detailed and deeply interactive virtual worlds that can run on mobile devices. Atomontage is working towards a platform where complex, persistent, interactive and deformable 3D worlds can be seamlessly shared by users without the need for special hardware or software on the part of the end user.

The most recent client software was released late last year (you can see the trailer below), but the company was at GDC to talk specifically about its applications in games and to meet developers.

From Minecraft and LEGO to virtual clay

Branislav got his start in the demoscene, so he’s no stranger to working magic with graphics using only a small amount of memory. Atomontage has been developing this technology for over a decade, solving complex issues related to data compression, management, and efficient rendering. The team of 15 has raised $4.5 million from angel investors, including Resolution’s Tommy Palm and CCP’s Hilmar Pétursson.

Their plan? Move away from polygons. In essence, Virtual Matter resembles a high-resolution Minecraft, using tiny voxels instead of blocky cubes.

“Everyone else does paper-thin polygons, with troublesome texture maps on top of them,” explains Tabar, commenting on the current state of photorealistic computer graphics.

“However, there's no depth at all to this data," he concludes. "And that makes it very hard to do deep interactivity, where the world is actually malleable. Instead, think about Minecraft: everything is built out of blocks that are very understandable, so even babies can stack blocks conceptually. But what if those LEGO-like blocks were so small that it almost becomes like virtual clay? You can start having realistic-looking things, but it's still very conceptually easy for kids to understand.”

It is very hard to increase the resolution of the cubes in Minecraft because you have “an explosion of data,” as Tabar puts it. "That is the essence of Branislav's quest. And now, we are building products around these breakthroughs that allow us to have vanishingly small cubes, running on mobile devices and in a cloud-native way. We are building towards something that will delight the hundreds of millions of people who grew up with Roblox and Minecraft.”

Atomontage anticipates that a whole generation of gamers will be looking for “a more grown-up” version of those same social UGC experiences, which will require exploring and sharing heavy 3D data.

Streaming and sharing virtual experiences

Two principles are core here. “The first is that everything is a remix; in all of art history, everything from music to literature to movies, every piece of content is based on previous pieces of content. This is just the way we evolve. When a technology makes this remixing or editing easier, then we have an explosion of content in that media form or art form,” says Tabar.

And the second? “Distribution is also everything,” he continues. “Roblox is one of the things in the games industry that did not take a dip after the pandemic ended. We feel the big reason for that is because distribution is so seamless and frictionless on Roblox. A kid can make a game over the weekend and hit publish, and instantly, it's available to millions of people. Versus more traditional games: I was an indie developer for 20 years, so I know this pain first-hand. Trying to get something done is hard enough, but then to get it in front of an audience…”

The pair describe the Atomontage platform for sharing these Virtual Matter creations as a sort of YouTube for 3D experiences, in terms of workflow at least. “You have a large video file in whatever format, you upload it to YouTube, and at that point, you stop caring how big the file was, or what format it was, and nobody else cares either,” says Tabar. “It's just a video in the cloud with a link to it, and you can send it to anyone with the confidence that they can, within seconds, start viewing that video. This is the same concept but for 3D content. So if you send a link to one of our Montages, you’ve got a 3D space in the cloud that you can fill with whatever you want. And really, there are no hard limits on the amount of detail you can have in there. And it's social: if you send it to a friend, you can see them join immediately, you can see each other in there, and you can explore and talk with each other across your different platforms, whether mobile, web browser or in XR, it just works.”

To prove a point, Tabar fires up the Atomontage Client app on his phone and connects to a server over in Europe in order to wander around in a simple virtual demo experience. It takes moments.

“There’s about 20Gb of content on this server here, but it's coming in only where you need it in high resolution,” says Tabar, as we zoom in on some chainmail armour that looks extremely detailed. “You will have sub-millimetre resolution on those voxels. There's also a sarcophagus over here, which is a real scan of a real archaeological artefact. You can read the hieroglyphs, but you can also get even closer and see the imperfections in the wood, the water damage, everything about it.”

This is coming in live over the internet as a 3D stream of content. “It’s a little counter-intuitive, in that you would think that 3D would be heavier than two-dimensional video,” continues Tabar. “But it’s not the case; it's very efficient. It comes in at the resolution you need from your point of view currently, and then it's cached, and it’s rendered locally. There are many benefits to this. It's very lag-insensitive; you can be far away from the server and still have a good experience. And it's also very cost-effective – we don't need a GPU on the server side, which is a big deal. You know, Stadia was a money pit for Google for many years. They sunk several billion dollars a year into having these essentially game consoles in the data centres, serving each device that was connecting to it!”

The promise of voxel worlds

Much of this was unveiled in December, but at GDC, Daniel Tabar and Branislav Siles finally showed game developers what could be built with it.

“The future solution is very much game-oriented,” says Tabar. "That's what we've been building towards for years and years now. It just happens to have unlocked other things on the way.”

Virtual Matter has already been used as a visualisation tool by neuroscientists doing brain tissue microscopy in Alzheimer's research (“They have such detail and data, they can see the plaque formed on the neurons. But they could not visualize that or send it to somebody easily. They desperately searched and found us!”).

After projects like that, the mission now is to create something with the visual appeal of GTA but with the interactivity of Minecraft and the immediacy and social functions of Roblox. “We are here now at the stage where somebody can come to us, just like they can on Roblox, and in a self-serve way build a compelling, commercially viable game,” claims Tabar.

The demo we play together on our phones demonstrates some of these strengths. It’s not as slick as Fortnite, but it looks good, it starts instantly (we both click on a URL and we’re in), and the world is deformable – we dig down into the ground to take shelter in holes and uncover buried weapons. A neat feature is that our avatars leave coloured footprints so you can see where we’ve been.

“You will see a bunch of tracks and a bunch of bullet holes and somebody's dead body,” explains Tabar. “It becomes this very interesting record of what has happened there. And it's all persistent, so if the server is up for a week or two, you will have a very different environment in there from the beginning; it's going to be a bunch of craters and bullet holes, and it's going to look like a war zone.”

So why has nobody done this before? Voxels are not new, having emerged as a concept in the 1970s and 1980s. And plenty of other teams are exploring UGC and building social games platforms.

“It's a great question,” says Tabar, but points at ten years of research by the company to even get to this stage and all the experience Branislav had with shrinking graphics files. “The voxel magic, to the point where you can stream those cubes to a phone and make it all render – this is why we're a deep tech startup. This was 10-plus years of R&D, and more before that of Branislav’s work, evolving it to this point.

“We've been tracking every other voxel project out there. Voxels as a concept is not something we came up with, it's been around since the ‘70s. We've seen other projects come and go. Even very smart people like Ken Silverman, who made the Build engine for Duke Nukem 3D. Classic games people like John Carmack, Jon Olick, they spent years trying to make these cubes small enough, and they gave up at some point, pretty much all of them.”

The role of AI in editing levels

Atomontage plans to make Virtual Matter something every end user can play around with.

“We want to make it as easy for people who are used to creating with Roblox to transition to this,” says Tabar. “They will just have this Virtual Matter, which makes all these game mechanics possible. And also Lua scripting. We're trying to make it as easy as possible for people to have this seamless transition, essentially, and maybe Virtual Matter will make it easy to import Roblox games to some degree. And the environment is quite fun on the scripting side; the Virtual Matter itself is so fun and intuitive to edit…”

Although Atomontage doesn’t have an AI product itself, the team is considering ways that AI tools could speed up the UGC process.

“We have AI integrated in the phone app with another startup called Common Sense Machines. They’re a Cambridge startup, former DeepMind folks,” reveals Tabar. “Their claim to fame is that they can take a single 2D image and turn it into a 3D model. "So with our app, you already now have a prominent button that you can [use to] take a picture of anything, like your teacup, and then you're done – it’ll send it to Common Sense Machines, and it comes back into your Montage, and we voxelise that on the fly. You come back in a bit to a completed 3D object that you can easily edit and remix further."

The Atomontage Client is available now on the App Store and Google Play store. You can find out more about Virtual Matter at the company’s official site. UGC and AI are hot topics at PocketGamer.biz's own upcoming conferences, including the Dubai GameExpo Summit in May.