Despite there being no standard definition for “the metaverse,” a raft of startups have emerged to offer specialist services for the 3D internet. One such company is Agora, which offers “real-time communication and live broadcasting features” for interactive apps. Its functionality covers a variety of platforms, but is mainly used at present for 3D web and Unity apps.

To discover how Agora fits into the metaverse trend, and what it offers from a development perspective, I spoke to Ben Weekes, the principal software engineer at Agora.

In one of the demos Weekes showed me during our Zoom call, he and four other people sat at a virtual table. A photorealistic avatar represented each participant (top half only) and the user was able to do a 360-degree circuit of the virtual room. Developers can build a feature like this into their app — for example, into a conference app or a game — using Agora’s SDKs and APIs.

“Metaverse meets real-time video”; source Ben Weekes’ LinkedIn profile.

However, it isn’t necessarily Agora’s SDKs that distinguishes it from similar developer platforms — or indeed, from developers building the same features using native web or Unity coding. What Weekes is most excited about is Agora’s global network, SD-RTN™ — which stands for “software-defined real-time network.” It’s similar to a CDN (content delivery network) from the likes of Fastly or Cloudflare.

“Low latency video is what we’re all about,” he said. “We have servers in every Tier 2 ISP around the world.” Currently, Agora does “something like 60 billion minutes a month” of online video.

Agora doesn’t go overboard promoting itself as a metaverse company — the word is used just once on its homepage. However, the company’s PR representative did pitch this interview to me as a discussion about interoperability in the metaverse, so I asked Weekes how Agora defines “metaverse” and what the company offers to developers in this domain?

He replied that a metaverse is “a 3D environment which could be rendered either with something like Unity, Unreal, or on the web.” He added that Agora has SDKs “for all three of those platforms, allowing you to put real-time voice and video into 3D worlds and games.”

One of the popular use cases for metaverse amongst Agora’s customers is streaming real-time video into a 3D world (see images above). The company is also exploring 360-degree videos streamed over the Agora network — for example having 360-degree coverage of a boxing match, so that you can change views while watching it. Weekes admitted that the latter is “a little bit futuristic at the moment, given that you typically need more like 10 megabits per second,” but he thinks the advent of 5G will make this a reality soon.

Another popular use case is spatial audio, which Weekes described as “when you’re in a group and you can talk to each other, and as you get closer it’ll get louder […] and if your headset supports it, you can get a sense of 3D if people are behind you, in front of you, etc.”

Agora also has SDKs to build Augmented Reality (AR) experiences, such as an example using Apple’s RealityKit framework for iOS. It’s a collaborative AR app “where you can build a scene together with anyone around the world, seeing live updates from others.”

As for which 3D platforms Agora’s customers are building on, according to Weekes it is mainly Unity or web at this time. There is a lot of discussion among their customers about whether to build on Unity or web, but he favors the latter.

“I’m a big advocate of the web, to be honest, but I’m aware that there are certain operations where Unity is more optimal than WebGL at the moment,” he said. “I always make the argument that Google and Meta, etc, are pushing the browser, [so] there’s no reason why it should remain behind. And I’ve spoken to quite a few of our customers that do the web first, because it’s much easier to share a link to a prototype than it is to put a Unity app in the app store and manage all the iterations around that.”

He added that the web makes it much easier for users too.

“You don’t have to install an application and then go and sync yourself up. So I could share a link to a session, and you in your Oculus [headset] would just click on it in your messages, and you’d be straight into it.”

Since the interview was supposed to be about the (as yet unresolved) question about interoperability in the emerging 3D internet, I asked Weekes for his views on this. Does he think the metaverse will be web-based, or will it evolve into a separate internet platform?

“I think much like on our iPhones today, you’ve got a mixture of web and apps — it will be the same. If you use an Oculus headset, you’ll see that there’s a browser on there, which makes it easy to click on a link and join different WebVR worlds.”

Weekes is referring to Oculus Browser, which Meta describes as “the built-in browser for Oculus Quest devices.” He also pointed out that it’s based on Google’s open source Chromium rendering engine.

“I think they [Meta] get to benefit from all the hardware acceleration work that Google does around video encoding and decoding,” said Weekes. “It certainly performs nicely with the demo apps that I’ve built, and I’ve been testing predominantly on it.”

From a developer perspective though, Weekes said that Agora is completely multiplatform. “It’s just a case of getting the SDK into your app, which is not a big job,” he said.

Although the code is different across platforms, according to Weekes “they all follow the same conventions of listening to users joining a channel, and then once you get that notification, you can subscribe and publish into the channel.”

We’re at a point in the metaverse hype cycle where we are waiting on the big companies to define their architecture going forward. Meta, which kickstarted the metaverse trend last year, has said that it will build an interoperable platform — but as yet it hasn’t released any technical details. Tim Sweeney of Epic Games, the company behind Unreal Engine, said in a recent interview that “we [the industry] need to move our proprietary technology to open standards, file formats and networking protocols so that all our systems can interoperate and we can all be participants in the metaverse.” But again, there are no firm details of how this might play out.

In the meantime, we have companies like Croquet and Decentraland building web-based metaverse platforms. On the AR front, Niantic (the company behind Pokémon Go) recently announced increased support for the web. So there is a lot of experimentation happening in the “open metaverse” space.

As for Agora, it is covering all platform bases right now — with a particular focus on Unity and web applications. You could almost call it “Interoperable-as-a-Service”. While it’s a savvy move to support all current metaverse platforms, much seems to depend on Agora’s CDN-like network — which is a very competitive field.