See: Thank you, Lindens
Bottom-line: the server-side must serve more than HTTP.
There are a few of these game servers out there, the vast majority of them proprietary. Which is fine for the companies that own them. In the scenario where the browser is the 3D viewer, those companies can easily develop the necessary JavaScript components that talk to their back-end servers via WebSockets.
All these proprietary servers, however, are a bottleneck for the massification of 3D content on the Web. Just like for the Web itself, we need open source 3D scene servers. At the very least, we need open, standard protocols, so that people can focus on developing content without having to reinvent the network connectors, and the servers, from scratch all the time. (But with open standards come open implementations, so the result is the same: we need open source servers). Yes, we need an Apache for RT 3D scenes. Hmm, where did I hear that before…?