Improve Uru Cost/Performance Using UDP Network Architecture

[Ned]

Ok guys / gals / and uru people....
This is a long post...make a coffee first.

Point of this topic: Comparison to other similar type model.
Main point: Can URU really be made open source and affordably LIVE?

Firstly there is no need to determine the URU open source project against SL and.....

When Google, Microsoft, Yahoo!, Amazon, et al have their new datacentres for "cloud computing" up

Secondly forget these new datacenters too.

These datacenters of which you speak are tcp specific. For mmo gaming they are slower than Linux running on a P2 with 128 ram. No apologies given. MMO LIVE gaming requires very careful UDP package management. Of Cloud Computing... I have been there, done that and it wasn't cheap, nor was it successful. SL started a long time back...their engine is of that day, and they afford it with their present user count and income. Hardly relative to the task at hand.

Ok Server Facts.....

This model has been thoroughly tested and is still running today....
10 mg guaranteed output per user with streamed udp packets
built-in client udp manager (most important part)-(the colonels secret recipe).

at full throttle a gamer on a gamer's machine laughs at how lovely a 10mg connection in a 3d world plays.
worst scenario a p3 in an office running on a single 100mb cable shared with 30 other machines.
this office avatar walks comfortably alongside the gamer avatar with the only indicative loss being to the depth of field before the avatars.
an almost zero lag achieved.

with this scenarion I can achieve 30 users per server at a cost of $1 per user per month.
+$2 to add voice servers per 20 users per month. a voice server has no lag impact on the 3d world performance.
I know this as i do this everyday.
I am also running a full 3d mmo.

this is the system that I have developed and this is the system that I will give to the URU Open Source team free of charge for the initial stages.
I cant of course give it away free for ever lol
but some arrangements regarding that can be discussed later.

So forget the SL numbers for now and disregard hype from cloud serving companies...I have the solutions in place already for you. What the others do is their thing...URU is something more than that.

Take it easy...finish your coffee and then when the code comes...it will all happen with ease... for we are the puzzle makers

Point of this topic: Comparison to other similar type model.
Main point: Can URU really be made open source and affordably LIVE?
Answer is: Massively YES. Because the people of URU want it to be.

[Mac_Fife]

These datacenters of which you speak are tcp specific. For mmo gaming they are slower than Linux running on a P2 with 128 ram. No apologies given. MMO LIVE gaming requires very careful UDP package management.

Someone like a'moaca' will probably be able to comment on this more definitively, but I think that the current URU networking is entirely TCP, so while your assertion may point the way to a "better" implementation, the down-side is that it requires a total re-write of the client server interface.

[Nalates]

No offense Ned, but I don't know if I'm reading a fan's rave or sales hype.

I do find the claims bang against what I know newer games like RealXtrend and Open Sim are doing. However, they are not necessarily telling all and Open Sim and Blue Mars make some claims that are way out there… 1,000 users in a node/region/age/server and no lag… I’ll have to see it.

Unless I missed it, there is no reference to a product name so I can't research your post or know what you refer to. I did dig through the info I can find and read, I guess, your blog, reference to a company, etc. I see some other posts and all relate to the same topic. I can’t tell, of course, if you are posting on other Uru forums, but I don’t think so. But, there does appear to be a real product (NiHao ?) and your offer does sound generous (but with a string).

I also see the mention that it cannot be free forever. I have no problem with that. But I wonder about someone that supposedly understands the community considering offering such a proposal. If Mac_Fife is right and the game needed major rewrite… it would make the community dependent. Since it would not be free forever, that would eliminate many of those that want to operate shards from their homes. So, this would seem to be something that would require a major code branch or the elimination of small home based shards, which we think may be possible.

I think most of us know why we are looking at SL… it gives us some ideas and some numbers in places where we are short hard data and some of it is just interesting trivia. I doubt anyone thinks MOOS will be the same. It gives us some idea of how the numbers work and we make more educated estimates about MOOS. Once we see the OS we can get real. I do think MOOS will use only open source code and resources. To use any proprietary software and incur a cost or future cost… I don’t see it happening.

[a'moaca']

Yes, the open-source code base we will get uses a network protocol that is all TCP, including the voice chat. The Prologue/UU protocol did use UDP, but when you get down to the details, the majority of the messages required application layer acks anyway, so aside from the voice traffic I'm not certain that the UDP version of the protocol was really all that much easier on the server. And some of the most common complaints in UU for visual problems with other avatars were presumably due to loss of the small subset of non-acked messages. The primary example would be avatars moving without the running/walking animations. So due to the particular environment being maintained, and the whole premise that everyone sees things the same, you would likely not get much traction going the route of converting to UDP and making many fewer messages have reliable delivery. Enough things already work badly enough and look wrong enough due to other factors at a higher level than the network transport protocol. Never mind the overhead of re-rewriting the transport-level wrappers around the actual meaningful messages. (In fact, for a subset of messages the MOUL protocol wraps the actual message in part of the old Prologue/UU message headers before wrapping them in the MOUL headers. This even includes, amusingly enough, the flag indicating whether the message requires reliable delivery. The networking rewrite was not as big as you may have been led to believe, at least from the protocol viewpoint.)

Also, application layer acks mean that things like retransmissions require crossing the kernel/user space boundary a couple times, and so if everything is acked anyway, I believe TCP will generally perform better in most any aspect you care to measure, including filling the transmission channel. If you were paying close attention in the TCP-bug days of late 2007-January 2008 you would know that for the big downloads, the MOUL protocol artificially constrains the bandwidth used. I would certainly not mind "fixing" that, but beware, since it was done as a bug fix for something worse. Point being, TCP fills the channel reasonably well.

Each of TCP and UDP have their advantages, but given the goals of the Uru environment I do not think either particularly outweighs the other. TCP is easier to use, and it is what we will be given. So... I think "we" should keep it. I would not mind seeing the voice chat moved away from TCP -- as it is now, if the voice chat is overwhelming a given connection the server would have to notice and throttle it, which is a pain. But then you get firewall/NAT problems with non-persistent connections to work through. Doable, of course, but it's a lot of effort for the gain.

- a'moaca'

P.S. One big advantage of TCP is when the client crashes the server knows right away instead of after a long timeout. This actually does have gameplay and environment consistency implications.

[Ned]

Wow a lot of feedback....at which I'll reply as I can.

Ok a link out to my stuff - Dont want to for privacy reasons and this is an URU board not my board...but you can PM me and I'll happily provide you with some of my links.

TCP / UDP argument.
Ok....me and my team spent like 1 year researching the pro's and con's of both. Admittedly your arguments re: TCP are accurate as we too had to face these arguments.

A rewrite for TCP to UDP and vice versa for the engine and client is not as much a task as one would think.
So we can put that aside for now.

Hard figures...well here is the example I talked about in my previous post.

Ok....
Blade servers in a server center, not a data center.
Each blade set (I call them towers but my server guy would correct me on that) hosts its own network router
the routers are set per user as in the game servers for CS and other FPS machines. This is the model we use.
So example...
Blade 1 = X amount of nodes / zones / shards to accommodate X amount of live users e.g. 30 users online.
user 31 enters tower and is redirected to Blade 2 via the router (or next available blade for those zones)
so a simple scenario = shards A, B, C in tower 1 - Tower 1 hosts 8 to 10 blades @ 30 users per blade.
Each user is allocated a router throttled 10mg connection and is never switched.
So at router each user has a static IP with an allocated 10mg connection.

With this kind of setup most of the issues described in the previous 2 posts are nullified as this is a proven method for handling game systems. This system is called Slot User system and is managed at the network router and is highly efficient.

Not only is it accurate but it saves on server load handling.

Ok internet pipes I host my 3d world on are both in the states and europe. These two locations guarantee full speed global coverage.
One doesn't really need more than these two locations to server well.
My example hosts to the extreme New Zealand as well as it hosts to China Korea USA England and so on. Little variance recorded other than user's own internet connection quirks.
My USA pipe map can be seen here...NiHao Host

When I spoke of free forever....
I did not ever mean to be charging users.
If hosting URU can be achieved without ever charging users then that would be my goal.
I wasn't too impressed with GameTaps $6 (I think it was 6 cant remember now) per month system.
Good SEO management can provide the required funding without ever having to ask the users to make up the deficit.

Ok...Am I sellng to you or here to help?

Good question and its the question that I so wanted to avoid here, but knew it would come.
I am here to help.
I loved URU as the most fascinating and intriguing game that I had ever played.
I bought every version, I had every update, and I collected URU paraphernalia for years. Totalling a messy hard drive folder simply called URU equalling 38 gb of storage space.

The problem with having the capacity to provide for the group as i can means this question will indeed arise.
and the simple answer is "I am here to help" as both an experienced game maker and an URU fan.

porting URU for UDP would be the wise choice
It sounds messy but its not too difficult.
One sad point of TCP is that there are a massive amount of users accessing the Wonderful World Window via TCP hence making server hops for many global locations slow and sluggish.
An advantage to UDP is that once the client has connected UDP packets can be piped at phenomal speeds and accuracy without dropping the connection.
Again true...on a UDP game when a user logs out the server needs time to close the connection
Simple solution....
10 seconds with a time bar "You are exiting URU LIVE" aint so bad really.

With a good server system as I described and a UDP package management system at client side the UDP asset is by far the prefferred system for online gaming.

One only needs to google this to know.

My offer Summary: Network Router + Blade Servers hosting Shards + 2 locales - the USA and Europe = Full Global coverage + UDP specific handling for accuracy and speed + a simple client side config for UDP and a server adjustment needed.

Its not a scary as it looks I can assure you.
End Comment...If I am giving this for free is it such a bad thing?
Go figure???

Recent Edit: My server guys just threw this over to look at - SCTP as a possible solution.
http://en.wikipedia.org/wiki/Stream_Con ... ort_layers
This too could be a solution. My guys are looking into it, but cannot yet report on the viability of it.

[a'moaca']

So... Your argument is that routing topology is sufficiently different for UDP and TCP, such that you get fewer hops with UDP, so latency is lower?

I don't buy it. In my experience they are not routed differently at all. Can you actually prove that TCP and UDP traffic take different paths through the network for a significant enough portion of the Internet for it to matter? I await such proof! It would be fascinating.

I also don't see how fewer hops, or lower latency, or any of that, makes your servers handle more concurrent sessions than if they were using TCP. And if you have this ultra-fast ... UDP pipe thingie, why does it take 10 seconds to "close" a connectionless ... connection? It didn't take 10 seconds to log out of UU and it used UDP to people's computers on their crummy DSL lines.

I also really must wonder what the heck a UDP package management system is, or why I should have to do any kind of configuration on the client side to use UDP.

All told, you have failed to convince me that you have any strong technical reason to change Uru to UDP. But hey... when it's open source, have at it. Nobody will stop you, all we can do is not use your version. When you have it all working, and providing better service than everyone else, people will use it.

I have merely attempted previously to explain how Uru is using, or used, TCP and UDP, and just a little bit about how they affected the way the game works. Ned is far from being my only target audience for this. If you, Ned, wish to cast that post as some kind of roadblock to you switching to UDP, that is your choice. I'm not here to argue, and I don't have time for it. But, I do think you will need to learn more about how the system works before you can provide a logical, convincing argument to someone like me that switching to UDP is "wise", based on the actual facts of the system under consideration, which is indeed NOT identical to yours. If it were, after all, your system would be Uru.

- a'moaca'

[Ned]

Ok dude....

I am tired...and regardless of what i say its falling on deaf ears.
UDP does not make the game faster.
It makes the speed from client to server router more efficient.
Hence a few miliseconds saved here effects the overall performance user side.
Ok router package handling is a proven method....
Cisco routers are a massive load distribution saving at server side.
If the server is not having to write the IP's then it is doing what it should be doing....
writing the update co-ords.
but I will not argue...as I dont feel I need to.
Have you got a better free system?

[a'moaca']

It makes the speed from client to server router more efficient.

My ears are not deaf. This is precisely the assertion I do not believe. If you can prove it, I would be happy to see the proof. Fascinated, in fact.

And I am saying nothing at all about routers at the server side.

Have you got a better free system?

I would say that, the moment the source comes out, everyone using the code as-is, with TCP, will have a better free system than one running UDP. I have no objection whatsoever to waiting and seeing what you do with your UDP-focused view of the world. If it has technical merit, and works better, at that point, you will have a better free system. I have no objection to that, either.

My goal here is to help people understand the Uru system. It is explicitly not my goal to be involved with any financial, hardware, or network protocol decisions on the running of servers, no matter whose. If you offer your setup and people use it, great.

- a'moaca'

[Mac_Fife]

UDP does not make the game faster.
It makes the speed from client to server router more efficient.
Hence a few miliseconds saved here effects the overall performance user side.

I'm trying to understand this UDP/TCP argument at a more basic level: Is saving a few milliseconds on the client/server interface really worth the effort of changing protocol?

Although "lag" was a commonly voiced complaint about MO:UL, I think those of use who've looked into this have determined that conventional lag (ie network latency) was never really a big problem in MO:UL - people with pings over 500ms could still play "comfortably" (and some satellite systems with pings around 1300ms!). This isn't a First Person Shooter, so timeous message delivery generally isn't a crucial factor, and data volumes aren't all that big. What most people percieved as "lag" was really down to client side rendering issues, hence those with lots of horsepower on their client machines never saw much "lag", even in a busy city. Bottom end machines struggled even in empty locations.

To me, improving the client side rendering pipeline will benefit more people more of the time that tweaking the network protocol. Is my perception completely wrong?

[realXCV]

No. But tweaking the network protocol may help to avoid teleporting avatars.