Client Update Concerns

Discussions About CyanWorlds.com Engine Client & Plugin
User avatar
Hoikas
Member
Posts: 344
Joined: Fri Jun 03, 2011 8:38 pm

Client Update Concerns

Post by Hoikas »

Given the recent amount of trouble experienced by users on the MOUL forum with the recent update, I think it would be relevant to share what I've learned about deploying client updates through 20+ Gehn builds. We've updated compiler versions twice quite successfully--it's very rare that we have any issues with an update aside from legit bugs that just weren't caught. The goal here is that an update should be an operation that requires one button press and causes little to no issue on the user's end. If your update methodology (or lack thereof) cause the player problems, especially on a mainstream shard such as MOULa, you are damaging your player retention.

Our Gehn process has changed significantly since the last time we livestreamed it. It now takes me approximately 5 clicks to update Gehn :). Err... and there is a beta shard.

Problem: The first problem we tend to run into is that when the launcher detects an update, it will often die a terrible death and tell the user that their "harddisk is full" -- generally users are immediately frustrated because their hard drives are nowhere near full. This is, in fact, a permissions error. Most folks tend to install Uru in the Program Files directory, which is the default for the MOUL installer. The advice most give is to run the launcher as an administrator. This is an unnecessary hurdle to updating the game and is a barrier to player retention (to the user, this is like waving a magic wand--how would they know to do this without being told?). Running the launcher as an admin is also a security risk. There are currently an unknown amount of security issues in the client. The launcher is responsible for starting the client, therefore the client inherits administrator privileges. If I am a malicious user with a zero-day exploit, I know to pay attention for an update--people will be running the game as an administrator!

Solution: Invest in a good installer! The Gehn Shard installer, in my not so humble opinion, is quite amazing. It toggles the permissions of the Uru client's directory such that the patcher can do its thing without requiring user intervention. It also takes care of some non-Cyan Shard things like copying data from a MOULa install, if found. If not, it launches the patcher in a special "repair mode" that downloads the game from Cyan. It also takes care to install the latest redistributables.

Problem: The other major issue we encounter when shipping down an update are dealing with the issues of missing DLLs for dependencies such as DirectX and MSVC++. With the recent update, the problem here was users not having d3dx9_27.dll. Windows' suggestion in this case is to "reinstall the program". Unfortunately, Uru is a large program at ~3 gigs, so if you're me, asking me to reinstall that over my 1mbps connection is tantamount to saying "stick it where the sun don't shine!" For other users, reinstalling a program is just too difficult. Furthermore, if your installer doesn't include the correct dxwebsetup.exe (see above) that advice is about as useful as a skillet made of cheese (cows are my friends). More savy users will be able to google the DLL name and download that file from a DLL website, but this is a security risk--who knows what sort of hacks have been applied by these unknown individuals. Extremely savvy users will be able to find the DirectX installer on the web, but it appears that this requires an amount of work because Microsoft has hidden lots of downloads in favor of prompts to update to Windows 10.

Solution: This problem was a bit more difficult to solve, but we began with the end in mind. We would like for the patcher itself to be able to patch the user's computer--including installing updated versions of DirectX and MSVCRT! We rewrote from the ground up the everything related to UruLauncher. Here are the relevant pull requests on GitHub: #352, #353, and #385. This adds another flag for files in the game manifests that signifies, "I'm an installer!".

Here is the script I use to generate manifests. Dirtsand accepts the output of this script as-is, however, I think MOSS wants this to be compiled into the network format. I believe I remember hearing of a tool to do that. (I am contemplating submitting a DS pull request to change it to the MOSS way--I really like sendfile.)

Here's how it works: they run UruLauncher.exe, and UruLauncher then asks for the patcher manifest. Any file flagged as an installer is executed FIRST. UruLauncher then runs the client EXE, which might be an updated copy of the patcher itself (whee). The obvious downside to this approach is that the patcher on the user's machine must already have this functionality before you update anything like DirectX or MSVCRT... because it's the old patcher doing the update, after all.

This solution will be difficult for y'all to pick over because it's pretty tightly wound in with other H-uruisms like plString and plFileSystem. I however do recommend pulling it over as it greatly improves the launcher. This code is able to check the Uru install for updates and download them concurrently, which is nice for folks with fast connections where the hashing takes longer than the actual download.
Image
User avatar
JWPlatt
Member
Posts: 1137
Joined: Sun Dec 07, 2008 7:32 pm
Location: Everywhere, all at once

Re: Client Update Concerns

Post by JWPlatt »

Agreed, especially as it all relates back to "player retention," probably the most important aspect or mantra of your post. The barrier is too high with the DLL issue. I think the DLL issue was a surprise here. It was to me at least. It's possible we didn't test on systems where it would have been revealed.

The Gehn installer is something we discussed recently as something needed to keep it simple for the user. I mentioned that you posted instructions on these forums before about its use, but I did not take the time to find the post. I imagine that post has been superseded by now anyway. I don't know the details, but it would be best, I think, if we could use it as a package verbatim from your repo without requiring a port for every update.

Thanks!
Perfect speed is being there.
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: Client Update Concerns

Post by Mac_Fife »

Yup. The DirectX (and MSVCRT) DLL issues date right back to the very first Open Source client builds and arose, I think, because the versions then available to MSVC 2003 had superceded the ones being used by Cyan. Whatever the history, most people who used any early shard and didn't already have those run time DLLs (some people would likely have them from other sources) would have run into those problems three years back, resolved them and forgotten all about them again. I suspect we fell into that category.
For a lot of people who have only ever used the "official client", this would mean they've never seen the issue before this update, so it was a big issue for them. And as Adam noted, even if you know what you need to do, the MS penchant for redirecting many downloads requests to Win10 downloads is frustrating.
The folder permissions issue is one that has persisted for a long time, at least as far back as GameTap, but then there were monthly updates and people got used to what they needed to do. But that didn't make it right. I suppose there are two aspects to consider - the first time install for new installs and the launcher/patcher behaviour. For the former, it's a case of making sure the launcher gets installed with all the resources and permissions it needs to work properly - I think that's something we didn't get quite right, àt least at first, on Minkata.
Mac_Fife
OpenUru.org wiki wrangler
Christian Walther
Member
Posts: 317
Joined: Sat Dec 13, 2008 10:54 am

Re: Client Update Concerns

Post by Christian Walther »

Thanks for the detailed writeup! I had no idea that there was this much to it.

Like JW, the DLL issues were a surprise to me, I wasn’t aware (anymore?) that we were using different versions than Cyan – but then I haven’t been paying much attention to a good installation user experience in the first place.

I agree that taking over these improvements would be a desirable thing. Regarding the porting issues, I think this should be irrelevant for the next MOULa update (what is on Minkata right now) because there will be no change to third-party DLL versions this time, and after that, as I have alluded to at the AGM, I would much rather switch to a H-uru-based codebase wholesale than spend more effort cherry-picking and massaging individual bits and pieces, now that the biggest obstacle, Cyan’s build system, is out of the way. There are some open issues regarding that that need to be identified and researched and discussed, but now seems like a good time for that. I will do my initial research and start that discussion when I get around to it.
User avatar
Hoikas
Member
Posts: 344
Joined: Fri Jun 03, 2011 8:38 pm

Re: Client Update Concerns

Post by Hoikas »

Remember that the H-uru codebase requires MSVC 2015 2013 (but don't be surprised if that's bumped to 2015) now, so being able to update runtimes is important if that's the route we're contemplating, even if it's not such a big deal for the next update. Also, unfortunately, H-uru master is no longer 100% compatible with MOULa (and MOSS for that matter)--there are some subtle changes that I need to document at some point. The last commit that was 100% compatible can be found here.
Image
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: Client Update Concerns

Post by Mac_Fife »

Those comparability issues were something I had a vague sense of and are probably part of what Christian was referring to as needing some discussion. Obviously, we don't have any control over what's in Cyan's server code at this point, so we need to keep with client code that doesn't break that link, at least for the foreseeable future.

One other aspect I'd be a little concerned about is whether any of the H'uru codebase changes remove support for XP: While there's no doubt that there's a diminishing user base there, I have the sense that there is still a significant, although reducing, number of keen fans that are kinda stuck with old systems due to financial constraints. I'd hate to be alienating anyone if that's at all avoidable.
Mac_Fife
OpenUru.org wiki wrangler
User avatar
Hoikas
Member
Posts: 344
Joined: Fri Jun 03, 2011 8:38 pm

Re: Client Update Concerns

Post by Hoikas »

Visual Studio 2012 originally removed support for Windows XP. This was a highly unpopular decision that caused Microsoft to reverse course slightly. The mainline compiler in later versions of VS do not support Windows XP by default, but there is an XP toolchain that can be selected to maintain compatibility with Windows XP. This will deny access to any features added to the Windows SDK with Windows 8, however. We currently use none of those features, so this is no loss. I can honestly say that I care not for users who run Windows XP -- it might seem nasty to lock them out of the game, but really the security issues in an unsupported OS, especially in this day? It highly concerns me.

It shouldn't be too big of an issue to maintain a parallel history that is 100% compatible with MOULa. The divergence only happened recently. Although, it would likely be best if we could either modify the PS code or come up with our own scalable uru server that does the job better than the currently available options. Note that this is not DirtSand, nor is it MOSS. Each have their own unique sets of bad problems.
Image
User avatar
Mac_Fife
Member
Posts: 1239
Joined: Fri Dec 19, 2008 12:38 am
Location: Scotland
Contact:

Re: Client Update Concerns

Post by Mac_Fife »

Hoikas wrote:I can honestly say that I care not for users who run Windows XP -- it might seem nasty to lock them out of the game, but really the security issues in an unsupported OS, especially in this day? It highly concerns me.
I take your point entirely, and on a shard it's quite reasonable to say "look, this is the way we're heading, and if you want to come with us then you'll need X, Y and Z". I just feel it's a bit trickier when we're talking about MOULa - there are some loyal fans who are either reluctant or otherwise unable to depart from an old system, so we'd need Cyan to buy in to excommunicating XP users (or somehow provide a "legacy client") - maybe the number is so few it's no longer an issue, I don't know. At the end of the day, the user chooses to use an unsupported OS on their own cognisance, and I've seen sufficient dumb comments about "why I turn off Windows Update" to know that you can only do so much to protect some people from themselves, whatever OS they're using.
Mac_Fife
OpenUru.org wiki wrangler
User avatar
Hoikas
Member
Posts: 344
Joined: Fri Jun 03, 2011 8:38 pm

Re: Client Update Concerns

Post by Hoikas »

The recent Minkata update debacle illustrates that this accumulated wisdom, especially the part about the installer, is especially relevant. Apparently we're supposed to give polite reminders around here, so here is that "reality."

Here's the truth "fantasy": OpenURU is obstinately retreading ground that has since been traveled by the maintainers of the modern fork of CWE. I fail to understand why those solutions have not been adopted aside from the obvious abdication of responsibility and mulishness. By my rough calculations, this information has been posted to this forum for 3 years and 7 months.
Image
cjkelly1
Member
Posts: 67
Joined: Mon Dec 29, 2008 6:08 am

Re: Client Update Concerns

Post by cjkelly1 »

Hoikas wrote: Mon Aug 19, 2019 4:09 pm the modern fork of CWE
There is absolutely no doubt that the H'uru/Plasma fork has a huge number of improvements over the CWE-ou version, but the incompatibility with Cyan's server remains an issue.

From what I understand from reading the "Should a future MOULa client be H-uru-based?" thread, the removal of the game manager was the initial breaking change. Have there been any other breaking changes after that?

Chogon also mentioned that dropping of XP compatibility was a change that was an issue for Cyan. At the time, H'uru/Plasma did not use anything that was not available on XP, so you could use the XP compatible toolchain to compile. Is that still the case today?


Thanks.
Post Reply

Return to “CyanWorlds.com Engine - Client & Plugin”