Minkata Update Build #190 2020.07.06

[cjkelly1]

Following up on some of the conversation in Discord earlier this week (maybe this should be split into its own topic for easier flow on reading?).

The core problem summary from my perspective is this:
There are almost 10 years worth of performance improvements, bug fixes, code cleanups, and maintenance to this project by a team that has over 15 years expertise in this engine, working on GitHub in open-source fashion with pull requests and code reviews, and all of that work is not able to make its way to MOULa users because Rarified is the only person able to review and approve changes to the OpenUru codebase.
This is not a fair request to make of Rarified's spare time, nor is it feasible for the scope of the changes on H-uru/Plasma.

"Help wanted" is said on occasion, and yet every time we ask how we can help there are no concrete answers.
Concerns about Cyan's requirements are frequently cited as a reason for the way OpenUru works, but when asked for details, nobody has been able to produce a list of those requirements so that H'uru could work within them.
We're asked to take this discussion to the forum, where topics about contributions are frequently ignored for years on end: Ex 1, Ex 2, Ex 3, Ex 4

Recently, there was some work to update the OpenUru project files from VC2003 to VC2010. This upgrading effort could have been avoided by adopting the CMake build system that was added to H-uru/Plasma in April 2011 which can automatically generate Visual Studio projects for a variety of versions (including cross-platform Makefiles for eventual Linux and macOS ports). It feels like there's a willful stubbornness to refuse to adopt any work that the H'uru team has done, except in extremely small pieces at an agonizingly slow pace.

The current process seems to be that Rarified has to read through 9 years worth of changes and 2500+ commits to decide if they are safe, and then cherry-pick them over to the OpenUru project. Again, this is not a fair request to make of Rarified's spare time, nor is it even feasible.

How did we end up in a situation where Rarified is expected to review and approve every change? How do we establish a process (such as pull request reviews that are already being used on GitHub) so that Rarified is not the single gatekeeper?


However, even addressing that issue for future contributions doesn't resolve the fact that there is 9 years of work and several major code restructurings and build system/library updates on H-uru/Plasma that should form the basis for future MOULa clients.

The problem with using tools like FishEye to track changes to H-uru/Plasma is that they can only notice changes after all the discussion has taken place and the code has been merged. It doesn't provide an opportunity for anyone from OpenUru to participate in those discussions.

H'uru is using GitHub, which offers us git code hosting, issue tracking, pull requests (code review and discussions), wiki documentation, and automated builds in the same place. We have a Continuous Integration system set up, whereby AppVeyor builds the entire codebase every time a change is merged (we're investigating moving to GitHub Actions, but there are some technical issues with building the Max plugin due to non-open-source libraries). We also have a repo that builds a package of all the required development libraries.

The current OpenUru clients for Minkata and MOULa are built with Rarified's self-hosted Jenkins system. This makes sense when you need to build using toolchains that aren't available on hosted CI tools (like VC2003 and VC2010 and Solaris), but it also means a lot of manual work to set up new build agents that would be compatible with the current state of the H'uru/Plasma repository. At minimum, it would need an agent running VC2017, and a copy of all the required libraries. If the intention is to make H-uru/Plasma builds through Jenkins, there are a few ways we can try to make the library management easier using tools like vcpkg. Once we have our clients building on GitHub Actions, I'd be happy to help get them building on Jenkins.


One concern that gets raised is the worry that malware might be merged into the codebase and get deployed to MOULa. I personally find this laughable for several reasons:
1. We have multiple sets of eyes on the pull requests, that could catch something like that.
2. We make our builds for Gehn Shard directly from GitHub, and hope to move that to a publicly visible GitHub Action pipeline soon.
3. We have a incredibly long multi-step process before anything ends up on MOULa: Gehn Shard -> Minkata Alpha -> Minkata Prime -> MOULa Staging -> MOULa


Now the important bit: While this is being discussed and Rarified is figuring out how to play catch-up on a decade of changes, we have 2 major changes that are in progress:

• Updating from Python 2.7 to Python 3.8: https://github.com/H-uru/Plasma/pull/650
  Python 2.7 was released in 2008 and is now past its end-of-life. Python 3.8 is the latest version of Python, but there are some language incompatibilities between 2.x and 3.x. The benefits of moving to Python 3 are security from a supported library, much much better built-in Unicode handling, easier compatibility for Linux and macOS, and a much nicer scripting environment for Age builders to work with. This work has already been tested on a private shard by some enthusiastic fan testers, and we are looking to merge it fairly soon.

• Updating PhysX to 4.1: No pull request yet, but work-in-progress
  The closed-source PhysX 2.6.x version that Uru currently uses is no longer easily available for download, is 32-bit only, and only available for Windows. PhysX 4.1 is open-source, cross-platform, and should provide a better supported and more mature physics system.

This is your opportunity to weigh in on these changes and raise concerns so that they can be assessed before merging.

Perhaps this should be split off into its own topic, so it is not missed? I am thinking that it is probably already overlooked, as it is in the "build #190" thread, and the current build thread is #193.

[rarified]

Already ahead of you on this — I’ll be scanning several topics for posts on this topic to collate into a new thread.

Right now all of my time is directed at getting the last bugs out of the current Minkata client and delivering everything on Minkata to Cyan. So the collecting of posts won’t happen immediately.

_R