My big plans for the code

[branan]

Since few people here have worked with me regularly before, this is an overview of my plans with the code, and where I see it going. I'm hopeful these are things that other developers share, and that I'll be able to merge this work into a repository here at OU - hopefully as the "clean slate" for future Plasma-based projects. Basically, this is what I'd like cwe-nextgen to start off as, and hopefully from there the code will be in a state where "the sky's the limit" so to speak.

1) Port the build to CMake. This is both because every windows developer has a different version of VS, and because eventually the code will need to build on mac and linux. CMake is something the GoW/Grey Hat/H'uru developers already have experience with. I've looked at other cross-platform systems (such as scons), and I find CMake to be the most robust.

2a) Write a plGLPipeline
2b) Replace PhysX with ODE or Bullet

DirectX and PhysX are both windows-only, so they have to go. plDXPipeline can and should stay around as an option for a while, but I'm hopeful that future graphical improvements will be done to the cross-platform OpenGL pipeline.

3) Abstract anything Win32-specific. There's a lot of this in Plasma. It looks like at one point Cyan tried to do this, but over time the lines have become very, very muddied.

4) Implement those abstractions on some other platform.

5) Start adding new features.


The CMake port has already begun in earnest in the H'uru clone of the code. Along with the CMake port, we're cleaning up all the #includes that use absurd relative paths, and removing any hardcoded paths to StaticSDKs (those external dependencies will be found by CMake).

What I'd like to see done along with this is to move the repository files so that Plasma20/ is the root. Without the StaticSDKs folder, there's little sense in having the actual code start three levels deep. This is more of a nitpicky thing than anything else, honestly.


All that being said: What do other developers have planned? What sorts of feautres/ports/fixes do you want, and how do you see those changes in the long-term health and stability of the Plasma engine?

[Nye_Sigismund]

I can speak as a content developer, I suppose.

-> A long, hard look at improving the development pipeline to facilitate faster content development. For the health of Uru and fan projects, this actually is pretty critical.
For those not aware, at the moment the development pipeline from idea to age in Uru, using the best tools available, is as follows:
1) Get copies of 3DS Max 8, POTS + MOULa, Drizzle, some form of Python development toolset.
2) Create age.
3) To test age: Export out of Max. Convert using Drizzle. Hope that all the python, sound etc files are in the right place. Boot up POTS. Check age.
4) If something does not work, repeat steps 2-3 until done.
5) Somehow get everything out to the general public (Yay Drizzle!). Note that this will mostly be in POTS format. The only easily available MOULa format age at the moment I believe is Llantern.

The issues with the above are easy to spot. There are significant flaws in the development pipeline -
1) The requirement for old tools and old versions of the game.
2) The shoddy testing proceedure that takes too long and is too easy to screw up.

So, as a developer, here is my big plan for this first important step:

1) Plugins that work with 3DS Max 9 and up. I'm sure that the talented GoW team can get PyPRP2 up to speed thanks to the Max plugin's code, so leave that stuff to them.
2) A stripped-down version of the MOULa client that can simply be used to test ages. It needs to run in the background and update when the age is re-exported. I'm pretty sure Cyan would appreciate this too, given that we've all seen the artifact from their attempt to do this in the Max plugin (SceneViewer).

Ideally, 2) would at least grapbically update on the fly, but that would probably be difficult.

I know this is a little bit on, timeline wise, from Branan's list, but I feel that keeping an eye out for ways to actually make Uru grow wider as well as taller is important.

[rarified]

Hey Branan, gotta be brief as my day's getting started, but a couple of questions...

Could you characterize your goal groups as being applicable to the hypothetical cwe-compat or cwe-nextgen we've been talking about?

One reason I ask is to know when we should seek Cyan input about how much structural change to the open source base would make it difficult or impossible for them to take advantage of for applying to the general MOULa servers. (Especially considering their resource constraints).

I'm open to restructuring, but with the proviso that we can keep the CWE entity in a ready-to-roll state (i.e. there's some way to build the "whole client" from the restructured repository.

As an alternative, would you consider using the new Mercurial subrepo facility a reasonable alternative (i.e. make the Plasma20 area and below a repository unto itself that can be either cloned by itself, or obtained by cloning the encompassing CWE repo)?

_R

[branan]

My changes would definitely be applicable to cwe-nextgen. Cross-platform compatibility includes things like changing the physics system, which would likely cause PRP format changes.

The CMake system will definitely be "ready to roll" when it's done (our goal is to have plClient buliding with CMake by Monday at the latest). The other major changes would be done in feature branches and merged in when they're ready. It will still be possible with CMake to distribute a bundle of the required libraries on Windows, and add special CMake code to handle having them in a known location - but so far I've found it's picked up on the default windows install locations for most of the SDKs I've had to add to get the code building. Because of that magnificent autodetection, I think getting rid of everything above Plasma20/ is viable, since a skeleton SDKs directory won't be needed.

I've had a couple discussions with mark re: getting code back to Cyan (basically every time I've fixed anything for him), and he's told me there's still really no plan. When there are major fixes we've been emailing things directly to him, and he integrates as he has time and resources available. As far as I'm aware that's still the workflow.

I think if Cyan had intended to pull changes regularly from the repository, it wouldn't have been quite so stripped down. The missing parts make it harder to test changes to the overall system (by which I mean client AND server), and likely would make it harder for Cyan to pull changes from the public repository en masse. This is why I've proposed keeping a cyan-compatible repo for bug and security fixes, but doing most major development in a separate repository that we can safely break away from Cyan compatibility with.

[rarified]

Let me make a couple of requests, then.

In your CMake infrastructure, see if it can be made to use specific versions of SDKs and not go arbitrarily looking for it's idea of the "best". Or at least make that a configurable option. My automated Windows build environment has copies of Visual Studio 2003, 2005, and 2008 installed, along with their corresponding SDK. I'd like to make constructing build artifacts as deterministic as possible (with my CI configuration determining the deterministic!).

Also, can you keep me up to date with updated tools, libraries, and other resources you will be introducing dependencies against, so I can acquire them and add them to the build environment(s)?

Thanks!
_R

[branan]

CMake by default assumes a unix-y filesystem layout. For PlasmaClient on windows, I've created a C:/Plasma folder that contains include/ and lib/ (just like on unix). For CWE, I'll try to have a zip file of my equivalent folder, containing all the redistributable dependencies built for VS 2010 express, since it's easy for developers to acquire. You'll be able to unzip that to any location you want, point CMake to it, and everything willsual studio to target at configuration time, so other VS installs won't cause any issues. be all set. For other SDKs (such as PhysX) a bit of manual configuration will still be needed, but I'll try to add automation (like searching for the PhysX installer's registry key to find the installed location). CMake lets you select which version of vi

And of course I'll write up a short introduction to CMake and how to work with it for CWE.

CMake also lets you completely bypass and override any detection it does and tell it exactly which include directories and library files to use (also useful if you have something in a weird location that it can't find). You can take advantage of this in scripted builds to pass it any special congiguration options that are needed.

Major SDK changes will (hopefully) only be done on feature branches, so there should be plenty of warning before they're integrated anywhere.

[rarified]

Here's another request for you and your project members.

Could you organize your changesets as you make modifications to your repos so that each changeset covers one particular small goal, rather than putting the entire reorganization, CMake implementation, etc all in one big revision?

I ask because I'd like to be able to import changes that would be applicable to other clones and the baseline repository from your work, independently if possible of major organizational changes. Things such as bug fixes which don't impact how the client code is organized, and could benefit other variations.

Thanks
_R

[branan]

Yes, we're already doing things that way. VS2008 fixes were first, we're almost done with CMake, and the next set of revisions will be the FS layout adjustments (moving Plasma20/ to the root). We can work out which of those points (if any) OU wants to pull from.