Scenegraph

[OrbWeaver 448]

No I'm not doing anything at this precise moment, feel free.

[greebo 29]

Ok, the Scenegraph is in its own module plugins/scenegraph now. I haven't checked whether it compiles in Linux yet, but I'm confident.

[OrbWeaver 448]

That was quick, is the GraphTreeModel rewritten to get around the implementation dependence, or is it just disabled for now?

[greebo 29]

That was quick, is the GraphTreeModel rewritten to get around the implementation dependence, or is it just disabled for now?

I converted the GraphTreeModel into a StaticModule for now. It acts as any other module and registers as scene::Graph::Observer at module initialisation. Apart from that, I only needed to extend the scene::Graph::Observer interface a bit, but that's all. Not too much work.

 

Rewriting the GraphTreeModel is another issue though - this will be necessary if we ever want to be immune to STL implementation changes.

[OrbWeaver 448]

Ah OK, so you've managed to disentangle them then, that's good.

 

I'm not sure it makes sense for the GraphTreeModel to be a module though, rather than a core part of the Entity List which is created on demand.

[greebo 29]

I don't know exactly how the inner mechanics of the GraphTreeModel work (it's a bunch of gtk-style methods), but I suspect that the GraphTreeModel must "grow" along with the Scenegraph. When the GraphTreeModel is instantiated at a point where the scenegraph is already populated, I assume that the GraphTreeModel can't catch up with the changes.

[OrbWeaver 448]

The way I would do it is:

 

1. Entity List owns the GraphTreeModel, and can implement it however it wishes (probably something wrapping a GtkTreeStore or similar).

2. Each time the Entity List is displayed, it traverses the Scenegraph using the standard tree-traversal interface to build up a fresh GraphTreeModel corresponding to the current state of the scenegraph.

3. Once populated, the Entity List uses its status as a Scenegraph Observer to automatically keep the GraphTreeModel in sync with changes to the Scenegraph.

4. The GraphTreeModel can be destroyed when the EntityList is closed (since a single traversal of the scenegraph doesn't take that long).

 

This makes the GraphTreeModel purely an implementation detail of the Entity List, and does not require any changes or additions to the module interface(s).

[greebo 29]

Yet another bump of this topic: I am subscribed to the OpenDarkEngine SVN commit mailing list and I was skimming through some of the code too. I don't know how advanced the OPDE scenegraph is, but maybe volca has considerable experience under his belt? I wouldn't want to ask him to actually write anything for us, but maybe he could give us some tips before we start?

 

On the same topic, I was reading through "Game Coding Complete" by Mike McShaffry (the guy who came to the Deadly Shadows project as some sort of firefighter and wrote the third person camera movement), and there is one topic about the scenegraph. It's a basic example, and I will try to pick the best tips from this book - maybe these tips are applicable to our scenegraph (but I somehow doubt it).

[OrbWeaver 448]

Those both sound like potential sources of information to me.

 

To be honest, I think we have probably gone as far as we can with the top-down design approach to the scenegraph, and any further time spent will be unproductive. I suggest therefore that we look to an incremental, bottom-up approach to refining the existing scenegraph, addressing issues that we know exist:

 

1. The unnecessarily-complex Node/Instance division. Moving all necessary interface functions onto scene::Node and removing scene::Instance altogether should be achievable and would simply the codebase quite a lot.

 

2. The piss-poor implementation of the tree itself: even without spatial division it should be possible to improve performance considerably by improving the tree implementation (it is not even a tree at all AFAIK, but a list of variable-sized "node path" lists).

 

Neither of these are trivial tasks to complete obviously, but I think they are at least realistic.

[greebo 29]

Agreed. Both tasks are indeed huge, especially the first one. It will be quite an odyssey, because it can't be done gradually. I estimate that it takes one or two weeks until the code will compile again, but there is no way around that, of course.

 

When I'm upright again (I'm quite ill at the moment), I will open up a new branch for this.

[greebo 29]

Ok, it's time for a small status report. This applies to the scenegraph branch only.

 

After a coding rampage of several dozens of hours, I can say that I'm over 90% through with the scenegraph refactor. The scene::Instance class is completely gone without replacement, everything is handled by the Nodes now. To achieve this, I had to move all code from the instances to the nodes, like Renderable, Bounded, LightInstance, TargetableInstance, etc.

 

I had to rewrite almost all the scenegraph walker classes (I didn't know we had so many!) and stumbled over a lot of old code. I could solve some things by using adaptor classes, though. I also had to rewrite the entitylist module and the TargetableInstance code (which still has bugs) plus countless minor things.

 

Two features (ExpandSelectionToEntities and the EntityList's click-to-focus-on-entity) had to be disabled for the moment being, as this requires more thought, it was too much to handle on-the-fly.

 

The code is definitely not stable yet. There is a crash on shutdown due to some NameObserver being incorrectly destructed (which makes me ask wtf?, because I didn't change anything in there) and duplicating map objects creates a nice segmentation fault, which I can track down more easily, I guess. Moreover, the code does not compile in Linux, because I haven't had time to adapt the sconscript (including the addition of the (temporary) scenelib static library, which I needed for the refactor). I will do this as soon as I've fixed the major issues.

 

Next issue is that I can't do without the scene::Path information at the moment. I had to write a quick-and-dirty method findPath() which locates the scene::Path for a given object, but this is requires a full scenegraph traversal and is way too slow. This must be resolved before DarkRadiant goes into production again.

 

Overall, it seems that I'm over the hill now, but a lot of bugfixing is awaiting us. As soon as the major issues are solved, angua will start to use the experimental branch for some mapping.

[Komag 20]

Wow, awesome! I'm impressed despite knowing pretty much nothing about coding. Looking forward to trying out the new version, and I'm willing to test stuff and try to help find bugs

[SneaksieDave 39]

Heh, I'm struggling to post something similar, wanting to say "wow, congrats! ...on, erm... that huge thing you guys have been working on... whatever it does!"

[OrbWeaver 448]

Good work, I've seen the email notifications from Sourceforge and you have definitely been attacking this problem. I can help with getting the Linux side of things up and running once you have finished the main changes.

 

Hopefully your foray into the codebase should give some idea of what walkers etc. can be removed or replaced; I don't doubt there is a hell of a lot of redundant, duplicated and generally-crap code in there.

[greebo 29]

Good work, I've seen the email notifications from Sourceforge and you have definitely been attacking this problem. I can help with getting the Linux side of things up and running once you have finished the main changes.

Most changes are file removals and I renamed one or two file pairs. The only major thing is the static scenelib library, which has to be set up like mathlib in the sconscript. It should not have any dependencies apart from the boost headers.

 

Hopefully your foray into the codebase should give some idea of what walkers etc. can be removed or replaced; I don't doubt there is a hell of a lot of redundant, duplicated and generally-crap code in there.

There are a lot of simplifcations possible, but there's not too much code in there that can actually be removed. I intend to remove some of the libraries like instancelib.h though. There is a lot of templated crap in BrushVisit.h which is so confuscated that not even the VC++ compiler detected the errors. I experienced a few "pure virtual function calls" during runtime due to these templates. This is a cascade of templates and it took me quite a bit to wrap my head around that.

[New Horizon 470]

Wow. Great work as always Greebo! I'm always blown away by how quickly you rip these things to shreds!

[greebo 29]

Next status update:

 

- I could fix the crashes (not saying there aren't any left, but the obvious ones are gone)

- The ExpandSelectionToEntities feature is functional again (had to rewrite the walker).

- I removed the traverselib.h and removed the scene::Traversable::Walker stuff.

- Some optimisations in the Node loops transformChanged(), etc. They should be a bit faster now.

- Map loading and saving works again (was crashing sometimes before).

- Cloning nodes is working again too.

 

So, I think we're ready to get the Linux build working, so that I can use the profiler to check for some performance drains I might have introduced. I don't expect DarkRadiant to be faster than before, but it shouldn't be slower either. I'll look into the sconscript tomorrow, if I have time.

[greebo 29]

Ok, the Linux build is working again in the scengraph branch. I figured I could do it right away just as fine.

[Tels 266]

Ok, the Linux build is working again in the scengraph branch. I figured I could do it right away just as fine.

 

I only have checked out the trunk, so I can't test it here. Is there a way from a checked out subdirectory to check out one directory above it, too?

 

te@te:~/src/darkmod/darkradiant$ update
Path: .
URL: https://darkradiant.svn.sourceforge.net/svnroot/darkradiant/trunk/darkradiant
Repository Root: https://darkradiant.svn.sourceforge.net/svnroot/darkradiant
Repository UUID: 80d3f8d2-1b0e-0410-a445-ca0696aa8c50
Revision: 3183
Node Kind: directory
Schedule: normal
Last Changed Author: orbweaver
Last Changed Rev: 3110
Last Changed Date: 2008-03-08 11:59:29 +0100 (Sat, 08 Mar 2008)

At revision 3193.

[greebo 29]

I only have checked out the trunk, so I can't test it here. Is there a way from a checked out subdirectory to check out one directory above it, too?

I don't understand what you mean. If you want to have a look at the scenegraph branch, you'll need to switch to it, it's a rather quick operation.

[Tels 266]

I don't understand what you mean. If you want to have a look at the scenegraph branch, you'll need to switch to it, it's a rather quick operation.

 

Ah, okay, that's another possibility. What I meant was that I check out the trunk and each branch etc, aka the entire repository. Which is clearly not necessary.

 

Should I switch to the branch to test it on my linux box, or do you just want to go ahead and merge them and let me test it later?

 

(In the former case, please remind me again how I switch

[greebo 29]

Ah, okay, that's another possibility. What I meant was that I check out the trunk and each branch etc, aka the entire repository. Which is clearly not necessary.

That's of course possible, you just need to check out this URL:

 

https://darkradiant.svn.sourceforge.net/svnroot/darkradiant

 

which contains the trunk, tags and branches folder. I'm pretty sure that's not worth the hassle.

 

Should I switch to the branch to test it on my linux box, or do you just want to go ahead and merge them and let me test it later? (In the former case, please remind me again how I switch

You don't need to test it if you don't want or don't have time. At any rate, switching is easy, go to your darkradiant working copy (the trunk) and type:

 

svn sw https://darkradiant.svn.sourceforge.net/svn...ches/scenegraph

[greebo 29]

I think I've covered the bases now. The instance removal is now completed and DarkRadiant should be as good as before.

 

Are there mappers who can help me doing a complete feature-check of the new version? If yes, I will assemble a new experimental release build and upload it somewhere. Angua is already using it, but she's quite busy with the AI door coding, so I need more helpers here. Let me know if you can help.

 

edit: The experimental build 1 is here:

http://208.49.149.118/TheDarkMod/DarkRadia...t-0.9.6exp1.exe

 

The task is to download the build, install it, test everything you can think of and report any crashes or weirdness that wasn't there in DarkRadiant 0.9.5. Known bugs are of course excluded, I haven't fixed anything on top of the refactored scenegraph - bugs that have been there before my changes will most probably still be in there.

 

Beware that the build can have critical bugs I might have introduced, so back up any work if you're saving any of your maps using the experimental version.

[Komag 20]

I've installed it and trying things out.

 

Right away I like the new lights. But I miss seeing the old lights because you could see the light color right on it, now it's just a green outline. But I really LOVE seeing the whole light size in the 3D window, very nice.

 

I found a pretty huge bug so far, when I save the map, all my "convert to func_static" entities are deleted! Good thing I backed up my map before!!! ... Just tried some more, selected a few brushes, made into func_static, selected some other brush, deselected (just to be all cleared), then saved, boom the new func_static disappeared! It's just gone, they're all gone man, gone gone gone

[greebo 29]

Wow, that's in fact a pretty huge bug. I'll definitely fix that.

 

Thanks so far.