New Feature: Layers

[sparhawk]

Wow! There are coming a lot of nice features with DR.

 

BTW: How about layers? You know like in Blender where you can put different objects in different layers and then switch some of them on, so that only the objects in the selected layers are visible. I think I posted such a request quite early in the beginning, because this would also be a very usefull feature to have. Especially when the screen gets more and more cluttered.

[greebo]

Layers could be created during runtime but the major problem will be how to save them. The map file format doesn't support this and we would have to find a clever solution of how to save this without breaking compatibility with Doom3. (There are really only custom spawnargs and special comments possible, as these are ignored by Doom3, anything else would break the portability).

[OrbWeaver]

Layers should be easy. The filter system can be used for this, since it does not have any fixed categories but just takes two string arguments, e.g. "texture" "textures/common/blah" or "entityclass" "worldspawn". This could just as well be used for layers as well by passing "layer" "15" or something.

 

Storing the layers could be done by adding an editor-specific keyval on affected entities, e.g. "editor_drLayer" "15". Then whenever an entity is rendered, it checks its layer key, passes it to the filter system to get a visible/invisible value, and then submits itself or not based on the result.

 

Some thought would be needed regarding how to configure layers (i.e. do you need to create them first, or can you assign an object to any layer, how is the list of layers managed, etc) but the infrastructure is basically there I think.

[greebo]

How would the worldspawn brushes/patches be handled by this? It should be possible to add arbitrary objects to any layer, but most primitives are children of worldspawn.

[OrbWeaver]

Good point. Layering would only work for discrete entities, hmm.

[greebo]

Perhaps it could be possible to add this as comment into the map file, like:

 

// primitive 0 // drLayer 1
{
patchDef2
{
"_default"
( 3 3 0 0 0 )
(
( ( 256 0 0 0 0 ) ( 256 48 0 0 -1.5 ) ( 256 96 0 0 -3 ) )
( ( 320 0 0 2 0 ) ( 320 48 0 2 -1.5 ) ( 320 96 0 2 -3 ) )
( ( 384 0 0 4 0 ) ( 384 48 0 4 -1.5 ) ( 384 96 0 4 -3 ) )
)
}
}
// primitive 1 // drLayer 2
{
brushDef3
{
( 0 0 1 -64 ) ( ( 0.03125 0 60.25 ) ( 0 0.03125 63 ) ) "_default" 0 0 0
( 0 1 0 -192 ) ( ( 0.03125 0 1 ) ( 0 0.03125 0 ) ) "_default" 0 0 0
( 1 0 0 -384 ) ( ( 0.03125 0 60.25 ) ( 0 0.03125 0 ) ) "_default" 0 0 0
( 0 0 -1 -64 ) ( ( 0.03125 0 60.25 ) ( 0 0.03125 1 ) ) "_default" 0 0 0
( 0 -1 0 96 ) ( ( 0.03125 0 63 ) ( 0 0.03125 0 ) ) "_default" 0 0 0
( -1 0 0 256 ) ( ( 0.03125 0 3.75 ) ( 0 0.03125 0 ) ) "_default" 0 0 0
}

[OrbWeaver]

Hell no. Comment means comment.

 

It might be possible to use a series of keys on the worldspawn, like

 

"editor_drLayer3" "15"

"editor_drLayer17" "4"

 

to indicate primitive 3 on layer 15 and primitve 17 on layer 4. The problem would be how to get the primitive numbers, since I don't think the scenegraph code makes any guarantees about primitive order or even allows you to query it.

[greebo]

Well, if the comments aren't an option, the spawnargs would certainly be possible, although I'm a bit shuddering imagining the layer spawnargs in the bonehoard map with its 3200 primitives.

 

I could think of another possibility where an additional file is saved along with the map (like bonehoard.dr or bonehoard.info) where all this information can be stored.

 

However, the primitive numbering wouldn't be much of a problem, it's just a matter of extending the scene::Node interface by methods like getLayer() / setLayer(). The actual numbers should be available while parsing the map file, shouldn't it? Or is this info swallowed by the Tokeniser?

[sparhawk]

IMO the layers wouldn't be a permanent assignment most of the time. If some object doesn't have a layer number just put it as default in layer 0. Anything else, that gets reassigned must be put there by the mapper. This way, it should already be helpfull, because then you can assign a few items to a different layer, and switch off layer 0.

 

As for saving. How about storing a secondary optionfile per map, like mapname.opt or something. You can do some hashes to make sure that the map file still matches that option file when you reopen it. This way you can store about any additional options that you want, for later use. Entitynames are to be unique in Doom 3 so you can use them as a key. In fact they are even unique at runtime. I once accidently created an entity with the same name and Doom 3 broke because of it. Entityname shouldn't change so often, and even if they do, because the mapper changed the map in the meantime, you can simply revert them to layer 0 if you can't match them. Worst thing that can happen is that some objects are in the wrong layer after loading, if the mapper messed with the map file, but that's not really a big problem. At least it would certainly not damage anything.

[OrbWeaver]

Using an additional file did occur to me as well, and might become necessary for the more advanced features. In fact, it would be possible for DarkRadiant to have its own complete native map format which would store any information necessary, and then "export" to the Doom 3 format when required. This might not be usable in practice however, because generally you would need to export very often in order to test the map in game.

[sparhawk]

You can write it at save time. This would be transparent for the mapper. You might make it optional, so that mappers can have it always on at saving, or make it explicit by using an "export" command.

[greebo]

Yeah, we could have DarkRadiant saving two files when you hit Ctrl-S. One .map file compatible with Doom3/DoomEdit and one DarkRadiant-File that contains the whole map plus all the additional data (the master file). It doesn't need to be very different to an ordinary map file, we could add a "DarkRadiant" entity that hierarchically contains all the editor-specific stuff.

 

The map loading algorithm has to be versatile enough to check for the "second", DR-specific map file and load this instead of the .map file.

[sparhawk]

I've split this into a sperate topic, since it's becoming a complete thread on it's own already.

[sparhawk]

The map loading algorithm has to be versatile enough to check for the "second", DR-specific map file and load this instead of the .map file.

 

It depends on waht you plan to store there. If it's just some option settings, then it should be rather straight forward. You can load the regular map file as usual, and the load the second file and set the appropriate options. If you intend to store data there, that actually modifies the loading process itself, it might be a bit more complex.

[OrbWeaver]

I don't like the idea of selecting an ordinary map file and then secretly switching to the corresponding DarkRadiant file during load. I think the process should either be:

 

1. Load a .map file. DarkRadiant searches for and opens the corresponding DR file if possible, which ONLY contains DR-specific information such as layers. All geometry is contained in the .map file, so if the DR file is not found the map is loaded without extra information as it does currently.

2. DarkRadiant uses its own file to contain geometry and all metadata (like the .blend file in Blender). When mappers want to test in Doom 3, there is a separate option to save in Doom 3 format (like "Export", although it could be made more accessible since mappers would need this a lot).

 

Option 1 is obviously a lot easier to implement, so this is what I would lean towards unless there is a compelling reason to use 2. Some design work would be needed to determine what is stored in the DR-specific file and how it maps to the geometry in the .map file.

[SneaksieDave]

And a question might apply about saving in DoomEd and then going back to DR. Will there be any need to take TDM-specific items or settings (such as those above, or objectives, etc?), saved in a map, and write them out to the separate file, so if the user opens their map in DoomEd, makes a few changes and saves it, next time they open in DR, their TDM-specific items and settings will be stored separately, or maybe merged in?

[OrbWeaver]

If you edit the map separately in DoomEdit then all bets are off regarding the DarkRadiant metadata. I'm not sure how DarkRadiant could tell that the map file had been separately edited however, unless it did something like testing the timestamps and not using a metadata file that was older than the .map file.

[sparhawk]

Since all the TDM specific stuff is set on spawnargs you shouldn't loose these. But of course you might loose metadata stuff that is only relevant to DR. Syncing a map can become pretty complex if you try to match up the metadata file with a modified map file.

[SneaksieDave]

If you edit the map separately in DoomEdit then all bets are off regarding the DarkRadiant metadata. I'm not sure how DarkRadiant could tell that the map file had been separately edited however, unless it did something like testing the timestamps and not using a metadata file that was older than the .map file.

Well, I mean something TDM-specific DR or the mod might add/require. For instance, Objectives. Will Objectives not be written to the map? Or startup equipment - separate file, or right in the map? I imagine (and I think I've seen you discuss) some kind of format for things that DoomEd will just ignore (like 'editor_' maybe? IIRC...), but the question becomes, if DoomEd can handle those items (like objectives or equipment) coming in just fine, will it also write them back out. That's the concern I'm trying to describe. It's great if DoomEd can still open a TDM map, but it loses most of the value if it can't also write it and all important info back out. If that 'extra' stuff was separated off into another file as discussed, no more worry.

[OrbWeaver]

Anything that affects the game, such as objectives or startup equipment, will necessarily be written in a form that DoomEdit can handle, i.e. keyvalues on game entities. These will have no problems in either DoomEdit or DarkRadiant.

 

The situations where DR-specific metadata would be required are things like moving brushes onto layers or saving the last camera position. Some of these (like the last camera position that greebo has already implemented) can be saved by using special "editor_whatever" keyvalues on the worldspawn; others, such as brush layering, would require a separate storage mechanism.

 

At no stage is there any question of important gameplay data being lost, it is just an issue of what happens when a separate file is used to store DR-specific information and the map is then edited separately in DoomEdit (without updating the DR file).

[sparhawk]

The issues are mostly how the loading of a map should be handled when two files are now used instead of one. The loading and writing is transparent. Obviously everything, that Doom 3 needs to run the map must be included in the map file. Since objectives are part of the game obviously they must be contained somehow in the map, just like before.

What might be an issue to think about though is: what happens if you move 5 entities into layer 2, then open the map in DoomEdit to fix something that DR can't handle, and also delete two of the entities that you assigned to the layer. Now when you load the map in DR, it will know about the layer and will look for these entities, but two of them are no longer there. Just as an example.

[Domarius]

Hm we were discussing this a few times a while back, and I'm still of the opinion - layers make sense in a modelling app where you're working on one thing - you want maybe the clothing on one layer, the jewelry on another layer, etc. so things don't get in the way.

 

But when you're level editing, its not useful to hide everything across the entire map - you want to section off "areas", and this is where DromEd's "area brushes" were really usefull. You're working on a mansion, so you activate an area brush aroudn the mansion so that the entire rest of the map outside is hidden. Or you're jsut working on the top floors, so you activate the top floor area brush you made, and hide everything else.

 

I'd say put the question of "do you want support for layers" to our actual map editors, and see if they even want it before spending time on this feature.

[sparhawk]

I often found layers to be usefull as well, because I don't always want to switch off the entire area. For example, I might want to keep the walls to have an idea of the area, but not all the furniture and such. The area brush is a good way to conveniently switch off a large chunk at once, without having to add everything individually, but the layers give you more detail. Actually, if a layer system is implemented, an area brush should be simple, because all it would have to do is to assign all covered objects to a specific layer. So an area brush would be a subset of the layer functionality.

[greebo]

Layering is definitely desirable. In the bonehoard map there are several "floors" and the top orthoview can get really cramped with all the overlapping brushes. Layers would really help easing the problem, so IMO it's definitely worth the time.

[OrbWeaver]

Layers are a superset of region functionality. You can use a "select partial tall" box to select all of the objects in a particular area, and consign them to a layer called "north entrance" or whatever. Then you can activate or deactivate that layer exactly like a region.

 

With regions you can do this, but you can't select a group of objects in different locations, which is an advantage which layers provide. The implementation difficulties are similar in both cases, so it makes sense to go for the feature which provides the larger set of functionality.