Announcement: SEED system

[Tels]

Makes me jealous:

 

[Sotha]

Makes me jealous:

 

 

Impressive, I really liked the autumn forest scenes, but the car (in which the whole thing focused more) was boring.

And what does that "for cinema" mean? Probably the engine is so heavy it works well only on some specific supersystems?

[jaxa]

You obviously missed the wave of "yeah, but can it run Crysis" jokes.

 

What I get from it is that this could be used as realtime rendering for games, or you can run it on an abacus for 7,000,000,000 years to produce non-realtime "For Cinema" scenes.

[Tels]

Impressive, I really liked the autumn forest scenes, but the car (in which the whole thing focused more) was boring.

And what does that "for cinema" mean? Probably the engine is so heavy it works well only on some specific supersystems?

 

The Forest scene was impressive in two ways:

 

* handles a LOT of detail, and that in real-time AND you can edit it in real-time

* it is still all manual, the entire system depends on the mapper to "paint" each detail in. Granted, its in real-time and you can quickly fill the scene, but if you want to change it, you have to start again from scratch...

 

The car is impressive from a technical point of view (lots of detail, lots of reflections, lots of shading). The thing you probably missed is that it is basically Cinema-quality CG done in real-time on one PC. Since most movies overdo on CG scenes, it is easily dismissed as "oh thats easy they do that all the time". But usually in a lot higher quality (which normal people like you and me don't even know is there) and not in real-time.

[Shadowhide]

I really don't care about new game technologies

 

Darkmod running on old idTech4 engine and it ok,this video doesn't make any sense IMHO doom3 better than Crysis2(yes,I play leaked beta version),technology didn't make games fun.... and someday idTech4 goes open-source

 

 

 

 

 

[Tels]

I really don't care about new game technologies

 

Then why are you reading and posting in a thread that discusses "new game technology implemented into the idTech4 engine"?

[Shadowhide]

Then why are you reading and posting in a thread that discusses "new game technology implemented into the idTech4 engine"?

 

implement into the idTech4 engine without source code ? Is that really possible to implement something in engine without source code ?

 

 

I think SEED is just some entity that spawn another enities via some code:)

 

 

[Tels]

implement into the idTech4 engine without source code ? Is that really possible to implement something in engine without source code ?

 

We have the SDK, which is somewhere between 30 and 70% of the original source code (we do not have the renderer, the particle engine, the sound engine, and a few other things I forgot), and it was extended *greatly* by us (so that maybe now 20..40% of the source code is just TDM code).

 

I think SEED is just some entity that spawn another enities via some code:)

 

No, it is much more. You would never get a good performance if every grass or bush or mushroom was a single entity - not only would each entity eat a few KByte memory (an entry in the model-list inside the SEED entity takes only 56 bytes), it would also consume time thinking and issue multiple drawcalls. (And you would run into the entity limit if you wanted f.i. 100000 mushrooms

 

So there is also a ModelGenerator, which can create new models from scratch by combining existing models with different. The thing can even change the models on-the-fly, f.i. when entities need to be hidden, or change their LOD stage.

 

And even this is not enough, as f.i. under certain circumstances you can have a lot of SEED entities, which all control only one entity. So for the near future I am planning to add a SEEDController entity, which takes control of these things, that means you have less entities (one SEED entity overhead, instead of multiple) and also the chance to combine models across SEEDs.

 

One instance is if the mapper places three small SEED entities that both create grass and these are either close together or overlap. In this case you get 3 SEED entities, and 3 StaticMulti entities with grass. With the SEED controller, you get 1 SEEDController, and possible only 1 StaticMulti with all the grass. It will be of course fully dynamic and completely hidden from the mapper who only places the SEED entities and does not care how it gets into the game.

 

SEED is much much more than just spawning a few entities randomly. It's an entire new way to build maps, and have them running inside the engine. IMO it is a new technology (from the point of the aging idTech4 engine).

[Shadowhide]

You would never get a good performance if every grass or bush or mushroom was a single entity

I have a lot of LOD trees in my mission

[Tels]

I have a lot of LOD trees in my mission

 

There is a difference between having 20, 50 or even 100 trees, and having 2000, 5000 or even easily 100000 grass models in your map

[nbohr1more]

I think I should bring up an old topic here...

 

The new Object Detail settings help NHAT 3/3 performance until halfway through the map where I get

 

IDRENDERWORLDLOCAL::RESIZEINTERACTIONTABLE: OVERFLOWED INTERACTIONTABLEWIDTH, DUMPING

 

errors...

 

If there is anything that can be done to adjust that table size, it may have significant consequences for SEED because spawning particle interactions is very much like performing SEED entity spawns...

 

 

http://www.doom3world.org/phpbb2/viewtopic.php?f=4&t=21040&view=next

 

 

You are lucky! I know something about it.

I had this problem in my mod time ago... so I searched everywhere (intenet and souce code) but nothing useful found. Then i did a lot of tests to understand the problem... and the result is that there is sort of table allocated (for interactions) when a map is loaded.

The lenght of this table is about the number of entities that have collisions + about 100.

The "100" is an extra probably used for entities dinamically spawned during the game like gibs and the ones spawned by scripts.

If you exceed the limits of this table that warning will be shown and the game will go much slower. Expecially in big areas with lots of objects.

If you reload the map this time the table will be larger (don't know why) so it was initially hard to reproduce the error.

 

Conclusions:

- it may happen because you script is spawing too many entities ( like i said before )

- yes, emitters seems having something to do with collisions.

- other than always make sure your scripts are removing what you don't need anymore, you could "trick" the game: create an empty box in the space and fill it with about 100 exploding_barrels. At the beginning of the map remove them all --> now you have some more free space in that damn table.

 

I know you are thinking "I'll put there 500 barrels instead of 100 so I'll have more space". The bad news is that it seems like it doesn't work in this way and after that about 100 entities removed you always gain the same amount of space in the table.

 

This idea saved me :-]

 

[Tels]

I think I should bring up an old topic here...

 

The new Object Detail settings help NHAT 3/3 performance until halfway through the map where I get

 

IDRENDERWORLDLOCAL::RESIZEINTERACTIONTABLE: OVERFLOWED INTERACTIONTABLEWIDTH, DUMPING

 

errors...

 

Hm, that is not good, but then, all the "Object Detail" setting does it reducing/increasing the visibility distance (in this map which doesn't use SEED at all), so I wonder why you'd get this error?

 

As for the table, I don't think we can modify it, as the error message is not in the public code.

[nbohr1more]

I got that error before SEED was even a twinkle in your eye

 

I was hoping that SEED would help that one too. It's great that I can make the first half of the map more playable but when I get past those statues... 4FPS all the way till I get to the fortress

 

I think that the "Interaction Table" might be sized in relation to the hardware that is detected (but I hope not...). I was just thinking of all the new stuff you have been able to spawn like combined light entities (etc) and how many "Interactions" are accumulating.

 

I think I heard Greebo talk about the Interaction Limit being the size of the Entity Limit...? Maybe there is a ratio of Interaction Limit to Interaction Table size?

 

If the Interaction Limit can be adjusted, it might be a good idea to raise it anyway to accommodate all the new activity?

[Tels]

If the Interaction Limit can be adjusted, it might be a good idea to raise it anyway to accommodate all the new activity?

 

As I don't even know where the limit is calculated or checked, I simply don't know.

[nbohr1more]

If modwiki's description of r_useInteractionTable is correct:

 

r_useInteractionTable ... create a full entityDefs * lightDefs table to make finding interactions faster

 

Then qhandle_t looks like the SDK virtual to access this Interaction Table if I read this code segment in Renderworld.h correctly:

 

	

//-------------- Entity and Light Defs -----------------

// entityDefs and lightDefs are added to a given world to determine
// what will be drawn for a rendered scene. Most update work is defered
// until it is determined that it is actually needed for a given view.
virtual	qhandle_t		AddEntityDef( const renderEntity_t *re ) = 0;
virtual	void			UpdateEntityDef( qhandle_t entityHandle, const renderEntity_t *re ) = 0;
virtual	void			FreeEntityDef( qhandle_t entityHandle ) = 0;
virtual const renderEntity_t *GetRenderEntity( qhandle_t entityHandle ) const = 0;

virtual	qhandle_t		AddLightDef( const renderLight_t *rlight ) = 0;
virtual	void			UpdateLightDef( qhandle_t lightHandle, const renderLight_t *rlight ) = 0;
virtual	void			FreeLightDef( qhandle_t lightHandle ) = 0;
virtual const renderLight_t *GetRenderLight( qhandle_t lightHandle ) const = 0;

[nbohr1more]

...I wonder if you could hack non-unified surface interactions by updating that table....

[nbohr1more]

Also... you may want to add "noDynamicInteractions" as a LOD characteristic...

[nbohr1more]

Can LOD \ SEED behavior be influenced by proximity to AI rather that the player?

 

If so, the "noDynamicInteractions" attribute could be applied universally and only when AI with dynamic light sources appear will the models be swapped to interactive lighting...

[Tels]

Can LOD \ SEED behavior be influenced by proximity to AI rather that the player?

 

If so, the "noDynamicInteractions" attribute could be applied universally and only when AI with dynamic light sources appear will the models be swapped to interactive lighting...

 

I am not sure what this should be worth. Apart from the fact that dynamic lighting is important (it would look simply wrong without it), the lighting ONLY happens when the player looks at it, anyway. (D3 is like Zen, if there is noone to watch a shadow, the shadow will not be cast...)

 

So you would only save if the player looks at things, but that amounts to almost the same as turning shadows off - it rarely looks good or correct, unless you are very far away. And it might not even improve performance, because all the checking and toggling takes time, too. (Instead of a constant 20 FPS you get then some that fluctuate between almost-all-the-time-22 and disp-down-to-5-briefly, and this is worse than having constant 20 FPS, f.i.)

 

Plus it is an all-or-nothing situation.

 

Once the source goes open, we can look at much better things, like exclude/include lists (where you can exclude certain lights to hit certain objects, or make them cast shadows). That would be more useful, f.i. you could have a candlestick always casting a shadow, BUT not from the candle flame itself, which creates an unrealistic blob-shadow. (Which is actually quite realistic compared to real candle-on-a-plate, but the light texture I made for this was rejected because a player could hold a candle over his head and be still in darkness. Not that this is not correct, but insted of fixing the AI to get alerted by floating candles, the candle light texture was removed again... but I digres.)

[nbohr1more]

It is my understanding that if an entity has "dynamic interactions" enabled that it checks for lighting changes at every render frame. To my mind, this means that each per-stage render-pass is drawn anew every frame incurring significant overdraw penalty for every successive draw.

 

Frame 1:

Draw Base Polys

Check Interaction table

Draw Diffuse polys

Check Interaction table

Draw Texture polys (modify by normals?)

Check interaction table

Draw specular polys (modify by normals?)

 

Each draw multiplying the number of vertices in the scene, every single frame.

 

If you set an object to "noDynamicInteractions" it bakes all those draws down the first render. If it works as it appears from the code comment it should save tons of polys and texture fill. That's why it was done for the Monorail map in Doom 3 AFAIK.

 

I know that Doom 3 has interaction culling that should help reduce that problem but I wonder how aggressive it is? If we could control when "interactions" are allowed it would be a great saving.

 

Mostly though, since the Interaction Table is limited, setting objects to non-dynamic might free-up space in that table.

 

I agree that too many distance checks for proximity to dynamic-lights would probably be a loss but at least it might be a good option for distant LOD objects.

[stgatilov]

nbohr1more, I don't understand the problem here.

 

AFAIU, there is a matrix (table) which has size MAX_ENTITIES x MAX_LIGHTS where there is a record per each entity per each light. Since you simply cannot have more than MAX_ENTITIES entities and more than MAX_LIGHTS lights, why do you need to increase the size of the table? What the hell this table is? What is it used for? Is it internal data structure of the renderer?

 

If you want to know how the rendering is done try to type "r_logfile 1" in console. You'll get huge list of GL calls, annotated with texture filenames and names of rendering phases.

 

P.S. And what do you expect to hear from me?

 

 

 

 

 

[nbohr1more]

Sorry stgatilov.

 

I've been puzzling over that table overflow error almost since NHAT was released.

 

The really bad thing is that I have a feeling that there may be at least two "Interaction Tables" so the overflow error might even be referring to either physics or light interactions or neither ...

 

I just thought you might be able to cook-up some interesting uses for the data in that table if I had correctly located it's interface.

 

[nbohr1more]

Ha!

 

Possible render-to-texture substitute?

 

1) Spawn a rectangular model with a transparent window-style post-process shader

2) Set it to "noDynamicInteractions"

3) Result: Rectangular Model with image baked onto it.

 

This is akin to how Red Dead Revolver does LOD replacement for distant vegetation\trees.

 

The theory is that: Once spawned a "noDynamicInteractions" model only processes the first sequence of "interaction stages" if a post-process shader is included in that set the background image will be baked in along with all the other interactions.

 

(I will have to test this on a transparent window in one of the FM's )

 

[nbohr1more]

I thought about that scenario a bit more and realized that the post-process (even if batched together) would still be too expensive at the LOD transition

 

~~~~~

 

After going back over the batching concept on-whole I to understand it further, ended-up right back at the Ground Zero thread at D3W.

 

It seems the amazing screenshots of foliage they are showing are also using "noDynamicInteractions":

 

 

 

So, Paul found a method of very quickly spawning and rendering simple foliage objects. These objects cannot be lit (must use "blend blend" or similar) and are non-interactive once spawned, but can still be dynamically generated and are ideal for things like dense, non-colliding grass. As long as they all use the same texture sheet, it can pretty much be rendered with a few draw calls. The user can define the groundcover draw distance.

 

 

(eg. They "cheated" )

Edited April 7, 2011 by nbohr1more

[nbohr1more]

More Crazy talks...

 

I was thinking that a kind of "mega-texture lite" could be made by having the SDK code create a virtual file-system that is referenced by dummy material definitions. You could stitch several textures together into a texture sheet in memory then move it to the virtual file-system location referenced by the dummy material definition. In this way you wouldn't need to make tons of texture sheets for different scenes using different varieties of textures and you could save on draw calls by referencing only a few materials per scene... Like a SEED combine for textures.

 

I suppose that would be a bit intensive on the Dark Radiant side... You'd have to have a way to UV map the virtual sheet consistent with the system that renders it in-engine...