[2.10] Entity limit removed

[stgatilov]

The recent changes removed a lot of constraints on the entity limit, and now it is free to be raised up.
Technically speaking, it is not removed, but it can be raised so high that it won't bother you anymore.

On the screenshot, you can see the game working fine with 120022 entities (almost all of them func_static):

These changes are included in the latest dev build, which can be freely obtained via tdm_installer.
In the end, I set the limit to 65536 entities. I hope you won't need more

 

The main point that I'd like to make here is: optimizations against entity limit should not be used any more!
You don't need to worry that something would occupy an entity slot.
You should never inline entities --- that's a very bad idea!

Of course, many of known optimization techniques have other benefits.
For instance, combining several models into one manually or via SEED reduces rendering overhead and should make the game faster. Just make sure that the merged objects are very close to each other. Good news is that now you are not forced to do so, and can postpone it indefinitely.

[peter_spy]

Holy shit, fuck yeah! Finally!

Edit: All in all, it's a balancing act. While having thousands of tiny models in your visleaf will be bad, combining everything into large models can hurt performance too. If the model is bigger than what you can see in your POV, and there are lights hitting it and casting shadows, those addidional DCs and shadows will impact your performance, even if you don't see such lights.

Edited March 2, 2021 by peter_spy

[HMart]

stgatilov I'm right to assume that to increase the entity limit to 65536 is just the case of setting the GENTITYNUM_BITS  to 16? There's no need to change anything else in the engine to support that amount of entities? I want to do it on my version of the dewm3 engine if is not too complicated for me. 

Edited March 2, 2021 by HMart

[demagogue]

Nice to see this limit go so that the size of an FM is defined by an author's ambition and not some technical limitation.

[nbohr1more]

8 hours ago, HMart said:

stgatilov I'm right to assume that to increase the entity limit to 65536 is just the case of setting the GENTITYNUM_BITS  to 16? There's no need to change anything else in the engine to support that amount of entities? I want to do it on my version of the dewm3 engine if is not too complicated for me. 

TDM may be more susceptible to more fragility around entities due to some of our more complex designs and alterations to the base game.

stgatilov made a number of infrastructural changes to handle the additional overhead that increasing entities can incur.

This tracker has a history of the work:

https://bugs.thedarkmod.com/view.php?id=5439

and you can compare that to the git revision history here:

https://github.com/fholger/thedarkmod/commits/master

[HMart]

Like I suspected that is way more complicated than I thought but I will try it none the less but to tell the truth, I don't think I will ever need such large amounts of entities.

[stgatilov]

9 hours ago, HMart said:

stgatilov I'm right to assume that to increase the entity limit to 65536 is just the case of setting the GENTITYNUM_BITS  to 16? There's no need to change anything else in the engine to support that amount of entities? I want to do it on my version of the dewm3 engine if is not too complicated for me. 

You can try: it will probably work.

I would say it's better to do r9167 about EntityPtr, since otherwise it will be hard-limited by 32768 or 65536 (two entity indices stored in one 32-bit number).

Most of the other changes are about optimization: the original code often creates local array with NUM_GENTITIES elements, which is not very good for performance if you raise it to 64K. But perhaps it's not so critical, since all these arrays are left uninitialized: it's only stack caching performance that's lost. That's the reason behind the largest commit r9158.
Without commit r9156 the map with many entities will shutdown slowly.

You will also need to port the interaction table from TDM, because the original [Enum x Lnum] matrix will eat awful amount of memory, also hard-capping the number of entities at something like 16K.

Network code will probably give some headache. In TDM it is partly removed, but there were some large arrays, like with 32 * NUM_GENTITIES elements. Note sure how they are used, but keep an eye on multiplayer performance in case there are many entities.

[Geep]

On 3/2/2021 at 9:52 AM, stgatilov said:

You should never inline entities --- that's a very bad idea!

Could you expound a bit on why this is true? For one of my planned wiki articles. Thanks.

[stgatilov]

7 hours ago, Geep said:

Could you expound a bit on why this is true? For one of my planned wiki articles. Thanks.

Because inlined geometry of a model goes through dmap processing.
The more such models you have, the longer dmap takes. Chance of complicated geometric algorithms failing also increases.

Are there any benefits of inlining?
One less entity is not a reason any more (and probably combining close models into one is better than inlining models). Yes, dmap can remove some invisible geometry and precompute stencil shadows. But with the costs outlined above.