[2.12] Multi-addons support

[Dragofer]

In the current user addons system, only one addon could ever be installed at a time. This was because every addon would try to overwrite tdm_user_addons.script, and only the one in the alphabetically latest .pk4 would win. Merging addons was not a practical solution for most since this required a similar level of proficiency as actually creating the addons.

Coming to the next dev build for 2.12, this limitation has been lifted by automatically including or calling all script files or functions that start with a specific prefix. Addon creators will need to choose unique names, however, which could for example include one's username.

For now, backwards compatibility with the old style of addons will be maintained so they will still work, but they will need to be updated before several can be installed at once.

Note to creators: as this may result in a surge of popularity of user addons to "mod the mod", it feels like some remarks are necessary:

• it's important to communicate to players downloading the addons that these are to be used at one's own risk and there's no certain way to ensure that no mission will break.
• releasing a stable mission with the default TDM package is hard enough. Addons create another layer of complexity that is out of the authors' hands. So while it's okay to ask, please respect the authors' decision not to support addons.
• savegames will only work with the exact combination and version of addons that they were created with.

Updated instructions for creating addons can be found here: https://wiki.thedarkmod.com/index.php?title=A_to_Z_Scripting:_Script_addons_for_players

[Dragofer]

Pinging addon creators that are known to me:

@MirceaKitsune@wesp5@snatcher@Dragofer

[jaxa]

I hear these addons make saves incompatible between installations (relevant when swapping saves during beta testing)? That should be noted if accurate.

[Dragofer]

6 minutes ago, jaxa said:

I hear these addons make saves incompatible between installations (relevant when swapping saves during beta testing)? That should be noted if accurate.

This would be accurate since saves are specific to the scripts (and TDM build) that were installed at the time, so getting a newer version of an addon will make saves incompatible. Changing other custom files like entityDefs will probably cause issues as well. I added this to the creators' notes.

[snatcher]

This is very much appreciated. Many thanks @Dragofer & Co!

Thinking of the "transition period", I have a question: what happens if by the time 2.12 comes out an add-on has both a "tdm_user_addons.script" and a "tdm_user_addons_spiders.script"?

1 hour ago, Dragofer said:

Addon creators will need to choose unique names, however, which could for example include one's username.

First come first served, I guess.

In the case of conflict the Modder (Addoner?) publishing his/her work last should be around listening and can be warned and take action.

[Dragofer]

2 hours ago, snatcher said:

Thinking of the "transition period", I have a question: what happens if by the time 2.12 comes out an add-on has both a "tdm_user_addons.script" and a "tdm_user_addons_spiders.script"?

There's nothing really hindering it, you just have to make sure that both scripts dont try to run at the same i.e. by starting with a version check to pick the right script.

2 hours ago, snatcher said:

First come first served, I guess.

In the case of conflict the Modder (Addoner?) publishing his/her work last should be around listening and can be warned and take action.

Core scripts already include the identifier "tdm" to minimise the risk of conflicts with FM scripts (originally: with Doom3 scripts). You could make up your own identifier such as "snatch", and in a community of this size it'll be unlikely that anyone else will think of the exact same one.

[snatcher]

21 hours ago, Dragofer said:

There's nothing really hindering it, you just have to make sure that both scripts dont try to run at the same i.e. by starting with a version check to pick the right script.

Not sure I was clear enough, sorry, but it is ok. With the information at hand I understand updating add-ons today to be 2.12-ready is a pointless exercise.

Let's then resume this topic sometime in February. Thanks for your work.

[snatcher]

Developers / Mappers / Modders / Addoners be warned. Boundaries are closer than you think.

Tag: Exceeded global memory size (196608 bytes)

[stgatilov]

41 minutes ago, snatcher said:

Tag: Exceeded global memory size (196608 bytes)

This is total size of all global variables.

Here are the limits:

#define MAX_STRING_LEN		128
#define MAX_GLOBALS			196608			// in bytes
#define MAX_FUNCS			3072
#define MAX_STATEMENTS		81920			// statement_t - 18 bytes last I checked

I suppose everything except for MAX_STRING_LEN can be raised.

Perhaps create an issue for 2.13?

[snatcher]

6 hours ago, stgatilov said:

Perhaps create an issue for 2.13?

And shelve new initiatives for 12 months.

At a glance, the below script-heavy missions are orbiting close to the boundaries:

• A house of locked secrets
• Hazard Pay
• Iris
• Painter's Wife
• Shadows of Northdale ACT I
• Seeking Lady Leicester
• Written in stone

All it takes is a little push.

Each new release of TDM comes with new core scripts and sooner or later something needs to happen. Let's hope the next "big one" doesn't get impacted.

[stgatilov]

I tried to debug this.

It turns out that "global variables" includes all immediate values (including string literals which take 128 bytes each) and all global function declarations (4 byte per function, including even engine events).

I see 119240 bytes before starting Iris, and 124924 bytes after starting it.
On Bakery Job, I see 109680 before starting game, 109700 bytes after starting it.

So It seems that core game takes about 110K, but Iris itself takes only about 10K, and the limit is at 196K.
Does not sound like a big problem for missions.
With addons, you have to compile all of them at once, and perhaps there is a lot of code, so you reach the limit.

Ok, let's double MAX_GLOBALS for 2.12.

[snatcher]

Very good assessment. Thank you.

I was surprised to reach the limit and I think it had to do with this:

10 hours ago, stgatilov said:

(including string literals which take 128 bytes each)

What I am trying to do looks very ugly (lack of arrays, unable to feed data from files) but I don't consider I abused of it (yet).

10 hours ago, stgatilov said:

Ok, let's double MAX_GLOBALS for 2.12.

Whoo-hoo! Thanks a a lot!

Edited February 4, 2024 by snatcher

[datiswous]

Quote

If your addon contains further files that you want to overwrite core asset files with, then your addon .pk4 name should sort alphabetically after tdm_base01.pk4, for example z_arachnophobia.pk4.

Quote

However, addons that have files, script functions or .pk4s with the same name will not be compatible.

This if from the wiki page. It sounds like these contradict each other. Or does it mean that same names are not compatible because they would override each other?

[Dragofer]

On 11/1/2024 at 9:13 PM, datiswous said:

This if from the wiki page. It sounds like these contradict each other. Or does it mean that same names are not compatible because they would override each other?

I don't think there's a contradiction: if an addon has the same .pk4/script/asset names as another addon they will interfere with each other's function if both are installed. However, addons can intentionally overwrite core .pk4 files as an intended feature.