[Resolved in TDM 2.10] When Will Loading Times Be Too Long?

[AluminumHaste]

I took a quick look at the long loading times in TDM with Valgrind and it looks like the inflate_fast function (for unzipping) takes up a large portion of the loading time according to Valgrind:

 

 

I had a quick glimpse at the function here: https://github.com/revelator/The-Darkmod-Experimental/blob/7d86b82fce9eb587952c4963b928fd7974ecfa78/src/framework/Unzip.cpp#L3475

and it looks like it's very old and optimized for old hardware. I've never looked at the deflate/inflate algorithm before but maybe there are faster ways to do this on modern hardware.

 

Unfortunately the reality of it seems to be different.

I tested this by loading up the largest mission we have: Crucible of Omens, in TDM and timing how long it takes to go from "Start Mission" to "Click to Start" appearing.

I then unpacked the ZIP file into the BCD directory to remove unzipping as a bottleneck in the loading process, and the times were the same, give or take a second or 2.

Here is what the directory looks like:

 

PK4:

 

Unzipped:

            Run 1        Run 2        Run 3        Run 4
PK4:        53.32 sec    53.32 sec    52.31 sec    53.10 sec
Unzipped:    52.12 sec    51.55 sec    51.33 sec    51.76 sec

[Hiradur]

 

 

Unfortunately the reality of it seems to be different.

I tested this by loading up the largest mission we have: Crucible of Omens, in TDM and timing how long it takes to go from "Start Mission" to "Click to Start" appearing.

I then unpacked the ZIP file into the BCD directory to remove unzipping as a bottleneck in the loading process, and the times were the same, give or take a second or 2.

 

Thanks for the test but what about all the pk4 files coming with TDM itself?

 

If you look at the Doom 3 BFG code you'll see that the Unzip.cpp changed a bit since Doom 3 and that inflate_fast() was removed: https://github.com/id-Software/DOOM-3-BFG/blob/master/neo/framework/Unzip.cpp

I don't know how BFG's loading times compare to Doom 3 though.

[nbohr1more]

BFG uses binary assets. It's not quite comparable to what we have. That said, you may have a point about packed mod assets loading too. Though the SVN where

we do development has unpacked assets and mission loading isn't substantially faster there.

[AluminumHaste]

 

Thanks for the test but what about all the pk4 files coming with TDM itself?

 

If you look at the Doom 3 BFG code you'll see that the Unzip.cpp changed a bit since Doom 3 and that inflate_fast() was removed: https://github.com/id-Software/DOOM-3-BFG/blob/master/neo/framework/Unzip.cpp

I don't know how BFG's loading times compare to Doom 3 though.

 

That test I did was with all the files unzipped, I'm using the SVN version which does not have pk4's.

[Hiradur]

I see, well, it was worth a shot.

[AluminumHaste]

Absolutely Hiradur, thanks for trying