[Resolved in TDM 2.10] When Will Loading Times Be Too Long?

[Mik James]

As Tdm missions have become more complex and larger the load times have continued to increase. This would be alright if they were reasonable to begin with.

 

Playing Crucible of Omens running off a Samsung 850 evo ssd I get about 1 minute of loading time. My best uneducated guess is too many assets are loaded at mission start rather than dynamically loading them in over time?

 

My other thought is this engine may not be suitable for dynamically loading assets and by extension large in game areas?

I suppose it was originally running Doom 3, a corridor style shooter from what I've seen.

 

Just speculation but it definitely negatively impacts the experience for me. Most of my games load up missions within 10-20 seconds. Seems like the only way to enjoy Dark mod will be to fire up the ram disk.

[nbohr1more]

There are plans to get RGTC compression working for normal maps and that should reduce load times for two reasons:

 

1) No load-time mipmap generation

2) Less data to load due to compression

 

It's on the todo list.

[AluminumHaste]

All missions assets are loaded in at mission start. Which is one of the reasons that it takes so long.

I already did RAMDisk testing two years ago:

 

http://forums.thedarkmod.com/topic/16215-comparison-of-3-different-storage-techs-with-tdm/

[Mik James]

Indeed I noticed little if any realworld difference on a ram disk myself. Maybe i've become spoiled by ssd's and better optimized engines but I just can't get into Dark Mod as it is. It also doesn't help that the dark engine games have received so many improvements over the last few years.

 

Probably best to wait until the Crucible campaign is complete anyways

[demagogue]

If it's any consolation, the levels load faster now than they did in v.1.0. They have done some optimizations already.

 

And at least our textures don't pop in, something I hated with idTech5.

[Bikerdude]

Playing Crucible of Omens running off a Samsung 850 evo ssd I get about 1 minute of loading time. My best uneducated guess is too many assets are loaded at mission start rather than dynamically loading them in over time?

I have the 840 evo with an i7 3770k and it takes about 30-45s to load.

[Nico A.]

It's on the todo list.

That's awesome.

[wesp5]

I still would recommend to think about loading the assets in the background while the mission intro is shown because this usually takes some time too and both amounts to a lot of waiting before a mission starts. I think there were some issues with setting the difficulty, but maybe different entities for different difficulties could be set dynamically after the map has been loaded? Also I want to remind you that is would be great to have a "press x or y to skip intro" info on the screen while the intro plays so you know how to skip it without trying random keys or doing research on the internet...

[Goldwell]

It's on the todo list.

 

is that on the 2.04 todo list?

[Nico A.]

I still would recommend to think about loading the assets in the background while the mission intro is shown

+1.

 

Please do this if this is somehow possible. Even if the mission cannot be loaded completely, I think that would quick up things a lot.

Edited January 9, 2016 by Nico A.

[Mik James]

I have the 840 evo with an i7 3770k and it takes about 30-45s to load.

Hmm, I didn't time it. I only have an fx 4350 though so it may have something to do with single thread performance.

I patiently await the improvements though

[AluminumHaste]

Yeah my conclusion of the hard drive tests was that the single threaded performance of your CPU is the biggest contributor to initial load times.

Video card has almost nothing to do with it.

The Radeon 7950, GeForce 670 and 780ti almost all had the same load times.

[jaxa]

I still would recommend to think about loading the assets in the background while the mission intro is shown because this usually takes some time too and both amounts to a lot of waiting before a mission starts. I think there were some issues with setting the difficulty, but maybe different entities for different difficulties could be set dynamically after the map has been loaded? Also I want to remind you that is would be great to have a "press x or y to skip intro" info on the screen while the intro plays so you know how to skip it without trying random keys or doing research on the internet...

 

Neat idea. It should be disable-able in a menu or config file in case it causes instability/lag.

[Azaran]

At least as of the latest update, the wait time actually matches the progress bar. Before that it didn't, which was really annoying. If that was improved, I'm sure eventually they'll manage to shorten the wait itself

[nbohr1more]

 

is that on the 2.04 todo list?

 

No, 2.04 is too close to final build version. It's going to be a mostly a bug-fix release since some of the larger changes were taking too long

and leaving players with some critical bugs like the "card player save state bug" and a nasty "SEED related memory allocation" behavior.

Load times will be shorter since the particle bounds fix is in there but not dramatically so.

 

RGTC support should probably get bumped to priority 1 or 2 for 2.05. We've been discussing it for over 4yrs now AFAIK and it's really been

brought to the fore with the size of Behind Closed Doors.

[STiFU]

Indeed I noticed little if any realworld difference on a ram disk myself. Maybe i've become spoiled by ssd's and better optimized engines but I just can't get into Dark Mod as it is. It also doesn't help that the dark engine games have received so many improvements over the last few years.

 

Probably best to wait until the Crucible campaign is complete anyways

It's your loss really. Some PC gamers are so spoiled when it comes to loading times. Check out any gaming console. The loading times in most games exceed the 1-minute mark by far. Also, after the first load, reloading is a lot faster.

[demagogue]

Man in my day we waited a dogdamn half hour for Bard's Tale 2 to load up on the C64.

[Deadlove]

It's your loss really. Some PC gamers are so spoiled when it comes to loading times. Check out any gaming console. The loading times in most games exceed the 1-minute mark by far. Also, after the first load, reloading is a lot faster.

 

Agreed. After first load, quick loads take only a couple seconds.

[SteveL]

No, 2.04 is too close to final build version. It's going to be a mostly a bug-fix release since some of the larger changes were taking too long

and leaving players with some critical bugs like the "card player save state bug" and a nasty "SEED related memory allocation" behavior.

Not to mention sound pops in this year's big maps like Sir Talbot's and House, and the bug with AI sitting their way out of the world. Those will be fixed too.

 

Load times are definitely a nice to have, and grayman and I have both dug into it in the last year or so looking for options and testing a few things, but it's not top priority because the most obvious changes are huge work to implement -- background loading and processing of assets, for example, presumably ordered by how close the asset is to the player's current position -- that's a complicated change, and it feels like less of a priority than new features (imo). nbohr1more's suggestion of a texture load fix is an exception: we can try that without boiling the ocean, and there will be other quick fixes too, like the one to speed up particle loading.

 

I've had confusing results from cpu and gpu profiling during the load procedure, which is another reason I haven't worked on it much: no clear lead.

[grayman]

I still would recommend to think about loading the assets in the background while the mission intro is shown because this usually takes some time too and both amounts to a lot of waiting before a mission starts. I think there were some issues with setting the difficulty, but maybe different entities for different difficulties could be set dynamically after the map has been loaded? Also I want to remind you that is would be great to have a "press x or y to skip intro" info on the screen while the intro plays so you know how to skip it without trying random keys or doing research on the internet...

 

I took a long hard look at threaded loading in 2.03, but the current design depends too much on sequential processes to just tweak it a bit to use threads.

 

A new design from scratch is necessary, and I abandoned the effort in a moment of sanity.

[wesp5]

I took a long hard look at threaded loading in 2.03, but the current design depends too much on sequential processes to just tweak it a bit to use threads.

That's a shame, because this is how the modern blockbusters hide their loading times behind pretty movies, like the Call of Duty series...

Edited January 14, 2016 by wesp5

[Hiradur]

I took a quick look at the long loading times in TDM with Valgrind and it looks like the inflate_fast function (for unzipping) takes up a large portion of the loading time according to Valgrind:

 

 

I had a quick glimpse at the function here: https://github.com/revelator/The-Darkmod-Experimental/blob/7d86b82fce9eb587952c4963b928fd7974ecfa78/src/framework/Unzip.cpp#L3475

and it looks like it's very old and optimized for old hardware. I've never looked at the deflate/inflate algorithm before but maybe there are faster ways to do this on modern hardware.

[New Horizon]

Would be awesome if that could be sped up.

 

 

I took a quick look at the long loading times in TDM with Valgrind and it looks like the inflate_fast function (for unzipping) takes up a large portion of the loading time according to Valgrind:

 

 

I had a quick glimpse at the function here: https://github.com/revelator/The-Darkmod-Experimental/blob/7d86b82fce9eb587952c4963b928fd7974ecfa78/src/framework/Unzip.cpp#L3475

and it looks like it's very old and optimized for old hardware. I've never looked at the deflate/inflate algorithm before but maybe there are faster ways to do this on modern hardware.

[AluminumHaste]

Would be nice to just grab to source for 7zip at least it takes advantage of modern CPU extensions.

[Goldwell]

Man in my day we waited a dogdamn half hour for Bard's Tale 2 to load up on the C64.

 

Ha.. Luxury!

 

http://www.youtube.com/watch?v=Xe1a1wHxTyo