Jump to content
The Dark Mod Forums
Sign in to follow this  
vozka

First impressions and overall polish

Recommended Posts

53 minutes ago, OrbWeaver said:

I haven't complained about it much because I assume it is heavily system dependent, but I can actually load and start playing entire other games in the same time it takes just to load a single mission.

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.
Another option is to load things when they are needed. This looks either like stutters when you see new places or like textures and object popping up in the view. I believe a similar problem in megatexture was a major compliant of Rage game. Anyway, I think both options are possible in TDM, but not recommended for players.

Developers dedicate time to the loading times sometimes, but it is not clear how to make it much better.

Quote

A good solution would be to fix issues that are present in the currently used levels and bring them up to date with best current practices.

You are talking about 100+ missions, released by a lot of people (most of them not even being official TDM developers) over 10 years.
Aside from the fact that this would take years of work, we simply cannot do it. Because every FM is an artistic creation of the author. We have no moral right to take his work and start "improving" it as we see fit. The only intervention we can do is small fixes to make mission playable if it got broken.

If you are talking about 3 prepackaged missions, then yes. Surely it is possible to fix issues in them. Although I would imagine the author is already pretty annoyed maintaining them 🤨

Quote

including being on Steam

Well, in my opinion the main reason why TDM is not on Steam is the spirit of the community. The core team wants TDM to be free for everyone, free of money, and free of legal issues and entities. This contradicts the idea of working with any distribution service like Steam.

Quote

I see people recommending System Shock 2 and Deus Ex on sites like reddit at least once per month

It is unfair to compare TDM to System Shock 2 on the polish level. System Shock 2 has preserved to this date in its original state, with its original missions. It is as polished as it was when it was released. And before release, I'm pretty sure a team of people who made it from the ground up polished it from dawn till dusk.
But TDM is constantly changing. Even if you spend all time on polishing it this year, it will go away in 5 years.

Quote

Missions have varying quality, sometimes simply because they were made for an older version of TDM when the state of the art was a bit different. Allowing voting and sorting by popularity would help tremendously. A featured most recently added mission could be shown by default, so that they have a chance to get ratings. 

 

While it sounds like a good idea, I am a bit afraid that this would start ranging FMs into "good" and "bad", which is not very encouraging for mappers, especially for the new ones. And mappers are the most important group of people in TDM community.

 

Anyway, while some of issues may be easy to fix, a lot of them are actually very hard to fix. Like the frobbing system, which I think should declared "unchangeable" and never ever changed again. Or the physics of item pushed into the wall (pretty sure "fixing" to get a bit of "polish" would break some FMs). The system of "installing" FMs is not easy to remove, because I'm afraid Doom 3 engine was not intended to hot-swap its ppk4 files, and loading several FMs at the same time may lead to disaster.

  • Like 2

Share this post


Link to post
Share on other sites
28 minutes ago, stgatilov said:

Another option is to load things when they are needed. This looks either like stutters when you see new places or like textures and object popping up in the view.

I'm pretty annoyed that this is actually quite often the case with Unreal Engine games too, even AAA: textures popping in visibly when the game starts! As for TDM, I think it would be very cool if the mission is loaded in the background while the intro is displayed, as was already suggested here.

  • Like 3

Share this post


Link to post
Share on other sites

I don't know whether this will help with the loading times, but I recently found I had image_useCompression and image_useNormalCompression set to the default 1 (I was wondering why the colour banding on my TGAs was so bad in the game). According to https://www.iddevnet.com/doom3/ these cause TGAs to get compressed at load time. Since TDM (currently) doesn't use pre-compressed normal maps, that might be one cause of loading delay, though I can't say I've noticed a big speed-up (and presumably setting 0 will use more RAM).

The question of nominating recommended 'showcase' FMs comes up occasionally, but besides the point stgatilov makes, it raises other questions: what if A is a stunning, impressive mission but utterly brutal to inexperienced players? What if B is third in a series where the first two no longer impress so much? What if C is great fun but long since ceased to be a visual showcase for TDM? (I was going to suggest that instead of trying to list 'best' missions it might be nice to have an occasional spotlight on missions that made a splash in their day - n year anniversaries for former competition winners, perhaps, since they were already entered into a public ranking - but that idea naturally biases things towards the sometimes less pretty past.)

  • Like 3

Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Share this post


Link to post
Share on other sites
On 3/24/2020 at 10:54 AM, STiFU said:

6.2. As far as I know, that's not supposed to happen. Can you give and example? Maybe a video or a location info (GetViewPos in console)

I just downloaded the 2.08 beta and tried it again and it seems that this issue has been fixed. 

  • Like 2

Share this post


Link to post
Share on other sites
15 hours ago, stgatilov said:

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.

In unreal engine, it seems to me like the lowest mipmap stages and the lowest LOD-Models are loaded first and then the rest is streamed in dynamically when needed and when ressources are free. You can occasionally see a full-resolution texture pop-in from a low-resolution version of it.

So looking at TDM and idTech, I guess VisPortals could be utilized to check which ressources should be loaded when. The question now is, can it be done such that the user does not get any stuttering? Would it be possible to do the loading in a separate low-priority thread such that game performance is not affected? This would need a handling for situations when the full resolution texture / model is not available yet.

By the way, Doom Eternal has bonkers loading time. It's unbelievably quick!

  • Like 2

Share this post


Link to post
Share on other sites
15 hours ago, stgatilov said:

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.

I wonder if some of the slow loading comes from the fragmented nature of the resource storage. A map contains references to various other assets which then have to be looked up and loaded from various other PK4 files: textures from here, models from over here. I don't know exactly how it's implemented at the PK4-loading level, but perhaps the need to seek and extract each of these individual assets from different files results in an inefficient, non-pipelined load process.

Perhaps it could be accelerated by writing out a "map package" which includes all of the assets for a particular map (not just custom ones added by the mapper), and then pre-emptively pushing every asset from this package into memory at the start of the load process, rather than waiting for the map to request particular assets individually which then have to be extracted from the PK4 virtual file system.

  • Like 3

Share this post


Link to post
Share on other sites

How slow is the loading really?

I have a a pc that is optimized for speed and performance, has colling issues.

General load time is a minute.
In that time i get cookies and tea.

I wouldnt recognize a loading problem on my side 😕


Can we have more scary Zombie Horror maps?

Share this post


Link to post
Share on other sites
19 minutes ago, Filizitas said:

I wouldnt recognize a loading problem on my side 😕

I agree, it's not that bad. But combined with the intro videos which sometimes are at least as long if not longer, I would really like the mission to start loading in the background. The old Call of Duty games did this perfectly and as they were based on the Quake engine too, it should be possible with TDM...

  • Like 1

Share this post


Link to post
Share on other sites

I dont know exactly but: Why not load everything first and then play the intro and stuff?
That might be hard todo now that the system is already coded.


Can we have more scary Zombie Horror maps?

Share this post


Link to post
Share on other sites
4 hours ago, OrbWeaver said:

I wonder if some of the slow loading comes from the fragmented nature of the resource storage. A map contains references to various other assets which then have to be looked up and loaded from various other PK4 files: textures from here, models from over here. I don't know exactly how it's implemented at the PK4-loading level, but perhaps the need to seek and extract each of these individual assets from different files results in an inefficient, non-pipelined load process.

The engine opens all pk4 files on load, and creates an index of all files contained in it. When you actually open the file later, it just sets the offset in zip file and starts reading. So everything is OK on zip level, but of course data is read from disk in the order in which you load it, so seeking is done.This is not HDD-friendly, so SSD is very helpful.

By the way, it seems to me that loading times are much longer on AMD GPU than on NVIDIA GPU. I suspect that loading textures to OpenGL is the major fraction of loading times.

2 hours ago, Filizitas said:

I dont know exactly but: Why not load everything first and then play the intro and stuff?
That might be hard todo now that the system is already coded.

Forcing player to wait on TDM start is even worse than waiting on level load.

The idea of loading level in background while player goes through briefing is quite hard to implement. It requires multithreading, and the engine itself does not work with it properly, so there would be plenty of bugs, some of them corrupting random stuff randomly --- very annoying, very hard to reproduce and debug. Duzenko already tried to load textures in separate thread (faster load for debug purposes), and recently we had hard time fixing random weird bugs caused by it. We had to make the whole D3 filesystem threadsafe. And there are many more systems which are not yet suitable to multithreading.

Moreover, it is hard to distribute resources between loading and other engine: for instance, FFmpeg videos can be rather CPU-heavy to decode.

 

I'm not sure that squeezing 25% of load times is worth the effort behind it. If someone really wants to invest time into it, why not. Otherwise, let's pretend they are already OK 😥

  • Like 1
  • Haha 1

Share this post


Link to post
Share on other sites
2 hours ago, stgatilov said:

This is not HDD-friendly, so SSD is very helpful.

Not as much as you'd expect. A while back, Aluminium Haste benchmarked loading times using hdd, ssd and RAM-Disk. Loading time went down from 1:45 min to 1:28 min at most.

 

I also did a quick benchmark just now in order to check what the effect of the zip/pk4 compression is. I used 2.08 Beta 1 and compared it to SVN (where everything is unzipped), while using the same executable. I loaded Saint Lucia for this test.

  • Loadtime zipped: 0:28 min
  • Loadtime unzipped: 0:23 min

Again, not much to gain here.

I wonder how loadtimes would be affected if all normals were converted to RGTC? 

 

  • Like 1

Share this post


Link to post
Share on other sites
On 3/25/2020 at 7:07 PM, VanishedOne said:

I don't know whether this will help with the loading times, but I recently found I had image_useCompression and image_useNormalCompression set to the default 1 (I was wondering why the colour banding on my TGAs was so bad in the game). According to https://www.iddevnet.com/doom3/ these cause TGAs to get compressed at load time. Since TDM (currently) doesn't use pre-compressed normal maps, that might be one cause of loading delay, though I can't say I've noticed a big speed-up (and presumably setting 0 will use more RAM).

This actually solved the slow loading problem for me, so thanks for the suggestion.

I guess that raises the question of why image_useNormalCompression is defaulting to 1 in the first place, given the problems it causes. Perhaps it is intended as a "safe" default for people with low-memory GPUs.

  • Like 1

Share this post


Link to post
Share on other sites

If I recall correctly, there used to be a menu option to enable / disable.  I haven't been involved much in recent times, has it been removed?  

Edit:  Just checked and don't see any sign of it.  Probably removed at some point with recent updates / optimizations.  Definitely a good thing to have for people who need it, but shouldn't be enabled by default.  If possible, I would say put the a texture compression setting in the menu for people who can accept the trade off of performance over loading time.

  • Like 1

Share this post


Link to post
Share on other sites
48 minutes ago, OrbWeaver said:

This actually solved the slow loading problem for me, so thanks for the suggestion.

I guess that raises the question of why image_useNormalCompression is defaulting to 1 in the first place, given the problems it causes. Perhaps it is intended as a "safe" default for people with low-memory GPUs.

Oh wow, yes, that's a huge gain in loading time. My loading time for Saint Lucia in 2.08 Beta 1 decreased from 0:28 min to 0:17 min when run from my regular HDD. Wow! Just out of curiousity, I also copied the game to my lightning fast m2-SSD and surprisingly, there was no difference in loading time: It was 0:17 min again! Tried it numerous times in both cases, to verify. I conclude, that hdd access are not the bottleneck in regards to loading time.

Share this post


Link to post
Share on other sites
On 3/30/2020 at 3:46 PM, STiFU said:

Oh wow, yes, that's a huge gain in loading time. My loading time for Saint Lucia in 2.08 Beta 1 decreased from 0:28 min to 0:17 min when run from my regular HDD. Wow! Just out of curiousity, I also copied the game to my lightning fast m2-SSD and surprisingly, there was no difference in loading time: It was 0:17 min again! Tried it numerous times in both cases, to verify. I conclude, that hdd access are not the bottleneck in regards to loading time.

I got even more dramatic gains than you did.

I have what used to be a mid-end laptop about 8 years ago, an Ivy Bridge i5 CPU upgraded with enough RAM and an SSD. 

With a bigger mission, Volta and the Stone, the loading time was 1:48 m with compression on and only 31 seconds!! with image_useCompression and image_useNormalCompression set to 0.

Looking at the FPS at the beginning of the mission, I seem to have gotten a 2 FPS reduction with the compression off, but that might have just been statistical noise, I would need to do more tests to find out if there's an actual difference. But note that I have a Radeon 7650M, which is as I said 8 years old and basically a rebranded 5xxx radeon chip - I don't know whether its performance is even relevant for you. 

On 3/26/2020 at 5:38 PM, STiFU said:

I also did a quick benchmark just now in order to check what the effect of the zip/pk4 compression is. I used 2.08 Beta 1 and compared it to SVN (where everything is unzipped), while using the same executable. I loaded Saint Lucia for this test.

  • Loadtime zipped: 0:28 min
  • Loadtime unzipped: 0:23 min

That's 18% reduction in loading time, not as dramatic as image compression, but imo nothing to scoff at. It might be more on slower CPUs like mine. I would think that it's worth considering depending on what the size difference is.

  • Like 1

Share this post


Link to post
Share on other sites

My understanding (and I might well be out of date on this) is that image_useNormalCompression should default to 0, because we do not supply precompressed normal maps and compression of normal maps tends to look bad, but image_useCompression should be left on (1), because compression of regular images is normal practice and these are supplied as pre-compressed DXT files anyway (in fact they may ONLY be supplied as precompressed DXT files, which means that some combinations of disabling image compression will result in all-black environments since the non-compressed original images cannot be found).

  • Like 1

Share this post


Link to post
Share on other sites
46 minutes ago, OrbWeaver said:

My understanding (and I might well be out of date on this) is that image_useNormalCompression should default to 0, because we do not supply precompressed normal maps and compression of normal maps tends to look bad, but image_useCompression should be left on (1), because compression of regular images is normal practice and these are supplied as pre-compressed DXT files anyway (in fact they may ONLY be supplied as precompressed DXT files, which means that some combinations of disabling image compression will result in all-black environments since the non-compressed original images cannot be found).

According to the descriptions given in the game console, image_usePrecompressedTextures is the cvar to 'use .dds files if present'.

If your only TGAs are things like light textures (which tend to have forceHighQuality set in the material) and cubemaps, image_useCompression shouldn't make much difference, but working with TGA assets I haven't compressed to DDS yet, I found they had horrible colour banding on my system and only looked decent with image_useCompression turned off.


Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Share this post


Link to post
Share on other sites

I'm just gonna throw in my two cents on the matter of the missions included in the package.

I personally think A New Job and Tears of St Lucia are both great missions and should pretty much remain as is.  The training mission... not so much. It is a sandbox sort of environment that is not so fitting with the rest, at least as far as tutorials go. At some point I would like to see a new training mission on a more step-by-step tutorial basis, preferably starting at the Thief's house and then venturing out on the streets to accomplish a very simple objective. This would better introduce the main character and set the tone for the following missions. As for the original training mission, it should be repurposed and renamed as a practice sandbox or something of the kind.

More importantly, I think the additional missions concluding the St. Lucia story should be finished. Like OP says, it would be lovely for the packaged product to have a small story with a beginning and an end. Isn't it possible to pull a community effort to make this a reality? I think this should have been at least considered after so many years...

Finally, an additional suggestion, and as a complement to a possibly finished St. Lucia campaign: I think three standalone FMs of good reputation should be included in the main package as to give the player a more complete experience, ideally one short, one medium, and one long. It would also serve as a showcase of the many possibilities of map making and of the variety contained within TDM, while recognizing long-standing FM creators within the community. All in all, the goal here would be to give the first time player a stellar impression of some of the best things the game has to offer.

Edited by zergrush
  • Like 1

Share this post


Link to post
Share on other sites

That is looking amazing Springheel, thank you for sharing. Hopefully someone will be able to pick it up in the future.

Share this post


Link to post
Share on other sites
17 hours ago, zergrush said:

The training mission... not so much. It is a sandbox sort of environment that is not so fitting with the rest, at least as far as tutorials go. At some point I would like to see a new training mission on a more step-by-step tutorial basis, preferably starting at the Thief's house and then venturing out on the streets to accomplish a very simple objective.

I completely agree with that! The training mission has no story at all and can't even be finished, so it might give a bad first impression to players unless they continue...

Edited by wesp5

Share this post


Link to post
Share on other sites
27 minutes ago, wesp5 said:

I completely agree with that! The training mission has no story at all and can't even be finished, so it might give a bad first impression to players unless they continue...

Isn't it why New Job FM was created?

  • Like 1

Share this post


Link to post
Share on other sites

I think the present training mission should remain as is as a simple practice sandbox, but it would be nice to have a real step-by-step tutorial mission. A New Job does good work of introducing the player to navigation , searching for alternate paths, and simple narrative, but it does not work as a tutorial in the classical sense.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...