Jump to content
The Dark Mod Forums

First impressions and overall polish


vozka

Recommended Posts

53 minutes ago, OrbWeaver said:

I haven't complained about it much because I assume it is heavily system dependent, but I can actually load and start playing entire other games in the same time it takes just to load a single mission.

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.
Another option is to load things when they are needed. This looks either like stutters when you see new places or like textures and object popping up in the view. I believe a similar problem in megatexture was a major compliant of Rage game. Anyway, I think both options are possible in TDM, but not recommended for players.

Developers dedicate time to the loading times sometimes, but it is not clear how to make it much better.

Quote

A good solution would be to fix issues that are present in the currently used levels and bring them up to date with best current practices.

You are talking about 100+ missions, released by a lot of people (most of them not even being official TDM developers) over 10 years.
Aside from the fact that this would take years of work, we simply cannot do it. Because every FM is an artistic creation of the author. We have no moral right to take his work and start "improving" it as we see fit. The only intervention we can do is small fixes to make mission playable if it got broken.

If you are talking about 3 prepackaged missions, then yes. Surely it is possible to fix issues in them. Although I would imagine the author is already pretty annoyed maintaining them 🤨

Quote

including being on Steam

Well, in my opinion the main reason why TDM is not on Steam is the spirit of the community. The core team wants TDM to be free for everyone, free of money, and free of legal issues and entities. This contradicts the idea of working with any distribution service like Steam.

Quote

I see people recommending System Shock 2 and Deus Ex on sites like reddit at least once per month

It is unfair to compare TDM to System Shock 2 on the polish level. System Shock 2 has preserved to this date in its original state, with its original missions. It is as polished as it was when it was released. And before release, I'm pretty sure a team of people who made it from the ground up polished it from dawn till dusk.
But TDM is constantly changing. Even if you spend all time on polishing it this year, it will go away in 5 years.

Quote

Missions have varying quality, sometimes simply because they were made for an older version of TDM when the state of the art was a bit different. Allowing voting and sorting by popularity would help tremendously. A featured most recently added mission could be shown by default, so that they have a chance to get ratings. 

 

While it sounds like a good idea, I am a bit afraid that this would start ranging FMs into "good" and "bad", which is not very encouraging for mappers, especially for the new ones. And mappers are the most important group of people in TDM community.

 

Anyway, while some of issues may be easy to fix, a lot of them are actually very hard to fix. Like the frobbing system, which I think should declared "unchangeable" and never ever changed again. Or the physics of item pushed into the wall (pretty sure "fixing" to get a bit of "polish" would break some FMs). The system of "installing" FMs is not easy to remove, because I'm afraid Doom 3 engine was not intended to hot-swap its ppk4 files, and loading several FMs at the same time may lead to disaster.

  • Like 2
Link to comment
Share on other sites

28 minutes ago, stgatilov said:

Another option is to load things when they are needed. This looks either like stutters when you see new places or like textures and object popping up in the view.

I'm pretty annoyed that this is actually quite often the case with Unreal Engine games too, even AAA: textures popping in visibly when the game starts! As for TDM, I think it would be very cool if the mission is loaded in the background while the intro is displayed, as was already suggested here.

  • Like 3
Link to comment
Share on other sites

I don't know whether this will help with the loading times, but I recently found I had image_useCompression and image_useNormalCompression set to the default 1 (I was wondering why the colour banding on my TGAs was so bad in the game). According to https://www.iddevnet.com/doom3/ these cause TGAs to get compressed at load time. Since TDM (currently) doesn't use pre-compressed normal maps, that might be one cause of loading delay, though I can't say I've noticed a big speed-up (and presumably setting 0 will use more RAM).

The question of nominating recommended 'showcase' FMs comes up occasionally, but besides the point stgatilov makes, it raises other questions: what if A is a stunning, impressive mission but utterly brutal to inexperienced players? What if B is third in a series where the first two no longer impress so much? What if C is great fun but long since ceased to be a visual showcase for TDM? (I was going to suggest that instead of trying to list 'best' missions it might be nice to have an occasional spotlight on missions that made a splash in their day - n year anniversaries for former competition winners, perhaps, since they were already entered into a public ranking - but that idea naturally biases things towards the sometimes less pretty past.)

  • Like 3

Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Link to comment
Share on other sites

On 3/24/2020 at 10:54 AM, STiFU said:

6.2. As far as I know, that's not supposed to happen. Can you give and example? Maybe a video or a location info (GetViewPos in console)

I just downloaded the 2.08 beta and tried it again and it seems that this issue has been fixed. 

  • Like 2
Link to comment
Share on other sites

15 hours ago, stgatilov said:

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.

In unreal engine, it seems to me like the lowest mipmap stages and the lowest LOD-Models are loaded first and then the rest is streamed in dynamically when needed and when ressources are free. You can occasionally see a full-resolution texture pop-in from a low-resolution version of it.

So looking at TDM and idTech, I guess VisPortals could be utilized to check which ressources should be loaded when. The question now is, can it be done such that the user does not get any stuttering? Would it be possible to do the loading in a separate low-priority thread such that game performance is not affected? This would need a handling for situations when the full resolution texture / model is not available yet.

By the way, Doom Eternal has bonkers loading time. It's unbelievably quick!

  • Like 2
Link to comment
Share on other sites

15 hours ago, stgatilov said:

I wonder how is it done in modern games.
I have an impression that developers push the loading responsibility onto artists' shoulders. Basically, they most likely put some markers where start "preload" a new bunch of textures and models. This won't help TDM because nobody will add such markers to existing missions.

I wonder if some of the slow loading comes from the fragmented nature of the resource storage. A map contains references to various other assets which then have to be looked up and loaded from various other PK4 files: textures from here, models from over here. I don't know exactly how it's implemented at the PK4-loading level, but perhaps the need to seek and extract each of these individual assets from different files results in an inefficient, non-pipelined load process.

Perhaps it could be accelerated by writing out a "map package" which includes all of the assets for a particular map (not just custom ones added by the mapper), and then pre-emptively pushing every asset from this package into memory at the start of the load process, rather than waiting for the map to request particular assets individually which then have to be extracted from the PK4 virtual file system.

  • Like 3
Link to comment
Share on other sites

How slow is the loading really?

I have a a pc that is optimized for speed and performance, has colling issues.

General load time is a minute.
In that time i get cookies and tea.

I wouldnt recognize a loading problem on my side 😕

Can we have more scary Zombie Horror maps?

Link to comment
Share on other sites

19 minutes ago, Filizitas said:

I wouldnt recognize a loading problem on my side 😕

I agree, it's not that bad. But combined with the intro videos which sometimes are at least as long if not longer, I would really like the mission to start loading in the background. The old Call of Duty games did this perfectly and as they were based on the Quake engine too, it should be possible with TDM...

  • Like 1
Link to comment
Share on other sites

4 hours ago, OrbWeaver said:

I wonder if some of the slow loading comes from the fragmented nature of the resource storage. A map contains references to various other assets which then have to be looked up and loaded from various other PK4 files: textures from here, models from over here. I don't know exactly how it's implemented at the PK4-loading level, but perhaps the need to seek and extract each of these individual assets from different files results in an inefficient, non-pipelined load process.

The engine opens all pk4 files on load, and creates an index of all files contained in it. When you actually open the file later, it just sets the offset in zip file and starts reading. So everything is OK on zip level, but of course data is read from disk in the order in which you load it, so seeking is done.This is not HDD-friendly, so SSD is very helpful.

By the way, it seems to me that loading times are much longer on AMD GPU than on NVIDIA GPU. I suspect that loading textures to OpenGL is the major fraction of loading times.

2 hours ago, Filizitas said:

I dont know exactly but: Why not load everything first and then play the intro and stuff?
That might be hard todo now that the system is already coded.

Forcing player to wait on TDM start is even worse than waiting on level load.

The idea of loading level in background while player goes through briefing is quite hard to implement. It requires multithreading, and the engine itself does not work with it properly, so there would be plenty of bugs, some of them corrupting random stuff randomly --- very annoying, very hard to reproduce and debug. Duzenko already tried to load textures in separate thread (faster load for debug purposes), and recently we had hard time fixing random weird bugs caused by it. We had to make the whole D3 filesystem threadsafe. And there are many more systems which are not yet suitable to multithreading.

Moreover, it is hard to distribute resources between loading and other engine: for instance, FFmpeg videos can be rather CPU-heavy to decode.

 

I'm not sure that squeezing 25% of load times is worth the effort behind it. If someone really wants to invest time into it, why not. Otherwise, let's pretend they are already OK 😥

  • Like 1
  • Haha 1
Link to comment
Share on other sites

2 hours ago, stgatilov said:

This is not HDD-friendly, so SSD is very helpful.

Not as much as you'd expect. A while back, Aluminium Haste benchmarked loading times using hdd, ssd and RAM-Disk. Loading time went down from 1:45 min to 1:28 min at most.

 

I also did a quick benchmark just now in order to check what the effect of the zip/pk4 compression is. I used 2.08 Beta 1 and compared it to SVN (where everything is unzipped), while using the same executable. I loaded Saint Lucia for this test.

  • Loadtime zipped: 0:28 min
  • Loadtime unzipped: 0:23 min

Again, not much to gain here.

I wonder how loadtimes would be affected if all normals were converted to RGTC? 

 

  • Like 1
Link to comment
Share on other sites

On 3/25/2020 at 7:07 PM, VanishedOne said:

I don't know whether this will help with the loading times, but I recently found I had image_useCompression and image_useNormalCompression set to the default 1 (I was wondering why the colour banding on my TGAs was so bad in the game). According to https://www.iddevnet.com/doom3/ these cause TGAs to get compressed at load time. Since TDM (currently) doesn't use pre-compressed normal maps, that might be one cause of loading delay, though I can't say I've noticed a big speed-up (and presumably setting 0 will use more RAM).

This actually solved the slow loading problem for me, so thanks for the suggestion.

I guess that raises the question of why image_useNormalCompression is defaulting to 1 in the first place, given the problems it causes. Perhaps it is intended as a "safe" default for people with low-memory GPUs.

  • Like 1
Link to comment
Share on other sites

If I recall correctly, there used to be a menu option to enable / disable.  I haven't been involved much in recent times, has it been removed?  

Edit:  Just checked and don't see any sign of it.  Probably removed at some point with recent updates / optimizations.  Definitely a good thing to have for people who need it, but shouldn't be enabled by default.  If possible, I would say put the a texture compression setting in the menu for people who can accept the trade off of performance over loading time.

  • Like 1
Link to comment
Share on other sites

48 minutes ago, OrbWeaver said:

This actually solved the slow loading problem for me, so thanks for the suggestion.

I guess that raises the question of why image_useNormalCompression is defaulting to 1 in the first place, given the problems it causes. Perhaps it is intended as a "safe" default for people with low-memory GPUs.

Oh wow, yes, that's a huge gain in loading time. My loading time for Saint Lucia in 2.08 Beta 1 decreased from 0:28 min to 0:17 min when run from my regular HDD. Wow! Just out of curiousity, I also copied the game to my lightning fast m2-SSD and surprisingly, there was no difference in loading time: It was 0:17 min again! Tried it numerous times in both cases, to verify. I conclude, that hdd access are not the bottleneck in regards to loading time.

Link to comment
Share on other sites

On 3/30/2020 at 3:46 PM, STiFU said:

Oh wow, yes, that's a huge gain in loading time. My loading time for Saint Lucia in 2.08 Beta 1 decreased from 0:28 min to 0:17 min when run from my regular HDD. Wow! Just out of curiousity, I also copied the game to my lightning fast m2-SSD and surprisingly, there was no difference in loading time: It was 0:17 min again! Tried it numerous times in both cases, to verify. I conclude, that hdd access are not the bottleneck in regards to loading time.

I got even more dramatic gains than you did.

I have what used to be a mid-end laptop about 8 years ago, an Ivy Bridge i5 CPU upgraded with enough RAM and an SSD. 

With a bigger mission, Volta and the Stone, the loading time was 1:48 m with compression on and only 31 seconds!! with image_useCompression and image_useNormalCompression set to 0.

Looking at the FPS at the beginning of the mission, I seem to have gotten a 2 FPS reduction with the compression off, but that might have just been statistical noise, I would need to do more tests to find out if there's an actual difference. But note that I have a Radeon 7650M, which is as I said 8 years old and basically a rebranded 5xxx radeon chip - I don't know whether its performance is even relevant for you. 

On 3/26/2020 at 5:38 PM, STiFU said:

I also did a quick benchmark just now in order to check what the effect of the zip/pk4 compression is. I used 2.08 Beta 1 and compared it to SVN (where everything is unzipped), while using the same executable. I loaded Saint Lucia for this test.

  • Loadtime zipped: 0:28 min
  • Loadtime unzipped: 0:23 min

That's 18% reduction in loading time, not as dramatic as image compression, but imo nothing to scoff at. It might be more on slower CPUs like mine. I would think that it's worth considering depending on what the size difference is.

  • Like 1
Link to comment
Share on other sites

My understanding (and I might well be out of date on this) is that image_useNormalCompression should default to 0, because we do not supply precompressed normal maps and compression of normal maps tends to look bad, but image_useCompression should be left on (1), because compression of regular images is normal practice and these are supplied as pre-compressed DXT files anyway (in fact they may ONLY be supplied as precompressed DXT files, which means that some combinations of disabling image compression will result in all-black environments since the non-compressed original images cannot be found).

  • Like 1
Link to comment
Share on other sites

46 minutes ago, OrbWeaver said:

My understanding (and I might well be out of date on this) is that image_useNormalCompression should default to 0, because we do not supply precompressed normal maps and compression of normal maps tends to look bad, but image_useCompression should be left on (1), because compression of regular images is normal practice and these are supplied as pre-compressed DXT files anyway (in fact they may ONLY be supplied as precompressed DXT files, which means that some combinations of disabling image compression will result in all-black environments since the non-compressed original images cannot be found).

According to the descriptions given in the game console, image_usePrecompressedTextures is the cvar to 'use .dds files if present'.

If your only TGAs are things like light textures (which tend to have forceHighQuality set in the material) and cubemaps, image_useCompression shouldn't make much difference, but working with TGA assets I haven't compressed to DDS yet, I found they had horrible colour banding on my system and only looked decent with image_useCompression turned off.

Some things I'm repeatedly thinking about...

 

- louder scream when you're dying

Link to comment
Share on other sites

  • 2 weeks later...

I'm just gonna throw in my two cents on the matter of the missions included in the package.

I personally think A New Job and Tears of St Lucia are both great missions and should pretty much remain as is.  The training mission... not so much. It is a sandbox sort of environment that is not so fitting with the rest, at least as far as tutorials go. At some point I would like to see a new training mission on a more step-by-step tutorial basis, preferably starting at the Thief's house and then venturing out on the streets to accomplish a very simple objective. This would better introduce the main character and set the tone for the following missions. As for the original training mission, it should be repurposed and renamed as a practice sandbox or something of the kind.

More importantly, I think the additional missions concluding the St. Lucia story should be finished. Like OP says, it would be lovely for the packaged product to have a small story with a beginning and an end. Isn't it possible to pull a community effort to make this a reality? I think this should have been at least considered after so many years...

Finally, an additional suggestion, and as a complement to a possibly finished St. Lucia campaign: I think three standalone FMs of good reputation should be included in the main package as to give the player a more complete experience, ideally one short, one medium, and one long. It would also serve as a showcase of the many possibilities of map making and of the variety contained within TDM, while recognizing long-standing FM creators within the community. All in all, the goal here would be to give the first time player a stellar impression of some of the best things the game has to offer.

Edited by zergrush
  • Like 1
Link to comment
Share on other sites

17 hours ago, zergrush said:

The training mission... not so much. It is a sandbox sort of environment that is not so fitting with the rest, at least as far as tutorials go. At some point I would like to see a new training mission on a more step-by-step tutorial basis, preferably starting at the Thief's house and then venturing out on the streets to accomplish a very simple objective.

I completely agree with that! The training mission has no story at all and can't even be finished, so it might give a bad first impression to players unless they continue...

Edited by wesp5
Link to comment
Share on other sites

I think the present training mission should remain as is as a simple practice sandbox, but it would be nice to have a real step-by-step tutorial mission. A New Job does good work of introducing the player to navigation , searching for alternate paths, and simple narrative, but it does not work as a tutorial in the classical sense.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 1 reply
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
    • nbohr1more

      Please vote in the 15th Anniversary Contest Theme Poll
       
      · 0 replies
×
×
  • Create New...