Jump to content
The Dark Mod Forums

Faster dmapping (testers requested)


SteveL

Recommended Posts

We're working on tweaking DMAP for 2.04 to speed it up by cutting down its console output. It's the console spam that makes DMAP take 4x as long when you have vsync turned on in the video settings. And even when you have vsync turned off, DMAP is significantly faster without the excessive console output.

I'm hoping for some feedback from current mappers who use Windows (I don't know how to compile for Linux, sorry). Did I suppress messages that you want to see? I'll post a patched TDM 2.03 engine in this thread with the dmapping changes. If you're using DMAP right now, you can use this version and it'll be faster.

https://www.amazon.co.uk/clouddrive/share/ELHMA5EQbln-K72wMZt6FKzRBc5BodbL5Qcgc-CICkE?ref_=cd_share_link_copy

To install: save the linked file in your darkmod folder overwriting the current file with the same name (TheDarkMod.exe).
To uninstall: run tdm_update.exe from the same folder

Notes:

  • With the patch installed, you can reactivate the original console output for comparison by using option 'v', e.g. dmap v mymap
  • Some console spam still comes out of other parts of the code during dmapping (renderer code used to remove triangles, AAS and collision world generation), but not enough to slow things down.
  • The console will now stay open throughout, so you'll see the output from the CM and AAS stages too instead of getting a frozen TDM at the end of dmap.
  • Your first DMAP after opening TDM will take longer than later ones. The first one loads and parses materials and decls that won't have to be done again until you restart the game. So when comparing, ignore your timings on the first run.

Let me know if any of the messages that I've hidden should be put back again.

Link to comment
Share on other sites

I'm using SVN, and dmap is considerably faster. I'm not missing any messages, and I'm thinking the "degenerate triangles" warnings could be suppressed, because that's internal to dmap and there's nothing a mapper can do about them.

Link to comment
Share on other sites

That's true. The only reason I left them in is that they come from general renderer code, not the dmap-specific code that I tweaked in this patch. I don't know whether that code is used by anything except dmap itself. But I'll look for a clean way to suppress the messages. It would be nice if dmap gave just the right amount of meaningful output, so that mappers will always want to read the text. I did some things in that direction in this patch... individual entities get mentioned selectively with their class and name, for example, if they take longer than 0.5 seconds to dmap. But that useful bit of output is still buried beneath messages from the code that's external to dmap, in the renderer, collision world, and AAS sections.

Link to comment
Share on other sites

Did a comparison run between 2.03 and svn using Moonbo's new map as a base -

 

2.03 vsync off

  • 3:58 console disappears, mouse cursor fixed.
  • 4:36 mouse cursor moveable, dmap finished.

SVN vsync off

  • 4:28 console always visible, dmap finished.

SVN vsync on

  • 4:36 console always visible, dmap finished.

New tdm.exe from OP, vsync off

  • 4:28 console always visible, dmap finished.

Map info

  • File size - 34mb
  • Entities - 7009
  • Brushes - 25171
  • Patches - 10120

Copied over the existing tdm.exe in the svn folder, ran tdm.exe and got a CTD - dump file.

 

Copied over the 2.03 tdm.exe and the game loads up fine.

Link to comment
Share on other sites

Copied over the existing tdm.exe in the svn folder, ran tdm.exe and got a CTD - dump file.

Use either SVN or the patch, but not both. The patch is for TDM 2.03, not the 2.04 WIP in the SVN repository.

 

Not sure I understand what results you got in your post above... Did you see no improvement over 2.03 by using the SVN build then? And when you say "dmap finished" do you mean you stopped the timer where that message appeared? That happens before the AAS and CM bits.

Link to comment
Share on other sites

  • Use either SVN or the patch, but not both. The patch is for TDM 2.03, not the 2.04 WIP in the SVN repository.

Not sure I understand what results you got in your post above... Did you see no improvement over 2.03 by using the SVN build then? And when you say "dmap finished" do you mean you stopped the timer where that message appeared? That happens before the AAS and CM bits.

  • Ah yes, I did wonder if that was the case, understood.

Yes SVN was faster than 2.03 with vysnc off by 8s, and with vsync on under SVN its was the same as 2.03. So good news as it is, the new .exe with vsync ON is as fast as 2.03 with vsync OFF.

Link to comment
Share on other sites

  • Yes SVN was faster than 2.03 with vysnc off by 8s, and with vsync on under SVN its was the same as 2.03. So good news as it is, the new .exe with vsync ON is as fast as 2.03 with vsync OFF.

I tested it on a few maps and got much more impressive results than that. Hopefully other maps will give you better results too. It will vary by map, based on the amount of FS built in DR (which cause dmapping slowdown).

Link to comment
Share on other sites

We're working on tweaking DMAP for 2.04 to speed it up by cutting down its console output. It's the console spam that makes DMAP take 4x as long when you have vsync turned on in the video settings. And even when you have vsync turned off, DMAP is significantly faster without the excessive console output.

 

I'm hoping for some feedback from current mappers who use Windows (I don't know how to compile for Linux, sorry).

I'd like to try it in Linux, unfortunataly, it doesn't compile here anymore. Is there anyone still testing TDM on Linux?

"The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw (1856 - 1950)

 

"Remember: If the game lets you do it, it's not cheating." -- Xarax

Link to comment
Share on other sites

I always find the first dmap after freshly booting up TDM to be the longest so I ran a dmap after a fresh boot and then another straight after to see if there was any difference. Overall I found it did trim down my dmapping speeds but not by a massive amount.

 

Mapsize 5.16mb
Without patch
dmap first time - 58 seconds
dmap second time - 20 seconds
With patch
dmap first time - 48 seconds
dmap second time - 15 seconds
Link to comment
Share on other sites

On the subject of doing multiple DMAP/MAP runs, this is greatly aided by having lots of system ram.

 

The more ram you have the longer you can go before TDM throws a memalloc error, for example with 16Gb of ram I can go for hours doing multiple runs with TDM/DR both open while working on a map

Link to comment
Share on other sites

Nice work Steve, I did have some requests if this is possible.

Once the first part of DMAP has finished, the console closes but now it's doing pathfinding and it's not done.

Could it be changed to keep the console open until Dmap is completely finished?

 

Also about the console output, would it be possible instead to change it so that every 2 seconds, a percent progress update shows instead of nothing?

`

I always assumed I'd taste like boot leather.

 

Link to comment
Share on other sites

Nice work Steve, I did have some requests if this is possible.

Once the first part of DMAP has finished, the console closes but now it's doing pathfinding and it's not done.

Could it be changed to keep the console open until Dmap is completely finished?

 

Also about the console output, would it be possible instead to change it so that every 2 seconds, a percent progress update shows instead of nothing?

`

 

Console closing: I thought I'd fixed this. It works for me. I didn't try to stop the console closing -- which happens outside of dmapping code -- but I put in a command to reopen it immediately. I don't even see it close, and at the end, it's still open. What's happening at your end exactly?

 

Progress updates: We could add some kind of progress update. It'd be nice to have a line of output for each of the 10 major steps in dmapping, and a 1-liner account of what got processed. The longer running stages could report a counter at least that got updated every 10 or 100 items: 550 / 2435 func statics dmapped, for example, if we can get it to write on the same line every time rather than scroll the console.

Link to comment
Share on other sites

It's the console spam that makes DMAP take 4x as long when you have vsync turned on in the video settings.

 

 

LOL, srsly?! Can't believe this! :blink:

"Einen giftigen Trank aus Kräutern und Wurzeln für die närrischen Städter wollen wir brauen." - Text aus einem verlassenen Heidenlager

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • Ansome

      Finally got my PC back from the shop after my SSD got corrupted a week ago and damaged my motherboard. Scary stuff, but thank goodness it happened right after two months of FM development instead of wiping all my work before I could release it. New SSD, repaired Motherboard and BIOS, and we're ready to start working on my second FM with some added version control in the cloud just to be safe!
      · 0 replies
    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
×
×
  • Create New...