TDM Packer 2 - another tool for managing and packing missions

[Skaruts]

This is an experimental GUI app for managing and packing Dark Mod missions. It's a successor to FM Packer but with a GUI, that I've been working on for some time. It's now at a stage where I feel somewhat comfortable sharing. Having a GUI unlocks potential for more in depth management of fms, which I intend to explore in the future. 

You can download the binaries here: https://github.com/Skaruts/tdm_packer_2/releases

Currently:

• it can pack your selected mission into the pk4 at a click of a button
• it provides an editor for a .pkignore file where you specify which files to exclude from the pk4
• it gives you a handy tree view of both the included and excluded files (Shift+LMB to fully expand/contract branches)
• it can launch TDM or DarkRadiant for the selected mission, or run a second installation of TDM to test your pk4 in isolation (see the menu Settings->Paths)
• it automatically writes your map sequence into the appropriate file
• all maps, except the ones in the map sequence, are auto excluded from the pk4

Important: This is still an experimental alpha version, so backup your missions before using it.

Note: For now I provided binaries for Windows and Linux. I don't have a Linux system to test the binaries yet, though, so they may or may not work properly. As a last resort, one can still run this app from the source code by running the Godot Engine itself from the terminal with the "--path path/to/project" argument. (Godot itself is just a simple executable and requires no installation.)

Edited February 12 by Skaruts

[nbohr1more]

Woo!

Any chance you could add a feature that warns about spaces or special characters in file or folder names?

That way it'll be less work to upload missions to the database.

[Skaruts]

14 minutes ago, nbohr1more said:

Woo!

Any chance you could add a feature that warns about spaces or special characters in file or folder names?

That way it'll be less work to upload missions to the database.

Quite possibly. I could even do this on the python version too. Spaces are easy to look for. For special characters I'll need to know more about what's supported. 

I suppose that includes the pk4 name, right?

Edited February 8 by Skaruts

[nbohr1more]

It's easy enough to rename the mission pack on upload but I guess for new mappers they might encounter a puzzle if the pk4 fails to be read by TDM so pack naming checks would be handy.

As for special characters?

( ) { } [ ] | ! @ # $ % ^ & * , + - " ' : ; ? < > ` ~ /

Are the ones I've seen cause upload failures. I am hoping that we aren't susceptible to UTF-8 localization characters. Thus far I haven't seen any failures in that regard.

[Skaruts]

1 hour ago, nbohr1more said:

( ) { } [ ] | ! @ # $ % ^ & * , + - " ' : ; ? < > ` ~ /

Are the ones I've seen cause upload failures. I am hoping that we aren't susceptible to UTF-8 localization characters. Thus far I haven't seen any failures in that regard.

Is this a problem specific to the server and not the game? I just need to know what kind of error or warning to display.

Also, I could actually enforce this, rather than just warn about it, if that would be preferable.

Edited February 8 by Skaruts

[nbohr1more]

It impacts the SVN server as I know. The game seems to handle these without issue other than - in some file names.

I would prefer enforcement since it would make my life easier as a mission maintainer

[Skaruts]

@nbohr1more

Added this to the python version.  Seems to be working great.

Also complains when the mission folder is invalid. Gonna port it to the GUI version.

[datiswous]

There's some rules for capital letters in naming folders I think that could have a check?

[Skaruts]

5 hours ago, datiswous said:

There's some rules for capital letters in naming folders I think that could have a check?

Possibly. I'll need to know the rules.

Edited February 9 by Skaruts

[nbohr1more]

For some reason, Linux won't recognize an FM folder if it has any capital characters. Sub-folders work though.

[datiswous]

Oh I thought you should use lowercase for all filenames and foldernames. That's what this video says at least:

 

[nbohr1more]

I prefer all lower case but I think I've seen some camel-case folders work in Linux. Just not the main ones ( materials, textures, dds, def, maps, guis, etc ) 

[peter_spy]

I think I have an idea for a really useful feature this thing could have: you could parse the .map file for e.g. models and material names, and you could compare them against a list of files found in a given FM folder. You should be able to find a way to exclude models, textures, and materials that didn't make it to the final version of a map, which would be very useful for people making custom stuff. Hunting for that stuff during final stages is a chore, and may end up with unnecessary pk4 bloat when you miss something.

[Frost_Salamander]

34 minutes ago, peter_spy said:

I think I have an idea for a really useful feature this thing could have: you could parse the .map file for e.g. models and material names, and you could compare them against a list of files found in a given FM folder. You should be able to find a way to exclude models, textures, and materials that didn't make it to the final version of a map, which would be very useful for people making custom stuff. Hunting for that stuff during final stages is a chore, and may end up with unnecessary pk4 bloat when you miss something.

That does indeed sound useful.  I ran into this problem with Foreign Affairs and all the custom textures.  But I wonder if that functionality would be better placed in a DR script instead? Then everyone would benefit and not just users of this tool. Obviously it wouldn't prevent stuff getting packed in the FM, but it could identify the things not being used and the mapper could then manually remove them, leaving them with a clean source folder.  Or even delete the stuff if you want it to?

[peter_spy]

Yeah, having it directly in DR would be brilliant.

[Skaruts]

@peter_spy @Frost_Salamander

I've actually been working on that kind of thing in the python version. I'm gonna have to do the same for this version, but some of those things are already usable in the other one in the dev branch, through the --validate command.

https://github.com/Skaruts/tdm_fm_packer/tree/dev

I've been doing this in that version because it's easier to iterate and test. Once I get to the point things seem to be going ok, I'll bring them to this one.

I forgot to update the readme, though, but you can use -h to see how it works. (EDIT: actually, I forgot to update the help too, omg... ) (EDIT2: got that sorted.)

Beware though that the paint is still fresh. It may not be working 100% reliably. Needs more testing.  

Edited February 11 by Skaruts

[datiswous]

On 2/8/2025 at 4:23 PM, Skaruts said:

You can download the binaries here

I can't find the binaries

Edit: nevermind, it's under Releases in the right menu (but how could I know this?)

I think you should rename the binary package though. alpha_0.3_linux.tar.gz is not a good name for it.

Edited February 12 by datiswous

[Skaruts]

Yea, I should've linked directly to the releases. Sorry about that. Fixed it.

17 minutes ago, datiswous said:

I think you should rename the binary package though. alpha_0.3_linux.tar.gz is not a good name for it.

I don't know what else to call it. Or maybe you mean I should include the app name in it? I should've done that, indeed. It didn't even occur to me, for some reason.

Maybe tdm_packer_a0.3_linux.tar.gz would be an improvement.

[datiswous]

On 2/8/2025 at 4:23 PM, Skaruts said:

it can pack your selected mission into the pk4 at a click of a button

This works in Linux

On 2/8/2025 at 4:23 PM, Skaruts said:

it provides an editor for a .pkignore file where you specify which files to exclude from the pk4

I guess you have to manually write down the files you want to exclude? I thought you could just double click in the file-tree, but this doesn't work. There's no save button for this it seems.

On 2/8/2025 at 4:23 PM, Skaruts said:

it gives you a handy tree view of both the included and excluded files (Shift+LMB to fully expand/contract branches)

yes. But this is not updated until you save, which you can only do when you pack, play, or edit.

On 2/8/2025 at 4:23 PM, Skaruts said:

it can launch TDM or DarkRadiant for the selected mission, or run a second installation of TDM to test your pk4 in isolation (see the menu Settings->Paths)

Works with DR, but not TDM. The console logs starting and stopping, nothing happens.

On 2/8/2025 at 4:23 PM, Skaruts said:

it automatically writes your map sequence into the appropriate file

I don't understand what this does.

Edit: Is this for campaigns?

On 2/8/2025 at 4:23 PM, Skaruts said:

all maps, except the ones in the map sequence, are auto excluded from the pk4

I don't see how this could be useful and why this isn't inside the Pack window.

Edit: Ah ok, I see.

Edited February 12 by datiswous

[Skaruts]

@datiswous by the way, is the executable icon a little cardboard box? (And showing everywhere, window, filesystem, taskbar, etc?) Just wanted to be sure I set the icon properly.

 

59 minutes ago, datiswous said:

I guess you have to manually write down the files you want to exclude? I thought you could just double click in the file-tree, but this doesn't work. There's no save button for this it seems.

Yes, you need to edit by hand, for now (and save). I may do something with the trees at some point, but that will probably be a rabbit-hole of its own.

There are no handy UI buttons for now either.  As far the text editors go, you still have all the hotkeys you might expect: ctrl-s, ctrl-z, ctrl-shift-z, etc. Including multi-cursors, if I didn't forget to turn them on.

 

1 hour ago, datiswous said:

yes. But this is not updated until you save, which you can only do when you pack, play, or edit.

Yes, you have to save the .pkignore in order for it to rebuild the trees.

 

1 hour ago, datiswous said:

Works with DR, but not TDM. The console logs starting and stopping, nothing happens.

Damn... that's a bummer. I don't know what to do about it, as I'm just using Godot's api to execute a process. Have you tried running the console version of TDM Packer? I don't know if it will show any useful information. I didn't export with debug, so probably not.  

 

1 hour ago, datiswous said:

I don't understand what this does.

Edit: Is this for campaigns?

Yes. If you have more than one map in the map sequence it will create a tdm_mapsequence.txt, instead of a startingmap.txt. This is something I still have to look deeper into, though. I've never made any campaigns, so this feature is probably not reliable as is, and I'm not even sure I'm on the right track. I only looked into the files of NHAT. 

  

1 hour ago, datiswous said:

I don't see how this could be useful and why this isn't inside the Pack window.

Edit: Ah ok, I see.

Just to clarify it, I made it this way because I tend to have lots of small test maps and even some local backups, or different versions of the main map. Excluding them all by hand in the .pkignore would be quite tedious. Since I couldn't think of a reason why one might want to include unused maps in the pack, I just went with the assumption that there probably isn't one.

 

Edited February 12 by Skaruts

[datiswous]

26 minutes ago, Skaruts said:

Damn... that's a bummer. I don't know what to do about it, as I'm just using Godot's api to execute a process. Have you tried running the console version of TDM Packer? I don't know if it will show any useful information. I didn't export with debug, so probably not.  

It shows this:

Spoiler

select_mission 3 crystalgrave2_1
ERROR: Attempt to disconnect a nonexistent connection from 'root:<Window#77393300089>'. Signal: 'focus_entered', callable: ''.
   at: _disconnect (core/object/object.cpp:1462)
ERROR: Attempt to disconnect a nonexistent connection from 'root:<Window#77393300089>'. Signal: 'tree_exited', callable: ''.
   at: _disconnect (core/object/object.cpp:1462)
TASK: Running TDM
TDM 2.13/64 #10932 (1435:10932) linux-x86_64 Jan 26 2025 17:32:50
/proc/cpuinfo CPU frequency: 4242.23 MHz
4242 MHz AMD CPU with SSE & SSE2 & SSE3 & SSSE3 & SSE41 & AVX
found interface lo - loopback
found interface enp8s0 - 192.168.1.64/255.255.255.0
found interface docker0 - 172.17.0.1/255.255.0.0
Found AMD CPU, features: SSE SSE2 SSE3 SSSE3 SSE41 AVX
TDM using AVX for SIMD processing.
no 'darkmod' directory in exe path /home/datiswous/Games/TDM-new, skipping
no 'darkmod' directory in cwd path /home/datiswous/Downloads/alpha_0.3_linux, skipping
WARNING: using hardcoded default base path
Found 0 new missions and 0 packages.
------ Initializing File System ------
Current search path:
  [C] /home/datiswous/Downloads/alpha_0.3_linux/
File System Initialized.
--------------------------------------
Unknown command 'vid_restart'
--------- Game Map Shutdown ----------
--------- Game Map Shutdown done -----
idRenderSystem::Shutdown()
signal caught: Segmentation fault
si_code 1
Was in fatal error shutdown: Couldn't load default.cfg
Trying to exit gracefully..
About to exit with code 0
TASK: Copying TDM logs to TDM directory
ERROR: Failed to open DarkMod.log
   at: copy (core/io/dir_access.cpp:348)
ERROR: Failed to open DarkMod.temp.log
   at: copy (core/io/dir_access.cpp:348)
INFO: TDM process ended

 

Hmm, it looks for a "darkmod" folder?

Edit: Changing the tdm folder to "darkmod" makes the play button work correctly, but this should not be required.

Edited February 12 by datiswous

[Skaruts]

11 minutes ago, datiswous said:

Hmm, it looks for a "darkmod" folder?

Edit: Changing the tdm folder to "darkmod" makes the play button work correctly, but this should not be required.

I'm inclined to believe that's some TDM or Linux sided quirk. The program just uses whatever path you give it. It only cares that the executable is in there. I can't reproduce this on windows.

Unless it has something to do with Godot not being able to set the current-working-directory (CWD) to the game's path when launching it. If that's the cause, I can't do anything about it, unfortunately.

[datiswous]

I remember there was a bug about the darkmod folder a while ago..

Edit: I meant this bug, but It's probably not related.

 

What's the command you give to start tdm from your program?

Edited February 12 by datiswous

[datiswous]

59 minutes ago, Skaruts said:

by the way, is the executable icon a little cardboard box? (And showing everywhere, window, filesystem, taskbar, etc?) Just wanted to be sure I set the icon properly.

It shows up in different places, but not the executable icon.

 

1 hour ago, Skaruts said:

Yes, you have to save the .pkignore in order for it to rebuild the trees.

But it's missing a manual save functionality.

[Skaruts]

14 minutes ago, datiswous said:

But it's missing a manual save functionality.

You can press ctrl+s. The hotkey is from the text editor itself, so you must click the text editor for it to gain focus and receive input.

I've been on the fence about just having a global save functionality that just saves all mission files at once, whenever you do ctrl+s. Might make it easier on users and also on me, because individual saving is harder to work with.

Or... I could add a timer, that counts down a couple seconds and auto-saves, after you make some changes. That could work too.

Edited February 12 by Skaruts