Automation features and discussion

[stgatilov]

I have previously posted about automation efforts in this thread about watching over asset files.

I decided to create a more generic thread for whatever discussions related to automation.

 

The most recent version of automation library and scripts (by the time this topic was created) can be downloaded from this link.

More information can be found in readme.md.

It should run properly against any beta version of TDM 2.07.

 

Some new Python 3 is required to run automation stuff.

I'm afraid that decent skills in Python are needed if you want to do anything advanced with it.

If you have some tedious work that you regularly do with TDM, and feel that it can be automated, post ideas here.

 

UPDATE: Current version (2019.02.22) is available here.
It should run properly against TDM 2.07.

UPDATE: Current version (2026.02.11) is available here.
It should run properly against TDM 2.14 at least.

[stgatilov]

Here is a video showcasing automation commands (you can type them into console.py, and I intend to integrate them into automation library in future).

I managed to complete the shortest TDM mission without directly interacting with the game.

 

Beware: the video is long and very boring

 

[stgatilov]

Just now automation has been used to start every FM released to date.

It is about 125-128 missions already

That's a huge number , and merely starting and cheat-completing them all takes something like 90 minutes

 

If anyone is interested in this feature, see complete_any.seq (commands sequence) and foreachfm.py (the runner script) in the automation package.

[Oktokolo]

That changes everything - hardcore ghosters will have to play the game without a mouse using your remote to get any satisfaction now.

[freyk]

At this moment, i can only say to this automation feature: WOW!

Immediately got some ideas for it. Like a TDM-Tweaker, video-problems debugger, benchmarker, etc.

 

Now to find a small python collection package (or one-single binary), to run scripts from.

Do you know something i can use.

Like winpython?

 

 

Btw,

nice to see midnight commander in action.

I'm a bit more a totalcommander (ultima prime) user myself.

Edited January 12, 2019 by freyk

[STiFU]

So, who is going to train a deep neural network now to play tdm with this? ;-)

[stgatilov]

Now to find a small python collection package (or one-single binary), to run scripts from.

Do you know something i can use.

Like winpython?

Never used anything except CPython myself.

People use py2exe or pyinstaller to produce a standalone executable.

I'll try them in future (after I get rid of ugly sys.path hack).

 

 

nice to see midnight commander in action.

I'm a bit more a totalcommander (ultima prime) user myself.

Thou shalt not confuse the great Far Manager with the ugly Midnight Commander!

Anyway, all of them are descendants of the same Norton Commander.

[jaxa]

Automation should be used to create missions, not play them! Like Swift Mazes but more advanced...

[STiFU]

I figured, I'll use automation for testing if my fix for #4952 works. :-)

I have added two new commandline args "num_repeats" and "repeat_seqFile" to foreachfm.py. If both are specified, the regular seqFile is only executed once as a form of initialization and then repeat_seqFile is executed periodically as specified in the num_repeats argument.

 

This way, I can now easily repeatedly enter the fighting area and let myself get killed by the AI with a simple a script as this:

#teleport to fighting area, wait to be killed

setviewpos -551.59 -1949.04 -180.94 -16.0 -176.5 1.1

$time.sleep(20.0)

This is really awesome and makes our lives a lot easier, so thanks stgatilov!! :-) I have the debugger attached. If I eventually hit a set breakpoint, my fix works! :-) If the game crashes, it doesn't. ^^

 

By the way, I accidentally once tele-fragged the AI because I had used a too low sleep timing. :-D

[STiFU]

Ok, so far, I didn't manage to reproduce the behavior with this sequence. So I was thinking to add random rotations and movements as well as the occasional attack. So, how to feed random values from python to the automation interface? One way to achieve this would be to have python write a new .seq file with the desired commands and have that .seq file executed. I guess, there is no other interface to achieve that right now, is there?

[STiFU]

After three hours of running, the game crashed in Automation::ParseAction() with a Null-pointer exception because

idFile *f = fileSystem->OpenFileRead(logFile->GetName());

apparently failed to open the file for some reason and f==NULL is not asserted. I wonder why it cannot open the file anymore. I backuped the pdb and saved both a full dump and a minidump, if you want to investigate the issue closer. The full dump is 5 gigs, though.

 

This is not the crash I was hoping for... If it had crashed during LeaveSearch() again, I would've known that my fix does not work. Now, as it stands, I haven't learnt anything, except that this is maybe the wrong way to reproduce the issue or that it is incredibly rare!

 

Callstack:

 

TheDarkModx64.exe!idCommonLocal::GetConsoleContents(int begin=31058, int end=31058) Line 502 C++
TheDarkModx64.exe!Automation::ParseAction(Automation::ParseIn & parseIn={...}) Line 298 C++
TheDarkModx64.exe!Automation::ParseMessage(Automation::ParseIn & parseIn={...}) Line 269 C++
TheDarkModx64.exe!Automation::ParseMessage(const char * message=0x00000232c0554cb0, int len=107) Line 256 C++
TheDarkModx64.exe!Automation::Think() Line 533 C++
TheDarkModx64.exe!Auto_Think() Line 538 C++
TheDarkModx64.exe!idEventLoop::RunEventLoop(bool commandExecution=true) Line 171 C++
TheDarkModx64.exe!idCommonLocal::Frame() Line 2450 C++
TheDarkModx64.exe!WinMain(HINSTANCE__ * hInstance=0x00007ff7e4f40000, HINSTANCE__ * hPrevInstance=0x0000000000000000, char * lpCmdLine=0x00000232df913bbe, int nCmdShow=10) Line 1363 C++

[stgatilov]

I figured, I'll use automation for testing if my fix for #4952 works. :-)

I have added two new commandline args "num_repeats" and "repeat_seqFile" to foreachfm.py. If both are specified, the regular seqFile is only executed once as a form of initialization and then repeat_seqFile is executed periodically as specified in the num_repeats argument.

To be honest, I hoped that script will only be used for massive testing over many FMs, not for ordinary checking.

Probably it's better to keep it that way, because otherwise a lot of stuff will fill it over time.

 

There is another script called repeat.py, which is very small and not really important.

It's better to fill it with regular testing stuff.

For instance, I used it for testing crashes in 4919 the following hacky way:

1) I created a file finish_mission.seq, which loads a quicksave, moves forward to mission exit, waits until mission finish screen loads up.

2) Then I created another file finish_mission_10.seq, which had one command @file finish_mission.seq copy-pasted 10 times, and exit at the very end.

3) Then I run repeat.py finish_mission_10.seq.

So it boots TDM, performs sequence 10 times in a row, then properly exits. Repeat 100 times. If something crashes, repeat.py will catch nonzero exit code.

[stgatilov]

Ok, so far, I didn't manage to reproduce the behavior with this sequence. So I was thinking to add random rotations and movements as well as the occasional attack. So, how to feed random values from python to the automation interface? One way to achieve this would be to have python write a new .seq file with the desired commands and have that .seq file executed. I guess, there is no other interface to achieve that right now, is there?

There is no plan to turn automation commands syntax into real programming language --- there is python for that!

Just write your own Python script and use automation class there directly.

For gameplay actions you can either try to use game_controls directly or try to invoke console.py (I seriously plan to move commands execution engine from this script to library).

 

After three hours of running, the game crashed in Automation::ParseAction() with a Null-pointer exception because

idFile *f = fileSystem->OpenFileRead(logFile->GetName());

apparently failed to open the file for some reason and f==NULL is not asserted. I wonder why it cannot open the file anymore. I backuped the pdb and saved both a full dump and a minidump, if you want to investigate the issue closer. The full dump is 5 gigs, though.

This is not the crash I was hoping for... If it had crashed during LeaveSearch() again, I would've known that my fix does not work. Now, as it stands, I haven't learnt anything, except that this is maybe the wrong way to reproduce the issue or that it is incredibly rare!

Did you look for this file on your machine? Was it present or not? empty or not?

When did this happened? Does your script rerun TDM process? install different FM?

Do you normally have logFile cvar enabled?

 

This may be some sort of race condition.

[STiFU]

Wowza, I expected you'd be happy about somebody actually using your automation stuff. ^^

 

To be honest, I hoped that script will only be used for massive testing over many FMs, not for ordinary checking.

Probably it's better to keep it that way, because otherwise a lot of stuff will fill it over time.

Well, that script already supported testing individual FMs. And using a repeatable .seq file might also make sense with massive FM testing. With that in place, you can very easily, without even knowing python, test about anything, and that's good right? I really see no harm in adding that feature, but if you don't like it, we can very well revert the change and I'll add a renamed copy of it. But look at the diff first. It's really almost nothing I have added.

 

Just write your own Python script and use automation class there directly.

For gameplay actions you can either try to use game_controls directly or try to invoke console.py (I seriously plan to move commands execution engine from this script to library).

Could you show me an example of howto do a random rotation in tdm with a python script? I tried to figure it out, but I am not that experienced with python and I also miss all those comfort features of Visual Studio. :-)

 

Did you look for this file on your machine? Was it present or not? empty or not?

When did this happened? Does your script rerun TDM process? install different FM?

Do you normally have logFile cvar enabled?

It was present and filled, if happened after 3 hours of "fighting" in the Training Mission using the described automation script. It does not restart TDM process inbetween, just 3 hours of continuous testing in a single FM without any reloading. I have logfile 2 always set.

[stgatilov]

Uploaded new version (see link in the first post).

 

Now this commands syntax is available as a part of automation library.

If you know a bit of Python, you can easily use it in your own script.

 

Also did some refactoring.

Hopefully testing framework would be next step.

 

[datiswous]

I wonder is this still being used by devs? Or there other methods being used now?

[stgatilov]

19 hours ago, datiswous said:

I wonder is this still being used by devs? Or there other methods being used now?

Yes, I still use it every year to check that missions are bootable.
Also I sometimes try to use it to reproduce problematic bugs, although it rarely helps.

But I never extended it to anything like a full-fledged testing framework.
There is a major feeling that such kind of tests will drain a lot of time to maintain and won't catch more issues than what any player can do in one play session.

The most up-to-date Python automation framework is in the assets SVN.

Also, hot reload feature in DarkRadiant uses the same code & protocol on the side of TDM engine (but not the Python part).

[datiswous]

I tried to script something recently. But couldn't start the mission from command. Menu sysyem has changed.

I don't know a lot of the commands possible.

I tried to start a mission and immidiatelly frob a readable somewhere in the mission (or a bunch of them in succession). I thought it could aid testing readable text in missions. But it seems too much work considering the other options.

[datiswous]

On 2/8/2026 at 4:59 PM, stgatilov said:

The most up-to-date Python automation framework is in the assets SVN.

Where can I find the assets svn? I looked for it here but couldn't find it:

https://svn.thedarkmod.com/publicsvn/darkmod_src/

[stgatilov]

20 hours ago, datiswous said:

Where can I find the assets svn? I looked for it here but couldn't find it:

It is not public.

I have added current version of automation to the first post of the thread.
But yes, usually using it is too much work.

[datiswous]

Cool thanks, got it working. Also the controls.txt is helpful. Took me a while to figure out how to type it though.

Guide says:

Controlled via gamectrl.impulse:

0: put away all weapons (backspace)
1-10: ready k-th weapon  (digits 1234567890)
19: objectives (O)
23: crouch          (usable only with "toggle crouch")
24: mantle

But you have to write:

@gamectrl impulse48

Not very straight forward, but I figured it out.

[datiswous]

Question:

In @help of console.py it states:

@guiscript: execute GUI script of specified window (allows clicking menu buttons)",

Is that a gui inside the mission, or some gui file that I have to specify? If I have let's say 20 readable's in a mission, can I open those directly this way? 

[stgatilov]

6 hours ago, datiswous said:

In @help of console.py it states:

@guiscript: execute GUI script of specified window (allows clicking menu buttons)",

Is that a gui inside the mission, or some gui file that I have to specify? If I have let's say 20 readable's in a mission, can I open those directly this way? 

I think this is the global GUI active at the current moment.
Most likely this is only main menu GUIs, I believe readables are "overlays" so they won't receive these signals.