Jump to content


Photo

Run console command automatically when files change

automation

18 replies to this topic

#1 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 22 October 2018 - 08:30 AM

*
POPULAR

The most recent TDM has the "automation" system, which allows an external Python script to connect to TDM game and programmatically do some stuff within it.
Right now only one basic action is supported, which is: run a given command in console.
Originally, the whole system was intended to implement automatic testing, and I hope I'll have time to get it there some day.
However, I already find it useful.
 
Recently I implemented a simple watcher script for interactive shader editing.
It works like this:
1) Python script watches over all files like glprogs/*.fsglprogs/*.vs and glprogs/*.gs.
2) Whenever one of them changes, reloadGLSLPrograms command is run automatically.
As a result, you can edit GLSL shader file in Notepad++ on the left half of the screen, which TDM on the right half reloads immediately after each save.
You can see some sort of demo:


 
Probably this scenario applies to other TDM resources which are reloadable without restarting map.
Something like reloading texture while editing it in special software, or doing the same with mesh/model.
 
In fact, it is even possible to auto-reload map with something like:

getviewpos                 // this gives coordinates and orientation
dmap {mapname}             // make the updated map playable
map {mapname}              // start the updated map
setviewpos {saved coords}  // teleport to the same exact location

If you have any ideas about using this system to reduce amount of dull work, please share.


  • Bikerdude, rich_is_bored, HMart and 3 others like this

#2 Skaruts

Skaruts

    Member

  • Member
  • PipPip
  • 279 posts

Posted 22 October 2018 - 11:26 AM

This is cool, also for tweaking 3D models. 

 

I never learned how to do this kind of thing in python though. Interacting with other programs and that sort of thing. I wonder what that script looks like.

EDIT: maybe I should look into the automation thing in the wiki, I suppose. Will do that. Hmm, it's not obvious what I should look for... I suppose this isn't a 2.06 thing, as you mention "the most recent TDM". 


Edited by Skaruts, 22 October 2018 - 11:31 AM.


#3 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 22 October 2018 - 12:08 PM

It's a cool idea, although you can have similar fast workflow by binding commands like reloadimages,  reloadmodels , reloaddecls to keys. Not s ure if this all can be automated, as some changes in materials for example require reloaddecls first and reloadmodels second.


Edited by Judith, 22 October 2018 - 12:09 PM.

  • nbohr1more likes this

#4 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9034 posts

Posted 22 October 2018 - 12:11 PM

On a related note, if image_downSizeLimit and image_downSize cvars could reloadImages automatically when set

then we'd have one less quality option that requires a restart.


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#5 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 22 October 2018 - 12:21 PM

This is cool, also for tweaking 3D models. 

 

I never learned how to do this kind of thing in python though. Interacting with other programs and that sort of thing. I wonder what that script looks like.

EDIT: maybe I should look into the automation thing in the wiki, I suppose. Will do that. Hmm, it's not obvious what I should look for... I suppose this isn't a 2.06 thing, as you mention "the most recent TDM". 

I meant the latest experimental/unofficial beta-or-not-beta build, supposedly available on moddb. They won't work with TDM 2.06

The scripts can be downloaded from here, they are supposed to be run from some subdirectory of TDM installation directory.

 

It's a cool idea, although you can have similar fast workflow by binding commands like reloadimages,  reloadmodels , reloaddecls to keys. Not s ure if this all can be automated, as some changes in materials for example require reloaddecls first and reloadmodels second.

Could you please be more specific? Do you mean that changing only image file is sometimes enough, but reloadDecls is required in addition to reloadImages?

 

Note that I have almost zero idea about how this all works.

Unless someone helps me with particular rules, all of this won't work.


  • Skaruts likes this

#6 nbohr1more

nbohr1more

    Darkmod PR, Wordsmith

  • Development Role
  • PipPipPipPipPip
  • 9034 posts

Posted 22 October 2018 - 12:37 PM

Hmm...

 

With testing Materials, I think you can point to a new texture filename and just use ReloadDecls.

You'd only need to ReloadImages if you update the texture image but keep the same filename.

 

I can't speak to models, etc.


Please visit TDM's IndieDB site and help promote the mod:

http://www.indiedb.c...ds/the-dark-mod

(Yeah, shameless promotion... but traffic is traffic folks...)

#7 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 22 October 2018 - 03:45 PM

For example, when you're changing material to include noshadows or noselfshadows, you need to reloaddecls first, but you won't see the effect on your model until you reloadmodels.



#8 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 22 October 2018 - 10:54 PM

For example, when you're changing material to include noshadows or noselfshadows, you need to reloaddecls first, but you won't see the effect on your model until you reloadmodels.

reloadDecls + reloadModels after changing anything like *.mtr.

reloadImages after changing something like *.tga, *.dds, *.jpg.

More examples?...

 

Do I understand right that reloadXXX commands only actually reload files which have changed (I saw that "last modified time" is checked somewhere), so they work pretty fast?



#9 freyk

freyk

    Advanced Member

  • Member
  • PipPipPip
  • 701 posts

Posted 23 October 2018 - 12:31 AM

Its a nice start of a new feature for TDM.

Can I use this for debugging problems in TDM on the system of the user?


Edited by freyk, 23 October 2018 - 12:31 AM.


#10 Judith

Judith

    Advanced Member

  • Member
  • PipPipPip
  • 1644 posts

Posted 23 October 2018 - 01:17 AM

Do I understand right that reloadXXX commands only actually reload files which have changed (I saw that "last modified time" is checked somewhere), so they work pretty fast?

 

Yeah, they don't reload the whole map. If you have this bound to a key, you just press it and the latest texture, material, or version of a model pops into place.


Edited by Judith, 23 October 2018 - 01:17 AM.


#11 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 23 October 2018 - 02:18 AM

Its a nice start of a new feature for TDM.

Can I use this for debugging problems in TDM on the system of the user?

The TDM game and the script are connected via ordinary TCP connection.

So in theory yes, you can ask user to enable automation in his TDM, and then configure the script to connect to remote machine.

If you can reach the player's machine by IP, then it should work.

 

My machine is not reachable directly from internet.

If you are interested to test it out, I can write a simple script.

It would connect to specified IP and allow you (me) to run TDM console commands with getting back whatever TDM writes to console.

 

Note that the system was never designed with network latencies in mind.

It should not matter much for running console commands.



#12 freyk

freyk

    Advanced Member

  • Member
  • PipPipPip
  • 701 posts

Posted 24 October 2018 - 03:20 AM

I want to create a script, that the user runs on their system and speaks to it and to TDM.

Is this feature for the published 2.06 or higher?

Is there any documentation about this automation feature? Cant find anything on the wiki.

 

 

And is there a mini portable package of python?

Like one single binary executable, that runs a python script?


Edited by freyk, 24 October 2018 - 03:30 AM.


#13 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 24 October 2018 - 09:14 AM

I tried to change some texture while being in-game, but all my changes have no effect.

First I tried to change a guard at the main entrance to the tavern in Inn Business, and changed all jpg textures in /models/md5/char/guards/proguard.

Then I tried to change the carpet in the tavern (near two guards playing cards), and changed \textures\darkmod\carpet\runners\geometric01_red_ed.jpg.

Every time I did reloadImages, reloadModels and reloadDecls in all combinations, but nothing changes.

 

Could someone please write complete steps how to change a texture without reloading engine?

I mean something like:

1) Load "..." FM and go to "..." well-known place and look at "..." object.

2) Find jpg file at "..." and modify it.

3) Run "..." and "..." console commands in TDM console.

4) Notice that the object looks different now.

 



#14 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 24 October 2018 - 09:37 AM

I want to create a script, that the user runs on their system and speaks to it and to TDM.

Ok, so you don't need real network.

Is this feature for the published 2.06 or higher?

The TDM-side support was started after 2.06. So you cannot run scripts against TDM 2.06, but you will be able to run them against 2.07.

Also, the latest unofficial build from moddb most likely supports it on TDM side.

The python code itself is contained in assets SVN, I have posted a download link above.

Is there any documentation about this automation feature? Cant find anything on the wiki.

Not yet.

The amount of documentation which will appear on 2.07 release depends on how much functionality I can implement now  :D

Right now it can only help you to reload GLSL, which is a purely developer's feature.

And is there a mini portable package of python?

Like one single binary executable, that runs a python script?

Probably not.

Some people say you can take ordinary CPython installation, configure paths and it should work.

Also, you can convert Python code into standalone executable, although it feels less secure for potential receivers.



#15 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 06 November 2018 - 06:39 AM

In the latest SVN version (not publicly available yet), I have supported user actions in automation.

Believe it or not, but the main incentive of this major work was to be able to bypass "Press Attack to start" screen programmatically.

Now you can use automation to make player walk, rotate view, attack and do various special actions internally known as "impulses".

 

Meanwhile, I did not extend the watcher script.

As I wrote above, I need a help from experienced mappers to implement automatic reloading of anything (except GLSL which I needed for myself).

 



#16 freyk

freyk

    Advanced Member

  • Member
  • PipPipPip
  • 701 posts

Posted 06 November 2018 - 10:55 AM

Is it possible to "beam over" the player to a specified location?

So we can debug a bug in training mission.

http://forums.thedar...ng-mission-bug/



#17 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 06 November 2018 - 11:09 AM

Is it possible to "beam over" the player to a specified location?

So we can debug a bug in training mission.

http://forums.thedar...ng-mission-bug/

What does "beam over" mean?

There is a command setviewpos (and teleport), which puts player to specified coordinates.

In all the publicly available versions it can only set yaw view angle, they have no way to specify pitch angle (although it would be changed in next version).

 

What exactly do you want to do?



#18 freyk

freyk

    Advanced Member

  • Member
  • PipPipPip
  • 701 posts

Posted 08 November 2018 - 05:03 PM

Beam over > Transport.
If people say the some locations slows their system down, you can pinpoint it using those commands.

Using the automation feature, for benchmarking, test settings, debugging, etc.

#19 stgatilov

stgatilov

    Lead Programmer

  • Active Developer
  • PipPipPip
  • 1049 posts

Posted 09 November 2018 - 12:23 AM

Beam over > Transport.
If people say the some locations slows their system down, you can pinpoint it using those commands.

Using the automation feature, for benchmarking, test settings, debugging, etc.

It is possible to write a script which will query FPS every second.

When you finish playing, it will produce file with a list of locations sorted by FPS increasing (x y z + pitch yaw + timestamp).

Then you can take any of these locations and use setviewpos to teleport to it.

Note that if player knows which place poses a problem, you can ask him to write getviewpos, and then use setviewpos to move there.





Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users