Feedback on wiki "GUI Scripting Language" Series

[Geep]

I'm wrapping up a new series, with hub named "GUI Scripting Language" and children (except preexisting) named "GUI Scripting: ..."

If you find an error for me to fix or otherwise recommend an improvement, please mention it here. Thanks.

[datiswous]

Just an idea: Make a subcategory of category Editing called GUI and put everything GUI related under it, so it's easier to find.

( Curently there is no letter G in the list of subcategories: https://wiki.thedarkmod.com/index.php?title=Category:Editing )

Edited November 3, 2022 by datiswous

[HMart]

I have not read your wiki about the gui scripting, so sorry if this was already known by you and talked about, but if not, imo is always good to know that gui scripting language, can do a little more than what idSoftware used and talked about. Unless i failed to see where idSoftware talked about this stuff.

I discovered this by accident, when I tried a few stuff that Doom Script supports, to see if they worked with gui scripting and they did.

For example the gui language, other than #include also supports #define, just like Doom Script, thou in a more limited way, so you can do basic text substitution using #define like:

#define COLOR_WHITE 1,1,1,1

#define WIND_HEIGHT 480

#define WIND_WIDTH  640

or more complex

#define RECT_SIZE   ((WIND_HEIGHT/2)-(320/2)),((WIND_WIDTH/2)-(240/2)),320,240

then used as:

rect RECT_SIZE 

-----------

It also has basic support for pointers! using $ sign, example:

transition "start_white::matcolor" "$Desktop::COLOR_WHITE" "$Desktop::COLOR_BLUE_07" "1000"; 

and other capabilities that I may be forgetting, have not messed with gui scripting for ages now.

 

Thou i remember macro and pointers support was not free of problems, there's some idiocentricities with this stuff, for example macros using #define and defined in the global space, meaning outside the main desktop windowdef or any other windowdef, work globally and are accessible directly by name (just like in Doom script), but seem to work, only in some things, like "rect", "backcolor" and others but not in transitions, I remember having trouble with those, the only thing that worked well for this ones, was defining the "macro" in another way, like this and inside the main desktop windowdef:

definevec4 "COLOR_WHITE"  1,1,1,1

Despite  #define COLOR_WHITE 1,1,1,1  and  definevec4 "COLOR_WHITE"  1,1,1,1 looking similar and apparently do the same thing, behind the scenes they seem to be handled totally differently.

I don't claim to have tested this macro stuff very deeply but imo has potencial to make gui scripting life easier, but caution in the few tests I made, transitions using macros, were the most unstable GUI feature, they work the best inputting values directly, specially because those are separated by spaces not commas. Thou using pointers like I show above, even using colors defined using commas worked, sometimes...

I think this, spaces versus commas, most be why macros and pointers were mostly ignored by idSoftware, most of their GUI's only use a "safe" subset of the GUI script language and I unless I missed it, I never saw #define being used in any of their gui's.

But they do work in the basic tests I made, Gui scripters just need to be aware of the limitations.

Edited November 3, 2022 by HMart

[Geep]

Thanks, HMart. Much of what you mentioned is touched on in the subtopics "GUI Scripting: Preprocessor Directives" and "GUI Scripting: User Variables". In TDM, a group of #defines are used to support the main menu. In some cases, a vector is #defined twice, with and without commas. I sort of de-emphasized definevec4, since that didn't seem to be used in TDM. Possibly because of problems you noted?

The "$" feature (which I guess could be called a pointer) I discussed just under Event Handler commands, specifically "set" and "transition". I saw no evidence it can be used elsewhere.

I'll review what you said and see what further might be added to what's already there.

Edited November 3, 2022 by Geep

[Geep]

6 hours ago, datiswous said:

Just an idea: Make a subcategory of category Editing called GUI and put everything GUI related under it, so it's easier to find.

( Curently there is no letter G in the list of subcategories: https://wiki.thedarkmod.com/index.php?title=Category:Editing )

Good idea. I'll have to research how to add a new wiki template.

[stgatilov]

2 hours ago, HMart said:

For example the gui language, other than #include also supports #define, just like Doom Script, thou in a more limited way, so you can do basic text substitution using #define like:

Gui language has full-blown support for preprocessor macros.
In fact, it is done via the same exact code as in game scripts.
If something does not work with them, it's either some custom settings of GUI lexer, or just the GUI language not understanding what you pass to it.

In fact, C preprocessor is the only way to implement reusable code in GUI scripts.
But in order to use it properly, you need to be an expert in C, with stuff like #include and include guards, ## -- concatenation and # -- stringization, and know about single expansion rule, etc.
Just look at tdm_objectives_core.gui and tdm_subtitles_common.gui in the latest dev build, and you'll know what I mean,

Quote

It also has basic support for pointers! using $ sign, example:
transition "start_white::matcolor" "$Desktop::COLOR_WHITE" "$Desktop::COLOR_BLUE_07" "1000"; 

This is not pointer for certain.
It is a very specific hack to link to variable specifically in Transition command.
The rules regarding what is considered a string and what can be turned in to a link are not obvious and consistent, as well as the rules about when engine evaluates all the expressions/conditions.

Quote

Thou i remember macro and pointers support was not free of problems, there's some idiocentricities with this stuff, for example macros using #define and defined in the global space, meaning outside the main desktop windowdef or any other windowdef, work globally and are accessible directly by name (just like in Doom script),

Yes, macros don't have namespaces, so you should always prefix your stuff with your domain, or/and undef your macros when they are not needed.
Actually, the GUI windows have no namespaces either, so you can get name collision here just as easily.

You need a lot of discipline (like in pure C programming) in order to work with large codebase in D3 GUI.
Even with the recent 2.11 improvements which add previously nonexistent error reporting.

Quote

but seem to work, only in some things, like "rect", "backcolor" and others but not in transitions, I remember having trouble with those,

According to the code, dollar only works in Set and Transition right now.

Quote

the only thing that worked well for this ones, was defining the "macro" in another way, like this and inside the main desktop windowdef:

definevec4 "COLOR_WHITE"  1,1,1,1

Despite  #define COLOR_WHITE 1,1,1,1  and  definevec4 "COLOR_WHITE"  1,1,1,1 looking similar and apparently do the same thing, behind the scenes they seem to be handled totally differently.

This is not a macro at all.

Here you define a custom window register.
And you reference it only in some locations, as far as I remember.

Quote

I think this, spaces versus commas, most be why macros and pointers were mostly ignored by idSoftware, most of their GUI's only use a "safe" subset of the GUI script language and I unless I missed it, I never saw #define being used in any of their gui's.

They probably ignored the advanced GUI scripting because it did not work... or the vice versa: they did not use it, thus it did not work
Doom 3 code had so many bugs in GUI handling, and lack of any error reporting. I am impressed that TDM guys managed to implement such a complicated main menu with this mess and maintain it for years.

As for C preprocessor, I don't know.
Maybe they did not reuse any of their GUI code, simply copypasted it around without any worries about future maintenance.

[HMart]

Hey thanks that does teach me a lot about how the gui language works and clears some things, thou I did knew definevec4 was not a macro, that is why I put macro, word within quotation marks.  

Edited November 3, 2022 by HMart

[stgatilov]

I recommend going to changelog and roadmap in bugtracker and looking through issues that are about GUI scripting.
I'm pretty sure there were some major changes which are worth taking into account when writing a tutorial.

Maybe even give a list of these tickets somewhere, because if people come from other Doom 3 mods, they can get wrong impression, since stock Doom 3 engine has more bugs and some different behavior.

Speaking of putting double-quotes vs not putting them: I think it is a good idea to enclose every item in double-quotes.
Literal numeric values are perhaps the only exception.

Internally, lexer breaks the text into tokens, and each token is treated as single item. Characters like colon, dot, or dollar break name into several tokens, so you have to enclose such items into double-quotes.
In fact, adding double-quotes is never a problem since lexer always removes them, emitting their contents as single token of type "string". The only case where it is a problem is with literal numeric constants, since the GUI parser expects to see them as tokens of type "number".

Regarding "initialization" and "updates" of properties.

When you declare a property, you can write any expression on the right side, potentially referencing some non-constant things there. Normally, when some of these things change, the property is reevaluated automatically. However, this can get you into trouble is you assign something into the property using Set command... so there is a concept of "active" and "non-active" properties, or "enabled/disabled" properties (not sure about canonical terms).

Some things disable/deactivate a property.
The rules of original Doom 3 were weird in this regard (6028), but in upcoming TDM 2.11 I think the only way to deactivate a property is to Set something into it, or start a Transition of it. Deactivation happens in runtime, not statically, so the property will keep updating before the first Set is done.

 

The section User Variable says this is OK syntax:

float crouch_scale=0;

I'm sure this is not correct, and hope that recent dev builds produces a warning on it.

String comparison section says that there is some special "quoted comparison operator":

if ("LSGSaveGameNameEntry::text" "!=" "") {

I don't recall seeing anything specific regarding double-quotes around operators. Moreover, GUI parser cannot see double-quotes, since lexer will replace it with a single two-character token of string type, that won't work properly with GUI parser.
Moreover, I don't see any occurance of quoted "!=" in current .gui files, so I guess it was a buggy code that got fixed.

I think there is no string comparison, because all expressions and their intermediate values are floats, they simply cannot hold strings.

 

In Precedence section, there is discussion about parentheses.
Yes, you can put parentheses in GUI expressions, and rely on normal evaluation order otherwise.

The discussion about macros here might be a bit confusing, because GUI engine has nothing to do with #defines, in fact it does not even see them. Just like in C language, all the directives starting with #sharp are handles by totally separate thing called preprocessor, the GUI parser will see all macro references replaced by whatever they point to, so for the GUI parser it would be just a gigantic expression with two levels of nested parentheses.

I would probably be better to stress this in tutorial, because this understanding of preprocessor as an entirely separate step can help on many occasions.

 

There is Multiple If section, which totally lacks the most important caveat of GUI ifs that usually kills all attempts of writing any complicated logic in GUI scripts!

While you can write expressions inside If conditions (and now also on the right side of Set command), they are evaluated not when you normally expect it to!
All the expressions are evaluated before script commands are executed. So if you Set the value of "gui::myvar" from 0 to 1, the following "if (gui::myvar)" will still be considered false.
I believe all the expressions are evaluated at the same time, regardless of whether they are on the right side of property definition (aka 'visible "gui::LootIconVisible"') or inside script.

This is very important to understand when you write expressions, since normal people assume that script commands are evaluated sequentally, and their changes are propagated immediately.

And yes, you can write nested ifs, but with the caveat of early evaluation they probably won't give you much.

UPDATE: I see it mentioned in Changing User Variables.
And yes, it should apply to properties too, simply because the problem is on the other end: the change is applied immedately, but conditions and expressions are evaluated earlier. Perhaps correct this misconception here, and mention it in Multiple If section too (never too much).

Regarding onTime event.
You wrote there '"reset" command', perhaps change it to '"resetTime" command'.

Regarding onTime 10 instead of onTime 0 in main menu.
I think one reason of doing so is to ensure order of triggering handlers.
Imagine that you have 50 "onTime 0" handlers, but you want to ensure that some of them are evaluated after the others. Adding small time offsets is the only way of hacking around it.

 

Regarding vector components.

The original Doom 3 code intended to make them accessible via indexation, like v[0], v[1], v[2], v[3]. But the code had a terrible bug, so it did not work. In TDM 2.11, it works (6028).

UPDATE: also here

[stgatilov]

Regarding ResetTime.

I don't think I changed this code, so most likely value was mandatory from the very beginning. And if you omit it... something will happen. Crashing is a best-case scenario
TDM 2.11 should warn about wrong usage, and hopefully not crash afterwards...

Quote

resetTime "InventoryPickUpMessage" "1900";

I suppose this stopped working in TDM 2.11, hopefully with a warning.

Regarding Set command.

I think you should mention in a separate subsection that TDM 2.11 allows expressions on the right side of Set (6028).
But please add the same ugly disclaimer that these expressions are actually evaluated before script handler starts executing.

There is a page with BNF.

I think when formal grammar is usually given only when it is 100% correct and ready to be put into a parser to parse code. I doubt anyone would do anything like that. So... what's the purpose then?

 

[stgatilov]

Overally, this is veeery detailed documentation!
I think I looked through most of it, except for various listDef, editDef and stuff like that.

I definitely support the idea of putting it into its own wiki category, since it consists of like 20 articles!

P.S. Maybe we should add support for:

1. runScript command: It seems to be supported in source code.
2. namedEvent: I got the same idea myself at some moment.

?

[datiswous]

16 hours ago, Geep said:

Good idea. I'll have to research how to add a new wiki template.

I don't know why you need to create a new wiki template, but I just created the category GUI and added the GUI Scripting Language page to it and made category GUI a subcategory of the Editing category, so the Gui category is listed here:

https://wiki.thedarkmod.com/index.php?title=Category:Editing

If this was not how you wanted it, you can revert it. I have not understood yet where a template is used for.

 

Edit: I have read a bit about wiki templates. Very interesting.

https://wiki.thedarkmod.com/index.php?title=Assigning_your_Page_to_a_Category

I guess you can just use one of the existing templates to build on:

https://wiki.thedarkmod.com/index.php?title=Category:Templates

 

Edited November 4, 2022 by datiswous

[HMart]

3 hours ago, stgatilov said:

Speaking of putting double-quotes vs not putting them: I think it is a good idea to enclose every item in double-quotes.
Literal numeric values are perhaps the only exception.

I remember some idSoftware dev on Doom3World forum saying exactly that, they also said that in 

float "somevar" "0"  the "0" is ignored 

Quote

Additional user variables may be defined with the 'definevec4', 'definefloat', and 'float' commands. Note you cannot set the initial value for the variable (it will always be 0). There are guis in Doom 3 that specify an initial value, but it is ignored

From iddevnet, but you can look at the c++ and see if is true or not just to make sure.

 

3 hours ago, stgatilov said:

The section User Variable says this is OK syntax:

float crouch_scale=0;

I'm sure this is not correct, and hope that recent dev builds produces a warning on it.

That is indeed not correct, unless is for Doom script...

idSoftware says for Gui script user var definitions you should do:

float myfloat

or 

float "myvar" "0"    // see above about zero or other value being defined here at user var definition

 

3 hours ago, stgatilov said:

String comparison section says that there is some special "quoted comparison operator":

if ("LSGSaveGameNameEntry::text" "!=" "") {

I don't recall seeing anything specific regarding double-quotes around operators.

Hum you maybe right but Doom 3 main menu does uses that "!=" 

if ( "SGC1Title::text" "!=" "" ) {
		set "forecolor" "0.6 1 1 1" ;
		set "SGCBtn1EdgeG::matcolor" "0.5 0.7 0.8 0.5" ;
		set "SGCBtn1BorderG::matcolor" "0.5 0.7 0.8 0.5" ;
		set "SGCBtn1CornerG::matcolor" "0.5 0.7 0.8 0.5" ;
		set "noevents" "0"
}

 

3 hours ago, stgatilov said:

There is Multiple If section, which totally lacks the most important caveat of GUI ifs that usually kills all attempts of writing any complicated logic in GUI scripts!

While you can write expressions inside If conditions (and now also on the right side of Set command), they are evaluated not when you normally expect it to!
All the expressions are evaluated before script commands are executed. So if you Set the value of "gui::myvar" from 0 to 1, the following "if (gui::myvar)" will still be considered false.
I believe all the expressions are evaluated at the same time, regardless of whether they are on the right side of property definition (aka 'visible "gui::LootIconVisible"') or inside script.

This is very important to understand when you write expressions, since normal people assume that script commands are evaluated sequentally, and their changes are propagated immediately.

And yes, you can write nested ifs, but with the caveat of early evaluation they probably won't give you much.

This is awesome info and explains why some of my ifs didn't seem to do what i expected.

but is that

Quote

"gui::myvar" from 0 to 1, the following "if (gui::myvar)" will still be considered false

still true, if you change it through c++, using "_gui->SetStateInt("gui_var", value);" or is only when doing it from Doom Script?

[HMart]

4 hours ago, stgatilov said:

namedEvent: I got the same idea myself at some moment.

Something like this.  

Quote

guiCommandDef_t commandList[] = {
.....
    { "namedEvent",            Script_NamedEvent, 1, 1 },
.....
};

 

/*
=========================
Script_NamedEvent
=========================
*/
void Script_NamedEvent(idWindow *window, idList<idGSWinVar> *src) {
    auto* parm = cast(idWinStr*)((*src)[0].var);
    if (parm) {
        window->GetGui()->HandleNamedEvent(parm->c_str());
    }
}

not sure if well done but works for me and was really easy, don't know why idSoftware never implemented it originally, seems very obvious and useful, only Quake 4 has similar ability.

 

edit: 

Took a fast look at the wiki being done and saw this there...

Quote

HMart in Ref 10 recommended TDM try to implement a way for GUIs to "use colors defined with #define on transitions as well [as float parameters], instead of having to use definevec4 in the Desktop window."

That is wrong and not what I recommended at all, if I passed the wrong idea then my bad, was not my intention.  What I really said, is exactly the contrary, what i said is that I add problems using, in this case colors, defined with preprocessor macro #define in transitions, only when using definevec4 for that coupled with the $ sign did they work.  But the better and widely used method by idSoftware was to input color values directly!

Also #define macros aren't really "float parameters", like definevec4 or definefloat, they are exactly like C macro preprocessors like stgatilov said, just text substitution.

Edited November 4, 2022 by HMart

[Geep]

Wow, lots of great feedback this morning. I'm going to need more coffee!

[Geep]

12 hours ago, datiswous said:

I have not understood yet where a template is used for.

Yeah, me either. But many of the existing TDM wiki pages seem to use a template, so I'll be looking those and through the mediawiki docs about templates today to fully understand pluses/minuses. Moving cautiously because it's easy to create or edit a mediawiki page (evidently including templates), but almost impossible to delete a page.

[Geep]

Got it now. TDM likes to define category templates with an optional pagename parameter, so that if need be a particular page can be placed in a non-default location in the alphabetic listing, e.g., instead of:

T

This page is about Foo

...it's...

F

This page is about Foo

I'll do likewise.

[datiswous]

On https://wiki.thedarkmod.com/index.php?title=GUI_Scripting_Language

Quote

Deploying Stock GUIs or the Entities that Use Them

I think this can be added: https://wiki.thedarkmod.com/index.php?title=Xrays_in_GUIs

[Geep]

Xrays in GUIs added. That and about 4 dozen other pages tagged in category "GUI". My brain is now dead. Time to step away from the screen.

@HMart, sorry about that misinterpretation of what you said. I'll revisit and correct that tomorrow. And then start to delve into the many points you and @stgatilov raised.

[Geep]

@HMart, you took issue with me saying:

Quote

HMart in Ref 10 recommended TDM try to implement a way for GUIs to "use colors defined with #define on transitions as well [as float parameters], instead of having to use definevec4 in the Desktop window."

This was my summary of what I thought you meant in these post fragments from a forum linked to by Ref 10 (namely, https://forums.thedarkmod.com/index.php?/topic/20100-idtech-4-gui-scripting/#comment-439778 )...

Quote

[After showing some code that experimented with macros and definevec4s in transition statements.] Right now by the looks of it I should avoid using #define macros and just do it like idsoftware did, work on the values directly. But man macros would really make life much easier...

[Responding to duzenko's offer to make C++ changes if needed...] Another thing you could try to implement, is make it so anyone making TDM guis can use colors defined with #define on transitions as well, instead of having to use definevec4 in the Desktop window, i'm sure that will help everyone using idtech 4.

I see I probably shouldn't have added the "[as float parameters]" to the ambiguous "as well" phrase, and perhaps "recommended TDM try" is too strong. Overall, would this be better?

"In Ref 10, it is noted that in transition statements, a 4-value color vector must be either a literal or (less reliably) a definevec4. Using a #defined macro for the color vector won't work currently... an idea for a future improvement?"

EDIT - I see that a broader revision was needed. So changed to this in

https://wiki.thedarkmod.com/index.php?title=GUI_Scripting:_TDM_vs_Doom_3,_Quake_4

====transition 4-6 parameters====
Often, transition statements have a pair of 4-value color vectors as their 2nd & 3rd parameters. As the discussion in [[GUI Scripting: References & Resources | Ref 10]] indicates, each vector is traditionally represented by a double-quoted literal:

    transition "matcolor" "1, 1, 1, 0" "1, 1, 1, 0.8" "300"

This was id Studio's preferred method, but using a definevec4 user variable was a possible alternative. In addition, TDM can #define colors for transitions. Unfortunately, due to syntax differences, for a particular color, it is not possible to create a single #define that would work with both properties and transitions; thus:

    #define INACTIVE_COLOR    0,0,0,0.50
    #define SINACTIVE_COLOR  "0 0 0 0.50"

See [[GUI Scripting: Preprocessor Directives]] for further examples. (Possible future improvement: Changing TDM's parsing to let a transition also accept color vectors with commas.)

See also the discussion above about 4vect properties and _x, _y, _z, _w suffixes.

Edited November 6, 2022 by Geep

[Geep]

Considering your comments, starting with preprocessor directives. More soon.

[Geep]

On 11/3/2022 at 6:29 PM, stgatilov said:

P.S. Maybe we should add support for:

1. runScript command: It seems to be supported in source code.
2. namedEvent: I got the same idea myself at some moment.

?

Regarding (1), it clearly was supported at one time, and would seem valuable. Did it cause problems in the past, or was just neglected as unneeded or too tricky when TDM engine merge occurred? Dunno. I'm a fan of this, because it will make GUI debugging easier.

HMart is supportive of (2), calling a namedEvent from elsewhere in a GUI. I'm neutral on this.

Is someone would like to add these to the bugtracker as feature requests, I'll reference them in the wiki writeups.

[Geep]

@HMart, @stgatilov Let's see if the 3 of us (and anyone else with a stake) can agree on the best initialization syntax of User Variables in a property list, which I discuss in 2 places in the wiki. As we know, a float user variable is ALWAYS initialized to zero.

Below are possible syntax forms, which I have numbered, and in the comments given my opinion

1a) float myvar 0 // recommended form. 0 is only real value, but needed for proper parsing. Matches syntax style of properties.

1b) float myvar "0" // alternate form

1c) float "myvar" "0" // alternative form (but I hate)

1d) float myvar=0; // DEPRECATED (float crouch_scale=0; from mainmenu_settings_guisize.gui)

2) float myvar; // alternate form with semi-colon, also initialized to 0.

2b) float "myvar"; // alternate form (but I hate)

3) float myvar // BAD. Parser may or may not recover from this, depending on what follows. (I've seen this fail)

3b) float "myvar" // Probably BAD

4a) float myvar 1 // MISLEADING, because variable is still initialized to 0.

4b) float myvar "1" // MISLEADING.

4c) float "myvar" "1" // MISLEADING.

[stgatilov]

39 minutes ago, Geep said:

As we know, a float user variable is ALWAYS initialized to zero.

That's not true.
It is initialized to whatever you set as the next token (with semicolon meaning zero right here).
 

Quote

1a) float myvar 0 // recommended form. 0 is only real value, but needed for proper parsing. Matches syntax style of properties.

2) float myvar; // alternate form with semi-colon, also initialized to 0.

These are the ones I know of.
 

Quote

1d) float myvar=0; // DEPRECATED (float crouch_scale=0; from mainmenu_settings_guisize.gui)

3) float myvar // BAD. Parser may or may not recover from this, depending on what follows. (I've seen this fail)

3b) float "myvar" // Probably BAD

All of these are certainly errors.
 

Quote

4a) float myvar 1 // MISLEADING, because variable is still initialized to 0.

I just checked that if I set variable it 1, the condition that checks it == 1 works, but if I change the condition to != 1, then it does not work. So this is OK way to initialize to 1.

[HMart]

Write what stgatilov says, I didn't looked at the code like he did, only transmitting what idSoftware has said themselves in their old wiki but even their employees, could have been wrong when writing that or it did worked like that but then they updated the gui handling code and forgot to updated the wiki.

Only way to be sure, is to look at the c++ code and or test a GUI.

[Geep]

Quote

4a) float myvar 1 // MISLEADING, because variable is still initialized to 0.

 

17 hours ago, stgatilov said:

I just checked that if I set variable it 1, the condition that checks it == 1 works, but if I change the condition to != 1, then it does not work. So this is OK way to initialize to 1.

Last month, when I tested, I recall I got the opposite result... it was still 0. So something's screwy.