Subtitles - Possibilities Beyond 2.11

[Geep]

Additional thoughts:

- Allowing width overrides would mean the width of a subtitle field could not be used as a reliable clue for story vs speech verbosity.

- Another alternative system would be for the engine to look at the subtitle (which might have an embedded \n), and calculate what the width the backing field should be, to be tight but not cause additional wordwrap. That value would be passed to the gui to implement.  There would be minimum and maximum width values. The engine would calculate string pixel length based on the specified font. (I've built utility programs with C# code for that, with carleton and stone fonts).

This would results in variable-width subtitle fields... too raggedy?

It could be further extended to calculate field height as well, if desired

[datiswous]

15 hours ago, Geep said:

#define FIELD_MARGIN is not anywhere official yet, just in my prototype tdm_subtitles_common.gui, hidden in yesterday's post.

I recently used the testSubtitlesCommander.pk4 file as a bases to test some subtitles, but the subtitles were centered small, so I asumed you already used the FIELD_MARGIN, but I couldn't find it.

[Geep]

True. The testSubtitles... just hard-codes the field width. The recent prototype gui was a draft to clean that up by introducing more #defines, in addition to playing with the sound source direction oval. I'll probably revisit this once @stgatilov pushes out a version of 2.12dev with sound source, maybe in time to incorporate it into testSubtitlesSimpleton in progress.

[stgatilov]

I don't like variable-sized subtitles in general, especially if people can set width at will.

If we know speech subtitles are not going to be large, then we can make them always less wide.
Although I'm not certain we can know it in advance.

Maybe we'd better implement some automatic size adjustment for subtitles in the future?
If the subtitle fits one line, then show it on one line.
After that, check 3 tiers of width and show the minimum one that is OK.
Of course, top-middle point of the subtitle box is always at the same location, even if the box is smaller.

[Geep]

3 hours ago, stgatilov said:

If we know speech subtitles are not going to be large, then we can make them always less wide.
Although I'm not certain we can know it in advance.

Don't have to know in advance... just calculate the string width just-in-time, and tell the gui. If you don't like continuously-variable width, it could be just an boolean or small enum passed to the gui, to select from 2 or 3 widths.

I'll post my utility code that (internally) calculates string width. The code - a bit ratty -  would need minor adaptation for the purpose we're talking here.

EDIT: I take that back, it's too hard to explain that pgm. Let me make a fresh pgm that better fits the current purpose.

Edited October 11, 2023 by Geep

[Geep]

Here it is:

calcStringWidth.exe - Oct. 11, 2023 Release. Calculates the display width of a input string, for 4 candidate subtitle fonts Carleton and Stone, uncompressed and compressed.

[stgatilov]

My idea is that the game engine chooses box size automatically.
It means that someone (e.g. me) will have to see where to take width calculate from in the C++ code, and add corresponding logic to subtitle integration code.

It's not like we run some script once and then live with hardcoded settings in .subs files forever.

[Geep]

7 hours ago, stgatilov said:

My idea is that the game engine chooses box size automatically.
It means that someone (e.g. me) will have to see where to take width calculate from in the C++ code, and add corresponding logic to subtitle integration code.

It's not like we run some script once and then live with hardcoded settings in .subs files forever.

Agreed that's a viable approach.

However, it would probably be more flexible if the game engine calculated and reported the string width (using the current font and its aspect ratio), and then let the gui use that to pick from a small pre-determined selection of field widths.

If there are two lines explicitly given, then the game engine would simply report the wider of the two (assuming we continue to use a single background behind both lines.)

If a long line is given without explicit linebreak, then no special treatment is needed... the gui would choose the widest maximum field width and wordwrap would occur.

[Geep]

It would be nice if, from the main menu (presumably under Settings/Audio, where "Subtitles" currently lives), one could have an additional option, "Subtitle Font", with choices:

• Carleton
• Carleton, Wide
• Stone
• Stone, Wide

where "Carleton, Wide" and "Stone, Wide" are the uncompressed fonts, and "Carleton" and "Stone" are the width-compressed fonts. All with the 0.25 scale factor, so in effect 12 pt. (This avoids the concern raised earlier than allowing an arbitrary scale factor to be user-set would result in raggedy-ass text characters.)

Further, change the default from "Carleton, Wide" to "Carleton".

This choice would be appropriately passed to gui's through a global string.

EDIT: This relates to bugtracker 6283

Edited October 13, 2023 by Geep

[Geep]

RE Glitches with the Stone (aka Stone Print) font.

Probably the minor artifacts with this font - mentioned in the bugtracker - are due to the fact that this file is missing from the distribution:

tdm_fonts01\fonts\english\stone\fontimage_12.dat <== MISSING

which I surmise causes the 24pt font to be used instead and scaled down.

There is some reason to believe the missing file was present at one time, because the corresponding .dds file is present:

tdm_fonts01\dds\fonts\english\stone\stone_0_12.dds

And the russian paths have both the .dat and .dds files (2 of the latter in the case of russian) for this font. (I think trying to copy the russian version of .dat to the english branch would fail big time.)

If someone with access to an archive of very early TDM full distributions can check to see if it is available, that would be great. That would be the best way to go, because that file may contain hand-tweaks to the font spacing.

Otherwise, it could be regenerated from the .ttf file and tweaked, a somewhat fraught endeavor (at least if I were to attempt it).

 

[stgatilov]

On 10/13/2023 at 8:25 PM, Geep said:

It would be nice if, from the main menu (presumably under Settings/Audio, where "Subtitles" currently lives), one could have an additional option, "Subtitle Font", with choices:

I'm strongly against this idea, at least until we have automatic box size and are sure fonts are not too different.

Right now allowing choice between several fonts will mean that some subtitles won't fit with one fonts, the others won't fit with other fonts.

[Geep]

20 hours ago, stgatilov said:

I'm strongly against this idea, at least until we have automatic box size and are sure fonts are not too different.

I agree this would be best in conjunction with rollout of automatic box sizing. Just looking ahead to that being available.

I suspect many players would appreciate some font choice.

BTW, if we were to implement font choice *before* automatic box sizing, we would just stick with the box size for the widest font (that would be carleton uncompressed, the 2.11 default), and the other 3 proposed font choices, being less wide, should fit. (I know that to be the case for English version of these fonts; possibly needs to be confirmed for distributed Russian version too.)

Edited October 15, 2023 by Geep

[Geep]

I noticed that there was a pre-existing "carleton_condensed" font, and was wondering how it compares with the other  versions I've floated as subtitle font options. So here's an eyeball comparison, all with the same gui scale factor of 0.25. In this shot, from top to bottom are:

 "carleton_condensed"
 "carleton"
 "carleton@aspect=16:9"

So "carleton_condensed" is intermediary between the two. If "carleton" is considered width=1 (designed for 4:3 screens), and carleton@aspect=16:9 is width=0.75, then "carleton_condensed" is maybe width=0.875

So, since it's pre-existing, it could be another reasonable subtitle font choice... maybe called in Settings "Carleton, Medium Wide"?

[Geep]

Feedback needed.

I plan to do all the "verbosity speech" subtitles for 2.12 (assuming Feb 2023 as target date), while leaving any "verbosity effects" for later. So I need a consensus on what we would like to see covered, or not, by "verbosity speech". Importantly, when you turn on "speech" subtitles in TDM, what would you expect to see or not see?

Easier cases:

• manbeast (hear at tdm_ai_humanoid_beasts02.pk4\sound\voices\monster\manbeast) I would call as "speech", because it's still spoken and in English.
• steambots and automaton (tdm_ai_steambots01.pk4\sound\sfx\ai\) I would call as "effects", because of the non-language sounds & no vocal cords.

Animals and Monsters [no human language spoken; so probably "effects"?]:

• raven (tdm_sound_vocals04.pk4\sound\voices\animal\)
• elemental (tdm_sound_vocals04.pk4\sound\voices\monster\elemental)
• spider (tdm_sound_vocals04.pk4\sound\voices\monster\spider)
• horse (tdm_sound_vocals05.pk4\sound\voices\animal\horse)
• zombie (tdm_sound_vocals05.pk4\sound\voices\undead\zombie)
• werebeast (tdm_ai_humanoid_beasts01.pk4\sound\voices\monster\werebeast)

Edge cases:

• player (tdm_sound_vocals01.pk4\sound\voices\player) [mostly grunts, groans, some hmmms]
• revenant  (tdm_sound_vocals05.pk4\sound\voices\undead\revenant)[a little english, a little latin, and much unintelligible/multiple-voices/sounds/reverb]

 

[Geep]

An alternative view would be to interpret "verbosity speech" as "speech from humans, and noises from all ai that could harm me".

That would exclude ravens and horses, but include everything else. Although the player himself and noises from fixed-location machine threats would then be edge cases. Among latter:

pivoting security cams ("camgoyles")

ticking sounds of mines?

[snatcher]

Hey Geep,

Sorry, I am not following this topic but perhaps this can help in getting some feedback.

In TDM 2.11 we have:

• Story: Display only story-related subtitles
• On: Display subtitles for all speech
• Off: Disable subtitles

Can you explain in few words without going technical..?

• Changes / improvements already in place
• Short term goals (achievable or not)
• Long term goals (wishes)

Does the current scope have anything to do with hearing impaired? Just asking because it is not clear to me: hearing impaired have very special needs the general audience don't need (nor want).

[Geep]

A subtitling capability was developed that offers 3 "verbosity" categories: "story", "speech", and "effects". Roughly, "story" are FM-specific talk clips, while "speech" is for ai barks, and "effects" are more general sounds. "Effects" is not yet exposed as a setup choice, nor are any sounds so tagged.  I have never seen an exact differentiation of the 3 categories... which is what the feedback request is about.

At the time I started this thread, there was only a few demo phrases tagged as "story" or "speech" and given subtitles. I've made many more "speech" subtitles available for eventual incorporation into 2.12 (and others have done "story"), but I'm not doing that incorporation myself.

As to purpose, I surmised they are -

- to help low-hearing players

- to help players for whom English is not native

- to demystify more obscure ai dialects

 

and, in conjunction with visual cues to sound source location (prototyped)...

- for players with audio turned off

- for deaf players

Hope that helps

 

 

 

[snatcher]

Thank you.

I see two different models: Hearing Impaired (HI) and Hearing Unimpaired (HU).

The HI model includes low-hearing, deaf, and players with audio turned off for which all sounds are relevant:

• Plot (key conversations, special effects such as footsteps upstairs...)
• Surroundings (generic barks, doors...)
• Environment (machinery, thunders...)

You somehow have to let players know how an event (sound) relates to them and their actions. I understand italics and colors are not available so you probably have to use brackets, in example, for anything that is of no concern to the player (but still relevant).

You then have to let players know where sounds are coming from and it seems this is going in a good direction (congrats).

A different story is how to make HI players know how much noise they are making...

The HU model would be for:

• Players for whom English is not native
• Translations for obscure ai dialects or accents
• Support for bad audio or recordings

I think we don't need effects in this model, players can hear effects already. I have a question at this point, what happens if a key conversation and general barks take place at the same time?

[Geep]

There can be up to 3 subtitles seen at a time, stacked with translucent backdrops. If there are more the 3, then priorities are involved, with "story" highest and "effects" lowest.

I agree that distinguishing more important from less important subtitles is desirable. One method was floated (background field width) but maybe that's not the answer. Could be background field color tint. I would not support putting all barks in brackets.

[Geep]

Effects in brackets might be OK.

A concern with color field tint is color blindness accommodation, as well as problems translucency can cause. I guess another idea is to use one font for story, a different one for speech. Instead of making font a user choice. Dunno.

[snatcher]

Unless you come up with fancy or elaborated alternatives you should do your best to limit how you present information to two options:

• Regular font and Italics
• No brackets and brackets
• White color and yellow color
• ...

The moment you introduce a third option players may not understand what's what anymore, or if more options are to be expected.

13 hours ago, Geep said:

I agree that distinguishing more important from less important subtitles is desirable. One method was floated (background field width) but maybe that's not the answer. Could be background field color tint. I would not support putting all barks in brackets.

I think it is important to understand how current events relate to the player and convey the information properly and you could probably have code support to know the current AI alert level (or similar tricks).

If the AI alert level is high they will most likely be shouting at you:

Quote

I shall slay thee intruder!

Otherwise:

Quote

[Who is making noise over there?]

[Geep]

I appreciate your desire for simplicity, but since the current system has 3 categories, you are actually proposing to split the barks category, creating 4 in total -

"story" (most vital info to know if you are to win)

- ai barks, dangerous now

- ai barks, not dangerous yet (but ai presence is important to know)

- effects, generally ignorable

Now, they don't all have to be visually distinguished... maybe 3 & 4 could be shown the same. Dunno.

[snatcher]

13 hours ago, Geep said:

Dunno.

I don't know half of the story so neither do I. I am just trying to help by putting some elements on the table for you to evaluate.

Comics could be a good source of inspiration.

[stgatilov]

I would agree with @snatcher, and I think it was the original idea.

Speech level is for people who hear sounds well, but don't understand spoken English well (like me), especially if it is stylized (e.g. drunk).

Effect level is something even more widespread. It might be useful in the distant future for people who have problems with hearing or just lack speakers/headphones today for whatever reason. I heavily doubt we'll be able to support this case properly, but added this level early anyway. For instance, "Heyyah!" should most likely be on Effect level.

Story level was added as default for TDM veterans who also hear well but don't understand English well (like me). I don't want to see these "I've got him cornered here!" and "Just you wait!" all the time, it's enough that I hear them all the time

Whether speech is dangerous or not does not matter. If some speech has angry tone instead of quiet, then you can add exclamation mark to carry this information (useful in case player has hearing problems). But whether guards are just talking calm or already want to hang player up --- that he should understand himself by reading the text and listening to intonation.

Actually, I don't like the idea of giving any visual difference between Story and Speech for this exact reason. We use subtitles to help player understand English, not to help him draw correct conclusions from it. He should be able to understand himself whether some text is important for his quest or interesting, or not.

[Geep]

@stgatilov, thanks for sharing your thoughts about the original ideas behind the 3 levels, which I had not previously seen put so well. I was surprised that you considered that some of an individual AI's barks might go under "effects" instead of "speech". @snatcher likewise thought that, in effect, an AI's barks should be divided, though draws the dividing line elsewhere.

I agree with you about the primary existing TDM audience for subtitles: those players appreciating help with spoken English, specially in dialects. I think partially-impaired-hearing individuals will also benefit.

But, given subtitles in TDM and our newly prototyped method of indicating speaker sound source, I think we have an opportunity to enlarge the potential TDM community to include deaf players, or those who like to play games with sound off. This does involve sound categorization and visual cues, beyond just toggling story/speech/effects. Such cues could be universal (seen the same by everyone), or controlled by new options, to accommodate personal interest (or lack thereof).

Let me be more specific. Consider the 2-level visual cues that snatcher preferred (while leaving open where to draw the line between the 2, or whether 2 is really too few). Examples of universal+fixed, which for barks could be done by me in the .subs file, are:

• bracketed vs unbracketed
• white vs yellow font, done by string markup. (This has significant drawback and limitations).

Alternatively, there is categorization performed engine-side and passed to the GUI as one or more variables. The results could be universal, or the passed value(s) could be affected by additional optional settings. Displayed results could be, for instance:

• white vs yellow font, done by text font color choice
• Stone vs Carleton font
• wide vs narrow font
• wide vs narrow background field
• two different background tints or opacities
• colored border around backgrounds, with two different colors

(Also, without involving new GUI variables, the engine could add brackets to selected strings)

Turning to what additional optional settings might look like, at the basic level, you could just have an option that turns visual cues on or off. So, for example, if the visual cue was font choice, turning the visual cue option to off would cause all subtitles to use the same default font. More specificity is, of course, possible.

(Maybe too, stgatilov, you could sneak in a related option to suppress "Just you wait!" audio and only play story clips!)

If the barks were to be subdivided by the engine into (in this discussion) 2 categories, it could be done by a new per-line option in the .subs file, e.g, -cue 1.

Or the categorization could be done entirely automatically on-the-fly by the engine, at the moment it's looking at the AI's current state & most recent state transition, in order to select a sound shader to play. It would probably use the standardized state-transition names as found in the AI .def files, e.g. "snd_foundDeadMale" rather than AI-specific sound shader names. So there would be a list of "snd_..." to determine when g_cue=1 instead of the default g_cue=0.