Jump to content
The Dark Mod Forums

Subtitles - Possibilities Beyond 2.11


Geep

Recommended Posts

Subtitle capabilities available in 2.10/2.11 are described in the wiki "Subtitles". For info about one new project using these capabilities, see the thread "English Subtitles for AI Barks". That project includes development of a TDM subtitle style guide. The thread here is mainly focused on ideas that would require future engine changes to implement.

A. Automatic Switching Between Subtitle Languages Based on Settings

This is a follow-on of a forum discussion between datiswous and Geep.

It is desirable to have a standardized method for:

  • subtitles to be authored in multiple languages, and so packaged and distributed as part of TDM and its FMs
  • a gamer to dynamically select subtitles in a desired language

The subtitle language choice might be controlled by

  • the existing TDM language choice within Settings, or
  • a separate "Subtitles Language" setting.

B. Other Possible Improvements Requiring Engine Changes

Most of these involve allowing the subtitle code to know something about game entities, and convey that to the gamer. Examples of useful knowledge:

  • the ID of the speaker (or other source)
  • the location of the speaker, relative to the player, the player's view, and other speaking AI.

This type of information helps to disambiguate when there are multiple voices present. There are lots of ways such info is visually conveyed in other subtitle/caption systems, and might be added to TDM.

C. Providing Subtitles for Existing FX

These would have the "verbosity effects" subtitle tag. Beyond 2.11 capabilities, some of the location aspects of (B) would be pertinent.

Edited by Geep
Add links
  • Like 2
Link to comment
Share on other sites

Languages... maybe it's time to finally get into this mess.
I'm afraid there that would be a long story, and subtitles won't even be the first issue.

Regarding ID of the speaker.
Let's suppose we pass the entity of the speaker into the subtitles code.
How exactly should it be used?
It would make no sense to automatically add names like "tdm_guard_generic07: I've got him cornered!"

  • Like 2
Link to comment
Share on other sites

COPIED FROM "English Language Subtitles for AI Barks". MORE RELEVANT HERE...

@datiswous, I imagine the TDM fonts don't support bold, italics, underline, etc. Maybe just different sizes.

I don't think color by itself would be helpful for speaker identification, unless there was a color halo or name tag above each vocalizing AI. Might be useful for word emphasis, though.

@stgatilov, if there is location data available to the subtitle code, that could be used in various ways, using a different GUI:

1) Let the 3 slots be either left justified, centered, or right justified, depending on relative speaker location (including off screen). This could be implemented by 9 actual windowDefs (all of same size; 3 each for each current slot, overlaid, with each of the 3 justifications.) Alternatively, with a 3x3 grid, all windowDefs the same size, but 1/3 the current width.)

2) Or instead, at the top edge of each slot, show a short horizontal bar, whose left/right position is moved to be under the relevant AI. If off screen, add an arrow head (<-- or -->).

Naming is harder, if the subtitle code can't get at that info. Though at the point the sound engine is passed the sound to render, presumably it knows the speaker, and could independently and in parallel visualize the name information (particularly if the sound engine passed back which GUI slot it was going to use for subtitle.)

It is true that associating a name (say, using a small tab-like field) with a slot is less useful when the player doesn't know the AI names (i.e., with no floating names above the characters).

If the technical issues of naming could be overcome, then there's still the question of when you'd want to give a name (Rupert) versus type (thug #3) versus generic (speaker #2). Also have to handle special cases of (narrator) and (player).

Dreaming... Instead of text names, more fun would be to show a thumbnail face of each AI next to their slot. With a question-mark face when they are off-screen?

  • Like 2
Link to comment
Share on other sites

7 hours ago, stgatilov said:

It would make no sense to automatically add names like "tdm_guard_generic07: I've got him cornered!"

Names that were autogenerated could be recognized and shortened, e.g., "guard #7: " or maybe in parentheses "(guard #7) " as some captioning systems prefer. (The post I copied above has other approaches, where the name text is separate from the caption text, or faces are used.)

Link to comment
Share on other sites

9 minutes ago, Geep said:

if there is location data available to the subtitle code, that could be used in various ways, using a different GUI:

1) Let the 3 slots be either left justified, centered, or right justified, depending on relative speaker location (including off screen). This could be implemented by 9 actual windowDefs (all of same size; 3 each for each current slot, overlaid, with each of the 3 justifications.) Alternatively, with a 3x3 grid, all windowDefs the same size, but 1/3 the current width.)

2) Or instead, at the top edge of each slot, show a short horizontal bar, whose left/right position is moved to be under the relevant AI. If off screen, add an arrow head (<-- or -->).

I don't think naming is a good idea, but this idea of location hints are good.
I think they can reduce the confusion from several people speaking simultaneously.

Perhaps keep brainstorming in this direction 😃

The problem with different positioning of text is that if you move, the text would suddenly jump to a different place, and it does not help reading. On the other hand, having smoothly changing position of subtitle in realtime might be hard due to GUI system interaction... although we can probably write just positions into gui::XXX variables and make "rect" of windows depend on it.

Another possibility is to show some marker on top of person who is speaking.
But I think this is OK only as last resort accessibility feature.

Link to comment
Share on other sites

The idea of using left/right/center justification comes from captioning of pre-recorded material, but as you indicate is problematic for live action where characters move around unpredictably.

A horizontal location bar could be implemented as a solid-color rect that is repositioned, I dunno, every 200 ms. The cheap and easy implementation would be just to determine the location from the origin of the source (AI, fixed-speaker, etc.) relative to the viewport. It would be more accurate but much harder to take account of sound propagation pathways.

If you wanted to be fancy, you could make the bar get less-wide but more-high (thick) as the source got nearer and visible.

I suppose you could even have the bar's vertical location relative to the upper edge of the slot field indicate something about the source's relative vertical location. Probably too fancy, that.

  • Like 1
Link to comment
Share on other sites

@stgatilov, another variation on that theme: to indicate voice location, just have a fixed-size dot that is constrained to travel around the edge of the subtitle field. When the source is behind the player, the dot would be somewhere on the lower edge of the field.

Also, thinking about color, that @datiswous brought up: another use would just be just to differentiate the 3 categories of "story", "speech", and eventually "effects". A color tint to the field for each of these makes more sense to me that trying to tie color to particular AI.

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

In the long run, figuring out a way to add font attributes (e.g., color, bold, underline, italics) would be great for subtitles and throughout TDM. Hard, tho.

I guess fontsize markup is an attribute that might be most easily conceived at the moment. But I don't know if TDM has any underlying support for interleaved mixed-size characters.

Turning to a potentially easier issue, on the "Barks" thread, @datiswous noted...

Quote

Sometimes the subs text overlaps the weapon and selected inventory item (left and right in interface), so I think the gui for in-game subtitles should be narrowed, so it doesn't overlap. For mission intro's this isn't an issue and therefore full width is fine.

I responded:

Quote

Geep: Most guidance for subtitles would restrict them to the central 2/3rds of the screen, width-wise. So maybe narrowing the fields a bit throughout would be good. Need to think about this. I guess the lowest field would also obscure the health and breathe bars a bit.

datiswous:

Quote

I think @stgatilovsaid that possibly more input was needed for the correct gui. Maybe the current gui is designed from the briefings and ai barks experience. In briefings there is no player interface with icons left and right and health bar and the short ai barks will perfectly fit in the central, so at first it seems it looks ok.

So in my opinion the in-game gui has to change so that the subtitles are not able to overlap any interface elements, instead of working around the limitation.

Geep: I agree.... Related: I'll probably do some more experiments with the current TDM font & size, to understand max char count versus field width.

  • Like 1
Link to comment
Share on other sites

I just did an experiment, with the field width margins increased from 10 to 100. That looks attractive with respect to the current weapon and inventory displays in the corners. And seems wide enough for expected content.

To confirm if they are wide enough, I took the 3 longest Thug subtitles (as representative of longest expected subtitle strings) and duplicated each text string, to guarantee that they would overflow (exceed 2 lines).  I then counted  where they overflowed: after respectively char 121, 134 [shown], and 126.

IMHO, as long as a typical phrase of 120 characters or less (and usually way less) can fit, it's all good. Where 120 characters = 6 second max subtitle duration x 20 cps max reading rate.

@datiswous, since this was your idea, perhaps you'd like to do this as a new-feature request on the bug tracker? Then I could upload the proposed revised tdm_subtitles_common.gui file.

testSubtitlesNarrowed_(2023-02-16)_1.jpg

  • Like 2
Link to comment
Share on other sites

3 minutes ago, Geep said:

just did an experiment, with the field width margins increased from 10 to 100. That looks attractive with respect to the current weapon and inventory displays in the corners. And seems wide enough for expected content.

What happens if "gui_iconSize" is set to a higher value in the settings by the player?
"Settings->Video->Advanced->HUD Size/Opacity - Adjust->Icon Size"

  • Like 2
Link to comment
Share on other sites

Yes, it is possible to make subtitle lines less wide. And it is possible to do it only for gameplay.

Not sure about avoiding overlaps in general.
That would not allow us to move subtitles lines around, although I guess the idea of markers on the border for positional hint are better than moving the lines.

As for font and size, I don't think GUI system supports anything like that.
Maybe I'm wrong, but I don't remember any possibility to mix italics and bold in same text, the same applies to several sizes. Such thing can only be done with some kind of escape sequences in the text itself, and I don't remember any in GUI engine.

 

  • Like 2
Link to comment
Share on other sites

If the Icon Size and Weapons/Item Text Sizes are adjusted to their maximum HUD settings, then it will be pretty impossible to avoid overlap [see image, still using previously-proposed 100 margins].

The 100 margins could, however, be increased. If the field width was narrowed slightly from about 60 characters to about 55, that would be about 2/3rds of the screen. That would still be fine, and provide very good flexibility to the subtitler to decide where best to break a long phrase into 2 lines. If narrowed to 42 characters (about 8 words), which other subtitling systems use, then verbatim subtitling would still be doable, but (for phrases > 70 chars) flexibility would be lost, so either poorly positioned linebreaks would occur, or more phrases would have to be subdivided, e.g., from inline into srt.

Now, other subtitling systems minimized average overlap by not using fixed-width fields. Common is to have, behind each text line, a black field that is just the width of the text. We could do that. For each text line, the engine would calculate the width needed, and pass that value to the gui. Our backing field could be black (best for quick readability) or translucent like now (best for world awareness). [This would be preclude augmenting the field edge with speaker positioning info, floated earlier.]

Also, again thinking about overlap, the subtitling field-choice system could be made smarter, so that if there's something to be seen at the bottom of the screen (e.g., enlarged light gem, breath metter, text like "Acquired 80 in Jewels"), the lowest subtitle field is the last of the 3 populated, not the first.

Slider(s) could be added to adjust subtitle font size (and other layout attributes). Could get pretty complicated... and make subtitlers life harder. 😬

As for changing individual characters/words with markup, yeah, there's only a bastardized "caret" markup at the moment to select primary colors. Having a real system would be no trivial effort.

testSubtitlesNarrowed_(2023-02-17)_1.jpg

  • Like 1
Link to comment
Share on other sites

It looks like the subtitles section will need to adapt to the other GUI element sizes.

  • When the light gem size increases, move the subtitles up.
  • When the weapon/item size increases, reduce the width of the subtitles.
    • When the reduced width reaches a threshold, pop it up above the light gem and weapon/item gui elements.

 

For colors, bold, italics, etc., I wonder if supporting a text2 field (text version 2) that supports a subset of HTML would be worthwhile.

Edited by Daft Mugi
Link to comment
Share on other sites

On 2/17/2023 at 12:37 PM, Daft Mugi said:

It looks like the subtitles section will need to adapt to the other GUI element sizes.

Could well be, but there's lots of ways that could go. You had some good suggestions for automatically adjusting vertical locations and field widths, on the assumption that the subtitle font size remains fixed. But maybe that assumption needs to be revisited. Certainly the current subtitle font size is bigger than the fonts with the weapon and inventory icons, so maybe it doesn't need to made greatly enlargable to address needs of low-vision folks. The automatic adjustment you suggested would also be different from other HUD settings, that are (mostly) manual and independent of other elements. Maybe there should be subtitle positioning sliders. I dunno. I worry that moving the subtitles still further towards the center of screen (particularly when 3 fields are active) would make the game unplayable. Making the fields narrower than at present I can certainly see. Possibly only as wide as their text (another form of automatic adjustment), as I mentioned earlier.

Quote

For colors, bold, italics, etc., I wonder if supporting a text2 field (text version 2) that supports a subset of HTML would be worthwhile.

I expect most everyone would agree with you and wish HTML-style font markup was available throughout TDM. The problem is, it would be such a major effort, since the engine doesn't support such font markup (and underlying font attributes and rendering) currently.

Still, maybe it's time to add that as an "aspirational" feature request to the bug tracker.

Edited by Geep
aspirational feature request mentioned
  • Like 1
Link to comment
Share on other sites

Copied from the AI Barks thread. In response to datiswous, I said:
 

Quote

 

Here's my understanding. There are 3 stacked non-overlapping "slots" for a subtitle field to appear in. You can see this in the testSubtitles FM if you hit any of the buttons fast enough. (Granted, this is all in the same voice, not different voices.) When a sound is about to play, the sound system looks for the lowest unused slot to put its field in. And it keeps that field until the sound clip ends (e.g., across multiple srt phrases). If a sound starts playing and there's no slot free, then no subtitle appears for it.

Because the sound system currently doesn't know who/what emits a sound, it can't "reserve" a slot for a particular AI or other entity across separate .ogg files.

Even without knowing the emitting entity, what the subtitle system could do now (but doesn't currently) is look at the standardized pathname of the sound file, and determine what vocal set it comes from. It could then "reserve" a slot for a while for future utterances from the same voice. Is that a good idea? Unclear to me. (There's a broader issue here about giving priority to "story" phrases over "speech" phrases. I'll move this to the Future thread.)

....

 

 

Link to comment
Share on other sites

So, just thinking about priority further, we have the 3 obvious levels:

1) story

2) speech

3) effect

Suppose, if all 3 slots are filled, a higher-priority subtitle could "bump" (take over the slot) of a lower-priority one. I'm thinking this would not be too difficult technically. From a reader's viewpoint, the main difficulty is that the bumped subtitle would appear on-screen for a shortened amount of time. How do you minimize the grief?

You have to choose which subtitle to bump. If they're different priorities (1 is speech, 1 is effect), this is easy. Otherwise, you might choose the subtitle of the sound clip that is closest to completion anyway. This implies knowing the total duration and the starttime of playing clips.

Maybe you need to put in a 1/10 second no-subtitle moment when the bump occurs, a visual cue.

  • Like 1
Link to comment
Share on other sites

This ties into another thing I would very much like, namely, the ability to extend the duration of a subtitle beyond the end of the clip. This is not possible now, so would require some build-out of threading capabilities for subtitles.

There are a lot of barks that are too short, below the minimum recommended 1 second for a subtitle. In theory, one could add dead-air to the .ogg files, but really, way too much tedious work.

Assume that the underlying tech issues to allow duration extension could be addressed. Then extension could be deployed through 2 complementary features -

- Have the subtitle system automatically enforce the 1 second requirement for inline only.

- Have the "inline" and "srt" commands take an optional additional parameter, for overall duration (i.e., greater than the clip duration). This would allow the subtitler to ask for a small time extension to accommodate CPS/WPM concerns. When specifying the final "srt" phrase, it could use that extended timeline. Typically, for longer clips, a small extension like 1/4 second more would suffice. A long extension is undesirable for voice-subtitle sync reasons.

Finally, going back to the previous post, if the subtitles were threaded and more independent of the sound playing, then another capability would be, if a high-priority clip wants to play but there are no subtitle slots, see if any of the lower-priority clips will conclude shortly (ideally, within 1/4 second), and queue the subtitle to appear then (importantly, without shortening). Instead of immediately bumping the lower-priority clip.

 

Edited by Geep
  • Like 1
Link to comment
Share on other sites

On 2/21/2023 at 12:16 PM, Geep said:

[I want] the ability to extend the duration of a subtitle beyond the end of the clip. This is not possible now, so would require some build-out of threading capabilities for subtitles.

There are a lot of barks that are too short, below the minimum recommended 1 second for a subtitle. In theory, one could add dead-air to the .ogg files, but really, way too much tedious work.

Assume that the underlying tech issues to allow duration extension could be addressed. Then extension could be deployed through 2 complementary features -

- Have the subtitle system automatically enforce the 1 second requirement for inline only.

- Have the "inline" and "srt" commands take an optional additional parameter, for overall duration (i.e., greater than the clip duration). This would allow the subtitler to ask for a small time extension to accommodate CPS/WPM concerns. When specifying the final "srt" phrase, it could use that extended timeline. Typically, for longer clips, a small extension like 1/4 second more would suffice. A long extension is undesirable for voice-subtitle sync reasons.

For inline, if there is the additional parameter, that specified duration (whether longer or shorter than 1 second) over-rides the 1-second rule. (Thus, if for a future "effect" sound of footstep, you wanted to provide a "(clop)" subtitle every 1/4 second, say, you still could; you'd just have to be explicit.)

@stgatilov, @nbohr1more, @Dragofer: The potential to get the foregoing capability - or not - will affect how I strike a balance between authoring verbatim and shortened subtitles. So, I need to get feedback ASAP as to whether or not this is reasonably possible for TDM 2.12. If so, I'll add it to the bugtracker as a New Feature request, and leave more subtitles as verbatim. If not, I'll continue shortening affected subtitles.

  • Like 1
Link to comment
Share on other sites

On 2/21/2023 at 6:16 PM, Geep said:

- Have the "inline" and "srt" commands take an optional additional parameter, for overall duration (i.e., greater than the clip duration).

It seems to me this could also be interesting if you quickly want to create a conversation based on no sound, but with added subtitles, then if it looks good, you can add the audio later. So I can just use one 1 second silent ogg file for multiple voices and for every voice sentence, specify the length and put the subtitle in that length.

  • Like 2
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • Petike the Taffer  »  DeTeEff

      I've updated the articles for your FMs and your author category at the wiki. Your newer nickname (DeTeEff) now comes first, and the one in parentheses is your older nickname (Fieldmedic). Just to avoid confusing people who played your FMs years ago and remember your older nickname. I've added a wiki article for your latest FM, Who Watches the Watcher?, as part of my current updating efforts. Unless I overlooked something, you have five different FMs so far.
      · 0 replies
    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
×
×
  • Create New...