Jump to content
The Dark Mod Forums

English Subtitles for AI Barks


Geep

Recommended Posts

Introduction

A project is underway, led by Geep, to eventually provide English language subtitles (or if you prefer, "closed captions") for all the non-story phrases (e.g., barks) of the stock AI characters. The non-story phrases are identified by the "verbosity speech" subtitle tag. The hope is that these subtitles would be distributed as part of the future TDM 2.12 core and so available to all FMs, both old and new. Since there are 5-10K such phrases in total, this project will take a while. Let me know if you'd like to help.

This forum thread can be used for project coordination, and to provide info about -

  • What subtitle sets (e.g., for particular AI) are underway and completed
  • Emerging tools or methods for bulk testing
  • Style guidance for TDM subtitle authors. To support this project, a style guide will be developed as experience accumulates. While motivated by providing some consistency to English speech-tagged subtitles, it may be of some interest to FMers creating story-tagged subtitles, and those subtitling in other TDM-supported languages.

Potential improvements to TDM's subtitling system - particularly those that would involve engine/GUI changes - can be discussed in this companion thread: Subtitles - Possibilities Beyond 2.11

Current Status - Subtitles for Barks

All TDM vocal sets planned for TDM 2.12 have been delivered.

New: Fixup Needed to Run These Under 2.12 Beta 3

The Thug - Update May 6, 2023 released as testSubtitlesThug2 FM. This takes advantage of new 2.12dev features and changes to style guidelines.

The Lord - Update May 4, 2023 released as testSubtitlesLord2 FM

The Wench - Two postponed April versions - Nov 25, 2023 released as testSubtitlesWenchOriginal and testSubtitlesWenchPatched FMs. Two versions because of Bugtracker 6284. IMPORTANT: See final version at end of this list, released Jan. 14, 2024.

The Young Builder (Builder 4) - Update May 30, 2023 released (again) as testSubtitlesYoungBuilder FM. (Corrects error in briefing syntax of original May 17 release.)

Average Jack - June 8, 2023 released as testSubtitlesJack FM.

The Pro - June 23, 2023 released as testSubtitlesPro FM.

The Maiden - July 8, 2023 released as testSubtitlesMaiden FM

The Grumbler - July 24, 2023 released as testSubtitlesGrumbler FM

The Mature Builder (Builder 3) - August 15, 2023, released as testSubtitlesMatureBuilder FM

The Lady (aka Noblewoman) - Sept 9, 2023, released as testSubtitlesLady FM

The Moor - Sept 14, 2023, released as testSubtitlesMoor FM

The Commander - Sept 30, 2023, released as testSubtitlesCommander FM

The Simpleton - Oct. 21, 2023, released as testSubtitlesSimpleton FM

Builder 1 & 2 (shared vocal set) - Nov. 8, 2023, released as testSubtitlesBuilder1and2 FM

The Critic - Nov. 25, 2023, released as testSubtitlesCritic_FM

Manbeast - Dec. 10, 2023, released as testSubtitlesManbeast FM

NEW! Lady02 - Dec 21, 2023, released as testSubtitlesLady02 FM. Assisted by datiswous.

NEW! The Drunk - Jan 6, 2024, released as testSubtitlesDrunk FM. Assisted by MirceaKitsune.

NEW! The Cynic - Jan. 13, 2024, released as testSubtitlesCynic FM. Incorporates early demo subtitles by Dragofer.

NEW! The Wench, Final Revision - Jan. 14, 2024, released as testSubtitlesWenchFinal FM . Assisted by Dragofer. See also original May version (released in November) in the list above.

All TDM vocal sets planned for TDM 2.12 have been delivered. Thanks, @nbohr1more, for integrating these into the release.

All AI vocal sets, including additional ones for future consideration - perhaps as "verbosity effects" - are listed here.

Current Status - Utilities

buildSubtitleShader.exe - Latest release of April 10, 2023. This program fabricates a TDM sound shader file specifically for the testSubtitles... series of FMs used here. It does so by wrapping each sound file name in a directory into a sound shader with uniform incremental naming. While limited to a single directory, the latest release simplifies merging runs from multiple directories. @datiswous reports that this Windows console program also works under Linux/wine, where it is used with similar "testSubtitle..."-derived FMs for verbosity "story" subtitling.

checkDurationsInSRT - May 6, 2023 Update (bug fix)
This Win/console program scans a directory for .srt files, examines the subtitles, and warns about those phrases/messages that are potentially too short or too long in time, or that seem to require too high a reading rate, expressed in characters per second. It also looks for within-file subtitle messages that overlap in time.

soundDurationsCSV.exe - March 7, 2023 Release
This Windows console program scans a directory of sound files, reporting their names and (using pre-installed ffprobe) durations with millisecond resolution. The resulting .csv file can then be imported into a custom Excel spreadsheet for subtitle editing. For more, see also Feb 20th comments .

findTooLongSubtitles.exe - Minor update, June 23, 2023 Release
This Win/console program scans a directory for .subs and .srt files, checks the length in characters of each subtitle line, and reports those that exceed a specified maximum.

analyzeFieldWidth of April 27, 2023 (release not scheduled; too idiosyncratic). Used to determine, for 12pt Carleton font, and a representative sampling of bark subtitles, how many characters might reasonably fit within a given gui field width. Later, parts of this code were used to build calcStringWidth.exe discussed next.

calcStringWidth.exe - Oct. 11, 2023 Release. Calculates the display width of an input string, for 4 candidate subtitle fonts Carleton and Stone, uncompressed or 16:9 compressed.

statsForSubtitles - June 25, 2023 Release.  Starting with The Pro vocal set, this Windows console program automates the gathering and reporting of the basic statistics included with the announcement of every vocal set release in this forum thread.

Current Status - Spreadsheets and Related Explanatory Documents, including Workflow and Templates

Work by Geep on each vocal set starts with instantiation of a pre-configured Excel spreadsheet. This is used for subtitle editing and to do automatic calculations to help manage constraints and decision making.

The announcement of each AI subtitle release here includes a link to the corresponding spreadsheet. At first, the spreadsheet's columns, formats, calculations, and highlighting (and its distillation into template form) would change quite a bit between each AI. As of June, 2023 and AverageJack, it has largely stabilized, and is now documented:

June 10 Documents Explaining Workflow and Excel Spreadsheet/Template (for AverageJack)

The v5 template was used for vocal sets from June's Average Jack to November's The Critic. Further clarifications are now available:

Dec 7 Update of the "Explained" Doc for Spreadsheet Template v5

Subsequent vocal sets, through to 2024's The Cynic, use template v6:

NEW! Jan. 10, 2024 "Explained" Doc for Spreadsheet Template v6

Current Status - Style Guide for Barks

As of Sept 2, Subtitle Style Guide - Part 1 is now available.

NEW! As of Jan 16, 2024: Subtitle Style Guide - Part 2 with more discussion and references.

Current Status - Third-Party Tools

For editing of "story" SRT files, datsiwous found "kdenlive" particularly helpful [to do: link to how-to below]. Geep has been working with the simpler, audio-only "Cadet" for barks.

Edited by Geep
Release of Subtitle Style Guide - Part 2
  • Like 4
Link to comment
Share on other sites

I'm starting with The Thug's barks (appx 400), for which maybe half of the needed subtitles were available on the Thug Vocals wiki page. (These pages are no longer maintained and so mostly obsolete.) I've got a test map suitable for testing a single subtitle, but it doesn't scale for bulk testing. Also looking at existing caption/subtitle style guides.

  • Like 1
Link to comment
Share on other sites

Is there an easy way to handle situations where multiple guards are barking at once?

  • Like 2

My missions:           Stand-alone                                                      Duncan Lynch series                              

                                      Down and Out on Newford Road              the Factory Heist

                                                                                                  A House Call

                                                                                                  The House of deLisle                                                                                                  

                              

Link to comment
Share on other sites

Multiple guards (up to 3) should have their speech appear in separate fields.

Within the limitations of 2.10/2.11, my draft guidance for generic barks ("verbosity speech") is never to include AI name info. However, for an FM-specific phrase ("verbosity story") just said by one character, you could include the name or other ID. (I'm still researching if there is a consensus on best-practice format to indicate the name).

Beyond 2.11, color is one of many way to go. Reminds me, I was going to set up another thread for that... done. See link to Subtitles - Beyond 2.11 in OP above.

Edited by Geep
Link to Beyond 2.11
  • Like 1
Link to comment
Share on other sites

I thought about naming sounds by who said them but did not got any particular scheme.
One problem here is that subtitles are supported inside the sound engine, where sounds no longer have any source: you just have a sound and probably location, but not something like who said it.

I made sure that a subtitle for a sound does not move between slots.
Other than that, there are 3 slots for simultaneous subtitles, and I guess some prioritization. But if there are many guards talking at once, it would be chaos.

As for colors: it should be possible to generate color automatically but it won't help anything.
It should be possible to specify colors inside subtitle files, similar to the way verbosity is now specified.

To be honest, I'm not sure it's a good idea to give names, since player usually does not know who speaks.
But that's of course mapper's preference: sometimes player surely knows who is talking in a cutscene, so why not.

  • Like 1
Link to comment
Share on other sites

@datiswous, I imagine the TDM fonts don't support bold, italics, underline, etc. Maybe just different sizes.

I don't think color by itself would be helpful for speaker identification, unless there was a color halo or name tag above each vocalizing AI. Might be useful for word emphasis, though.

@stgatilov, if there is location data available to the subtitle code, that could be used in various ways, using a different GUI:

1) Let the 3 slots be either left justified, centered, or right justified, depending on relative speaker location (including off screen). This could be implemented by 9 actual windowDefs (all of same size; 3 each for each current slot, overlaid, with each of the 3 justifications.) Alternatively, with a 3x3 grid, all windowDefs the same size, but 1/3 the current width.)

2) Or instead, at the top edge of each slot, show a short horizontal bar, whose left/right position is moved to be under the relevant AI. If off screen, add an arrow head (<-- or -->).

Naming is harder, if the subtitle code can't get at that info. Though at the point the sound engine is passed the sound to render, presumably it knows the speaker, and could independently and in parallel visualize the name information (particularly if the sound engine passed back which GUI slot it was going to use for subtitle.)

It is true that associating a name (say, using a small tab-like field) with a slot is less useful when the player doesn't know the AI names (i.e., with no floating names above the characters).

If the technical issues of naming could be overcome, then there's still the question of when you'd want to give a name (Rupert) versus type (thug #3) versus generic (speaker #2). Also have to handle special cases of (narrator) and (player).

Dreaming... Instead of text names, more fun would be to show a thumbnail face of each AI next to their slot. With a question-mark face when they are off-screen?

Edited by Geep
  • Like 2
Link to comment
Share on other sites

The wiki has AI vocal scripts for most but not all characters. I'm guessing the wiki info is derived from early-draft vocal scripts supplied to the voice artists. Looking at the Thug's script, I see "I'd drink horse piss [etc]" 🐎🍺

There are occasional differences between an .ogg file and it's draft text phrase. Some of that is just the voice artist riffing. Other cases seem to be a more intentional change of plan. For instance, with respect to swearing, "shit" is generally suppressed... "horse shit" became "horse dirt" or "horse filth". "Bloody" became more widely used, probably to make the English more British, less American.

I'm about 2/3rds through the Thug translation. Going slow because I'm also working up related tools and style guides. And I don't plan to upload subtitle sets for any AI until after 2.11 is out, presumably Real Soon Now.

  • Like 1
Link to comment
Share on other sites

Custom-FM-to-test-subtitles.thumb.jpg.99d1cc49b880789fb26ff5e9bf71bca1.jpg Here's a screen shot of an FM I've been building to help test/review subtitles for an AI, in reproducible order with 100% coverage. It uses the TDM-distributed .ogg files, but has custom soundshaders. Each such soundshader wraps exactly one .ogg file, and has a uniform naming that includes an index number. The collection of soundshaders is housed in a single file. The hope is that that (prior to embedment in the FM) this file can be easily generated from a directory listing of the .ogg files for a particular AI, and subsequent manipulation.

There are 3 buttons to step through the file. After each button press, you hear the vocal and see the subtitle. Also, briefly the index number within the list appears (a floating "7" near the statue's shoulder in this screenshot.) You can also - using a custom CVar in the console - jump to a particular index number.

Unsurprisingly, you can't edit the subtitle within the FM. Just note what needs changing and do it with a text editor. Also, this FM is not intended to evaluate TDM's stock soundshaders, nor AI lip syncing (so just a statue speaking here.)

If you hit the buttons fast enough, multiple sound files play at once, and multiple subtitle fields appear (up to the max of 3 the .gui offers)

  • Like 3
Link to comment
Share on other sites

2 hours ago, datiswous said:

Ehhh, cool idea, but why not do everything with a video editor and then you can immediately edit the subtitle in the editor?

For the Thug, all the subtitles I've done so far have been rather short and so have lent themselves to the simple "inline" approach. I agree that a different character that has more long monologues, and so needs "srt", would benefit from a video editor (even if there's no real video, just audio) as you described earlier. I may need to call on your srt expertise later.

I've been creating the subtitles with 3 windows open -

1- to select and play each .ogg with a minimal Windows player.

2- A view of the AI vocal script (e.g., source view of the Thug wiki entry), to use as a copy/paste starting point.

3 - The .subs file being built in a text editor.

So, the FM I described is not used in the first place to create subtitles, but rather to do a second-pass review (e.g. by me) or third-pass review (as part of quality control during incorporation into TDM).

 

  • Like 1
Link to comment
Share on other sites

I have a question about the current TDM implementation of "srt"...

For an audio clip using srt with multiple subtitle phrases, the start and end timestamps for each phrases are relative to the clip start. For implementation reasons, it would not be reasonable to have negative time. But it could be both reasonable and useful to have a subtitle that continues on-screen a bit longer than the audio clip. Is that currently possible?

  • Like 1
Link to comment
Share on other sites

You mean only the last phrase? So for example you have an srt subtitle file with 20 phrases and the last phrase will be shown a couple of seconds longer than the audio file plays?

Or do you mean for every audio phrase in an audio file? You can do whatever you want. You can set the time for every subtitle phrase how long it will display during playback.

Edited by datiswous
  • Like 1
Link to comment
Share on other sites

2 hours ago, Geep said:

For an audio clip using srt with multiple subtitle phrases, the start and end timestamps for each phrases are relative to the clip start. For implementation reasons, it would not be reasonable to have negative time. But it could be both reasonable and useful to have a subtitle that continues on-screen a bit longer than the audio clip. Is that currently possible?

As far as I remember, it is not possible.
If sound ends, it is no longer in the sound system, so it does not generate subtitle anymore.

On 2/3/2023 at 10:45 PM, Geep said:

Unsurprisingly, you can't edit the subtitle within the FM. Just note what needs changing and do it with a text editor.

I think some command can pull the subtitle changes from disk.
It's either reloadDecls or reloadSounds.

  • Like 1
Link to comment
Share on other sites

The first-draft of the thug subtitles is done, and also successfully loaded into my test jig with custom soundshaders for the second-draft review/revise. That review will be paused while i work through some issues about best subtitling practices to target. For example, word per minute reading constraints and their impacts.

  • Like 1
Link to comment
Share on other sites

@stgatilov, in the Subtitles_decls section of the Subtitles wiki page, you show example code, that includes an srt reference.

Wouldn't it be better to guide people to place the corresponding .srt files within the "subtitles" tree (where their .subs live), rather than the "sound" tree?

Also, should FM authors be encouraged (if not required) to prefix .srt files with "fm_" ?  Like "fm_sound8_long.srt" ?

Finally, in the "displayed text" section, under the srt command, there's a desperate need to see example .srt content, in this case, something made up for sound8_long.srt (or fm_sound8_long.srt) like:

1
00:00:06,612 --> 00:00:10,376
Something's wrong with this crystal ball.

2
00:00:15,482 --> 00:00:20,609
Bugger me!
It's not showing the right dream.

3
00:00:25,336 --> 00:00:28,167
Ah! Here we go.

--end

Link to comment
Share on other sites

22 hours ago, Geep said:

Wouldn't it be better to guide people to place the corresponding .srt files within the "subtitles" tree (where their .subs live), rather than the "sound" tree?

Technically, it does not matter.

It might be convenient to have .srt files at same location and with same name as the sound files that they correspond to.
I think that was the reason to suggest placing .srt into sounds.

Or maybe I did not think to much about it 🤔

Quote

Also, should FM authors be encouraged (if not required) to prefix .srt files with "fm_" ?  Like "fm_sound8_long.srt" ?

Are you encouraged or required to prefix your sound samples with "fm_"?

I know many mappers do use some kind of prefix, and that's good.
Others don't and that can cause accidental name collisions.
Nothing new here.

Quote

Finally, in the "displayed text" section, under the srt command, there's a desperate need to see example .srt content

There is a link to .srt description. Also, .srt is kinda standard format.

  • Like 1
Link to comment
Share on other sites

Btw. there's some confusing text in that article:

Quote

FM-specific subtitles are always included from the decl named fm_root, which must be in the fm_root.subs file. For instance, the contents of the fm_test.subs file from the example above will have no effect until we add the file fm_root.subs with contents:

https://wiki.thedarkmod.com/index.php?title=Subtitles#include

There's no fm_test.subs in an example. I think it should be fm_conversations.subs . Not sure, but maybe you could check?

Edited by datiswous
  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recent Status Updates

    • nbohr1more

      Congrats to all the busy Dark Mod developers! TDM 2.12 is here!
      · 0 replies
    • JackFarmer

      @joebarnin
      Jeff, I sent you PM a few weeks back but you did not respond. Maybe your inbox is full?
      · 5 replies
    • The Black Arrow

      I was playing Skyrim, got bored fast. Got back to The Dark Mod just for one mission, played "Somewhere above the City" which is not really a great mission but still good, as in above-average.
      My biggest regret is living in this country, where there's a heat wave, I am sweating at 20c and I really wish for temperatures below 5c.
      · 12 replies
    • Skaruts

      Is there something wrong with the forums lately, or is it my browser? I've been having trouble formatting posts, and just now I couldn't format anything at all.
      I'm using Vivaldi.
      Usually I have to: select text, click bold, nothing happens, select again, click bold, then it works. 
      Same for other stuff, like creating spoilers, bullet points, links. Nothing works the first time. 
      · 1 reply
    • STiFU

      Back from a spontaneous 1-week trip to Lanzarote with wife and son. I hope beta testing has been going well...
      · 1 reply
×
×
  • Create New...