Jump to content
Forum Login Changes ×
The Dark Mod Forums

English Subtitles for AI Barks


Geep

Recommended Posts

A project is underway, led by Geep, to eventually provide English language subtitles for all the non-story phrases (e.g., barks) of the stock AI characters. The non-story phrases are identified by the "verbosity speech" subtitle tag. The hope is that these subtitles would be distributed as part of the future TDM 2.12 core and so available to all FMs, both old and new. Since there are 5-10K such phrases in total, this project will take a while. Let me know if you'd like to help.

This forum thread can be used for project coordination, and to provide info about -

  • What subtitle sets (e.g., for particular AI) are underway and completed
  • Emerging tools or methods for bulk testing
  • Style guidance for TDM subtitle authors. To support this project, a style guide will be developed as experience accumulates. While motivated by providing some consistency to English speech-tagged subtitles, it may be of some interest to FMers creating story-tagged subtitles, and those subtitling in other TDM-supported languages.

Potential improvements to TDM's subtitling system - particularly those that would involve engine/GUI changes - can be discussed in this companion thread: Subtitles - Possibilities Beyond 2.11

Current Status - Subtitles for Barks

The Thug - NEW UPDATE May 6, 2023 released as testSubtitlesThug2 FM. This takes advantage of new 2.12dev features and changes to style guidelines.

The Lord - NEW UPDATE May 4, 2023 released as testSubtitlesLord2 FM

The Wench - Soon! To be released as testSubtitlesWench FM; pending resolution of rename of a few sound files (Bugtracker 6284).

The Young Builder (Builder 4) - NEW UPDATE! May 30, 2023 released (again) as testSubtitlesYoungBuilder FM. (Corrects error in briefing syntax of original May 17 release.)

Average Jack - coming soon

TDM vocal sets are listed here.

Current Status - Utilities

buildSubtitleShader.exe - Latest release of April 10, 2023. This program fabricates a TDM sound shader file specifically for the testSubtitles... series of FMs used here. It does so by wrapping each sound file name in a directory into a sound shader with uniform incremental naming. While limited to a single directory, the latest release simplifies merging runs from multiple directories. @datiswous reports that this Windows console program also works under Linux/wine, where it is used with similar "testSubtitle..."-derived FMs for verbosity "story" subtitling.

checkDurationsInSRT - NEWLY UPDATED (BUG FIX)! May 6, 2023 Release
This Win/console program scans a directory for .srt files, examines the subtitles, and warns about those phrases/messages that are potentially too short or too long in time, or that seem to require too high a reading rate, expressed in characters per second. It also looks for within-file subtitle messages that overlap in time.

soundDurationsCSV.exe - March 7, 2023 Release
This Windows console program scans a directory of sound files, reporting their names and (using pre-installed ffprobe) durations with millisecond resolution. The resulting .csv file can then be imported into a custom Excel spreadsheet for subtitle editing. For more, see also Feb 20th comments .

findTooLongSubtitles.exe - March 21, 2023 Release
This Win/console program scans a directory for .subs and .srt files, checks the length in characters of each subtitle line, and reports those that exceed a specified maximum.

analyzeFieldWidth of April 27 (release not scheduled)

Used to determine, for 12pt Carleton font, how many characters might reasonably fit within a given gui field width.

Current Status - Spreadsheets

Work by Geep on each vocal set starts with instantiation of a pre-configured Excel spreadsheet. This is used for subtitle editing and to do automatic calculations to help manage constraints and decision making. The columns and calculations are still evolving, as are related templates, how-to documents, and workflow documents. It is hoped, when evolution solidifies, to release a stable version of this material. Interim spreadsheets are being released as snapshots with corresponding subtitle releases.

Current Status - Style Guide for Barks

Work in progress, as major Word doc. Expect summer release.

Current Status - Third-Party Tools

For editing of "story" SRT files, datsiwous found "kdenlive" particularly helpful [to do: link to how-to below]. Geep has been working with the simpler, audio-only "Cadet" for barks.

Edited by Geep
Update of YoungBuilder FM
  • Like 3
Link to comment
Share on other sites

I'm starting with The Thug's barks (appx 400), for which maybe half of the needed subtitles were available on the Thug Vocals wiki page. (These pages are no longer maintained and so mostly obsolete.) I've got a test map suitable for testing a single subtitle, but it doesn't scale for bulk testing. Also looking at existing caption/subtitle style guides.

  • Like 1
Link to comment
Share on other sites

Is there an easy way to handle situations where multiple guards are barking at once?

  • Like 2

{ 0 | 🞵 } = funk_tastic

My missions:           Stand-alone                                                      Duncan Lynch series                              

                                      Down and Out on Newford Road              the Factory Heist

                                                                                                  A House Call

                              

Link to comment
Share on other sites

The subtitle system afaik has one graphical look for all subs, so it's hard to see the difference except if you specify it.

Quote

Guard1: If there's trouble, I'm ready

Guard2: Me too!

It would be nice if you could set colors for different persons.

Edited by datiswous
  • Like 2
Link to comment
Share on other sites

Multiple guards (up to 3) should have their speech appear in separate fields.

Within the limitations of 2.10/2.11, my draft guidance for generic barks ("verbosity speech") is never to include AI name info. However, for an FM-specific phrase ("verbosity story") just said by one character, you could include the name or other ID. (I'm still researching if there is a consensus on best-practice format to indicate the name).

Beyond 2.11, color is one of many way to go. Reminds me, I was going to set up another thread for that... done. See link to Subtitles - Beyond 2.11 in OP above.

Edited by Geep
Link to Beyond 2.11
  • Like 1
Link to comment
Share on other sites

I thought about naming sounds by who said them but did not got any particular scheme.
One problem here is that subtitles are supported inside the sound engine, where sounds no longer have any source: you just have a sound and probably location, but not something like who said it.

I made sure that a subtitle for a sound does not move between slots.
Other than that, there are 3 slots for simultaneous subtitles, and I guess some prioritization. But if there are many guards talking at once, it would be chaos.

As for colors: it should be possible to generate color automatically but it won't help anything.
It should be possible to specify colors inside subtitle files, similar to the way verbosity is now specified.

To be honest, I'm not sure it's a good idea to give names, since player usually does not know who speaks.
But that's of course mapper's preference: sometimes player surely knows who is talking in a cutscene, so why not.

  • Like 1
Link to comment
Share on other sites

@datiswous, I imagine the TDM fonts don't support bold, italics, underline, etc. Maybe just different sizes.

I don't think color by itself would be helpful for speaker identification, unless there was a color halo or name tag above each vocalizing AI. Might be useful for word emphasis, though.

@stgatilov, if there is location data available to the subtitle code, that could be used in various ways, using a different GUI:

1) Let the 3 slots be either left justified, centered, or right justified, depending on relative speaker location (including off screen). This could be implemented by 9 actual windowDefs (all of same size; 3 each for each current slot, overlaid, with each of the 3 justifications.) Alternatively, with a 3x3 grid, all windowDefs the same size, but 1/3 the current width.)

2) Or instead, at the top edge of each slot, show a short horizontal bar, whose left/right position is moved to be under the relevant AI. If off screen, add an arrow head (<-- or -->).

Naming is harder, if the subtitle code can't get at that info. Though at the point the sound engine is passed the sound to render, presumably it knows the speaker, and could independently and in parallel visualize the name information (particularly if the sound engine passed back which GUI slot it was going to use for subtitle.)

It is true that associating a name (say, using a small tab-like field) with a slot is less useful when the player doesn't know the AI names (i.e., with no floating names above the characters).

If the technical issues of naming could be overcome, then there's still the question of when you'd want to give a name (Rupert) versus type (thug #3) versus generic (speaker #2). Also have to handle special cases of (narrator) and (player).

Dreaming... Instead of text names, more fun would be to show a thumbnail face of each AI next to their slot. With a question-mark face when they are off-screen?

Edited by Geep
  • Like 2
Link to comment
Share on other sites

The wiki has AI vocal scripts for most but not all characters. I'm guessing the wiki info is derived from early-draft vocal scripts supplied to the voice artists. Looking at the Thug's script, I see "I'd drink horse piss [etc]" 🐎🍺

There are occasional differences between an .ogg file and it's draft text phrase. Some of that is just the voice artist riffing. Other cases seem to be a more intentional change of plan. For instance, with respect to swearing, "shit" is generally suppressed... "horse shit" became "horse dirt" or "horse filth". "Bloody" became more widely used, probably to make the English more British, less American.

I'm about 2/3rds through the Thug translation. Going slow because I'm also working up related tools and style guides. And I don't plan to upload subtitle sets for any AI until after 2.11 is out, presumably Real Soon Now.

  • Like 1
Link to comment
Share on other sites

Custom-FM-to-test-subtitles.thumb.jpg.99d1cc49b880789fb26ff5e9bf71bca1.jpg Here's a screen shot of an FM I've been building to help test/review subtitles for an AI, in reproducible order with 100% coverage. It uses the TDM-distributed .ogg files, but has custom soundshaders. Each such soundshader wraps exactly one .ogg file, and has a uniform naming that includes an index number. The collection of soundshaders is housed in a single file. The hope is that that (prior to embedment in the FM) this file can be easily generated from a directory listing of the .ogg files for a particular AI, and subsequent manipulation.

There are 3 buttons to step through the file. After each button press, you hear the vocal and see the subtitle. Also, briefly the index number within the list appears (a floating "7" near the statue's shoulder in this screenshot.) You can also - using a custom CVar in the console - jump to a particular index number.

Unsurprisingly, you can't edit the subtitle within the FM. Just note what needs changing and do it with a text editor. Also, this FM is not intended to evaluate TDM's stock soundshaders, nor AI lip syncing (so just a statue speaking here.)

If you hit the buttons fast enough, multiple sound files play at once, and multiple subtitle fields appear (up to the max of 3 the .gui offers)

  • Like 3
Link to comment
Share on other sites

1 hour ago, Geep said:

Unsurprisingly, you can't edit the subtitle within the FM. Just note what needs changing and do it with a text editor.

Ehhh, cool idea, but why not do everything with a video editor and then you can immediately edit the subtitle in the editor?

Link to comment
Share on other sites

2 hours ago, datiswous said:

Ehhh, cool idea, but why not do everything with a video editor and then you can immediately edit the subtitle in the editor?

For the Thug, all the subtitles I've done so far have been rather short and so have lent themselves to the simple "inline" approach. I agree that a different character that has more long monologues, and so needs "srt", would benefit from a video editor (even if there's no real video, just audio) as you described earlier. I may need to call on your srt expertise later.

I've been creating the subtitles with 3 windows open -

1- to select and play each .ogg with a minimal Windows player.

2- A view of the AI vocal script (e.g., source view of the Thug wiki entry), to use as a copy/paste starting point.

3 - The .subs file being built in a text editor.

So, the FM I described is not used in the first place to create subtitles, but rather to do a second-pass review (e.g. by me) or third-pass review (as part of quality control during incorporation into TDM).

 

  • Like 1
Link to comment
Share on other sites

I have a question about the current TDM implementation of "srt"...

For an audio clip using srt with multiple subtitle phrases, the start and end timestamps for each phrases are relative to the clip start. For implementation reasons, it would not be reasonable to have negative time. But it could be both reasonable and useful to have a subtitle that continues on-screen a bit longer than the audio clip. Is that currently possible?

  • Like 1
Link to comment
Share on other sites

You mean only the last phrase? So for example you have an srt subtitle file with 20 phrases and the last phrase will be shown a couple of seconds longer than the audio file plays?

Or do you mean for every audio phrase in an audio file? You can do whatever you want. You can set the time for every subtitle phrase how long it will display during playback.

Edited by datiswous
  • Like 1
Link to comment
Share on other sites

2 hours ago, Geep said:

For an audio clip using srt with multiple subtitle phrases, the start and end timestamps for each phrases are relative to the clip start. For implementation reasons, it would not be reasonable to have negative time. But it could be both reasonable and useful to have a subtitle that continues on-screen a bit longer than the audio clip. Is that currently possible?

As far as I remember, it is not possible.
If sound ends, it is no longer in the sound system, so it does not generate subtitle anymore.

On 2/3/2023 at 10:45 PM, Geep said:

Unsurprisingly, you can't edit the subtitle within the FM. Just note what needs changing and do it with a text editor.

I think some command can pull the subtitle changes from disk.
It's either reloadDecls or reloadSounds.

  • Like 1
Link to comment
Share on other sites

The first-draft of the thug subtitles is done, and also successfully loaded into my test jig with custom soundshaders for the second-draft review/revise. That review will be paused while i work through some issues about best subtitling practices to target. For example, word per minute reading constraints and their impacts.

  • Like 1
Link to comment
Share on other sites

@stgatilov, in the Subtitles_decls section of the Subtitles wiki page, you show example code, that includes an srt reference.

Wouldn't it be better to guide people to place the corresponding .srt files within the "subtitles" tree (where their .subs live), rather than the "sound" tree?

Also, should FM authors be encouraged (if not required) to prefix .srt files with "fm_" ?  Like "fm_sound8_long.srt" ?

Finally, in the "displayed text" section, under the srt command, there's a desperate need to see example .srt content, in this case, something made up for sound8_long.srt (or fm_sound8_long.srt) like:

1
00:00:06,612 --> 00:00:10,376
Something's wrong with this crystal ball.

2
00:00:15,482 --> 00:00:20,609
Bugger me!
It's not showing the right dream.

3
00:00:25,336 --> 00:00:28,167
Ah! Here we go.

--end

Link to comment
Share on other sites

1 hour ago, Geep said:

Wouldn't it be better to guide people to place the corresponding .srt files within the "subtitles" tree (where their .subs live), rather than the "sound" tree?

Yeah, I thought the same and this is what I effectively do (putting srt files in folder subtitles).

  • Like 1
Link to comment
Share on other sites

22 hours ago, Geep said:

Wouldn't it be better to guide people to place the corresponding .srt files within the "subtitles" tree (where their .subs live), rather than the "sound" tree?

Technically, it does not matter.

It might be convenient to have .srt files at same location and with same name as the sound files that they correspond to.
I think that was the reason to suggest placing .srt into sounds.

Or maybe I did not think to much about it 🤔

Quote

Also, should FM authors be encouraged (if not required) to prefix .srt files with "fm_" ?  Like "fm_sound8_long.srt" ?

Are you encouraged or required to prefix your sound samples with "fm_"?

I know many mappers do use some kind of prefix, and that's good.
Others don't and that can cause accidental name collisions.
Nothing new here.

Quote

Finally, in the "displayed text" section, under the srt command, there's a desperate need to see example .srt content

There is a link to .srt description. Also, .srt is kinda standard format.

  • Like 1
Link to comment
Share on other sites

Btw. there's some confusing text in that article:

Quote

FM-specific subtitles are always included from the decl named fm_root, which must be in the fm_root.subs file. For instance, the contents of the fm_test.subs file from the example above will have no effect until we add the file fm_root.subs with contents:

https://wiki.thedarkmod.com/index.php?title=Subtitles#include

There's no fm_test.subs in an example. I think it should be fm_conversations.subs . Not sure, but maybe you could check?

Edited by datiswous
  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...