English Subtitles for AI Barks

[datiswous]

On 4/10/2023 at 8:27 PM, Geep said:

An update of command-line tool "buildSubtitleShaders" is now available:

Win executable

C++ source code file

With this Feb. 10 version, if you ask for help (buildSubtitleShaders -h or ?), there's an additional option shown:

-c counter start value     (default: 1). Eases merging results from different sound file folders.

As before, comments in the source code file provide full documentation.

 

Thanks, this will help me a lot.

I'm currently creating the subtitles for The Black Mage, which has sound files organized in different folders.

Edited April 11 by datiswous

[datiswous]

On 3/11/2023 at 6:09 PM, Geep said:

For that, I've downloaded and installed "Cadet", free captioning software produced by public TV station WGBH in Boston. Working with one clip, I was able to produce a srt file. Painful process, though. I clearly need to look at more Cadet tutorials....

I just tested Cadet because I wanted to know if it's good at something and I have to say, it seems very limited. I don't really understand why you need an editor for inline text though (maybe you told that already).

Oh and I just used

-c counter start value

and it worked really well. Saved me a lot of time by creating one sndshd file for all sound files in different folders.

Edited April 19 by datiswous

[datiswous]

Btw, I found that some subtitles still fit in inline subtitles although they're over 6 seconds because the speach is spread out. For example:

Quote

Oh... (1 second pause) Ahh... (1 second pause), Good question... (2 second pause) I'll think about it.

[Geep]

16 hours ago, datiswous said:

I just tested Cadet because I wanted to know if it's good at something and I have to say, it seems very limited.

Yeah, it's pretty simple. I'm using a beta build, that adds an automatic cps calculate & display, and better supports my workflow... don't know the beta has seen public release yet. I found I could live with Cadet's quirks better than those of kdenlive. Chief among the latter: because kdenlive is based on video, it insists on changing exported srt times, presumably to conform to the frame timestamps (of a non-existent video). This bothered me too much, maybe more than it should.

16 hours ago, datiswous said:

I don't really understand why you need an editor for inline text though (maybe you told that already).

I don't specifically, though I do use utility programs and Excel to help calculate constraints and make decisions about -

• inserting line breaks
• when to use the new inline durationExtend option, and what value to give it
• SRT instead of inline
• Non-verbatim instead of verbatim transcription

For a given vocal set, it appears just a small number of bark clips need the SRT treatment (and so Cadet) instead of inline.

17 hours ago, datiswous said:

Oh and I just used

-c counter start value

and it worked really well. Saved me a lot of time by creating one sndshd file for all sound files in different folders.

Another satisfied customer of buildSubtitleShaders.

I've got another console utility about ready to release, for group checking SRTs. Soon.

7 hours ago, datiswous said:

Btw, I found that some subtitles still fit in inline subtitles although they're over 6 seconds because the speach is spread out. For example:

Quote

Oh... (1 second pause) Ahh... (1 second pause), Good question... (2 second pause) I'll think about it.

For me, this is an example of where it CAN fit in an inline, but should be placed in an SRT instead. I would definitely put the final phrase into a subtitle message, and at least consider breaking into 3 or 4 messages.

For The Wench, there were some clips (of 7 - 30 seconds) where she's idly humming. Rather than a single inline, e.g., "(humming long tune, hoarsely)" that stayed up the whole time, I went with srt, showed that message briefly, and then towards clip end showed "(humming ends)".

[Geep]

checkDurationsinSRT is now available:
checkDurationsInSRT.exe
checkDurationsInSRT.cpp

This Win/console program scans a directory for .srt files, reporting on each in turn. It warns about those subtitle phrases(aka messages) that are potentially too short or too long in time, or that seem to require too high a reading rate, expressed in characters per second (cps). It also looks for within-file subtitle messages that overlap in time.

Invocation:
checkDurationsInSRT -l lowerBound [default is 1.0 sec] -u upperBound [default is 6.0 sec] -c maxCPS [default is 17] -d dirWithSrtFiles [default is current dir] -o output file [default is stdout]

For more, run with -h or ? options, or see comments in the C++ source code file.

EDIT: The above links now access a May 6, 2023 update, that corrects a significant error in character counting for the cps calculation. In addition, if a given subtitle's cps is too high, the warning now includes a suggestion: "try x.xxx seconds". That says what longer duration would be needed to achieve the target cps. How or whether than longer duration could be achieved is left up to you.

Edited May 7 by Geep
May 6 update

[datiswous]

16 hours ago, Geep said:

Chief among the latter: because kdenlive is based on video, it insists on changing exported srt times, presumably to conform to the frame timestamps (of a non-existent video). This bothered me too much, maybe more than it should.

I don't know what you mean by this. When you change the srt file inside kdenlive you have to export it (which is saving), or if you saved as a project before, you can just click save to (also) export the srt.

Or do you mean this import window? I always ignore it, because I don't know what it means in relation to audio files and it didn't give me any problems:

 

16 hours ago, Geep said:

I'm using a beta build, that adds an automatic cps calculate & display, and better supports my workflow... don't know the beta has seen public release yet.

Where can I download the beta?

Edited April 20 by datiswous

[Geep]

8 hours ago, datiswous said:

I don't know what you mean by this.

In the kdenlive subtitle editor, use the "End" counter to adjust the subtitle phrase end point. Then export to an .srt file, and look within that file. Does the phrase have the same specified endtime, or has it been changed?

8 hours ago, datiswous said:

Where can I download the beta?

The latest Cadet release version, 2.0.044 of April 14, incorporates the beta. The main feature added was that, if you go into Tool/Preferences, there's a new line "Show cc reading rate with alarm threshold of [...] in units of [CPS or WPM]". There's also a bit more flexibility in how to advance from one phrase to the next.

BTW, I found the Cadet project feature adds no value. I just (re)use an anonymous project, and export the .srt. If later I need to adjust something, I can use "Import" of the .srt to do so.

If I strongly needed a tool that had speech-to-text capabilities, I would not use Cadet. Since I've got preexisting text for the vocal sets I'm currently processing, I'm not needing speech-to-text right now.

[datiswous]

4 hours ago, Geep said:

In the kdenlive subtitle editor, use the "End" counter to adjust the subtitle phrase end point. Then export to an .srt file, and look within that file. Does the phrase have the same specified endtime, or has it been changed?

Hmm, yes I see it now. Actually it doesn't actually change it, but it just looks slightly different in the editor. So if you resize, then export, then import, you see the same end time, but it is different from the srt. Very weird, I never noticed it. But I can understand this is an issue.

Btw, although Kdenlive has built-in speech to text and I was initially excited about it, I now use Whisper for that, which is MUCH better at it.

[Geep]

The update to subtitles for The Lord vocal set ("Lord2") is now available, that makes use of the new -dx (durationExtend) feature of 2.12dev:

testSubtitlesLord2.pk4

Corresponding Excel file - TheLord2Subtitles.xlsx

As usual, the testing FM is used as the vehicle to distribute this update. Out of 390 subtitles (all of which are inline, no srt needed), the changes are as follows.

Durations Extended

• 67 times, -dx was used to pad duration between 0.25 and 0.50 second (instead of default 0.20), calculated based on a presentation rate of 17 cps. The value of 0.50 was set as a cap by policy. When reached, it leads to a presentation rate higher than 17 cps.
• The cap is ignored if the presentation rate reaches the quite fast 20 cps. For Lord2, this occurred once (-dx 0.61, a value calculated for 20 cps presentation rate). This particular subtitle now is verbatim, unlike the original Lord subtitle release, where the text was shortened.
• Also for presentation rate reasons, leading optional "(...)" descriptors were dropped or shortened (1 case of each).
• In the spreadsheet, additional columns were added to help in the foregoing calculations. This will be documented when The Wench is released.

Style Issues

• Capitalize leading 'Tis or 'Twas 14 times.
• Replace [...] with (...) for descriptors.

Improvements for testSubtitles... FMs

• This applies to Lord2, forthcoming Thug2 and Wench releases, and later.
• In-game, the "Prev", "Again", and "Next" buttons are now twinned with a group of 3 more buttons labeled "Same but Quiet". As requested by @datiswous, these affect the cursub numeric counter without either playing an audio sound or showing its subtitle.
• The cursub counter (as an HUD overlay) shows itself more consistently, thanks to an implementation hack.
• Each subtitle appears in a field that is now Geep-standardized to a half-screen wide, compatible with a 42-char/line limit. This is done with an override of  tdm_subtitles_common.gui, based upon a customization of TDM 2.11 code. (So it does not include visual indicators of AI location or speech volume found in recent 2.12dev's tdm_subtitles_common.gui).

Edited May 4 by Geep
Oops, option is "-dx" not "-xd"

[Geep]

The update to subtitles for The Thug vocal set ("Thug2") is now available:

testSubtitlesThug2.pk4

Corresponding Excel file - Thug2Subtitles.xlsx

The testing FM is used to distribute this update. (See the preceding description of "Lord2" for recent testingSubtitles... improvements.) Out of 392 subtitles, the changes are as follows.

Durations, Phrases, Linebreaks

5 formerly-inline subtitles are now each 2 srt phrases. Of the 5, 4 are muttering or snoring; 1 has a yawn between 2 sentences.
For inline, 14 -dx extensions are added to pad duration between 0.25-0.50
4 previously-shortened inline captions are  now restored to verbatim, due to extension (-dx or auto 0.20 seconds)
10 enforcements of 42-char/line policy are done by adding linebreaks

Style Changes
15 Square brackets are replaced by parentheses
3 Sentences starting with 'evening are changed to Evening

Transcription More Like Vocalization
1 "is" replaced by 's
1 "?" replaced by "."

[Geep]

The subtitles for The Young Builder (Builder 4) vocal set is now available:

testSubtitleYoungBuilder.pk4

As usual, the testing FM is used as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

• There were 471 subtitles. Only 10 were SRT; the rest inline.
• Of the inline, about 87 required a -dx extension of duration in the range 0.25-0.50 seconds (By policy, 0.50 was the cap, hit by 35 of these. In a worst case, that cap could be raised; none required that).
• Of the srt, two had a first phrase that had to be slightly shortened (non-verbatim).

Corresponding Excel File

TheYoungBuilderSubtitles.xlsx

The final form of the Young Builder Excel spreadsheet was also retained (without data) as Version 4 of the evolving Subtitle Template. Documentation about the spreadsheet/template usage will be available.

[Geep]

The subtitles for the Average Jack vocal set is now available:

testSubtitlesJack.pk4

Subsets of the Average Jack vocal set are defined for civilians, guards, beggars, cooks, labourers, and merchants.

As usual, the testing FM is used as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

• There were 388 subtitles. Almost all were inline.
• Only 12 were SRT; these were all two-message, except 1 was 3-message.
• Of the inline, 52 required a -dx extension of duration, as follows:
  • 31 in the range 0.25 to 0.49 second.
  • 17 at the cap (by policy) of 0.50 second.
  • 4 worse-case fast-talkers, raised slightly beyond the cap.
• One of the cap-exceeders also needed a word dropped (non-verbatim).

Corresponding Excel File

AverageJackSubtitles.xlsx

The final form of the Average Jack Excel spreadsheet was also retained (without data) as Version 5 of the evolving Subtitle Template. This and documentation about its spreadsheet/template usage will be available shortly.

 

Edited June 8 by Geep

[Geep]

FYI, there are now two documents that describe my current workflow for the bark subtitles associated with any AI.

The "Overall" Doc

This gives a 1-page summary of how an Excel  spreadsheet serves as the hub to manage and tune the inline subtitles, and flag those that need an SRT treatment. SRT is then covered in more detail, as is preparation of an FM for testing the results and releasing them for incorporation into TDM:
Subtitling Overall Workflow, SRT, Testing (Jack - June 2023).docx

The "Explained" Doc

This concentrates on the latest version of the spreadsheet, beginning with its instantiation from an Excel template, followed by population of the first four columns with program-generated data and with strings copied from the vocal set. Manipulation of each subtitle, and its affect on generated column info and your decision making, is then covered. Finally, export to a text file and further text editing creates the content needed to be incorporated into the testing FM:
Explained - SubtitlesTemplate(v5) & AverageJackSubtitles.docx

[Geep]

The subtitles for The Pro vocal set, popular for upper-level guards, is now available:

testSubtitlesPro.pk4

As usual, the testing FM is used as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

• A new console program "statsForSubtitles" now calculates this info.
• There were 353 subtitles. As usual for barks, almost all were inline.
• Of the 5 that were instead SRT -
  • 3 were 2-message
  • 1 was 3-message
  • 1 was 4-message.
• Of the inline, 54 were arranged on 2 lines with a manual line break.
• Of the inline, 46 required an explicit -dx extension of duration, as follows:
  • 26 in the range 0.25 to 0.49 second, for a 17 cps presentation rate.
  • 18 at the cap (by policy) of 0.50 second. (For one of these, a near-inperceptable word was dropped to achieve this.)
  • 2 worse-case fast-talkers, raised slightly beyond the cap, for a 20 cps presentation rate.

Corresponding Excel File

TheProSubtitles.xlsx

This spreadsheet used (without any changes) Subtitle Template V5, documented previously in the context of the Average Jack vocal set.

 

Edited June 24 by Geep
update statistics

[Geep]

The console program "statsForSubtitles" is newly available, to produce the type of information shown in the previous post (although exact output format is now somewhat different).

statsForSubtitles.exe - Window executable

statsForSubtitles.cpp - C++ source code

As usual, check the source code comments for more.

[Geep]

The subtitles for the The Maiden (aka The Maid) vocal set are now available:

testSubtitlesMaiden.pk4

This character was described in the original vocal script thusly: "This is a young female from the common class. She is probably a servant or tradeswoman. She is earnest and somewhat timid and used to being ordered around by her betters. She might complain when alone but she's always polite and proper to someone's face. Although some lines are included for armed characters, she is primarily going to be an unarmed character...."

As usual, the testing FM serves as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

In file fm_root.subs there are 228 "inline" subtitles, categorized as:

•  39 with an explicit linebreak, intending 2 lines
•  189 without

 4 of the inlines have explicit duration extensions. These are all in the range of 0.25 to 0.49 seconds, for 17 cps presentation rate.

There are 6 SRTs, all with 2 messages apiece. Of the 12 total SRT messages, 2 have an explicit linebreak, intending 2 lines.

In all, in this vocal set captioning, there are 234 voice clips with subtitles, showing 240 messages.

Corresponding Excel File

TheMaidenSubtitles.xlsx

This is based on Version 5 of the Excel Template, which was also featured in the preceding Average Jack and The Pro work.

Edited July 9 by Geep

[Geep]

The subtitles for the The Grumbler vocal set are now available:

testSubtitlesGrumbler.pk4

This character was described in the original vocal script thusly: "a cynical complainer. He’s got a rough job and a rough life and he wants to make sure everyone knows it. He’s pessimistic and grouchy. Not well educated."

As usual, the testing FM serves as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

In file fm_root.subs there are 360 "inline" subtitles, categorized as:

• 64 with an explicit linebreak, intending 2 lines
• 296 without

  64 of the inlines have explicit duration extensions, as follows:

• 37 from 0.25 to 0.49, for 17 cps presentation rate
• 26 capped at 0.50 seconds, for 17-20 cps
• 1 with more than 0.50 seconds, for 20 cps

There are 4 SRTs, all with 2 messages apiece. Of those 8 total SRT messages, 4 have an explicit linebreak, intending 2 lines

In all, in this vocal set captioning, there are 364 voice clips with subtitles, showing 368 messages.

Corresponding Excel File

TheGrumblerSubtitles.xlsx

This is based on Version 5 of the Excel Template for TDM bark subtitles, which was also featured in the preceding work for Average Jack, The Pro, and The Maiden.

 

[Geep]

The subtitles for The Mature Builder (aka Builder 3) vocal set are now available:

testSubstitlesMatureBuilder.pk4

The wiki's "Vocal script: Builder3" suggests these traits for this cleric: "strong, intimidating voice, determined; archaic language; patient, talks about servitude, orthodoxy." Compared to other builders, this one seems to enjoy quoting scripture more often and at more length.

As usual, the testing FM serves as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

In file fm_root.subs there are 315 "inline" subtitless, categorized as:
  - 52 with an explicit linebreak, intending 2 lines
  - 263 without
 
3 of the inlines have explicit duration extensions, as follows:
      - 2 from 0.25 to 0.49, for 17 cps presentation rate
      - 1 capped at 0.50 seconds, for 17-20 cps
      - none with more than 0.50 seconds

There are 9 SRTs, including:
  - 8 with 2 messages
  - 1 with 4 messages

None of these were given a duration extension.

Of the 20 total SRT messages, there are:
  - 12 with an explicit linebreak, intending 2 lines
  - 8 without

In all, in this vocal set captioning, there are 324 voice clips with subtitles, showing 335 messages.

Corresponding Excel File

MatureBuilderSubtitles.xlsx

This is based on Version 5 of the Excel Template for TDM bark subtitles, which was also featured in the preceding work for Average Jack, The Pro, The Maiden, and The Grumbler.

 

[Geep]

A "Style Guide" for how I have been doing the bark captioning has been percolating on my desktop for some time. In order to move it along, I have split it into 2 parts, and the first part is now available as a Word doc here:

Subtitle Style Guide - Part 1

Part 2, the remainder, with more supplementary material, hopefully will be available around year's end.

[Geep]

FONT NOTE: Starting with this test FM, to improve readability, the Carleton subtitle is compressed in width to 75% of its size in previous test FMs. The backing field is similarly reduced in width. These are implemented in the FM's custom-override guis/tdm_subtitles_common.gui. If desired, you may copy that file to previous test FMs to propagate the change.

The subtitles for The Lady (aka Noblewoman) vocal set are now available:

testSubtitlesLady.pk4

The wiki's "Vocal script: Lady" suggests these traits:

Quote

educated; upper class; restrained and refined. The Lady is used to being obeyed. She speaks well. Try for a vaguely aristocratic accent if possible. Think along the lines of Elizabeth Swann in “Pirates of the Carribean”

As usual, the testing FM serves as the vehicle to provide these subtitles for eventual incorporation into TDM.

Statistics

In file fm_root.subs there are 243 "inline" subtitles, including:

• 37 with an explicit linebreak, intending 2 lines
• 206 without

  17 of the inlines have explicit duration extensions, as follows:

• 11 from 0.25 to 0.49, for 17 cps presentation rate
• 6 capped at 0.50 seconds, for 17-20 cps
• none with more than 0.50 seconds

There are 2 SRTs, each with 2 messages that respectively indicate start and end of singing/humming.
No duration extension was needed, and none of the 4 messages contained a linebreak.

In all, in this vocal set captioning, there are 245 voice clips with subtitles, showing 247 messages.

 Corresponding Excel File

LadySubtitles.xlsx

This is based on Version 5 of the Excel Template for TDM bark subtitles, which was also featured in the preceding work for Average Jack, The Pro, The Maiden, The Grumbler, and The Mature Builder.

[Geep]

The subtitles for The Moor vocal set is now available for eventual incorporation into TDM:

testSubtitlesMoor.pk4

"Vocal Script: Moor" describes him as "a foreign soldier from the south. He is a man of few words, fluent in English but with a strong (Nigerian) accent. He occasionally uses awkward phrases, and is as liable to communicate with a grunt or single word if that will do. He is tough, stern, and reserved, not given to a great deal of emotion."

While not spelled out in the original vocal script, the accent/dialect performed had words that start with "th" (e.g.,"there, that") pronounced as "d" (e.g., "dere, dat"). This is what linguists call "th-stopping". Serious consideration was given to rendering the subtitles that way for flavor, but ultimately the original standard spellings were kept for ease of understanding.

Statistics

In file fm_root.subs there are 221 inlines, including:
  - 12 with an explicit linebreak, intending 2 lines
  - 209 without

Of these, 15 were given explicit duration extensions, as follows:
      - 12 from 0.25 to 0.49, for 17 cps
      - 3 capped at 0.50 seconds, for 17-20 cps

There are 3 SRTs, each with 2 messages apiece. Two of these were "tune hums" with a 2-second start message and later 2-second end message.

The remainding SRT, with two sentences of text, could have just fit into an inline, but manual-word-wrapping within the second sentence would be poorly placed. With SRT, each sentence got its own message, and the manual linebreak in the seconds sentence was at a reasonable place.

In all, for this voice, there are 224 voice clips with subtitles, showing 227 messages.

Corresponding Excel File

SubtitlesMoor.xlsx

This is based, as usual, on Version 5 of the Excel Template for TDM bark subtitles.