English Subtitles for AI Barks

[stgatilov]

10 hours ago, datiswous said:

Btw. there's some confusing text in that article:

https://wiki.thedarkmod.com/index.php?title=Subtitles#include

There's no fm_test.subs in an example. I think it should be fm_conversations.subs . Not sure, but maybe you could check?

Yes, you are right.
I think that's what happens after a lot of edits

[Geep]

@datiswous, made that correction fm_test.subs --> fm_conversations.subs

@stgatilov, about srt naming and file location, would you be OK with the following edit? New/changed stuff in italics:

srt command is followed by paths to a sound sample and its .srt file, typically with matching filenames. An .srt file is usually placed either with its sound file or in a "subtitles" folder. The .srt file format is described e.g. [1]. The file must be in engine-native encoding (internationalization is not supported yet anyway) and have no BOM mark. It contains a sequence of text messages to show during the sound sample, each with start and end timestamps within the sample's timeline. It is recommended to use common software to create .srt files for sound samples, instead of writing them manually. This way is more flexible but more complicated, and it is only necessary for long sounds, for instance sound sample of a briefing video.
 

Quote

There is a link to .srt description. Also, .srt is kinda standard format.

It's a simple enough standard that it can be shown as an short example, demonstrating that subtitle segments can have time gaps between them. And the example can show correct TDM usage, without requiring a trip off-site and picking through features that TDM doesn't support. Specifically, the example shows how to define two lines by direct entry, rather than using unsupported message location tags (X1, Y1, etc.). And skips other unavailable SRT font markups like italics, mentioned in the wikipedia description. The example would also show the TDM-specific path treatment.

The example could be inserted before the sentence "It is recommended to use common software...."

[datiswous]

How much dialog can fit in an inline subtitle until you need srt ?

[Geep]

It's the duration that matters. If the clip exceeds 6 seconds, use srt. Individual segments within srt should be 1-6 seconds long.

There's more to it than that. I'll DM you later today with a Word doc, a draft fragment of the style guide under development, that delves into this. Specifically, if there's too many words to read in the time that a subtitle is shown, you may need to edit out some words. That is, move away from a verbatim caption. The fragment has quantitative guidance on this.

[Geep]

BTW, under Win11 File Explorer, you can see the duration (called "Length") of sound files in a directory, as well as sort on that. Use the "Details" view. Right-click on the columns header, and checkmark "Length" to include its column.

Resolution is only rounded to the nearest second, tho. Adequate for making the "inline" vs "srt" choice. Not for calculating WPM.

 

[Geep]

For the Thug, out of 393 utterances, only 9 required subtitle editing to stay within a 240 words-per-minute reading rate, the highest anyone thinks reasonable.

There were:
 

Verbatim --> Shortened

Let's have a look. --> Let's look.

I'm ready for him. --> I'm ready. [only needed for shortest clip with this phrase; 2 other clips were fine verbatim.]

How'd you like a taste of this? --> Like a taste of this? [2 clips]

I'll piss where I want to piss. --> I'll piss where I want to.

I guess I have to do it. --> Guess I have to do it

I guess I have to do it. --> Guess I have to. [shortest clip]

The son of a whore was right here. Look around! --> (curse) ...was right here. Look around!

Look, you bastard! --> Look, bastard!

If I used the still high but slightly lower reading rate of 200 WPM, about 21 additional edits would be needed.

Edit: Actually, those last 2 don't need editing. Spreadsheet calculation problem fixed. Might revisit some criteria too.

 

Edited February 12 by Geep

[Geep]

I changed over to finer-grained characters-per-second metric instead of words-per-minute. Using 20 cps, 3 of the above phrases could be left verbatim.

[Geep]

The subtitles for The Thug vocals have been posted to the bugtracker #6240 for review & eventual incorporation into 2.12dev. They are in FM form, embedded into QA-program testSubtitlesThug. The .pk4 is available here:

https://drive.google.com/file/d/1VNP2fNge-2Ff3RkdUvCLagRENUESVW40/view?usp=sharing

Easy instructions on using this FM can be found in both the Notes and Briefing. That info also sketches how the test program can be adapted by FM authors to present & test sets of their own custom subtitles. (I'm not claiming getting your data into this FM is real easy.)

As I refined these subtitles, a particular style evolved, and particular tools/methods were used. I'm working on documenting this, but it won't be ready to release for a while. And things may mutate further as I tackle the next AI's subtitle. I'm thinking I need to do a character with longer speeches next, e.g., a noble.

[datiswous]

Currently I just write down the subtitles till 6 seconds length from the audio files in the subs file. For longer I use Kdenlive and check correctly formatting in that app. Interested in your workflow though.

Edit: Actually this is really nice for testing out inline text, but I guess it should also work for srt... Interesting,this means I don't have to testplay the whole mission. This should be sufficient.

Edited February 14 by datiswous

[datiswous]

Maybe you can put the instructions in the objectives screen as well so you can revert to that when "playing" It's easier than supplying it as note.

[datiswous]

I got it working now with my own audio and inline subs and srt files. Works really well for testing ingame. It's a bit work to set up at first.

Sometimes the subs text overlaps the weapon and selected inventory item (left and right in interface), so I think the gui for in-game subtitles should be narrowed, so it doesn't overlap. For mission intro's this isn't an issue and therefore full width is fine.

Edited February 15 by datiswous

[Geep]

6 hours ago, datiswous said:

Maybe you can put the instructions in the objectives screen as well so you can revert to that when "playing" It's easier than supplying it as note.

Interesting idea, though I wouldn't want the text to be interpreted as objectives by the player.

Quote

I got it working now with my own audio and inline subs and srt files. Works really well for testing ingame. It's a bit work to set up at first.

Glad that's working for you. Yeah, getting the data into shape is a pain. I did it for me with a lot of Excel in/out and text find/replace misadventures. Some potential for process improvement there, next time.

Quote

Sometimes the subs text overlaps the weapon and selected inventory item (left and right in interface), so I think the gui for in-game subtitles should be narrowed, so it doesn't overlap. For mission intro's this isn't an issue and therefore full width is fine.

Most guidance for subtitles would restrict them to the central 2/3rds of the screen, width-wise. So maybe narrowing the fields a bit throughout would be good. Need to think about this. I guess the lowest field would also obscure the health and breathe bars a bit.

 

[datiswous]

On 2/15/2023 at 5:37 AM, Geep said:

Glad that's working for you. Yeah, getting the data into shape is a pain. I did it for me with a lot of Excel in/out and text find/replace misadventures. Some potential for process improvement there, next time.

I used Excel as well, but I did:

Spoiler

1. First in command prompt dir /b > voices.txt (actually I used ls, because I use Linux, but this is how it is done with dir). The /b gives you a file-list without extra info around it.

2. Pasted the voices.txt contents in Excel. Paste in column E.

 3. Fill in the first line like this:

4. Select the first 4 cells. A1, A2, A3 and A4. Then use the + sign right under the cell-selection and move it down till the end of the list. After that move column D to Column F and delete Column D. So it looks like this:

5. Then in cell F1 you do command =A1&B1&C1&D1&E1

You get then the combination of cells A t/m E together in cell F1. Then you do the same as with column F what you did in step 4.

6. Then select column F, copy and then right mouse button paste values over it.

So you see this:

5. Now you can copy column F to your sound shader:

It can be done in even less steps, I think..

I guess, maybe it's useful to put this on the wiki..

 

On 2/15/2023 at 5:37 AM, Geep said:

Interesting idea, though I wouldn't want the text to be interpreted as objectives by the player.

I don't understand why this is an issue. This is not used by real players, only by subtitle testers. Putting it in the objects field is easiest and you can acces it easilly by pressing O (or the key set for it). But you can also add a note file to the inventory, but I thought this is more work.

On 2/15/2023 at 5:37 AM, Geep said:

Most guidance for subtitles would restrict them to the central 2/3rds of the screen, width-wise. So maybe narrowing the fields a bit throughout would be good. Need to think about this. I guess the lowest field would also obscure the health and breathe bars a bit.

I think @stgatilovsaid that possibly more input was needed for the correct gui. Maybe the current gui is designed from the briefings and ai barks experience. In briefings there is no player interface with icons left and right and health bar and the short ai barks will perfectly fit in the central, so at first it seems it looks ok.

So in my opininion the in-game gui has to change so that the subtitles are not able to overlap any interface elements, instead of working around the limitation.

Edited February 21 by datiswous

[Geep]

6 hours ago, datiswous said:

So in my opininion the in-game gui has to change so that the subtitles are not able to overlap any interface elements, instead of working around the limitation.

I agree. I'm copying this idea over to the https://forums.thedarkmod.com/index.php?/topic/21741-subtitles-possibilities-beyond-211/ thread. Related: I'll probably do some more experiments with the current TDM font & size, to understand max char count versus field width.

[Geep]

Regarding Excel, I likewise used its integer incrementing feature. My spreadsheet was more complicated - ultimately multiple worksheets. That's because I was using it to include the draft subtitles, from which were auto-calculated character and word counts, as well as CPS and WPM, which were then color-coded against specific thresholds

Those latter calculation required the sound file durations. I originally used Windows Powershell to gather those, but Windows only exposes duration to the nearest second, which proved inadequate. I then installed the Windows version of ffmpeg. That includes ffprobe, which reports duration in seconds to 2 digits after the decimal point. Regrettably, the output format from ffprobe is a problem, and Powershell was a big fight for several reasons (among them I'm a Powershell novice). The results were usable but barely.

So I'm not going to write up my method as the hodge-podge it currently is. Maybe one or more specialized C++ or C# formatting programs would be the way to go next for me.

Sadly, the extraction and import of first-draft subtitles from the existing wiki AI vocal scripts will probably continue to be a manual process. Because the .ogg file names have changed from when the old vocal scripts were written, so auto-matching can't easily be done. (This is a problem FM subtitle authors don't have.)

[Geep]

I noted this in the other thread, but let me quote myself here as well:

Quote

 

Just confirmed that the caret markup does work for subtitles... ^1good whore^0 appeared in red.

This markup and its limitations are described in https://wiki.thedarkmod.com/index.php?title=Text_Decals_for_Signs_etc.#Signs_with_Illuminated_Colored_Letters

 

I've updated the "displayed text" section of the wiki's Subtitles to include this and other stuff.

[Geep]

I've chosen "The Lord" vocal script to subtitle next.

[SeriousToni]

Let me chime in just to say that this is a very honorable task - thank you!

Edited February 18 by SeriousToni

[Geep]

I just built a tiny command-line program, "buildSubtitleShaders.exe", to expedite creation of the specialized sound shaders needed by my testSubtitles.... You point the program at a directory of .oggs, and its spits out the completed .sndshd file. Basically, it does what datiswous (and similarly, me) did in Excel earlier, as shown above. I've used it to now generate The Lord sound shaders.

Details -

This was a rather trivial VS 2020/C++ 17 console program. The only minor complexity was in programmatically truncating the directory path down to what TDM wants, namely, a path starting with "sound/..."  There's some sanity-checking to make sure the path provided (either implicitly or explicitly) is of TDM-compliant-form. This runs under Win11; because of file path issues, it might need to be tweaked to work right under Linux.

To use with stock tdm-distributed sounds, you would point it at an unzipped .pk4 directory of interest, and generate the needed file. At run (i.e., testing) time, because of the truncated directory path, it works fine against the usual zipped .pk4, like every other sound shader.

At some point I'll probably post source to github. Going on now to try to build a similar program to gather sound file durations, for import into Excel as a csv file. Will be trickier.

Edited February 19 by Geep
Add name of program

[Geep]

Another C++ program now functional, "soundDurationsCSV.exe", that fetches sound file names from a directory and gets their durations in seconds, to millisecond resolution. The result is output to a .csv file, for import into Excel. Durations will be used to calculate WPM and CPS reading rates for subtitles.

Details:

This Windows-specific console program relies on separately-installed ffprobe (bundled with ffmpeg) to get each duration. The program builds a batch file that calls ffprobe repeatedly, for each sound file. Then quietly executes the batch file, and finally reformats the results into a .csv

The program needs further polishing, but good enough for the moment to start working on Lord subtitles next.

[datiswous]

I wonder how subtitles show up when two seperate voices are played at the same time. This would happen all the time with ai-barks and look bad if they overlap. Probably best if tdm does not allow overlapping of subtitles (so choosing one sound to display, closest).

Edited February 21 by datiswous

[Geep]

Here's my understanding. There are 3 stacked non-overlapping "slots" for a subtitle field to appear in. You can see this in the testSubtitles FM if you hit any of the buttons fast enough. (Granted, this is all in the same voice, not different voices.) When a sound is about to play, the sound system looks for the lowest unused slot to put its field in. And it keeps that field until the sound clip ends (e.g., across multiple srt phrases). If a sound starts playing and there's no slot free, then no subtitle appears for it.

Because the sound system currently doesn't know who/what emits a sound, it can't "reserve" a slot for a particular AI or other entity across separate .ogg files.

Even without knowing the emitting entity, what the subtitle system could do now (but doesn't currently) is look at the standardized pathname of the sound file, and determine what vocal set it comes from. It could then "reserve" a slot for a while for future utterances from the same voice. Is that a good idea? Unclear to me. (There's a broader issue here about giving priority to "story" phrases over "speech" phrases. I'll move this to the Future thread.)

Later in the year, when more bark subtitles for different voices are in 2.12dev, it should be possible to just drop your player into a room of miscellaneous AI and fill up those 3 slots big time.

[Geep]

@datiswous and others, you can get my Windows console program buildSubtitleShaders.exe now at:
https://drive.google.com/file/d/19Pf513nv5gwOzZ5tyWHJN7Ka-D9UxBET/view?usp=sharing

Do "buildSubtitleShaders -h" to get help on parameters.

The output should be compatible with an instance you clone (and rename) from the testSubtitlesThug FM, provided you go into the FM's .script file and change

#define SHADER_PREFIX "fm_ai_thug_shader"

to

#define SHADER_PREFIX "fm_test_subtitles_shader"

This will be the fixed sound shader prefix I use for all the AI vocal sets subtitles I will make available (embedded in various testSubtitles...) to QA in the future.

I'm not releasing the full VS project for this program right now, but you can make your own console project and build it yourself with the only file needed, buildSubtitleShaders.cpp:

https://drive.google.com/file/d/16Epx2V2iGa2b91Mnt0cbzGSb3IQBJpk7/view?usp=sharing

Needs C++ v17 to compile. Rest are VS2022 defaults.

For more on purpose and deployment, see comments in

• start of buildSubtitlesShaders.cpp
• generated output file

 

[Geep]

Now available for QA & incorporation into TDM 2.12 - subtitles for The Lord

Testing FM (now with instructions also in Objectives, as requested by @datiswous). Assumes TDM 2.11

testSubtitlesLord.pk4

Description of workflow, including use of Excel spreadsheet:
SubtitlingWorkflow with Excel (for Lord, March, 2023).docx

Excel spreadsheet used to develop subscripts. Calculates various metrics:

TheLordSubtitles(v10, March 3).xlsx

[Geep]

The Windows console/C++ program to populate the first 2 columns of the foregoing spreadsheet, with the sound files and their durations, is now available, as executable or as source code:

soundDurationsCSV.exe

soundDurationsCSV.cpp

To run this successfully, you must have ffprobe/ffmpeg installed on your machine, to fetch the durations. The comments of the cpp file has a link to info about how to do that.

For command-line parameters, do: soundDurationsCSV -h

If you want to compile this yourself, it requires C++ 17 (or later).