datiswous
-
Posts
3224 -
Joined
-
Last visited
-
Days Won
54
Posts posted by datiswous
-
-
Automatic subtitles generation with Whisper
I found a far better alternative to auto generate almost perfect srt files: Whisper.
https://openai.com/research/whisper
https://github.com/openai/whisper
For example I did a test with file Simeon3.ogg, a 44 seconds voice file from fm A house of locked secrets. By using command in terminal:
whisper Simeon3.ogg --model small.en
After a very short time (could be due to it using nvidia cuda, not sure) it creates a bunch of export files, including an srt file with contents: (be warned that you will read the contents of an audio file of a mission, potentially spoiling something)
Spoiler1 00:00:00,000 --> 00:00:07,000 When our time in this world is at an end, our body returns to the earth and our soul goes to the builder's side. 2 00:00:07,000 --> 00:00:15,000 Sometimes the body refuses to embrace the earth, these become the undead, to be purified with the hammer and holy water. 3 00:00:15,000 --> 00:00:21,000 However, sometimes it is not the body, but the spirit that stays in this world after death. 4 00:00:21,000 --> 00:00:26,000 These spirits are known by many names, ghosts, shades, phantoms. 5 00:00:27,000 --> 00:00:34,000 Some are benign and some are harmless, but others can wreak terrible havoc on the world of the living. 6 00:00:34,000 --> 00:00:40,000 As it is our duty to repel the undead, so it is also our duty to repel these spirits. 7 00:00:40,000 --> 00:00:44,000 The holy symbol you have picked up shall enable you to do so.
This is almost exactly how it is supposed to be. Not only does it pick up the language all perfect, it also creates full sentences with punctuation. This is far better than the VOSK method in Kdenlive, which I had to edit afterwards.
After that I load it in Kdenlive together with the sound file and make a couple of easy corrections to the flow. Basically you have to make sure that in the gaps in the audio file the subtitle sentences also stop. See example below:
Spoiler
This is probably a 10x faster workflow.
Edit: You can also list all the files in the command. For example (from dir with voice files):
whisper --device cuda Carlotta1.ogg Carlotta2.ogg Carlotta3.ogg -o ./../../subtitles --model medium.en -f srt
This command generates the subtitles from these 3 files and saves them in the subtitles folder in (only) srt format.
If you want inline, you copy them over from the srt files. Instead you can just use srt for all voice files.
Edit 2
Currently I use this workflow:
- In the folder with voice audio files I open a terminal window.
-
Then I do the following command in the terminal:
whisper --device cuda Carlotta1.ogg Carlotta2.ogg Carlotta3.ogg -o ./subtitles --model medium.en -f srt - Afterwards I move the subtitle files to location root_fm/subtitles .
Note that Cuda can only be used with nvidia gpu.
-
I added the full list of missions that have voice sound files in them, but don't have subtitles yet. See first post.
-
2
-
-
Question: If an fm author overrides an ai bark with it's own soundfile by using the same filename (assuming this goes this way), does the core subtitle still apply?
Edit: maybe it is set in the sndshd file(s). Still same question.
-
1
-
-
I just found that if you save in kdenlive as a kdenlive project, the subtitle srt track is automatically exported as well and every time you save in kdenlive again the srt file is updated. So manual exporting isn't actually needed. The srt is saved in format filename.kdenlive.srt .
-
2
-
-
Some of us are natural writers and some of us are not. I personally have a hard time expressing positive remarks. When I'm satisfied, I've not much to say.
-
1
-
-
On 3/2/2023 at 3:09 PM, gokudo said:
4.) Prey
Which version? The 2006 version comes closer graphically (similar engine) I think, but qua gameplay the 2017 version is probably closer.
-
On 10/1/2016 at 11:05 AM, Obsttorte said:
Does this still work, or can it made workable? This uses arb glprog instead of glsl currently used by tdm. Or can it be converted?
-
Why is subtitles not an integrated part of the hud gui code (tdm_hud.gui)?
-
On 1/28/2023 at 12:03 PM, snatcher said:
Just to give two examples, @Obsttorte's throwable hook and grabber arrow wonderful initiatives didn't make it into any mission
Pretty awesome! Never seen them before.. Maybe not enough people know about them? I think what would be cool at least if a map would be made, like an extension of the training mission, where such tools can be tested in an environment which makes sense. This gives mappers ideas for implementation.
-
What are the current missions with story subtitles present?
To my knowledge (I update them from your responses as well):
A house of locked secrets (datiswous)
A new Job
A Night in Altham
Braeden Church (datiswous)
By Any Other Name
Coercion (datiswous)
CoS1: Pearls and Swine (datiswous)
Deceptive Shadows (datiswous)
Down by the Riverside (datiswous)
Eye on the Prize
Hazard Pay
Hidden Hands: Blood & Metal (datiswous)
Hidden Hands: Vitalic Fever (datiswous)
Lord Edgar's Bathhouse (datiswous)
Mandrasola (datiswous)
No Honor Among Thieves (datiswous)
Now and then (datiswous)
One Step Too Far (datiswous)
Requiem (datiswous)
Seeking Lady Leicester
Sir Talbot's Collateral (datiswous)
Sneak & Destroy (v2) (datiswous)
Somewhere Above the City (datiswous)
Spring cleaning (datiswous)
Tears of St. Lucia
The black mage (datiswous)
The Elixir (datiswous)
The Gatehouse (datiswous)
The Hare in the Snare, Part 1 (datiswous)
The Imperial Sword (datiswous)
The Last Night on Crookshank Lane (datiswous)
The Painter's Wife (datiswous)
The Terrible Old Man
Ulysses 1: Genesis (datiswous)
Who watches the watcher
Winter Harvest 2.5 (datiswous)
Written in stone
WS1: In the North (datiswous)
WS2: Home Again (datiswous)
WS3: Cleighmoor (datiswous)
WS4: The Warrens (datiswous)
WS5: Commerce Bank (datiswous)
You Only Fly Thrice (datiswous)I added the included in mission ones to the mission list on the wiki:
https://wiki.thedarkmod.com/index.php?title=Fan_Missions_for_The_Dark_Mod
Missions that have voice sound files in them, but don't have subtitles yet (I have looked in all the mission pk4 files for such voice soundfiles):
Springheel A Reputation to Uphold
Springheel A Score to Settle (ready for testing)Goldwell Accountant 1: Thieves and Heirs (ready for testing)
Goldwell Accountant 2: New In Town
Goldwell Noble affairs
Goldwell Shadows of Northdale ACT 1
Goldwell Shadows of Northdale ACT 2
Goldwell & Kingsal Snowed Inn
Goldwell Sneak & Soufflé
Sotha The Golden Skull (minor)
Sotha TP1: Knighton Manor (minor)
Sotha TP4: The Transaction
Sotha TP5 Phrase Book
Sotha TP6: Lich Queen's Demise
Sotha Ulysses 2: Protecting the Flock (made, not included yet)Bienie CoS0: To Catch a Thief (minor)
Bienie CoS2: A Precarious Position (minor)
Bienie CoS3: Sacricide (minor)
Bienie Langhorne Lodge
Bienie The Night of Reluctant Benefaction (made, not included yet)Jackfarmer Hidden Hands: The Anomaly
Jackfarmer Hidden Hands: The Lost Citadel (made, not included yet)joebarnin The Heart of Saint Mattis (made, not included yet)
Kingsal Volta 1: The Stone
Kingsal Volta 2: Cauldron
Kingsal Moongate Ruckus (made, not included yet)
Kingsal Volta III - Gemcutter (subtitles for intro video made, not included)Melan, Bikerdude PD 2: All the Way Up (minor)
Melan, Bikerdude PD 3: Erasing the Trail
Bikerdude & team Crucible of Omens : Behind Closed DoorsMortem Desino The Creeps (made, not included yet)
Jedi_Wannabe PtB:0 Moving Day
RJFerret Inn Business (minor)
Wellingtoncrab Iris (minor, only for shops I think)
Spooks King of diamonds
bwyan Cole Hurst 1: Eaton-
2
-
-
I always remove the swort from equipment in the shop at mission start. But I like it more when you can pick up the equipment you want inside the mission, like some missions allow (Requiem, Who watches the watcher, The Accountant 2)
-
Here I release the subtitles I made for this mission. The subtitles are only for the opening conversation which starts in the first 3 second of loading the mission. Therefore it's quite easy to test if it is done correctly.
Unzip the 7zip file into the root of the fm. It adds the subtitles folder.
It would be nice if people can test it.
-
1
-
-
Personally I rather play small to medium sized missions, but this is against the will of most mappers currently.
Why not make a serie of small to medium sized missions with a storyline, maybe campaign, instead of one super large mission that takes 2 years or more to create. Seems more productive. If it's a campaign, just add missions to it via updates later.
-
1
-
-
Instead of that, you could, in that time period, start learning your own missions and in doing so become more active on the forums, learning from others.
It's also great to understand how things work in the engine, when you play. It might spoil some immersion though..
Or learn to build games with other engines, like Godot for example.
-
1
-
-
-
Well this at least got fixed in 2.10
-
On 2/26/2023 at 9:29 PM, SeriousToni said:
But layers don't work with the automation tool, do they? I mean switching on and off the visibility and having that also ingame.
No not really, but you can hide everything except the room and then it's easier to move to the room. But not a good solution.
On 2/27/2023 at 7:02 AM, Geep said:In TDM console, you can use "teleport" to get to a named entity. I found it useful to have certain things (like teleport targets or rugs) with very simple, memorable names to teleport to.
Interesting. Could be used with info_locations. Although HMart's solution is probably better.
On 2/27/2023 at 7:02 AM, Geep said:RE automation tool - which I have little knowledge of because my rig is too underpowered to run active DR and TDM at the same time - I can imagine there could be a way in the future for DR Layers to also send vast numbers of hide/unhide commands to individual ingame entities corresponding to layers. That approach probably would be slow, though.
Well, if you have DR active with a map file with one room-layer active, then in TDM teleport to another room and in DR activate that room-layer and sync camera, it will probably not be so harmful on system resources, depending on the size and complexity of that specific room.
I think this stuff can already be done with scripts directly in DR (make a menu, jump camera to specific positions, activate/deactivate layers, etc.), but I find DR-scripting too difficult to understand (no Python knowledge).
-
1
-
-
On 2/27/2023 at 1:42 AM, Xolvix said:
There's also the Godot Engine which is supposedly increasing in popularity if you believe certain people on the Internet, although I've yet to encounter a commercial game actually using it.
There's plenty of games that use it, mostly 2d.
On 2/26/2023 at 6:30 PM, datiswous said:I often wonder what the use case is for Unity nowadays..
So what I see people say is Godot is way better at 2d than Unity (already in 3.5). I personally can't give an opinion about it, because I haven't used the engines to make games but @Moonbo's new 2d game is using Unity (I think) so I wondered why he made that choice, but I didn't ask yet..
In 3d there are also better options.
I think it's probably the assets store. And possibly the amount of tutorials.
-
I often wonder what the use case is for Unity nowadays..
-
Is there a console command (in tdm) that makes you jump to specific info locations? Coupled with automation, you could quickly jump to different rooms in a large map. I guess instead I could make a filter that filters everything away except the info locations. Then navigate between them..
Probably most mappers use a layer for every room (hide everything except the room)?
-
On 2/21/2023 at 6:16 PM, Geep said:
- Have the "inline" and "srt" commands take an optional additional parameter, for overall duration (i.e., greater than the clip duration).
It seems to me this could also be interesting if you quickly want to create a conversation based on no sound, but with added subtitles, then if it looks good, you can add the audio later. So I can just use one 1 second silent ogg file for multiple voices and for every voice sentence, specify the length and put the subtitle in that length.
-
2
-
-
I used a Lumia 735, which was a mid-range model. This also got the upgrade to Windows Mobile 10,
-
1
-
-
21 hours ago, kano said:
I sure am glad these guys failed so spectacularly in the phone market.
I used to have a Nokia Lumia (Windows) phone for 3.5 years (free updates in that period), I was very happy with it. So I disagree. Their ui was i.m.o. better than Android and ios. It was also relativally fast on low resources.
-
Maybe someone
wants to make a blue-filter mod.
-
1
-
Automatic subtitles generation
in TDM Editors Guild
Posted · Edited by datiswous
Btw Whisper can apparently also do direct text translation, which could be useful later if TDM will support multi-language subtitles.
Edit: This might not work for En -> Other language.
Edit2: Aparently Whisper can directly transcribe for a language, so if you would make speech files in a different language, it would generate to that language subtitle. Maybe I could test it with translated missions, if they exist.