Jump to content
The Dark Mod Forums

How about AI voice generation using the already existing voices?

STRUNK

By STRUNK
in TDM Editors Guild

 Share

Recommended Posts

STRUNK

STRUNK

Posted
Posted

I think it would be quite a handy tool to have an AI that can generate dialog in the voices that are already included, and even the various voice actors, if they are ok with that ...
and have it run local on your own computer.

Any thoughts on that?

STRUNK

STRUNK

Posted
  • Author
Posted
4 hours ago, datiswous said:

This seems pretty good:

https://huggingface.co/spaces/collabora/WhisperSpeech

I was looking at this one yesterday:
https://github.com/neonbjb/tortoise-tts
 

 

 

6 hours ago, nbohr1more said:

See:

 

Thanx.

So I see voice actors don't like it and some other people neither.

My idea came when I locked out a guard on a balcony and thought; would be nice is he was yelling some
about being stuck on the balcony in his normal guard voice.
Maybe I/we just have to try what the quality of speech generation is these days.
But .. is there any copyright on these "in game" npc voices?

Goldwell

Goldwell

Posted
Posted

I experimented with using AI voices as a placeholder before I asked human actors to do the lines. There are still morality and copyright questions regarding AI voice usage, but that aside, I just find the technology isn't there yet.

 

Here is a comparison from a Builder sermon scene I have in Shadows of Northdale Act 3.

 

This is an AI voice I used for the scene originally as a placeholder

 

 

And this was the final human voice acted version

 

 

In my opinion the human version is infinitely better than the AI generated one. The human voice sounds much more natural, plus there are nuances and inflections the voice actor can bring to the script which can change it in ways that an AI voice can't. Additionally, some voice actors i've worked with will go off script or offer alternative versions, some of which end up being used (a couple of improv lines that AndrosTheOxen used for the shopkeeper in Noble Affairs we're featured in the final game).

  • Like 2

Shadows of Northdale Campaign
ACT I: A Curious Mind | ACT II: Down The Rabbit Hole

Stand Alone Missions
Accountant 1: Thieves and Heirs | Accountant 2: New In town | Spring Cleaning | Lord Edgar's Bathhouse | Snowed Inn | Noble Affairs

datiswous

datiswous

Posted
Posted (edited)

Well if you have voice actors available than the choice is easy. But I think it could be used to generate alternative voices, or when you don't have the time to contact voice actors, or maybe just for some testing purposes. Also, if you want to have voices in different languages, it could be helpful.

 

34 minutes ago, Goldwell said:

This is an AI voice I used for the scene originally as a placeholder

 

This is pretty good I have to say. What software/site did you use?

Edited by datiswous
Goldwell

Goldwell

Posted
Posted
10 minutes ago, datiswous said:

This is pretty good I have to say. What software/site did you use?

https://elevenlabs.io/

  • Thanks 1

Shadows of Northdale Campaign
ACT I: A Curious Mind | ACT II: Down The Rabbit Hole

Stand Alone Missions
Accountant 1: Thieves and Heirs | Accountant 2: New In town | Spring Cleaning | Lord Edgar's Bathhouse | Snowed Inn | Noble Affairs

datiswous

datiswous

Posted
Posted (edited)
15 hours ago, STRUNK said:

I was looking at this one yesterday:
https://github.com/neonbjb/tortoise-tts

Yeah I saw it, but the demo page is not able to generate a voice from audio file. The WhisperSpeech demo does have this ability. I tested this and it does generate a pretty realistic voice, it's not really a copy of the voice you supply. Also if the supplied voice has a heavy accent it makes mistakes.

Edited by datiswous
chakkman

chakkman

Posted
Posted (edited)

To be fair, the A.I. voices are better than I expected, but, I agree with Goldwell that humans still have the edge. Expectedly. ;) 

If that's not an option, then A.I. voices surely will do. I like that the timbre of the voice is really close to the Hammerites Stephen Russell voiced. I guess you took that as a sample?

Edited by chakkman
STRUNK

STRUNK

Posted
  • Author
Posted
6 hours ago, datiswous said:

Yeah I saw it, but the demo page is not able to generate a voice from audio file. The WhisperSpeech demo does have this ability. I tested this and it does generate a pretty realistic voice, it's not really a copy of the voice you supply. Also if the supplied voice has a heavy accent it makes mistakes.

I tried that yesterday and I thought it sounded pretty bad : P
The thing is, there must also be a way to steer the speech output, and train voice sets of the dfferent npc's; bored, angry, alarmed etc. to be really usefull .. I guess. Like training LoRa's for image generation ...
I'll try to install and use tortoise TTS this weekend.

STRUNK

STRUNK

Posted
  • Author
Posted

Ik got this tortoise TTS up and running after some hassle and made a model for Builder1.
Builder one has just 4 audiofiles and I ran 500 epochs on it, what might be way too much for such a small sample size, and the model is about 1.6Gb, but it seems to work quite nicely.
Some outputs sounded a bit strange and for most of them I had to cut of the start of the audiofile for there was some garbage.
I still have to play around with settings but for now it's looking (sounding) quite nice.

 

To install it I followed this tutorial, though some things differ a bit when you install version 3 that came after this tutorial:

He is using some other programs to remove background noise and to prepare the audio, but the voices in TDM are already clean so no need for that. What you will have to do is convert all the ogg files to wav (batch convert with vlc player) for tortoise TTS to be able to handle them.
You also need an Nvidia graphics card.
That said, on my rtx5000 laptop GPU it all takes a lot of time ...

 

 

 

  • Like 1
STRUNK

STRUNK

Posted
  • Author
Posted

The cloning of the characteristics of the voice works quite nice.
I selected sets of audio clips by how "loud" the speech is, Loud, Normal and Soft (speaking up, speaking normal, speaking soft).
I didn't figure out how to get the best audio quality, without weird "audio artifacts".
But as demonstrated down here, the moor certainly sounds like the moor:

 
STRUNK

STRUNK

Posted
  • Author
Posted
1 hour ago, datiswous said:

That's pretty amazing. So this is not related to tortuose-tts?

F5-TTS is a different, non autoregressive model.
The moor voice was done with tortoise-TTS.

ChronA

ChronA

Posted
Posted (edited)

It's pretty remarkable what's possible these days. Maybe large voice samples are no longer even needed.

If they are though, and anyone were looking for samples to train new characters, I suggest considering LibreVox recordings. Since these readings are all in the public domain, the legal and ethical case for using them to create derivative works with AI is much less fraught. LibreVox even says so themselves: https://wiki.librivox.org/index.php?title=LibriVox_and_Artificial_Intelligence_(AI)

In particular Frankenstein, or the Modern Prometheus (version 3) narrated by Caden Vaughn Clegg is excellent. I could imagine a voice and style of intonation like Clegg's main reading pattern fitting really well in the The Dark Mod. His monster voice is not bad either. Some other interesting ones are Greg Bryant in Paradise Lost. His performances could make for a good player character voice. He has a similar gruffness to Stephen Russel as Garrett, but with a different overtone. Lastly Cori Samuel gives some really good performances that could be suitable for young women characters, especially those of noble or plucky-roguish backgrounds.

Edited by ChronA

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing

  • Recent Status Updates

    • datiswous

      datiswous

      (I just write this down, because it's on my mind)
      I think it's time somebody makes the city hub fm 😉
      Technically I see 2 possible ways.
      1. One is where everything takes place inside one mission and you teleport to different enclosed areas of the map to do subquests.
      2. Another is a campaign style mission. You start inside the cityhub. When you do a subquest, the mission ends, but the next mission you load the specific quest-map. This questmap you load is depended on what subquest you choose to start inside the map. Afterwards you load the 3th mission in the campaign which will be the cityhub map again, but via persistant info, changes some things inside the map (for example, you can't do the same subquest twice) and you have more gold maybe, etc, etc. So if the cityhub has 5 subquest missions that can be played in any order, the total amount of missions in the campaign would be 10, but the amount of map files 6. Because if you start in the cityhub map, after each subquest mission, you return to the cityhub map again, except for the last mission. When you make missions later you can add them to the cityhub again and make it bigger.
      (This seems all possible with changes made in TDM 2.13 , but I'm not 100% sure. It's at least an interesting project to work on)
      · 0 replies
    • datiswous

      datiswous

      I started using Unexpected Keyboard for Android phone.
      It's pretty cool, because it has an actual Ctrl key, which you can use to do cut, copy and paste actions via x, c and v. pretty handy in the forums, to move quoted text around for example.
      · 0 replies
    • STiFU

      STiFU

      New home, who dis? 🙂
      · 3 replies
    • Xolvix

      Xolvix

      Dammit there are too many missions available now. That's what I get for being away for so long. Such a first-world problem.
      · 1 reply
    • DeTeEff

      DeTeEff

      The best thing about not mapping more than like every other year is the awesome feeling to see how far TDM has gotten in all appartments. I can just create some parallaxing cobblestones and walk to and fro and just let the jaw drop
      · 2 replies
×
×
  • Create New...