Crazy Idea: Analyzing Ogg Vorbis Sounds

[Ishtvan]

So for sound propagation, a suspicious sound is approximated as an impulse sound (square pulse in 1/60th of a second (1 frame ) ). I'm going to make a "def file" for sounds, so you sound designers will provide a "base volume" and a "frequency band" (the frequency band is very simplified, it will just be one of 6 options I think: very low, low, mid, high, very high and ALL.

 

Then when you play the sound on an entity, you can modify the base values for the sound (for example running on stone is louder than walking on wood, so if the default footstep is walking on wood, multiply the base volume by a multiplier when the player is running on stone).

 

Now, I was thinking: Ogg vorbis sound format is completely open sourced. It would be cool if we could do an analysis of a sound automatically to get the base volume and frequency band numbers.

 

If we did this, frequency band wouldn't have to be so simplified either: it could have the actual Fourier coefficients of the sound when it's decomposed into frequency components in those bands, rather than simple bits (this would take up a little more RAM, but whatever).

 

So basically my crazy idea is: Write a program that takes in a list of Ogg Vorbis sounds. It decodes the sound, calculates some sort of max volume (maybe time averaged around a short time on either side of the peak volume). This is used to generate the "base volume" for the sound prop. impulse sound approximation. (ie, when the sound is propagated to AI, it just has this base volume on a square wave 1/60th of a second long).

 

2nd, the program performs a Fourier transform of the volume vs. time data for the sound, and gets the fourier coefficients for various frequencies. It then puts these into one of our divisions (say we divide the audible frequency spectrum into 8 bands or something like that), and also normalizes all of them such that all the coefficients added up equals 1.

 

Do you sound designers know of a program that does something like this with Ogg Vorbis sounds?

 

I found the Ogg Vorbis SDK here: http://www.vorbis.com/download_win_1.0.1.psp

 

Anyway, if there's an existing program that does something close to this, we might be able to get this in by the first milestone. I'm not going to attempt to write it from scratch though, so if not, you guys might have to deal with putting in by hand, for each suspicious sound, something like: "soundname = footstep_default", "baseVol = 25 dB" and "freq = freq_mid + freq_very_high" or something like that, in a def file.

[Fingernail]

It would be a computationally expensive system ??? I just get that impression from understanding it.

 

However, if it works without too much performance loss, I say we go for it.

 

Not Milestone 1, though?

[Ishtvan]

It wouldn't run ingame, it would run once to generate your "suspicious sound" def files and "environmental sound" def files. (These are files that determine things like what volume/frequency the sound has according to out sound prop. model). You'd have to run it whenever you put in a new sound as a sound designer, but never during gameplay.

 

Yeah, I'm afraid this isn't something for the first milestone (at least not something I have time for by myself), unless we somehow happen upon a miracle program that already does this.

 

If we have to do it from scratch, there is probably a signal processing library or something for C++ (I can try and ask some of my friends who do signal processing all day). All we really need is a Fourier transform algorithm (people use this all the time so I would imagine it's pretty easy to find) and some way to get a "base volume" that is roughly based on the max volume of the sound, but also takes into account how long the sound is at max volume over a short time range.

[sparhawk]

Signal processing should be available for linux. I'm pretty sure I saw something like that in a package description, but I can't really say where as I never was interested in such stuff myself. But there are all kind of sound processing applications, so there is certainly something there.