Jump to content

FFT window on OpenAL buffer (for spectrum data)


CrazyM
 Share

Recommended Posts

I'm trying to tap the OpenAL raw audio data to see if it can be processed with an FFT window to build a live stream of the spectrum or amplitude of a playing audio file. This is quite simple in FMOD since it ships with a getSpectrum function. However, we are evaluating the feasibility of bringing our automated lip sync solution, originally built for Unity on top of FMOD, to Leadwerks, and after speaking with Josh, including FMOD as a built in option for Leadwerks users isn't an option at this time. Personally, we would love to see an asset store for Leadwerks as we think Unity has proven it's a good source of income for the host company, encourages developer involvement, and makes it easier for new developers. It is popular enough to convince Unreal Engine to get on board and open their own asset store.

 

Josh informed me that the Bank class pointer references the raw data but I'm struggling to determine if the data in the bank->buf could be processed through an FFT window.

 

Are there any OpenAL pro's here that know if this can be done, and might be will to share some expertise?

 

In case anyone is interested or curious, here are some of our promotional videos for our solution on Unity.

 

Thanks

  • Upvote 1
Link to comment
Share on other sites

The Workshop is not monetizable, yet.

 

The buf is a memory address. The bank class has another member that indicates the size.

 

The bank class also has a lot of built-in functions for reading memory, like this:

unsigned char c;
for (int i=0; i<sound->data->GetSize(); i++)
{
c = sound->data->PeekByte(i);
}

My job is to make tools you love, with the features you want, and performance you can't live without.

Link to comment
Share on other sites

Esenthel have it's own store also.

I agree a monetized asset store is missing , i doubt this will be possible in Steam as only products owners can sell DLC.

The other way would be like the Sci Fi DLC, make some agreement with Josh than let him put on sale your product as DLC on the workshop, but that would represent too much time and effort to manage if they were several people wanting to sell stuff as DLC.

Let's give time to LE3 to maturate first, the store can wait.

Stop toying and make games

Link to comment
Share on other sites

Thanks Josh, I'm working with the data now.

 

To anyone interested, I found a really great article on processing audio with an FFT window. I think this might work, though I may have to read the article 3-4 more times. wink.png

 

Part I

http://www.edn.com/electronics-news/4383713/Windowing-Functions-Improve-FFT-Results-Part-I

 

Part II

http://www.edn.com/electronics-news/4386852/Windowing-Functions-Improve-FFT-Results-Part-II

 

If I can build a getspectrum function, then we can try convince Josh to include it in the API. Having this ability brings a lot of cool possibilities, lip sync, music-based games, etc.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...