General Purpose GPU roundup

I spent quite a few hours yesterday researching what's currently available utilizing graphics cards to do processing. Graphics cards are essentially stream processors - meaning they are well suited to doing repeated operations on arrays of data. Audio processing is a stream process and could benefit from being run on a GPU. Since GPU's are used to tailored for graphics, essentially we would be using textures as our audio buffers. Currently there are several options available for exploiting GPU's that help make it more like regular programming and hide the complexities of dealing with textures and other strictly graphics based idioms. Here's what I've found:

• Brook. Brook takes programs written in a special language and converts them into C++ code. That code generates custom GPU instructions at runtime, which are
  tailored to your specific hardware.
• CUDA. Specific to NVidia GPU's, but a very nice package overall. Uses a custom compiler to compile special programs which are linked into your code. You can call these functions right from your code.
• Sh. A library that allows you to create GPU programs by interacting with objects just as you would in a regular C++ program (adding, multiplying, etc). It uses overloaded operators and other tricks in order to leverage C++ as a language, rather than making it's own custom language like CUDA and Brook. The advantage of SH is that it's all done in realtime - there is no need for compilation of special programs (like with Brook and CUDA). Sh also does graphics programming if you want to use it for that.
• And while not really a general purpose library, there's also nvidia Cg which can compile programs in realtime.

Now for my needs, I was looking for something that doesn't require me to compile programs ahead of time. I'd like to be able to take my intermediate audio language (DNA) and either run it on the CPU or on the GPU at runtime. This leaves out Brook and CUDA. Sh might kind of be OK for this, but isn't really aimed at this sort of thing and I would probably be circumventing a lot of stuff to get it to work how I want. Plus it's probably not worth the extra overhead and dependencies if I am just stripping out a lot of the functionality. So, that really leaves me with writing my own backend.

Now the other big downside to using the GPU is that there is a lot of overhead getting data on and off the graphics card. It's best to use the GPU when you can offload all or most of your processing to the card. It's not worth it to process little chunks, one at a time. So, for an audio system what would be cool is if I can offload the entire song (all modules) to the GPU card. This would require a significant reworking of Genome's internals so I am not sure I want to go this route just yet. Plus, it remains to be seen how many people have GPU's worth taking advantage of and how many people would be interested in buying such a product. It might be better suited to a research project some day. For now, it might be better to target Multi-core CPU's as opposed to GPU's.

In the near term though, I can think of a couple audio applications that might be suited to using CUDA (or another of the above libs):

• My planned Doppelmangler rewrite. Doppelmangler uses FFT's for analysis. CUDA already has an optimized FFT written for it. The Doppelmangler of my dreams is able to take audio samples and manipulate them in fantastic ways without loss of quality. If it can do that in realtime, with low CPU usage then it would be awesome.
• Physical modelling. True 'physical' modelling using actual 3d models might be interesting (and might be possible to do using Vertex programs), in addition to the more traditional numerical models using waveguides and such (fragment programs). I think it would be awesome if someday I can actually 'build' an instrument in 3D (rather than just dealing with DSP processes). In theory anyway..
• Reverbs and convolution effects (already been done, I know)

Looking ahead, Stream processing promises to make it's way into everyone's machines eventually. Intel is working on the Larrabee processor which will perform GPU functions in addition to General Purpose streaming functions. Multicore is great for the average user who is running several programs at once, but difficult to program efficiently for. Stream processing can be orders of magnitude faster for the kinds of tasks I'm interested in. Future processors will probably include multiple cores and multiple stream processors - essentially everyone will have a supercomputer on their desk.