Touchlib page updated

I updated the touchlib page to include some more details about compiling the source and running the config app. On the source code, I removed the need to include the directshow base classes since I took out the dscapture filter (it was redundant).

No Wii for Mii

I got to Toys 'R Us at 9:00 (they open at 10:00) and the Nintendo Wii pre-orders were already all spoken for. Apparently a bunch of people were camping out in line since 4:00 pm saturday. The line was at least 50% full of people waiting for the new Tickle Me Elmo (which is a hilarious toy, according to people in the line). Also amusingly, one of the grandmothers waiting in line for the tickle me elmo called the Wii, the Wiz. Looks like Walmart is my last hope, though judging by the interest that fans have shown so far, I'm going to have to camp out pretty early in order to get one.

another FTIR video

I found this video on youtube yesterday:




Could this be someone who's been to this blog before? Looks like cooking oil on the surface to improve the FTIR effect.

Maintenance

I just set up a little page to house all the important info on the touchscreen. You can find it here. Also, I'm making a page that has all my projects on it here.

What I'm up to

Dunno about you guys but I am eagerly awaiting Nintendo's new console, the Wii. (For those that don't know what it is and are looking for a multitouch tie in, the Wii is probably the first console that will allow for gestural control). There's only about a month left to go before it's released. I missed the EB games pre-order so I am currently weighing my options for how to get a console on launch day. There is a Walmart down the street from me, so it might be possible to get one at Midnight as some people have said they are doing. However, I haven't seen anything official about that so I dunno if I want to take a chance with it. Has anyone else waited in line for something at walmart around midnight? Toys R' Us is also reportedly going to offer Wii pre-orders next weekend. I'll probably go with that. Final Fantasy XII also comes out next week, so that should keep me busy until Wii time (and distract me from getting anything useful done).

On the programming end I've shifted my focus a bit. I'm pretty satisfied with how the multitouch table has turned out so far - it's not perfect but it's probably as good as it can get without me plopping down large sums of money for new cameras or exotic screen materials. So, I'm going to focus on Genome again. Genome is a music sequencer application that I have already put a few months into developing. All the basics are there - you can create instances of instruments and effects, write note patterns, audio is working, saving and loading works, etc. The thing that will set Genome apart from all other music apps is that it will be the first multi-user music application. It will allow multiple people to work on one song at the same time over the internet. And yes, I know there are a few programs which already allow something like this, but I think Genome will be the first to do it the way that it does it. Online collaboration will be the most important part, and I hope to show that working on a song with others can be fun and addictive - just like playing a game. Genome is influenced by Buzz's modular nature and I would like to include a module that allows you to build new effects and synthesizers similar to how the Nord Modular works (by connecting simple building blocks). Anything you create can also be exported and shared. There will be a big emphasis on community and sharing resources which were important parts of Buzz's success too. The GUI will be designed to provide a fast workflow and be easy to use - expect lots of keyboard shortcuts for all the major functions (everything will be accessible by mouse too of course).

Touchlib Source Code

For those that missed it, all our source code is available on a public subversion server at http://publicsvn.edmstudio.com/multitouch . It's written in C++ (the BlobTracking / Analysis is all written by me) and has a Visual Studio 2005 Solution ready to compile. No docs are available right now and it's windows only (though it should be possible to make everything compile under other OS's with a little work). It currently depends on OpenCV, DirectShow, VideoWrapper and the DSVideoLib. The source code includes our main library which you can link into your application to start capturing touch events. It has support for most major camera/webcam types. It also includes a basic config app which will need to be run in order to calibrate your camera, and has a couple example apps. Alternately, I've heard other people have used things like vvvv, EyesWeb and Max/MSP in order to do blob tracking / processing and make applications.

DIY Instructions

Looks like more and more people are getting in the multitouch table act. Here's a wiki about building your own table. Thanks, Chris for the link. Be sure to also check out my Freemind Mindmap which has some step by step instructions and also info where to find things.

Claymod in action

I just uploaded a couple videos of Claymod in action. Definately not perfect yet - in fact you can see some of the problems with Claymod in these, but the multitouch stuff is working.

More Vids

Old news, but worth checking out again. I like the Phycon thing. It should be pretty easy to build interactive tangible elements that work similar to how the Augumented Reality stuff works - just make some kind of pattern on the bottom that shows up in Infrared. The tangible item could have it's own IR LED inside it. Software could pick up the patterns visually and each item could be uniquely recognized.

Touchscreen Progress

I got Claymod partially working as a multitouch application. I just need to do a little more work on it before it's ready for prime time. I'll be sure to post some videos when it's ready.

Sweet links

Well, I must admit that I've been saving this one for a rainy day. Hey, it's not like there is news everyday in the world of interactive multitouch tables.

Anyway, here are a couple really good links about interactive tables in general:

Pasta and Vinegar: Interactive tables - links to a bunch of tables (using a variety of techniques), some that I hadn't heard of before

reacTable: other musical tables - a bunch of musical table links. Includes videos. I haven't looked at all of them, but I'm particularly fond of the Augmented groove one.

Thanks to Futnuh for passing these along. If anyone else has any great links they'd like to share feel free to send 'em over. My email is admin[at]whitenoiseaudio . com.

Mindmap

I uploaded my mindmap for the touchscreen here. I hope it's useful for some of the people reading this who are interested in doing their own screens. You'll need Freemind (which is coincidentally, free), in order to view it. I'm a big fan of freemind for planning out projects and getting ideas down. It's hierarchical nature allows you to organize and hide parts you are not interested in. It can also link to images, other mindmaps and web pages.

Links

• Some nice info on the Tangent table.
• Links on del.icio.us tagged with multitouch

DIY IR Pen - Pics

I uploaded a new Flickr Set of pics of an IR Pen I put together for use with my table. I actually made two pens, but I only have construction pics of the second one. The basic technique is to use a clicker style pen and use the clicker portion to push a small momentary switch that turns on an infrared LED placed inside the clicker. The switch is a small one that radio shack sells. For power, I used a AAAA battery in one pen and a watch battery in the other.

vNes

Check out this NES emulator that runs in a browser, complete with sound. It's already got lots of games online - you don't need to have any unsightly illegal roms on your machine.

Helping blind people see using sound?

I had a brainstorm yesterday about a way that might help blind people see using sound. Someone has probably already thought of this, but I figured I'd post it anyway. Anywho, you may have heard of this boy who has learned to get around by making clicking sounds with his mouth and listening to the echoes that come back to get a feel for his surroundings. Bats, as you know, use ultrasonic clicks to create a picture of their surroundings. Basically they are getting an impulse response. All animals have the ability to process this kind of information - everyone can tell the difference between being in a large room or small room based on the way the sound echoes. There have been attempts to turn images into sound (by converting colors to different frequencies) to help blind people 'see', but it really makes more sense to help them focus on the impulse response of the room they are in since the human brain already has this kind of circuitry - it just needs help to be able to learn how to use it better.

So, here's my idea for how it could work. You'd need a high frequency capable speaker, a high frequency stereo recorder, a pair of headphones, and a computer to process the input and output. The speaker, headphones, and recorder would be part of a hat that the sight impaired person would wear. High frequency clicks are outputted from a speaker (ultrasound clicks would probably be best? I don't know much about ultrasound). A stereo recorder simultaneously records all the sounds. You could Highpass filter out everything except ultrasound. The results should resemble the impulse response for the room. Then you use the impulse response to re-create the reverberations of the room (you could feed it with white noise or some other kind of sound) and play it into headphones for the sightless person. Everything would happen in realtime so as you move your head you could hear how the impulse response is affected. You'd be giving them the ability to focus on the sound of the shape of the room and since it's in stereo, get a 3d picture of it. Since it would use high frequency sound, above the normal hearing range, it would be pretty unobtrusive, though there's a chance it might bother dogs and some other animals. ;) The big challenge would probably be making it portable (it might require a lot of processing power) and finding audio equipment capable of high frequency operation. Consumer cards already purport to being able to handle 192khz, but finding speakers and microphones that can handle that might be difficult..

Touchscreen Progress

I started working on a IR pen input device (pictured on the left) using a pen, a momentary switch from radio shack, an LED and a watch battery. I still need to find a way to mount the battery, but otherwise it seems to work. It lights up in IR when you press down.

I also started thinking about converting Irrlicht's GUI code to work with multiple inputs and I think I have an idea on how to do it that shouldn't require too many changes or break it for non-multitouch appliactions. It will be kind of nice getting this working, since then I'll have a complete GUI system (with buttons, dropdowns, windows, etc) that is multitouch friendly.

I am also going to think about how the screen can be made even more sensitive and user friendly. Dragging operations are still a bit difficult since when your finger starts moving, the FTIR spot tends to get less bright. Fixing may take a combination of getting better quality images and more predictive blob tracking.