Building a Theremin With My Webcam

This is my first writing on my website and I fully expect it to be messy because 1) I haven't journaled in decades and 2) despite using AI to help assist me in my projects/website building I want my art and writing to be as raw and human as possible. I find that this is so rare these days, we are all starting to sound like bots talking to each other!! I am not a bot. This is why I am also being intentional with building a website that looks messy or like a zine? I am not a web designer.

Inspiration irl of theremin

Theremin at the Making Music Museum in Carlsbad

I recently went to the Making Music Museum in Carlsbad, CA. Really, I stopped at Armada dr to have a nice overview of the Flower Fields but I saw this awesome museum across the street. In the museum, I saw an exhibit on the theremin, an instrument from the 1900s where you play two antennas to hear the electromagnetic signals. I remember seeing Chris O'Dowd play the theremin in the Apple TV Show the Big Door Prize. After seeing the instrument, I felt inspired to go home and make a modern day version with my computer and a webcam.

Big door prize

I'm not sure what in particular drew me to the theremin, but later on I saw this quote about the plot purpose of the theremin in the show. For the next three months, I'm on a life adventure, feeling my way into an unknown space. Maybe the theremin called to me because the magic of it and seeing how it works felt symbolic of where I am in life.

Before jumping into the project I mentioned above, I spent some time researching other musicians and creators to draw inspiration from.

Inspiration from Imogen Heap's music gesture gloves

Inspiration from other creators on Youtube

Claude code

I am not a software engineer, but I use Claude Code so much professionally that it was relatively straightforward to set it up to use what I needed: puredata, opencv, mediapipe to get started.

Code review claude output with Codex

I asked it to check a few things: 1) Code efficiency, 2) Out of date packages, and 3) Security risks

Codex review example: Anyone on the same network (ie shared wifi in a cafe) could send a packet and cause a nuisance. With this catch, I was able to modify the code so that my firewall blocks it from reaching Puredata.

To be honest, I am probably overkilling security for this little music app but I like to build habits early and setting these early on will help me when I do build something more seriously.

Final touches: Watching youtube instruction videos to start are still very helpful

I had watched this PureData tutorial to understand how to use PD and still protect my hearing.

What is PureData?

It's a visual programming language used for creating interactive computer music

Improving: Claude initially made music notes too close to each other

Theremin use

Within a few hours, I was able to set up my first theremin proof of concept. I realized I could overlay lines for notes so I could read music and play the theremin. I'll spare you my off tune twinkle twinkle little stars.

I don't think I will continue building on my theremin, but I will continue to build on the concept. I've started building out other instruments like the electric guitar, piano, and drums. I plan on practicing getting good at using the instruments by playing along to my favorite songs that I find midi files for like Washing machine heart by Mitski.

This will also quickly tell me how my instruments need to be recoded to more intuitive or expressive hand gestures. I don't want to build another air guitar or piano where the hand gestures are similar to playing the real instrument. I want to create my own mapping of gestures to control chord changes, volume, pitch etc.

How can I make my hands more expressive? Where I can go with this music? More to come!