How I hacked my sons Duplo train to go faster using my voice
So my son got a Duplo train for his birthday. The train is fairly advanced. There are different blocks you put on the tracks that make it do different actions: one will make the train turn on the light, one will reverse the driving direction of the train, one will make the train play a specific sound and one will make the train stop.
On top of that you can control the train with an app (because of course). In the app you can control the speed, different sounds, the lights and the driving direction. The app connects to the train via Bluetooth and that gave me an idea 💡
What if you could control the train using your voice? Wouldn't that be cool!? 😎
Now I had to research if someone had tried something similar. That led me to a Dutch guy that had put his sons duplo train on the blockchain, where the train made sounds and drove around based on some events on his own alt coin.
✅ Was it possible to control the train - yes
Connecting to the train
Now I knew it was possible to control the train, but could I make it work?
My first stab at it was on my Windows machine. I started developing a bit too quick (story of my life) without checking the prerequisites. This gave me a lot of problems. When I finally found the prerequisites I saw that it was a lot easier using a Mac compared to Windows.
So I booted up my work computer and quickly got a successful connection. Yeah 🥳
The script that the Dutch guy had made was outdated though. The entire node-poweredup api had changed so into the documentation I dove 🏊♀️
It took a bit of time but I finally made the train drive and quickly learned to debug the train by putting it on its side. My arm was getting a bit tired after constantly moving the train back and forth.
✅ Could I control the train - yes
Getting the pitch of someone's voice
On to the next problem: Figuring out if I could get the pitch of someone's voice.
I knew this was possible. I had made a previous project in python where clapping would start playing some music. This was done using a technique called fft where you transform the audio signal into its frequency domain.
But the connection to the train was made in nodejs. Therefore I had to do the pitch detection in nodejs. This proved to be a huge problem 🤔 I am still not really sure if it is possible. I tried out a few libraries but didn't really get anything working.
Then I tried to approach it from another angle 🔎 What if I instead detected the pitch in the browser? Luckily there was a great library called Pitchy that solved this exact problem. Look at this simple, magnificent api with a great, clear example 👌
Now all I needed to do was to send the pitch to nodejs.
✅ Could I detect the pitch in someone's voice - yes
Gluing everything together
All I had left from here was to glue everything together by sending the pitch to nodejs and then using that pitch to control the speed of the train. That was done using Socket.io. Socket.io lets you send data bidirectionally in real time. So from the client to the server and from the server to the client.
Luckily I had also worked with Socket.io before while building an online slime volley game, so it did not take too long to get it working.