How I hacked my sons Duplo train to go faster using my voice
So my son got a Duplo train for his birthday. The train is fairly advanced. There are different blocks you put on the tracks that make it do different actions: one will make the train turn on the light, one will reverse the driving direction of the train, one will make the train play a specific sound and one will make the train stop.
On top of that you can control the train with an app (because of course). In the app you can control the speed, different sounds, the lights and the driving direction. The app connects to the train via Bluetooth and that gave me an idea 💡
What if you could control the train using your voice? Wouldn't that be cool!? 😎
Now I had to research if someone had tried something similar. That led me to a Dutch guy that had put his sons duplo train on the blockchain, where the train made sounds and drove around based on some events on his own alt coin.
✅ Was it possible to control the train - yes
Now I knew it was possible to control the train, but could I make it work?
My first stab at it was on my Windows machine. I started developing a bit too quick (story of my life) without checking the prerequisites. This gave me a lot of problems. When I finally found the prerequisites I saw that it was a lot easier using a Mac compared to Windows.
So I booted up my work computer and quickly got a successful connection. Yeah 🥳
It took a bit of time but I finally made the train drive and quickly learned to debug the train by putting it on its side. My arm was getting a bit tired after constantly moving the train back and forth.
✅ Could I control the train - yes
On to the next problem: Figuring out if I could get the pitch of someone's voice.
But the connection to the train was made in nodejs. Therefore I had to do the pitch detection in nodejs. This proved to be a huge problem 🤔 I am still not really sure if it is possible. I tried out a few libraries but didn't really get anything working.
const [pitch, clarity] = detector.findPitch(input, sampleRate);
Now all I needed to do was to send the pitch to nodejs.
✅ Could I detect the pitch in someone's voice - yes
All I had left from here was to glue everything together by sending the pitch to nodejs and then using that pitch to control the speed of the train. That was done using Socket.io. Socket.io lets you send data bidirectionally in real time. So from the client to the server and from the server to the client.