Specdrums
Web API
NYU Ability Project
Summer 2020
Hacking into handheld musical gadgets to enhance their accessibility
My Role
I was responsible for parsing complex numerical data, translating it into MIDI,
and creating a Tone.js synth instrument to turn this MIDI data into music.
Problem
Our goal was to adapt the Specdrums, easy-to-use rings developed by Sphero
that produce sounds when tapped on different-colored surfaces, for use
in medical waiting rooms for people with
disabilities. However, the lack of an API for the Specdrums means they
required the official app, which has limited functionality.
Parsing
To get the data, we had to connect the Specdrums to Chrome via Bluetooth Low Energy
and print out all of their data to see what was useful. When the Specdrums are tapped
against a surface, they send out data corresponding to the color of the surface. I wrote
code in React.js to turn the data into either singular notes or chords, and to specify each pitch in MIDI.
Output
I created a synth instrument using Tone.js which takes the MIDI and turns it into sound.
I allow the user to choose a sound and a key, and then I make the synth play notes when
the user taps and release the notes when the user lifts their finger.
Research
I learned about Bluetooth Low Energy, its organization of data, and how devices can be
accessed by and connected to other devices. This included learning about the GAP
(Generic Access Profile) and GATT (Generic Attribute Profile), and how a client can read
or write data from a server, using a UUID (Universally Unique Identifier) to access these
services and characteristics.