Hi internet! I want to sonify this brick using camera and MediaPipe - Object detection or Blob detection and transfering it to midi notes in Ableton in real-time
What are you seeing and how it should sound:
The green brick is 4x4 and green means drums. From the bottom it is: kick, snare, hihat, crash. 4 beats. The blue one is piano and it is from tone C to Fsharp (because we count the halfsteps) and it also is loop of 4 beats. The last one is a 8x8 and scale is from C to D2. It plays a C Major chord. This means that the 4x4 loops plays 2 times during one 8x8 loop and they play at the same time. There can be also odd number of beats that would create Polyrhytms.
Sound of white blocks:
The white block can obviously differ the round are stacciato and square and longer / if you create a rectagle out of squares it is held even longer.
How would you approach this problem?