A Kinect sequencer ? First Steps and method discussion

Hey wonderful human beings ,

we`re planning a music and interactive Theater Piece for February 2020.

One part of it will be a “recipient sequencer” where peoples position is translated to a musical notes scale and other data.

with another technique i did an “Light Sequencer” some time ago (https://github.com/circat/TD-movie-to-midi)(old version), but would love to discuss my idea before i mess it up from the first attempt. :smiley: Also it`s a huge project and this is one part out of 5. no time to waste ! :mrgreen:

My question : you have a technique for this or a good hint how to solve this problem ?
Should i convert the kinekt data to a Table?

Thank you for your time and blessingz,
Erich // https://www.circat-sound.de/

Part I : The Recipient Position Sequencer

Explanation : 1 or 2 Kinects are mounted to the ceiling of a room to scan the position of recipients. By scanning the floor horizontal,vertical or radar-wise, the recipients position is translated to midi messages/notes and corrected to music scale . A 5m x 5m space is virtually tiled to 8 x 8 squares for 64 possible positions // musical notes.

  • Kinect2 to locate points on a 5m x 5m floor
  • Tile the floor to a grid with 64 Steps / Positions for Midi Notes
  • Scan Grid Horizontal / Vertical / Radar - wise.
  • For each active grid cell ( when a person is on it ) generate a message 0/1 or 0 - 127 ( depth from Kinect )
  • Sync grid scanning to Ableton
  • Mapping // Projection of Fluid Simulation, grid and other graphics on floor

I think about this Technique from lightsequencer , but i resize the video for the lighsequencer and this is not what i want to do for kinekt…

get the depth video from kinekt2,

threshhold the values that i get values from 20 cm to 200cm above the floor,

resize to 64 x 64 pixel for my grid ? ( this is nonsense,or isn`t ? )

TOPTO ( RGB Units 255 ),math to 127 and then CHOP TO row per Table,
then convert to midi osc with python:


source = int(op(‘select12’).numRows) # get number of rows in the table as int
arrayMidi = [source] # number = nuberof Rows in the Table

def onValueChange(channel, sampleIndex, val, prev):

for x in range(0,source):
	if op('select12')[x,0] > 0.1:
		velocity = op('select12')[x,0]
		op('../AbletonSync/abletonMIDI1').SendMIDI('note', x, velocity-30)
		op('../AbletonSync/abletonMIDI1').SendMIDI('note', x, 0)