Camera MIDI Solution

Beginners
, ,
1

Hi, I’m new to TD, and I didn’t know about python before…
But I a m a composer, I am using TD for some cool design.

Here’s my original thoughts: I can possibly split the videos into maybe 48 blocks, and each one of them stands for one MIDI note.
I’ve look through the forum and found one interesting mouse tracking midi output, but I feel 127 in each axis is too much. I want them be set to blocks to be performable.

Here’s the problem that I encounter: I don’t know how I can count the blocks in TD.(or in python), So now I have the real-time coordinate of my mouse like (4,4), which I think should be block_index 25. (As they starts from 0) I’ve tried chop to DAT and scrip DAT for calculation, but I don’t know how to loop them per frame, so the table dat output isn’t in real time.
Also, I think Script CHOP might work, but I have no idea about their languages.
And here’s a bonus: I may want to rename the blocks to trigger some more organized MIDI notes, for example, block 1’s output will actually be 12ch1, to trigger the 12th midi note.

In the end I want to map the information gathered by Mediapipe in math and do a camera MIDI. But there are more difficulties than I imagine.

I appreciate a lot if anyone can help me with that, or if there’s already some cool midi triggering patches, please let me know.

2

Hi @hastingssu,

your challenge is to convert x/y coordinates into an index and we can do that all in CHOPs.

First you’d to define your grid so you know how many cells you are dealing with. To simplify things, let’s consider a regular 5x5 grid - so you can in the end trigger 25 different midi notes.

We also have to decide on the order of the grid cells. For this example we would say that bottom-left is cell 0 while top-right is cell 24.
Your mouse input coordinates should be normalized to a 0-1 range making it easier to adapt to any kind of grid size.

image
image2544×628 44.4 KB

First step would be to convert the normalized coordinates to a cell index. For that, use a Math CHOP and set the Multiply parameter on the Mult-Add page to the number of cells in any direction -1 - so in our case, set the parameter to 4. We also don’t want floating point numbers but rather integers, so on the Math CHOP’s OP parameter page, set the Integer parameter to “Round”. Moving the mouse around, you should now be seeing the Math CHOP containing values that correspond with the rows and columns of your grid.

Now since every row has 5 cells, to calculate the cell index, we’d need to take the rowindex, multiply by 4 and add the current column index. So the complete formula would be:
rowIndex*4+colIndex

To do this append 2 Select CHOPs to the previously created Math CHOP and in the first, select the horizontal value of your mouse, while in the second select the vertical value of your mouse.
Corresponding with the formula, append a Math CHOP to the vertical value and set the Multiply parameter to 5 (the number of columns).
Next append another Math CHOP and also connect the second Select CHOP you created and set the Combine CHOPs parameter to “Add”.

The result should be an index going from 0 to 24. To trigger individyual notes, we will require individual channels - for this we can use the Fan CHOP with it’s “Fan Out” operation. In this mode, the Fan CHOP will creat as many channels as you specify in the Channel Names parameter (in our case set it to “chan[1-25]”) and will set the channel value of the channel with the same index as the input value to 1.

That’s pretty much it. Append a Null CHOP and you are ready to go.

For learning more about the involved operators, make sure to check out OP Snippets and also have a look at our curriculum at learn.derivative.ca providing short lessons with foundational knowledge of TouchDesigner.

cheers
Markus
base_gridIndex.tox (1.5 KB)