Signal-Driven Video Installation

Hey everyone!
I’m working on a TouchDesigner project. It’s a 15-minute signal-driven video piece—basically, I have a CSV with time + signal values and a set of time markers that trigger video clips.

I need help with:

  • visualizing the signal as a waveform in relation with the time (15 min total)
  • Using the time markers to play videos at the right moment (I have another CSV with timestamps that determine if there is a clip or not in each timeslot, and if yes which one). basically I want to make it work as a kind of editing timeline
  • adding some transitions between the video/ non-video states

hope it makes sense
if you have any leads I’d me more than grateful:))

Hi @yanakiz,

welcome to the community.

It might help if you could share your project to illustrate what you already tried and where you are struggling.

Make sure to include the data you are trying to use or alternatively some sample data.

Best
Markus

Hello @snaut ,

thank you for your message.

You can take a look at the project here:
NewProject.toe (176.3 KB)

At this point I’m a bit stuck with how to make a total duration of 15min of the signal animation, and secondly I would like to import the time markers CSV file that determines if there’s a video clip triggered on top of the signal animation and in which timeslots.

I’ve tried to visualize this process in the following timeline-like graph:

On the zip you can find the eeg signal data that I’m using, as well as the CSV with the time markers.
data.zip (154.9 KB)

Thanks again!

Hi @yanakiz,

so there are 2 different datasets, first the eeg data which looks to be a very regular interval of measuring points and secondly the time markers.

First thing to try would be that all data is depended on your timeline. As your framerate is set to 30 fps, the length of the timeline should be 27000.

Let’s start with the eeg data. I would solve this by only converting the EEG Data from the DAT via the DAT to CHOP, scaling it and then converting it to a SOP with a CHOP To SOP.
The CHOP To SOP also works without an input and will create as many points as necessary - it’s a nice easy way to just create geometry without having to keep track of number of points. It also has parameters for Start Position and End Position so you can keep it in a normalized space.


To only show the data up to the current frame, you can utilize a Trim CHOP placed before the CHOP To SOP, setting the Unit Values to “Absolute” and the End Parameter Unit (little unit menu behind the parameter) to “Fraction”. Now we need to calculate the fraction the timeline is currently on. To do so, use a Timeline CHOP and append a Math CHOP where in the Multiply parameter on the “Mult-Add” page you enter the expression 1/me.time.end where me.time.end refers to your timeline’s end frame (in your case 27000).
Now use the resulting value and export or reference it on the End parameter of the Trim CHOP.


For the time markers it might be best to use a Timer CHOP in Segments mode where each row in your data table describes a segment with a start time. The Timer CHOP also allows to show data of specific columns of the currently playing segment to be shown in a Info DAT making it ideal for your case.
To run the Timer CHOP properly and get as much information as possible, we should add a column that specifies a length of each segment. This we can derive from the “Timestamp” column by subtracting the current timestamp value with the timestamp value from the next row.
For this first add in another column with the Insert DAT. A nice feature of the Insert DAT is that we can add a Column with an expression, meaning we can fill the column with data right away.
So choose for the Insert parameter “Columns with Expressions”. For the At parameter i choose “Index” and set the Index parameter to “1” to add this column as the 2nd column in the table.

Here now some python…

Now for the Expression, first set the Names parameter to “length” and for the Cell Expression parameter, switch it to expression par mode and use this:

900 - me.inputs[0][me.subRow+1,0] if me.subRow == me.inputs[0].numRows-2 else me.inputs[0][me.subRow+2,0] - me.inputs[0][me.subRow+1,0]

now this is a bit much but first of it uses a conditional if - else structure that can be written in a single line:

value if thisConditionIsMet else otherValue

The most important part of the expression is me.inputs[0][me.subRow+2,0] - me.inputs[0][me.subRow+1,0] where:

  • me.inputs[0] is a reference to the input table
  • me.subRow is the index of the created row not including the header

so for the first segment, me.subRow would evaluate to 0 but when accessing table cells, the index of the first row is 0 in this case where our header is placed. Hence the bit where to reference values in the current row, we need to use me.subRow+1 as an index and referencing values in the next row we need to add 2.

The whole if condition is so we can take care of the last segment - we have start times but no end times - so for the last segment we need to subtract the start time from the total runtime of the project.

sorry about this… but it works and keeps things tidy.

finally we need to rename the columns for the Timer CHOP to understand the values and their meanings. Here first select out everything but the first row using a Select DAT setting the Select Rows parameter to “by Index” and the Start Row Index parameter to 1.
Next add a Insert DAT and add column names in the Contents parameter: begin length stage content title

Finally add a Timer CHOP, on the Segments page add the just modified DAT as the Segments DAT and fill out the Columns to Info DAT parameter with the columns you want to see during playback (stage conetent title). Now back on the “Timer” page, hit initialize and start and after, set the Time Control parameter to Lock to Timeline.
You can now also add a Info DAT and reference your Timer CHOP to see the columns data we wanted to have returned.

The Timer DAT also allows for outputting various channels. Useful here could be the “Segment” channel as well as the “Running Time Count” in Frames.

Cheers
Markus
NewProject (5).2.toe (179.9 KB)