I’m exploring the possibility of using Touch for live visualization of brain signals (electrophysiology) digitized at 30kHz. It would be great to be able to pull in at least 16 channels, display the data, do some live filtering, and maybe even pipe a channel or two to audio to get started (hopefully more exciting things down the road). I could down-sample the data before sending it into Touch, but that would eliminate the possibility of doing the latter two things. Should this be possible? When trying to pull in the data, using OSC, to the “OSC In” DAT, with buffer sizes of 1000 or less floats per channel (there appears to be a limit to the OSC message size in touch that’s below the standard UDP limit?), I noticed the following things:
For trying to pull in 1 to 4 channels at 30kHz, with each OSC message being one buffer (1000 floats) of one recording channel (i.e. each buffer represents 1/30th of a second), things run well.
When I try 8 channels, the frame rate quickly approaches 0, and CPU memory usage increases dramatically.
If I down-sample to 10kHz, I can get 8 channels to work pretty well, but the program chokes at 16 channels.
If I further down-sample to 3kHz, 16 channels run alright, but the program chugs when I try to visualize the signals through a DAT-to-CHOP.
Is there a more optimal way to pull in data like this? These data channels have a lower sample rate than typical audio – has anyone been able to pull 16 or more channels of audio (44kHz) into Touch successfully? I’m using a reasonably powerful machine (quadcore, 4gb RAM) with Windows 7.
Just a fast thought: maybe reduce the parameter “Maximum Lines” of the Osc In DAT to 1 and instead point the Script DAT parameter to a script which takes the values (accessible via $args or $arg1 $arg2 …) and adds them to a Table DAT via the expression tabinsert. Maybe that would make a difference…
Also there is a Udp In DAT as well - does that show the same behavior of a smaller message size then standard?
what software are you using to send the OSC messages?
Is it hand-coded or built in PD, Max etc?
maybe some filtering of repeated values might help
the ‘change’ object in PD is good for this for example - prob. also in Max
rod.
Markus: Indeed I’ve clamped my input to both 1 line and 16 lines at various times. If I don’t clamp, there’s a runaway memory leak and the program quickly crashes. As for creating a script DAT, I haven’t tried scripting yet in Touch but I’ll give it a shot. My intuition is that it won’t help because I have problems scaling the input channels even when the only DAT in the program is the OSC DAT (so no additional processing is being done). I have tried the UDP DAT in the nightly-release versions of the software (I’m using the latest stable release FTE for the most part, which doesn’t seem to have a UDP dat) with similar results. I can’t make the buffer size much more than 1000 floats. I had read somewhere that typically UDP packets are limited to 64kB, so it seemed to me that I should be able to send more data per message (but maybe all of the meta data associated with the OSC message is taking up a lot of space).
Rod: I’m sending the OSC messages using the oscpack library (audiomulch.com/~rossb/code/oscpack/) for C++. I hadn’t really considered compressing the data, but perhaps that could help. Typically the electrophys signals I’m analyzing jitter from sample to sample, but I could maybe set a threshold to define a significant change in the signal.
Is there a fundamentally different way of pulling in and visualizing the data that would make things more scalable and run more smoothly? Like maybe writing the data to shared memory, and pulling it directly into a GPU-accelerated TOP (somehow)? Is it uncommon for Touch users to pull in a lot of audio channels at once for visualization?
I’ve tried the Pipe in CHOP, using the example C++ code. It works well for a couple of channels, but there is a slow memory leak for several channels at 30 kHz, and it can’t handle 16 channels at that rate. In general I’d like to steer away from TCP though, and use a protocol like UDP/OSC that won’t potentially slow things down on the computer that’s dealing with the digitizer callback (I can afford a loss of a little data – for visualization anyway – if that means that things run deterministically).
I’ve tried the OSC CHOP. Giving it one channel with an arbitrary buffer size, say 100, it creates 100 channels – I’m not sure why. Touch crashes when I give it 16 channels at 30kHz.
And just following up on Markus’ scripting suggestion, I tried having the OSC DAT point to a Text DAT with the script: tabinsert /project1/fifo1 r $args (writing to a fifo clamped at 1 or 16 lines), and it didn’t seem to make much of a difference – Touch still crashed with 16 channels at 30kHz.
I’ve been using the expression “crash” sometimes when I mean “causes Touch to slow down to the point where I can’t do anything with it”. The latter is the case for this particular scenario. Basically the CPU memory ramps up quickly, and everything stalls. I’ve attached a screenshot.
Hi Brian, after some inhouse discussion this is what we’ve found:
We’re able to pipe through 30 channels @ 16KHZ between two Touch processes running
on the same machine very smoothly (60 fps).
However, this uses the proprietary CHOP TouchIn/TouchOut.
We think the best solution would be to extend the CHOP OSC In/Out in a similar
manner to also allow receiving timeslices of samples.
This would open up interoperability between different data sources quite well.
We’re going to explore this change and hopefully have something coming up.
Thanks a lot, Rob. In case it’s helpful for debugging the OSC input, I wrote a little C++ multi-channel digitizer simulator for arbitrary numbers of channels, digitization rate, and buffer size – though I imagine it’d be just as easy for you to create an arbitrary signal in Touch, use OSC out to pass it through the socket, and then pull it back in. If you do choose to use the code, you’ll need the attached source files (MultiChannelSimulatorOSC.cpp, HighResTimer.h), and you’ll need to include the source files from oscpack. You’ll also need to tell your linker to add WS2_32.LIB and WINMM.LIB as dependencies.
Hi Brian.
Thanks for your tester. Im going to begin by just optimizing Touch->Touch and then move onto testing with an external app such as the one youve provided.
Cheers
Rob.
Hey Brian.
I ended up not compiling your source code, but I was able to opimize both CHOP OSC Out and OSC significantly…
I also added an option to discard samples while timeline stopped.
With these changes I was easily able to maintain 60fps, sending 16 channels @ 16khz each (4 byte floats) between two processes running on my machine.
Look for it in an upcoming experimental release.
Cheers
Rob.
Hi Brian.
The next experimental is up.
You should find you can send 16 @ 16khz channels between two Touch’s on the network easily.
We also tried 6 channels @ 48khz.
One thing to keep in mind, is that we now have an option to send multiple samples per message at once. (Format: Timeslice)
The CHOP actually sends up to 90 samples per channel, not the 4 shown above.
This cuts down dramatically on the data pushed through compared to one sample at a time.
