I work as a Technical Artist and VFX TD who is about to enter the hardware era in my career.
I want to build an interactive setup that is corresponding with the real time build on the PC, getting its input from the camera + translating the data into visuals. I would like my LEDs to be lit up based on the real time data like colour dominance or any other coded condition.
What would be your recommendation for the entry level hardware proof of concept?
I would like to avoid investments in high quality chips and microcomputers when I’m not sure about the outcome.
Anything like soldering tools is covered. I’m prepared in this case.
Looking forward to your explanations and recommendations!
All the best to you guys and thanks in advance,
Your best bet with small scale proof of concept would be ESP32. Not only is it dirt cheap, but has great community support and documentation. Then look into WLED, and read the documentation thoroughly. Depending on the size of the installation, you MAY be able to build it purely with several ESP32s. If what you’re planning is on a medium to larger scale (1000+ pixels to drive), then best to stick to hardware-free previs software like GeoPix.
What do you think about Arduino?
For now I’d simply build a little 8x8 led square or a little cube that would respond to colours from the input. What I struggle with is understanding which setup I need to prepare first to make it readable from my PC (TD or UE) through the microcontroller and then to the LED panel. I understand how to make a project driven by code (aka triggering certain led pixels over time), but I don’t know how make the controller suck out the real time data from the project first.
You can absolutely use Arduino for that amount of LEDs, but I cannot comment much on the implementation since I’ve only used ESP8266, where the pixel data was sent with UDP, as opposed to serial communication. There are great free learning resources about pixel mapping for Arduino and Immersive & Interactive HQ offers a course on this exact topic.
Essentially, you would code your microcontroller to expect a list of rgb values for each pixel, then “paint” each value to the corresponding pixels to the LED strip. On TD side, you would use toptoCHOP, a shuffleCHOP might be necessary depending on your LED wiring order, then serialDAT to send the data to your Arduino. This tutorial shows this exact process in detail. Good luck!
Other options (although more expensive and not necessary for a starter project) are LED controllers which typically have an Ethernet input and serial output. You’d feed them ArtNet over the Ethernet connection via a DMXOut CHOP. I use this in an installation which is exactly the same as what you’re describing, a temporary install with TD and UE5 driving projections and LEDs and this is the setup I use.