I’m trying to create a video mosaic using multiple input videos. The mosaic should be divided into a grid, with each segment of the grid filled dynamically from a corresponding segment of one of the input videos. Some details;
The final Output is a video composed of multiple segments arranged in a grid (e.g., 3x3 or 4x4 layout). Each segment must come from a specific region (e.g., top-left, bottom-right) of one of the input videos.
There will be multiple video sources (e.g., 4–6 videos). Each input video must be divided into the same number of segments as the final output grid.
Each segment in the final grid (e.g., top-left) must be filled only by the corresponding region from one of the input videos (e.g., top-left of Video 1, Video 2, etc.).
The selection of input video for each region should be dynamic and changeable.
The number of segments in the grid (rows × columns) should be configurable.
Ideally, I’d like to have control over which input source populates each segment dynamically.
Please suggest some starting points for this? Jack dilaura’s Texture Instancing: 3D Slicing Technique (https://www.youtube.com/watch?v=cspOWtB_ZHk). The tutorial solves for 1 single input channel. Is this a good starting point?
If you don’t have experience with glslTOPs in TD, I’d recommend looking up some of those.
The main esoteric thing going on in this project is that the value from the noiseTOP is being sampled out and cast into an integer, which is the required type for array indices.
Here’s some annotations for the GLSL code in case it’s useful. I realized I just threw it together and provided no information about it:
out vec4 fragColor;
void main()
{
// we sample the 7th input TOP which provides the grid layout and
// varying values for each grid 'segment'
vec4 mosaic = texture(sTD2DInputs[6],vUV.st);
// cast the sampled value (multiplied by five) into an integer.
// we only use the red channel of the noise texture, hence the '.r'
// in the uploaded example I forgot this part but the compiler
// silently truncated the input for us. here we're being explicit.
int source = int(mosaic.r * 5);
// we use the 'source' integer to flip between different inputs
vec4 color = texture(sTD2DInputs[source], vUV.st);
// assign the output color for the fragment shader
fragColor = TDOutputSwizzle(color);
}
@flowb i chatgpt’ed the code and got a hazzy understanding. But this helps a lot, especially mosaic.r. Will have to stare it longer for this to sync in. Cool trick, thank you much.
Getting rid of the ‘magic numbers’ and instead making the code automatically handle the number of inputs is a smart idea. Generally, I’d solve this by evaluating the number of connected operators outside of the GLSL code and passing it in by way of a ‘uniform’.
In TD uniform variables can be made available to shaders hosted in glslTOPs by way of the ‘Vectors’ page.
The number of inputs connected to any operator can be evaluated with the python expression:
# where 'me' is the node that has the expression entered into
# a parameter field
len(me.inputs)
First we put this expression into the Vectors page:
// the name of our registered uniform value needs to match the
// specification on the Vectors page
uniform int numInputs;
out vec4 fragColor;
void main()
{
// array indices in most programming languages are base 0
// so the 7th (last) input is actually 6
// subtracting 1 from the number of inputs reliably returns
// the last input. this arithmetic can be done in place.
vec4 mosaic = texture(sTD2DInputs[numInputs-1],vUV.st);
// just to dive into this a little more:
// in most (non-HDR) cases our input colors will be normalized
// ie: a value between 0-1. multiplying this by the total
// number of inputs and casting to an int automatically rounds
// the value into an appropriate number.
// we want a number that excludes the index of the grid/mosaic
// texture, so here we subtract 2
int source = int(mosaic.r * (numInputs - 2) );
vec4 color = texture(sTD2DInputs[source], vUV.st);
fragColor = TDOutputSwizzle(color);
}
edit:
if you want to forgo using the python expression for number of attached OPs, there is actually a constant TD_NUM_2D_INPUTS that TD creates for you that you can use in your code:
