CPlusPlusTOP: rendering video frames using GL

Beginners
1

I had this code (github.com/remap/YoutubeTOP/blo … .cpp#L1101) working some time ago, where I render arbitrary (A)RGB buffer (uint8_t*) using OpenGL and it was working fine.
Now, (using macOS as target platform) most of this code does not compile as there’s new “#include <OpenGL/gl3.h>” header, which renders many gl functions that I use as undefined.
Moreover, I initialize my TOP with info.executeMode = TOP_ExecuteMode::OpenGL_FBO; and would like to use OpenGL FBO (I think, it’ll be faster?).
So far, that’s what I tried and it doesn’t work:

  • create texture (called once):
glGenTextures(1, (GLuint*)&texture_);
glBindTexture(GL_TEXTURE_2D, texture_);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width_, height_, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
  • initialize framebuffer with texture (called once):
glFramebufferTexture2D(GL_FRAMEBUFFER,
                           GL_COLOR_ATTACHMENT0,
                           GL_TEXTURE_2D,
                           texture_,
                           0);
glGenRenderbuffers(1, &depthBuffer_);
glBindRenderbuffer(GL_RENDERBUFFER, depthBuffer_);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT, 1024, 768);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthBuffer_);
GLenum DrawBuffers[1] = {GL_COLOR_ATTACHMENT0};
glDrawBuffers(1, DrawBuffers);
  • inside execute(), render into texture (called on every cook):
context->beginGLCommands();

glViewport(0, 0, width_, height_);
glClearColor(0.0, 1.0, 0.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT);
glBindTexture(GL_TEXTURE_2D, texture_);
// TODO: convert ARGB to RGBA
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, width_, height_, GL_RGBA, GL_UNSIGNED_BYTE, buffer_);

context->endGLCommands();

I don’t work with OpenGL much except when I need to render a random video on an arbitrary texture, so I’m clearly missing something crucial here. Any ideas are welcome!
Thanks

2

That will only upload the texture to the GPU, you then need to do something with it, such as draw a quad with it onto the FBO.

If you are just looking to upload an image from the CPU to the GPU though, you should use the CPUMemoryTOP example instead. That way you don’t need to deal with GL at all.

3

Hm. I never thought about it… it makes sense. Apparently I was stuck with the way my old code worked.
But isn’t “glTexSubImage2D” supposed to draw into the texture, thus rendering the image?

4

Right but that’s a texture you’ve created, it’s not the texture the TOP is going to use as it’s output. That texture is bound to the FBO which TD sets up for you.

5

Ohhh… interesting. Is there a way I can write an image to this texture?
Or I shouldn’t bother and there’s no really performance benefit of not using CPUmem?

6

In this case using the CPU mem will be the higher performance route to go.

7

Ok thanks! I’ll re-write it to use cpu memory.
And the reason for that is that texture is already prepared by TouchDesigner and will just copy memory into it under the hood?

8

Right, and it’ll do it in the faster way possible, and it avoids TD having to protect GL from things that occuring in the plugin

9

Thanks for the explanation.
It’s quite expensive as I observe CPUMemoryTOP with 1280x720 frame size taking ~15ms to cook (using macOS system) :frowning: . I suppose, it happens due to iterating over every pixel in the buffer, right?

10

Hmm no, it should just be doing a texture upload operation, just as you were in your code. It uses PBOs though instead of system memory.
If your code does nothing, does the cook time go to near 0?

11

Yes. Sorry, I meant that it seems to be CPU-costly to iterate over every pixel. I’m playing with sample CPUMemoryTOP code. If I change output size for 1280x720 it eats up ~9ms in cooking.

bool
CPUMemoryTOP::getOutputFormat(TOP_OutputFormat* format)
{
	// In this function we could assign variable values to 'format' to specify
	// the pixel format/resolution etc that we want to output to.
	// If we did that, we'd want to return true to tell the TOP to use the settings we've
	// specified.
	// In this example we'll return false and use the TOP's settings
    format->width = 1280;
    format->height = 720;
    return true;
}
12

Right but why iterate over every pixel? Can you post the rest of your code?