Build 3D cube from GAN video

I would like to create a cube based on GAN video frames.
The xy dimension = GAN video (4000px*4000px), z dimension = frames in a loop, each layer +1 frame

Similar to this example: https://bigumigu.com/wp-content/uploads/2019/01/refik_anadol_nasa_jpl_data_-sculpture_bigumigu_1.jpg

Can someone help me with this or point me to a tutorial?

Thank you so much!

With the newer version of TouchDesigner it’s possible to instance directly from 2D Texture arrays. However, a 4000x4000x1 image is 16 million particles, which is way too many. If there were a z dimension it would be even worse. If you want to instance plainly from a texture 2D you’d have to pick something like 200x200x200 = 8 million objects. And if those objects were spheres rather than 4-point rectangles, it probably wouldn’t work well either because spheres are far more points. The reason you’d want spheres is because you can get shadows easily with them. If you wanted to get good shadows with points or rectangles, you’d have to do the shadow pass from a light manually, have the rectangles face the light, and probably do some other tricks.

Another issue is that the example you shared is a sparse cube. Not every “voxel” in the cube is occupied. This is a good thing, because it means you don’t have to use 2D texture arrays with “wasted” pixels being used for instancing. Instead you can use a compute shader to fill up a 2D texture (not array!) with ~8 million points, one input image at a time, taking care to not copy in blank pixels. I have a YouTube demo on a compute shader here TouchDesigner GLSL Compute Particles Tutorial - YouTube It could serve as a good starting point for taking advantage of the sparseness of the cube.

1 Like