I have what I think is a fairly simple question, but due to my beginner level skills im struggling to solve it myself. The situation is the following: I am getting face tracking data from a TouchDesigner MediaPipe plugin and want to cut out specific parts of my face (eyes and mouth). Im doing this by simply drawing a white rectangle at the given coords and masking out that part. Works perfectly fine. My issue is, that I need that “cut out” of the face to be a static TOP thats completely filled out. Let me illustrate:
The white rect is the face cut out (e.g. my eye, not visible here tho since I overlayed white). It moves as I move my face, since it’s just a composited version of the webcam feed as I said. I need this to be static and always filling out the TOP/screen, so if i move my face, it doesnt move “off screen” or anything. In other words, the cut out always should fill 100% of the image/TOP so I basically just want to take the cut out and make it fill the frame, while also being on it despite it moving.
I tried automatic cropping with the face tracking coordinates instead of masking (giving the crop node the x/y values of the facial feature and then cropping accordingly), and while it gives me the result I want it is messy at best. Also I cant crop just down to the alpha values like in this video, because my input is 1280x720 and its too intensive on my PC, especially since I’d have to do that for every facial feature I want.
Using the “Fit” node doesnt do anything. There has to be fairly obvious solution to this, that I am not seeing. Help would be very appreciated, as this is for an important project. ^^
welcome to the community.
There are 2 different approaches:
Cropping (maybe you can share your file and we can determine why this is a messy process)
Creating a dynamic uv map and use it in conjunction with the RemapTOP to grab the right area
For the UV Map, a component might be useful that takes the x,y coordinate of the box and a width/height parameter. Is that the data you have available?
Have you thought about how to deal with changing aspect ratios of the image or is this not an issue in this project?
If in the end you are using this texture in a 3D context, it could also make sense to right away adjust the texture coordinates of a rectangle and prepare it for rendering…
I looked into my old cropping method again and finetuned it a bit and now I’m happy with the results. Previously, I had the issue that when changing the camera position (which is inevitable since I’m still just prototyping the setup) it would mess up all the offsets and crop sizes that I need to frame the facial feature properly, due to the coordinates changing. I made this less of a headache by implementing some easier ways of adjusting the cropped cut out width and length.
Although I was able to fix it myself, I am quite interested in the UV example you mention. UVs have a habit of tripping me up, especially when they are used in purely 2D image processing. I tried looking into some examples and have a vague idea of what you mean, but I’m not sure how I’d proceed. The data I have is an x and y coordinate and the width/height is set by me. How would I go about making this dynamic UV map? If you have some tutorials or examples to point me at, I’d appreciate it! If not, it’s fine, since I’ve progressed with the project anyways. It just seems like a useful technique to know.
i’ve attached an example here of a component that hopefully explains the idea of an uv map. You could do it all in a shader, but this way might be a bit more visual.
Generally uvs are normalized, hence resolution independent, coordinates in a texture. Normalized meaning bottom/left is 0,0 and top/right is 1,1.
This can be described as a horizontal ramp in red and a vertical ramp in green. Now to create a uv map that is equal to a cutout, you’d have to calculate the bottom left starting position of the cutout: u0 = cutPosX / imgWidth and v0 = cutPosY / imgHeight. Equally, your top right endposition can be calculated like this: u1 = u0 + cutWidth / imgWidth and v1 = v0 + cutHeight / imgHeight.
Hope this helps a bit.
cheers
Markus UVer.tox (1.2 KB)
