An interactive guided tour through matrix transformation pipelines

Helpful resource. I was struggling with matrices & transformations recently Thank you for sharing this Lucas

No problem! I keep bumping into things in the matrix world I thought I had learned for goodâ€¦ and end up spending a couple hours just to realize I got the order wrong, or forgot to inverse this or that - figured a visual aide would be a useful way to cut through some of that

Thank you very much. Did help me a lot!!

A question: I am trying to use the world-to-view transformation based on a camSchnappr camera. This does not work (the camera is not correctly aligned with the world coordinate space) I think because this camera uses a view with a Custom Projection Matrix, which is generated during the camSchnappr calibration process.

I have no idea how to solve this. Would be very helpful if I could get a starting point about what to do in this situation. Thank youi.

@knut I have not messed much with camSchnappr, so canâ€™t advise much on the specific workflow for that module, but generally speaking, there is usually either a view matrix, a projection matrix, or a matrix representing the â€śsum totalâ€ť of both.

I was just poking around to see, and the first custom matrix I saw was this one here:

You can use the excellent tdu.Matrix python class in Touch to create a matrix, filled with the values from these tables. You can either do this during initialization, or manually for each of the 16 indicies after.

the tdu.Matrix object is the same type of object returned when you get the default view or projection matrix from cameras and such.

the tricky thing is to just make sure the operations that youâ€™re trying to do are undone, in the proper order, this part is usually what gets me stuck for a bit sometimes.

I think youâ€™d want to build a matrix from the custom matrix data, and then do a matrix.invert(). to get itâ€™s inverse, then you can use it to perform the reverse operation.

Hope this helps!

@lucasm thanks a lot for your reponse. I guess I will go that path: Get the Projection matrix, calc the inverse and then do the transformationâ€¦although I have to admit: I dont really know what I am doing here.

I need probably a little bit of reading about the Camera projection matrix and what it does. Could not find very much in the TD docs so far.

I totally get that, it can seem a bit confusing even after understanding the steps.

The way I conceptualize the projection matrix that helps me differentiate it from other world space and view space matricies is that itâ€™s purpose is to take a 3d point in an â€śinfiniteâ€ť space, such as world space, or view space , and put it into a space that is â€śfiniteâ€ť. clip, then NDC space.

There isnâ€™t technically a limit to those spaces in any real way, but the purpose of those spaces is for culling, so things are remapped into a -1:1 cube where things outside of those ranges are clipped/removed by the renderer. So I like to think of it as if there is sometimes.

Also, worth mentioning here that when you do matrix math in GLSL for example, and many other places you need to do the perspective divide step (divide .xyz by .w) to get coords from clip space, to proper NDC space.

When you use TDâ€™s matrix library, it handles this for you automatically. Iâ€™ve gotten so used to this that when I try porting my code to GLSL I sometimes forget and get stuck debugging because I didnâ€™t do perspective divide.

Scratchapixel.com has some really helpful material on these matricies:

Link1

Link2

Iâ€™ve also written a pretty comprehensive blog post on one of my websites about the process (*it started out as a big cheat sheet for me * )

Goes really into depth about how the projection matrices are constructed from simple numbers and camera parameters we already understand using systems in Touch.

Thank you for all your thoughts on these things. Also the links are very helpful for me. And its also clear to me what to do next: I read through the whole stuff but then I have to implement several examples, because thatâ€™s for me the only way to learn and to really check, if I understood everything.

TD and in addition your tool here is very important because itâ€™s always easy to see what I am doing.

And its now the right time for such an exercice: We have a nearly complete shut down over here in Germany, everybody stays at home if possible. So its time to read and learn.

Thanks Lucas! As always, amazing explanations about the fundamental topics.

I must add that to start diving into this topic I watched first 3 lessons from this Intro to Computer Grapchics course (Math, 2D transformations and 3D Transformations) and then looked at your resource to learn how to do it all in TD.

And now I can compute the screen space coordinates for my Kinect joints when they are looked at from any arbitrary camera position, awesome!