Augmented Reality and Touch Designer

Dear all,
here are some screen shots of my Augmented Reality authoring with Touch project.

interface overview01800px.jpg

Above is the overall interface. You drag the marker icons onto your workspace then use them to parent the geometry you want to use in the scene. The extra null is to move the object up a bit so it is not place inside the marker but on top of it.

imagesurface_600px.jpg

here the real world image is used as both a texture and surface in real time. This kind of stuff you can’t do with most AR authoring tools out there without a lot of hand coding.

It works in a similar way to Jörg’s PrevizART application. There are a couple of extra tricks used to read the ARToolkit’s camera calibration file to adjust for lens distortion etc. and reconcile the openGL matrix representation of the camera with Touch’s camera parameters.

If anyone wants to try it out, you need the pro version of Touch077 in order to access shared memory.

um… video … once I get around to joining vimeo …

Rod.

super nifty!

Hi Rod,

It’s cool! I have tried ARToolkit, but don’t understand how you make it works in Touch. Did you use OSC to connect to ARToolkit result? Or you write the tracking code by yourself using scripts in Touch?

Kam

Yep we cooked up our own program.

first it looks in the camera frame for black squares.

when i finds one, it then looks inside it to see if there’s a glyph it’s been taught to recognize.

if it sees one, then it checks which way up it is to get z rotation
then checks how big the black square is to get Z translation
checks where it is in the frame to get x and y translation
then checks if the square is more trapeziod than square to get the x and y rotations

then it puts the original frame into shared memory and at the same time puts the tracking data for that frame into shared memory as well.

touch designer then brings in the data using shared memory TOPs (pro version only, sorry)
a TOPtoCHOP operator and a bunch of chops then sort out the data to send to the translation fields of the geometry.

I’m cooking up a nice GUI to make this a drag and drop operation and also a control panel with which to launch the various camera calibration and marker training etc. I also hope to cook up a control panel to actually make and print markers ready for use (maybe even an automatic pattern data file generator).

If you have the pro version of Touch077, then I can let you try it out. … but give me a month to tidy it up a bit first.

Rod.

Rod,
I see. I don’t have the Pro version, but thanks anyway!
Kam

i’d love to see how you handled the matrix transforms… i’m working on a project that requires some projection translation functions and am having a tough time figuring out how to do it in touch!

Short answer is I got someone smarter to handle it. :slight_smile:

the registration is tidied up by reading the ARToolkits camera parameters file and using the lens distorion info to bend a mesh around before sticking the ‘real’ image onto it as a texture.

the matrix is translated into rotations and translations in degrees in the tracking program before feeding it into touch. owever, if you look in any null or geometry component, you can see a pre-transform tab which has a ‘matrix chop’ input’ that lets you use a chop to feed in the 4 x 4 matrix. ARToolkit only needs 4 x 3 but same thing. you should be able to feed it to a chp then to the pretransform chop field.

rod.

looks great! would love to give it spin in pro

Hi,
I’ve put it on the backburner for a bit while trying to get my PhD work under control. However, I plan to get some interface improvements into it soon and use the pallette so you can just select a AR project and drop it ready to start work.

I’ve also cooked up some things to link with the marker-training and camera callibration programs so they get launched from a panel inside touch.

problem is, a student helped me with it and now I have to reverse-engineer his project to see whay it doesn’t quite work yet!

rod.

We are looking into to creating Augmented reality projects.
pros- clients are really excited
cons- none of us knows programming

I came across touchdesigner searching for augmented reality for non-programmers but reading this forum it looks like I need the pro version.

Is TouchDesigner Pro capable to create augmented reality or do I required different softwares? And can I get them publish over the net for the my clients?

thanks
Wolfman

Hello wolfman,

The shared memory operators that are required to interface with the ARToolkit are now also available in FTE Commercial, which is available for $599. You will need to use ARToolkit however, which is another software package.

TouchDesigner is a full application that does not run in web browsers, however Touch files are very small which makes them easy to share over the web.

rodberry is the AR guru on this forum, he can provide you with much more information on using AR with TouchDesigner.

Also talk to “dajoegl”. He developed a AR environment in TouchDesigner called Previzart:

viewtopic.php?f=19&t=456

cheers
markus

HI all,

I’m moving back to australia in March and will reopen this project definitely soon after that.

If anyone’s keen to try it before then, send me a message. As ben noted, now the $599 FTE commercial will work happily with this. I don’t have an installer for the tracking program but it will run from an executable. there are a few .dlls that must be either in the program’s directory or in the windows directory.

I’ll have to recompile it for windows 7 64 bit soon! I’m not a programmer so there will only be changes to the actual code when one is kind enough to help me - or I have enough money to compel one to help. :slight_smile:

Rod.

I should say that the big disadvantage of my approach is that it requires the end user to have a FTE commercial or Pro license. If touch ever gets to a stage where it can generate an executable that you can hand a client, then it will become viable to do this stuff for other people.

I have to check with the ARToolkit people if it’s okay to release this as a closed source binary for free without paying for the artoolkit (it’s a special build of ARToolkit 4, not the GPL open source one).

anyway, I’d love to get some people playing with it and giving feedback. I quite like it. I will get off my bum and tidy up the interface to make it nicer to work with.

Rod.

Hi Rod,
Your AR setup is really cool. Did you ever finish cleaning up the interface? I have a commercial version of touch and would love to play around with it if possible.

Also, has anyone else done any AR projects and have any tips?