3D Network Visualization Tool

Hey all!

This was a quick half a day project that started off as a tangent, and ended up here, more or less a functional TOX component with some interesting options broken out to custom parms.

https://www.facebook.com/enviraldesign/videos/1436804489759676/


I have a few items on my todo list for this, but wanted to share, get feedback and see what everyone thinks before diving too much deeper.

My wishlist:
A) more optimized code for caching / gathering network data (crashes / hangs on some very big projects, or can take up to an hour.)
B) draw more things, like node wires, or flash containers when they cook.

Problem with B) is that there isn’t a way I know of, to efficiently gather cook info about everything in a network, with out killing performance on medium - large scenes.

the OpFind DAT works technically, but it’s very slow on big networks and kind of defeats the purpose of seeing things fire off organically in real time. (think neurons in a brain sort of visual)

The other thing I wanted to do with this, was build a visualization over TIME, where each incremental TOE save represented a frame and then render out an animation of the growth / evolution of a network over time…

It would be very interesting to see a graph like this shrink, and grow, and evolve over the course of a project’s lifespan. :slight_smile:

My biggest road block there is how to automate going through thousands of .toe files programatically. Command line? from another TOE file that’s always opeN? Looking for suggestions!!

Lucas
network_3d_visualizer.tox (18.9 KB)

1 Like

Nice visualization tool. My first thought when seeing one of my networks would be that it would be helpful to see wires connecting nodes that are linked by expressions.

It could be an interesting way to see the multiple places that a parameter is linked to, or chains of linked items

:mrgreen: :mrgreen: :mrgreen:

Hi Lucas, I am very interested in visualizing the touchdesigner system I have created
https://github.com/GregDav/Machines/tree/master/born/Orage
Have you pursued your idea of a visualization through TIME in such a way?
Thank you kindly

No but I still want to - I might circle back around to this topic soon, but at present I’ve done no more work on this tox. will certainly post here if / when I do, maybe I can find some time in the next couple weeks.

The big issue is automation of opening/modifying many toe files via windows or command or batch.
essentially, need the ability to load a tox and run a script inside that tox at minimum.

@raganmd if I remember correctly from the last summit, didn’t you talk about a way of doing things with a toe, via environment variables, or something of that nature?


My structure only displays comps, would you remember what would be the cause?

@lucasm - I didn’t talk about it at the summit, but did write a little tut about this kind of process:

The idea here is really how to pass an environment variable to your toe file, then use that variable in your network at start.

When it comes to iterating through lots of toe files, in the past what I’ve done is use a python wrapper to automate that process - look at a folder, open a project, perform some operation, close that project, move to the next. Automating what happens once it’s open is a little harder… one way (though maybe a little shady) would be to alter a tox that’s in
C:\Program Files\Derivative\TouchDesigner099\Config\System
that you know is going to init on start for any network. You could sneak some python executable code in there for onStart or onCreate that might help you get the data you need out. I don’t know that’s a sanctioned approach but would be a tricky way to get there.

Thinking out loud, I wonder if rather than saving out a whole frame, you might instead save out a point file. Those are tab or comma separated, and might be an interesting way to pack data into a file that’s easy to instance from / sequence over time.

There might be some direction to take from gource - I’ve used that before to visualize contributions to a project over the course of its development:
https://gource.io/

1 Like

Wow that’s absolutely clever about C:\Program Files\Derivative\TouchDesigner099\Config\System
, never occurred to me that Touch is injecting certain things into every file. Going to give that a whirl and see how it goes.

Agreed, about a better format for packing info about a network. Thanks for the tips!

1 Like

Ok quick update - I managed to actually “render” out a sequence of dat’s to disk totally automated, for toe’s saved from years ago using @raganmd’s trick - I’ve started re working my original code to be a bit more robust, and performant on very large networks, and will either update this thread or make a new asset post with the network viz animation tool.

More on that soon!

4 Likes

Throw it up in COMMUNITY section when you do, excited about an update to this!

@ben certainly will do!

I am taking a slightly different direction, took a look at Gource, and turns out it accepts a custom log format that was pretty trivial to reproduce by iterating through each toe, and comparing it’s entire structure to the structure of the previous one.

So what I’ve got right now outputs a log file that Gource can accept, and another part of the toe actually launches gorce + ffmpeg to automatically render out a video of the animation.

Users should be able to point this tool at a folder of thousands or tens of tousands of toes, and let it do it’s thing and should be able to generate this sort of thing almost entirely hands off.

Here’s a short sample I built today from a series of 20 toe files:

Bit more cleanup to do, but I think I’ll be able to release it quite soon.

2 Likes

@lucasm- that’s rad! Glad gource was worth giving some attention.

It was! I had seen it before but never knew the name. thanks for the pointer there as well.

Ok I can’t help but post a few more screens before I post this tool up tomorrow - this is what a very large TD network looks like in gource:
image
image
image

since I loaded a single “frame” that was massive already, it took quite a while to “settle” as it’s a dynamic system. I think that an animation going from small to large will probably look better and less chaotic since it has time to spread itself out as it grows.

Still! so crazy to see things this way.

3 Likes

Ok I’ve posted the official asset to a new post, check it out, and please share some cool stuff that you make!

2 Likes

This is brilliant!

Are you moved on to using Gource now rather than the first version?

Ya I went with Gource since it’s beautiful, and already exists etc. It builds it’s own graph based on relational data and nixes the spatial data of nodes for it’s own spatial arrangement.

I may one day return to the original idea - but I was having trouble coming up with an aesthetic and clear way to represent an entire touch network in 3d, because of how much stuff overlaps and intersects when you use the raw node coordinates and depth data.

Node coordinates are all relative to their parent, and even when their parent coordinates are accounted for (think world transform vs local transform) they still overlap, because of how we design networks and lay nodes out.

The idea of showing the 3d structure somehow is still a really interesting though, may come back when I have more time. :slight_smile:

2 Likes

Thanks for your work lucas. I have pushed your original work to visualize the progress of my own project in time through lots of versions. Thought I would share the result:

https://www.youtube.com/watch?v=UVyyc7EMgOU

Cheers
alivemachine

2 Likes

This is rad! awesome to see the addition of the streaming data shown, also the subtle lighting on the statues at the end is very nice too :slight_smile:

@Monomyth very cool, thanks for sharing