Optimizations for Join CHOP?


I have a scenario where I have many geo COMPs with a variety of internal layouts. I am generating paths to a sub chop in each of these objects, via an op FIND dat, and then feeding this into a join CHOP to collect xyz positions from each into one big chop for instancing.

@ 150 selected chops, the join chop’s performance degrades quite considerably coming in at around 16 ms per cook. I would have just assumed this is necessary overhead, except I am able to cut the cook time down by a little over 16x by building my own custom “join chop” using a script CHOP:

I should note as well, that I am providing the paths using a dat’s .col() method. Not sure if this is relevant, but my script CHOP approach uses it as well.

While it’s not a biggie to write a few lines of code for me, this optimization might be applicable to other nodes as well like in script DAT’s which are considerably more expensive than their compiled node equivalents… so I figured I’d mention it here in case there’s something to be gained across the board!

Here’s my test file:
joinChopOptimizationRequest.1.toe (8.2 KB)

Ok just did a similar test using dats, the results are strikingly similar, which makes me think even more that there is something going on. both script approaches have very similar cook times while both node approaches have similarly high times:

I’ve updated the below file to contain examples for both.
joinChopOptimizationRequest.2.toe (9.1 KB)

I think the conversion of op(‘paths’).col(0) is whats slowing it down so much. I usually convert my oppaths table to text (default convert DAT) and use op(“convert1”).text in the join CHOP. Not sure why col(0) is fast if used in the script solutions

Ya you’re right - this is definitely a .col(0) performance issue. If I place the .col() expression in an eval dat, it comes in again right at ~16 ms. Very strange indeed how accessing this in python gets around it.

I discovered something else:

In the eval dat, I can optimize it greatly by using a str map()

op('paths1').col(0) (~16 ms)
list(map(str,op('paths1').col(0))) (~.15 ms)

Would be interesting to see what insight the devs have here - doing the last example above is very easy and yields the expected gains and results, and can be used directly as expression to a merge or select operator - so a very reasonable solution as well.