What data does Shared Memory In TOP expect?

Hi all

I am trying to use the Shared Memory In TOP to pass an image from a Python script to TD (2021.15800).
When I run my script, nothing happens on the TD side. As you can see from the test script below I am trying to save a 256x256 image as a numpyArray. Then have a basic TD project with a Shared Mem In Top and a Null TOP.

The warning I get in TD is “Warning: Shared Memory segment named TOPShm not found”

Is it because TD expect the TOP data in a different format, or is there something about naming that I am getting wrong?

Thanks in advance,
Christian

import numpy as np 
import cv2
from multiprocessing import shared_memory

#texture = np.zeros((256, 256,3))
texture = cv2.imread('sd-input-1.png',cv2.IMREAD_COLOR)

shm = shared_memory.SharedMemory(name="TOPShm", create=True, size=texture.nbytes)
b = np.ndarray(texture.shape, dtype=texture.dtype, buffer=shm.buf)

b[:] = texture[:]

window_name = 'image'

cv2.imshow(window_name, b)

cv2.waitKey(0) 

shm.close()
shm.unlink()

There is a header you need to fill in. Check out Samples/SharedMem in your installation folder for information on that.

Thanks for the tip!
I am afraid though, that I don’t really understand how to implement this in a python script. Is that even possible?

Can I just “translate” the code to python or will I get in trouble when I try to store the header, which is now some sort of Python object, in the buffer?

Does the approach make any sense, or do I need to stay in the C++ realm somehow?

import numpy as np 
import cv2
from multiprocessing import shared_memory
import pickle


class TOP_SharedMemHeader:

    def __init__(self):
        # Begin version 1 
        # Magic number to make sure we are looking at the correct memory
        # must be set to TOP_SHM_MAGIC_NUMBER (0xe95df673)
        self.magicNumber = 0xe95df673 

        # version number of this header, must be set to TOP_SHM_VERSION_NUMBER
        self.version = 2
        
        # image width
        self.width = 256

        # image height
        self.height = 256

        # X aspect of the image
        self.aspectx = 1

        # Y aspect of the image
        self.aspecty = 1

        # Format of the image data in CPU memory (RGB, RGBA, BGR, BGRA etc.)
        self.dataFormat = 0x80E0

        # The data type of the image data in CPU memory (unsigned char, float)
        self.dataType = 0x1401

        # The desired pixel format of the texture to be created in TouchDesigner in GPU memory (RGBA8, RGBA16, RGBA32 etc.)
        self.pixelFormat = 0x8058

        # This offset (in bytes) is the diffrence between the start of this header,
        # and the start of the image data
        # The SENDER is required to set this. Unless you are doing something custom
        # you should set this to calcDataOffset();
        # If you are the RECEIVER, don't change this value.
        self.dataOffset = self.calcDataOffset()

        # End version 1 

    def	calcDataOffset(self):
        return len(pickle.dumps(self))
    
        # Both the sender and the reciever can use this to get the pointer to the actual
        # image data (as long as dataOffset is set beforehand).
    def	getImage(self):
        c = self
        c += self.dataOffset
        return c


#texture = np.zeros((256, 256,3))
texture = cv2.imread('sd-input-1.png',cv2.IMREAD_COLOR)

header = TOP_SharedMemHeader()
header_pickled = pickle.dumps(header)

shm = shared_memory.SharedMemory(name="TOPShm", create=True, size=texture.nbytes + header.dataOffset)
shm.buf[:] = header_pickled[:]

b = np.ndarray(texture.shape, dtype=texture.dtype, buffer=shm.buf[header.dataOffset:header.dataOffset+texture.nbytes])

b[:] = texture[:] #This doesn't actually work: "ValueError: memoryview assignment: lvalue and rvalue have different structures"

window_name = 'image'

cv2.imshow(window_name, b)

cv2.waitKey(0) 

shm.close()
shm.unlink()

The data needs to be carefully packed in both size in bytes of each member, and offset each member within the structure, to match the C++ data structure. You’ll need to use the ‘struct’ python module to layout the data precisely.

Hi again!

Thanks for the tip with the struct module. I’m getting somewhere I think.
It was a little hard to understand from the example, but I have assumed that I can leave the getImage() and calcDataOffset() out of the header, assuming the data is packed correctly. Does the header version play a role here?

I have tried the code below, but I still get the same warning in TD “Warning: Shared Memory segment named TOPShm not found”. Is that expected even though there is such a segment with this name? Shouldn’t I be expecting an error that it was wrongly packed or something along those lines? In other words, how can I know that I am still packing the data wrong and I don’t have a problem where TD doesn’t identify the memory segment correctly to begin with?

In any case, this is the way I am using struct.pack(), however it is not easy to find a way to check whether the packing below actually correspond to the way C++ would pack the class. Any ideas?

import numpy as np 
import cv2
from multiprocessing import shared_memory
import struct

#Open image file 
texture = cv2.imread('sd-input-1.png',cv2.IMREAD_COLOR)

#Figure out the siez 
headerSize = struct.calcsize('Iiiiffiiii')

shm = shared_memory.SharedMemory(name="TOPShm", create=True, size=texture.nbytes + headerSize)

print(headerSize)
packed_header = struct.pack('Iiiiffiiii',0xe95df673,2,256,256,1,1,0x80E0,0x1401,0x8058,headerSize)

shm.buf[:headerSize] = packed_header 

b = np.ndarray(texture.shape, dtype=texture.dtype, buffer=shm.buf[headerSize : headerSize+texture.nbytes])

b[:] = texture[:]

window_name = 'image'

cv2.imshow(window_name, b)

cv2.waitKey(0) 

shm.close()
shm.unlink()


The code now definitely stores something that looks like meaningful data in shared memory. This can be checked with the code below (even though it is just going to spew out a long list of byte values)

from multiprocessing import shared_memory

existing_shm = shared_memory.SharedMemory(name='TOPShm')
c = existing_shm.buf.tolist()

print(c)

existing_shm.close()

Right, sorry. There is quite a bit more to it than that. To allow the shared memory segments to get resized there is a secondary hidden shared memory segment used to let the apps know what the ‘correct’ one to look at it. It’s a bit more complicated and you’ll need to look at the code in that samples folder to see what the UT_SharedMem and UT_Mutex classes are doing.
I do think it’ll be easier for you to do this your own way with reading/writing the shared memory yourself in Python, and then uploading the data to the TOPs in a Script TOP.

1 Like

Wow… I can’t believe I had my mind set squarely on using the Shared Mem In TOP that this didn’t occur to me. Thanks for helping out!

It took 20 minutes to rewrite it for a script TOP. Below is the code if anyone needs it. The shared memory functionality was included in Python in 3.8, so you need a recent version of TD to run this out of the box.

With this simple implementation you should probably run the sender script first, then open TD. Afterwards, close TD first and then the sender script to avoid errors and making sure that all memory is released.

Sender script:

import numpy as np 
import cv2
from multiprocessing import shared_memory

#Open image file 
texture = cv2.imread('sd-input-1.png',cv2.IMREAD_COLOR)

shm = shared_memory.SharedMemory(name="TOPShm", create=True, size=texture.nbytes)
print(str(texture.shape) + " " + str(texture.dtype))

b = np.ndarray(texture.shape, dtype=texture.dtype, buffer=shm.buf)

b[:] = texture[:]

window_name = 'image'

cv2.imshow(window_name, b)

cv2.waitKey(0) 

shm.close()
shm.unlink()

Receiving Script TOP:

# me - this DAT
# scriptOp - the OP which is cooking
import numpy as np 
import cv2
from multiprocessing import shared_memory

import numpy
  
# press 'Setup Parameters' in the OP to call this function to re-create the parameters.
def onSetupParameters(scriptOp):
	return

# called whenever custom pulse parameter is pushed
def onPulse(par):
	return


def onCook(scriptOp):
	
	existing_shm = shared_memory.SharedMemory(name='TOPShm')
	img_array = np.ndarray((256,256,3), dtype=np.uint8, buffer=existing_shm.buf) #Use the shape and dtype that the sender script prints here. 
	
	scriptOp.copyNumpyArray(img_array)
	
	existing_shm.close()
	
	return