SOLVED: Multiprocessing python module not working

[sensormusik]

Hello,

I am trying to use the multiprocess module. According to these posts, it should work:

SOLVED: Popup-Error before first Python Script Execution

UPDATE, revisiting this in 2024: SOLVED: Multiprocessing python module not working - #5 by JetXS Revisiting this in 2023. Note that using the multiprocessing library, you can use the multiprocessing.set_executable() method and pass the python exec of TouchDesigner. Use app.binFolder + '/python.exe' This should fix the popup. In our future release you can also pass Path Objects. Original issue, if it’s still occurring, could try to call multiprocessing.set_executable() first… Best, Michel …

I made the following example:

# me - this DAT
# 
# channel - the Channel object which has changed
# sampleIndex - the index of the changed sample
# val - the numeric value of the changed sample
# prev - the previous sample value
# 
# Make sure the corresponding toggle is enabled in the CHOP Execute DAT.
import multiprocessing as mp


def process(conn):
	conn.send('Hello.')
	conn.close()
	
def onOffToOn(channel, sampleIndex, val, prev):
	mp.set_executable(app.binFolder + '/python')
	conn_parent, conn_child = mp.Pipe()
	p = mp.Process(target=process, args=(conn_child,))
	p.start()
	print(conn_parent.recv())
	p.join() #it does not matter wether I block the thread or not
	return

def whileOn(channel, sampleIndex, val, prev):
	return

def onOnToOff(channel, sampleIndex, val, prev):
	return

def whileOff(channel, sampleIndex, val, prev):
	return

def onValueChange(channel, sampleIndex, val, prev):
	return
	

When the script gets executed, TouchDesigner crashes (becomes unresponsive, I have do force quit).
I am using a Macbook Pro M1

[JetXS]

Hey @sensormusik

Apologies about the delay.

I am not around a Mac at the moment and this will take a minute before I can test this and debug.

Best,
Michel

[JetXS]

Hello @sensormusik

I stand corrected. The statement you linked is misleading, apologies.

Back when this was tested, the test setup was actually weak and wrongfully made me think that everything was working as expected. The process was actually silently dying during execution.

1. Your problem of hang is easy to explain and understand, the recv method is blocking. To prevent this method from being blocking, you should call the poll() method on the connection object.

2. However, even then, the method you pass to your Pipe will fail in the subprocess. This is part of TouchDesigner and exist only in this instance of TouchDesigner. This is caused by the current architecture of TouchDesigner where the TouchDesigner process itself is the python interpreter, but where most of the python representation of the TouchDesigner objects, including extension and DATs as module, is generated at runtime.

3. There is currently no way that I know of to run the multiprocess library in TouchDesigner. Instead, you could rely on the Threading library which will create threads and stay in the TouchDesigner context, or design a standalone Python app / process that you would start in parallel to TouchDesigner, using a .bat or .sh file.

I hope this covers most of it,

Best,
Michel