Using asyncio module in TouchDesigner

asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.

I tested the following program. But this program didn’t work in TouchDesigner. It seems that the event loop does not return control to TD until the event loop has completed. That’s why I created TDAsyncIO.tox for using asyncio module in TouchDesigner.

import asyncio

async def test():
    await asyncio.sleep(3)
    print('hello world!')

asyncio.run(test())

TDAsyncIO.tox

TDAsyncIO.tox is a Component for using asyncio module in TouchDesigner without blocking the TD’s main thread by running the event loop only once after every frame.

This is a sample project for using TDAsyncIO.tox. Please download it from here.
https://github.com/sndmtk/TouchDesigner-asyncio

Parameters

[Active] - the Event Loop can run asynchronous tasks while ‘Active’ is enabled.

[Cancel All Tasks] - Cancel all tasks you created.

Code example

AsyncIO COMP is set to ‘OP Global Shortcut’ parameter to TDAsyncIO so it can be reached anywhere by op.TDAsyncIO. It can call op.AsyncIO.Run() and op.AsyncIO.Cancel() by Extensions.

import asyncio

async def test():
    await asyncio.sleep(3)
    print('hello world')

# Run coroutine
coroutines = [test()]
op.TDAsyncIO.Run(coroutines)

# Cancel all tasks
op.TDAsyncIO.Cancel()

Now you can easily use asyncio! For more information about coroutines, please refer to Python Coroutines and Tasks.

References

See also these helpful links.

AsyncIO for the working PyGame programmer (part I)

Is it possible to run only a single step of the asyncio event loop

6 Likes

I always thought derivative would need to add support for asyncio. If this works this is really really nice ! Thank you

1 Like

Ha what an interesting hack, clever! I did not even know you could run the event loop step by step.
I hope to be able to play with this soonish, thanks for sharing.

@MarkusHeckmann
do you think this could also be a solution to the “execute on next timeslice” issue we talked about ?

tested a bit and so far it’s pretty awesome for doing simple network requests. Thanks @sndmtk !
I hope to test with more advanced asyncio stuff soon, curious how it will hold up when receiving lots of traffic.

import asyncio
import aiohttp


async def test():
    async with aiohttp.ClientSession() as session:
        async with session.get('http://python.org') as response:

            print("Status:", response.status)
            print("Content-type:", response.headers['content-type'])

            html = await response.text()
            print("Body:", html[:1000], "...")
			
         
# Run coroutine
coroutines = [test()]
op.TDAsyncIO.Run(coroutines)
1 Like

@nettoyeur did you try calling functions (op(),…) that usually trigger a “not thread safe” warning ?

I assume/hope those will work just fine when calling them from a coroutine

@Achim You can call TD module inside a coroutine :+1:

like this

import asyncio
import requests

async def post(url):
	# Clear textDAT
	op('text1').clear()

	# Get the current event loop
	loop = asyncio.get_event_loop()

	# Arrange for func to be called in the specified executor. 
	r = await loop.run_in_executor(None, requests.post, url)

	# Set the result to textDAT
	op('text1').text = r.text

coroutines = [post('https://derivative.ca/')]
op.TDAsyncIO.Run(coroutines)
1 Like

Yes @Achim I tested that and it works fine as asyncio coroutines are all in the same thread

This looks really amazing, and sounds like testing is going well. Threaded operations are not usually my domain, but if there is specific TouchDesigner Python support needed to support this project definitely let me know and I will do my best to make it happen!

Great work, @sndmtk

2 Likes

hey @sndmtk thanks again for this great hack.
Here a tip for future TD asyncio users:

I did a lot more testing, to see if I could replace some of my existing external asyncio Python applications running high-speed messaging. I kept running into a glass ceiling where the amount of incoming messages/second I could process in TouchDesigner asyncio code was much lower than in my external asyncio apps, and my TD asyncio client would crash and disconnect after a while. After lots of head scratching it became clear to me that the incoming socket message buffers where overflowing, as they ‘only’ get emptied every 1/60 second (once per frame) in this TD asyncio version, whereas a loop in an external asyncio program can be started much more often. Once I upped the incoming buffer size, so more messages could accumulate during each frame, I was able to match the amount of messages/second TD could process to that of my external Python apps, and the websocket connection now also stays stable. Fantastic!

3 Likes