asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.
I tested the following program. But this program didn’t work in TouchDesigner. It seems that the event loop does not return control to TD until the event loop has completed. That’s why I created TDAsyncIO.tox for using asyncio module in TouchDesigner.
TDAsyncIO.tox is a Component for using asyncio module in TouchDesigner without blocking the TD’s main thread by running the event loop only once after every frame.
[Active] - the Event Loop can run asynchronous tasks while ‘Active’ is enabled.
[Cancel All Tasks] - Cancel all tasks you created.
Code example
AsyncIO COMP is set to ‘OP Global Shortcut’ parameter to TDAsyncIO so it can be reached anywhere by op.TDAsyncIO. It can call op.AsyncIO.Run() and op.AsyncIO.Cancel() by Extensions.
Ha what an interesting hack, clever! I did not even know you could run the event loop step by step.
I hope to be able to play with this soonish, thanks for sharing.
tested a bit and so far it’s pretty awesome for doing simple network requests. Thanks @sndmtk !
I hope to test with more advanced asyncio stuff soon, curious how it will hold up when receiving lots of traffic.
import asyncio
import aiohttp
async def test():
async with aiohttp.ClientSession() as session:
async with session.get('http://python.org') as response:
print("Status:", response.status)
print("Content-type:", response.headers['content-type'])
html = await response.text()
print("Body:", html[:1000], "...")
# Run coroutine
coroutines = [test()]
op.TDAsyncIO.Run(coroutines)
import asyncio
import requests
async def post(url):
# Clear textDAT
op('text1').clear()
# Get the current event loop
loop = asyncio.get_event_loop()
# Arrange for func to be called in the specified executor.
r = await loop.run_in_executor(None, requests.post, url)
# Set the result to textDAT
op('text1').text = r.text
coroutines = [post('https://derivative.ca/')]
op.TDAsyncIO.Run(coroutines)
This looks really amazing, and sounds like testing is going well. Threaded operations are not usually my domain, but if there is specific TouchDesigner Python support needed to support this project definitely let me know and I will do my best to make it happen!
hey @sndmtk thanks again for this great hack.
Here a tip for future TD asyncio users:
I did a lot more testing, to see if I could replace some of my existing external asyncio Python applications running high-speed messaging. I kept running into a glass ceiling where the amount of incoming messages/second I could process in TouchDesigner asyncio code was much lower than in my external asyncio apps, and my TD asyncio client would crash and disconnect after a while. After lots of head scratching it became clear to me that the incoming socket message buffers where overflowing, as they ‘only’ get emptied every 1/60 second (once per frame) in this TD asyncio version, whereas a loop in an external asyncio program can be started much more often. Once I upped the incoming buffer size, so more messages could accumulate during each frame, I was able to match the amount of messages/second TD could process to that of my external Python apps, and the websocket connection now also stays stable. Fantastic!
I loaded the test .toe file that you have and the ASyncIO component seems fine.
But when I tried to import it into my own project I get this error.
File “/project1/TDAsyncIO/execute1”, line 21, in onFrameEnd
td.tdAttributeError: ‘td.baseCOMP’ object has no attribute ‘Update’ Context:/TDAsyncIO
Any idea why the component is reacting differently? I imported it into the root of the project the same as in the test toe file. Is there some setting somewhere that is missing?
EDIT: I think the component had to be re-initialized. I eventually opened the edit component menu itself and clicked init on the Extension just in case and that seems to have solved this.
Also in the test file when you click cancel all it throws an error
Traceback (most recent call last):
File “/project1/TDAsyncIO/chopexec1”, line 12, in onOffToOn
File “/project1/TDAsyncIO/TDAsyncIO”, line 61, in Cancel
AttributeError: type object ‘_asyncio.Task’ has no attribute ‘all_tasks’
This all_tasks issue seems to be resolved by changing the Cancel code from
asyncio.Task.all_tasks(self.loop)
to
asyncio.all_tasks(self.loop)
was this aspect of asyncio updated within the last year?
Great component @sndmtk, thank you very much for making it!
I might have one question. Please do you think it might be possible to use await asyncio.sleep(x) also within TouchDesigner running without Realtime on (meaning that specified x could be much faster or slower than actual real-time x)?
Based on current code I believe using await asyncio.sleep(x) now sleeps exactly for x, no matter how fast is TouchDesigner running (with Realtime flag turned off). I initially thought it would be cool to use this for triggering some complex sequence of events in time, but then I realized it won’t follow TouchDesigner’s rate (when it is not running in real-time). I am not sure if there is some way of telling asyncio’s event loop how much time should have passed between frames…
@monty_python From my experience the asyncio(sleep) command measures its own time from when the command was called. So it’s independent of how often the asyncio update is called (its time is not based on how often the update is called). Although its output will only be rendered (afaik) when its timer has passed and the next asyncio update comes in.
It would seem to me in your case you would want to use timers from TD (like a Timer CHOP) which follow the non-realtime mode,and have them send a callback to your asyncio coroutine when the time is up.
Aha! Thank you very much @nettoyeur, using Timer CHOP for this is a great idea. I have just tried it and it seems to work nicely. I am attaching simple example in case someone would like to take a look.
import asyncio
async def sleepTD(time):
print('starting sleep...')
timer = op('timer1')
timer.par.length = time
timer.par.start.pulse()
while True:
await asyncio.sleep(0) # avoid blocking the event loop, continue on the next frame
if timer['done'] == 1:
print('done sleep')
return
async def main():
for x in range(10):
await sleepTD(3)
coroutines = [main()]
op.TDAsyncIO.Run(coroutines)
If you open up textport window and jump into the perform mode, it swooshes through sleeps (as realtime & v-sync are disabled and TD is running at very high frame rate) - just as it should. Thanks once again for help! non-realtime_async.1.toe (6.6 KB)