How to filter messages from TCP IP?

I have some data coming in on TCP/IP that looks kind of like this:

message: aaa, bytes: 255 255 255 255
message: aaa, bytes: 255 255 255 255
message: aaa, bytes: 255 255 255 255
message: bbb, bytes: 122 124 068 014 022 124 124 122 120
message: aaa, bytes: 255 255 255 255
message: aaa, bytes: 255 255 255 255
message: aaa, bytes: 255 255 255 255
message: bbb, bytes: 224 227 169 115 112 121 114 102 132

I get 3 lines of repeating “a” messages and then a line of the data I need in the “b” messages.

I am using Clamp output so I’m seeing one line at at time. I am using Convert to turn this into a table but because the incoming bytes are different lengths my Table keeps jumping from 4 columns to 9 columns and when I try and pull values from the cells above 4 it jumps from value to red X when the table drops to 4 columns and my Constant Chops (where I am feeding the values) drops their value. I tried Lag but it seems the error stops that from working.

How can I filter out the A messages and push only the B messages into the table, holding onto the previous B message values until the next B message comes in?

Or is there a better or different way to handle this?

I can do this with python outside of TD but I am wondering how I would assemble this to work with TD nodes, limiting the python code needed?

Thanks!

Handling TCP pretty much requires doing it in the Callback DAT in Python.
TCP/IP is just a stream of bytes, with no notion of message start/ends. You will get random groups of bytes and you need to wait until you get a completed message as defined by your protocol, not by when they arrive. That is, on one callback you may get 50 bytes, and then the next you get 40, but the message was only 60 bytes long. Your protocol needs to define how you know where those boundaries are. Usually a protocol has a ‘header’, that tells you how many bytes long the message will be, and then you now know to accumulate the bytes until you get at least that amount of bytes (and save any extra since that’s the start of a new message).

The messages always come in as A or B.

A messages are always 4 bytes
B messages are always 55 bytes.

I just need to kill the A messages… They don’t have any useful data. I only need the B messages.

I apologize if this is obvious but I’m still learning… but I’m not sure how to kill A while still getting B. If I could prevent A from reaching the Output I’d be good to go.

Pushing data around with Python in nodes is still something I don’t have a good grasp on.

This is basically analog clock circuitry signal that is being digitized and sent down TCP and then turned back into analog clock circuitry signal. It’s incredibly rudimentary.

Thanks!

That’s luck though, there is no guarantee that TCP won’t merge/split those messages on you. Only if you are using UDP can you know that a message is a message.
You’re setting yourself up for headaches by assuming the messages will always show up like this.

I’ve already got the incoming data formatted how I want it.

I have no idea how to move forward and filter and push out only the data I need.

I’m not really looking for help on how to use TCP. I’ve already got that but I have no idea how to push only the data I want from the tcpip1 node to another so I can use it.

I’m not sure how else to explain what I need… I’m already catching the data just fine - now I need to filter it…

I’ll correct myself to say, if your messages are using newlines at the end of them then the DAT will separate out the messages into different rows dependably.

Yes it does and its awesome. I’ve got nice data coming in to the TCP-IP node and I can send it through a Convert to turn it into an -almost- useable table but the data keeps scrolling/changing as the messages keep coming in. I can’t figure out how to ditch the short messages and keep the long messages and/or keep the data locked into a set of specific cells so I can drive a CHOP without it dropping values because the cell goes away when the length changes…

https://youtu.be/lf7b-3Bm-T0

as malcolm said, you can inspect incoming messages in the callback of the tcpip DAT, and choose there what should happen with each received message, using Python.

So to check if a message is longer than let’s say 25 characters, an if so, write that message to another DAT:


def onReceive(dat, rowIndex, message, bytes, peer):
	if len(message) > 25:
		op("fifo1").appendRow(message)

example with tcpip server & client
filter_tcpip_message_based_on_lenght.tox (9.1 KB)

1 Like

The “op(‘fifo1’).appendRow(message)” and the fifo DAT and how to do that are the answer I’ve been looking for. That’s the part I’ve been missing and trying to figure out.

My if-statement setup is a touch different but its the same basic concept.

I was trying to keep the unwanted data out of the TCPIP DAT going into the Convert DAT but this makes so much more sense.

Thank you so much!! Success!!

As you already know Python, it’s handy to click the blue/yellow question mark topleft in each node’s parameter window.
This will drop you right in the Python class documentation for that node, and tell you all possible methods, such as how modify content in a table DAT, like adding a row:

Unfortunately I am not smart enough to be able to read that and figure out what its actually saying or how to apply it to what I am trying to do. There’s nothing in there that would have led me to the concept of using the fifo DAT node to solve this issue. I didn’t even know the fifo DAT existed until you mentioned it.

The problem is I just don’t know how I should be doing things in Touch. Like right now I am trying to figure out how to get a modulo result from a Math node… and I’m struggling. Maybe that’s not the node to do that but I don’t know that until I fail trying for a few hours and then show up here to ask dumb questions.

expression CHOP might be useful for modulo:

Also see Help->Operator Snippets for many practical examples for most nodes

And this is why I am here in the Beginner forum. Never quite sure what’s got a node and what’s going to need a bit of code. I’ll check out Expression, thanks!!