If you try to load a CSV that has, for example, semicolons as separators, the File In DAT will give the “File not found” error. I can understand how trying to load said non-standard CSV in a Table DAT should maybe throw an error if there’s no commas in the file, but I would assume the File In DAT should retain the ability to open anything anywhere for more obscure usage.
In this case the CSVs in question are exported from GrandMA2 software (latest version 3.7.0.5) and is a rather niche function so will probably never be changed by MA Lighting. Hopefully this bug can be fixed in TouchDesigner so we can then use a convert DAT to take the raw text and turn it into a table by splitting at semicolons.
also as stated elsewhere, regular / otherwise valid CSVs are getting their last row chopped off unless there’s a empty line at the end of the file… but that’s probably a different bug
Just to clarify, if I change the file extension to anything else that it wouldn’t otherwise recognize (like scsv for the imaginary “semicolon separated values” filetype I just invented) then it can open the file fine as just plain text, but as soon as I change the file extension back to “.csv” it suddenly “can’t find the file”.
This is irregardless of the settings in the common tab, which I’m not sure what they’re supposed to affect anyways (“Edit/View Extension”? “Custom Extension”?)
I would agree that since it is not a legal CSV file, it should not be loaded into TouchDesigner as a Table, but I was hoping that it would fail gracefully (IE just load it as raw text like any other weird file extension it didn’t recognize) at which point we could do what we need with it after the fact.
There is precedent for the semicolon, especially in non-US locales where the comma is a decimal separator, and thus the semicolon is used instead:
Which considering MA Lighting is a German Company, might explain why their software spits out CSVs in this way. Either way, it would be very useful to not have to rename all of the exported files from MA2 since there are a lot of them and sometimes other programs look for said CSVs in this same location.
Yeah, .csv has lots of confusion with it since it wasn’t standardized until RFC 4180, which is what we follow. The goal being to be forward compatible with this standard. I would contend that MA needs to be the one to get onto the standard here.
I’ll think more about the error case suggestion. Loading the raw binary data from a file into a Text DAT in the case where it can’t be recognized doesn’t seem like the right move offhand though…
Touchdesigner is useful when it can tie into various external inputs, and as developers, we don’t always have control over what we get as input. So ,the more adaptable TD is the more useful it is.
To this end, I would say standards be damned and allow the file extension to be disregarded, and provide manual overrides for the parsing.
It would be great if, by default, the parsing type can be Auto and based on the extension (for backward compatibility), but can also be set to TAB, Comma, or whatever weird delimiter we input.
Also, a few other settings for how to handle quotation marks, and to ignore the first N number of lines (so header data can be ignored).
I’m dealing with a case where we’re getting weather data from a science station, and it would be much easier to adapt TD than go through a potentially year’s long process of asking a bureaucracy to alter its data format.
My workaround for now is writing a python script to manually parse the incoming files.