Hello,
I am not entirely sure about this memory of mine, but I feel like there was some mention about “git compatible toe / tox files” long time ago. Therefore I just wanted to ask if this is planned for the future? Thanks 
2 Likes
Yes, this is something we’re looking at, but I don’t have a timeline or plans to share at this point. If you’ve got any thoughts or requests on this, feel free to post them here and we’ll keep them in mind as we move forward.
Great, thank you for info.
I have one little idea that I would like to mention. I was thinking it might be cool if tracked toe / tox files would show git related status information on operators and parameters. If operator or parameter was added or modified since last commit, some small indicator might be nice.
I am attaching visualization in vscode as inspiration (see yellow M and green A).
I think this would be nice addition to bold non-default parameter RFE that I tried to mention at the bottom of this post.
1 Like
Direct Git integration is on our wishlist, but might not necessarily be in an initial release of the file format change since it requires some UI work that is dependent on other projects.
1 Like
Sure, thank you very much for info 
I don’t know if it is even possible but
it is pretty neat that with files that are text based witch makes it possible to look at git comits and diffs
fx KiCad
yes, the idea is to have a text based mode to allow for merging and diffs. It’s somewhat possible now if you try the toecollapse and toeexpand utilities that are in the bin folder, but the idea is to make it a little cleaner and more seamless than those utilities do right now.
1 Like
I have been thinking about binary data stored in toe / tox files (like for example locked TOP operator storing texture in a scene file). Please may I ask how are you planning to save such data when it comes to text based files (if there is already some plan on what to do with such data)?
(I am just curious whether they will be visible as jibberish in diffs, or is there some trick to hide them somehow?)
The common way for storing binary data in text is using Base64 encoding ( Base64 - Wikipedia ). It is essentially jibberish, but it uses a subset of characters so that it displays in text editors fairly cleanly and doesn’t interfere with formatting.
This gets used whenever data has to remain binary like if you lock TOPs, use the virtual file system, or have file protection (encryption) enabled .
1 Like
Would it not also be possible to simply save stored data in it’s own binary file? This way we could easily exchange stored and can maybe update stored data without putting dirt on the logic-part of a file.
Another question would be how granular you would do the setup? Having every node as its own file would pollute everything, but the whole project as one bix ol textfile is also not ideal. I feel that every COMP should simply have its own file and maybe subdirectories when there are nested comps.
I feel it would be good to only store non default parameter values (even though when they change later on that might get tricky).
Also, this could simply be like a word file, which in itself is simply an archive that can be unpacked with 7ZIP or similiar (which I find kinda funny tbh).
1 Like
We’re looking at multiple ways to split the files right now and likely it will be configurable to some extent. Using a single file will definitely still be an option as it is now, but to reduce merge conflicts and make reading the files a little easier there will likely be one or more modes such as file-per-node, or file-per-component.
Splitting the data portions out on their own is an interesting idea that we haven’t explored much yet beyond the existing use of external files with dats and file in nodes. Its definitely something that would be compatible with the system though, so it could be an option at some point.
Our current system only saves non-default parameter values and that will still be the way going forward. We have internal systems already in place for when we change a parameters default (it doesn’t happen often, but there have definitely been cases).
Our current toe/tox files and the binary versions going forward already do use a compression system to reduce file size. They are not directly readable by 7zip because of header data and a few other differences, but we intend to continue with tools like the existing toeexpand/collapse utilities to let them be parsed outside of touch.
2 Likes
This is super interesting, thanks for sharing all info.
I mentioned this at the last summit, Flash did the same thing at some point. They had a binary FLA old way or the newer xml based way .xfl format . Flash files had graphics and other binary assets and I think what they would do is not store them in the xml but reference them and store them in a lib folder next to the xml file.
Here is a basic Flash file with a banana bitmap asset in the library.
test.zip (269.0 KB)
2 Likes