MathCHOP operation not correct when in "shortened Number" display mode

Hello,

not sure what to call this, but it seems to me, that when adding small integers to very big integers
(big enough to be displayed as 3.1536e+07 instead of 31536000 the results are wrong.
Like if there was a rounding error. However i dont think that 31536000 is a craaazy high number and i would expect to be able to add 1 to it ; )

Is this a bug or am I missing some crucial setting?

WIN10, 64bit, Build 2021.13640

Best
Stefan

Our CHOP channel values all use 32-bit floats so you are indeed running into a floating point error where it can’t fully represent the least significant digits.

32-bit floating point has 24-bit precision, so 2^24 = 1.6777216e+7 maximum for whole numbers before precision needs to be sacrificed. You’re over that, but not by much, so it would only be missing that extra bit of precision to represent odd numbers (and fractions), but can still fully represent whole even numbers without precision loss, hence the rounding.

1 Like

Hey Eric,
thank you for your answer. It makes total sense, yet it is highly unexpected.
The naive assumption would be that TouchDesigner is good with Math (but not that good ; )
Without a good reason most users that come from a design background like myself
will not expect that behaviour at all. It is mega-counter-intuitive
and I think should be addressed more clearly (by e.g. a warning Pop-Up when one enters the range
where rounding errors will occur - or better - a behind the scene fix for it).
Just my 50p ; )

Best
Stefan