32 bit mode, Color Picker value snapping/precision behaviour.

  • 1
  • Question
  • Updated 2 months ago
Also mentioned here -
but I was told that's for answers from users and I'm unlikely to get a definitive answer there.

I've a linear RGB 32 bit per channel image and want to use as exact as possible colour value, e.g. R=1.224747, G=..., B=....  An example use case would be to apply a simple linear RGB rescaling to the whole image so that some colour-picked pixel RGB value can be re-scaled exactly on to some known linear RGB, fill a 'multiply' layer with this colour, and observe the target pixel(s) now exactly match this 'some known linear RGB'.

The above number is almost exactly representable in IEEE754 floating point:  https://www.h-schmidt.net/FloatConverter/IEEE754.html

However, when I type this into the Color Picker, the text entry boxes truncate the value to "1.2247 ".  Then, when I 'OK' the Color Picker and then re-open it with this recently set colour, it now shows a value of 1.2184, which is quite a long way past the nearest representable floating point value of the originally entered value.

So, I'd like to understand what it's doing to my desired values to make them land quite a way off from what the nearest representable float value should be.  Does anyone know?

Also, is there any way to get it to use exact IEEE754 float values and not quantise/snap them to some other value or is there a more precise way to perform the above separate RGB linear rescaling/multiply example scenario so that maximum precision can be maintained throughout the multiply, leaving any colour profile quantising (or whatever else might be snapping intermediate values) 'til the end?

Photo of dan h

dan h

  • 4 Posts
  • 1 Reply Like

Posted 2 months ago

  • 1

Be the first to post a reply!