Camera Raw/Lightroom: Enhanced Profile Issue: CSV looktable files with large dimensions are not able to be embedded in presets

  • 2
  • Problem
  • Updated 6 months ago
  • (Edited)
First, I love having the ability to embed custom HueSatMap style looktables inside the enhanced presets in ACR 10.3. I think this will be a gamechanger and some of the results I'm getting in testing are jaw-dropping.

However, I'm finding that when I try to create a new enhanced profile in ACR, it will not embed looktables which have large dimensions (for example, 90, 30, 30). I have no trouble with smaller tables (for instance, 36, 16, 16). 

It's really important to be able to have a table this size because the changes are mapping to linear Prophoto RGB space. So the coordinate system for value is very dense close to white, and then becomes considerably less dense. So in a small table size, there is very little control over the outer limits.

(And Adobe themselves uses tables with 90, 30, 30 dimension in many of their own DCP files emulating camera styles). 

Is there a way that this can be fixed, so that we can embed larger table sizes inside ACR? 

OR - if you could you give us the tool to encode our looktables directly rather than having to do it through ACR? It looks pretty close to Ascii85? 
Photo of Nathan Johnson

Nathan Johnson

  • 20 Posts
  • 22 Reply Likes

Posted 6 months ago

  • 2
Photo of Simon Chen

Simon Chen, Principal Computer Scientist

  • 1490 Posts
  • 471 Reply Likes
It is intentional to restrict the size of the LUT (I think it is restricted to 32x32x32) to reduce the rendering performance impact and the memory and disk space required to store/load/sync settings with such large LUT.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3857 Posts
  • 1014 Reply Likes
Oops, Adobe Standard is a DCP.  But Adobe Color is an enhanced profile, and it's look table is stored by reference in the .xmp sidecar:

   <crs:Look>
    <rdf:Description
     crs:Name="Adobe Color"
     crs:Amount="1.000000"
     crs:UUID="B952C231111CD8E0ECCF14B86BAA7077"
     crs:SupportsAmount="false"
     crs:SupportsMonochrome="false"
     crs:SupportsOutputReferred="false">
    <crs:Group>
     <rdf:Alt>
      <rdf:li xml:lang="x-default">Profiles</rdf:li>
     </rdf:Alt>
    </crs:Group>
    <crs:Parameters>
     <rdf:Description
      crs:Version="10.3"
      crs:ProcessVersion="10.0"
      crs:ConvertToGrayscale="False"
      crs:CameraProfile="Adobe Standard"
      crs:LookTable="E1095149FDB39D7A057BAB208837E2E1">
Photo of Simon Chen

Simon Chen, Principal Computer Scientist

  • 1490 Posts
  • 471 Reply Likes
That only applies to the default LUTs built by Adobe. Those default (or well known) LUTs are embedded in Adobe's code and shipped with every copy of Lightroom. In the case that you want to build and use your own custom LUTs in your custom profile, Lightroom does't know them and the full custom LUT table would need to be embedded in the XMP. This is the same reason that Lightroom does not need to embedded the Adobe Standard.dcp, because it is installed with every copy of Lightroom. That is not the case with custom dcp or custom LUT.
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 3857 Posts
  • 1014 Reply Likes
Make sense, thanks much.
Photo of Nathan Johnson

Nathan Johnson

  • 20 Posts
  • 22 Reply Likes
Simon - the documentation mentions that the max RGB LUT is 32x32x32... but does not mention the max dimensions for dng_hue_sat_map"  look tables. 

Whatever the max dimension is, it seems to be smaller than the RGB LUT... 

I just tried a dng_hue_sat_map csv at 32x32x32 and it would NOT parse it.  :(

The documentation does mention that "the Adobe Raw profiles use 36 hues, 16 saturations, and 16 values." 

In my testing, this is fine for make broad changes to an image. 

But to me, the beauty of the hue_sat_map is making very precise, finely tuned adjustments that wouldn't be possible any other way (and in dimensions that I get to chose)... and 36x16x16 gives me LESS precision than an RGB LUT...


Photo of Max Wendt

Max Wendt, Employee

  • 13 Posts
  • 18 Reply Likes
You have a bit more flexibility with look tables than you do RGB tables. The maximum size cannot exceed 36 * 32 * 16, but you can have more that 36 hue divs by reducing the number of sat or val divs.

The maximum values for each are:
hues: 360
sats: 256
vals: 256

You can arrange them however you like, as long as the product of the three of them doesn't exceed (36 * 32 * 16).
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 3857 Posts
  • 1014 Reply Likes
"could you give us the tool to encode our looktables directly rather than having to do it through ACR? It looks pretty close to Ascii85?"

See this related feature request for the RGB LUTs in enhanced profiles: https://feedback.photoshop.com/photoshop_family/topics/camera-raw-and-lightroom-document-the-format-...

The ASCII85 encoding is easy to figure out. The stumbling block seems to be the compression algorithm -- I tried all the common ones without success.
Photo of Max Wendt

Max Wendt, Employee

  • 13 Posts
  • 18 Reply Likes
Note that you don't need to use linear ProPhoto - you can specify the encoding when you make your looktable. Using sRGB still provides you with plenty of power; I made all of the initial profiles using sRGB with divs of 36, 16, and 8. 
Photo of Nathan Johnson

Nathan Johnson

  • 20 Posts
  • 21 Reply Likes
Max - can you speak to the advantages / disadvantages of sRGB vs linear ProPhoto for encoding in the dng_hue_sat_map? As far as I can tell, tables with the sRGB encoding would still it's entries somewhat clustered towards white, no?
Photo of Max Wendt

Max Wendt, Employee

  • 13 Posts
  • 18 Reply Likes
No - the sRGB encoding is non-linear, and the distributions of the values are more even. Making a few quick "toy" tables will help you get a feel for how the divisions map; you can target the highlights, shadows, and midtones in a few different table to get a sense of what values the divisions affect.
Photo of Nathan Johnson

Nathan Johnson

  • 20 Posts
  • 21 Reply Likes
Max - thanks so much for your response on this. Yes, I've made a number of "toy" tables. 

As a developer though, I want to get more than just a "sense" of what values each division affects (as well as how much each values is affected by the scale factor). I'm interested in getting mathematical precision.

Just as an illustration, I have this toy table:  https://www.dropbox.com/s/g507vbaeqbxfsww/playground.png?dl=0  that is equally divided between 36 hues, 10 sat division, and 10 val divisions. In theory, I should be able to precisely target any pixel on this table.  



Let say for illustration that I wanted to completely desaturate the entire 9th "column" of saturation levels. So I build this 36x10x10 table (sRGB encoded), and set the sat scale to 0.0 for all entries where sat is 9/10 (link here: https://www.dropbox.com/s/3vkiypp8rikahkw/srgb-sat-9-10.csv?dl=0 ) Here's what results:



Not exactly what I expected...

Now if I make sure my playground image is in the ProPhoto color space https://www.dropbox.com/s/u86s9w6mjfet1uz/prophoto-playground.tif?dl=0 , things get way closer to what I would expect, but is still off in that it seems to be effecting brighter vals differently when it should only be effecting saturation (also, the experiment doesn't give me a lot of confidence that I can expect the nearly the same results from the looktable on a JPEG  vs linear RAW file..)




I'm also having a hard time understanding the math behind the scale factor... for instance, if I set a scale factor on all saturation to 0.5 ( https://www.dropbox.com/s/l05owgkf35jlw0u/srgb-sat-halved.csv?dl=0  ), I would expect HSB measurements to show half the saturation they were before. But this is not the case. For instance, saturation that was previously at 100% is now at 11-40% saturation (weirdly depending on brightness - with the brightest values being severely desaturated). 



Very strange.

Any input you could provide on how to understand mathematically where the divisions are being made, and how the scale factors are being applied would be greatly appreciated! 

-Nathan