Lightroom 5: ICC Table Profiles clipped shadows under OSX

  • 12
  • Problem
  • Updated 3 years ago
  • Acknowledged
  • (Edited)
Hi, I've just found a really bad issue occurring in Lr 5 (but also in all other Lightroom versions) under Mac OSX 10.9 with a calibrated monitor: dark shadows (from a value of 20 to 0) are all clipped (pure black with no detail and no textures) while the histogram remains ok, indicating NO clipped shadows at all. This issue afflics also ACR.

Photoshop for now is the only software under MAC that reproduces dark shadows correctly: Library Module shows a bit darker and shifted shadows than Ps but acceptable, Develop Module is really bad showing brutally clipped shadows (but you work in the Develop Module right?!).

The same problem occurred also in OSX 10.8 but it was related only to LUT profiles, creating a Matrix based profile problems were solved.

Now the issue occurs with both Matrix and LUT profiles, v2 and v4. There's no apparent way to make Lr working right.

Under Windows no problems at all: Bridge, Photoshop, ACR, Lr (Library Module and Develop Module) show the same correct NOT clipped shadows.

I tested 8 different Mac running 10.9 with different GPU, different monitors, different profiling Softwares (Color Eyes Display Pro, Eizo Color Navigator, BasICC Color, i1 Profiler). Same results.
I tried to change the gamma value (2.2, sRGB, L*) problems remain. I tried to change ICC version (v2, v4) problems reamain. I tried to change profile type (LUT, MATRIX) problems remain.

How can a photographer work professionally on RAW images if shadows are bad reproduced?

Why Photoshop can reproduce shadows correctly while Lr isn't able to do that?

Why this happens only on a Mac enviroment?

Is Lr based on ColorSync (that can't handle profiles correctly) while Ps isn't (because it can handle and it has no problem)?

Please Adobe, FIX IT for all professional photographers, we can’t use Lr for serious works under Mac.

Max Ramuschi
Adobe Certified Expert

p.s.: Added a 100% Crop screenshots that shows the problem, some photos are even worse...

Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
  • frustrated

Posted 6 years ago

  • 12
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Chris,

I appreciate you have removed my second post because it was referring to this issue.

The problem is: we are asking for support, and no one seems picking up.
What does "official representative" exactly mean? Because if you are the SPOC of this issue, at least a simple "we are investigating, we will let you know soon" would be great.

Of course, I appreciate any kind of support you will be able to provide.

Thank you
Andrea
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 846 Reply Likes
I don't work on LightRoom, I just asked a few questions to clarify the nature of the problem.
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Thank you for your reply, Chris.

Do you get to know someone working on Lightroom, or in the Adobe development team?

I think this is something to have some Lr Engineer aware of.

Thanks again
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Any chance to have at least an answer by Adobe? It seems they love the approach "best answer is silence".

Never seen such support, frankly speaking. Really frustrating.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Engineers are aware of this since Lightroom 5.2. The issue is easly reproducible, but nothing has been done.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
I think this is a major issue though few people noticed it.

But can you imagine a landscape photographer who takes a lot of nightscapes with a great amount of shadows and bare visible details? 50% of the photo will change from Library to Develop to Ps. Not a consistent and professional workflow at all.
Photo of Hans van Eijsden

Hans van Eijsden

  • 32 Posts
  • 4 Reply Likes
Today Mac OS X Yosemite has been released. And the problems are still here: clipped shadows. For commercial images I'm using Capture One now, because my clients have high demands and shadow areas must be absolutely perfect (well controlled).
Switching on the soft proofing is too much hassle for me, so Capture One does the job with consistent shadows.
I hope Lightroom 5.7 or 6 will fix the issue, so I can also use Lightroom for my commercial work again.
Photo of Kelly C.

Kelly C., Senior Lightroom QE

  • 86 Posts
  • 39 Reply Likes
I apologize for the slow response to this thread. We had an open bug on this, but it was closed as a third party compatibility issue that was reportedly fixed with the release of 10.9.2. I will reopen the bug and try to get some answers for those of you still experiencing the problem.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Hi kelly castro, I updated the bugs I noticed on Yosemite and Mavericks, so please read my last post. Thanks for your interest.
Photo of Wojciech Brzozowski

Wojciech Brzozowski

  • 9 Posts
  • 2 Reply Likes
macOS Sierra, Lightroom 6.7 and bug still exist in Library module and Develop (if Graphics Processor turned OFF)
Photo of Hans van Eijsden

Hans van Eijsden

  • 32 Posts
  • 4 Reply Likes
Thank you Kelly.
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Good to hear it is open again. It was never fixed but for a while appeared to be an apple bug but it is now getting clear that it is a Lightroom bug that only gets triggered in Macericks and Yosemite. Apple fixed it in their own apps but Lightroom never got it right. Every other app except Safari is doing shadows right now.
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Thank you Kelly, I really appreciate your help.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Just to summarize and update the bugs list with Adobe products on OSX (Mavericks 10.9 and Yosemite 10.10):

- With Matrix Based Profiles: Lightroom shadows are bad (darker, clipped with no details in Develop Module), Camera RAW and OSX (Preview) are ok.

- With LUT Based Profiles (which are a reference point for professional users with pro displays): Lightroom shadows are worse (even darker and clipped in both Library and Develop), Camera RAW and OSX are bad (yes you've read right, the bad and clipped shadows issue occurs in Camera RAW too with LUT profiles and even OSX didn't fix the support for them).

- On OSX, Photoshop is always right, perfect with both profiles.

- On Windows, Lightroom and Photoshop are right, perfect with both profiles.

Please focus on the LUT Profiles support as these are profiles that professionists normally use. (Eizo pro monitors perform way better with these kind of profiles for example).

Tip: to see clearly the bug, calibrate your display with gamma 2.2 (which is the standard).

You can download this image as a test image here: https://www.dropbox.com/s/e6sldv4l4s5..., but you can also use every image with a big dark shadows zone.

Massimiliano Ramuschi
Adobe Certified Expert Photoshop
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
Massimiliano, Jao, and others experiencing this problem: Can you please send me your ICC display profiles? You can contact me at madmanchan@gmail.com. I'm assuming these are small enough to send by email, but you can also use a standard file transfer service if not.

Yes, I can use my own matrix-based and LUT-based ICC profiles for development and testing, but it's always better to have the customers' profiles as well for our further verification.

Thanks,
Eric
Photo of Hans van Eijsden

Hans van Eijsden

  • 32 Posts
  • 4 Reply Likes
Done! Thanks! :)
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Same here. Glad to see you looking into it. It really doesn't matter at all what kind of profile you use. Every single Mavericks and Yosemite machine I tested has this problem regardless of the calibration device/settings used and this even happens with the built-in Apple profiles.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Done.
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 846 Reply Likes
If LR is using the GPU for color management, then you probably also need information on the GPU in the system -- because we have seen precision problems with some GPU models and driver versions.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Does Lr use GPU? I found the issue on every recent mac with every GPU (Intel HD 4000, Iris, Geforce, Radeon). Though I don't think it's a GPU related issue, I sent my GPU info.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
I noticed precision problems related to certain GPUs in Photoshop color management. But I also noticed (in my case and few other) that if you set GPU to Basic in the performance tab will fix every issue. Can you confirm that?
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 846 Reply Likes
Yes, that's the GPU precision problem I was referring to. Some claim to be accurate to a certain number of bits, but aren't really. We're still trying to figure out which is which and disable them for tasks that really need the stated precision.
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Chris, this is a problem in Lightroom. Photoshop is basically correct in all cases no matter how you set its preferences safe perhaps a small 1-bit error that you should only be able to spot in a smooth color ramp. The problem in Lightroom Develop is MUCH larger than a small precision error. There is no performance tab in Lightroom and Lightroom doesn't use the GPU in the same way as Photoshop anyway.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
The Intel HD 4000 has some problems: magenta shift in dark tones and bad transitions (banding) in shadows. Some Geforce have same issue even on Pc and, in my experience, Ati Fire Pro are ok...
Photo of Chris Cox

Chris Cox

  • 20280 Posts
  • 846 Reply Likes
Jao - I know, but Lightroom has been trying to use the GPU to accelerate display, and I suspected that Eric might need a bit more info.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
To be clear, the issue being experienced here in Lr is unrelated to the GPU.

The reason I asked some of you (offline) GPU-related questions for Ps is that the color values you will see in Ps can (often does) depend on whether you have GPU support turned on in Ps's Preferences panel.

(If you want to see what I mean, take Jao's test image and convert it to a color space that has a linear gamma, e.g., create a customized version of Adobe RGB, sRGB, or ProPhoto RGB with gamma of 1.0 -- you can use Ps's Color Settings panel to do this -- and then view that in Ps with the GPU option enabled vs with the GPU option disabled.)
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
We are aware of that kind of problem, but in my experience setting GPU to basic in performace tab, will solve every issue.
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Well it cannot solve the Lightroom issue. That is completely impossible. There is no GPU setting in Lightroom and Photoshop preferences have no effect on Lightroom.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
no no, it's a completely different thing. Maybe Eric asked us if we were using GPU in Ps, because he wanted to be sure that we were comparing the same image in Lightroom and in a non GPU-issue-affected Photoshop .
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
I am getting my signals crossed here. Didn't see who was asking questions and what the thread was in the email notifications. I thought you wanted me to try to change the setting in Photoshop and see if it fixed Lightroom. That didn't make any sense. Will try what Eric is asking.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Eric was saying another thing about PS. There's nothing we can do to solve Lr issue. We must wait.
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Don't really see a noticeable difference between GPU basic or GPU advanced in Photoshop using a linear ppRGB version of my little test image. I don't even see a difference when I turn of the GPU acceleration completely. They both look correct and hugely different from the affected Develop.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
Jao, when you converted to Linear ProPhoto and switched GPU mode on/off, did you make sure to quit/restart Ps completely in between tests?
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Yes, I was thinking that might matter so I quit PS completely and restarted between. No difference that I could tell. Maybe in the very very dark patches (1,2, and 3 in sRGB 8-bit numbers), but those are very hard to make out to start with. Any effect is far smaller than what I see in Lightroom where every dark patch upto and including 15 is strongly affected when switching between Library and Develop with the same linear file. Library looks like Photoshop and looks like what the file looks like in Lightroom in both modules on Mountain Lion.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
Interesting.  (Doesn't match my experience re: GPU.  I'll have to follow up on that later, and separately.  Sorry for the distraction ...)  In the meantime, I've identified the smaller issue regarding table-based display profiles in the Camera Raw plug-in (ACR) and this should be fixed in the final version of ACR 8.7.  This issue should have occurred on both Mac and Windows, by the way.

I'm still looking into the (much more noticeable) issue regarding Lr's Develop module, which happens on the Mac and for both gamma-based and table-based ICC display profiles, and hope to consult with the team's engineers in the next day or so to determine the root cause.  (I have a pretty good idea, but I'd rather know for sure rather than just guess ...)

I just wanted to let you know that this issue is being investigated, and to thank the folks here for all of the details, which have been very helpful.
(Edited)
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Let me know if I can help more with that. Glad that you are investigating these issues. Very much appreciated as it has bothered me for a long time now.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Eric, you're right: the issue you identified is affecting ACR on Windows also, when using a table-based profile. I'm not a good observer...
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Jao, please try to create a LUT based or Table based Profile for your display and check if the library module is darker than PS.
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 211 Posts
  • 70 Reply Likes
Massimiliano. I did the LUT based profile and indeed the dark patches in Photoshop are slightly darker in "Advanced" mode vs "Basic". Not much but still.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
I noticed some magenta shift and bad transitions between contiguous tones... But it depends on the GPU and drivers. you can observe the issue on both mac and win. For me it's not a major issue because it's easily fixable.
Photo of Kelly C.

Kelly C., Senior Lightroom QE

  • 86 Posts
  • 39 Reply Likes
Official Response
An update: Eric has identified the source of the clipping issue and we are currently testing a solution. More info as it becomes available. Thanks for your patience.
Photo of Hans van Eijsden

Hans van Eijsden

  • 32 Posts
  • 4 Reply Likes
Yay! The best news of the day! The future is smiling to me again - with LR! :)
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Good news. Keep us informed!
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Very good news :-) awaiting for a fix. Thank you very much !!!
Photo of Jao van de Lagemaat

Jao van de Lagemaat

  • 203 Posts
  • 66 Reply Likes
Fantastic. Can't wait!
Photo of Adrian Delsi

Adrian Delsi

  • 1 Post
  • 0 Reply Likes
Great! I'm having this issue with a series of black on black images. Need a fix soon!
Photo of Wojciech Brzozowski

Wojciech Brzozowski

  • 9 Posts
  • 2 Reply Likes
macOS Sierra, Lightroom 6.7 and bug exist in Library module and Develop (if Graphics Processor turned OFF)
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
The really important thing (for me at least) is that Lightroom will become able to handle table-based profiles in every module. I personally don't use matrix based profiles at all, as they are not too precise.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
FYI, Massimiliano, you may be confusing two different concepts: the color transform and the gamma encoding (1D functions). The color transform may be matrix-based, or table-based. The gamma encoding can be polynomial-based, or table-based.

Both ICC profiles that you sent me actually use matrices for the color transform, so these are both considered matrix-based profiles.

However, one of the profiles uses a polynomial gamma encoding, whereas the other profile uses a table for the gamma encoding.

So, the underlying issue here actually is not matrix-based vs table-based for the color transform itself, but rather polynomial-based vs table-based for the gamma encoding.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Ok understood, I call that profile table based or LUT based because Eizo Color Navigator calls it this way. The options you can choose in that software are 'LUT Based' or 'Gamma Curve', to be honest I've never looked into the profile itself. So the problem is actually about the gamma encoding right? It's a gamma issue with perfect matching colors indeed...
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
Yes, exactly Massimiliano.
Photo of k_gs_201305

k_gs_201305

  • 2 Posts
  • 0 Reply Likes
I to have this issue. I don ́t occur when viewing TIFFs or JPS ́s only dark CR2 and DNG ́s.

It ́s very annoying.
Photo of Andrea Riela

Andrea Riela

  • 13 Posts
  • 0 Reply Likes
Kelly, Eric, any update on this?

More than 10 days from last update, can you please share the status?

Thank you very much for your effort

Andrea
Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 15633 Posts
  • 2368 Reply Likes
We're hoping to sneak this into the next update. Won't be too much longer. Stay tuned.
Photo of Coleman

Coleman

  • 4 Posts
  • 0 Reply Likes
I'm having the same issue. Is ir related to ICC versions? (ver. 2 vs. ver. 4). My problem is particularly noticeable when comparing the same image in PS CS6 (or CC 2014) and LR 5. Wondering if LR5 is still operating on ICC ver. 2? Asking because when I switch monitor profiles to those generated via Apple profiler (rather than those generated by ColorMunki), crushed shadows dissipate and image looks the same in both programs. W/ ColorMunki profile, LR 5 crushes shadows andimage is very different than in PS.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
Hi Cole, 

No, I do not believe this is related to ICC profile versions.  This issue has to do with specific details of how the gamma encoding curve is represented in the profile.  (Multiple ICC profile versions, including 2.x and 4.x, support the affected gamma curve representations; this is why I do not think the version is relevant here.)  We should have a fix soon.
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Eric is right: EVERY kind of profile is affected. I tried all combinations, v2, v4, table, matrix, etc. Shadows are bad in all cases. And it occurs viewing all kind of images: tiff, jpg, dng, etc.
(Edited)
Photo of Coleman

Coleman

  • 4 Posts
  • 0 Reply Likes
OK, I'll go w/ that and hope it gets fixed soon. In the meantime, I'll go w/ PS version for printing. Not t be a pest...but if ICC versions are not the culprit, I can't figure out why the visual difference goes away when I revert to older monitor profiles? Here is an older discussion that initially lead me in that direction:

https://forums.adobe.com/message/4440968
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 631 Posts
  • 133 Reply Likes
It is possible that the older profiles use a different type of gamma curve representation.  I believe it is the internals (how the profile correction data is represented) that is at the heart of the issue, rather than things like the profile version or other basic metadata.
Photo of Coleman

Coleman

  • 4 Posts
  • 0 Reply Likes
Thanks for the explanation.
Photo of Coleman

Coleman

  • 4 Posts
  • 0 Reply Likes
Hi Eric,

for whatever it's worth, I just reprofiled my monitor w/ Colormunki, using version 2 ICC and LR 5 now matches w/ PS 6. Don't know if this helps or hurts but figured I'd post it. Cheers, Cole

Photo of SatisfiedCustomer

SatisfiedCustomer

  • 3 Posts
  • 1 Reply Like
This is another 10 days later after a fix was promised. Any news?
Photo of Hans van Eijsden

Hans van Eijsden

  • 32 Posts
  • 4 Reply Likes
Same question. Refreshing the Adobe Labs page automatically twice an hour for many weeks and also that RSS feed on high priority, and monitoring all the Twitter thingies and also this forum... but still nothing. :(
Maybe for Christmas?
Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 15633 Posts
  • 2368 Reply Likes
Official Response
Should be fixed in Lightroom 5.7: http://blogs.adobe.com/lightroomjourn...
Photo of Massimiliano Ramuschi

Massimiliano Ramuschi

  • 65 Posts
  • 4 Reply Likes
Not fixed for me: fixed only for matrix based profiles, table gamma profiles (used by pro monitors like Eizo Color Edge and NEC Spectraview) are still showing clipped shadows.
Photo of Wojciech Brzozowski

Wojciech Brzozowski

  • 9 Posts
  • 2 Reply Likes
macOS Sierra, Lightroom 6.7 and bug exist in Library module and Develop (if Graphics Processor turned OFF)