Lightroom: Resize DNG without lossy compression

  • 2
  • Idea
  • Updated 5 months ago
  • (Edited)
With more and more photographers outsourcing their editing to the likes of shoot dot edit, broadand transmission speeds are becoming an issue. This is a growing trend.

I don't want to shoot sRAW, but would happily convert to a smaller DNG for the editors to work on, and then revert to the full size DNG when I get the LR catalog back from them.

However, the DNG converter doesn't allow resize without also selecting "lossy compression" at the same time. Is there a reason why these two options can't be disconnected? I want the outsource editors to see the original quality files - just a LOT smaller..

This would make a real difference to my workflow. I could transmit files in about 6-8 hours rather than 1.5 days.

Thank you!
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
  • frustrated

Posted 5 years ago

  • 2
Photo of stuartpeckphoto

stuartpeckphoto

  • 226 Posts
  • 2 Reply Likes
Is that a step you can add before importing into LR? An automater step that resizes DNG before hitting LR. You would then have two catalogs, one for full size, one for smaller size. Any metadata could be imported once you conform later.
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
The question then is.. what's the software that will allow me to do that without changing to lossy DNG? I found one piece of software which might work.. but nothing other than that. LR won't allow me to resize DNG without switching to lossy compression at the same time :(
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 134 Reply Likes
"Is there a reason why these two options can't be disconnected?"

Yes...to downsample, the file must be demosaiced. I suppose you could losslessly compress the demosaiced file, but that would make it perhaps three times as big with pretty modest gains in image quality. So, either leave the original sensor data or not.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Hi Phil,

I agree, in theory, that the option should not be disallowed, however I also question the value of it. You can probably get better over-all quality for same filesize (or smaller) by sending higher-rez lossy, than lower-rez lossless. Have you tried the lossy compression? Color and tone-wise it's identical to the original raw (whilst editing with 8-bit monitor anyway) - the only way to tell the difference is by zooming in, and if you're doing that, you'll get better results with the lossy (greater-rez) than the lossless (at same file size).

If you insist on a lossless reduced-rez DNG, you'll have to export a tif first, then wrap it in DNG (via subsequent "re"-export). I could probably roll that into a plugin for you to automate it, but again: I encourage you to first make sure that's really what you want, by trying both (manually) and comparing. If you do make such a comparison, please let us all know your conclusion.

PS - I assume you are thinking 16-bit for the reduced-rez lossless, not 8-bit - right?

Rob
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
Thanks. The adobe video I saw suggested that highlight (and shadow) recovery could be compromised? It's that which I need to be intact, but I can check out the compressed version to see myself.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
I'd be curious to know which video suggested this. I did a lot of testing before creating EditInLightroom, and from what I could tell, the handling of clipping and preservation of equivalent raw image beyond clipping..., in lossy compressed version, is identical to raw.

Note: the image inside a lossy compressed ("raw") DNG is *very* different than an "equivalent" jpeg (despite using same compression scheme for data-size reduction) - it's more like a shrunken version of the raw data - intended for further editing as raw, not for final consumption.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 132 Reply Likes
" I did a lot of testing before creating EditInLightroom, and from what I could tell, the handling of clipping and preservation of equivalent raw image beyond clipping..., in lossy compressed version, is identical to raw. "

It's not, but you have to push really, really hard to find differences.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
|> "It's not"

Other than the fact that lossy compression does not preserve exact pixel data, are there any other differences you are aware of? - I couldn't tell any.
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
Thanks guys. I'm going to do some testing to see how much latitude you lose. I'll let you know...
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
Well.... I'm part way through my tests:
2.35Gb for the CR2
2.05Gb for the DNG
0.73Gb for the lossy DNG (about 30% of the size!)

.... and so far I've yet to see a single pixel difference and I'm choosing files that I'm really pushing.

Wow! I'll update again later.

So far it proves you can't believe what you see online - even from Adobe videos!
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 371 Reply Likes
Yeah - it's top quality jpeg compression, and other than the expected compression artifacts (which you have to strain to see), the highlight/shadow handling seems identical to me. I'm not sure whether Lee Jay's comment "it's not" was just refering to the inevitable pixel imperfection of compression, or whether there are some other corners cut that I am not yet aware of...
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 131 Reply Likes
The shadows and highlights won't be identical (not just blocky JPEG artifacts) if you push hard enough on certain images.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 371 Reply Likes
Well of course there will be some quality loss, since the data is *lossily* compressed. I never meant to say that the edited results would be identical, only that the "handling" is identical, meaning, for example one can recover the same amount of highlights/shadows from beyond the clipping points, etc... - and the same is NOT true when editing an exported jpeg of the same filesize, in which case, all info outside clipping points is discarded, and even info inside the clipping points may be purposely thinned to optimize for final baked viewing...

The bottom line is, I think, for the purposes of this thread, that although quality differences may emerge when lossily compressed DNGs are pushed hard enough in post-production, such differences would not likely affect the *settings* one would come up with (or very little in any case), and so, for example, the settings that an edit house would come up with could be transferred back to original raw for full quality.
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
Here's the video:
https://www.youtube.com/watch?v=XVRama...

Go to 3:20 and listen for about two minutes. I can't see a breakdown any earlier than with the original CR2 or DNG so far... but I've yet to do my full tests. I'm going to try the resize too, to 13MP.

It's an official Lightroom video.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 367 Reply Likes
Thanks Phil,

I think the deal is that there just isn't all the same density and accuracy of information due to compression, so you'll not be able to get 100% the same quality out of it, in general.

As you're finding out by testing, it's not at all the same as editing an exported jpeg of the same size, since there is no discarding of clipped data, nor redistribution (data thinning in regions where it won't be missed...) to optimize for final baked viewing...

Rob
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
Just tried the rescale to 13MP and the files came down from 785Mb to 635Mb (from 21MP). This surprised me - I expected them to be closer to half of the size - something like 400Mb.

I'm wondering if I did something wrong, but I can't see it..

Anyone else fancy having a go? See if I'm being an idiot? For that level of difference it wouldn't be worth rescaling.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 132 Reply Likes
The original size was probably a lot larger.
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
So with further tests I'm having to push shadows 5 stops to see ANY difference at all - and then it's just a few pixels.

Well... I don't push shadows 5 stops so, so far, this is looking like a great option!

Thanks for getting me to try it out :)
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 371 Reply Likes
Yeah, I can't tell yet if Lee Jay is aware of something I'm not, or if there is just some difference in language usage, or experience with some relatively rare corner cases... but I have also been *very* impressed/pleased with the lossy compression technology for DNGs - color, tone, and handling have been "spot on" in my experience, and even detail is very close, other than some relatively minor compression artifacts.

Thanks for letting me/us know your experience as well.

Cheers Phil,
Rob.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 131 Reply Likes
My issue was with your term "identical" whereas "very, very close" might have been more appropriate.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 371 Reply Likes
To be clear: I was reporting my personal experience, which was:

* In all cases I tried (which was a fairly large number), color, tone, and handling was literally identical. I'm not saying pixel-for-pixel identical, but if you zoom out, there is no discernable difference.

I'm not saying I disbelieve it's possible for there to be noticeable differences in some cases, it's just that I have not yet experienced any such cases.

I understand you may be reporting something "theoretical", but if you do have personal experience with cases when there were noticeable differences, I would love to hear a description of such differences, and if possible, see a sample.
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1008 Posts
  • 214 Reply Likes
The best reading I have found on lossy compression:
http://forums.adobe.com/thread/947446

I really think it is an excellent choice for Phil : keeping about all pictures qualities and editng flexibility for outsourced editing editing when you know your output won't require maximum size/resolution.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Even if you know your output *will* require maximum size/resolution. i.e. one can transfer editing from lossy version to original version. But yeah, outsourced editor won't be able to create max size/rez output.
Photo of Phil Drinkwater

Phil Drinkwater

  • 24 Posts
  • 6 Reply Likes
I've found one more interesting thing out. I can substitute the original full DNG at my end if I choose to. LR isn't fussy about whether it find the lossy or original DNG.

So I can:
Convert to DNG and Lossy DNG and delete the CR2's
Upload the lossy DNG
Get the LR catalog back from outsource
Substitute with the original DNG's, just to be sure of quality

That way there's no issues any way. As far as I could see, the edited file from a lossy DNG was the same as from a full DNG, so it should just "work"... :)

I might also convert all of the outtakes that I don't use at all to Lossy DNG and save a ton of disk space too! But not until after the job is completely finished...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 371 Reply Likes
Phil, yes: all changes made to the lossy version can be readily transferred to the original raw (such is the basis for Edit-In-Lightroom). And, if Lr doesn't recognize the swap, just do an "Update Metadata and Preview" - that should be a sufficient prod.
Photo of Eric Chan

Eric Chan, Camera Raw Engineer

  • 617 Posts
  • 121 Reply Likes
When it comes to quality/compression issues like the ones being discussed in this thread, my recommendation is that users really need to do their own evaluation based on the types of images they shoot and -- even more important -- the types of edits they tend to make. Some folks tend to push the sliders a lot, whereas others don't. And with some types of images you can push the sliders a lot with lossy compressed DNGs and get away with it fine -- and in other cases less so.

So what Phil did here was the right thing in my view.
Photo of Shilpa Nandi

Shilpa Nandi

  • 1 Post
  • 0 Reply Likes
Here is a new wordpress plugin https://wordpress.org/plugins/way2enjoy-compress-images/ thats allows you to compress the site's image and acceleration, totally free and without registration.