Preview generation performance in LightRoom

  • 2
  • Problem
  • Updated 4 years ago
  • (Edited)
I'm often struck by how long Preview generation takes. On a brand new, top of the line MacBookPro with DNG images on SSD drives, LR takes up only 18% CPU utilization while generating previews. Last night I tried to generate previews on a folder with lots of images and it only got through 20% of them over 12 hours. I notice that the additional threads aren't being utilized at all which means this process isn't multithreaded is it?

This problem is compounded because there are poor controls for managing previews on large catalogs. When the Previews file gets above 100GB we're forced to throw it away and start generating new previews on just the pertinent images (potentially with different settings - 'low' quality for example saves a lot of space and is visually acceptable). I wish previews would expire after a period of inactivity just as 1:1 previews do. Some of my catalogs are 8 years old...

So I'm wondering 1) what can be done to improve the performance of preview generation and 2) what can be done to improve previews management.

p.s. on a similiar note, I'd love to see DNG Conversion multithreaded in LR. Demanding pros need performance!
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes

Posted 4 years ago

  • 2
Photo of Simon Chen

Simon Chen, Principal Computer Scientist

  • 1610 Posts
  • 529 Reply Likes
Where are the source raws located? On the same fast SSD or on some slower NAS/external drive? If it is the later, that could explain the lower CPU usage by Lr, because Lr is blocking most of the time waiting for the disk i/o to complete. When the negatives are loaded, the preview rendering could use up to 1/2 of the available physical cores on your machine. That number is reported by the "Maximum thread count used by Camera Raw" number in Lightroom's system info dialog.
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes
Hey Simon - thanks for your response! Off the SSD it's bad and even worse off an external RAID. Would be nice to see this performance at least double if it were multithreaded and utilized all the available threads... Could it not also be made to generate several previews in parallel?
Photo of Simon Chen

Simon Chen, Principal Computer Scientist

  • 1610 Posts
  • 529 Reply Likes
On speeding up the DNG Conversion, you could consider batching a couple (or more depending on your system configuration) DNG conversion tasks in parallel. That could allow you to squeeze more juice out of your capable hardware. It would be an option if you are not doing interactive work in Lightroom during conversion.
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes
Testing this now.... Looks like queuing up two DNG conversion processes in LR doesn't match the speed of a single process in the Adobe DNG Converter, unfortunately.

The main issue at hand, IMO, is either people use the "Copy as DNG" option when importing images from a camera's memory card. Lots of people don't do this because the performance hit is significant. And since we can only perform one import session at a time we can't do this DNG Conversion in parallel.

Because I work in timelapse, some of my import sessions involve thousands or even tens of thousands of images. For me, it makes more sense to copy them to the hard drive, use the DNG converter to convert them, and THEN import them into Lightroom. Would be a lot nicer to just import into LR straight form the camera's memory card using the "Copy as DNG" option if only it was multithreaded and the performance as good as the DNG Converter...

Demanding pros need performance! I'm just going to keep saying that. :-] I travel around and work with a lot of people and I hate to see people not using great features in LR, or using Photo Mechanic or Capture One because LR isn't as optimized as it could be....
Photo of Steve Sprengel

Steve Sprengel, Champion

  • 2673 Posts
  • 348 Reply Likes
Or just not use DNGs.
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes
No way Steve. I'd rather have the performance hit upfront than later down the line. The file size savings, embedded metadata, decreased backup times, and file verification all make it worthwhile for me. I *want* the DNG workflow to work and that's why I'm pointing out these shortcomings.
Photo of Simon Chen

Simon Chen, Principal Computer Scientist

  • 1610 Posts
  • 529 Reply Likes
I hear you Scott. We made a change in Lightroom 6 to allow export taking advantages of multiple cores on capable machines (>=4 cores), the same techniques that are used in the DNG Converter. We hear customers who loves the speed up and we also hear customers who want Lightroom to be interactively more responsive while an export session is running. It is certainly a tradeoff that we have to be mindful.
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes
I think high volume pros would always vote for speed. Thanks for your perspective Simon!
Photo of vortex

vortex

  • 7 Posts
  • 0 Reply Likes
I would like to throw in a second request for a faster performing Lightroom when it comes to preview generation and many other features which could benefit from being multi threaded. It would be much appreciated if Lightroom took advantage of my 12 core threaded Mac Pro!
Photo of eartho

eartho, Champion

  • 1135 Posts
  • 342 Reply Likes
Preview generation has always been extremely slow and i'm not sure Adobe has ever understood what users are asking for.

The sad thing is, there's an option on import that could give us what we want, but it's never been implemented correctly... Embedded and Sidecar.
Photo of Scott Martin

Scott Martin

  • 199 Posts
  • 17 Reply Likes
Earth, can you tell us more about this? Why do you use this option, how does it appear to misbehave, and how would you like it to perform?