Lightroom Classic: Import running in parallel with Preview building

  • 1
  • Problem
  • Updated 11 months ago
  • (Edited)
I delayed upgrading to LR 7.x until Loupedeck+ hardware forced me to.  So far, commencing with 7.5, I have seen some improved performance except during import.  Import is glacial slow, worse than LR6 ever was.

The problem seems to be LR 7.5 performing the Import in parallel with the building of Previews.  I see no way to disable this, and force LR to perform the tasks in serial, as previous versions of LR 6 and before always did.  (I cannot speak for previosversions of LR 7—I never went there.)

Adobe is known to poorly support multi-core CPUs, so why this decision to multitask was made is unclear, and clearly unwise.

Is there a way to disable this poor attempt at multi-tasking?  If not, this must be an option in the next release as Lightroom has now become too slow to use.  ON1 2019 is coming next month, promising to provide a complete migration path (i.e., develop settings, keywords, and collections) from Lightroom to ON1.

Adobe should be working overtime to improve performance rather than working overtime to drive users to competing platforms.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes

Posted 11 months ago

  • 1
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
You can force preview previews to be built after import by unchecking the option Preferences > General > Replace Embedded Preview With Standard Previews During Idle Time and using the Import option Build Previews: Embedded & Sidecar.  After the import completes, select all the imports and do Library > Build Standard-Sized Previews.

From the forum posts, I don't see evidence that a large number of people are experiencing import performance problems with 7.5 or 8.0.  (Note that the most recent release of LR is 8.0.) The recent posts about performance are consistent with the long-term pattern, which is that a significant minority of users experience performance problems, with sometimes hard-to-discern patterns of who gets affected. 


Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Sorry, John, but you missed the point.  Starting with at least LR 7.5 (and I am now on LR 8 which exhibits the same behavior) one did not have to manually build previews AFTER the import process, it was automatic.

Now it is in parallel, and the performance suffers.  Look at all the other posts complaining about poor import performamce.  You need to read the posts here; they are manifold.

Your suggestion flies in the face of the situation; to whit, Lightroom now performs Import and Build Previews in parallel—that's at the same time in case you're confused—instead of performing in series; that's one after the other.  (Another point of edification.)

You are saying that users now have to disable automatic Preview Build and do it manually afterwards?  Excuse me sir, but that is asinine.

(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
I understood your point perfectly well. You asked,

"Is there a way to disable this poor attempt at multi-tasking?"

and I answered your question.  I didn't suggest it was a desirable solution.



(Edited)
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
John, you didn't answer the question.  You offered a half-baked soultion, and then you attempted to bolster your BS response by stating—without ANY factual evidence—that no one else was seeing this issue. 

Why offer it at all?  You're supposed to be a Champion, yet you offer a completely ridiculous solution instead of answering the question: "Is there a way to disable this poor attempt at multi-tasking?"

(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"Why offer it at all?"

I offered it as a plausible workaround, a solution that might help to get work done while waiting for a better solution that doesn't require a workaround.  If your import performance is bad enough to force you to consider another product, then a three-click (annoying) interim workaround is possibly better than switching products.  (Click Previous Import, Cmd + A to select all photos, Library > Previews > Build Standard-Sized Previous.)

I don't experience slow imports, thankfully, so I can't evaluate how much faster the workaround actually is.  
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Oh sweet Jesus take the wheel, John.  It might be simply a three click workaround, but it is TIME CONSUMING, and time is money for professional photographers.

Gawd, protect me from people and Adobe who never actually use Lightroom to make money!
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
And dammit, this still does not answer my question:  "Is there a way to disable this poor attempt at multi-tasking?"

Clearly, nobody knows.  Nobody knows why Adobe made this change, nor when.  (Since I adopted LR7 late in the game, at 7.5, and now 8.0, I cannot say.  But it was NOT the case in any version of LR 6.x.)

Adobe?  Are you listening?  WTF..., over?

BTW, I have been sitting here waiting for 2300 images to import for over two hours—on a MacPro with eight cores and 128GB of RAM.  In that time, my Colorado Buffaloes lost, the Dodgers are on their way to the World Series, and my Lakers are ahead.  And my bottle of Woodward Reserve is lower than when the import started.
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
Which is faster?

- Importing n photos with Build Previews: Standard

- Importing n photos Build Previews: Embedded followed by Build Standard-Sized Previews.

You reasonably hypothesized that the latter is faster.  If it is significantly faster on your configuration, then you could save a lot of time by using that workaround while waiting for Adobe to fix the problem. 
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
I never made any such hypothesis, John.  However, I am importing with Standard previews as I typically do (and as I am doing in this specific case).

Actually, when importing with 1:1 previews I find—without a valid statistical analysis; just a gut feeling—that the import process is not as slow.  This might be due to the smaller import volume, so I would not care to make a concrete statement.

I never use embedded previews.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"I never made any such hypothesis."

You wrote, "The problem seems to be LR 7.5 performing the Import in parallel with the building of Previews.  I see no way to disable this, and force LR to perform the tasks in serial."  In other words, you're hypothesizing that importing first and then building previews is faster.  That's what importing with embedded previews followed by Build Standard-Sized Previews does.  (Importing with embedded previews copies the small, existing JPEG previews from the photos into LR's preview cache.)
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
I am not hypothesizing a damn thing, John.  My Import with Standard Previews duration has increased by nearly 300% since 'upgrading' to LR 7.5.  That is a fact.  It was never this slow under LR 6.  I have completed imports with 1:1 Previews of 10,000 images in less time than LR 7 is requiring.

I do not Import with embedded previews; I am damn sure I told you that.

It is a known fact that Adobe does not support mulitple core CPUs; they have stated this more times than can be counted.  Yet, they have now embarked upon parallel tasks without such support, which is why we—and I am NOT the only one—are realizing poor import performance.

My import with standard previews of 1,998 images just completed...after three hours and 26 minutes!  An import of this size under LR 6 would not have taken more then 60 minutes...I know from having imported over 1 million images under LR 6!

The question remains: "Is there a way to disable this poor attempt at multi-tasking?"

If you cannot answer John, it really is okay to say "I don't know" or shut the heck up.  No one, especially me, is going to think the less of you.

However, when one continues to pontificate without any clear knowledge (or understanding of the issue), then one is being Trumpian, and few respect that. 

Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"I am not hypothesizing a damn thing, John."

So the statement, "The problem seems to be LR 7.5 performing the Import in parallel with the building of Previews", is not a hypothesis?  The fact that you're experiencing poor import performance supports that, but it's certainly not definitive.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
No, it is not a hypothesis.  I have the facts to prove that it is past the hypothesis stage and is now a theory, waiting to be disporoven.  That is how the Scientific Method works.   (A Lightroom Champion might not be expected to know this basic process of science.)

It is clearly not definitive; it is in position to be challenged by facts demonstrating the opposite...but not your BS assertions.   That is how the Scientific Method works.  Present the facts, and the theory—stemming from the hypothesis—might be proven to be false.

The observations indicate and support the theory that the change in the Adobe Lightroom Import from a serial process to a parallel process is the root cause of the decrease in performance, given that nothing else has chaged.  Occam's Razor:  The simplest explanation usually proves to be the correct one

If you have an alternative explanation, please present it, AND the factual evidence supporting your claim.  (Note:  Being a Champion carries no weight.  The Nobel Laureate Shockley, who is credited with inventing the transistor, also held that Africans were of less inteeligance than other races simply because they were black skinned.  He was, of course, full of it.  Just goes to show what a appellation is worth.)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
It's very possible that building previews concurrently with importing is causing your problems, but as you said, you're assuming "that nothing else has changed".  But an awful lot has changed between LR 6.12 and LR 7.5 with respect to performance -- Adobe has posted here frequently about its efforts to improve LR performance, especially on machines with more than 4 cores.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"you [stated]—without ANY factual evidence—that no one else was seeing this issue."

That baldly misstates what I said: "The recent posts about performance are consistent with the long-term pattern, which is that a significant minority of users experience performance problems, with sometimes hard-to-discern patterns of who gets affected."  

In the context of Lightroom, a "significant minority" of users is a fair number of users.  

As evidence, I searched for topics mentioning slow import performance and merged a few into the main existing topic: https://feedback.photoshop.com/photoshop_family/topics/import-lightroom-cc-classic-is-slow-after-upd...

That 11-month-old topic has 24 Me Toos and 37 Followers. That is consistent with problems that affect a "significant minority" of users.  For problems with core functionality that affect the majority of users, there is typically a higher rate of Me Toos and Followers.   
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
John, you clearly have no knowledge of statistics.  Please provide the statisitcal method utilized, the power of the statistical method you utilized, the confidence interval you selected, and the population analyzed.  Otherwise, what you stated is without any merit whatsoever.  It's pure swank.

Please do not try to bolster your "Champion' rating with BS; it just makes you look bad.  I have seconded all of your opinions previously because what you stated was correct.  I was impressed.  But your are clearly out in left field now, hanging on by your reputation, and it's time to stand down or furnish the evidence.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
I wrote, "That 11-month-old topic has 24 Me Toos and 37 Followers. That is consistent with problems that affect a 'significant minority' of users."

I offered that as a judgement made on eight years of experience contributing to this forum (often very critical of Adobe).  But others can easily test that judgement for themselves by searching for all topics marked Problem (solved or not) and sorting them by number of Me Toos (recipe below).  Here are the top 30 problems, in decreasing order:





Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
To do this search:

1.Next to the search box, click the magnifying glass to search.

2. Click Advanced Search.

3. Under Filter By Type, check Problems only.  

4. Under Filter By Category, check Lightroom Classic CC only.

5. Select Sorted By: Number Of Me Toos.

6. Click Search.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Did you bother to add up the number of complainants who replied to those initial posts?  That number alone would alter your statistical results.

I also notice that there are quite a few—no, I have not dome an analysis—posts which are not accounted for in your sample and which I have seen.  (And no, I am not going to waste my time collecting them.  I'll leave that to you.  I have money to earn, waiting on f@#%ing Lightroom.)

Why don't you just admit it, John.  The problem is pervasive and I am not the only one complaining about it.  Just because you don't see it does not mean it does not exist.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Gawd protect us simple folk from the thin-skinned; i.e., those who have been annointed with the appellate "Champion."  They cannot EVER give up.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"Did you bother to add up the number of complainants who replied to those initial posts?  That number alone would alter your statistical results."

The number of people following a topic are the second number displayed.  (By default, someone becomes a follower by replying.)  You can see that the number of follows is reasonably close to the number of me-toos.   You can sort by the number of followers instead. 

The 24 Me Toos / 37 Followers of the slow-import topic makes it comparable to many other topics that affect many LR users but clearly not a majority of them.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
I never said it was a majority of reported complaints, only that is was it was "manifold."

You are sounding like Trump, offering 'facts' to fit your worldview.  Please stick to the real facts and the issue at hand, lest you sully your trophy of 'Champion.'

I repeat: "Is there a way to disable this poor attempt at multi-tasking?"

Answer the question or leave it lay.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"I never said it was a majority of reported complaints"

And I never said you did. I was providing evidence to support my statement that poor import performance affected a "significant minority" of users.  

I'm trying hard to stick to the facts and avoid responding to ad hominem comments.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"Why don't you just admit it, John.  The problem is pervasive and I am not the only one complaining about it.  Just because you don't see it does not mean it does not exist."

Again, that's baldly misstating what I wrote in my first reply: "The recent posts about performance are consistent with the long-term pattern, which is that a significant minority of users experience performance problems, with sometimes hard-to-discern patterns of who gets affected." 
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
John, do you actually USE Lightroom?  I mean to say, do you actually earn an income fro the use of Lightroom?

I think NOT!  It is NOT simply a matter of three clicks.  It is a matter of setting Lightroomnto perform a task, haiving some expectation of how long that taks will take to complete, and the user going aobut his or her business of doing something else whilst Lightroom does its damn job.

Lightroom does NOT sound a freaking alarm when it is done.  Lightroom does NOT call your cellphone when it is done.  Lightroom does not do a damn thing when it is done.

So, the user is forced to either babysit this POS software or continually check back to see if it is done.  How much time passes between the import and the three  clicks in your scenario?  How much time?

Time is money.

Which tells me that you earn NOTHING from the daily use of Lightroom, other than what being a shill pays.   Professional photographers have deadlines to meet.  Failure to do so costs us money.  This cannot be a concept which escapes you.

Please stop the BS.  Answer the freaking question.  Or shut the heck up.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
Yes, I make quite a bit from LR, thank you.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"Is there a way to disable this poor attempt at multi-tasking?"

Yes, the workaround I provided above clearly accomplishes that, at the expense of three clicks after the import.  

Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Not an answer to the question.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4573 Posts
  • 1225 Reply Likes
"The problem seems to be LR 7.5 performing the Import in parallel with the building of Previews. I see no way to disable this, and force LR to perform the tasks in serial, as previous versions of LR 6 and before always did. "

You've been insistent about the overlap of import (adding photos to the catalog) with building of previews. It may be that your performance issue is not caused by that overlap but rather by the generation of previews in parallel (building multiple previews concurrently), which was added to LR 7.   

You can go back to generating one preview at a time, as in LR 6, by unchecking the option Preferences > Performance > Generate Previews In Parallel.  Note that with this option unchecked, importing will still be overlapped with preview building.

This doesn't do what you've been asking for, but it may get you LR 6 import performance.  The actual importing of photos (adding them to the catalog) goes roughly an order of magnitude faster than generating the previews  (at least on my well-performing configuration) -- you can observe this by importing photos with Build Previews: Minimal (which is even faster than Embedded). 

It's preview generation that takes the bulk of the time, which is why Adobe parallelized it in LR 7.  And LR has previously struggled with parallel performance on machines with more than 4 cores, though the frequency of those reports appeared to tail off with later versions of LR 7.

Photo of Mayuri Jain

Mayuri Jain, Employee

  • 64 Posts
  • 25 Reply Likes
@dmeephd:

You can disable parallel preview generation by unchecking option "Generate Previews in Parallel" in Preferences> Performance.

Let me know if you are still facing the issue.

Thanks
Mayuri
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Unfortunately, about 15 minutes into the Import, Lightroom 8 again started building Standard previews in parallel.
Photo of dmeephd

dmeephd

  • 300 Posts
  • 77 Reply Likes
Thank you, Mayuri.  This works and the performance has improved.  The problem is that whilst Lightroom is performing these tasks in parallel, it cannot do much else, including loading the images in the view panel.   Now I can commence keywording sooner (as soon as the Import) is complete) while Lightroom is building the previews afterwards.

Thankfully someone knows the answer instead of so-called 'champions' spending hours and numerous responses trying to blame the messenger.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4562 Posts
  • 1221 Reply Likes
Note that the option Generate Previews In Parallel, recommended by Mayuri and me, isn't at all an answer to your question, how to stop previews from building until after the import:

"Import running in parallel with Preview building. The problem seems to be LR 7.5 performing the Import in parallel with the building of Previews.  I see no way to disable this, and force LR to perform the tasks in serial"

The option causes previews to built one at a time, but they still get built in parallel with images being imported into the catalog.  You were very insistent that was the cause of the problem, so definite as to not be deserving to be called a "hypothesis".  


Photo of John R. Ellis

John R. Ellis, Champion

  • 4562 Posts
  • 1221 Reply Likes
"trying to blame the messenger"

Could you please identify with specific quotes where I blamed the messenger?