Lightroom Classic: Disable smart collection

Yes, I believe that a possible never ending loop is indeed one of the reasons why Adobe did not implement searching for images in another smart collection. The other reason is that this is functionality that already exists anyway. You can simply include the criteria of smart collection B in the criteria of smart collection A. It may be a little more work, but in the end it's exactly the same result.

The problem with feature suggestions is that people can suggest anything they like. Thy don't have to check how difficult it is to add this to the existing code. They don't have to check wether Lightroom would become even more bloatware with all these features. They don't have to check whether more than a handful of people might ever use this. And of course they always think that their feature request is the most important request in the history of Lightroom and so it is unbelievable that it is still not implemented...

Coming back to your initial question: I doubt that turning off smart collections would make Lightroom significantly faster. When Lightroom starts, it needs to run the query for each smart collection. That is why you see the image count behind your smart collections appear not instantly, but (depending on the size of the catalog) after a few seconds. But after that, when you change the criteria of an image, Lightroom only needs to check that one particular image against the smart collections. When I change an image from 4 stars to 5 starts, I see my four star and five star smart collections update instantly. BTW, my catalog contains 170,000 images.

Hi Johan, you are quite right. But I find that my smart collections, I have about 12, are rebuilt every time LR starts. This background process takes about 5 minutes and slows down LR. I have read that some users have a lot more smart collections. I imagine they are just simple SQL statements and thus selecting on an index-able criteria like the "star rating number column" then they will be very quick. But if using a CONTAINS or LIKE search (e.g. lens criteria text column contains "24-105") they take a lot longer as they need to read all rows in the database rather than using a pre-built index.

"Each group takes over 2 minutes to process before the numbers appear. If I close LR and re-open it immediately it takes 20 seconds to refill them. But by the next day when I start LR it is back to over 2 mins per group."

This strongly indicates that unusually slow or excessive disk i/o is slowing down the "cold starts" compared to the "warm starts" by a factor of 6.

With a cold start, the operating system's in-memory disk cache is empty, so LR has to read the relevant parts of the catalog file from disk. But with a warm start (restarting LR immediately after you exited), the catalog file is already in the disk cache, so little extra disk i/o is required. 

There are two possibilities: Your disk is responding to read requests unusually slowly, or your catalog file is excessively large for some reason.  Once you post the size of your .lrcat catalog file, we can compare the size / photo with my catalog and others to see if yours is excessively large (per photo).

 Before someone shouts "Optimize!", consider: LR Classic optimizes the catalog when it converted from an earlier version, so your catalog was optimized in the last week or so. So while you could try File > Optimize Catalog, it's unlikely it will have any effect. (It also compresses the history field of each photo, resulting in catalogs that can be 50% or more smaller than LR 6.)

 ------------------------

I did some quickie timings with a plugin script to verify that on warm starts, your LR is executing smart collections roughly as fast as mine, further pointing the finger at cold-start disk i/o.

 I timed the execution of a smart collection very similar to the ones you posted.  For warm starts, I observed times similar to what you (roughly) observed: 0.9 x 10^-5 sec/photo on my catalog, versus 1.8 x 10^-5 sec/photo on your catalog.  But for cold starts, I observed just slightly slower times of 1.1 x 10^-5 sec/photo, versus order-of-magnitude larger times of 11 x 10^-5 sec/photo on your catalog.

 (My timing was done on a somewhat smaller catalog stored on a reasonably fast external hard drive, with a Macbook Pro mid-2015 with 16 GB memory. The test smart collection returned 14K photos. The SQL query execution time is directly proportional to the number of photos, since SQL indexes can't be used for these queries.)

Below is the timing script I used. You can run this script in your LR by doing Preferences > Presets, clicking Show Lightroom Presets Folder, and creating a subfolder "Scripts" under the "Lightroom" folder that's selected in Mac Finder or Windows Explorer.  Then copy the script into a file "timesmartcollection.lua" in that folder and restart LR.  You'll see a Scripts menu appear on the far right of LR's menu bar, and the script "timesmartcollection" will appear in that dropdown.

 If you want to measure cold and warm starts, you'll have to make a copy of your catalog folder and measure with the copy.  Delete the smart collections from that copy so they don't affect the timing.  To measure cold starts, you can either reboot your computer each time; or on Mac, use the "purge" command in Terminal. On Windows, the following recipe supposedly works, though I haven't tried it: https://stackoverflow.com/questions/478340/clear-file-cache-to-repeat-performance-testing

If you want to time a different smart collection, in the LR's Collections panel, right-click the smart collection and do Export Smart Collection Settings.  Open the exported .lrsmcol file in a text editor and copy and paste into the "timesmartcolleciton.lua" script where indicated.

local LrApplication = import "LrApplication"
local LrDate = import "LrDate"
local LrDialogs = import "LrDialogs"
local LrTasks = import "LrTasks"
local catalog = LrApplication.activeCatalog ()
local s
--[[**** Insert the contents of the .lrsmcol file here ***************]]
s = {
    id = "CC90F266-2911-427C-8753-1469B13A76AD",
    internalName = "Focal 28-105mm 24-105 lens",
    title = "Focal 28-105mm 24-105 lens",
    type = "LibrarySmartCollection",
    value = {
        {
            criteria = "lens",
            operation = "all",
            value = "24-105",
            value2 = "",
        },
        {
            criteria = "cameraSN",
            operation = "==",
            value = "1831227178",
            value2 = "",
        },
        {
            criteria = "focalLength",
            operation = "in",
            value = 28,
            value2 = 105,
        },
        combine = "intersect",
    },
    version = 0,
}
--[[******************************************************************]]
LrTasks.startAsyncTask (function ()
    local t = LrDate.currentTime ()
    local photos = catalog:findPhotos {searchDesc = s.value}
    t = LrDate.currentTime () - t
    LrDialogs.message (s.title,
        string.format ("%d photos\n%g seconds", #photos, t))
    end)

 

As a workaround, you could significantly reduce the number of smart collections by removing the camera and lens from the criteria.  Then define a filter preset for each camera/lens combination:



So you'd first select the desired smart collection and then the appropriate camera/lens filter.

So your catalog has is actually smaller in terms of bytes / photo: Yours is 15 KB / photo, mine is 26 KB / photo -- that doesn't explain the slowth you're seeing on cold starts.

The camera serial number, lens id, apertural, and focal length are all stored in a few small SQL tables, and each of those fields have indexes. But the tables are small enough such that a linear scan through 90K in-memory entries should be extremely fast even if it doesn't use the index (e.g. for the "contains" operators), and if the tables don't happen to be in memory, they could be read in a small fraction of a second.   That's all demonstrated by timings on my catalog.

So clearly something bad and out of the ordinary is happening with your catalog, but I don't see any clues as to what.  

In addition to using filter presets as suggested before, a couple more workaround suggestions:

- Use "is" rather than "contains" for serial number and lens id for any remaining smart collections using those fields. That will allow the indexes to be used (but as discussed before, the tables are so small that  a linear scan without the indexes should be plenty fast enough, unless the SQLite query optimizer has gone off the rails with your particular queries).

- Try exporting a copy of the catalog with File > Export As Catalog, with all of the options unchecked, and open the copy in LR. That will export a "clean" SQL database, perhaps "cleaner" than what the Optimize command does -- I believe the Export essentially rebuilds the tables from scratch.  So maybe there are some rotted bits that are embedded in the old catalog that exporting will remove.  Just an educated guess, probably only a 15% chance of having an effect.

Hi,

I have logged a feature request internally so that we can track this idea.

For now users can do the following workaround (not great but still it works) to get the same effect. Create a Collection set called "Disabled smart collections" or any name you like. Move all the smart collections you want disabled into it. Collapse the collection set and restart Lightroom. As long as this collection set remains collapsed they won't be evaluated. One drawback of this approach is that users need to remember to keep the collection set collapsed. If they open it they need to remember to close it and restart Lightroom to get the benefit.