Lightroom Classic: Limit of 52 reorderings in custom-ordered collections and folders

  • 13
  • Problem
  • Updated 1 month ago
  • In Progress
  • (Edited)
[I've verified this bug still exists in LR 9.2, both for collections and folders. See this post. -- John Ellis] 

After a while of custom sorting the order of photos and stacks within a collection (using Grid View), Lightroom starts to unpredictably refuse to sort photos. Some will get positioned where I drop them, others won't move at all and yet others will get positioned somewhere close by (eg. 4 or 5 photos before or after).

After digging into the catalog file I've come across what I think is the problem, but don't know how to fix it. In the attached file you'll see a screenshot of the database table for the collection, you'll see I've hilited the images that are part of the same collection, but their positionId is identical (which should never happen I'm assuming), probably due to the field size reaching it's maximum length. This is what I believe is causing the problem. Tested this on both Lightroom 5.7 and CC 2015.8.

This is a major bug and effectively stops user sorting from being functional, as well as now having potentially lost weeks of work. Any suggestions?



Thanks,

Adrian
Photo of Adrian Cleave

Adrian Cleave

  • 3 Posts
  • 1 Reply Like

Posted 3 years ago

  • 13
Photo of Eric Webb

Eric Webb

  • 12 Posts
  • 3 Reply Likes
In response to an e-mail just circulated by Rikk Flohr - who appears neither to have read nor understood the accumulated e-correspondence - I further confirm that from my standpoint this bug is definitely still active in a fully updated Win 10 installation.

Photo of Sunil Bhaskaran

Sunil Bhaskaran, Official Rep

  • 473 Posts
  • 172 Reply Likes
Eric,
I am sending you an e-mail off-list to get some more details.

Thanks,
Sunil
Photo of Ryan Caliendo

Ryan Caliendo

  • 3 Posts
  • 2 Reply Likes
Sunil the images are rearranging themselves! I’ll be done with arranging an entire batch of photos on the timeline and if I make one false move with a virtual copy, most images get scrambled.
Photo of jbedford

jbedford

  • 154 Posts
  • 40 Reply Likes
What's going on with this bug?? It still persists over THREE years after being first brought to the attention of this forum. Honestly, do you guys have no clue how to fix this? It's really annoying that it appears to be a back-burner issue when your users are demanding it to be resolved. Please update us as to what is being done to resolve this major bug.

As to what Ryan (above) mentioned, the virtual copies getting scrambled, this has to be the most frustrating part of this experience. Even after exiting LR, reopening, and going back to my collection, I still can't move around the pictures that are now out of order. Thanks a lot.
(Edited)
Photo of Mary Ella Jourdak

Mary Ella Jourdak

  • 1 Post
  • 1 Reply Like
Just chiming in that I'm fully updated and having this bullshit problem. This is SO frustrating. It's getting easier and easier to consider leaving behind the Adobe programs...
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 6991 Posts
  • 1578 Reply Likes
Greetings all,

The Lightroom Classic team marked this issue as fixed in 8.3 based upon the confirmation of the original reporter of the issue. It appears now that the issue was fixed with regard to the 52 image threshold but is still encountered at higher numbers of images. 

I've reopened the bug and am looking for more information regarding how many images it takes to break the reordering. Can you provide me with solid numbers of how many images it takes to break?
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
I retested the recipe I tested in LR 8.3 with 1025 photos:
https://feedback.photoshop.com/photoshop_family/topics/lightroom-major-bug-with-custom-sort-order?to...

and it still fails the same way. I tried with significantly fewer photos and couldn't observe any problems. But this test surely points at some problems with the current renumbering algorithm.
Photo of Brendan Bullock

Brendan Bullock

  • 4 Posts
  • 0 Reply Likes
Chiming in to say that a 3 year old bug of this magnitude needs to be addressed.  As a professional wedding photographer, my final collections for every project are near 1,000 images - I can't begin to tell you the frustration when I reach the end of a project, go to export, and find that LR has moved files around within the custom sort here and there.  It can take hours to go through, find them all, and rectify the problem.  Basically once a week I am dealing with this problem on a grand scale - which adds up to many lost hours per month.  C'mon Adobe - lets have a fix already!

PS - I have noticed that this random re-sorting primarily targets images I have color labeled Red - but not ALL images labeled Red, just some of them.  Occasionally a non-color labeled image will move but I would say that 95% of the time its a Red that's moved.  This seems somehow baked into the algorithm.  I generally don't use other color labels, so I can't confirm or deny that it affects Reds as opposed to certain other colors.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
And now Adobe has removed the ability to load any of the older releases where ordering was not broken!
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
"...any of the older releases where ordering was not broken!"

Unfortunately, the collection ordering has been broken at least back to LR 5.7.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
I'm not even talking about ordering collections.  I could order my regular photo sets in 8.0 but not 8.3.  I've reproduced the ordering problem on every release since.  I don't suppose there is a way to revert a catalog back to LR 6......
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
The same ordering algorithm is used for both folders and collections, in LR 8.4 and in all previous versions.  So reverting back to LR 6 won't in general avoid the bug.  If anything, the version of the algorithm in LR 6 may trigger the bug with fewer custom reorderings than in LR 8.4.

LR 8.3 changed the ordering algorithm in an attempt to fix the bug, changing the observed behavior, but it's still buggy for both collections and folders.  That you observed the bug in 8.3 but not in 8.0 is unsurprising -- the bug's behavior isn't completely understood (it's a bug after all).  

If you have a precise step-by-step recipe for reproducing the bug that's substantially different than this one:
https://feedback.photoshop.com/photoshop_family/topics/lightroom-major-bug-with-custom-sort-order?to...

then please post it here. The posted recipe requires moving large blocks of photos within a collection or folder, repeatedly, which doesn't represent typical user behavior and thus might be prone to de-prioritization by Adobe product managers when deciding on which bugs to fix.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes

I previously posted this in another Adobe forum and received no official response.

Win10 Laptop

All actions are performed on a brand new catalog with 100 images (0.jpg-99.jpg) copied from the same original image.

(same creation date)

I selected all images except the first 2. (So 2-99) I then dragged them to be between 0 and 1.

Next deselected only the first of my selected images (2) and dragged the rest between the first 2 images.
I repeated this until all images were moved. (0, 100, 99, 98, .....)

Next I repeated the above, selected all images but the first 2 and moved them between the first 2, deselected 1 image from my group each time.

Eventually I got to a gallery view of 0, 73, 72, 71, 70, ....... 1 74, 75, ..... 100 which 72->1 selected.
When I moved the group of 72->1 between 0 and 73 the ordered was invalid.

I ended up with 0, 1, 10, 11, 12, 13, ...... not what I expected (0, 72, ......)


Photo of Sunil Bhaskaran

Sunil Bhaskaran, Official Rep

  • 473 Posts
  • 172 Reply Likes
@StormyTheCat,
Thanks for the reproducible steps. (The issue is reproducible.)
We are currently investigating.

Thanks,
Sunil
Photo of Phil Burton

Phil Burton

  • 59 Posts
  • 10 Reply Likes
I can tell you that I first encountered this bug in LR 9.1 (not 8.x !!).  My collection is about 290 photos, and I could no longer rearrange photos after about ONLY 10 operations.

My effective workaround is to shut down and restart LR.  That works, but it's a huge timewaster, considering the time to back up the catalog and then restart LR.

To the dev team:  How about a fix that works for a collection of say 50K images, larger than anyone is likely to have, instead of these band-aid (plaster, for our UK friends) solutions.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
Since it doesn't look we will get a fix for this, please post a workaround.  i.e.  Will something like closing down and restarting work - I don't think it will.  How about exporting the catalog and importing into a new catalog?  Will that fix the broken decimal indexes?

This has become much more critical now that Adobe has discontinued access to older version of the product that did not have this problem.  

Thanks you,
Steve
Photo of Phil Burton

Phil Burton

  • 59 Posts
  • 10 Reply Likes
@StormyTheCat

Please see my post from a month ago.  Shutting down and restarting Lightroom does work, but it's a huge timewaster.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
I did see your post.  I didn't work in the past for me.
Photo of Phil Burton

Phil Burton

  • 59 Posts
  • 10 Reply Likes
@StormyTheCat,

I solved this problem by trial and error.  At the time I encountered the problem, I didn't know that it was a long-standing issue.  If it matters, my system has 16 GB of memory.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
Ram doesn't matter. They store order indexes as floating point and renumbering enough gets to where the precision isn't enough.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
"Since it doesn't look we will get a fix for this, please post a workaround.  i.e.  Will something like closing down and restarting work - I don't think it will."

Restarting won't fix the problem of too many insertions into a collection. But it may be that some other users here are experiencing another bug for which restarting helps.

 
A workaround for too many insertions into a collection was posted above:

1. Select all of the photos in the collection.
2. Do Library > Create Collection.
3. Select "Include selected photos".
4. Delete the original collection.
5. Rename the new collection to have the original's name.

Unfortunately, this workaround won't preserve any stacking in the original collection.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
What about the case where the original problem is not part of a collection?
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
In the case of custom order for a folder, I just tested a similar workaround, which appears to work:

1. Select all of the photos in the folder.
2. Right-click the parent folder and do Create Folder Inside.
3. Select "Include selected photos".
4. Delete the original folder.
5. Rename the new folder to have the original's name.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
What is the sort order after this set of steps?  Is the custom order retained?

Thank you
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
It appears to be (I haven't done extensive testing).
Photo of John R. Ellis

John R. Ellis, Champion

  • 4972 Posts
  • 1385 Reply Likes
I retested more thoroughly in LR 9.2, and it fails with collections as small as 3. To reproduce:

1. Make a new catalog with photos.
2. Create a collection with all n photos.
3. Select the collection and do View > Sort > Custom Order.
4. Select all n photos and deselect the first and second, leaving the last n - 2 selected.
5. Drag the n - 2 selected photos between the first and second photos.
6. Repeat steps 4 and 5 for m - 1 more times.
7. Note that what was the first photo is no longer first (incorrect).

Here's a table of and m values that provoke the bug:



In general, for - 1 a power of 2, m = 1 + floor (52 / log2 (n - 1)). (Intuitively, each insertion reduces the number of bits available in the 53-bit floating-point mantissa for representing the different positions.)

The values in the table column AgLibraryCollectionImage.positionInCollection specify the positions of the photos within the collection, and they should all be unique (since each photo has a different position). But after step 6, most of the values in the column are identical:



That means LR has lost track of the custom ordering.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
John, thank you for the investigation. 

Is there any way to flag this to the powers to be at Adobe?  It's been affecting me since LR8 so I haven't been able to upgrade.  I've opened tickets in the past but they just sit with no action.  Even after I get feedback from Adobe that the issue is reproducible.
Photo of StormyTheCat

StormyTheCat

  • 40 Posts
  • 4 Reply Likes
Have you been able to confirm the sort order for your suggestion of moving the contents of a folder to a new folder as a workaround?
Photo of Robert Somrak

Robert Somrak, Champion

  • 558 Posts
  • 187 Reply Likes
StormyTheCat said

"Is there any way to flag this to the powers to be at Adobe? "


Adobe is well aware of the ongoing issue as they have acknowledged it several times.  They have more than enough info provided in this thread, especially the detailed info provided by John Ellis, to address the issue.  Unfortunately they have chosen not to do so.  I agree that if you do a lot of custom sorts this would be very irritating issue.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 6991 Posts
  • 1578 Reply Likes
I have added the new notes to our existing bug and brought it to the Product Manager's attention for prioritization. Thanks @John Ellis for the new repro instructions and analysis.