Lightroom: hit a folder limit and unable to select the newly imported folder.

  • 2
  • Problem
  • Updated 3 months ago
  • Acknowledged
  • (Edited)
Lightroom bug - hit a folder limit and unable to select the newly imported folder.452497 photographs across many drives in one catalog.The S drive has 214341 photos. It's the largest drive and so had many more folders than the previous drives.S:\Master Photos\ contains ~1633 folders. I know that's a lot but that is how her filing system was started. Those are the sequential numbers from 2220 - 3853 but there may be a sub folder or two or deleted folder in there so that's why the number is ~ approximate.Folders are labeled "sequential number"-"date" so 3853-20150316Adding one more folder caused Lightroom 4 to not allow the last folder to be clicked on and not show any images. The T:\ drive (offline, collapsed and below the S:\ drive) was collapsed and the label was overlaid on top of the "Collections" section label. Updated to latest version of Lightroom 5 via her new cloud account, had it rebuild the catalog and the same problem occurred.Created a sub folder S:\Master Photos\2012 and moved 15 of the folders into that folder. The 2012 folder was expanded so the list was actually one row larger and the last good folder wouldn't open. I collapsed the 2012 folder and now all the bottom folders were accessible.It appears there is an object limitation or a fixed array or something that is causing the folder list to become finite. She said a similar thing happens on her keyword list until she collapses some of the groups but I didn't check that out.
Photo of SCOTT MCDANIEL

SCOTT MCDANIEL

  • 2 Posts
  • 0 Reply Likes

Posted 3 years ago

  • 2
Photo of Steve Sprengel

Steve Sprengel, Champion

  • 2643 Posts
  • 337 Reply Likes
This may be an operating system or programming language limit not something LR is doing different than how it was designed, which is what the term "bug" normally means. Any system, physical, or virtual, is going to have some design limits, and the only question is whether those limits will be encountered when using the system. The elevator in the building where I work has, maybe, a limit of 2000 pounds or is 6x6 feet square or whatever the actual limits are, so if too many people want to use the elevator at once, the limits may be exceeded and the solution is to split the number of people into smaller groups and use multiple elevator trips.

If I'm understanding what you're describing, it seems there is some sort of display-items limit, not an actual folder limit since collapsing a folder will let you see more, further down on the list.

I would suggest a simple reorganization of the physical folder structure, either breaking it up under 2 parent folders, a 2000 and a 3000, then move all the 2xxx folders under the 2000 and the 3xxx folders under the 3000 which should double the capacity but would require having to scroll down through 1000 folders with the other one collapsed, which still might be too many, so perhaps a more useful set of parent folders would be ones that have the year-month followed by the first sequence number by that month, so for your example folder, 3853-20150316, the parent folder that would contain it an all the other folders for March 2015 would be 2015-03-3nnn where 3nnn is not the literal folder name, but the lowest sequence number of the earliest March 2015 folder.

The parent folders would have both the month and first sequence number for the month so it'd be easy to locate the correct parent folder when either the date or sequence number is known. I have chosen to put the date part first instead of having the sequence number first, because the yyyy-mm is more of a regular pattern without any gaps so easier to scan down with your eye, and would also allow non-sequence-number subfolders inside if the organization is ever changed, it is unlikely that dates will never be part of the organization, so I put them first in the folder name structure. One more reason to put the date, first, is if you eventually find that older dates are rarely if ever needed, then another layer of parent folders can be made, but this time just yyyy-firstseq so an entire year's worth of subfolders can be collapsed at once instead of having to collapse individual months to get the list short enough to scroll to the bottom of.

There may be some other partitioning of subfolders in a few parent folders that makes sense depending on the subjects in those photos, but these ideas are what come to mind seeing that the existing folder names contain only a sequence number and date.
--
Basically I'm saying LR may not be able to accommodate the existing folder structure so the folder structure may need to be changed. Even if LR can be changed, it would take some months or years, perhaps, so what will you do in the mean time? Folder reorganization seems reasonable as a temporary if not permanent solution.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
The Keyword List panel on Windows LR does indeed have the same limitation on about 1600 items, at which point it fails ungracefully. The problem does not occur on Mac. As Steve said, the workaround is to introduce hierarchical folders and keywords.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 3808 Posts
  • 715 Reply Likes
Confirming what John and Steve have already said:

There is a vertical limit on how much data can be displayed within a panel. The limit doesn't affect the number of folders, just the number that can be displayed. As John has indicated, the workaround for now is to have a more hierarchical folder arrangement to mitigate the number of same-level folders.
Photo of SCOTT MCDANIEL

SCOTT MCDANIEL

  • 2 Posts
  • 0 Reply Likes
Steve, thanks for the info. I'm already having her move into "Year" folders to keep the number of sub folders in the 300-900 range. She'll probably move to something like a Drobo or Synology system where all her images will show up on one drive so the "year" folders will have to be in place before that.

The report was more for the developers/documentation to be aware of the limitation as they probably didn't build at test case with 1600+ folders on an 8GB Windows 7 machine to see what happens. When that limit is hit, Lightroom eats up all available memory and doesn't gracefully close. Maybe a better error trap to make the user experience more pleasant, although I don't know what they would check for to trigger an error if there even is an error. Maybe some documentation on "possible system based limits" in the best practices section. Documentation is cheap. I know this is an edge case but it may help others if they hit this problem.

5TB drives are out and some photographers might not have a good folder structure.

If you would like to remove the word "bug" in the first line of the first post be my guest as this appears to be a system limitation.

John, thanks for the keyword comment. I wonder if it is the way Mac handles memory or your Mac has more memory than the Windows machine I was working with.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
"The report was more for the developers/documentation to be aware of the limitation"

Adobe is already aware of the limitation on Windows -- it occurs much more oten with keywords than folders: https://helpx.adobe.com/lightroom/kb/.... There have been many reports of people bumping into the limit with keywords, but this is the first one with folders I've seen over many years.

The Mac and Windows implementations of LR have different implementations for the lower layers of the user interface, and it appears that the Windows implementation of a scrollable list of items has a limit of about 1600 items. It is sloppy programming that the limitation isn't handled more gracefully, much less that there is a limit at all. There's nothing inherent in Windows that imposes such a limitation.
Photo of john beardsworth

john beardsworth

  • 1041 Posts
  • 239 Reply Likes
" There's nothing inherent in Windows that imposes such a limitation. "

I understood that the problem is because Windows has (had?) an approx 32000 pixel limit on the size of a single control, and that Adobe haven't coded the UI to work around it.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 3788 Posts
  • 713 Reply Likes
That is my understanding as well, John.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
It's true that a low-level Windows control can have maximum height and width of 2^15-1 pixels. But nothing stops an application from coding its own scrolling controls, using a low-level window whose height is less than the screen height (or using a library that has done this). Any Windows application that has to display a very large list has to do this (and most do). For example, Windows File Explorer can display lists of files that exceed 32K pixels. Even LR does so for Library grid view, which can far exceed 32K pixels in height.
Photo of Daniel Wiercioch

Daniel Wiercioch

  • 4 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Keyword list is truncated in LightRoom CC.

I've spent a lot of time adding keywords to my photos. Well over 200000 photos. Probably several thousand hierarchical keywords. When I look at my list of keywords in the keyword panel, the list is truncate -- that is, I see about 90% of them, but near the end, the keywords stop showing and I can't scroll any further. I know the keywords are still there, because I can use them in searches. I just can't see or select them any longer. This was not a problem in Lightroom 5.
Photo of Daniel Wiercioch

Daniel Wiercioch

  • 4 Posts
  • 0 Reply Likes
Thanks for merging my thread in here. When I collapse some of my hierarchies as suggested, that allows me to work around the problem for now. Hopefully, Adobe addresses the problem at some point.
Photo of Daniel Kennedy

Daniel Kennedy

  • 1 Post
  • 0 Reply Likes
We supply a patient photographing system and use Lightroom as the DAM. The majority of our clients on Windows hit the list limit for folders as each tethered session creates a new folder. We have a system for splitting into hierarchy through a plugin but it is inconvenient and it would be great if Adobe could fix this.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
Every Windows application with large scrolling lists  has to cope with the maximum size that Windows imposes on low-level windows (32K pixels) by repainting the windows as necessary. Most applications do that by using commonly available libraries of user-interface controls that do the work for them.

But Adobe took a shortcut when they implemented LR Windows, choosing to implement their own controls but failing to complete the work. They did it for Library grid view but not Folders, Collections, Keyword List, and other panels. And they've never owned up to taking that shortcut.  See here for details: 
https://feedback.photoshop.com/photoshop_family/topics/bug-that-causes-collections-and-publish-servi...