Lightroom Classic: loses connection with files after folder is renamed

Thanks Rikk. Mostly the fix introduced in 10.3 has worked for me, but twice it has failed in the exact same fashion as before. Soft errors are always hard to nail down!

Thank you. The fix seems to work sometimes! Just seems a bit odd that it ‘loses’ some files occasionally

Why conversations get merged: 

https://feedback.photoshop.com/conversations/lightroom-classic/lightroom-auto-import-to-folders-organized-by-date/5f5f44f14b561a3d422ab0ba?commentId=5f5f48984b561a3d422bdf45 

@bill_3305731

No, that's not what users are experiencing in this case. It's a consistent and reproduceable bug with the latest version on macOS. Renaming a folder following import causes the contents of that folder to be falsely reported as "missing". They are not. And the filenames are unique. That's not an issue.

I always rename the folder in LR, and I always rename files during import into LR. I have been doing this since LR v3, and have never had a problem in the past. There are no files already in the catalog with the same names as the ones I've just imported.

The current LR behavior I've described in my original post is a bug.

Are you renaming the folder before the import is complete?

Never. As I stated in my original post:

"I usually rename the folder with something more descriptive, just after the upload is complete. For example, 20210403_family-gathering."

I am not able to replicate your issue. 

Information about your system might be helpful

Mac or Windows?

OS version?

Location of Catalog?

Location of folder where files reside?

As the Lightroom Classic Import dialog will allow you to custom name a folder for the import, I have to ask if you have a reason for bypassing this and doing an after the fact rename. 

Mac OS 10.14.6

I use a 2018 MacBook Pro, plus two external hard drives. One of the drives contains the Lightroom catalog. The other drive contains my "photo library" — all of the image files I've imported into Lightroom.

As for why I rename a folder after importing, the simple reason is that's the way I've always done it. Sometimes I don't decide on a custom name until after I've browsed through the images. Would it be possible for me to choose a custom folder name prior to importing the files? Yes, but why should I have to?

Without additional information regarding step-by-step instructions for reproducing, I am not able to replicate the behavior and, by extension, unable to identify this as a bug. 

I would recommend your resetting your preference file to see if some cruft has accumulated on your local machine and is causing this behavior. 

Reset Procedure:

1. Close Lightroom.

2. Hold down [Alt/Opt]+[Shift] while restarting Lightroom.

3. Overwrite the Preferences when prompted by the dialog.

4. Close Lightroom.

5. Restart Lightroom.

I followed your reset procedure, and the problem I originally described persists. As Alexander noted, if I navigate away from the renamed, problem folder by clicking on another, previously imported folder, and then return to the renamed folder, the question mark disappears and it appears that Lightroom has "found" the renamed folder. But each and every file in the renamed folder still displays an exclamation mark, and if I attempt to edit any of them, in Develop mode, the Histogram panel reports that the photo is missing. If I quit Lightroom and relaunch, all of the missing photos in the renamed folder are now found. None of them display exclamation marks, and I can edit them in Develop mode, as usual.

You may not be able to identify this as a bug, but as far as I'm concerned it's a bug.

My current workaround is to edit any newly imported photos before renaming the folder, and then rename the folder just before I quit Lightroom.  

I have attempted this 3 more times on Mac 10.15.7 and LrC 10.2. It performed as expected each time. 

I tried Copy and Copy as DNG and both worked with no issues. 

Perhaps more detailed step-by-step instructions are necessary since I cannot duplicate the issue?

Here are the steps I currently follow to import image files from a camera memory card (Sandisk SDXC - 64GB).

1. Launch Lightroom 10.2 Release, Build [202103041821-226a1211]
2. Remove card from camera and insert into IOGEAR SD card reader USB stick.
3. Insert IOGEAR SD card reader USB stick into an Anker USB hub, which is connected to a CalDigit TS3 Plus Thunderbolt 3 Dock, which is connected to a Thunderbolt 3 port on my 2018 MacBook Pro, running Mac OS 10.14.6. 
4. Lightroom recognizes the memory card and automatically opens an import window. I have "Copy as DNG" selected as my default.
5. Rename the files, choosing "Custom Name - Original File Number" from the Template drop-down menu. ("Extensions:" Leave as is)
6. Add keywords.
7. In the Destination panel, "Into Subfolder" is not checked. In the Organize drop-down menu, "By date" is selected. The Date Format is 20210407.
8. Image files are imported into my "Photo Library" external drive, and folders are organized chronologically by Year>Month>Day. So, for example, the 2020 folder contains 12 "month" folders, from January through December, and each month's folder contains individual folders for each date that corresponds to a photo shoot.
9. After the image files are fully imported, I usually browse through the images in the folder before renaming it to something more descriptive, although if time is short, I rename the folder just after import, and I always rename it in Lightroom, never in the Mac Finder.
10. If circumstances require it, I'll edit photos in this session. Otherwise I may just cull and rate photos to prepare for a future post-processing session.

I've followed these instructions as precisely as possible and still get the same result. Folder is intact, images are intact, nothing needs to be repointed/found/located.

In your step 8 - if you substitute your local drive for the external drive does the issue happen there also?

I just imported 10 image files to a folder on my computer's internal, SSD drive. After the import was complete, I renamed the folder, in Lightroom, and the image files did not go missing. The issue I originally described did not manifest.

However, this is not my standard workflow, and I will not be importing images to my laptop's internal drive, renaming the folder, and then dragging it to my preferred location on my external drive, just so that I can avoid this annoying issue.

Prior to the last two releases of Lightroom, I have always been able to follow the steps I listed, above, without issue. Again, as far as I'm concerned, this issue is a bug.

At this point, I've checked multiple times on a single 10.15.7 machine. I've also checked an M1 with Big Sur and a Windows 10 machine. I cannot reproduce this on any of the machines. 

Given the test you just ran for me, I would start checking settings for your EHD in Mac OS.

Other things to try: 

• Are you plugged into power or on battery when this happens? 
• Is the drive set to sleep?
• Have you tried using a different EHD?

At this point, there are no other reports of this behavior that I've seen and I cannot reproduce. That leads me to believe it is system-specific to your machine. 

You might try a clean reinstall to see if your system was in a bad state when you did the previous update.

Clean Lightroom Install Procedure

1. Close Lightroom
2. Restart the computer
3. Use the Adobe Creative Cloud App to uninstall Lightroom
4. Restart the computer
5. Install Lightroom via the Creative Cloud App without launching any other programs.
6. Restart the computer
7. Launch Lightroom
8. Wait 5 minutes

• My laptop is always plugged into power when I'm using Lightroom.
• In my Energy Saver preferences, the "Put hard disks to sleep when possible" option is unchecked.
• I don't have a different EHD.

There is at least one other report (which was merged into this conversation) at the top of this conversation, in which another Lightroom user, Alexander, claims to have what sounds like a nearly identical issue. And a new post from David, below, describes a similar problem.

I will consider a clean install of Lightroom, when I have the time.

• In my Energy Saver preferences, the "Put hard disks to sleep when possible" option is unchecked.

For some hard drives, this is built into the drive and they will go to sleep independent of commands from the OS. Drove me nuts until I discovered what was going on. You'll need to research the specific brand/model. For the drives that are doing that on my system, this feature cannot be disabled. 

I can reproduce this bug consistently across several of my own and my client's Macs. I am running the latest version of Catalina and it is consistently throwing this error.

It is exactly as described. Following import, the renaming of a folder results in that folder's contents falsely reported as being missing. Any attempt to reconnect the images results in the error message indicating it is already associated with another photo in the catalog.

Quitting and restarting fixes the issue immediately.

Please address this bug. It is indeed repeatable. Happy to provide any data you need.

Current test rig:

iMac 27" Core i9
macOS Catalina 10.15.7

Lightroom Classic 10.2

Have done preference reset and more.

Have checked disk access permissions in Security and Privacy System Preference.

Bug remains.

Don't enable a "Power Schedule" in your Synology Control Panel/Hardware & Power/Power Schedule. This way it won't turn off ever.

Carlos, if drives have sleep enabled in the drive, it cannot be disabled. It's hardware. Drove me nuts until I figured out what was going on. I wouldn't think that NAS drives are built this way but it is common for normal desktop drives. Some people try to save money by using desktop drives in their NAS, perhaps that's not a good idea?

For me it's only like that since v10.2.

Trying to find a sure fire way to reproduce the behavior.  To summarize, the NAS is off 9 hours and then come back.  The folder that was selected when the connection went off, show connected after the reconnection, but all the raw files show missing.  I tried shutting off the NAS for 1 hr and then 2 hrs but no missing files after reconnection.  The interesting thing is the folders that were selected yesterday and there were 4, are all showing connected after the test of shutting down for 1 hr and then 2.  All the other folders are greyed out during disconnect.  The other strange thing is that of the 4 folders that were selected before the night shutdown, two of the folders only have the first image missing and the other 2 have all the images missing.

Because I've lots of miscellaneous hardware lying around, the first thing I'd do is attach a hard drive to a USB port with a powered cable. I wouldn't use a case with any electronics. Then see if the problem occurs with this setup in place of the SAN.

If this setup works then the odds are that there's a driver problem for the SAN. While possible, it's unlikely to be a hardware problem with the SAN. 

People who use non-SAN drives in their SAN do experience all kinds of strange problems but the SAN usually reports them. SAN failures are rarely subtle. 

Using MOVE during import does NOT move the folder, only the photos in the folder to a new destination in a different folder as designated in your import destination dialog.

@Logan31639

"I imported a batch of photos to the second EHD, and after the import was complete, I renamed the destination folder in Lightroom and there was no problem."

You've proven that your primary hardware is defective. Time to replace it. 

I strongly disagree. My ability to import photos to a second, older EHD, and rename an imported folder in Lightroom without issue doesn’t prove anything, and that includes proving that there is no bug in Lightroom v 10.2.

Are you serious? You need to take a class in logic. "A" fails. "B" works. So the fault must be with "C". 

You need to take a class in science. There are these elements called variables, which you don't seem to be familiar with. There may well be different variables at work here, which you clearly haven't taken into account with your simple logic statement.

You have ample evidence that you have a hardware problem and zero evidence that the problem is with Lightroom.

I have some evidence that I might have a hardware problem. That in no way constitutes proof.

But you have zero evidence that you have a software problem. 

Logan, Bill is a troll. Just stop feeding him, please.

Alexander doesn't like answers that differ from his preconceived ignorance. He blames the lack of world peace on Adobe. 

@Logan31639 using a RAID is much different than just a USB hard drive. Is this hardware or software RAID? Have you installed any firmware updates? 

We have multiple people who all began to see the same problem at the very same time here. I doubt that's coincidence.

Folder operations in LrC have received improvements in v10 (changelog entry). That's related to what people are having problems with now.

At the same time, there's probably also some Mac users who held out on v10 until the "slow UI" problem was fixed in v10.2, so this is only beginning to show for more people now.

Have you noticed whether the number of photos in the folder has an impact? The catalog keeps track of the the full folder / filename for each photo so it has to be updated for every photo in the hierarchy. If you externalize all updates then the photos and XMPs have to be updated as well. 

I have tried with single/multiple images, and with and without subfolders. Always does the same. The folders are being correctly manipulated, as shown in Finder and being shown correctly after quitting and relaunching. No files/folders are being moved / renamed outside of Lightroom, so somehow the catalog is losing sight of the folder during the rename process.

Can you provide screenshots of the Finder window showing the full path to one of these 'missing images'

Also provide a screenshot of the Lightroom folders panel for the same image. 

I wonder if you have a capitalization problem between the catalog and the OS?

https://imgur.com/a/wIfzt6X

Here are screenshots of my problem, in the first image you can see from Library view that there about 6k photos missing from folder Klijenti. One of those photos is one that is shown.

In the second image, you can see a successful export even though it is said to be missing. Alongside is file info.

When I right click on the image -> Show in Finder, correct location on disk opens in Finder and image is selected. Folder in folders panel also correctly opens, with nothing being greyed out. 

I also tried right click on the folder -> Update Folder Location. Lightroom opens the already correct location in Finder, and when I click on Choose nothing happens.

Thank you for posting those but unfortunately they don't provide the information I need. 

I really need to see the screenshots of Finder showing the full path rather than Get Info showing me the same. 

In Lightroom I need to see the Folders Panel with the folder where the reported missing image resides - not the Missing Photos system collection. 

Neither of the screenshots help diagnose the issue. Is it possible for you to post those I've requested?

I am sorry, I misunderstood you.

https://imgur.com/a/SwHcCyS

Does this provide all the info you need?

Thank you. Yes that is a better view of what is going on. 

As a test, can you create a new Mac OS user and test the same catalog from the new user account to see if it behaves the same?

That took some time - creating a user account, giving access etc.

The problem still persists, everything is as before. I even tried copying the original catalog and opening the copy, but the files are still missing.

What happens if you use the Find All Missing Photos in the Library menu?

Then you will get a new filter line in the catalog panel, showing all offline photos.

https://imgur.com/a/dFFXuJt

Still nothing.

The first photo was when I went Library -> Find All Missing Photos, pop up with No missing photos appears. 

Second photo is when I right clicked on folder Klijenti -> Synchronise Folder -> Show Missing Photos, about 6k photos show up in catalog filter Photos Missing From Folder "Klijenti"

Could the Missing photos be duplicates?

Nope, they aren't duplicates.

Ok, here's an update:   I opened up another Lightroom Catalog which I don't use very often and it works fine with changing folder names on all external drives.   So it appears that the bug is catalog-specific.    How do we fix this Adobe?   Let's see what happens if I optimize the catalog that's working...  I'll post a reply if anything changes after optimization.  

Optimization didn't break it.   I went back to my latest lightroom catalog backup of the catalog with the issue, from about 3 weeks ago... it's got the bug.

How old is the catalog where this happens? I'm asking because mine is essentially "ages old" (begins in 2007) and I also went from Windows to Mac with it, and the photos and folder structure in it. Actually @Rikk had that catalog in his hands a while ago to do some repairs on it (on a different issue).

My EHD is a NewerTech Guardian Maximus enclosure RAID-1, with two 3 TB Toshiba hard drives that I purchased from OWC. I've changed a few variables since discovering the problem, including upgrading from Mac OS 14.6 to 15.7, rebuilding the EHD's directory with DiskWarrior 5.2, and swapping the EHD's connection to my 2018 MacBook Pro from an external, CalDigit TS3 Plus Thunderbolt 3 dock to directly in one of the MBP's Thunderbolt ports. I also performed the recommended "Clean Lightroom Install Procedure." None of these changes have resolved the problem, which I still consider to be a bug, regardless of whether a single Adobe administrator can't reproduce the problem. It will only be further investigated by Adobe engineers, and defined as a bug that needs to be addressed, if enough end users complain to hurt the company's image, or profits. If there's any good news, it's that more people are speaking out about the same problem, and that's not a coincidence. Also, for however many people are protesting about the same issue in this forum, we can be sure that others are experiencing the problem who will not register a complaint in this forum.

Have you tried creating a new catalog, adding a few images, and testing to see if the problem occurs with the new catalog?    I have one older catalog that works fine, but the rest of them have the same bug.  However new catalogs seem to work ok, though I've not tested a new catalog with a LOT of images.   I personally don't want to have to create an entirely new catalog and re-import all of my images to get rid of this bug. 

I've not tried creating a new catalog yet. I'm not unwilling to try this, but I'm not prepared, at this time, to invest time and energy into re-importing roughly 92 thousand images into a new catalog. And I can't know, with the current available data, whether creating a new catalog would be a permanent fix for this issue. For now, I'm going to live with the bug by adjusting my workflow so that I'm renaming folders just before I'm ready to quit Lightroom. If I learn, later, that creating a new catalog and re-importing files permanently eliminates this bad behavior, then I'll seriously consider following that path.

I determined last night that creating a new catalog and importing all of your images doesn't fix the problem.   It seemed to work when I had just a few images, but when I imported significantly more, the problem came back.    My only solution so far has been to make a copy of the only catalog I have that doesn't have this bug and add/remove images from there.    FYI my working catalog has about 70000 images and my new copied catalog now has about 75000 images.  

Thanks Rikk. I can provide any data points you need regarding this issue. I have seen it across numerous client installations.

Thanks, I will let the team know. 

I have the exact same problem with the following configuration:

MacBook Pro 15' macOS Big Sur 11,3,1

2,4 GHz Intel Core i9 8 cœurs

32 Go 2400 MHz DDR4

LrC 10.2

LrC catalog path (SSD inside) :
Macintosh HD/Users/Milek/Pictures/Lightroom

At your disposal if you want to upload my catalog to reproduce the bug.