Skip to main content
Adobe Photoshop Family

9 Messages

 • 

162 Points

Sun, Nov 10, 2019 12:00 PM

Lightroom Classic: Batch rename crash due to upper/lower case extension mix

Short version: if you rename/renumber a series of photo's, the MacOS version of LR Classic can crash if your files contain mixed case extensions (notably .jpg and .JPG) and the target names overlap with the original names.
MacOS allows you to have file names like image001.jpg and image002.jpg and image002-1.JPG. The point is that MacOS allows image002-1.jpg and image image002-1.JPG in the same directory. And LR is inconsistent about its upper/lowercase convention.

Scenario that cause LR Classic to crash (with a "uniqueness constraint" on the database used for the catalog):

1. create a directory containing image001.jpg and image002.jpg and image002-1.JPGThe "-1" suffix is essential here, because LR generates that file name suffix internally on a name clash.

2. import the 3 files into LR Classic (9.0, latest build) without renaming. This works.

3. make sure the Grid shows the files in order: image001.jpg ; image002.jpg and image002-1.JPG (use custom order if needed - order matters here)

4. select the three files in LR, and automatically renumber them (Library>Metadata>FileName) with a starting number of 002 and a resulting name like image002.

Expected result: image002.jpg, image003.JPG, image004.jpgActual result: crash and files on disk are image002-1.jpg, image002-1.JPG and image003.jpgSee screenshots below.

What is happening here is that image001.jpg needs to get renumber to image002.jpg.But the name image002.jpg is already taken. So it then tries image002-1.jpg as a temporary name.Normally it will then rename image002.jpg to  image003.jpg. And apparently later rename image002-1.jpg back to image002.jpg
The problem is that this step causes image002-1.jpg and image002-1.JPG to simultaneously exists. MacOS, being a UNIX variant, accepts that. But somewhere in LR, the database seems to store the temporary new file name. And the database says that the name is already taken, because it considers image002-1.jpg and the existing image002-1.JPG identical (as it would be in Windows). Result of the crash:
  1. simultaneous image002-1.jpg and image002-1.JPG in the folder
  2. the original image001 photo now has an "!" sign because there is no valid path to the file in the database
  3. if the upper/lower case problem happened after a series of 50 images, you get 50 or so images with temporary names, and 50 or so list images from the perspective of LR.
Workaround that I did, was fix the file names in MacOS finder to rename .JPG into .jpg. This required manual renumbering in some cases. And then finding the images which had become lost one by one in LR. If the catalog doesn't contain settings / metadata worth saving, it would be faster to remove the images from the catalog and re-import.
Can this happen in different cases? Probably. Anytime you rename to a new file name that already exists but with at least one character in a different case. So say renaming the name of your photoshoot from MynameNNN.jpg to MyNameNNN.jpg is likely to fail if one of the target names already exists.
Practical point: this is the ONLY site where I am going to file this bug report. So if it needs to be somewhere else, maybe an Adobe representative can copy it over.
Q: Likelihood of this happening to users?A: I ran into it twice in a raw when I received files from different sources (mix of .jpg and .JPG),renamed them to create a series, and then had to insert images in the middle of the series. Renumbering gave the problem.

Q: Likelihood that this has been in LR for years?A: Probably, but can't tell.

Q: Does it corrupt the catalog?
A: No. At least the defects in the catalog that this create seem localised to one folder and leave the catalog and folder in a standard state (but unSynchronised).

Responses

9 Messages

 • 

162 Points

10 months ago

Here are the three 200x200px JPEG files I made for testing (_after_ they got renamed by LR). Saves time if anyone wants to see/confirm the issue.

Can anyone with a Mac retest on an older (pre 9.0) version of Lightroom?

Retesting on Windows is also worth doing, but please state on whether the image files were stored on an internal C: type drive or a NAS. In my case: Mac + Synology NAS. For background information: https://en.wikipedia.org/wiki/Case_sensitivity#In_filesystems . I suspect the problem won't occur on Windows, but haven't tried that.






9 Messages

 • 

162 Points

10 months ago

Sidenote on using "Synchronize Folder..." to recover from this problem:

If I let LR automatically fix/Synchronise the test folder containing both image002-1.jpg and image002-1.JPG it only finds one of the two (in my case: image002-1.JPG). So relying on Synchronise to cleanup the damage (if you are ok to lose your catalog edits/metadata) can still cause images that exist on disk to be ignored in LR.

7.9K Messages

 • 

114.4K Points

10 months ago

Do you have your operating system set to case-sensitive or case-insensitive?

Adobe Photography Products
Quality Engineer: Customer Advocacy

9 Messages

 • 

162 Points

Standard settings of MacOS. But see next remark...

9 Messages

 • 

162 Points

10 months ago

Thanks for the reply Rikk.

Turns out that I CAN reproduce the problem when the image files are stored on a case-sensitive NAS drive (Synology DS214 in my case). The NAS is accessed via the AFP protocol (afp://DS214.local where DS214 is the name of the device).

But the problem does NOT occur when the image files are stored on the internal drive of the Mac. This is probably "case insensitive but case preserving".

So, I guess from Adobe's perspective the question is whether this is typical behaviour on a NAS. I suspect that many LR/PS desktop users store their image files on a NAS. Especially if the desktop has say a limited size solid-state drive. Laptop users are more likely on (usually case insensitive) external USB drives?

7.9K Messages

 • 

114.4K Points

10 months ago

Please note that Adobe does not support Case-sensitive Operating systems. 
See: https://helpx.adobe.com/creative-suite/kb/error-case-sensitive-drives-supported.html
and: https://helpx.adobe.com/lightroom-cc/system-requirements.html

While this is file location and not installation as referenced above, I suspect it is the root cause for this issue. 

Adobe Photography Products
Quality Engineer: Customer Advocacy

9 Messages

 • 

162 Points

Thanks for the reply.
"I suspect [case-sensitivity] is the the root cause for this issue". Agree, the tests look pretty conclusive about that.

"Adobe doesn't support case-sensitive operating systems". 
It looks like the software is careful about the install (binary and catalog), and actually does a test to block installation on anything less solid than a dedicated internal hard disk or SSD drive. But the image file storage is indeed a separate matter: the software doesn't check it, so there is no warning. And I, for one, ran using this setup literally for years before running into this problem.

But because the issue leads to a crash, you might want to escalate this to the LR software architects and product managers.

Some ideas on what I would do:

You can't expect typical users to figure all this out. Adobe currently allows users to store their data on a case-sensitive drives (NAS, some USB external drive file systems, network drives, cloud?) without a guarantee that it should always work. Or giving users a warning if there is a risk.

Blocking case-sensitive (image) storage is probably not a real option: too many users have this without knowing it, and it works fine - at least the vast majority of the cases. So Adobe could quietly plug the holes one-by-one. Without a change to the system-requirements (which doesn't really cover this - and probably couldn't summarise the requirement in a simple way without providing compatibility-test software). So I would first fix the bulk renaming with overlapping source/target filenames (because it can give a crash). Then the import (that can today, in rare cases, overlook input image files). And maybe then fix the export of files from case-insensitive storage to case-sensitive storage. Mainly a matter of creating a first test setup and fixing the issues one-by-one. And then retesting on a different type of test setup. And waiting whether you missed anything.

Champion

 • 

5K Messages

 • 

91.6K Points

10 months ago

"Please note that Adobe does not support Case-sensitive Operating systems."

The LR system requirements specify that the volume where LR gets installed must be case-insensitive. But I've never seen a system requirement restricting the location of catalogs or imported photos to case-insensitive volumes. 

In fact, the most recent LR SDK (8.3) includes a change to better support case-sensitive volumes:

    "2. LrCatalog:
    - There is a change in catalog:findPhotoByPath() API signature.
      The API now takes an optional boolean(true/false) parameter which indicates whether
      the user wants to do case-sensitive or case-insensitive search.
      eg: catalog:findPhotoByPath(path,true) indicates case-sensitive search for path."

However, over the years there have been several bugs reported involving differing case of what's supposed to be identical filenames (e.g. .xmp and .XMP), and some of these bugs have left the catalog in an inconsistent state.

In general, I recommend against trying to use case-sensitive volumes. (LR isn't the only app that can choke on them.) So you might find fewer hassles in the long run by reconfiguring your Synology to be case-insensitive.

Also, note that the AFP protocol is deprecated by Apple.  With Synology devices in particular, many people have reported in these forums solving problems by switching to SMB.

9 Messages

 • 

162 Points

I am looking into switching from afp protocol to Samba. Somehow I have one NAS on Samba and the other on AFP. Thanks.