Lightroom Classic: Importing Files into a folder with the file system date rather than metadata date?

  • 1
  • Question
  • Updated 3 months ago
  • (Edited)
I had an issue with lots of repeated files and in order to fix it I wrote my own script to calculate a hash on all files and then check which are the same. This worked very well and with it I found that I have the same exact file imported into different folders...

In my setup all the files enter those folders through an import on Lightroom.

When I digged further I found out that:

1) the files have the same MD5 hash
2) they are exactly the same
3) they have the same EXIF data and it matches reality (CREATED DATE)
4) They have a different date on the filesystem
5) The file was imported to the date the file has on the filesystem

I am very surprised with this. When we move files around the file date changes but it seems that Lightroom is importing based on that and not on the METADATA inside the file.

This is happening for .JPG 

Is this normal behaviour?

I have the latest release of LR Classic 9.2
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
  • confused

Posted 3 months ago

  • 1
Photo of Brian Pierce

Brian Pierce

  • 73 Posts
  • 24 Reply Likes
If you have multiple physical copies of files in different folders on your computer and add the to lightroom, then lightroom will import all of them - they are different files, even though they have the same hash and metadata - this is normal and to be expected. 
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
That's not the case. I "move" files into this directory structure by importing them. They land in the wrong folder date... It's not normal and not to be expected. What's the logical behind your comment of it being normal and expected?
Photo of Just Shot Me

Just Shot Me

  • 531 Posts
  • 144 Reply Likes
If you MOVE them during the import process then they are not from a camera Memory card.
Are you Really Moving them or do you have Copy selected?
Although LR is not supposed to import files that are already in the LR catalog that system can be fooled fairly easily. I think you have hit on one of the reasons LR can get fooled about duplicate files.
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
Mostly I copy them directly from the SD/CF cards and in that case I haven't seen any issues. Where I do see issues if when I move a bunch of folders with pictures into the same drive and then ask Lightroom to import and MOVE the files into the folder structure I use to keep my master files.
Photo of Just Shot Me

Just Shot Me

  • 531 Posts
  • 144 Reply Likes
If you move files outside of LR and then use LR to import them from the new location you will end up with duplicated files in the LR catalog.

LR is based on having ALL your images imported into the catalog file. Not just Some of them. Then In LR you manage them. Whether that is moving them into different folders or combining one folder full of images into some other folder or even Editing them.
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
I am not worried with the duplicates, I can handle that. I am worried than LR importing them into wrong folders. Right folder being the one with the appropriate date.
Photo of Erich Schlaikjer

Erich Schlaikjer

  • 41 Posts
  • 7 Reply Likes
Are you importing directly from a phone on a USB cable? I find that that often gets dates wrong, at least on Windows. So I copy to a temp folder on the hard drive using the OS, then import from there, which works properly. I do not have this problem when using a card reader.
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
I am running on mac. For the phones I tried to import directly from Lightroom but it never works well, I end up with pictures that weren't imported for unclear reasons and thousands of videos (really just the .MOV From the Live Pictures) that are not imported. I now found the best way is to copy the files from the iPhone to the mac using  "Image Capture" which works well and then import the folder with the option to move into lightroom. Whatever fails (the videos) I will then use a home made script to place them into the right folder using the CreateDate from the EXIF and then sync the folder from Lightroom (guess what? The movies get imported and no issues with the video files anymore!)
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
By the way, the script I used for this was just a dumb bash shell script:

for f in *.mp4;
do
echo Moving $f
mv $f "/Volumes/2017-2019/Pictures/"`exiftool $f  | grep "^Create Date" | awk -F: '{ print $2"/"$2"-"$3"-"$4 }' | awk -F' '  '{print $1$2}'`;
done

and the same for the .mov
for f in *.mov;
echo Moving $f
mv $f "/Volumes/2017-2019/Pictures/"`exiftool $f | grep "^Create Date" | awk -F: '{ print $2"/"$2"-"$3"-"$4 }' | awk -F' ' '{print $1$2}'`; 
done
(Edited)
Photo of Sunil Bhaskaran

Sunil Bhaskaran, Official Rep

  • 486 Posts
  • 178 Reply Likes
Thanks for letting us know. We need some of those files to produce the issue.
I have sent you an e-mail off-list.

Thanks,
Sunil
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
Thanks for reaching out. I sent you an example.
Photo of John R. Ellis

John R. Ellis, Champion

  • 5123 Posts
  • 1452 Reply Likes
LR has always had problems renaming and moving files based on capture date. Some of the known issues:

- The files don't contain a capture date in the industry-standard field EXIF:DateTimeOriginal. 

- The files are imported via a USB cable.

- The files are imported via the Devices panel in the left column of the Import window.

Note that what Exiftool calls EXIF:CreateDate is not the same as the capture date (EXIF:DateTimeOriginal).   EXIF:DateTimeOriginal is when the shutter was pressed (capture date). What Exiftool calls EXIF:CreateDate is what the EXIF standard calls DateTimeDigitized, and it represents when the image was converted to digital form.;

Most (but not all) cameras and software set DateTimeDigitized (CreateDate) to be the same or nearly the same as DateTimeOriginal.  Some scanning software sets DateTimeDigitized to the date the image was scanned, and the user is then responsible for setting DateTimeOriginal to when the shutter was pressed.

When LR can't read EXIF:DateTimeOriginal for some reason (either because it is missing or nonconforming to the standard), then for capture date LR will use EXIF CreateDate, EXIF:DateTime (the last time an imaging app modified the image), or the file system's date modified. But when LR uses one of these other fields, it can behave inconsistently with respect to what it considers the "capture date".  Adobe has never been able to remove all of these bugs.

If you upload one of the images that LR misfiled into a folder, upload the image to Dropbox or similar and post the sharing link here. Then we can put it under the microscope and see what precisely is going wrong and which bug report this belongs with.


Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
Hi John, that's very good information. I have an example I can share but I would rather not share the dropbox link publicly since it happens that the subject of the picture is a minor totally unaware of any of this. If you can send me an email to paulo . jorge . pereira @ gmail.com I will share the dropbox link with you.

I went back to check, the JPG in question have both fields, they match and they are correct and still one of the pictures ended up on a different folder and my guess is that just because the date on the filesystem is different.

The file is actually an export from when I used to run LR on Windows.
(Edited)
Photo of Paulo Pereira

Paulo Pereira

  • 10 Posts
  • 0 Reply Likes
I just did a check and I have 7886 files where the folder they were placed doesn't match the metadata of the file. It happens for JPG and HEIC files but never for CR2 files...
Photo of Just Shot Me

Just Shot Me

  • 531 Posts
  • 144 Reply Likes
And all of these files are from your iPhone? If that is true then it is a problem with the phone placing the wrong date, or the right date, in the wrong place.
Photo of John R. Ellis

John R. Ellis, Champion

  • 5123 Posts
  • 1452 Reply Likes
Paulo sent me some example files, one of which got imported into the wrong dated folder. As he had already determined, the EXIF metadata of the two files was identical, both with the same EXIF:DateTimeOriginal field, and both imported the correct dated folder in my LR 9.2.

You've said you now use Mac Image Capture to import from the iPhone to your disk and then from the disk into LR. Is it possible that these affected photos were imported via USB cable?  If not, then your symptoms don't seem to match previously reported bugs.