Lightroom: No suspected duplicates when imported from catalog and some imported photos don't show in Library

  • 7
  • Problem
  • Updated 3 years ago
  • (Edited)
I freequently use å laptop in addition to my computer at home. Once, I imported a group of photos from one shoot to the laptop, adjusted them in Lightroom. Then I eksported them as a Catalog, to get the adjustments over to my home computer, where I imported the photos from Catalog.
Next time I connected the camera CF-card, the previous photos could be imported again. Lightroom did not suspect duplicates the same way as when photos are imported from the card. How can that be?

Also, a friend of mine were importing av portrait/landscape mix of scanned photos, but all the portrait photos did not show in Library. How can that be? She sent me a few, and I did the import without any problems, std jpeg files and no errors.

Regards
Arnold
Photo of Arnold Hoddevik

Arnold Hoddevik

  • 53 Posts
  • 10 Reply Likes

Posted 7 years ago

  • 7
Photo of Jeffrey Tranberry

Jeffrey Tranberry, Sr. Product Manager, Digital Imaging

  • 14190 Posts
  • 1787 Reply Likes
When you imported the files the first time and exported the catalog and imported them again, did you modify the file names?
Photo of Jim Burgess

Jim Burgess

  • 87 Posts
  • 11 Reply Likes
This is a problem I have also observed for at least the past 2 or 3 versions, maybe more. I haven't had a lot of time to test it, but since Arnold started the discussion, here's my observations.

The problem is reproducible on the Mac version with this scenario:

On a Macbook Pro, import a set of images from a memory card to a folder named "~/Pictures/2011/11-29.
Rename the folder to "11-29 Anyname".
Export the folder as a catalog, including the negative files (as they are referred to).

Copy the folder to a Mac Pro.
Do a File Import from Catalog with file handling set as Copy Photos to New Location. The new location is "volume/2011-2012/CategoryName".
The photos are imported properly into "volume/2011-2012/CategoryName/2011/11-29 Anyname".
Note that none of the original filenames have been changed.

Now import the same memory card that was used on the Macbook Pro.
All of the images on the memory card are available for import. And it does't matter if New Images is selected, or if "Don't import suspected duplicates" is checked.

So it appears that if a set of images has been imported from a catalog, then they are not treated as duplicates in a subsequent import of the same images from the original memory card.
Photo of Arnold Hoddevik

Arnold Hoddevik

  • 53 Posts
  • 10 Reply Likes
I allways change filename when importing from the camera/card, also this time. When I exported the photos to Catalog and imported them from Catalog on the other computer, I did not change names.
Then, next time I connected the same card with the same photos on, there were no suspected duplicates.

When importing directly from the card/camera, this works completely different. I always change filenames using my own template preset. Next time I connect the card, Lightroom always discover suspected duplicates, even thou the file names in Lightroom and the card are different.

So I bouth wonder what the mecanism is, and why it works differently.
Photo of Arnold Hoddevik

Arnold Hoddevik

  • 53 Posts
  • 10 Reply Likes
About the "lost-photos-issue". Regarding to Lightroom, this is solved. The mix of landscape and portrait images imported were digital, but was copied to a CD from an old computer. However, but peculiar enough, the portrait photos were missing Capure date. Don't ask why, because that is a well hidden mystery so far.

The only date Lightroom could find on the portrait photos, were edit date, three years after capture date. So Lightroom sorted the photos nicely in different folders, landscape photos in folder 'capture date' and portrait photos in folder 'edit date' (three years after capture date. I found this by looking at latest import and Show in Explorer. - Nice.
Photo of Beat Gossweiler

Beat Gossweiler

  • 238 Posts
  • 38 Reply Likes
What Jim Burgess describes can be easily followed in the AgLibraryFile.importHash column the catalog. This is an example when performing
Import into Catalog1 -> Export as Catalog to Catalog2 -> Import from Catalog into Catalog3:


  • After first import in Catalog1:
    143494041:WR9H0028.CR2:15232206

  • In the exported Catalog2:
    143494041:WR9H0028.CR2:15232206

  • After Import from Catalog in Catalog3:
    NULL



Beat
Photo of Beat Gossweiler

Beat Gossweiler

  • 238 Posts
  • 38 Reply Likes
Actually it is the Import from Catalog function which looses the importHash in this case, which then leads to non-detection of duplicates upon subsequent imports, since the importHash does not match.

Beat
Photo of Arnold Hoddevik

Arnold Hoddevik

  • 53 Posts
  • 10 Reply Likes
Thanks a lot. Then this is a part of the characteristics of the import routine. Not a problem to me. I just noticed it, and thought it might be interesting, but is it something Adobe is going to change?
It will be interesting for me to know, because I have written a book about using Lightroom in Norwegian, and I also am instructor at Lightroom courses.
Photo of Beat Gossweiler

Beat Gossweiler

  • 238 Posts
  • 38 Reply Likes
I would consider it a bug, that the importHash is set to NULL during an Import from Catalog.

Beat
Photo of Robert Kiessling

Robert Kiessling

  • 1 Post
  • 0 Reply Likes
It would appear that this is a relatively simple bug to fix, yet disappointingly it is still present in the current version.

Here's a little Perl script that copies the importHashes from one catalog to another.

Robert



#!/usr/bin/perl

use DBD::SQLite;
use strict;

main();
exit(0);

sub main {
scalar(@ARGV) == 2 || die "usage: $0 <source catalog.lrcat> <destination catalog.lrcat>";

my $from_fn = shift(@ARGV);
my $to_fn = shift(@ARGV);

-e $from_fn || die "Database file '$from_fn' does not exist";
-e $to_fn || die "Database file '$to_fn' does not exist";
my $from_dbh = DBI->connect("dbi:SQLite:dbname=$from_fn","","",
{
AutoCommit => 0,
RaiseError => 1,
});
my $to_dbh = DBI->connect("dbi:SQLite:dbname=$to_fn","","",
{
AutoCommit => 0,
RaiseError => 1,
});

my %stats = (updated=>0, not_updated=>0, candidates=>0);
my $sth = $from_dbh->prepare(<<EOT );
SELECT idx_filename,importHash,absolutePath,pathFromRoot
FROM AgLibraryFile f,AgLibraryFolder o,AgLibraryRootFolder r
WHERE f.folder=o.id_local AND
o.rootFolder=r.id_local
EOT
$sth->execute();
while (my $row = $sth->fetchrow_hashref()) {
$stats{candidates}++;
my $sql = <<EOT ;
UPDATE AgLibraryFile set importHash="$row->{importHash}"
WHERE id_local IN (
SELECT f.id_local
FROM AgLibraryFile f,AgLibraryFolder o,AgLibraryRootFolder r
WHERE f.folder=o.id_local AND
o.rootFolder=r.id_local AND
r.absolutePath || o.pathFromRoot='$row->{absolutePath}$row->{pathFromRoot}' AND
f.idx_filename='$row->{idx_filename}'
) AND
importHash IS NULL
EOT
my $sth2 = $to_dbh->prepare($sql);
$sth2->execute();
if ($sth2->rows == 0) {
$stats{not_updated}++;
}
elsif ($sth2->rows == 1) {
$stats{updated}++;
}
else {
die "Unexpectedly updating multiple rows";
}
}

print "Found $stats{candidates} importHash in source DB\n";
print "Updated $stats{updated} importHash\n";
print "Did not update $stats{not_updated} importHash\n";

$from_dbh->disconnect();
$to_dbh->commit();
$to_dbh->disconnect();
}
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 384 Reply Likes
|> "It would appear that this is a relatively simple bug to fix, yet disappointingly it is still present in the current version".

Don't get me started..

A big thank you to Robert for sharing the repair script.

R
Photo of xvrl

xvrl

  • 5 Posts
  • 0 Reply Likes
I have the same problem (reported here http://feedback.photoshop.com/photosh...) It's disappointing to see that Adobe has not even acknowledged the problem after 3 years.

It's still a problem in Lightroom CC/6, and makes it hard to work on a separate catalog when traveling
Photo of xvrl

xvrl

  • 5 Posts
  • 0 Reply Likes
This reply was created from a merged topic originally titled Importing form exported catalog does not preserve duplicate file information.

Lightroom does not seem to preserve duplicate file information when importing photos that were exported from another catalog.

Steps to reproduce:
1. Open catalog A
2. Import new photos from card reader into catalog A
3. Export some of those photos from catalog A into catalog "A-export" using "export as catalog"
4. Open catalog B
5. Import photos from "A-export" into catalog B using "import from catalog"
6. Import photos into catalog B from same card / card reader as in step 2, making sure to check "ignore suspected duplicates"

Expected Result:
Photos already imported in step 2. should be marked as duplicates when importing from card reader into catalog B. The corresponding information should have been preserved when exporting and importing from catalog A.

Actual Result:
Photos already imported through catalog A are not marked as duplicates and reimporting would create duplicate copies in catalog B. Some of the information available about the original file names got lost when exporting and importing from catalog A.
Photo of Andreas Aronsson

Andreas Aronsson

  • 1 Post
  • 0 Reply Likes
Same issue here. I just imported a catalog from my laptop, only to discover Lightroom thinks all my photos on my memory cards are new, even though they exist in the catalog, same filenames and everything. I tried synchronizing my folders to see if Lightroom would grab more meta-data so this would actually work, but no difference :(

Kind of sad this was discovered years and years ago and it's still an issue.