Lightroom: Publish Service sometimes duplicates image with -2 or -3 extension to filename

  • 2
  • Problem
  • Updated 7 years ago
Lightroom 3.x. Win 7 64-bit.

I use Publish Services extensively to publish images to folders prior to uploading to my web site. It mostly works perfectly. However with some images, the image is first published with the correct name (Say W1234---MyImage.jpg) but then it gets published with an extension, usually -2 (for example W1234---MyImage-2.jpg), leaving the original image in place.

I have tried deleting the image from the Published Service collection and republishing the image. The image is not a virtual copy. I can see no difference between the image and any other images (they are all tif's and they all have quite a lot of metadata). There definitely isn't an image with a duplicate name in the catalog or on the disk.

It does seem as though the Lightroom catalog might have a duplicate entry ... it looks like a bug. Has anyone come across this and is there a workaround? BTW - I have optimised the catalog, tested integrity (as part of backup) closed and re-opened etc.

This is entirely reproducible!!

Thanks
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
  • bugged by this because it can waste quite a bit of time tracking down duplicate files

Posted 7 years ago

  • 2
Photo of jdv

jdv, Champion

  • 728 Posts
  • 56 Reply Likes
I suspect that it /isn't/ actually a dup in the catalogue, but that the publish service you are using is folder or disk-based. That is, the file name you are talking about is only expressed on disk, in the target folder for the publish service for that rendition for published catalogue image/

In this case it is supposed to overwrite the rendition on republish, but I bet when Lightroom ever runs into trouble overwriting the rendition, it will simply break the connection to the last rendition and create a new one with a guaranteed unique name. This part of the application is working exactly as designed, if this is the case.

Check the destination folder and device and make sure it has the correct permissions. If this happens occasionally, it is an indication that something else is interfering with the access to the rendition. If you are publishing to a network or USB/FW connected device, you should be aware that these do not behave exactly like a directly connected device when it comes to permissions. There might also be file locking implications on these devices that can manifest as the sort of thing you are seeing.

In cases like this, it is usually real-time anti-virus services, or (more rarely) automated backup services, that can interfere with other applications. In fact, the sort of file API calls that Lightroom makes to a rendition like this are exactly the sort of suspect calls that can trigger AV heuristics.

Try disabling real-time anti-virus services, or excusing Lightroom from their attention. It really doesn't make you any safer to have applications like Lightroom constantly monitored, and almost always leads to usability and performance problems.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
|> This part of the application is working exactly as designed, if this is the case.

I beg to differ - Lightroom should not pick a unique name when publishing a file, regardless of the problems encountered.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Which Publish Service?
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Rob - the Publish Services I use are Hard Drive (to a local hard drive).

John - thank you for your long response. However the problem isn't one of permissions. The publish service is to a local hard drive and there is full permission to it. I am publishing hundreds of images to the destination folder and only 3 give a problem. What happens is that the image (Say W12345.tif) in the Lightroom catalog gets published as W12345-2.jpg whether or not W12345,jpg exists in the destination folder. This is consistent and repeatable (with anti-virus running or not).

I can export the same file to the same destination folder and it exports with the correct name (that is W12345.jpg).

So unless I am very much mistaken, this is a an error in the database. I'll have a look with SQL but I would be interested to know if anyone else has come across this problem.

Robert
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
I have seen similar behavior in the past, I think - I was unable to rename a photo to asdf.ext, instead it would give asdf-2.ext. So, I think you are right, that it is possible for Lightroom to think it needs to use the unique name suffix when it shouldn't. Others have seen similar behavior too, maybe - unable to sync some photos in a folder, even though they don't exist in the catalog, per se, perhaps their "ghost" does, or something...

One thing to consider: export your entire catalog as a new catalog. That comes closer to flushing out catalog wonkiness than "integrity check + optimization".

If that don't get it, then maybe SQL(?)

Rob
Photo of jdv

jdv, Champion

  • 728 Posts
  • 56 Reply Likes
This may indeed be a timing bug, but there is no proof that anything at all is wrong with your catalogue.

We are talking about /renditions/ here, not images as they reside in your catalogue that reference the original "negative" (be that TIFF or raw [which tend to be variations on TIFF anyway]).

That is, the service publishes renditions as concrete file representations in a particular location, and are created and rewritten on an as-needed basis.

I suspect this is a timing issue of some sort, where the atomic file writes (which probably use temp files in the same location) go to rename the file at the end and the OS tells it that it is still there. So it picks the next safe name and uses that.

If you can duplicate this by forcing a re-publish of this single image (as opposed to republishing a whole set, where this one might be in the middle somewhere) then you might want to inspect the metadata and full OS details (Windows users can use "cacls" for this) and see how the compare with some other image.

I've heard of issues with non-image data in JPEGs causing interesting problems like this.

Also, try publishing to some other location, maybe even a different device. Try publishing a virtual copy of this image that you have given a custom name.

The information you get from trying things like this will be invaluable for solving the problem.
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Hi Rob,

Thanks for your input. I exported the entire catalog as you suggested and this does fix the problem ... but unfortunately the export doesn't include the Publish Services so that they need to be re-created (I just created a small one with the offending image to check to see what happens). I will probably have to do this but it is a pain as my Publish Services have a lot of images and also settings, meta-data filters (through J Friedl plugin) etc. The good thing is that the export does include collections, so I can copy my publish service images to a collection, do the export and recreate the publish service from the collection.

Still I think I will ask Adobe for a solution first as I think this is a LR bug.

Thanks

Robert
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
|> Still I think I will ask Adobe for a solution first as I think this is a LR bug.

I agree it smells like a bug...

I just noticed too that publish services don't come with an exported catalog - that sucks - I wonder if there is a (better) way to get them back (other than recreating manually).

Please tick the counter at http://feedback.photoshop.com/photosh...
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
Don't even get me started on the catalog import/export publish data retention. I was pretty unhappy about that choice. It's not without basis (there is a sticky issue it is trying to avoid), but it's an overly draconian choice.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
OK - I'll try to resist goading you on... ;-}
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
The OS will not name a file asdf-2.ext - Lightroom is doing that, and it shouldn't, given its the 'Hard Disk' publishing service, which should name it according to the naming convention specified, and if it can't, then provide an error message. This is a bug in Lightroom. That said, there may be some other software at the root. Still, if Lightroom does not react appropriately, then that's a bug in Lightroom. Naming a file with a unique suffix when that option has not been chosen by the user is not an appropriate reaction, regardless of the root of the problem.
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Quite agree!
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Yes, I agree - I was a software engineer (before becoming a professional photographer) and I'm pretty confident there is no way that Windows is interfering with this ... it's almost 100% certain to be in the Lightroom database. I'm going to have a look at the database to see if I can pin it down further (the Adobe support staff are helpful, but this would be in a different galaxy and all they would end up doing is referring it to the development team and in six weeks time a reply would come back saying it's a known issue that will be corrected in a future release).
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
I really don't know if the root of the problem is in OS, anti-virus software, Lightroom, catalog, or some combination. But there is definitely at least one bug in Lightroom - not detecting the erroneous behavior and reacting appropriately.

But if exporting your catalog solves the problem, there's a good chance the catalog was at least part of the problem...

I wouldn't know what to look for with SQL - but I'd sure start by trying to create the smallest catalog I could that exhibits the symptoms, if exporting the catalog didn't solve the problem in the process...

Did you make sure the file you are exporting is not read-only? (Shouldn't matter, but might...) What was the source file format?

Can you rename the source file in Lightroom, or does it add the -2 when you try? (I had this problem with some photos - that was definitely a Lightroom and/or catalog problem - no dup file on disk was "colliding" - I don't remember how or if I ever got past it). After renaming, do you still have the problem?

Rob
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Well I've had a look at the database but without a schema it's too much of a pain trying to find out if there is an error or not. I guess this is going to have to go back to Adobe!
Photo of john beardsworth

john beardsworth

  • 1042 Posts
  • 240 Reply Likes
I doubt it's anything to do with the database. Try a different filenaming scheme - there's a similar error in batch renaming - and note John Verne's comment:

"I suspect this is a timing issue of some sort, where the atomic file writes (which probably use temp files in the same location) go to rename the file at the end and the OS tells it that it is still there. So it picks the next safe name and uses that. "
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Note: Robert said its entirely reproducible and always happens with the same file(s). - right Robert?

What flavor of problem does *that* suggest?

|> Also: I exported the entire catalog as you suggested and this does fix the problem ...

Still true?
Photo of Robert Ardill

Robert Ardill

  • 45 Posts
  • 1 Reply Like
Yes, with an exported catalog and re-created Publish service the file gets published correctly (same file, same export folder).

Also, something I had not mentioned is that I have 3 Publish Services (all to hard disk): the first publishes the image resized to long size 900px, the second resized to 600px and the third resized to 250px. The three services have identically the same images as they are all used for my web site www.IrelandUpClose.com. All three services publish the file with -2 added (to three different folders obviously).

The file is not read-only. It is a tif. I can rename it in Lightroom fine. I can Export it to the same folder that the Publish service exports to and the export does not add the -2 to the file name. The -2 gets added by the Publish service whether or not the target file exists.

If I rename the file (its name is in fact W47262---Clare-Limestone-Clints-and-Grykes.tif) to W47262---Clare-Limestone-Clints-and-Grykes-X.tif, the publish services still export it as W47262---Clare-Limestone-Clints-and-Grykes-2.tif. If I Export the file the Export correctly exports it as W47262---Clare-Limestone-Clints-and-Grykes-X.tif

If I delete the file from the Publish service and Publish, the file with the -2 extension is correctly deleted. When I re-introduce it and then Publish, the file now no longer has the -2 added.

I think this demonstrated conclusively that this is a Lightroom problem and has nothing whatsoever to do with Windows.

So - the easiest solution is to delete the file from the service, publish, re-introduce it and publish again. However this is a lousy work-around because the same problem can happen again with a different file - what then happens on my web site is that the image is not updated (I might for example just add a keyword to the metadata) and I only notice this if I go and check the image or do a search for all files with -2 added.

Is this the best forum to flag a problem of this kind to Adobe? I cringe at the thought of going through Adobe support and getting a nice guy from India suggesting all kinds of things like re-installing Lightroom, Windows, moving swap spaces, deleting preferences and all of the things they suggest as standard solutions!

Robert
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Thanks for the update Robert - this thread is your bug report - no additional steps need be taken.