Lightroom: Export destroys keyword hierarchy when creating derivatives to be included in the catalog

  • 6
  • Problem
  • Updated 2 years ago
I use a large keyword controlled vocabulary that I have organized by creating top level terms that have the "Include on Export" unchecked. When exporting or publishing for external use this works as expected. These top level terms are removed from the images.

The problem occurs if I want to create and save some special derivative files and keep them in my catalog by checking the "Add to This Catalog" option in the "Export Location" and "Write Keywords as Lightroom Hierarchy" in the Metadata panel.

The result is that the derivative is created and automatically imported back into my catalog just as I intended. However, the keyword hierarchy is now completely missing and the bare keywords are added into the Keyword list as individual terms.

When the intent is to round trip the image back into the catalog the original keyword hierarchy should be kept completely intact.

This is also the case if I want to use export to create a derivative file to be processed by an external application and eventually imported back into Lightroom.

-louie
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
  • frustrated

Posted 3 years ago

  • 6
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 3850 Posts
  • 730 Reply Likes
I was able to duplicate this, Louis. I don't think this is expected behavior but I can see why the logic is thus. I filed a bug and we will see what the team says.

It may come back as-designed because you are Exporting and you have asked that top-level keyword not to export.
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
Thanks Rikk for passing this up the food chain.
Photo of robgendreau

robgendreau

  • 61 Posts
  • 18 Reply Likes
I'm not sure if it's the same thing, but with some plugins if I do an "edit in..." and create a TIFF, then apply changes, after the TIFF is imported it will show a flattened hierarchy in the keyword box. But the hierarchy is still present in the file's XMP hierarchical namespace (eg USA|CA|San Francisco).
Photo of John R. Ellis

John R. Ellis, Champion

  • 3697 Posts
  • 968 Reply Likes
This is a different issue (the behavior of Edit In with some plugins). Please start a new topic.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3734 Posts
  • 977 Reply Likes
There are (at least) two use cases for adding an exported photo back to the catalog:

1. Tracking the exact copies of images exported for downstream use, e.g. sending images to a client and recording exactly what got sent, in case the client needs them again.

2. Creating a cataloged derivative using an external tool or application to process a copy of the image, which then gets reimported into the catalog.

Add To This Catalog supports use case 1 well, but as Louis observes, for use case 2 it doesn't work very well when some keywords are not marked for Include On Export.

However, the Edit In command was designed specifically for use case 2, preserving hierarchical keywording. Louis, perhaps you could elaborate on your use case and why Edit In doesn't work well for it?

I suspect those who rely on use case 1 would be upset if Add To This Catalog were changed, which suggests instead adding an additional option or feature.
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
Actually the problem happens when using export and "Add to this Catalog". The export removes any terms flagged not for export from the hierarchy and the subsequent import creates new keyword terms not originally in the catalog.

If you don't use any hierarchy then the problem is more subtle in that imported images are just missing any of the flagged keywords.

In either case imported image does not have the same keywords as the original which is why I use the description "destroy" as important meta data is being removed.

As you note "Edit in" does not have this problem.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3734 Posts
  • 977 Reply Likes
"Actually the problem happens when using export and 'Add to this Catalog'."

Understood and agreed that your use-case isn't well supported by Add To This Catalog.
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
Edit-in only works for one image at a time. Export is the only way to batch process a set of images.

This initially came to my attention because Nik HDR Efex Pro uses Export to prepare the HDR set, converting raw to TIFF for processing. And every time I would process an HDR image it would muck up my key word hierarchy.

I finally determined that it was easier to always remove all keywords prior to export and copy them back to the processed image afterwards.

-louie
Photo of John R. Ellis

John R. Ellis, Champion

  • 3723 Posts
  • 973 Reply Likes
Edit In can also batch process multiple images at a time. But I can guess why Nik used Export rather than Edit In for HDR Efex -- they wanted the ability to combine several files into one. While Edit In can also do that, it doesn't have the ability to delete the temporary intermediate files (e.g. the TIFFs) that are created as part of the process -- users would have to clean them up themselves.

So if Adobe were to address your use case, there are (at least) three approaches:

1. Change the behavior of Add To This Catalog. (This would annoy users who rely on the current behavior.)

2. Add an additional metadata option for Export that preserves metadata exactly (the same as Metadata > Save Metadata To File).

3. Enhance Edit In so that if the editor deletes one or more of the file copies passed to it, LR doesn't try to import the copies (as it currently does).

Option 1 would be transparent to Nik but would annoy users relying on the current behavior. Options 2 and 3 would require Nik to change their plugins.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3723 Posts
  • 973 Reply Likes
Based on past experience, it's unlikely that Adobe will address this before the next ice age. But Nik has a couple of options for fixing this issue:

1. They could enhance their Export service by adding a post-process action that copies the keywords from the input photos to the final photo.

2. They could implement the HDR merge as a plugin that appears under File > Plug-in Extras. That plugin invokes export internally and then copies the keywords from the originals to the final.

Their release of Version 1.2.8 (November 10, 2014) appears to have fixed the same issue in some of their other plugins:

https://support.google.com/nikcollect...
"Analog Efex Pro 2, Color Efex Pro 4, Silver Efex Pro 2: Lightroom metadata hierarchy is now preserved"

So maybe they'll fix HDR Efex too?
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
John, I'm not sure that Nik can solve this. As I recall when I studied this issue a while ago the removal of excluded keywords happens before the image is passed to the plug-in code. So they are already gone and the plugin has no chance to save the original keywords.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3723 Posts
  • 973 Reply Likes
The Lua code that implements the Nik export service has access to both the original photo in the catalog and the exported "rendition" (the TIFF) that will be passed to the Nik external application. That code can easily extract the original keywords and either apply them to the rendition (e.g. using Exiftool) or pass them as arguments to the external Nik application (if it's convenient for them to reapply the keywords in the application). The Exiftool approach would take roughly 30 lines of Lua code.

If there was someone at Nik who was interested, I'd be glad to explain to them how to do it. (But don't the rumors indicate that Google has cut way back on further Nik development?)
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
I can ask to see if they will implement this. Although as you note there has not been much new development on the Nik Collection in the last year. Probably due to the roll out of the new Photos system and apps.
Photo of robgendreau

robgendreau

  • 61 Posts
  • 18 Reply Likes
This reply was created from a merged topic originally titled LIGHTROOM - Exporting and editing in plugins flattens keyword hierarchies.

Exporting and adding immediately to the catalog can result in flattened keyword hierarchies. http://feedback.photoshop.com/photosh...

In addition, using a plugin like HDR Efex Pro via the Export... menu will also flatten the keyword hierarchy in the resultant file (TIFF) that is imported into Lr.

Using Photo>Edit in... and selecting certain other plugins, like those by MacPhun, can produce the same flattening. You are prompted to create a TIFF, which is edited by the plugin, and upon committing the changes the image is imported into Lr with a flattened keyword hiearchy.

In the case of HDR Efex, the flattened hierarchy is separated by carats. In the latter case, by commas.

In both cases the correct hierarchy is actually present in the files; it's written in correct order within the file's metadata in XMP in the proper hierarchy space. For some reason Lr ignores it.
Photo of robgendreau

robgendreau

  • 61 Posts
  • 18 Reply Likes
Here are two:

https://dl.dropboxusercontent.com/u/2...
https://dl.dropboxusercontent.com/u/2...

The first displays a flattened hierarchy; the second doesn't. Made from the same JPG.

I get this from the Tonality:
[XMP] Hierarchical Subject : tesst, tesst|tesst2, tesst|tesst2|tesst3

[IPTC] Keywords : tesst, tesst2, tesst3

And this from the Nik:
[XMP] Hierarchical Subject : tesst|tesst2|tesst3

[IPTC] Keywords : tesst, tesst2, tesst3

Indeed it seems Tonality garbles it, and I think it may only be upon commitment of the edits in Tonality, when it saves to the TIFF. Didn't notice the difference initially; sorry. Seems a MacPhun error.
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 30 Reply Likes
This will happen when the "Write Keywords as Lightroom Hierarchy" is not selected. You should double check your export settings and make sure it is set.

Nik for some reason has chosen to hide most of the export settings including this one. It took quite a while for them to set that option in their HEP export plug-in Lua code.
Photo of robgendreau

robgendreau

  • 61 Posts
  • 18 Reply Likes
It's selected. It does write them as a hierarchy, just not correctly.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3697 Posts
  • 968 Reply Likes
"Indeed it seems Tonality garbles it,"

Yes, the input has one keyword appearing in the input XMP:HierarchicalSubject and three in the output. They should be the same. But in Tonality's defense, the XMP:HierarchicalSubject field is a proprietary Adobe field for which I've never found a specification. There is an industry standard for hierarchical keywords (the Metadata Working Group spec), but I'm not aware of any applications that implement it.
Photo of robgendreau

robgendreau

  • 61 Posts
  • 18 Reply Likes
MacPhun is aware of the problem (common to more than one of their plugins). Will update if they fix it.
Photo of Louis Sherwin

Louis Sherwin

  • 139 Posts
  • 31 Reply Likes
Back to the original topic. Adobe has unilaterally decided that there is only one right way to handle keywords through the export process. And while this may in fact be the most common usage it is not the only valid method. There are also valid uses of Export where one does not want this to happen.

As far as this topic is concerned I maintain it is a bug to offer the "Add to This Catalog" option and then to unilaterally remove some of the metadata.

Adobe could easily solve this whole situation by simply adding an option in the export dialog to keep ALL keywords. There is precedence for this in that they already recognize other classes of private data, people and location, where you can make explicit choices to keep or exclude.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3723 Posts
  • 973 Reply Likes
"Adobe could easily solve this whole situation by simply adding an option in the export dialog to keep ALL keywords."

Agreed that would be a clean and simple solution.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3734 Posts
  • 977 Reply Likes
See this topic for an another discussion of the issue: https://feedback.photoshop.com/photoshop_family/topics/lightroom_add_an_option_to_always_export_comp...I don't want to merge the two topics, because merging will hide all of the useful replies in the merged topic.
Photo of Hallgeir Holien

Hallgeir Holien

  • 2 Posts
  • 0 Reply Likes
I have been struggling with the same problem in the interface between LR and Photomatix for several years. My workaround as tedious as it is: remember to remove all keywords on selected images before exporting to Photomatix, then sync keywords with originals after result is imported. When I forget my keyword hierarchy is just a mess.
Photo of Shai Schcolnik

Shai Schcolnik

  • 26 Posts
  • 0 Reply Likes
so, this has been going on for roughly 5 years in parallel threads.
it affects a large number of users (the more advanced ones)
isn't adobe going to fix this?
I rather not get my keyword list messed up instead of getting a new feature added to LR