513 Messages
•
11.1K Points
Thu, Apr 7, 2011 5:33 AM
6
Lightroom: Prevent loss of Edit Histories when Reimporting Photos
When importing DNGs with stored edits (included XMP data) then the history of the photo just shows "Imported..." instead of the list of edits.
I have a corrupt catalogue. (I did nothing to cause the correction :()
The catalogue contains photos which are not associated to folders in the library module. When I choose "Got to folder in Library module" from the context menu for such photos, nothing happens. I imported them just like any other photos, but somehow the corresponding library folder wasn't created or lost.
I tried synchroning the parent folder but the missing subfolders are not created again.
That's why I decided the only way forward is to create a new catalogue. However, the new catalogue doesn't have any of the edit history. The rendering is OK and I can reset it to see the original version of the photos but I cannot see the edit history anymore.
Why is the edit history not recreated? The essence of it must be available because otherwise the correct final rendering could not be created.
I believe edit histories should be available for JPGs, RAW and DNG files. When I decided to use DNG files vs RAW files with sidecar (XMP) files, I didn't know that I'd lose the history with a fresh import of a DNG file. I suppose that if I had XMP files, I could copy these and still had my edit histories.
I have a corrupt catalogue. (I did nothing to cause the correction :()
The catalogue contains photos which are not associated to folders in the library module. When I choose "Got to folder in Library module" from the context menu for such photos, nothing happens. I imported them just like any other photos, but somehow the corresponding library folder wasn't created or lost.
I tried synchroning the parent folder but the missing subfolders are not created again.
That's why I decided the only way forward is to create a new catalogue. However, the new catalogue doesn't have any of the edit history. The rendering is OK and I can reset it to see the original version of the photos but I cannot see the edit history anymore.
Why is the edit history not recreated? The essence of it must be available because otherwise the correct final rendering could not be created.
I believe edit histories should be available for JPGs, RAW and DNG files. When I decided to use DNG files vs RAW files with sidecar (XMP) files, I didn't know that I'd lose the history with a fresh import of a DNG file. I suppose that if I had XMP files, I could copy these and still had my edit histories.
Ideas
•
Updated
10 years ago
76
19
6
Helpful Widget
How can we improve?
Tags
raw + xmp
dng
edit histories
Responses
john_beardsworth
1.3K Messages
•
22.5K Points
10 years ago
If catalogue corruption is the real problem, it's that problem that needs to be resolved. eg by the user using a backup (which contains edit history), or by Lightroom's backup being smaller (ie a transaction log allowing rollback/forward)
John
1
0
rob_cole_2221866
4.5K Messages
•
76.3K Points
10 years ago
Whoa. Yeah, people act as if backups are a panacea, but that assumes your backups aren't wonky.
I've known that edit history is only in the catalog for a long time, so it wouldn't have been a surprise, but it was a surprise the first time... - I feel for ya.
So, I take it the catalog integrity check that I assume you were smart enough to perform before the backups (if not, I've heard confession is good for the soul...) was passing, despite the problem? I hope you've sent the funky catalog to Adobe for inspection(?)
Anyway, if the catalog is mostly OK, it may be possible to reconstruct the missing pieces using an SQL client. Or at least someone at Adobe maybe could, I'm not sure I know enough, or you...
2
0
john_beardsworth
1.3K Messages
•
22.5K Points
10 years ago
Overall, I am not blindly against saving history steps to XMP - though XMP does have the adjustment settings so you're only losing the "route". I do think there should be an option (haven't we had this conversation before?) to save history to XMP, but it should include other Lightroom work that isn't currently saved out. So I'd like it to be my choice whether to save stacking info, assignment to collections, virtual copies etc.
On the other hand, I wouldn't want Adobe to see this as anything more than a very secondary level of backup.
John
2
0
rob_cole_2221866
4.5K Messages
•
76.3K Points
10 years ago
John's suggestion may just fix you right up. If not, if you send me both catalogs + a list of missing info, I'll transfer what I can, which may be nothing...
R
2
0
tk_images
513 Messages
•
11.1K Points
10 years ago
I discovered that the corrupted catalogue had only one corrupt entry. It was an image whose "root folder" was "nil".
I exported all photos in the corrupted catalogue -- except for the one problematic one -- to a new catalogue and, surprise, surprise -- the new one shows all the missing subfolders again!
So LR stumbled across one corrupt database row, causing it to drop a number of library folders from the display. I also noticed that LR became very slow with the corrupted catalogue in some operations (displaying "All Photographs").
Maybe (wild speculation) corrupted catalogues can sometimes be the source of some "performance" problems?
1
0
rob_cole_2221866
4.5K Messages
•
76.3K Points
10 years ago
One time, simply optimizing my catalog improved performance by A LOT, meaning it went from buggy slow to normal.
0
0
tk_images
513 Messages
•
11.1K Points
10 years ago
It seems that I cannot convert this bug report into a feature request anymore, so should I create a new one?
2
0
lee_jay_fingersh
946 Messages
•
13.8K Points
10 years ago
3
john_beardsworth
1.3K Messages
•
22.5K Points
10 years ago
In any case, LR catalogues are impressively robust and we have very few reports of corruption. That's why they are worth backing up.
1
0
dantull
Employee
•
166 Messages
•
3K Points
10 years ago
Which reminds me of another mini-feature I've wanted for the history panel: a "minimize" option that elides multiple (even non-contiguous) adjustments of the same slider into a single step. It would turn a messy history list into a simplified summary of all the actions taken to get from original to final.
The reason that's possibly relevant here is that minimize could probably be implemented in such a way that it could compare the default settings with the current settings which would mean an image imported without history would get a concise summary of the changes. Not as complete as the original, but it provides at least some of the utility.
8
dantull
Employee
•
166 Messages
•
3K Points
10 years ago
The minimize feature would definitely lose the complete path from start to finish as that isn't the intention. For me, I never (ever) care about the whole path. I care about a summary of the set of adjustments I made (that is, what path would I have taken if I'd known where I was going and made no mistakes). That's a distinct way to view history.
On the other hand for storing complete history in XMP, even if you store the deltas, you can reconstitute any step along the way (source control systems use this trick for efficient storage). It is undiluted, just very smartly compressed.
As an example of the reason for using deltas, I once was sent a corrupted catalog that was 7 GB. The size was dominated by the data for a small handful of photos. The history for the largest of them was was 450 MB (it had been heavily retouched with localized corrections). As an experiment, I blasted that data (history step by history step) into a Mercurial (source control) repository and committed each step. The repo at completion compressed to just under 700KB (and was only a couple of MB total, only about half again or maybe double the size of the XMP file containing only the final settings which were about 1.6MB).
All that is to say, while there are reasons not to store history in XMP, size is not one of them unless the implementation is naive.
2
0
tk_images
513 Messages
•
11.1K Points
10 years ago
=== Feature request ===
LR catalogues store full edit histories. However, when changes are saved to image files (DNG, RAW+XMP, JPG) only a set of instructions to recreate the final rendering is stored, all of the edit history is lost.
When importing such image files one only gets a "Imported..." entry, instead of the full list of edits.
I suggest that users have the following options regarding data stored in image files:
a) no history stored (current situation)
b) essential history stored
c) full history stored
An "essential history" only contains those steps that are necessary to create the final rendering. For instance, all "back and forth" settings to a single parameter are replaced by a single parameter change. It would actually be nice if users were given the option to reduce the in-catalogue histories to "essential histories" as well.
The motivation for having histories outside catalogues is twofold:
1. It would simplify backup strategies in that one would only need to back up image folders to retain final renderings with their edit histories.
2. It would create a safety net against catalogue corruption. If a catalogue develops a problem one may not immediately notice and thus create multiple backups with the same problem. Instead of being required to rebuild a clean catalogue from (potentially very old) clean backups, one should have the option to just create a new catalogue from the image files.
The size of histories stored outside of catalogues could be addressed by efficient storage schemes (e.g,. binary compression).
0
0
tk_images
513 Messages
•
11.1K Points
10 years ago
I though it was worth a new feature request to suggest that catalogues are stored in compressed form.
2
0
Victoria_Bampton_Lightroom_Queen
Champion
•
6.2K Messages
•
106.2K Points
10 years ago
Victoria Bampton a.k.a. The Lightroom Queen
www.lightroomqueen.com
Author of Adobe Lightroom Classic - The Missing FAQ and Adobe Lightroom - Edit Like a Pro books.
8
0
Ian_Lyons
Champion
•
28 Messages
•
532 Points
10 years ago
The default should be Off, just as it is with Photoshop, which has had the ability to save history to XMP and/or a text file for years. Actually, take a look at how it done in Photoshop.
Also, be warned, set default to On and there will be h.ll to play!
3
0