Lightroom: Catalog - robusten the integrity checker and add repair capability.

  • 10
  • Idea
  • Updated 7 years ago
Presently, backup does integrity check and optimization, but more slips through the net than if you do an import-from-catalog.

The problem is that often catalog problems get backed up for weeks or months before a problem is detected.

So, this 'Idea' is to have a more rigorous check of catalog integrity, and to whatever extent is possible, report errors in a more descriptive and detailed fashion than "Unknown error", and preferably repair them - with or without human intervention as appropriate...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes

Posted 7 years ago

  • 10
Photo of Victoria Bampton - Lightroom Queen

Victoria Bampton - Lightroom Queen, Champion

  • 4173 Posts
  • 1496 Reply Likes
I'm not so sure robusten is a word, but I agree with the idea. If nothing else, I'd like to see a 'check catalog integrity' option in the menu along with optimize catalog.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Robusten isn't a word yet, but I'm hoping it will catch on... ;-}
Photo of Ino Nimus

Ino Nimus

  • 16 Posts
  • 0 Reply Likes
Robust is a word (an adjective) but not a verb!
Photo of John R. Ellis

John R. Ellis, Champion

  • 3606 Posts
  • 938 Reply Likes
Have you heard that in English, you can verb any noun? So if you can verb a noun, you can certainly verb an adjective. :->
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Robustify is not in the dictionary, but it is in the wiktionary.
Robusticize is not in the dictionary either, but is referenced in the wiktionary under robustify, but is not in the wiktionary itself.
Robusten is not in the dictionary nor the wiktionary, but it has a nicer ring to me.
btw. wiktionary is not in the dictionary either.
All of this subject to change...
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
I've suggested this a few times (since version 1, actually), but it never quite makes the feature cut. The integrity check is just a low level SQLite file format validity and does nothing to analyze whether the data itself is valid.

I've taken to calling this semantic corruption (where referential integrity issues are a subset of that category) to distinguish it from the low level SQLite format damage indicated by the "your catalog is corrupt" dialog.

I see less of this than I once did, so it makes it even harder for me to lobby for it.

That said, if there is a third party developer that wants to build it, I'd be more than happy to provide guidance and answer questions.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Thanks Dan.

Since, during import of one catalog into another, more checking is done than during the integrity check, one way to "robusten" the integrity check function would be to do a mock import of the catalog into an empty catalog, behind the scene...

Rob
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
That would be extremely expensive... Besides, the catalog import process only achieves this "scrubbing" incidentally and is unfortunately lossy in other ways.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
I'm surprised on both counts. but lossy ya say? - do tell...
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
The main data that isn't retained is related to published collections (it's left out to avoid some complexities that would result from multiple copies of the same published collection syncing with a service, especially in the case where only part of the published collection was exported into a separate catalog). I think it also leaves behind preferences (otherwise importing a 100 image catalog into a 10000 image catalog would stomp all of the latter catalog's preferences). Lastly (unless I'm forgetting something), catalog import will only preserve non-empty collections and keywords (though it does copy all smart collections, IIRC). Basically, catalog import and export operate in terms of moving photos from one catalog to the other and everything else is only pulled through if it is somehow referenced by a photo in the catalog.

The expensive part is due to some complexity in the design of catalog import for supporting arbitrary merging of (possibly conflicting) keyword and collection hierarchies and photo/file sets. It doesn't bother special casing to avoid this logic for the case of importing into an empty catalog.

For the third part, catalog import only incidentally tends to scrub the structure by leaving behind cruft that isn't referenced, but only has limited logic for actually repairing other kinds of referential mistakes.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
I appreciate the info. Sounds like robustened integrity checking would not be able to take advantage of too much existing code without a lot of re-work - I can see why its a tough sell. - thanks for trying...
Photo of Dorin Nicolaescu-Musteață

Dorin Nicolaescu-Musteață, Champion

  • 703 Posts
  • 38 Reply Likes
What about image file integrity? Add that too...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
There is a separate FR for photo/file check/repair, but its easy to imagine them being rolled into one checkamous everythingamous feature. +1 vote...
Photo of Dorin Nicolaescu-Musteață

Dorin Nicolaescu-Musteață, Champion

  • 703 Posts
  • 38 Reply Likes
Sorry.
I have shovelled about 15 pages of most popular FR's today and haven't stumbled upon it.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Sorry I didn't provide link, maybe you'll stumble upon it eventually...