Lightroom: LR 2 catalog corrupted after power failure. Need help with recovery.

  • 3
  • Problem
  • Updated 7 years ago
  • Solved
  • (Edited)
My computer crashed during a sudden storm last night and Lighroom 2 was running. The catalog is now corrupt, and I really need it fixed. I followed the prompts to correct the corruption, and it didn't work. Can you please help me?
Photo of Kimberly Ward

Kimberly Ward

  • 4 Posts
  • 0 Reply Likes
  • worried, anxious, upset.

Posted 7 years ago

  • 3
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
All I know to do is fetch a backup, then reimport new photos (if you've saved xmp, then edits will be intact, although you'll lose virtual copies...) If you haven't saved xmp, then hopefully you will from now on...

Others know more and can do more...
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
I can try to help. While my e-mail addresses are fairly broadly proliferated at this point (lots of people find me via Google searches and forum threads on this topic), I'll send you a private message with instructions.

DT
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Lightroom catalog is highly vulnerable to this kind of corruption. Its a critical file that's constantly being written. I'm not sure how other apps handle this, or what can be done to alleviate in say Lr4, but some ideas for now:

1. Battery backup if you live in a place with intermittent power (or if you have extra money ;-}
2. Backup catalog each time Lightroom starts, or shuts down, (and/or after a large round of edits...).
3. Remember to also back up smart collections, presets, preferences, etc...
4. Periodically (or automatically) save xmp as you work (CustomMetadata will save custom metadata, xEmP will save virtual copies, for restoral in the future should the need arise).

Note: DNGs are also vulnerable when xmp is being written - if power lost while xmp is being updated, you may end up with a forever corrupt / unreadable / uneditable photo. This is one of the many reasons I prefer a sidecar for metadata that's constantly being written and re-written - worst case you have to re-save metadata or lose most recent edits, but you'll never lose the photo itself.

Keep us posted Kimberly - how're you getting on with this problem?

Rob
Photo of Kimberly Ward

Kimberly Ward

  • 4 Posts
  • 0 Reply Likes
Hey, thanks guys. Sorry for the delay in reply, I came down with a cold. Dan, my plan is to email you my file and see if you can repair it. In the meantime, I'm using a backup from the prior week and moving ahead with work. Thanks for the information, Rob, on DNGs being vulnerable. I'll keep an eye on them, just in case. Right now it appears it's only the catalog that's broken.

I really appreciate the help you guys. Thanks.

Kim
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Hi Kim,

Glad to hear it wasn't a total disaster for you.

I don't mean to spark a scare that the sky is falling - in practice the DNGs rarely get corrupted, but there is more risk when re-writing xmp constantly, which also necessitates re-writing the entire file including raw photo data occasionally - if power lost when that's happening, the DNG may very well be corrupted. Unfortunately, if this happens, you might not find out about it until further on down the road... - one of the reasons I push for photo integrity checking along with catalog integrity checking and repair.

Good luck,
Rob
Photo of Kimberly Ward

Kimberly Ward

  • 4 Posts
  • 0 Reply Likes
Thanks for the heads up, Rob. Would it be advised to re-download the raws from my memory cards?
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
|> Would it be advised to re-download the raws from my memory cards?

No.

I assume you are using DNG format for your working copies?

And that you have a backup of your original raws on a disk somewhere?

In any case, Lightroom was probably writing the catalog (not updating the xmp in a DNG/RGB or sidecar file) at the critical moment when the power was lost. Most often one or the other will be corrupted, but not both.

You can tell when re-importing newer files if there are any problems with the raw files - Lightroom may catch it, or you will see a funky band or block.

If you want peace of mind, consider running an independent photo integrity checker, like 'ImageVerifier' (if it turns up a corrupt file, then fetch a replacement from backup).

If you aren't using DNG, then re-saving metadata of whole catalog will overwrite any corrupted xmp sidecars, if there were any (again, its unlikely given that your catalog *was* corrupted).

Rob
Photo of Kimberly Ward

Kimberly Ward

  • 4 Posts
  • 0 Reply Likes
I use DNG for working, no backup on the RAWs at present (the memory cards act as a backup until I'm done processing, at which point I backup the DNGs to another drive).

Thanks for the tips. I'm pretty sure I just lost the catalog file. We are considering going to a battery backup at this point in time.

K
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
I generally recommend archiving the original raws, since one can always create a DNG from it, but not the other way around, and not all software supports DNG, and even software that *does* support DNG can't always read *converted* DNGs. But, its a personal thing I guess. In any case, I would definitely recommend keeping at least one backup of original raws or converted DNGs that are never touched, because of the previously mentioned possibilities - working DNGs being constantly re-written. If you backup a corrupt DNG (or any other type of file), then you have a corrupt backup too. Its one of the problems with catalog backups at present - many problems go undetected, so people end up with problem catalogs being backed up for weeks or months... - one of the reasons people want xmp and such to be as inclusive as possible - sometimes its a better option to rebuild a catalog from xmp, than to try to reconstitute a backed up catalog.

Cheers,
Rob
Photo of Eric Scot

Eric Scot

  • 3 Posts
  • 0 Reply Likes
OMG my catalogue just got corrupted.. I use DNG files also. What do I do? I am screwed.. Can anyone help me. And, no.... I don't have a back up!
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Hopefully in the future you will backup your catalog and related files, and save xmp.

Dan Tull sometimes helps people with fouled up catalogs - be very nice to him...

PS - I recommend using xEmP to save "xmp" for virtual copies, since otherwise virtual copies are lost when you have to resort to a backup or rebuild from xmp.
Photo of Geoff Walker

Geoff Walker, Champion

  • 214 Posts
  • 43 Reply Likes
xmp data is written into DNG files IF Autowrite is turned on in Catalog Settings.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Right. Begging the question: "How did Eric Scot have that setting"?
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
IIRC, it was disabled in the catalog he sent me (which I repaired).
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
What would we do without you Dan? :-)

To Eric - I suppose this is like reminding a person to watch their step right after tripping, but:

- Make sure you backup your catalog, and save (+backup) xmp too.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
This same problem just happened to me. I use LR 2... and have a backup from the last time I started LR. Would you send me the step by step of how to fix? Thank you so much!

I guess I just lost the 7 hours of editing I did today? On a wedding edit that was due yesterday...

Thanks again!
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Do you have xmp saved (embedded in DNG/rgb files, or as sidecar)?

Note: if you have auto-write xmp enabled in Lightroom preferences, then xmp is saved, or if you saved it manually using Ctrl/Cmd-s, metadata menu, context menu...

If you *don't* have xmp saved, then you have indeed lost those 7 hours of editing, unless Dan Tull can revive your catalog or something.

If you *do* have xmp saved, then you've only lost edit-history and maybe a few other incidentals (like virtual copies, stacking, picks, new collections...), but all "real" edits at the time xmp was saved will be intact after re-importing into restored backup catalog.

Note: Many of us have been pleading with Adobe to include xmp for virtual copies, so that (among other things) they are not lost when this kind of thing happens, since often virtual copy settings are just as important as "real" copy settings. In the mean time, I recommend Lr3+xEmP for saving virtual copy settings and metadata in the future.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Thanks! Honestly I don't know if I am xmp saved, but guessing I do. Can you walk me through what to do? Restoring from my backup?

I'm not worried about virtual copies. I don't use DNG. I import raw files.

Thanks so much!
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
1. Do you have .xmp (sidecar) files in same directory as your raws? (these contain the saved xmp info). If so, are their dates recent, for the most recently edited photos?

[[ Before the catalog went under, was the little down-arrow icon being displayed (indicating photo has changed since xmp last saved).
- Do you recall if the 'Automatically write changes into XMP' Lightroom preference was checked? (Catalog Settings -> Metadata tab).]]

Assuming #1 is true, then simply rename your existing catalog "old-name.lrcat.corrupt" or something of that nature, then copy your most recent backup catalog to the same directory, open it, and re-import the new photos (choose add-to-catalog, not copy...). If you know what folder the new photos are in, they can be "re-imported" by synchronizing the parent folder instead (right/ctrl-click parent folder, then choose 'Synchronize Folder').
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Oh shoot! No, there are no .xmp files in my RAW folders. If only I had known!
I don't recall if the auto write changes into xmp was saved and can't access that now.

So I guess I have lost today's work.
Going forward now, what is the step by step?
Select "choose a different catalog" from the dialog box and choose my last backup? Anything else?

Also, I do have LR 3.0 purchased and downloaded but am not using it just because I wanted to wait til the end of wedding season to update so as not to loose anything. Probably silly. Should I start that up now using my lastest backup as the first catalog?

THANK YOU, THANK YOU!
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
I believe the "auto-write xmp" setting is disabled by default (which is how I prefer it, but you have to know to save xmp manually so all your marbles are not in your catalog basket, so to speak). I could argue for having it enabled by default, but that would be a change that might cause its own set of problems...

I can't speak for Dan Tull, but he has been able to revive corrupt catalogs for some folks, dunno if it would be in time for your wedding client, but consider shooting him an email (dunno his address, but it's supposedly findable on the net...).

One thing: Do NOT just open your backup catalog where it stands. Instead, copy it to the same location as previous catalog (so all the photos and previews are found), and open it there (see my post above).

Note: I would definitely install Lr3 and begin using it with the restored backup. It won't interfere with your Lr2 work - if any troubles with Lr3 you can continue to use Lr2 - but make sure you save XMP, then you can import those photos with settings intact into Lr3 later once the Lr3 troubles are settled, or export them as Lr2 catalog and import that into Lr3.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Thanks!

So, just to be sure, as I do NOT want to screw up this work, I will:
1. Rename my current catalog by adding name corrupt.
2. Copy the backup to this same location where the corrupt catalog is.
3. Open LR using this new catalog.
4. Nothing more to do now, correct? It will be like it is 6 AM again.

4. Then, when ready, export the catalog and open the new LR 3 by importing this catalog.

Yes?

Thanks again! I swear, this is my last reply :)
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
You've got it, but note:

If you install Lr3, then double-clicking the restored backup or the exported version of it, will open it in Lr3, instead of Lr2.

You can import it into the default Lr3 catalog (or any other Lr3 catalog), but you can also just open it directly with Lr3 (which will convert its format). I'm not sure what the pros and cons would be, but the latter will preserve its name.

PS - You're welcome, and please feel free to post again if need be...

Rob
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Hi again :)

I've started up LR with the backup catalog in the new location.
It shows the files in the current wedding folder that I am working on to be offline or missing and only some thumbnails appear.
I tried clicking on the ? and locating the file, leaving checked the option to locate nearby files, hoping it will "find" the whole wedding folder. Unfortunately, during my edit today, I renamed the files I had in my final 2 star edit. Therefore, since this was done since my backup of this morning, the original filenames are different than the filenames in LR. Is there a way to solve other than going through and locating all 900 photos?

Thanks again,
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Given that you don't have the edits for them anyway, you might as well just delete them (remove from catalog) and re-import, or re-sync the folders they are in (or parent folder(s)) to "import in place".

If there is anything you are able to preserve from previous edits, then don't do this, otherwise why not?
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Two thirds of the photos I have edited (yesterday before the backup). About 600 photos. The whole folder, including those, have the question mark. Usually locating one folder works for the whole lot, but not this time since I have renamed in LR after the backup. Worst case I go one by one, searching for the corresponding image by looking at the thumbnail in Finder. But, wow, more time wasted. Any thoughts?
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
I bet there is somebody else on this forum with better ideas than me for matching up those photos with their new filenames. But if the critical catalog tables are still readable by a general purpose sqlite client - in your corrupt catalog I mean, I may be able to pull out the most recent develop settings to be automatically re-applied. - post a link to the corrupt catalog (just the lrcat file, preferrably zipped) and I'll take a whack.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
One obvious solution: use explorer/finder to rename the photos back to the way they were before, so they're found again. Then you can rename them again in Lightroom.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Hi,

Unfortunately I made a collection from images from 2 folders on my hard drive and renamed a selection of those combined in a collection. So I can't rename back. I can use finder to locate files one by one. And that is what I will have to do if nothing else.

Here is the link to the zip to the corrupt file (75 mb). Hope that is not too large. Let me know if you can figure something out. Thanks so much!
65.36.140.139/dropbox/2011corruptlrcat.zip
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Maybe put on some music and type to the rhythm...

I should be able to recover your develop settings from the corrupt catalog (it is readable by my sqlite app). I assume you can recreate other metadata with less trouble, no?

UPDATE: See post below about WreckHover.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Use WreckHover (for Lr3) to recover develop settings from your previous corrupt catalog (Lr2 or Lr3 catalog) and apply them to corresponding photos in the current catalog.

Note: Photos must have same ID (current catalog must be a recovered backup catalog), but may have different filenames.

PS - This plugin includes source code, and can easily be extended to recover other metadata as well, but @2011-11-07 only recovers develop settings.
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
I repaired that catalog file you linked to above:
http://dl.dropbox.com/u/7739202/repai...

The only damage I saw was in the develop history and even that seemed fairly minor. Hopefully that'll save you some trouble.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Thank you, thank you, thank you! Both to Rob and Dan! I spent all night clicking through to link associated files because of the renaming issue, and you just saved me an entire day of editing with this repaired catalog. The job is already past deadline so this really helps. Can't thank you enough!
Page
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
You're welcome Page.

PS - I nosiced you are from SF bay area when I was combing through your catalog metadata (me too). Maybe post link to some of your photos if you have any online? (or send me a private message...)

So, Lr3 yet??
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
The technique is fairly simple. I just use the sqlite command line tool's .dump command and do some very basic sanity checks on the schema afterward.
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
I have some ideas on a more advanced technique for cracking the (rare) tough cases, but once LR3 showed almost no corruption incidents, I tabled it. No pun intended.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
Thanks Dan.
R
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
I follow up question to the corrupted catalog. Now my LR destination folder that am using currently (with LR 3 now!) looks like this with the a dew versions of the corrupted catalog. The most recent one, which is now the fixed one, is called "2011.corrupt 2.lrcat". Should I move the others to a different folder using Finder to clean up this folder? Can I rename this catalog that I am using just by renaming in Finder? Will that screw up LR? This is a screen shot of my folder currently:



Thanks for the help!
ALso, is there any reason to keep old LR backups? I have a lot and they take up space. Should I just keep just the most recent ones?

Thanks again!
Page
Photo of Dan Tull

Dan Tull, Employee

  • 172 Posts
  • 38 Reply Likes
Lightroom only uses one lrcat and its accompanying Previews.lrdata package at a time, so it would be a good idea to move all the alternate copies elsewhere so you don't get thrown off by them at some point in the future.

Also, as long as you keep the names of the lrcat and lrdata matched, you can remove "corrupt" from the name as well so that doesn't create any confusion.
Photo of Page Bertelsen

Page Bertelsen

  • 12 Posts
  • 0 Reply Likes
Perfect, thank you!
Photo of Duncan Smith

Duncan Smith

  • 1 Post
  • 0 Reply Likes
I got an error message "The catalog "Lightroom 2 Catalog" is corrupt and cannot be repaired at this time."
As far as all the details go... long story.
2 months ago the logic board on my macbook pro fried. I purchased a new macbook pro and swapped hard drives and everything seems to be functioning fine. I'm traveling around the Americas on motorcycle so in the two months that I was waiting for my new macbook pro, I purchased a PC netbook to download and store my images. I moved all the images from the netbook to a external hardrive (formated for mac and on the PC netbook I have MacDrive 9) and imported them into my Lightroom catalog. Everything seemed fine and about 12 hours later Lightroom crashed and cannot read the catalog file since. I was not using the computer when it crashed.
Also there were duplicate files in the import as windows wasn't smart enough to realize it but I expected Lightroom not to have a problem with it.
Any advice?

This reply was created from a merged topic originally titled
Lightroom catalog is corrupt....