Lightroom Classic CC 7.3.1 Corrupts Catalog

Do you have any text from the error messages? Any screen captures? 

Hi Rikk--

Yes, I do. Here is a sequence of four. The first is after the backup process started...



Then, after clicking on OK...



Then, after clicking on Repair Catalog...



and shortly thereafter...




I then just clicked on Quit. As I said in one of my posts, after all this I was able to double-click on "Lightroom Classic Catalog_2018-04-20.lrcat" and Lightroom opened it without a complaint. So, it seems that either it wasn't corrupt or it was repaired (or maybe something else).

Thanks,
Dan

Hi Dan,

Thanks for reporting. 

- Just wanted to know since you are saying you rolled back to 7.3, so did you open the catalog second time in 7.3 or in 7.3.1 ?
- Is your issue resolved currently ?
- Can you try this again with another catalog(after taking a manual backup) ?

Thanks,
Sumeet

Hi Sumeet

Yes, I have been able to open the catalogs (more than one) in both 7.3 and 7.3.1. I have tried it with a couple of catalogs. Please see my post to the Lightroom Community forum referenced in my original post here..

Do I think this issue is resolved? NO. 7.3 has a number of issues that 7.3.1 attempted to fix, so that’s not a preferred version to use. I would prefer to use the most recent version but it now has a bug that it “corrupts” catalogs upon trying to back them up. How do I know I don’t have a database integrity issue like the error messages say? Has 7.3.1 somehow corrupted my database and there is something lurking in there?

Are you saying for me to use 7.3.1 and just do manual backups? Is 7.3.1 OK to use except for backing up?

Thanks,
Dan

I encountered the same issue. It appears backups made using 7.x can be corrupted, but not reported during the backup  integrity check. Both the original catalog file and backup file may in fact appear to be working fine when unzipped and used as the working catalog, but fail integrity checking.

I was able to "fix" my catalog corruption by installing LR 7.3.1 and running the catalog repair process again. No more corruption and it now passes the integrity check and can be backed up. At the LR Forum post it appears you've already tried catalog repair with 7.3.1. Is that correct?

Well, 7.3.1 is where the trouble started. Since 7.3.1 is where the problem exists for me, there’s no way I know of to fix it with 7.3.1. Nothing passes the integrity check. I have gotten the error messages above on several catalogs using 7.3.1. No problems with 7.3 or 7.2 for that matter. As I said earlier, I am using Mac OS 10.13.4. From what someone wrote in the Lr Forum, it was Windows OS that had the problem in 7.3.

It sounds like your current working LR Catalog file is OK and It's just the catalog backup files that fail the Integrity Check. Is that correct? If so close LR, hold down the Alt/Option key, and restart LR. In the popup check the option for 'Test integrity of this catalog' and make sure your current working LR 7.2 catalog file is selected. Does that pass the Integrity check? Try the same with your previous catalog backup files created with LR 7.3 or earlier that were NOT opened in LR 7.3.1. Do they pass?

This is a long post. Hopefully it is read all the way through. I know it might be confusing, so I apologize in advance, but the amount of testing with different catalogs and different versions of LR made it so.

Hmm. Well, initially I was grateful you pointed out how to run an integrity test outside of the backup process. Now, not so much. :-) Before I forget, is there a way to find out when I first installed 7.3?

It seems as though I have nothing but a bunch of bad catalogs and no way to fix them. Here's a run-down of what I've done today:

At the start of this run-down, I am running 7.3
1. My "Current Catalog" (has seen 7.3.1, the catalog that initially failed the integrity test but would still open in 7.3.1 and 7.3) opened OK. I then closed LR, held down Option key and restarted LR, selected "test integrity" and it proceeded to fail integrity test.

2. Went back to a catalog that had not been opened in 7.3.1(one that had been manually backed up), but had been opened in 7.3. Opened fine in 7.3 again. Quit LR, Option-Restart, checked test integrity, and failed. Tried to repair, but no go.

3. Rolled back to 7.2.

4. Double-clicked on catalog that had been opened successfully in 7.3 (not tested in 7.3.1) and 7.2 said it was corrupted from the get-go. This was a catalog from 4/15/18 that was backed up by LR.

5. I've gone back to a LR-backed up catalog from 4/9/2018 and tried to open it in 7.2 and it won't open (it's been trying to open it for 15 minutes). I'm not sure what version of LR I was using back on 4/9, so I don't know what it was backed up with. Do you know of a way to tell?

So, it looks like I am dead in the water. I seem to have catalogs that are corrupted in some way that will open in Lightroom, but can't pass an integrity test. I have older LR-backed-up catalogs, but Don't want to use them (the first one I tried LR doesn't appear to want to use either) because I have created a book since then (spending days and days) and that is saved in the catalog. 

I have two catalogs that seem to work: the "Current" catalog and one from 4/20/18 that is manually saved on two separate external hard drives. I am pretty sure I was using 7.3 when I manually saved them. None of these pass an integrity test, and I don't think any of them can be repaired.

As I was writing this and waiting for 7.2 to open the 4/9 catalog (which I gave up on), I saw another LR backup folder dated 4-30. This was created using 7.3 yesterday when I was testing things. It is a copy of my "Current" catalog. Since it was a part of a zip file, I know it was created by LR, and it had to pass an integrity test to do that. I unzipped it and moved it to another folder. It opened in 7.2. I quit LR and Option-started, selected the catalog, and checked test integrity. It failed. I clicked to allow it to repair. It failed. Will a catalog created in a later version of LR (7.3) fail an integrity test using an earlier version of LR (7.2) just because 7.2 can't understand some of it?

After this, I'm still not sure I have any good catalogs, and it appears as though I am toast. Quite an unhappy camper. Sure, I can run LR, but am I just laying the groundwork for something bad down the line? I don't really want to put a lot of work into something and have it all go down the tubes.

@Sumeet, if you've read this, do you follow? While writing this I went and click don the link Todd included in one of his posts. It seems as though I am caught up in the same issue that johnshore uncovered and documented in his post. I have backups created by LR that are corrupt. How could Adobe let this happen and not warn people back in February? How am I going to get one of my catalogs repaired?

This all sounds very similar to my experience with corrupted LR 7.x catalog files, but mine were repairable using LR 7.3.1. LR 7.2 failed the repair process with all catalog files that had integrity check errors. The fact that you can still open and work with them in LR is promising and means the catalog probably can be repaired. Perhaps an Adobe SQlite Database Engineer can take a look at it.

I ran my catalog through a free trial copy of SQLite Expert Professional and there were some missing develop settings history step indexes. It was also able to successfully repair the catalog same as LR 7.3.1. If you can post a working catalog.lrcat file to a file sharing site we can take a look at it. I can't promise anything, but at least it should tell us what's corrupted or missing. Use your current working catalog file and ZIP a copy manually outside of LR then upload the ZIP file. Make sure you have all of your catalog files copied to a separate backup drive. I'm sure you already know this, but better safe than sorry.

@Robert
Thanks for the suggestion. As far as I know, the problem with exporting as catalogs are: no VCs (as you said, but you do have a workaround), no collections (not sure of this), and most importantly for me if there are no collections, no books. If I'm incorrect on these, I'd be happy to learn otherwise. But...

Even with those "problems", I gave it a go. Unfortunately, Lr "Quit due to an unexpected failure". I also tried to create a new catalog and then Import from Another Catalog.... It likewise quit after importing only 1505 of more than 57,000 photos. It "shutdown prematurely due to a problem reading the catalog."

Maybe that shouldn't be too surprising. Upon opening the 1505 image New Import catalog, I noticed that none of the keywords are present. Maybe that happens later in the process (after it imports all photos)? Also found it interesting that Importing did include virtual copies. Unfortunately, it didn't go to completion and it doesn't look like I can only import a portion of the "From" catalog.

Dan, I did a check using SQLiteExpert Professional 5 with the below results. I then ran the SQLite ExpertProfessional 5 ’Repair’ utility, which was successful. It also passes SQLiteExpert Professional 5's integrity check, but fails LR 7.3.1's check. It also fails the LR repair process, but try it on your Mac system. I've forwarded the catalog file share links to Adobe Engineering (and you via PM) so perhaps they will have better luck.

Todd

Thanks for all the work you've done and time you've spent. I downloaded the repaired catalog and unzipped it. I double-clicked on the catalog and Lr 7.3.1 would not open it "due to an unexpected error."

I then Options-started Lr, selected the repaired catalog with an integrity test and it came back as corrupted. I clicked on Repair and Lr "encountered problems reading or writing from disk". The dialog box said it might work in future attempts, so I clicked on Try Again. Half a dozen tries did not result in anything new, so I gave up retrying. It sounds like this is familiar territory for you with this catalog.

Just for the heck of it, I tried double-clicking on the repaired catalog again, but to no avail. Thanks again for your help. I look forward to hearing from the Adobe Engineers.

Dan

Adobe Engineers have recovered my catalog and it appears to be working fine (it also passed the integrity test upon opening the catalog). I am back up and running, with a copy of the catalog manually copied to two external hard drives.

My sincerest gratitude and thanks to Satish and any other Adobe Engineers who worked on this, to others who have replied to my post, and especially to Todd Shaner.

Special thanks to SatishT and the Adobe Engineering Team for a speedy catalog recovery!  It looks like SatishT and I posted at about the same time. Anyhow the tips I provided for saving more catalog information to metadata should still be helpful.

I am running the same OS X operating system version (10.13.4) and Lightroom version (7.3.1) as this person on two macs. I use a SAN to store my image files and Export as Catalog from one computer and Import from Another Catalog on my other mac. When attempting to import the catalog, the second mac gets the following error:

Lightroom cannot import from this catalog.  The catalog does not appear to be a valid Lightroom Catalog.  

I don't want to revert to 7.3, I don't even know how.  I just want the problem fixed so I can continue to move files from one catalog to another, maintaining my Lightroom edits. 

Please fix this problem. I feel this is a fundamental flaw in the current mac release and should be fixed ASAP.

Dan