Elements: How to process an Organizer catalog that can't be repaired

  • 1
  • Problem
  • Updated 7 months ago
  • (Edited)
I have a Organizer Catalog that can't be  repaired.   However Photoshop Elements seems to be able to open it and I don't know what if any functionality is affected.     Any backup that I have, has the same error so being advised to go back to a backup is not helpful.   I think two things should be done.  1) An error report should indicate what the error has to do with and 2) a tool should be developed that will recover what is not in error from a catalog and offer to create a new one.    The situation I am in should not be left as it is.  I was given now warning that there was a problem.  As a result, all of my backups have been bad and are not usable for me.   Please fix this.    roger@oblad.org 
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes

Posted 7 months ago

  • 1
Photo of Steve

Steve

  • 52 Posts
  • 14 Reply Likes
What error are you getting? Providing a screenshot of the error message would be really helpful. Please describe what you did just prior to getting the error may give a clue to where the error.

steve
Photo of Steve

Steve

  • 52 Posts
  • 14 Reply Likes
Also, please provide what OS are you using? What version of EO?
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
rpoblad3,   

One question:   Did you do a backup from Windows on one PC then are trying to get it to recover on separate PC?   Please answer.  It makes all the difference for what I think is happening.   


Steve Lehman, MCSE for Windows   
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I haven't been making backups of the catalog.  I have backups of all the media files on a cloud service.  The database corruption may have been in place for a long time.  I can't know.  I do know that I have five thumbnails that I can't delete from the catalog.  I was able to link them to an image file but I can't delete the catalog entries for them.   I don't know but I suspect the database corruption may be associated with these.
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I found this issue when I attempted to do a backup of the catalog.    The backup failed because of the unlinked thumbnails.  I linked these to an image file and the backup completed.  The error however is still there and I assume it is in the backup I made as well.   The time I see this error is when I attempt to do a "repair".
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I am using Windows 10.   The thing I did to see the error was to attempt to do a "repair".  I also see an error if I attempt to delete one of the six thumbnails that I am aware of having problems.
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
So my reason for asking was, it didn't come from a Windows incremental backup.  Windows backups when done with Windows cannot be recognized on a different PC, only a copy of a file can.   So we won't  go down that path.  I'm not altogether experienced with catalogs since I don't use them.   But, I have been with Photoshop since it was first developed.   At Microsoft we tested them in the early 90's when Adobe first published one.   I have almost every version of it plus Light Room.   

My 1st suggestion is to remove and reinstall your Photoshop. Your files will stay in Windows Explorer during that time - but don't do that until we try my 2nd suggestion, which is to get a friend of mine who is very familiar with the catalog in Photoshop.   I will give him the link to this your forum, and I will come back to check on you.  His name is Michel Bretecher, and he's a Champion.  I am a former Champion who has come back to help again i the forums and I don't have my ranking anymore, but I am a Windows engineer so if it has to do with that, I'll be there for that fix.  

Update:  I'll have already given Michel a link to your forum.   He knows your name and will send you a note.   Hang tough.   This forum has lots of experienced people who can help.   

Steve Lehman, MCSE  

  
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
Hello rpoblad3,

Be prepared to be patient...
We can't give  a positive 'one click' answer before we agree on the definitions of  'backups' and 'catalogs' in the context of the Organizer, but with some more questions we can understand better your issue.
First, if you are curious, you can learn more about backups and catalogs and all their issues in the Elements user to user forum:
https://forums.adobe.com/community/photoshop_elements/content?filterID=contentstatus[published]~obje...
To begin with, a recent one where I gave links to the online and to the pdf help files (2nd post)
https://forums.adobe.com/thread/2452632

Organizer catalog:
It's a folder. It is hidden by default (you may have to set the Windows Explorer to show hidden files and folders) and you find its location from the menu Help >> System Info.
It does not contain any image file, only links to the physical location of your photo files (path and drive internal serial number). All the information about each item is stored in a powerful database for instant searches; a small thumbnail is stored in a cache in the catalog folder as well as other types of info like face recognition.
To keep the links connected, you should NEVER move, delete or rename files outside of the organizer (Explorer...)
We need to know where your photo files are stored: conventional internal drives, removable media, cloud or network drives.

Organizer backups
It's absolutely necessary to understand that they are very different from other external types of backups:
- they are not simple copies of your files
- they are folders.
- they contain both renamed copies of all your media files and your catalog.
- Only organizer backups can restore both the files and the catalog where you want.


Our first task will be to understand where your files are stored and how exactly you did make backups. More details after that about various potential issues depending on your description (error messages...)



 
(Edited)
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I have not been making backups.   I understand now that backing up files is not sufficient.  
The catalog is stored in the default location on the hard drive.   The media files are in a user profile.
When I found that some thumbnails could not be deleted, I tried to do a "repair" to fix the issue. When trying to do a repair, I am told that the catalog is corrupt and a repair is not possible.  I want to find a solution so I don't have to keep using a corrupted catalog and crossing my fingers that it will eventually get to a point where I can't use it.  Ideally, I'd like a tool that I could run that gives me a option to discard elements that are corrupted.  At least I'd know what I have to recreate.  Thanks
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
I have a Organizer Catalog that can't be  repaired.   However Photoshop Elements seems to be able to open it and I don't know what if any functionality is affected. 
If I understand well, the option to 'repair' catalog returns an error:
When trying to do a repair, I am told that the catalog is corrupt and a repair is not possible.
but you can use it to a certain extent
I don't know what if any functionality is affected.

When working with the organizer, you can meet the following issues:
- corrupt or not recognized photo/video files
- missing or disconnected files (ie moved outside of the organizer); the files are not located where they were located when registered in the catalog.
- corrupt catalog itself (the main database component or its associated files)

In your case, it seems that the catalog is corrupt, but can be used. The 'repair' option does not deal with the issues with media files, only with possible errors in the catalog main database. This is very rare.

Issues with corrupt media files are found when backing up for instance.
Reconnecting missing files is possible as well as finding and deleting them if you want.
Issues with corrupt catalog itself will depend on how much you can work with the computer. I assume you won't be able to perform a full backup.

Can you search files, add keywords or captions?
Can you use the menu File >> Write metadata to files? If so, there is the possibilty to store your information in the files themselves and reimport them in a new empty catalog.
You can delete a catalog when another one is selected.
Ideally, I'd like a tool that I could run that gives me a option to discard elements that are corrupted.
A tool to find corrupt files blocking a backup process would sure be very useful.
Finding and deleting missing files is available.
Repairing a corrupt catalog does not work everytime, as you have found.

Other questions: do you use face recognition or geotagging? Which Elements version on which platform (Mac or Win)?
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I’m using the most recent of Elements. I’m using Windows 10. If I try writing all tags to the image files and then create a whole new catalog , will albums be restored or just tags?
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
Yes I have used face recognition. Images that came from my smart phone do you have Geo tags associated with them. I was able to perform a back up after I connected all of the thumbnails to an image file. The repair features still fails. I have four thumbnails that cannot be deleted. I can’t connect him to images but I cannot delete them.
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
I have four thumbnails that cannot be deleted. I can’t connect him to images but I cannot delete them.
When you get the list of missing files from the menu 'Find All missing files', on the second line of the dialog, in the middle, you have a small icon with 3 horizontal bars: it opens a dropdown menu, in which you can 'select all' and/or 'delete all'. You can't delete them from the disk, since they are not connected, but you can remove them from the list in the catalog.
If I try writing all tags to the image files and then create a whole new catalog , will albums be restored or just tags?
You'll restore keywords, captions, ratings and notes, but not albums, stacks, version sets or projects.

If you can perform a full backup, there is hope:
https://helpx.adobe.com/photoshop-elements/kb/backup-restore-move-catalog-photoshop.html
A successful full backup contains renamed copies of all your media files, and a two important files: catalog.pse16db and backup.tly. Can you see them?
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I had a typo in what I wrote you. I said “I have four thumbnails that cannot be deleted. I can’t connect them to image files”.

I meant to have said, I have four thumbnails that cannot be deleted. However, I CAN connect thim to image files. It was after I did this that I was able to complete a full back up.

Are you suggesting that if I restore from that backup, that the corruption in the catalog might go away?
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
Are you suggesting that if I restore from that backup, that the corruption in the catalog might go away?
Possibly. The only way to know for sure if the backup is ok is to restore it (I would try to restore on an external drive and see if the restored catalog and file folders work the same as the original).
If it does, I would try again the 'repair' command.
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
Hope you two won't mind I put in at least 1 percent...

I have found that when we use a cloud service, some of them have a conflict with files which Windows uses for certain other applications or services - which could have began this problem.  

Even if you have certain file types, cloud services can change their files to another file format as their own back up to your initial files in the cloud.   For example, Drop Box has .ppt  files which are also found in Outlook's personal profiles, and when downloading those files from Drop Box onto Outlook (if a user was emailed a link to download), both file formats for a file can show in Outlook, (JPG) and .ppt files together, as the files to use from Drop Box.   If a user selects the .ppt format, it can corrupt Outlook's personal profiles immediately, rendering Outlook as corrupt and not usable.  At that point all personal profiles need to be completely deleted then separately rewritten for Outlook to work again.   And, this can continue its problem in other apps as well.  

I have notified Drop Box of this problem and they say they are working on the problem but have not notified me of their solution yet.  I suggest not using a Cloud service without naming any particular one.   As you mentioned to back up files onto an external drive, that would be the better storage in my opinion also.  

Also, while backing up, do not use the Window's back up utility if you want your backed up files to be loaded onto a new (or reloaded to the same) computer.   Back up by using the copy/paste format to a folder on the external drive.  The Windows backup utility has a recognition problem.  If the files were backed up with Windows' incremental utility, it will recognize those files on the same computer but only 75% of the time, and it is never recognized on a new or other computer, since the other computer didn't do the initial back up.   

And finally, it's always my recommendation to use an external drive that has a real disc that rotates rather than a mobile external drive which has silicon like a flash drive which needs to be plugged in at the USB to keep warm.   Silicon can lose its charge within 6 to 8 weeks.   Use a disc type which uses AC, that stays plugged into the wall, then unplug the USB to preserve the back up from viruses and to have clean files.   A rotating  disc type will mention on the box, 7200 rpm or 8400 rpm and of course I always buy Seagate.   Out of 5 external drives, we only have one that is Western Digital.   If a computer comes with Western Digital, as tested by the computer company as a good then its a known model with few problems.   

I hope this helps.   

Steve Lehman, MCSE
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I will try this. It will take me a few days to get back to this.
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I tried what was suggested.  I can create a backup without error.  I can recover the same backup without error.  However, I still can't get the "repair" to work.  I get the same message.  It suggests contacting support.   What else can I do to get a "clean" catalog?
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
The problem with 'repair' not working is generally that you can't backup. So, I think you are safe even with this error message.
Another way to check if the error is really specific to the catalog.pseXXdb sqlite database or to other components of the catalog folder:
- create a new folder named like 'my test catalog'
- locate the catalog.pseXXdb file in your urrent catalog folder (XX is the version number, 15 for pse15, 16 for pse 2018)
- copy and paste it in the newly created folder
- with organizer closeed, double click on that file. That should open the organizer with the minimal catalog. Don't wait necessarily for the thumbnails to be created.
- try the repair command. If no errors are found, that means that the main catalog itself is safe.

This 'minimal' catalog copy may be useful in many situations. It will recover all your tags, captions and also albums, stacks and version sets. This will not restore accessory features like face recognition, geotagging, visual similarity.
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
Michel is the "go to guy" for a catalog problem.   There's always mine: remove-reboot-reinstall, but this easy way out is never a guarantee.  

SL
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
Thanks --
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
About the repair features in Windows 10;  they don't always work because, when an app is tested it pings back that it exists (only).   The only thing that Windows will repair is the .dll file that connects Windows with an app.   Most of the time, unless an error message is the only thing holding me back, I usually look for a work around and that's what you did.   When there isn't one, I dismiss it and work with what I have left.  

Example, I have a laptop which has a Norton update but the update will not install, and unless it installs the update, Norton will not open.  When that happens, Norton has its own remove/reinstall program to download and run, which they want us to use, but that will not download.   To first check everything, I need to determine if this update is really malware in disguise because it won't respond to my installation.   Due to lack of file loss and checking Windows for activity, if there is no activity in Windows, it's not malware.   My next course is to force a removal, reboot then reinstall.  I want to reboot because otherwise, Windows will merely install Norton into the same page-frame on the drive and it will run the same way it did.   I want Windows to random-select a new page-frame on a new install so I will reboot and actually I will completely shutdown the computer to make sure Windows dismissed Norton.   Only because it's a virus scanner, it's still an app that's treated like any other.   The laptop has Malware Bytes installed which is my only work around.  But if I want to completely repair Norton, I will get rid of this problem and install a new one - especially when I have an unlimited user license.   That will take the same serial number.   I'll give the links to get your serial, later.  

If you're at least half-satisfied with what you have, make note of it.   There is little doubt, it'll come again.  

Also, a new Elements could adapt to the same characteristics of another Elements on the same drive.   So my advice is to remove the older before installing the new.   

More advice:   I have almost all of the Elements versions installed on one computer.  If I open a new Elements first, the older Elements will not open completely, it freezes.   I need to open the older one first, then the newer one.   After that, I can open any full-grown Photoshop and Light Room too.   I don't run any Photoshop CC so I'm not sure what it does with that.      

If you need your license key to reinstall follow these steps:

Start by signing in the Adobe site:  https://accounts.adobe.com
Sign in using your Adobe name and password at the link above,
then go to Manage Account, next click Plan and Products,
then click View My Products, click your licensed products and key.  
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1085 Posts
  • 227 Reply Likes
Steve,
Your input is precious for the repair features of Windows.
In the present issue, there is no doubt that the 'repair' feature in the organizer has been present in all older Elements versions, some managed through a Microsoft access databe (a single file suffixed .psa) and from PSE6 on, from an sqlite database (a folder including among others a catalog.psexxdb sqlite file).

The 'repair function is a necessary feature of a relational database engine. For instance, if I use an sqlite managing software, I can run a 'repair'. From many tests in all previous versions, I conclude that the 'repair' command in the organizer calls that 'repair' function of the database - and most probably that's all it does...
In the hundreds of cases where users have been using the 'repair' function, there are practically no 100% sure testimony that it helped. (By the way that proves that the sqlite engine is very trust worthy.)
What I have no way to verify is if the current implementation of the 'repair' command in the organizer tests other catalog properties. If my suggested tests works with the 'minimal' catalog copy, there is some other test going on, perhaps on another sqlite database (thumbnail cache, face recognition...) but I doubt it. Maybe a bug creating a false warning.
(Edited)
Photo of Steve Lehman

Steve Lehman

  • 620 Posts
  • 78 Reply Likes
Hi there Michel,  

Those are good thoughts.  But I was referring to his repair of Windows as an attempt to repair the app.   I had assumed he was repairing his app by using the Windows repair in the PFC or to reset the app to default in the settings>apps and features module.   I didn't run those instructions to him as I assumed he knew them.  

Then again, he may have been referring to what you mention as the Db base of the application itself.   SQL is a relational database, and using any command that begins with SQ will bring up such a database. As for SQ programming in Adobe's Catalog, I was not aware it in this one.   It could be limited in its programming to what you can do to organize the organizer using the SQ catalog in a limited SQL programming language.  

BTW, Access has the same type of Db programming using SQL if you want to use the SQL commands it will allow those commands, but you need to be logged in as Administrator, then use those commands as an administrator would in a SQL limited programming.  If you are into SQL, you will start SQL software and begin creating a database with its programming key.  It takes query language in its programming.  It's complete programming is outside of Access as Access is limited in its SQL programming.   It's limited because Access could give a user open administration privileges beyond a manager, and these databases are used by telemarketers who will mess with it if they can.   You will need the actual SQL programming software to complete it anything in Access as Access is the client side.   SQL is the host side.    


Steve Lehman, MCSE   
Photo of rpoblad3

rpoblad3

  • 13 Posts
  • 0 Reply Likes
I am gong to get back to this in two weeks after I finish a presentation that needs images from my Photoshop Elements images.   Hopefully my corrupt database won't cause me problems in the next two weeks.   After then I really want to fix this.  Currently I have four thumbnails that can't be deleted. I have occasional application crashes that occur at the end of taking an action.   The search feature is broken meaning that once I successfully do a search, I can't get out without closing the app.