Elements 2019: Understanding organizer backup

  • 1
  • Question
  • Updated 4 months ago
  • Answered
  • (Edited)
My Product:  Adobe Photoshop Elements 19
My System:  Windows 7 prof, 64 bit
My storage: C:drive.  500gb SSD - Program files instlled here
D: Drive.  1,000gb SSD My Catalogs and all my media stored here
F: External Drive:  1,000 gb HDD

I did my first "Full Backup"  on  17 Jan 2019 and directed it to a new folder I put on the external drive titled backupPSECatalog_17Jan2019".  When I look it up using Windows Explorer it says there are 117,206 files and 193gb.  There are no .PSE files here as I expected, since my "Catalog.pse" is on my D drive.

So today, 24 Feb 2019 I want to do an incremental Backup
Here are the process steps:
- Calculating total media size.
- Select destination drive.  (It list my 3 drives)
==> Here it is confusing.  The "Options" window has 2 entry boxes: 
1.  "Backup Path"   :  I created a new folder for "BackupPSECtalog_24Feb2019" so I point to that one.
2. "Previous Backup File"  I assume this means the .tly in the 17Jan2019 folder.
There is a note: "Size: Load previous backup file to determine size"

Using the "Browse" tool next to entry 1. I select the F: Dive and point to the new folder for 24Feb2019 (which is empty).  
When I use the "Browse" to point to "Previous Backup file"  in the 17Jan2019 folder  "Backup.tly"  I get this message:
"F:\...\Backup.tly does not contain a backup set for the current catalog.  Please locate the last backup you made for this catalog."
I have searched my entire F: drive and there is only 1 "Backup.tly" file and it is dated 17 Jan 2019 when I did the one and only "Full Backup"

I can't get past this point.

Please help

I would also like a couple of related questions answered:
Q1;  Is it OK to keep my .pse catalog file on a separate drive?
When I initiate the Backup catalog it goes through step where it is calculating the total media.  Q2:  What's that all about?  How is that used?  Q3:  Is that when the Catalog.pse file is updated? Q4:   If not when is the Catalog.pse updated when I open the organizer or close it?
Thanks in advance for your help.
Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes

Posted 5 months ago

  • 1
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
There are no .PSE files here as I expected, since my "Catalog.pse" is on my D drive.
Don't expect that. The backup folder must include a 'catalog.buc' file which is a renamed version of the 'catalog.pse17db' sqlite database in the catalog folder.

Be warned that the incremental backup process is very rarely used, mainly because it does not save enough time compared to a full backup. Anyway, don't trust an incremental backup if you don't understand the process and above all, unless you have successfully tested the restore feature.
Q1;  Is it OK to keep my .pse catalog file on a separate drive?

Yes, no problem. You can move the whole catalog folder elsewhere from your explorer / finder.


When I initiate the Backup catalog it goes through step where it is calculating the total media.  Q2:  What's that all about?  How is that used?
It's to check if you have enough free drive space.

Q3:  Is that when the Catalog.pse file is updated?
I don't understand. The 'catalog.pseXXdb' database is just copied and renamed in the backup process. Each full or incremental backup folder keeps the 'catalog.buc' which is the latest state of the database at the time of the backup. Yes, every incremental backup holds that database which covers all the images of your catalog.
The catalog.pseXXdb which is created in the restore process is the catalog.buc copy made in the most recent incremental backup. As expected, it reflects the last state of the catalog. That's why it's counterintuitive to start from the newest  backup and to go back to the initial full backup.
The job of the backup.tly file is to store everthing needed to restore the final structure of the media folder tree.

Q4:   If not when is the Catalog.pse updated when I open the organizer or close it?
The catalog.pseXXdb is not updated when opening or closing the organizer. It's updated from all actions you are doing when managing your files in the organizer.



Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
Michael
Excellent reply.  My understanding of the 'Catalog/Backup/key files' schema has emerged from the fog.  Especially grateful for the recommendation to skip the incremental backup.  Your point:  "unless you have successfully tested the restore feature" has been a worry from day 1.  If the backup process is so complicated I imagine the restore must be a nightmare.
Since posting my questions i went ahead and attempted a new full backup.  Nothing is easy with Adobe Photoshop!!  My external drive still has 450gb of free space and my first backup was 193gb yet the backup aborted because it determined I did not have enough space.  It takes 15 minutes to get to that determination!  I will try again.
Got another question.  Q5:  Is the catalog.buc possible to open with MS Access?  II tried to open the pse file and it did not work.  I'd love to see how everything is organized with tags, paths, albums, etc.
Thanks again for your help.
Jerry

Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
Got another question.  Q5:  Is the catalog.buc possible to open with MS Access?  II tried to open the pse file and it did not work.  I'd love to see how everything is organized with tags, paths, albums, etc.
Thanks again for your help.
Old versions up to PSE5 used the MS Access database engine, but since PSE6, the database is Sqlite (just like in Lightroom). You can download and use free Sqlite editing utilities to have a look at the structure.
Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
Well, I just went through every table and other than file name and path I could not identify where things like my tags and albums are stored. Has Adobe ever published a data dictionary? Something that explains the relationship of the tables and the field names.
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
No, nothing has been published even if the main concepts about tags and albums are the same since nearly twenty years. You'll need more than a couple of hours to find by yourselves about tags... Sometimes, I wonder if there are many engineers in the Elements team who really grasp the whole picture, especially after the different ways other features like face recognition or smart tagging, visual similarity etc have been added.
Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
My 'Gut' tells me the face recognition add was a "bridge-to-far" and it is gumming up the works.  Way-to-much complexity for marginal benefit, but appealing to the Facebook types,  I wonder if there is a way to turn it off completely and wipe the tables clean.
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
Yes, in the organizer Edit menu, set your preferences for the Media-Analysis feature.
I have always been using the standard keywording process to manage persons (I don't care for faces).
Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
Hi Michel
Just a quick note to let you know I solved my back-up problem.  Thanks to you.  Sending me to the other thread gave me the insight for what was causing my Back-up" to fail and as suggested in that thread it was connected to Apple IOS Video "MOV" type files.  Using Database .Net (Thanks for that tip also) I was able to clearly see I had multiple duplicates of certain MOV type files in my catalog.  I was able to determine they were being generated using Premier Elements for video editing.  Luckily I have not been doing a lot of that so I was only dealing with a manageable number of duplicates.  I deleted these in Organizer (catalog deletes not file) and was then able to successfully create a new back-up. 
I will be putting this comment at the tail of the main thread here and mark the issue  of "understanding back-up" as answered.
Thanks again
Jerry
Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
Hi Michael
Thank you again for the help you are giving me in my journey to understand Backup.  Unfortunately, I get past one hurdle and another presents itself.
Following your guidance I have decided to do "Full Backup" instead of "Incremental".  I have tried unsuccessfully 3 times to create a new full backup. 
New question: Q6:  It takes 15 minutes but I always get the same message box with no supporting details.

Details:
I have a brand new 1TB HDD dedicated for backup.  It currently contains 2 folders, one of which is the first Full Backup I did in January.  It has 193gb of data.  I have 450gb of free space.
I have run diagnostics and defrag on this HDD.  It is 0% Frag.
I have run repair, repair anyway and optimize on my catalog several times with no issues.
I have 48,000 items in my Organizer.
I have considered wiping the drive (reformatting) and starting again but I am more interested in Understanding Backup and the root cause behind the problem,  My confidence in this product is shaky and a successful "work-around" will not help me in that regard.  I know the 15 minutes leading up to that msgbox involved some analysis.  Maybe it's buried somewhere in the Adobe folders in Program Files.
What do you think?
Thanks
Jerry
(Edited)
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
The time to calculate the necessary free space is not normal, as well as the message you are getting. I don't get similar issues and I suspect there is something new like in that other discussion:
https://forums.adobe.com/thread/2598112
I don't use SSDs, I don't use any automatic background tasks like face tagging, smart tagging, auto-creations, and I don't use the new video formats used in iPhones.
The 'repair' and 'optimize' features have little chance to help here.
Have a look at the other discussin to see if disabling automatic features will help. You can also try to create a sample catalog without videos to see if that works as expected.

Photo of Jerry Delaney

Jerry Delaney

  • 12 Posts
  • 0 Reply Likes
Hi Michel
Just a quick note to let you know I solved my back-up problem.  Thanks to you.  Sending me to the other thread gave me the insight for what was causing my Back-up" to fail and as suggested in that thread ( https://forums.adobe.com/thread/2598112 ) it was connected to Apple IOS Video "MOV" type files.  Using Database .Net (Thanks for that tip also) I was able to clearly see I had multiple duplicates of certain MOV type files in my catalog.  I was able to determine they were being generated using Premier Elements for video editing.  Luckily I have not been doing a lot of that so I was only dealing with a manageable number of duplicates.  I deleted these in Organizer (catalog deletes not file) and was then able to successfully create a new back-up. 
I am formally marking the issue  of "understanding back-up" as answered.
Thanks again
Jerry
Photo of Michel BRETECHER

Michel BRETECHER, Champion

  • 1236 Posts
  • 259 Reply Likes
Congrats for solving this mysterious issue.
I have marked this discussion as answered.
I'll also tag it as a Premiere Elements issue.