Lightroom CC: Catalog backup speed optimization by changing order of operations

  • 4
  • Idea
  • Updated 1 year ago
Lightroom CC now zips the catalog backup to save space (excellent!).

Catalog backups are often stored on a networked drive (e.g. Drobo5N) to provide data security in case of a drive failure on the computer.

This suggestion reduces the network usage for the backup (and so elapsed time) to less than a third of the current network usage.

Currently the order of operations followed when the backup takes place to a networked drive effectively trots the .lrcat across the network TWICE and the zipped .lrcat.zip file ONCE:

1) "Copying Catalog" - the catalog file (.lrcat) on the local hard drive is copied across the network to the backup location. This uses network bandwidth and is slow (relative to normal hard-drive speeds), but uses little CPU.
2) "Optimizing Catalog" - the local catalog file is optimized. Uses no network bandwidth as it all takes place on the local hard-drive, but uses some CPU.
3) Compressing Catalog Backup - the backed-up catalog file (.lrcat) is zip-compressed, which for a networked drive means the .lrcat file is copied back across the network to the computer, zipped, and the resulting .lrcat.zip file copied across the network to the networked drive. Once the copy is complete, the backed-up .lrcat file is deleted from the networked drive. Again this uses network bandwidth both for the .lrcat file "coming back" and the smaller ".lrcat.zip file "going out" and is slow.

The suggestion is to change this "slightly" so the ONLY file copied across the network is the zipped .lrcat.zip:

1) "Compressing Catalog" - make a local .lrcat.zip by zipping the local .lrcat. No network bandwidth used, some CPU.
2) "Copying Catalog" - copy the .lrcat.zip to the networked drive. When the copy is complete the local .lrcat.zip gets deleted. Much smaller file so much less network bandwidth, and uses little CPU.
3) "Optimizing Catalog" - the local catalog file is optimized. Uses no network bandwidth as it all takes place on the local hard-drive, but uses some CPU...so this could really take place concurrently with step (2).

My .lrcat is 1.1GB and .lrcat.zip is 251MB, hence my noticing the time taken for the backup...

Kevin
Photo of kevk61

kevk61

  • 5 Posts
  • 0 Reply Likes

Posted 3 years ago

  • 4
Photo of J0NNO

J0NNO

  • 8 Posts
  • 1 Reply Like
We also face the same issue and agree with your proposed solution... I'm also posting a new request to have Adobe consider updating the compression used as it's currently slow and inefficient.. 

Checkout the post if you like which I think you might also find interesting as it will allow you to save allot of LR Catalogues/history in very little amount of space
https://feedback.photoshop.com/photoshop_family/topics/lightroom-catalogue-backup-speed-and-size-opt...
Photo of kevk61

kevk61

  • 5 Posts
  • 0 Reply Likes
Looks like this (or similar) is implemented in CC 2015.8 - backups across a network are now MUCH faster. Thank You very much people of Adobe! Well done!
Kevin