Lightroom: Reset your Preferences: Moving from the magic spell to a more serious approach

  • 3
  • Problem
  • Updated 3 weeks ago
  • (Edited)
Hi,

Under many circumstances, whenever your copy of LR crashes, freezes, doesn't start,... the most frequent answer you'll get from the support is "Reset your preferences". And in many cases, that works. But 1) you have to re-configure LR again and 2) this will not fix the bug generating the problem so, it will happen again . Should we be happy with this ? No.

If resetting the Preferences file(s) solves a problem, that's only because

- The piece of LR code reading or writing from or to this file is buggy
OR
-  A particular combination of parameters generates a problem.

Do not accept the idea that the Preferences file was corrupted. If it's corrupted, it's by LR itself. Spontaneous file corruption does not occur that often.

Since the developers at Adobe don't seem ready to investigate this case more deeply, I guess it's time to help ourselves and to collect information that could help them debug this issue. There's a few things that are easy to do :

1. Make a backup copy of your Preferences file(s) while LR is behaving correctly. This could also help you restoring a standard behavior instead of resetting the Preferences.

2. When a problem occurs, make a copy of the current Preferences file(s).

3. If resetting the Preferences solves the problem, then make a comparison of the files created in #1 and #2. You can also compare with the new Preferences file(s).

4. Make a ZIP archive of these files possibly including the comparison result and send them to the support or better, post a link to the archive in this thread.

Comparison can be made manually or by using a text comparison tool. I'm personally using Beyond Compare since years (this is not an ad) but there are others (a Google search will show you a lot).

Actually, there are 2 Preferences files :

C:\Users\\AppData\Roaming\Adobe\Lightroom\Preferences\Lightroom Classic CC 7 Preferences.agprefs

C:\Users\\AppData\Roaming\Adobe\Lightroom\Preferences\Lightroom Classic CC 7 Startup Preferences.agprefs

(note that the name didn't change with version 8)

So apply the above to both files. This procedure also applies to the Mac version although I don't know the file path.

Hoping that this will help solve this very old and annoying issue.
Photo of Patrick Philippot

Patrick Philippot

  • 445 Posts
  • 127 Reply Likes

Posted 4 months ago

  • 3
Photo of Cristen Gillespie

Cristen Gillespie

  • 1590 Posts
  • 502 Reply Likes
I agree with your steps to help reduce the amount of time you spend setting all new preferences. I'm on a Mac and haven't actually had corrupt preferences apart from installing a new version — maybe installing a new OS or significant update is another occasion when you're likely to get "corrupt" preferences.

BTW, it's user Library> Preferences> version Settings file on a Mac. You'll find a whole bunch of files and folders in the Settings file. If you hold down the 3 primary modifiers when launching, it asks if you want to delete the entire thing. That's the usual way of "trashing Prefs."

My take on it when installing a new version is there may be conflict between the old Settings file and the new one—we could say "as designed with unintended consequences."<G> Though it's probably close to a bug since no one seems to know in advance that this new thing will affect that old thing. Or the new setting that didn't exist with the old preferences can't possibly play well with others? It seems  over the years they change what's under the hood, and or actual preferences, more often than they used to. The industry is moving faster as a whole—we're getting new technology more quickly and the OS turnover is much faster as well. There's good and bad news in all of that.

Comparing files is a good idea. It may be impossible to avoid the problems that arise from new versions conflicting with old ones, but it surely should help if they know where the difference lies and if it matters.

But also, just writing prefs over and over and over again is statistically bound to cause digital corruption. Why we're not immune to that, I don't know. Maybe there are introduced problems when something else causes us to crash out or force quit. I can't really be any harsher on Adobe than everyone else who often also require us to jump through hoops as they fix the problems an update caused, or try to help when someone's program, but not everyone's, is acting up.

I've never forgotten standing with one of our engineers in a company I worked for a very long time ago when he said "I wonder what would happen if I accidentally hit this key right next to the one I want" — and brought down a mainframe for 3 days that operated coast to coast with hundreds of thousands of customers. It has always gone with the territory. IBM, you've got a bug.<G>

Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
On Windows, at least as of three years ago, LR was directly overwriting the preferences file rather than writing a new version to a temp file and then atomically renaming the temp file on top of the old version.  If the computer or LR halts during the overwriting, you're left with a corrupted preferences file. This bug was acknowledged by an Adobe engineer but never marked as fixed, and I've never retested:
https://feedback.photoshop.com/photoshop_family/topics/lightroom-writes-preferences-file-directly-ra...

While this is an elementary programming error that's easy to fix, I think it's unlikely to be responsible for most of the situations in which "resetting preferences" fixes a UI issue. If the file is only partially written, LR wouldn't be able to read it at all, since internally it's a large Lua expression on which the Lua loader would fail if it encountered a syntactically incorrect expression.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
It's clear there's a serious design flaw in the core architecture of LR's preferences, since "resetting preferences" can fix so many different user-interface problems.  Here's my educated guess about what might be wrong, based on many years of writing LR plugins and pushing the limits of the LR SDK's capabilities:

When you examine the preferences file, it's clear that LR stores a large amount of internal application state in preferences, not just user settings, and if that state gets recorded inconsistently, then chaos in the user interface results.   

I'm guessing LR has an internal preference facility implemented as a special Lua table of key-value pairs.  Other parts of the LR app store something in preferences simply by assigning a value to a key (with an assignment statement). There'd be a background task that periodically writes preferences to disk. 

Corruption could occur if a client module attempts to store multiple key-value pairs at the same time, and the background task decides to run in the middle of storing the key-value pairs.  This assumes there is no locking mechanism that ensures the multiple values are handled atomically, or if there is a mechanism, many parts of LR don't use it correctly.

LR's internal tasking is supposed to be cooperative non-preemptive, so in theory the programmer shouldn't need to worry (as much) about handling such concurrency.  LR doesn't even expose any locking primitives in the plugin API.  (I don't know if there are any internally.)

But the reality is that so many internal APIs can preempt a task's execution that it can be very tricky for the programmer to handle concurrent tasks without race conditions.  See for example this bug report:
https://feedback.photoshop.com/photoshop_family/topics/lightroom-sdk-many-methods-yield-preventing-r...

Suppose a LR module wants to store two key-values in preferences, with code that looks like:

prefs.key1 = f1()
prefs.key2 = f2()

If f1() calls an internal API that causes task preemption (e.g. it does file i/o), the scheduler could run the preferences background task between the two statements, writing to disk preferences state that is inconsistent, containing a new value for prefs.key1 but an old value for prefs.key2.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
I wrote a simple test script reliably demonstrating the race condition described in my previous post. It's available as this plugin:
https://www.dropbox.com/s/1gufmz0bwubu8o9/testprefswriting.lrdevplugin.zip?dl=0

I'll bet a modest amount that this race condition accounts for many of the "corruptions" of preferences experienced by users. The programming workaround is simple in concept but difficult in practice -- identify all occurrences where there are multiple fields that should be set consistently in preferences and assign their values at once from local variables. E.g. transform

prefs.key1 = f1()
prefs.key2 = f2()

to:

local key1, key2 = f1(), f2()
prefs.key1= key1
prefs.key2 = key2
(Edited)
Photo of Patrick Philippot

Patrick Philippot

  • 445 Posts
  • 127 Reply Likes
Very interesting. I have never studied LUA and the plugin SDK enough to be sure of this but AFAIK, the only task synchronization tool available in LUA seems to be yielding. Apparently semaphores, mutexes and the like are not implemented. This is far from being enough to cover all concurrency situations.

This raises again the question about the choice of LUA for the development of LR.
Photo of Jerry Syder

Jerry Syder

  • 271 Posts
  • 136 Reply Likes
Not sure what the results mean @John but ran it for the sake of it 
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
That means it detected an inconsistency caused by the race condition.  The two values should be identical.  Technical details are provided at the top of the file "testprefswriting.lua".
Photo of Jerry Syder

Jerry Syder

  • 271 Posts
  • 136 Reply Likes
Cheers, will give it a read 
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
Also, you should restart your LR after running that plugin -- otherwise, it will run continually.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
Instead of:
local key1, key2 = f1(), f2()
prefs.key1= key1
prefs.key2 = key2
the prefs object could provide this method:
prefs:store {
    key1 = f1(),
    key2 = f2()}
which would be trivial to implement and would provide the same safe semantics.

Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
Building on Patrick's suggestion for saving backup copies of LR's preferences, I wrote this simple plugin:
https://www.dropbox.com/s/vd0pd48b3u9mtkt/snapshotprefs.1.1.zip?dl=0

From its _README.txt:
-----------------------------------------------------------------------------------------------------
Snapshot Prefs - Lightroom Plugin

This plugin runs in background, making snapshot copies of Lightroom's preferences. If at some point you need to reset Lightroom's preferences, you can submit these snapshots to Adobe, helping them identify the bug that corrupted the preferences. 

The plugin will make a snapshot copy at most once every 5 minutes, and it will retain at most 15 copies, intelligently pruning them to keep increasingly older versions, up to 56 days. The plugin uses negligible disk space and processor time doing this.

To see the snapshot copies in Finder / File Explorer, do File > Plug-in Extras > Snapshot Prefs > Show Snapshots.

Photo of Patrick Philippot

Patrick Philippot

  • 445 Posts
  • 127 Reply Likes
Thanks a lot, John. Excellent idea.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 5215 Posts
  • 1057 Reply Likes
Actually, John your timing is really good.  This is something we are kicking around on our side right now.  (In response to this thread)

Basically, we want to be able to identify a bug (with reproducible steps) and have a copy of the Preferences before and after the reset of the Preference file.  In order for this to do engineering any good we need to know the following:
  1. A complete description of the bad behavior prior to reset
  2. A copy of the preference file before reset
  3. A copy of the preference file after the reset
  4. Verification that the preference file reset cured the issue in item 1
With those four pieces of information that may help us track down:
  • what in preferences changed by viewing the differential
  • what behavior it affected
  • clues as to what process, plugin or other circumstance generated the preference file issue and the subsequent aberrant behavior. 
Having the before/after preference files allows us to test user-reported issues in greater depth. 

If we were to create a process for the gathering before/after and possibly reporting of bad behavior resolved by a preference file reset how would it work best for you?
Photo of Patrick Philippot

Patrick Philippot

  • 445 Posts
  • 127 Reply Likes
Hi Rikk,

> 2. A copy of the preference file before reset

The preference file before reset might have been already corrupted. So, if I were the developer, I would ask for 3 files :

#1 : a copy of the preference file when everything is working correctly (thanks to John for providing the tool allowing to get it automatically)
#2 : a copy of the preference file before reset (the bug just occurred)
#3 : a copy of the preference file after reset, which will be different from #1 and less useful since everything will be... reset (witch exception of values specific to the user's environment).

My two cents.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 5215 Posts
  • 1057 Reply Likes
#1 would likely never be obtained with surety, I fear.  Until you stumble upon an area of bad behavior, it might lurk quietly in the background for days...months.  At least #2 and #3 are known quantities. 
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
The plugin I posted will keep up to 15 snapshot copies of the preferences, of increasing age, e.g. 

1 min, 5 mins, 10 mins, 20 mins, 40 mins. 80 mins, ..., 56 days.

So it's likely that one or more of those snapshots is prior to the corruption.  The total size of the snapshots in my LR is less than 1 MB on Windows, 10 MB on Mac.

Having preferences right before and right after the Reset would likely be helpful for diagnosing, but having a longer-term history might narrow down some kinds of problems more effectively.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4185 Posts
  • 1110 Reply Likes
Here are the preferences snapshots my plugin has created since I started using it three months ago:


In nine years of using LR, I've only had to reset preferences once, so it's unlikely I'll have to do it again. But if I do, I've got enough of a history here to make it very likely that the culprit could be identified.
Photo of Jerry Syder

Jerry Syder

  • 271 Posts
  • 136 Reply Likes
John, this plugin has saved me twice already. Both instances, LR crashed and created a new preference file when I re-launched. Cheers for that - I'd only downloaded the plugin as a trial to see how it worked and forgot to remove it(thinking I won't be needing it) but happy that I'd forgotten.