Why can't JPEG originals be exported with an XMP Sidecar?

  • 1
  • Question
  • Updated 3 months ago
I'm not sure where the limitation exists here. The use case is, I'd like to be able to export my JPEG originals + settings from CC and import them with adjustments into Classic (or vice versa), but I'm not able to do that, I'm presuming because Lightroom does not give them XMP sidecars. I'm mostly just curious why and where that limitation exists. But also, I guess, if it's possible, I'd love to be able to include XMP sidecars and/or be able to export adjustments from my JPEGS.
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes

Posted 3 months ago

  • 1
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
Side car files are intended for edits to be placed OUTSIDE proprietary raws. The same data can and is saved inside non proprietary files like TIFF, JPEG, PSD and DNG. IOW, there's very little reason for side car files in such documents and side car files were produced by Adobe because they will not edit proprietary raw data files. That data exists inside your JPEGs.
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
Does not matter!
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
It does matter, because Adobe should be able to read it’s own metadata. If it can sync the edits between the two, there’s no reason they couldn’t write the software to allow for manual transfer of the edits between the two.
(Edited)
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
And Adobe can! It doesn’t matter as you have been repeatedly told.
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
It may not matter to you, but it does to me. I don’t want to make those edits twice.
(Edited)
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
It would matter if you understand the request you made and why it's utterly unnecessary nor why you'll see it implemented. Concentrate on the bug/user error not the need for sidecar files for non proprietary data files. 
(Edited)
Photo of Just Shot Me

Just Shot Me

  • 233 Posts
  • 61 Reply Likes
I don't work with the CC, cloud based, version of LR but with the classic version when you export any image, RAW/DNG, PSD, TIFF or JPG that you have made edits to LR C CC Burns those edits into the newly created exported file (that is unless you select Export Original and then it does just that with No edits included).
With JPG, TIFF, DNG, PSD and I think PNG files the edits you make are saved to the catalog file and if you have the option checked to Auto Write changes to XMP the edits also get stored in a special area of the actual file. That is a Feature of the File Format. Camera makers proprietary RAW file format do not have that ability so LR and ACR store the edits in a XMP sidecar file.
In any event the only program that can read and display the edits made with either LR or the ACR plugin to any file format is either LR or ACR. No other RAW converter can read and display LR/ACR edits. They are just ignored by other programs.
So I don't see the point of exporting JPGs with a Sidecar XMP file.

If you edit a JPG in LR and you have the option checked to Auto write changes to XMP you don't need to Export that JPG. All you have to do is Copy it as all the edits are in the file already. Sending that file to some other user that uses LR or ACR will see the edits when viewed in their copy of LR or ACR.
(Edited)
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
But the bottom line, bug or user error, there is NO reason for sidecar files for JPEGs or non proprietary formats.Consider what happens when sidecar files disappear, get corrupted, etc instead of having that data embedded into the original file format. It's not a feature. It's a way in which Adobe can (for mostly legal issues), avoid writing that metadata into proprietary raws. Other files, no problem. Proprietary raws are treated as 'read only' by Adobe. Other's may not. 
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Sidecar, no sidecar, I don’t really care so long as the non-destructive edits come over to LR Classic or vice versa with my JPEGs.
(Edited)
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
Once you understand what non destructive editing really is, you'll see that sidecar files have absolutely nothing to do with anything you're asking about. 
Photo of Robert Frost

Robert Frost

  • 406 Posts
  • 63 Reply Likes
"Camera makers proprietary RAW file format do not have that ability so LR and ACR store the edits in a XMP sidecar file."
.......................................................................

My raw nefs DO have that ability; it is just a 'political' decision on the part of Adobe only to use that ability for capture time edits. And even then they only put the new capture time into the xmp part of the raw file; they don't edit the actual capture time in the Exif part of the metadata. So you end up with two different capture times in your raw file. They made up the tale about raw files being 'untouchable'. Rubbish; Nikon software edits them, so do Photomechanic and Exiftool, and probably others I haven't tried. Adobe want everyone to use their Dng, so they disparage editing other types of raw files.

Bob Frost 
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
Bob is correct that Adobe doesn’t write to proprietary raws for “legal” (he preferred political) reasons. It isn’t a technical limitation.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
Jon wrote:

"When I export an RAF or DNG from CC using the “Original + Settings” format, I can then import it to Lightroom Classic and all the adjustments (light, color, etc.) come with it. When I do the same for a JPEG, the stars and tags come with it, but the adjustments do not."

This is clearly a bug in LR Classic. The develop settings should come through with JPEGs as well as with raw and DNG.  LR CC is writing the develop settings to the JPEGs (in the XMP metadata section), but when they get imported into LR Classic, the effect of the settings don't show in Library.  However, if you open those imported JPEGs in Develop, the effect of the settings does show, and the sliders are set correctly.  Doing Library > Previews > Build 1:1 Previews has no effect.  However, exiting LR, deleting the Previews.lrdata file, and restarting LR, the imported JPEGs show the settings correctly in Library.

Try that out with your LR Classic catalog. Import one of the JPEGs saved from LR CC and go into Develop -- do you see the effect of the settings and sliders correctly?  If so, then go to Catalog Settings > General and click Show to open the catalog folder in Finder / File Explorer.  Exit LR, and then delete the Previews.lrdata, which appears as a file in Finder and a folder in File Explorer. Restart LR and wait for it to rebuild previews.

(Edited)
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Okay, one more test, took an image where the develop settings were not transferring, and changed one setting (increased contrast from 0 to +1) in LRCC, and this time when exporting to Classic all the develop settings transferred with it.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
In your tests,

1. Are you going to Develop in LR Classic to examine the photo there? 

2. Have you tried the workaround of exiting LR Classic, deleting the Previews.lrdata, and restarting LR Classic?
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
1. Yes, see video posted below.

2. I have not yet, will that affect the settings bars in the Develop Settings?
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
"2. I have not yet, will that affect the settings bars in the Develop Settings?"

Deleting the Previews.lrdata file is harmless and doesn't affect the contents of your catalog. The only effect is to cause LR to rebuild the previews it uses for displaying photos in Library (which could take a very long time depending on the size of your catalog).  So I am 99.9% confident that it won't affect the settings you see in Develop (it would be an extremely wierd-science bug if it did).
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Okay, I confirmed that deleting the previews.lrdata file does not fix the issue.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
Steps to reproduce the bug:

1. On an iPhone, check the setting Settings > Camera > Formats > Most Compatible (to force it to generate a JPEG). 

2. Use the Camera app to snap a photo.

3. Go to iOS LR CC and wait for it to sync the photo to the cloud.

4. Start desktop LR CC, select the photo, and set Exposure = -3.

5. Select the photo and do File > Save To, Original + Settings.

6. Import that saved JPEG into LR Classic with the Import > Add option.

7. Observe that the Library thumbnail does not show the effect of Exposure = -3.

8. Go into Develop and observe that Develop does show the effect of Exposure = -3, and the slider is set appropriately.

9. In Library, do Library > Previews > Build 1:1 Previews and observe the thumbnail doesn't change.

10. In Library, do View > Toggle Loupe View and observe that the photo still doesn't show Exposure = -3.

11. Exit LR Classic, delete the Previews.lrdata file/folder, and restart LR Classic.

12. Observe that Library shows the effect of Exposure = -3 correctly.

Tested with LR Classic 7.5 / desktop LR CC 1.5 / mobile LR CC 3.4.0 / Macos 10.13.6 / Ios 11.2.1.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
I posted this as a separate, clean bug report so that Adobe will notice it:
https://feedback.photoshop.com/photoshop_family/topics/jpegs-saved-from-lr-cc-dont-import-correctly-...
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
The bug I've encountered appears to be different than the bug you encountered, as the settings do not show in Develop.
Photo of Jim Wilde

Jim Wilde, Champion

  • 295 Posts
  • 98 Reply Likes
John, I'm not able to reproduce this issue on either my MacOS or Windows10 systems. In all cases the develop settings are correctly included in the exported jpeg, and seen in Library when importing them into Classic.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
Jim, after iterating a bit with Rikk Flohr, I narrowed down the symptoms I observed to a confusing bug in the import option Build Previews: Embedded & Sidecar:

https://feedback.photoshop.com/photoshop_family/topics/lightroom-importing-with-embedded-sidecar-pro...

If you followed my recipe using Build Previews: Standard, you wouldn't observe the bug.  

But John Anscher's problem is something different, as already discussed.
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
How's this for evidence? Note there was something weird where auto import did not import the photo the second time. Not sure what happened there.


(Edited)
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Okay, so I figured out what happened with the 2nd import. It apparently went into the Testing folder I made when I imported the 2nd photo. I probably should have deleted rather than just moving it, but this does show the develop settings not showing. I'll try deleting the previews.lrdata first, then re-test this again but delete the photo after the first import instead.
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
I ran another test where I did not use auto import, and I deleted the photos between. I replaced the video above, with that, so you can see it happening on my machine.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
"Okay, I confirmed that deleting the previews.lrdata file does not fix the issue." 
"
The bug I've encountered appears to be different than the bug you encountered, as the settings do not show in Develop."

Most likely, we're seeing different manifestations of the same underlying bug, which we don't completely understand yet. (That's the nature of bugs.)

Can you take a LR CC-saved photo that still won't import correctly into LR Classic, upload it to Dropbox or similar, and post the sharing link here? I'd like to examine the XMP metadata of the file, which encodes the Develop settings. The more detail we can hand to Adobe, the more likely the bug gets fixed sooner.


Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
I’m not near my computer at the moment, but these are from my iPad, the edits on that photo.



(Edited)
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
And again, if I make a single adjustment to that photo, then save it, it will import with all those adjustments. (I’m happy to start a fresh bug request by the way, now that I know this is a bug and not a feature issue.)
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
The problem with DSCF9619.JPG is that LR CC appears to have incorrectly saved it with this develop setting:
$ exiftool -G -xmp:alreadyapplied DSCF9619.JPG 
[XMP] Already Applied : True
The AlreadyApplied field is supposed to be True when all the develop settings recorded in the XMP metadata have already been applied to the underlying JPEG image. For example, when you export a JPEG from LR Classic, the underlying image will have all the develop settings "baked in", so the XMP metadata will include AlreadyApplied = True to indicate that.

But in this case, the settings clearly haven't been applied to the underlying photo. But LR Classic thinks that they are due to AlreadyApplied = True, so it ignores them. When I use the Exiftool utility to change the value to False and then import into LR Classic, the settings come in properly.

Do you remember the history of this photo?  The metadata indicates it was edited with LR Classic 7.5 as well as LR CC 1.5. I've tried unsuccessfully reproducing this bug both with migrating a Classic catalog and with importing directly into LR CC.

I think you should post a new bug report with the screenshots and the link to the saved file.  Also include screenshots of the Basic panel in Develop after you've imported the file into LR Classic, showing the sliders are all zero.   Or a video showing the sliders in CC, saving the photo, importing into Classic, and then Develop sliders in Classic. 
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Unfortunately, I don't remember the exact history of that photo. Generally, since I migrated to LR CC I only edit them in LR Classic if they are merged panos or HDR. It's possible that when I grabbed the photo, I grabbed the version that was imported into Classic. I'll make sure I put a clean file when I repost this as a bug.
Photo of Andrew Rodney

Andrew Rodney

  • 644 Posts
  • 119 Reply Likes
So you can not replicate this bug on all JPEGs?
Photo of John R. Ellis

John R. Ellis, Champion

  • 3894 Posts
  • 1035 Reply Likes
Also please copy the first ten lines of Help > System Info from both LR Classic and LR CC, so we can see the exact configuration you're running.
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
LRCC:

Lightroom Classic version: 7.5 [ 1186149 ]License: Creative Cloud
Language setting: en-US
Operating system: Mac OS 10
Version: 10.13.6 [17G65]
Application architecture: x64
Logical processor count: 8
Processor speed: 2.6 GHz
Built-in memory: 16,384.0 MB
Real memory available to Lightroom: 16,384.0 MB
Real memory used by Lightroom: 2,430.8 MB (14.8%)
Virtual memory used by Lightroom: 4,888.3 MB
Memory cache size: 1,922.8MB
Internal Camera Raw revision: 1001
Maximum thread count used by Camera Raw: 5
Camera Raw SIMD optimization: SSE2,AVX,AVX2
Camera Raw virtual memory: 365MB / 8191MB (4%)
Camera Raw real memory: 366MB / 16384MB (2%)
Displays: 1) 2560x1440

Graphics Processor Info: 
Metal: NVIDIA GeForce GT 750M
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes
Lightroom CC version: 1.5 [ 20180807-1548-54800d7 ] (Aug  8 2018)Operating system: Mac OS 10
OS Version: 10.13 [6]
Application architecture: x64
Logical processor count: 8
Processor speed: 2.6 GHz
Built-in memory: 16,384.0 MB
Real memory available to Lightroom CC: 16,384.0 MB
Real memory used by Lightroom CC: 806.6 MB (4.9%)
Virtual memory used by Lightroom CC: 5,696.3 MB
Memory cache size: 3,829.8 MB
Internal Camera Raw revision: 1001
Maximum thread count used by Camera Raw: 5
Camera Raw SIMD optimization: SSE2,AVX,AVX2
Camera Raw virtual memory: 1933MB / 8191MB (23%)
Camera Raw real memory: 1934MB / 16384MB (11%)
Displays: 1) 2560x1440

Graphics Processor Info: 
Metal: NVIDIA GeForce GT 750M
(Edited)
Photo of Jon Anscher

Jon Anscher

  • 185 Posts
  • 26 Reply Likes