Lightroom: Export jpg as original is not the original file

  • 3
  • Problem
  • Updated 7 years ago
  • (Edited)
When exporting jpg's as original the file is not the original file.
This could easily be shown with either diff or md5.

1. file date is not preserved
2. EXIF-data is re-ordered / altered.
Amongs EXIF-tags altered is:
* Software
* Modify Data
* Artist & Owner Name
* User comment

LR 3.4.1 on Mac OS 10.6.7
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes

Posted 7 years ago

  • 3
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 135 Reply Likes
Saving the metadata to the file will save it right into the JPEG. You can do that with control-S or with autowrite turned on.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Yep. 'ORIGINAL' is not exactly original.

Note: Even when export format is original, you still have the option to write keywords as lightroom hierarchy (this is a clue).

Indeed, jpegs are being re-written: image data is original, meta data is not.

Summary:
========
'ORIGINAL' format is handled more like an export (re-write) than a backup (copy). If you truly want an unaltered original, use ScooterSoftware's BeyondCompare or something similar (e.g. Explorer/Finder) to simply copy.

See related 'Idea': http://feedback.photoshop.com/photosh...
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
A quite annoying bug, especially since it has been introduced in some of the later updates - unfortunately I can't tell exact which one. However, I'm 100% sure it was not there a year ago.

Please fix this bug (if the Adboe people read this), or rename the export option to "almost original".

To export the original files, one now have to use the "reveal in Finder" and then drag'n'drop / copy&paste to wherever you want to export them. This could become quite labour intensive if you are exporting files whose file dates spans over longer periods - the reside in different folders.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 135 Reply Likes
It's not a bug. Metadata is saved into the original files for all formats except proprietary raws, which use xmp sidecars instead.
Photo of Mark Sirota

Mark Sirota

  • 146 Posts
  • 29 Reply Likes
This has been true forever, it's not new. Export as original has always meant the original image data with updated metadata.
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
Some seem to give the word "original" a slightly more inclusive interpretation.. I beg to differ.

I'm still insisting that it is a bug, and that it has not been there "forever". My reason for that is, except the quite obvious designation error, that the EXIF-tags has been altered.

The EXIF tags is written to the files in different parts; for example ifd0, thumbnail, exif, interop, makernote etc. Camera information, at least for Canon pro dslrs have, in original-exports from previous versions of LR been found in IFD0->make & IFD0->model. This might no be visible in other exif readers since they also look in the location for propriety tags; EXIF->MakerNote. This does, however, becomes clearly visible when using a software (for long term storage of photos) that suddenly does not find camera information, that it has done for ears. As of what I can see it seems this bug was introduced at the same time the tag IFD0->Software being to be used; 3.4 that seems to be using a newer version of Adobe XMP Core (5.2).

If a transformation in to a feature request could help getting this issue solved, I would gladly appreciate instructions of how to do that. Such a feature could include a check box that shows up upon selecting original, that lets the user decide whether to include LR meta data or not.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
Well, there is no question, but that Lightroom has forever saved xmp in source jpegs, whenever xmp metadata is saved. It also will write some stuff upon export, keywords I guess and I dunno what else. So, is the offending behavior happening when xmp is saved, or upon export? I would think the answer to that question would help isolate the problem. Perhaps make a separate feature request, for an option to not have the jpeg touched upon export, if you would still want that even if the specific problem you are having were fixed (or maybe regardless). But if Lightroom has a problem in xmp saving or export metadata, that should remain a problem report, IMHO.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 132 Reply Likes
"I'm still insisting that it is a bug, and that it has not been there "forever"."

Since this behavior is written into the LR help documents, I'd claim it's by-design:

"File information is stored using the Extensible Metadata Platform (XMP) standard. XMP is built on XML. In the case of camera raw files that have a proprietary file format, XMP isn’t written into the original files. To avoid file corruption, XMP metadata is stored in a separate file called a sidecar file. For all other file formats supported by Lightroom (JPEG, TIFF, PSD, and DNG), XMP metadata is written into the files in the location specified for that data. "
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
Along with saving xmp (in original jpeg), Lightroom also re-writes 'Original' jpeg upon export. Question remains in my mind - is the metadata problem Korv has noticed occurring upon xmp save (perhaps due to new change in xmp handling), or upon export (has that changed?)?

PS - Many of us would prefer an option that Lightroom use a sidecar for all original files, whether jpeg, DNG, or tif..., see related idea: http://feedback.photoshop.com/photosh...
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
No doubt abut RAW files or the XMP sidecar - that's no problem at all. This is about export of jpg's (not embeded in other files, just simple jpg) using the "Original" setting.

Regardless of documentation, exporting a jpg file using the setting "Original" produces different files depending on version of LR. Pre 3.4 ? gives you the original file (in the true meaning of the word) versions after that gives you a non-original file - that have the changes described above.
Photo of Mark Sirota

Mark Sirota

  • 146 Posts
  • 29 Reply Likes
While there is a difference between 3.4 and earlier versions, I think you'll find that 3.3 didn't really produce the original file either. Just try adding some metadata (like Creator, Copyright, keywords, etc.) and export from 3.3, then compare with the original.
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
I could accept Creator, Keywords, Copyright, or a thousand other tags - as long as it does not mess up with data in existing ones. That's what's this thread is about.

After an export of a jpg (using the setting "Original") with LR 3.4 the tag IFD0->make contains some binary code I cannot read (), the IFD0->model Null and IFD0->DateTime the same binary value found in model.

Files exported in previous versions contains relevant data in these fields. For example "Canon", "Canon EOS-1Ds Mark III" and "2011:06:11 13:14:49"

I'm I the only one thinking this is bug, really?
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
"After an export of a jpg (using the setting "Original") with LR 3.4 the tag IFD0->make contains some binary code I cannot read (), the IFD0->model Null and IFD0->DateTime the same binary value found in model." - that sounds like a bug to me too, although I haven't confirmed it m'self.
Photo of Mark Sirota

Mark Sirota

  • 146 Posts
  • 29 Reply Likes
I can't reproduce this. Here's what I did (in LR 3.4.1):

- Import a JPEG which includes values in Make and Model.
- Export that JPEG as Original.
- Compare the values of Make and Model in both files.

They're identical in my case. Can you make your original file available for testing?
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
The key to see the bug is too look at the exact location of the tag,
IFD0->model
IFD0->make
IFD0->DateTime

Most exif reading software don't give you the locate of the tag being read, it will just serve you with a value regardless of where it was found. Since Model and Make could also be found in the propriety tags EXIF->MakerNote-> it could be hard to spot this bug.

I will post another reply when I've made the data available.
Photo of Mark Sirota

Mark Sirota

  • 146 Posts
  • 29 Reply Likes
I still can't replicate this. Here's the IFD0 section of a camera JPEG I've got:

[EXIF:IFD0] Make : Panasonic
[EXIF:IFD0] Model : DMC-LZ6
[EXIF:IFD0] Orientation : Horizontal (normal)
[EXIF:IFD0] XResolution : 72
[EXIF:IFD0] YResolution : 72
[EXIF:IFD0] ResolutionUnit : inches
[EXIF:IFD0] Software : Ver.1.0
[EXIF:IFD0] ModifyDate : 2011:06:18 20:20:17
[EXIF:IFD0] YCbCrPositioning : Co-sited

I imported this into LR 3.4.1, then exported as Original. Here's the IFD0 section of the exported file:

[EXIF:IFD0] Make : Panasonic
[EXIF:IFD0] Model : DMC-LZ6
[EXIF:IFD0] Orientation : Horizontal (normal)
[EXIF:IFD0] XResolution : 72
[EXIF:IFD0] YResolution : 72
[EXIF:IFD0] ResolutionUnit : inches
[EXIF:IFD0] Software : Adobe Photoshop Lightroom 3.4
[EXIF:IFD0] ModifyDate : 2011:06:30 10:45:00
[EXIF:IFD0] YCbCrPositioning : Co-sited

Again, can you make your original file available for testing?
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
After digging even deeper in to this matter it seems to me that "some" data indeed is written in EXIF:IFD0->Make, Model, etc. but - it's stored using some sort of encoding. This encoding, or whatever used, makes it impossible to read for at least php.

As you can see in the original (real original) exifdatadump, is in readable cleartext almost at the top of the file : http://83.227.160.130/original.html

Whereas the data from the LR "Original" export only shows up in the hover box. Also, in this file, FD0 tags is at the end of the file.
http://83.227.160.130/lr_original_exp...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 379 Reply Likes
It still looks like you've encountered a bug to me - I still think it would be a good idea to make your original available so it can be confirmed by others (if you are the only one voicing this issue, it could get overlooked by Adobe).
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
You wrote: "Whereas the data from the LR "Original" export only shows up in the hover box."

On lr_original_export.html, when I hover over "IFD0-00 Make", I see:

IFD0-00 Make
Tag ID: 0x010f
Format: string[6]
Size: 6 bytes
Value offset: 0x00aa
Value: Canon

The dump shows the string "Canon" at offset 0x00aa.

What's the issue with this?

(If you posted the files, e.g. on the Adobe forum or pixentral.com, then others of us could probably help resolve this much more quickly.)
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
There is no problem with "canon, offset etc." in the hover box. the problem is that info is not present in the middle column, which it is for the original file. This becomes a big problem in applications using php to extract the data.

I've just uploaded a newly shot picture. I've also included data dumps from php and as you can see, some of the tags the IFD0 group seems corrupt.

Original file
image : http://83.227.160.130/1/_M3I7722-orig...
exiftooldump : http://83.227.160.130/1/_M3I7722-orig...
phpdump : http://83.227.160.130/1/_M3I7722-orig...

LightRoom export
image : http://83.227.160.130/1/_M3I7722-lr-e...
exiftooldump : http://83.227.160.130/1/_M3I7722-lr-e...
phpdump : http://83.227.160.130/1/_M3I7722-lr-e...
Photo of John R. Ellis

John R. Ellis, Champion

  • 3690 Posts
  • 963 Reply Likes
You wrote, “the problem is that info is not present in the middle column, which it is for the original file.”

The information is “in the middle column” of the Exiftool HTML dump, but perhaps not where you are looking – that is, it is correctly encoded in the metadata according to industry standards. For example, in the lr-export version, according to the hover box the value of the metadata field Make is stored at offset 00aa. If you look at offset 00aa in the HTML dump (the 12th line), you’ll see the string “Canon” correctly stored there. I don’t see any problem here.

I used Exiftool to compare the metadata fields of the two files -- the differences are included below. All of the metadata from the original is preserved in the exported version except EXIF:ModifyDate, which the standards require to be updated whenever a program writes the metadata. LR has added some additional fields, which all appear correct, consistent with LR’s design of Export Original, in which it writes the file’s metadata from what’s stored in its catalog.

You PHP tool doesn’t appear to be finding the value strings for Make or Model in the exported version. I’m not familiar with that tool, but Exiftool is generally recognized as the most authoritative implementation of metadata standards. So if I had to bet, I’d bet in favor of Exiftool.

$ diff original.txt export.txt
2c2
< [File] FileName : _M3I7722-original.jpg
---
> [File] FileName : _M3I7722-lr-export.jpg
5c5
< [File] FileModifyDate : 2011:07:02 08:16:27-07:00
---
> [File] FileModifyDate : 2011:07:02 08:17:42-07:00
9a10
> [File] CurrentIPTCDigest : 77f304a14dd06d1cf19b4f059d6c
904f
22c23,25
< [EXIF] ModifyDate : 2011:07:02 11:30:27
---
> [EXIF] Software : Adobe Photoshop Lightroom 3.
4.1
> [EXIF] ModifyDate : 2011:07:02 12:37:02
> [EXIF] Artist : Xxxxxx Yyyyyy
37a41,42
> [EXIF] MaxApertureValue : 1.2
> [EXIF] SubjectDistance : 0.54 m
41,44d45
< [EXIF] UserComment :
< [EXIF] SubSecTime : 00
< [EXIF] SubSecTimeOriginal : 00
< [EXIF] SubSecTimeDigitized : 00
273a275,296
> [XMP] XMPToolkit : Adobe XMP Core 5.2-c004 1.13
6881, 2010/06/10-18:11:35
> [XMP] SerialNumber : --redacted--
> [XMP] LensInfo : 50mm f/?
> [XMP] Lens : EF50mm f/1.2L USM
> [XMP] LensID : 241
> [XMP] ImageNumber : 0
> [XMP] ApproximateFocusDistance : 0.54
> [XMP] FlashCompensation : 0
> [XMP] OwnerName : Xxxxxx Yyyyyy
> [XMP] Firmware : 1.2.0
> [XMP] ModifyDate : 2011:07:02 12:37:02+02:00
> [XMP] CreateDate : 2011:07:02 11:30:27.00
> [XMP] CreatorTool : Adobe Photoshop Lightroom 3.
4.1
> [XMP] DateCreated : 2011:07:02 11:30:27.00
> [XMP] RawFileName : _M3I7722.jpg
> [XMP] Creator : Xxxxxx Yyyyyy
> [IPTC] CodedCharacterSet : UTF8
> [IPTC] ApplicationRecordVersion : 4
> [IPTC] DateCreated : 2011:07:02
> [IPTC] TimeCreated : 11:30:27
> [IPTC] By-line : Xxxxxx Yyyyyy
> [Photoshop] IPTCDigest : 77f304a14dd06d1cf19b4f059d6c
904f
275a299
> [Composite] DateTimeCreated : 2011:07:02 11:30:27
288,290d311
< [Composite] SubSecCreateDate : 2011:07:02 11:30:27.00
< [Composite] SubSecDateTimeOriginal : 2011:07:02 11:30:27.00
< [Composite] SubSecModifyDate : 2011:07:02 11:30:27.00
294a316
> [Composite] DOF : 0.01 m (0.53 - 0.55)
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
John: Thanks for the explanation about where and how the Make/Model is stored, that was relay clarifying! And sure, I definitely agree that exiftool is the most authoritative reader. Php is programing language, mostly used for web applications, used by sites as wikipdia, wordpress facebook. You can read more about it's exif reader here http://php.net/manual/en/function.exi...

However, the core of the questions remain.
1. Why was this behavior changed in 3.4? Earlier versions did export the EXIF data in the same as the original file, (readable in php). Has the EXIF spec changed recently?

2. Why have a select option called original, when you don't get the original? It's like asking to see the Mona Lisa at the louvren and you get shown a copy, almost identical.

A real original export would be really helpful! especially for those using LR as a way to sort out and delete images and then exporting them to an other application. To do this manually is quite time consuming since one have to use "Reveal in finder" for the images and then drag'n'dropp / copy&paste.
Photo of Lee Jay

Lee Jay

  • 990 Posts
  • 132 Reply Likes
A lot of metadata stuff was changed in 3.4.
Original means original image data without LR adjustments.
Photo of Mark Sirota

Mark Sirota

  • 146 Posts
  • 29 Reply Likes
Specifically, the change in LR 3.4 was to meet Metadata Working Group 2.0 guidance. http://www.metadataworkinggroup.org/
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
I'm betting php software is just not completely up to spec (made to work as needed for the now, but not to adhere to all future possibilities...), and so the recent Adobe software change has broken it. - just a guess...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 376 Reply Likes
Korv - if you want more of a backup/copy without using Finder, consider using a folder-sync program instead (and bypass Lightroom altogether) - might help...
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Korv - I dunno bout details, but it seems you need to split this in two.

1. An FR/Idea for unaltered originals. I can see two possible variations of this:
1. don't alter originals when saving xmp (use sidecar instead).
2. don't alter originals upon export (export more like backup...)

2. Can you work around the problem with php reading the exif meta? If not, consider working php end from makers of that software too.

Its possible php software people will say this is Adobe software problem, and Adobe will say this is php software problem, or neither cares enough to do anything about it anyway..., but if you can get closer to the source of this problem *and* separate out your complaint regarding the way 'Original' is handled in the first place, you may be more likely to get this resolved.
Photo of Korv Jan

Korv Jan

  • 10 Posts
  • 0 Reply Likes
Rob: It sure seems reasonable that the truth is somewhere in between Adobes and Php implementation of exif data. The fact that it showed up during a LR upgrade made me come to the conclusion it was LR ́s "fault". whether it's a bug or a step for better EXIF spec compliance, I will leave to others with better knowledge of the inner workings of LR.

I will probably rewrite the exif reader functions in the image database, so that it uses exiftoll instead of the native php reader. It's a little performance trade off, worthy to get the data.

And for the original - not so original export, I will just keep my fingers crossed for a original original export.
Photo of Rob Cole

Rob Cole

  • 4831 Posts
  • 382 Reply Likes
Sounds good to work-around if you can - even if it gets fixed, it will be months if not years. But keeping fingers crossed may not get you to the original-original export. Although, a plugin can - since you're a coder, why not learn some Lua?