Lightroom CC: JPEG format tripping up other programs

  • 21
  • Problem
  • Updated 4 years ago
  • Solved
Lightroom CC writes JPEGs in an atypical layout that trips up other programs. While they appear to be strictly conforming to industry standards, the unusual layout has caused problems for at least two users: one who had problems uploading photos to real-estate services, and another whose own software tripped over the layout.

LR writes the APP1 header as: TIFF header, 114 unused bytes, ExifIFD, IFD1, IFD0. Whereas Photoshop CC uses a more traditional layout: TIFF header, no unused bytes, IFD0, ExifIFD, IFD1. Here's an Exiftool dump of the beginning of a LR JPEG:

And here's a dump of the beginning of a Photoshop JPEG:

I've tested LR JPEGs with 12 Mac and Windows programs and 2 online services, all of which read them just fine:

Mac: Preview, Photoshop CC 2014, ColorSync Utility, Firefox, Chrome, Lattice, Paintbrush, Safari, Word
Windows 8.1: File Explorer Preview, Irfanview, Paint, Windows Photo Viewer
Online services: Flickr, Zenfolio

Even though LR's JPEGs may be strictly conforming, if it wrote the JPEGs without the unused header bytes and it put IFD0 immediately after the TIFF header, as Photoshop does, then there would be fewer problems with other programs choking on LR's JPEGs.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes

Posted 4 years ago

  • 21
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
Also, I seem to recall that years ago, LR 4 Beta had a similar issue, and the final release changed the layouts of JPEGs to be more traditional.
Photo of LRuserXY

LRuserXY

  • 426 Posts
  • 41 Reply Likes
You are right - but I think it was LR 5 beta (or perhaps both?)
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
I checked my email, and yes indeed it was the LR 5 beta. Unfortunately, Adobe has restricted access to the beta discussion forum, but one of the emails contains a link to a JPEG exported by LR 5 beta showing the exact same atypical layout. One of the users affected was complaining about Wordpress choking on LR 5 Beta JPEGs:

________________________________________
From: Les Bessant2 [mailto:forums_noreply@adobe.com]
Sent: Tuesday, May 14, 2013 7:23 AM
To: John R. Ellis
Subject: [Photoshop Lightroom 5] Lightroom 5 exported images and EXIF information, Camera Make points to keywords instead

Re: Lightroom 5 exported images and EXIF information, Camera Make points to keywords instead
created by Les Bessant2 in Photoshop Lightroom 5 - View the full discussion

Thanks for following this up. This is a sample image exported from LR5:

https://www.dropbox.com/s/3sqj3nr66rg...
Photo of Mark Swaisland

Mark Swaisland

  • 10 Posts
  • 1 Reply Like
I've encountered the very same issue: I publish a photo to Picasaweb/Google+ and the metadata isn't displayed. Been doing this for years via LR and it's always worked perfectly - stopped working as soon as I started using LR CC.

No metadata displayed in other applications is a BIG issue for me. The bugs are stacking up...
Photo of Mark Swaisland

Mark Swaisland

  • 10 Posts
  • 1 Reply Like
I just did some testing: if I use an exported JPEG created by LR CC or PS CC 2014 (so ACR 9) then the metadata seems to work OK and is displayed in Google+ as it should be. However, if I use a publish service then it still fails to display the camera metadata.

So what gives? Bug with the publish services?
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
Which publish service/plugin are you using?
Photo of Mark Swaisland

Mark Swaisland

  • 10 Posts
  • 1 Reply Like
Jeffrey Friedl's "Export to PicasaWeb" plugin.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
Unfortunately, I'm not able to reproduce the problem using that plugin, on either Mac or Windows. It would have been nice to narrow down the problem and confirm whether or not it is caused by the atypical JPEG format.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
This problem with Picasa is caused by the XMP bug: http://feedback.photoshop.com/photosh....
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
The problem seems to strike TIFF files as well (which use the same representation of metadata). In this thread:

https://forums.adobe.com/message/7493633

a user reports that HP MediaBin (HP's enterprise digital-asset management product) is not able to display the metadata of TIFFs exported from LR 6.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
LR 6 can write invalid XMP metadata (a true bug), as described here: http://feedback.photoshop.com/photosh...

To distinguish that bug from the issue here, you'll have to use a photo with and without a large number of develop settings.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
To work around the problem described here, you can use the Run Any Command plugin: http://regex.info/blog/lightroom-good...

Use this command line in the Command To Execute box of the Run Any Command post-process action:

exiftool -all= -tagsfromfile @ -all:all -unsafe "{FILE}"

This idiom deletes all of the existing metadata from the image and rewrites it from scratch using a more typical layout that doesn't confuse software that doesn't strictly obey the industry standards. Make sure you have Exiftool version 9.94 or later -- earlier versions would choke on LR images containing large numbers of brush strokes from the adjustment brush.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
That command line should have the "-m" option to handle XMP metadata with large numbers of brush strokes:

exiftool -m -all= -tagsfromfile @ -all:all -unsafe "{FILE}"
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
The maintainer of Exiftool pointed out the command line should be:

exiftool -m -all= -tagsfromfile @ -all:all -icc_profile -unsafe "{FILE}"
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
The latest version of the command line:

exiftool -m -all= -tagsfromfile @ -all:all -icc_profile -unsafe -overwrite_original "{FILE}"

This will avoid creating backup files of the originals, which aren't needed in this case.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
PHP's exif_read_data() appears to be incompatible with LR 6 JPEGs. PHP, of course, powers a huge number of Web sites. See this thread: https://forums.adobe.com/thread/1835290
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
That user was using PHP 5.6.
Photo of Tobias Frenzen

Tobias Frenzen

  • 5 Posts
  • 0 Reply Likes
I'm using 5.6.8 and since upgrade to LR6 my EXIFs are messed up on my site!
Photo of Stefan Uchrin

Stefan Uchrin

  • 4 Posts
  • 1 Reply Like
Yes indeed, we use Lightroom 6.01 to export our photos as JPEG for Piwigo under PHP 5.6. Und here you can see http://s.edv-tipp.de/fpvth that

COMPUTED.Copyright: efan Uchrin, (C) by St
COMPUTED.Copyright.Photographer: efan Uchrin
COMPUTED.Copyright.Editor: (C) by St

are broken. OK, we go back to LR 5.7.1 and wait for a better LR 6!
Photo of Phil Harvey

Phil Harvey

  • 12 Posts
  • 1 Reply Like
Interesting. It seems that the unused bytes are an aborted attempt at IFD0. It contains most of the information from IFD0, except for the orientation (in my sample), then IFD0 is rewritten at the end of the EXIF APP1 segment.
Photo of John Ellis

John Ellis

  • 117 Posts
  • 12 Reply Likes
Another LR user can't upload LR 6 JPEGs to his client's Web site: https://forums.adobe.com/thread/1842441
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
Dave Palmetto reports that the JPEGrescan plugin also works around the problem:

http://www.capturemonkey.com/lrjpegre...

https://forums.adobe.com/message/7540...

You need to set the Strip Metadata option in the plugin, so this solution is only useful for those, like Dave, who were uploading to sites that choked on LR 6's JPEGs and don't care about transmitting the metadata.
Photo of David Clark

David Clark

  • 1 Post
  • 0 Reply Likes
I'm the user who had the problem John R Ellis is reporting on May 13, 2015. When I switched from LR-5.7 to LR-CC, my jpgs began showing up very blurry after being uploaded into my client's reporting software. Apparently the client's report software that worked fine with my LR-5.7 jpgs was ignoring the full uploaded resolution from LR-CC jpgs, and instead magnifying the embedded thumbnail. This produced a low-resolution image that appeared out of focus. This is a high-volume client with multiple uploads daily and hundreds of photos every month, so a reliable, clerically simple solution that required minimal additional labor was needed.

Due to the assistance of the LR/PS/CC community, I can report four workarounds:

1. Export TIF files. For my user reports, these worked fine but of course were much larger than the JPEG files I had been using.

2. Use the Lightroom Web module to export, thereby producing classic jpgs in the "Large" folder. The jpgs created this way do not have the problem with mis-interpretation by my client's web reporting system. The disadvantage to this is that a multitude of unnecessary files are also created every time the jpg is exported.

3. Use the Run Any Command plug-in to execute an exiftool command line to edit the metadata back to classic style. The first command line I used created a duplicate copy of the original jpg with a ".original_jpg" suffix, but I am advised that using the command line

exiftool -m -all= -tagsfromfile @ -all:all -icc_profile -unsafe -overwrite_original "{FILE}"

would eliminate this unnecessary duplicate file, leaving only the LR-5.7-compliant jpg. This works but requires installing two pieces of software (exiftool and Run Any Command), plus having some expertise with scripting. On the plus side, this can be easily added to an export preset so it's very set-it-and-forget-it.

4. Use the JPEGrescan plugin. This requires installing only one piece of code, the plugin itself. Its disadvantages are that all metadata is stripped (not a concern in my situation) and there is a quirky pop-up every time the plug-in runs. This can also be included within an export preset.

I would like to see Adobe provide an export option that supports exporting in the "classic" format used by LR-5.7.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
Here's another person whose Android app(s) on a Sony Xperia phone and tablet can't read metadata in LR-exported JPEGs:

https://forums.adobe.com/message/7544...
Photo of Mark Swaisland

Mark Swaisland

  • 10 Posts
  • 1 Reply Like
Some other users reporting issues relating to exported files and metadata:

http://www.dpreview.com/forums/thread...
Photo of Stefan Uchrin

Stefan Uchrin

  • 4 Posts
  • 1 Reply Like
For the "Bugs in the LR 6.0.1 EXIF-Data" (here, I wrote about the problem: https://forums.adobe.com/thread/1835290) now we use "Jeffrey’s “Metadata Wrangler” Lightroom Plugin" as workaround - with these settings:



Link to Jeffrey: http://regex.info/blog/lightroom-good...
Photo of Patrick Fondanèche

Patrick Fondanèche

  • 13 Posts
  • 2 Reply Likes
It works for me but I made a small change. I had to preserve all the Date/Time fields otherwise I could not display this information in WordPress.
Great plugin!
Photo of Tobias Frenzen

Tobias Frenzen

  • 5 Posts
  • 0 Reply Likes
My website has started showing incomprehensible strings in EXIF data display fields since upgrade to LR6 too. Not cool! Any idea when this might be fixed???
Photo of Stefan Uchrin

Stefan Uchrin

  • 4 Posts
  • 1 Reply Like
Please see the last message above your entry
Photo of Tobias Frenzen

Tobias Frenzen

  • 5 Posts
  • 0 Reply Likes
Thanks, I'll give it a go, but this is a workaround, not really a fix. Would be nice to hear something official and find out whether this was an intended change and if the problem lies with the PHP library reading the EXIF info or if this is indeed a bug in LR. Seems it could be either since Windows/Mac reads the data correctly.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
It would be nice to get feedback from Adobe on this. However, I think it's best to avoid discussions about the meaning of the word "bug" and speculations on what was intended and focus on what LR's customers want.

As explained above, LR 6 strictly conforms with the industry standards, while PHP exif_read_data(), along with other applications, does not. As a practical matter, however, I think nearly all LR's customers expect LR to export JPEGs and TIFFs in a format that nearly all widely used software can read. Previous versions of LR met that expectation, but LR 6, in our opinion, does not. That is, while LR 6 conforms with the de jure standards, it does not conform with de facto practice.

The more evidence we can collect on that point (by having people come here to vote and add details of their experience), the more likely that Adobe will prioritize the issue.
Photo of Stefan Uchrin

Stefan Uchrin

  • 4 Posts
  • 1 Reply Like
Photo of Patrick Fondanèche

Patrick Fondanèche

  • 13 Posts
  • 2 Reply Likes
I have the same problem with LR6. When I import JPGs to WordPress, camera EXIF field is missing in the Wordpress Library
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
Some newly discovered details: LR 6 makes three passes to export a file: 1) It writes the file, including with full metadata. 2) It reads the file back into memory. 3) It rewrites the file with updates to face-detection region information. Past versions of LR only used one pass, as does nearly every other program.

The .jpg written on the first pass uses the typical EXIF IFD layout. But the re-written file from the third pass uses the atypical layout. I'd wager the second two passes were introduced to support face recognition, since it's the face region information that's getting changed on the third pass. I'd guess an inexperienced programmer couldn't figure out how to change the existing code to write all metadata on the one and only pass, or he/she was pressured to hack something quickly, so he/she added additional passes (with bugs to boot).

See this post for more details than you want: http://feedback.photoshop.com/photosh...
Photo of Eider Oliveira

Eider Oliveira

  • 1 Post
  • 0 Reply Likes
Also having this problem... disappointing.
Photo of Patrick Fondanèche

Patrick Fondanèche

  • 13 Posts
  • 2 Reply Likes
Use the Metadata Wrangler, it works very well. It takes a bit of time to install bu then you can save parameters as a preset and it becomes completely transparent.
Photo of Jose Vazquez

Jose Vazquez

  • 2 Posts
  • 0 Reply Likes
Same problem here. Re-saving file in Photoshop fixes the EXIF data but it shouldn't have to be that way.

Hope this gets fixed soon!
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
A workaround is to set the export option Limit File Size To to be a very high number, e.g. 30000 K (30 MB). This was suggested by an Adobe employee in another bug report that might have the same cause as this issue:

http://feedback.photoshop.com/photosh...

I've tried a couple of exports and it seems to work.
Photo of Jose Vazquez

Jose Vazquez

  • 2 Posts
  • 0 Reply Likes
Yep, it does work. Thanks for this workaround John.
Photo of John R. Ellis

John R. Ellis, Champion

  • 4272 Posts
  • 1137 Reply Likes
This bug still exists in LR CC 2015.1.
Photo of Edward Kidwell

Edward Kidwell

  • 1 Post
  • 0 Reply Likes
Many people having this issue are on PHP-based servers. Just an FYI, this issue has been reported as a bug on the PHP side of things, as well.

http://news.php.net/php.bugs/193576

Hopefully, someone somewhere can get this fixed before I have to write my own EXIF-parser. :)
Photo of Glen Tillyard

Glen Tillyard

  • 2 Posts
  • 0 Reply Likes
I've upgraded to LRCC and exported pictures do not show EXIF data Picasa version 3.9.139. Bit of a pain so pleased I didn't uninstall LR5.
Photo of jukka vuokko

jukka vuokko

  • 5 Posts
  • 4 Reply Likes
Same here. Fortunately the workaround that John R. Ellis describes works with Picasa too.
Photo of Glen Tillyard

Glen Tillyard

  • 2 Posts
  • 0 Reply Likes
30000k fix worked for me too.