Adobe DNG Format Improvement -- Immutable and Mutable Regions for Efficient Metadata Update

  • 1
  • Idea
  • Updated 1 year ago
DNG puts the processing settings in the same file as the raw data, which in some ways is nice (single file vs. two files), but the process of updating the XMP data is very slow because it appears the entire file is rewritten.

Two fixes come to mind.

1)
Optionally allow a sidecar file. The location would be metadata in the dng, and this would even allow the sidecars to be in a separate directory. This could even use a catalog variable to relocate the sidecars specific to a photographer, meaning that images could be shared, with developing data separated.

2)
To retain the single file but to allow rapid metadata updates define the DNG as starting with an immutable section that is the result of converting a raw format into the DNG format. Fundamentally this part should not change. This section would start with an offset to the end of this section.

To do a metadata update the DNG is opened,and a small amount is read from the beginning, just enough to know how many bytes exist from the file start to the start of the mutable section. A disk seek operation then positions the I/O to the start of the mutable area and a much smaller amount of I/O is needed to record say a new value for exposure adjustment.
Photo of Mike Hanafey

Mike Hanafey

  • 3 Posts
  • 0 Reply Likes

Posted 1 year ago

  • 1
Photo of David Franzen

David Franzen, Employee

  • 96 Posts
  • 16 Reply Likes
For 1, If you make your DNGs read-only in your filesystem, Camera Raw and Lightroom will write metadata to sidecars instead of to embedded metadata.

For 2, you are describing essentially how updating XMP in file formats based on TIFF already works.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3372 Posts
  • 844 Reply Likes
"For 2, you are describing essentially how updating XMP in file formats based on TIFF already works."

For those not familiar with the DNG format, it is based on TIFF, so David is saying that updating XMP in DNGs already works as Mike proposed.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3372 Posts
  • 844 Reply Likes
"If you make your DNGs read-only in your filesystem, Camera Raw and Lightroom will write metadata to sidecars instead of to embedded metadata."

If you import a read-only DNG into LR CC 2015.9 and do Metadata > Save Metadata To File, LR complains rather than writing a .xmp sidecar:



Too bad, since I know there are many who would rather have metadata always written to sidecars to make their backups more manageable.
Photo of David Franzen

David Franzen, Employee

  • 96 Posts
  • 16 Reply Likes
I apologize. I forgot Lightroom had this limitation that the Camera Raw plugin does not.
Photo of John R. Ellis

John R. Ellis, Champion

  • 3372 Posts
  • 845 Reply Likes
"Optionally allow a sidecar file."

Please add your me-too vote and why you want the feature to this existing feature request: https://feedback.photoshop.com/photoshop_family/topics/have-the-possibility-to-store-the-xmp-metadat...