Camera Raw/DNG: Loss of information when converting raw to DNG

  • 1
  • Question
  • Updated 3 months ago
  • Answered
  • (Edited)
Hello. My questions to developers of DNG concern the DNG format, into which I have been converting all RAW photos, considering that DNG is a complete analogue of RAW. I use Canon equipment. 
1. I recently have found that in many photo contests, if a photo reaches the final and claims prizes, confirmation of the rights to this photo (authorship) in the form of RAW files is required, and the DNG format is not considered as such. 

2. Once I had a claim about a Canon lens purchased - on specific photos as an example. The support required me to submit RAW files. The DNG files were not accepted by Canon support and I was told that they lacked some important information. 

Question 1. Please confirm that when converting RAW to DNG some information is lost, that 
a) does not allow to identify the authorship of the photo, 
b) does not allow to perform photo equipment service support.
Please indicate which information is being lost. 
If the statements written above are incorrect, please give a professional refutation. 

Question 2. What recommendations can you give to avoid the problems described above when using DNG?

I believe that these issues are very important for many photographers around the world.
Thank you!
Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes

Posted 3 months ago

  • 1
Photo of David Converse

David Converse

  • 680 Posts
  • 207 Reply Likes
Demanding a RAW file to confirm ownership is meaningless. What if you shot using JPEG? Or on film?

Canon would need to let you know what data is missing. Many vendors have their own proprietary data that is not industry-standard and the DNG spec wouldn't deal with that. I'm not sure why its a big deal- shoot a sample photo and send the RAW file.

Or, embed the RAW inside the DNG.


Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
Thank you, David!
1. Regarding the photo contest, it would be more correct to say about the authenticity of the photo, not about the authorship. Here is a quote from the rules of the one contest in my translation:
"Also, in order to establish authenticity, it is necessary to provide RAW files (.CR2, .NEF, .ORF, .PEF, etc.) or original uncorrected JPEG files ... DNG files are allowed only if they are the main format of the RAW camera"

2. Here is the answer of Canon support in my translation:
"ORIGINAL files in both JPG and RAW format contain extensive service information based on which we can often draw some conclusions. Any conversion of the original files leads to complete or partial loss of it and partial or complete loss of even standard EXIF ​​information. This does not allow us to make any reasonable assumptions about the reasons for the occurrence or the presence of problems when analyzing images. Converting original RAW files to DNG format leads to a loss of debug information. We can not comment on whether there is a loss in image quality and the possibilities of its processing in this transformation."

I use old Canon EOS 6D camera.
(Edited)
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
It's my understanding that Adobe includes all camera manufacturer proprietary data in the DNG file in MakerNotes. Adobe will sometimes support a new camera model with a new feature such as Canon's 'dual-pixel' technology before supporting that specific feature. In most cases Adobe will attempt to support the new feature in an update, which they did for Canon 'dual-pixel support in LR 6.8.

https://helpx.adobe.com/x-productkb/multi/editing-canon-dual-pixel-raw-cr2.html

As far as authorship is concerned the camera embedded copyright information in a raw file cannot be overwritten using LR, but there are many other metadata editors that can. Given that fact I don't see how a camera raw file proves authorship any better than a DNG file. Perhaps someone else landing here knows otherwise and will respond.

https://en.wikipedia.org/wiki/Comparison_of_digital_image_metadata_editors




Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
Thank you, Todd!
See, please, my comment given above on David’s answer.
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
"ORIGINAL files in both JPG and RAW format contain extensive service information based on which we can often draw some conclusions. Any conversion of the original files leads to complete or partial loss of it and partial or complete loss of even standard EXIF ​​information.
That's would be the case for "conversion" to JPEG, TIFF, or DNG file format. However, I believe some of the metadata editors at the link I posted simply copy ALL of the raw CR2 file data and then only change specific non-proprietary EXIF data. This shouldn't disturb any Canon "service information," at least from the standpoint of a non-Canon staff person examining the file (i.e. contest judges).
 Further, Canon's raw editing software (DPP) can "change" information in the file as a 'Save' operation, which makes it no longer an "original" file. User who do so cannot submit these Canon SW edited files? I'm remain skeptical of the value for such a restriction. As they say "Rules are Rules," but often able to be broken!

Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
I have started this conversation in the hope of getting arguments in defending my rights for photo equipment service and participation in photo contests, since all my photos are now in DNG format.
As far as I understand, the answer of the Canon service was given about all possible forms of converting RAW to DNG, regardless of what program was used. At the same time, they did not even ask me what programs I use. Information is lost - that's all!
But I converted the photos in question to DNG when importing from them from flash drive to a disk in LR.
Todd, one personal question: are you a representative of the Adobe development team? If not, do you know how to draw their attention to my questions?
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
Todd, one personal question: are you a representative of the Adobe development team? If not, do you know how to draw their attention to my questions?
No I am not, but Adobe Engineering does monitor posts in this forum.

IMHO it's a question that has only one "academic" answer. Neither raw files or DNG files are valid proof of ownership since the file metadata can be edited without detection by lay people (i.e. judges). However, there is one possible way to prove original file ownership of the images through use of the file's Maker Notes Serial Number field. The camera would then have to be physically inspected by the judges to confirm that its serial number, make and model matches the file data. Lacking that a DNG file is just as valid proof of ownership as an original raw file.
Photo of David Converse

David Converse

  • 680 Posts
  • 207 Reply Likes
I'm not sure that getting service or entering a contest is a "rights" issue. For service, you must still have the lens or you wouldn't care. So shoot a few RAW files and send them.

Contests, well, that's up to the judges. What Adobe says is irrelevant.
Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
It is a pity that so far none of the engineers have responded. In such a situation, the only argument in the dispute can be the phrase: "You know, but here Adobe engineers write that ..."
As for authorship and ownership, I corrected above: in the rules of the photo contest it was said "authenticity". What is meant by this was not explained. Perhaps - "just a RAW file", the rule for the sake of the rule, which is pretty stupid :-)
Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
David,
"shoot a few RAW files", yes, I know, but those shots was in specific place and specific lightning conditions. 
I also think that my question about the completeness of the conformity of the DNG and RAW formats is crucial and can be important in many situations.
Photo of David Converse

David Converse

  • 680 Posts
  • 207 Reply Likes
Have you looked at the files with a hex editor? And I gave you a solution- embed the original RAW file in the DNG.

I'm not sure why you are converting files anyway, leave them as RAW is probably better.
Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
David,
What should I see in the hex editor? What information should I search there and for what purpose?
Your solution is not for the specific case under discussion (photo contest and lens problem), but advice for the future.
Yes, it is probably better to leave Raw, however I began to convert files because I didn’t like to have two files per photo, and also following the advice of Martin Evening from one of his books.

Let us summarize the current results of the discussion on fundamental general issues. So far, no confirmation or refutation of the following statements has been received:
1. that when converting to DNG important information is lost;
2. that DNG files cannot be used to confirm the authenticity of photos;
3. that DNG files cannot be used for servicing photo equipment.

Alas!
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
Yes, that about sums it up concerning the issues of converting raw files to DNG format and deleting the original raw files.

Along with service/debug information being lost the ability to use proprietary features not supported in LR/ACR such as dual-pixel are lost forever! In addition when using 'Save Metadata to File' your backup software detects this and will transfer the entire file to the backup location (external drive, NAS, Cloud). When using raw files a very small XMP file is created or updated, which greatly reduces the backup time.

I only use the native DNG files created by LR Photo Merge to Panorama and HDR, but still retain the original raw files for all the reasons stated above. DNG file format is useful for specific purposes such as a multi-user editing environment. Lossy DNG file format is also useful for the purposes I've outlined here: https://feedback.photoshop.com/photoshop_family/topics/lightroom-cc-lossy-dng-workflow-suggestion

You can choose to embed the original raw file into the DNG and later extract it using the DNG Converter. This should be an exact copy of the original raw file, but the judges still may not accept it as "authentic." The file size increase by ~2x so no real benefit other than reducing file count and it also increases your file backup time by ~2x. This doesn't help you with your current orphaned DNG files, but something to consider going forward.
Photo of David Converse

David Converse

  • 680 Posts
  • 207 Reply Likes
All three of your statements are subjective, not objective. There is no definitive answer that will cover all situations.

What matters more is, will DNG fit YOUR specific needs? Apparently the answer is no.
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
David, just our of curiosity do you convert your files to DNG file format on import and delete the raw files? If so you can explain WHY you do so and WHAT benefits that provides you over the raw file format? That would be helpful to the OP and myself. Thank you.
Photo of David Converse

David Converse

  • 680 Posts
  • 207 Reply Likes
Do I? Absolutely not.
Photo of straannick

straannick

  • 8 Posts
  • 0 Reply Likes
Todd. Given the problems we are discussing and to which Adobe engineers are not responding, your question can be summarized as: "why there is a DNG format and what is the point of using it?" :-)
(Edited)
Photo of Todd Shaner

Todd Shaner, Champion

  • 1452 Posts
  • 486 Reply Likes
Adobe's objective in creating the DNG specification was to establish a "standard" file format that could be adopted by camera manufacturers and software developers. To date many camera manufacturers have adopted DNG file format, but the major manufacturers (Canon, Nikon, Sony) have not. Why?

This is a very competitive market and camera manufacturers maintain their product sales by offering proprietary and sometimes patented "new features," which will never be supported by the Adobe DNG specification. If you own a camera with one or more of these unsupported features your only option for using them is the "original" proprietary camera raw file or camera JPEG file. In fact adoption of these features in the DNG specification makes them no longer proprietary and could possibly affect the manufacturer's patent rights. I don't see this situation changing in the future, but camera manufacturers have worked with Adobe to support specific features such as built-in lens profiles.

So back to your question, "Why there is a DNG format and what is the point of using it?" I've provided two examples where (to me) DNG file format is useful, which are Photomerge Panorama and HDR DNGs and Lossy DNGs. I also convert raw files to DNG file format for troubleshooting in the forums and for sharing with family and friends who also use Lightroom, but I always save the original raw file.

Here's what Adobe says:

https://helpx.adobe.com/photoshop/digital-negative.html#key_benefits

To see what others say Google raw versus dng.
(Edited)