Adobe Lens Profile Creator, version 1.0.4. for Windows
on a 64-bit Windows 10 computer with 16 GB RAM, 128 GB system SSD.
The problem: I tried to calibrate a lens for my Pentax K-1 and its 36 MP sensor - and found that the Adobe Lens Profile Creator (ver. 1.0.4.) was not willing to process the full resolution DNGs with the required test chart shots. It gave a warning that - looking for the corners of the test chart - the memory requirements were exceeded and that I should try with a lesser number of images. I reduced the nine provided shots to three, but got the same warning - starting with the first pic analyzed.
I then converted the DNGs to full resolution JPGs using LR CC and tried again - same result.
For a third try I limited the longest edge of newly converted JPGs to 3000 px - and the lens profile creation process with the same pics (now downsized) went through. But I found that the created profiles weren't available for my DNGs - and that's what I wanted them for.
I found a workaround for that - simply switched the RAW tag in the created .LCP file to "true" - but of course am not satisfied with this solution. I would really like to be able to process the full rez DNGs without any workarounds. Are the 36MP DNGs/JPGs simply too big for processing?
The Lens Profile Creator has not been updated in several years, so there may be a technical limitation with the larger K1 images.
Would you be willing to upload the images to DropBox (or similar) so that I may do some troubleshooting?
If so, please send me a link to firstname.lastname@example.org.
Also, please let me know which lens(es) you are attempting to profile, if it isn't obvious in the image metadata. I ask because Pentax typically sends Adobe pre-built profiles to be included in ACR/LR. It's possible we may already have a profile on the schedule for a future release.
Thanks for the quick reply.
The lens I profiled on the Pentax K-1 is a Sigma 24mm AF Super Wide II for K-Mount. It's an older lens that doesn't have specific lens info in the metadata - LR just identifies is as "Sigma" (like another Sigma I have, an older lens too - the Sigma 50-500 f4.0-6.3 APO DG for K-Mount which also identifies as "Sigma" without any further info).
The next lens I plan to profile on the K-1 is a Pentax FA J f4.0-5.6 18-35 - AL - and boy, this lens does seriously need profiling ... :-)
I'm uploading a 374 MB ZIP file right now - it contains the nine f11 DNGs I made profiling the Sigma 24mm lens mentioned above - via WeTransfer. You'll get some download info from WeTransfer hopefully in the next 30 minutes or so. I have further profile shot groups available if you need them - from f2.8 up to f22 of the same lens.
Thanks a lot for your help!
Edit: The WeTransfer upload has been completed. You should get a corresponding message soon.
But see attached a screen shot with the warning message popup - I was just trying again to create the profile with exactly the same DNGs I sent you.
Thanks for your offer for further profiling. Since a lot of data is involved - I plan to do 180 single shots for the FA J 18-35 profile (4 apertures, 5 focal lengths, 9 shots each) giving around 7 GB - I'm a bit hesitant to bother you with it. :-) We'll see. Maybe you'll find a solution before I find the time to take all those shots. ;-)
One extra thing: I tried varying the size of the Calibration Cache in the preferences dialog, but this didn't help either. Default was 1 GB, but even with a setting of 20 GB the behavior was the same.
First I followed your suggestion and increased virtual memory in Win10 on my computer - it was on automatic handling (with a maximum short of 3.000 MB) - I switched it to manual, minimum 5.000 MB, maximum 10.000 MB, re-booted Windows so that the new settings could take effect, and tried again - same error message appeared in ALPC, again while processing the first pic.
Hadn't any more time this morning, but later had access to two other machines - first a fairly old Mac (but running with the current Mac OS El Capitan), second a Win10/64bit machine again with 16 GB of RAM, but using a different processor - an i7; my own computer uses an AMD A10.
First tried everything on the Mac: No problems at all, everything went smoothly, profile was created fast.
Then tried the Win10/i7 machine: processing went further than on my Win10 computer - but didn't finish either: it stopped at the 6th pic with the same error message as posted before,
Finally I increased virtual memory on this computer too - switching again from automatic handling to manual, but this time trying settings of 20.000 MB minimum and maximum. After a re-boot ALPC was started again - and this time created the profile completely and without problems. Wow.
Still have to check if the 20.000 MB min/max settings for Win10 virtual memory also works on my home machine. Since there is a difference in handling the files if Win10 automatically allocates virtual memory - my computer stops processing at the first file, the other one with the i7 at the sixth - I sincerely hope that I can get my machine to work through all files too. Will have access again to it in about 10 hours.
Meanwhile any recommendations on your side how to adjust min and max setting for virtual memory?
Unfortunately the problem was only partly solved. Memory management of ALPC is so weird that I can - with my new virtual memory setting - profile exactly one set of nine images. If I try to profile a second one in the same session - and for my wide angle zoom I had to profile 19 sets of nine images yesterday - I get the same memory error message as before. I had to close ALPC after profiling one set, restart it, reload the current project, select the next set and profile that.
A very slow and cumbersome process which I gave up on after the first five sets or so - another reason being that processing of the large DNGs took a long time on my AMD A10 machine, and due to heavy vignetting of my lens wide open a lot of those shots with the calibration target placed in the corners weren't detected by ALPC.
I instead turned to my "solution" described in the first post in this thread - converting the DNGs to 3.000 px wide JPGs, processing those and enabling the "RAW" flag in the resulting profile. This was done much faster, and I was able to process all 19 sets in one go - without any memory errors. Also with the JPGs APLC could detect almost all charts (I upped exposure half an EV before exporting, that helped with the strong light falloff in the corners).
I have to test the resulting .lcp file a bit, but it sure looks good so far - applied to JPG and to DNGs alike.
So if you ever update APLC, it really would be a good idea to make it 64 bit compatible - as you suggested yourself. And hopefully the memory problems can be solved with that.
Given the ALPC is 32-bit, I am not surprised it struggles with the larger images and multiple profiling within a single session. Also, there may be other implementation issues with how it handles memory independent of the ability to address larger amounts of system RAM. I'll bring this up with the team as something to consider addressing in the not-too-distant future.
If your camera has a setting to reduce the image resolution when capturing the raws for creating the lens profiles, try that as a workaround. The resulting lens profile generated from these lower resolution image capture would be just as effective for the lens geometric and vignette corrections, regardless of the image resolutions. For correcting the chromatic aberrations, Lr/ACR currently relies on an automatic algorithm to estimate and remove them. So it does not matter. either.