Lightroom Classic: JPEG Mini standalone is multicore but as a LR plugin, it's constrained to one core

  • 1
  • Idea
  • Updated 4 weeks ago
  • (Edited)
I use JPEG Mini and "JPEG Mini plugin for LR". Lightroom totally hamstrings the software by only allowing one core to work on the export group. 

Tested with 61 NEF Nikon D750 24MP files. 6-core i7 8700K at 4.9GHz

LR + JPEG Mini plugin export combo
5:06

LR export then drag into JPEG Mini standalone
2:40s + 18s = 2:58


Please fix. Unleash the beast.
Photo of seanhoyt-dot-art

seanhoyt-dot-art

  • 312 Posts
  • 51 Reply Likes

Posted 7 months ago

  • 1
Photo of Just Shot Me

Just Shot Me

  • 307 Posts
  • 89 Reply Likes
Why not post this to the plugin writers forums. IMHO it has nothing to do with LR. LR does not control the number of CPU cores used for anything other than LR itself. And even then it is up to the OS to give up CPU time.
Photo of Rikk Flohr

Rikk Flohr, Official Rep

  • 5376 Posts
  • 1119 Reply Likes
Have you contacted the plugin manufacturer for support regarding this?
Photo of seanhoyt-dot-art

seanhoyt-dot-art

  • 312 Posts
  • 51 Reply Likes
The JPEG Mini plugin manufacturer, Beamr Imaging Ltd, did respond that their hands are tied by the LR software. It's up to Adobe to allow this plugin to access multiple threads. It's insane the difference in export time between EXPORT+PLUGIN vs EXPORT then pull into desktop JPEG mini app. 50% time reduction. 
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4290 Posts
  • 1138 Reply Likes
See if you can get Beamr Imaging to post directly here with their views on how this affects their plugin. 
Photo of seanhoyt-dot-art

seanhoyt-dot-art

  • 312 Posts
  • 51 Reply Likes
They said it’s out of their hands.... it’s up to Adobe to implement. They can’t allocate more cores.
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4290 Posts
  • 1138 Reply Likes
While LR 8 renders exported photos in parallel, it does appear to run post-processing actions like JPEGmini serially. 

Here's a CPU graph of exporting 100 JPEGs using JPEGmini as a post-processing action:



And here's a graph of exporting the same JPEGs using the Run Any Command plugin to run the utility ExifTool to remove ICC profiles from the exported photos:



They show the same pattern -- an initial burst using 75% of all the processors (typical for the rendering phase), then a tail where the post-processing action is running the external tool, one photo at a time.

Note that the author of the Run Any Command plugin is the leading expert on LR export plugins, likely more knowledgeable than any engineers currently on the LR team.  (He designed the export/publish plugin architecture and offers many export plugins for sale.)  So it's unlikely that the JPEGmini developer did something "wrong". 

If the JPEGmini developers want forward progress on this issue, they'll likely make more progress if they engage directly with Adobe (e.g. through this forum) rather than through customer proxies.


Photo of Amit Amit

Amit Amit

  • 2 Posts
  • 0 Reply Likes
From my analysis, it seems that the problem is probably because they are using the suggested loop code from LR manual for "processRenderedPhotos()" which is processing one picture at the time.
The plugin call JPGEGminiTool one jpeg at the time and wait for it to complete before moving to the next one:
The call command is something like: "JPEGminiTool -action process -path E:\Lightroom Export\SM5D7315.jpg -result AppData\Local\Temp\LR-3"

If JPEGmini would have a command line option to get a list of files to process that won't be the case.
You could use "Jeffrey’s “Run Any Command” Lightroom Export Plugin"
with a command like: "JPEGminiPro.exe -files {MANIFEST}"
and the process of all the files would be done multi-process just like dragging multi-files to JPEGmini.


It is solve-able by doing different implementation - don't do one picture at the time if your tool knows to run in multi-core.
Collect the names of all the processed jpegs, export the list once everything else done and do one call to your tool with all the jpegs which would run multi-core.


BTW I made the experiment with I9 9900K and 220 RAW files and the times are:
LR + JPEGmini Plugin: Total export time: 15 minutes and 59 seconds
LR + JPEGminiPro : Total time: 4 minutes and 51 seconds. (LR: 3 minutes and 36 seconds. + JPEGminiPro: 1 minute and 15 seconds. )
(Edited)
Photo of John R. Ellis

John R. Ellis, Champion

  • 4290 Posts
  • 1138 Reply Likes
I agree with your analysis. Have you communicated this to JPEGmini support?
Photo of Amit Amit

Amit Amit

  • 2 Posts
  • 0 Reply Likes

I first contact them asking if they are aware of the slowness in LR plugin compare to the JPEGminiPro tool and the answer was they are aware of it and said “Adobe is 1 core processing making the process slower”.

When I asked the support if JPEGminiPro support command line to pass multiple files the answer was NO.

When I suggested the change in the LR plugin I didn’t get a response.

 

BTW I found this AppleScript that might help automate the process without the JPEGmini support:
https://github.com/JamieMason/ImageOptim-CLI/blob/master/osascript/run-jpegmini.applescript

Photo of John R. Ellis

John R. Ellis, Champion

  • 4290 Posts
  • 1138 Reply Likes
The plugin's source code is uncompiled. It would be straightforward for a plugin developer to modify it to run n concurrent instances of the program JPEGminiTool, rather than 1 at a time. A few of my plugins use similar concurrency.