After testing with some single negatives I bought this awesome plugin today. I think the batch-processing possibility is not the key feature of NLR however the one which will reduce the time spent on the keyboard the most.
Trying to batch process 5 Negs I was puzzeled to see the relatively slow processing. Looking at the CPU load while doing so I figured out that either Lightroom (in my case Version 6) or NLR 2.1.2 itself is not working with a lot of threads. Whereas in this exactly scenario you could create a thread for each negative in the batch job and thus run through a job like this with ease.
Therefore I kindly ask you to take this into consideration for future releases
Thereâs a few factors here as far as I understand:
NLP needs to export images from Lightroom to process them, since Lightroomâs SDK doesnât provide any access to the actual image/pixel data. A list of images to export is passed to a Lightroom API, which exports images one at a time. Only Adobe can really speed this part up, threading or not.
NLP is entirely written in Lua 5.x, which is what Lightroom provides for plugin authors. Lua supports coroutines, but not native threading. Lua can call C functions with a bit of encouragement, so if any heavy lifting is done in Lua, parallelism could be achieved by porting that portion to a compiled language and using OS threads that way.
NLP uses a few external compiled programs to do itâs heavy lifting (imagemagick, exiftool, jpegicc). The API Lightroom provides to execute external programs is blocking (LrTasks.execute), but the SDK docs state that these tasks can run cooperativly using Lua coroutines. Not sure if NLP takes advantage of this to run these external programs in parallel, but thereâs some possible gains to be had if not.
I suspect that the limiting factor for speed will be the way Lightroom exports images one at a time. If NLP processes images faster than LR can provide them, any further performance gains would be moot.
For similar images, you may be able to speed up batch processing by analysing/converting a single image and using âSync Sceneâ across the rest. That means youâd only pay the cost of analysis for a single image instead of repeatedly.
@mae - Yes, thereâs definitely room for improvement with the speed of batch processing.
As @stecman has pointed out, the biggest limiting factor is the speed at which LR exports files (which is necessary for the analysis). This takes up more 50% of the processing time (at least on my machine).
In terms of threading, I actually did use coroutines in one of the earlier versions of NLP (and it was faster), but found it to be very unstable and prone to crashing. It was a while ago, and I should probably revisit it. There may just be a piece Iâm missing somwhere.
This is a great technique. Iâve thought about making this an option during batch conversion (so you could select an entire roll, and during convert, youâd have the option to only run the analysis on a few images, and then use that on the entire roll to save time).
I did some tests too (12 cores / 24 threads AMD CPU and nvme drive)
I was able to export 20 Scans in less than 4 seconds (38-49MB each, exported as original nef with xmp data)
Doing the same with 300 Images resulted in a CPU Load of around 33% therefore It seems LR uses around 4 cores / 8 threads at itâs best. Could be better - yes but it was quite fast anyway
Going through the NLP process with the initial 20 Files converting every image it really looks sequential to me. CPU load is unsteady between 8% and does peak at around 50%
I have no experience programming in Lua but what steman wrote about blocking execution could indeed play a part in this game.
Syncing one conversion to the other 19 is almost instant. But this way we loose the benefit of analyzing each image separately which results in a much better conversion.
Personally I think syncing scene is only a valid option if you have a roll of film with the exact same lightning conditions (e.g. studio work)
Upon further research and testing, while Lua 5 does support binary modules it seems that Lightroom disables this feature for plugin developers. The documentation doesnât explicitly say this, but it does say:
Lightroom defines a require() function that works in a similar, but more narrowly-defined, fashion from the version that exists in Lua. The require() function takes a single parameter, which is the name of another Lua file in the same plug-in.
The standard Lua require function normally supports loading binary modules the same way neighbouring Lua files are loaded:
# Binary module compiled to native.dll or native.so
local native = require("native")
Trying this in a Lightroom plugin fails:
Error
An internal error has occured: error loading toolkit script ânativeâ (Could not load script native.lua: doesnât seem to be in the toolkit.)
Additionally the Lua built-in package table isnât defined in the Lightroom Lua environment, so pacakge.loadlib is not available. Lua 5.0âs loadlib is not defined either.
This means that to move anything to a compiled language, the only option is to build a standalone program and run that as a subprocess from the Lua side, by the looks of it.
