This is of course happening the very day after I sold my other scanners and am relying on NLP for my workflow - there’ve been some metadata growing pains finding the path between my old systems and NLP’s metadata workflow. But tonight, my two most recently scanned rolls will not to write the NLP custom data to the appropriate fields on export. I’ve tried multiple variations: exporting singly with a positive to the desktop and other folders, I’ve tried exporting in-place or to subfolder and adding back to the catalog, stacked and not stacked; i’ve turned off and on dozens of options in the export selection fields; i’ve renamed the files, manually synced the metadata; I’ve used the Flickr and IG export screen options, as well as just the default and selected/deselected options manually myself, on and on. Once the file export is complete there are some occasions that ONE option works which is “Retain Custom LR Metadata”: for some file exports in the folder that data will still be present when LR adds it to the catalog, but the camera make, model, etc do not appear. Verified outside LR in explorer and by uploading a number to flickr. The title and captions do not populate, newly or appended, etc etc.
I had this happen several times during my first few runs in previous weeks, but only occasionally, and then it would suddenly work fine - I thought it was just something I did incorrectly on a one-off, but it seems there’s a deeper issue here. Or, more hopefully, something painfully obvious is going on that I just can’t see and have been simply missing during the 6 hours I’ve been chipping away at this. I can certainly provide more detail but do have the energy left tonight, so am posting this much in the hopes it may be enough. Many thanks.
KPR
Hmm… let’s see if we can figure out what is going on.
What version of Lightroom Classic are you on?
Dumb question, but I’m assuming that you have added custom metadata in the Negative Lab Pro metadata section in Lightroom? And that you’ve done this for all the files you’re exporting?
Another dumb question, but also, I’m assuming you are using the Export module in Lightroom, and starting with one of the Negative Lab Pro export presets (like “NLP - Export for Flickr”)
If you look at the metadata of the exported file, do you still see the original metadata for things like camera make and model, or is it now blank? For instance, if you digitized with a Fuji XT2, does the metadata in the exported file still show “Fuji XT2” as the camera make/model, or has it erased this?
Have you tried exporting just one image (rather than many images) and see if that works?
Does the path to the original file, or the path to the Negative Lab Pro plugin, contain any symbols or anything else non-standard?
Input types: Multitouch: Yes, Integrated touch: Yes, Integrated pen: Yes, External touch: No, External pen: No, Keyboard: Yes
*Graphics Processor Info: *
DirectX: Intel® UHD Graphics 620 (26.20.100.6951)
*Installed Plugins: *
1) AdobeStock (not setup)
2) Facebook (not setup)
3) Flickr
4) jf Flickr
5) LR/Instagram
6) LR/Mogrify 2
7) Negative Lab Pro
Yes, all the metadata was added in the custom NLP fields using that new view
Yes, export module within LR is being used. I’ve started with “NLP-Export for Flickr” and have also attempted using the other NLP preset sidebar selections, as well as choosing none of these (just the default NLP selection chosen from the top drop-down, then choosing various options for the export in the main window individually, including leaving everything as default (unchanged). I’ve also tried the “Positive Copy JPG” and TIFF NLP presets.
Original metadata for the file remains; I’ve digitized with Pentax K-3 and FA 50mm Macro, both of which always remain. Like I mentioned, the only change that seems to stick is the preserved, new NLP specific metadata fields when I choose the option to “Retain Custom LR Metadata” when also choosing to “add to this catalog”… Though not always.
Yes, in fact what I have not tried is exporting multiple images. All my attempts have been just for a single export thus far.
No, no extended or non-standard characters are present in any of the files names. This is an example of one that I’ve tried: File Name:Konica FT-1 Motor - Ektar 100 - 1st Test Roll r-0380-20.DNG
However, (and perhaps this may get us somewhere) I renamed all files in this folder after the NLP conversion took place, thus there is also populated the Preserved File Name": untitled-24.DNG
EDIT** The filenames were all changed initially using the batch-rename function with LR. I now just tried changing the name of the one I noted above back to the original name, untitled-24.DNG, and exported again. Same result unfortunately.
One more note: one change I made to my LR install was in the catalog settings (for performance) to disable “Automatically write changes to XMP”. Not sure if that’s pertinent here, but I just re-enabled that setting and exported again; no change.
Can you show me the full path to both the plugin and the original image?
The metadata is being written to file by a tool called ExifTool… there are a number of “gotchas” based on both that tool and the environment that LR executes in that could be preventing it from writing to file (as an example, if there are chinese characters anywhere in either of those paths, it will likely fail, or if the plugin in on a different drive than the original file). So would be helpful to see that!
Nate - (I was typing this reply out before you recent reply came in - going to post this and then address your most recent question re: plugin and file paths)
I just had some success but trying to determine exactly what went wrong in the first place. The three recent, different rolls that are having this issue were all added within a day of one another, at different times. One was positive film, so it received no NLP conversion, only attempted metadata additions. The second was a roll of Delta 400 which was converted with b&w white profile in NLP. The third a color negative roll of Kodak Ektar. Only mentioning this to point out they each had different variables at work but all had the metadata write problem.
My success just now was this: I went back to a one-off scan of a single frame from about a month ago (a color negative Portra 400 scan, same Pentax K-3 equipment used, etc.) which is one that I’ve been able to export with proper metadata writes this entire time, then and currently. I also went to a known problem file from the current Kodak Ektar scan batch, one of the many current that have never exported properly. I took both of these .DNG files and copied them into a new subfolder within my current scan directory (adjacent to the problem scan folders), then renamed them in Explorer, and synchronized that new folder location in Lightroom. They each showed again as negative images, but I then went to reconvert but Lightroom recognized each of them specifically and my previous NLP conversion setting were reapplied without need to reconvert (as well as their crop settings, etc.). The custom NLP metadata for each was not present in these new versions however so I added just basics (Camera Make, Model, Film Stock, Film Speed, Film Format) respectively to each (which were both different, but both populated from the autofill drop-downs for previously used selection). I then exported the first, older known-good Minolta Autocord file with default settings, unticked resize and sharpening and chose to add to same folder as original, add to catalog and add to stack. It worked fine as previously. I did the exact same for problem file and it too worked fine in this case.
File example not working ORIGINAL:
path: C:\zDrive\zFOTOS_RAW_Master\zFOTOS RAW 2020\Filmscan RAW\Konica FT-1 Motor - Ektar 100 - first test roll
filename: Konica FT-1 Motor - Ektar 100 - 1st Test Roll r-0380 RESCAN-1.DNG
File example working ORIGINAL:
path: C:\zDrive\zFOTOS_RAW_Master\zFOTOS RAW 2018\Filmscans RAW\Medium Format\Autocord - Portra400 - Woodard Bay Early March r7814
filename: untitled-1.DNG
File example both working NEW LOCATION/NAMES:
1.
path: C:\zDrive\zFOTOS_RAW_Master\zFOTOS RAW 2020\Filmscan RAW\tests for NLP
filename: NLP Test 02 GOOD - Minolta Autocord - Kodak Portra 400.DNG
2.
path: C:\zDrive\zFOTOS_RAW_Master\zFOTOS RAW 2020\Filmscan RAW\tests for NLP
filename: NLP Test01 BAD - Konica FT-1 Motor - Kodak Ektar 100.DNG
More info - OK, I’m getting somewhere it seems. I hope this gives you some ideas.
I reimported another RAW file from that problem Ektar roll into the same directory with the rest of them, but I just left the camera-generated filename (disabled file renaming on import) as it was to differentiate it from the first copy of the same .DNG. I unticked “Don’t Import Suspected Duplicates” obviously as well to get it into the catalog. The only import setting ticked was my normal metadata preset which adds general creator IPTC fields and a few keywords that are always the same.
It imported and I proceed to convert it with NLP in the standard way (flip, crop, set WB etc) and the conversion was fine. I then went to the NLP metadata fields and added the same basic camera info as I did for that last successful test. I then exported just the same as before (previous settings still in place), same file location, add to catalog, add to stack etc. The export worked; NLP metadata was preserved and camera info showed in the default view as the film camera/lens specified in the custom fields. Cool.
So I then left that conversion there and clicked on the original non-working DNG, double-checked it’s NLP metadata (which had much more detail: lab, date, scan equipment, digitization notes, etc.) and then CTRL-selected the newly imported .DNG along with (old file still highlighted) and clicked Sync Metadata to copy those more complete NLP fields to the new file. That worked fine - double checked and all was there. I then exported the new file again with the export settings still untouched as before and this time, as with the original, the new .JPG file did not write the new data; the camera as usual still showed as Pentax K-3 with FA 50mm Macro, though the NLP custom fields did remain preserved.
Clearly it’s something going on with that metadata sync but sure have looked over all the fields it writes when the screen comes up before it’s all applied and I can’t find anything strange. The previously affected rolls/files that I mentioned earlier (the positive scans and the B&W conversion rolls) all had different custom metadata fields since they were all obviously very different, but the one thing they do have in common with this color-neg roll is that I synced NLP metadata using the LR sync feature.
Sure hope this puts a lightbulb above your head. Mine is raw from the head-scratching.
Just gets weirder and weirder. That last .DNG which failed to write the meta on export after syncing metadata from the other rolls in the group I could not seem to fix. So I renamed it and imported that exact same .DNG for a third time renaming that one again. Performed the same test as before - converted it, added the same few NLP custom fields and exported it with a successful write just like last time.
But this time, after that successful export, I went back into the NLP metadata fields for it (the new, working one) and filled out the rest of the info that I wanted for that entire roll. I then performed the LR “sync metadata” using that new-and-working file as the base to sync to the rest of the problem DNGs in the folder/roll. I also, in the sync dialog box, turned back on a number of basic and IPTC fields that I had turned off (because I never use them or didn’t want them overwritten in sync previously) but left them all blank (so they should not write anything if no new data is in the field). After the sync was completed the broken files were all still broken as before (the two random ones I tried would still not export correctly just as before)… But weirder than that, the new and working import DNG which was the metadata source for that last sync also no longer works.
I’m losing my mind at this point and I’ve scanned just 3 broken rolls in 24 hours now of the 20 or so I need to get through this week (along with ironing out the rest of my workflow changes to accomodate NLP).
ANY HELP or thoughts you have in the near term would be hugely appreciated, Nate. At the moment I’m rather hosed. This is just bizarre.
EDIT: (a new development) I THINK I HAVE FOUND THE ISSUE (or very near to it) !!
I just attempted the previous steps with a newly imported Minolta Autocord TMax400 roll and something interesting happened…
This is long, but please bear with me.
As before, I added just basic scan info to just one of the .DNG files, e.g. camera make/model and film stock info.
I then exported just that file to .JPG leaving default NLP metadata options but tick options to add to the same location, retain custom NLP metadata, add to stack. I also turned off any resizing or compression. The export went fine (as usual with non-sync-metadata files previously); the new .JPG appended the title, changed the camera information, etc. All looked well.
I then used LR ‘sync metadata’ as before but to just two additional .DNG files in the folder. After sync I exported those, one at a time. They both worked without any problem just as the original did.
I went back to the original .DNG from which the metadata had been synced and added all additional fields; e.g. shooting notes, digitization notes, lab details, etc.
I then re-synced to the additional two .DNGs that I’d previously synced. I exported these one and a time once more and AGAIN both of them failed: camera info was still Pentax K-3, no title appending, no keywording etc etc.
OK… I went to one of the synced DNGs and REMOVED the values in the NLP digitization fields (all of them). I re-exported and it WORKED!
I went back to the original source .DNG and removed the same values from it and exported. IT WORKED.
I again synced that source DNG back to the first DNG I’d modified (in step #6) - all values should be the same, the only difference now was that it had been synced, not manually adjusted. I re-exported the synced file (not the source) and IT ALSO WORKED.
I then re-added the digitization fields info again to that synced file but OMITTED the camera equipement field (e.g. Pentax K-3 and FA50 2.8 Macro) and then exported it yet again. It did NOT work.
I then once more removed all NLP digitization field values. And exported. IT WORKED.
I did the same (step #10) on the original source DNG. IT also WORKED.
I synced the source DNG back to the other two test DNGs (now with all NLP custom metadata that I desired EXCEPT for the digitization fields). I exported them again and BOTH WORKED.
(lucky number 13) I then synced the source file to ALL DNGs in the folder (even those I’d previously not touched), picked one at random, checked the metadata and exported it to JPG. AND IT WORKED (as did 3 other spot checks including one of the previously sync’d files from the above steps.
So… there’s something going on, at least in my particular instance, with the NLP custom fields for digitization information. (shown in screenshot)
Trying to see if I can reproduce over here… just got a new MacBook that has Catalina on it, so having to upgrade / migrate my VMware Fusion software to get a Windows environment going again for testing, and running into a few hiccups…
In any case, a thought did occur to me.…
It’s possible that WHAT you are putting into the NLP Metadata fields is causing it to fail…
Some of the fields expect a very specific format, and if that format isn’t being used, it could cause something to fail… that could explain why when you first import a DNG and put limited info into it, it is exporting properly, but when add additional metadata, it is failing.
Could you share a screenshot of a filled in NLP metadata section? Here’s an example of the formats it expects:
Here is a screen of the Digitization field information added back again.
After adding it just now for this screenshot, I re-exported just this file once more and it again did NOT work, did not write any metadata to the JPG export.
(As I mentioned before, removing ONLY the scan equipment field values does not fix the problem - I have to remove it all.)
What are the specific formats expected are and for what specific fields would be incredibly helpful.
Is there a post I’ve missed somewhere? THANK YOU for popping back in.
I’ve narrowed it further to the FILM HOLDER field.
First, red boxed, example (which is the original) does not work on export.
The two following, green boxed, values (one empty, one “arbitrary test value”), both export correctly.
Do you know off-hand what it doesn’t like about the first (red) example (which has been a common denominator - or at least I think so - throughout this adventure).
Bonkers. Same number of characters, same words, just rearranged worked just fine.
I’m baffled.
I’m walking away for now but would really like to know how to avoid this in the future
… if you’ve some insight from all of this or your own testing.
ARRRRRRRRRRRRGH!
I did NOT walk away but I found the culprit!!
The ONLY difference in working and not working (which I discovered by simply re-typing the original value just as it was before in a known “broken” file) was ONE carriage return at the end of the line.
e.g. I went back to another known-broken file and arrowed to the bottom of the field and hit the backspace to remove it, and voila. Working.
Dude.
@nate
EDIT: CONFIRMED
I was able to recreate it with an extra carriage return on a previously working DNG…
And it seems to apply to other fields as well. I introduced an extra return in that same file as the last screenshot (the working one) into the Camera Model field, and it too would then not export correctly. Problem found!
Cheers. Mostly just time (I like a good frustration, sort of, if the problem is solved) lost this week. My weekend is jammed and definitley wasn’t expecting this. I cranked through a bit more scanning but I had to use a hybrid metadata method, or else leave everything half finished, or have a bunch of rolls left in the hybrid state - I like clean breaks.
Are you able to reproduce it in any of your environments, or is this just insanity in my particular setup?
