Zone System Film Calibration for the Digital Age

[ArnoG]

Hi film community,

After building my scanning rig and shooting and scanning many rolls of B&W, I am still optimizing film stock and development, and after wanting to try the Zone System, I took some months to understand how this could be made to work in our digital scanning environments.

In this exercise, I developed a procedure to obtain a full H-D (density versus log(exposure)) curve for a B&W film using one frame only. This is possible both using scanning and PS, and obviously similarly using a classic densitometer, but still obtaining the full curve with only one frame.

I believe it could be useful for those still shooting (and scanning) B&W film and therefore wanted to share my findings here, which are summarize in one of my blog pages:

Zone system calibration for film in the digital age

Cheers,

Arno

[Digitizer]

Thanks for the link, Arno.

Tried to read the page…but it’s too long and detailed for my taste.
Can you add a management summary on top of the text?
1 or 2 paragraphs, readable without scrolling

[Col]

Very interesting work, it will take me a while to digest that. It would be interesting to see some conversion comparisons with and without your linear profile approach.

[nate]

Thanks for the article, @ArnoG!

Just to clarify re: profiles, the camera profiles already included with Negative Lab Pro are already linear and make additional corrections beyond the approach mentioned the article (i.e. using the DNG Profile Editor and changing the tone profile to “linear”).

Even if you set the tone profile to “linear” in the DNG Profile Editor, there are still a number of things you would need to do to get the optimal negative conversion profiles… for instance: 1) correcting for “highlight” rolloff (to match the actual highlight data, as was done prior to PV2012, 2) removing any “hue twists” from the profile (this will matter more for color negatives than black and white), 3) Removing any black point compensation (which will impact toning in the highlights once the negative is inverted), 4) adding protection for gamut to make sure the inverted colors stays in the right color space…

All that is already being done in the Negative Lab Pro camera profiles, which you are free to use even if you are inverting manually!

-Nate

[Mark_Segal]

I would also suggest that histograms and tone curves are the digital replacement for the Zone System.

[ArnoG]

Hi all,

Thanks for the feedback and suggestions.

The reason I invested quite some time to write it down is since I believe it’s an entirely new approach to obtain full H-D curves from one shot using a suitable adaption of the zone system for our current digital environment. It’s a very powerful method to obtain in one frame the film’s tonal response, especially for quickly comparing film A with film B. I’m even contemplating whether I should submit it to some scientific photography journal, but that would take another blob of time in which I’m not exactly swimming, so having it available online should suffice for now.

@Digitizer : I did add a short version on top, that is very short in text but the tables and graph make it seem longer. I couldn’t really compress it further since I need to sketch the issues shortly and then the solution, but it should be very accessible now, with a longer version below for those who want to understand the details.

@nate : Yes, I am aware. We had this discussion on a custom linear profile before where you mentioned there were more issues than just linearity of the profile, which is indeed entirely correct. What frustrated me somewhat is that Adobe seems to make changes from process version to process version that don’t seem to be documented, and also that in my case I couldn’t exactly reproduce what others over at Rawdigger had found to obtain tonal accuracy using a zeroed out PV2. This still puzzles me. I suppose that you are keeping track of what Adobe is doing in order to maintain your linear-overall NLP profile so I would be perfectly happy relying on that profile. As far as I could tell, my custom version and your NLP version seem both to retain linearity but I might be missing more hidden Adobe things that you are obviously aware off. In the short version I did add the suggestion to use the NLP profile, so folks can use that instead.

@Col : I did include a comparison to Adobe’s Monochrome profile, which is non-linear and adds visibly an s-curve to the conversion. This kills tonal accuracy, but is likely done to provide punch to a B&W converted digital color photo by adding more contrast.

[rh_photo]

This is what happens when you allow ChatGPT to play unsupervised.

[VladS]

This is brilliant! From my point of view this methodology can be applied to the camera scanning backing light. Just few days ago I took the reading of one popular lightsource and found out that the center is at least 1 stop brighter than corners if one uses RawDigger to read the raw file. So I started tinkering with numbers to calculate what mask at with density as function of x and y should be printed to make source uniform to better degree. I actually have means to print neutral/colorless grey mask with density field given as image. Basically I am talking about special- film based version of flat field correction. Let me read more and dig into the math. Great work!

[praeburn]

I am a little confused by the insistence on a linear stops to density relationship in the digital values. Print paper has a shoulder rolloff that helps encapsulate higher exposures above the straight-line portion of the curve. This is evident in in negative and slide films and deliberately engineered into the chemistry to allow for pleasing contrast in the middle values and rolling off on the top.
sRGB at gamma 2.2 fails to do this and looks rubbish if this is not compensated for, usually in the colour science when encoding to a device relative colour space for viewing. So in the case of film emulation at this point.
VFX uses a linear scene referred colour space that is as linear as possible as an intermediate working space. Nominal exposure (zone V) is set at a float value of 0.18, zone IV being .09 and zone VI at .36 and so on. So all operations are performed in colourspace as close to the scene phyics as possible. This retails all possible super brights as the work is done in float math, not 0-255 interger- math that clips on the top end. Values above one are rolled off in the same way film does when viewing using a curve that is derived from a film response. Paper does the same thing.
Stepping up at a .5 stop step above zone V may fit the dynamic range you want into the 8 bit range, but worry this is not giving you the pleasing contrast in the straight line portion of the curve.
Illford and Kodak both publish the response curves of the print papers that ould be a good guide on what the rolloff does on painting paper. perhaps that is a good model of a response to aim for to achieve what Adams modelled his system on.

[ArnoG]

The core goal of this exercise is to be able to extract an entire H-D curve from a single frame on B&W film. The question then becomes what the optimal luminosities of the patches are, and, contrary to what most others do, I chose to retain connection with camera stops. The only item defined therefore is L*, which follows from light intensity. I do not at all limit myself to 8 bit RGB: These values are only calculated as a bonus since they are, for most, more easily accessible in software. By choosing changes of 1 stop below Zone V and 0.5 stop changes above Zone V, I artificially limit myself to 7.5 EV. In practice this means that I will not drive films into full saturation, so the "shoulder’ is not reached. At the moment, I can derive the “foot”, see how linear the tones are, and determine gamma from the linear section of the curve, but the “high end” of the curve is not accessed this way. Typical results look like so:

Deriving such comparisons of film tonal behavior from a single frame was what I set out to accomplish, and this is working just fine now.

You make a good point though, in that there is no reason to not make larger steps above Zone V and extend the accessibility range to also include the shoulder (i.e, extend the range to 10 EV total by defining Zone X at +5 EV above Zone V). This is just simple math for the luminosities of the patches. I will think about a useful way to do this, but it should not be so complicated. I do insist on retaining connection to camera stops, since this also enables easy connection to a camera and a light meter. Full or half stops are easier to access and handle on cameras and light meters than some values in between.

I have no idea what you mean by VFX, so it would be useful if you could explain that some more.

Cheers,

Arno

Update: Thinking this through a little I conclude that extending the exposure range to 10 EV is not possible without loosing connection to the way light meters are calibrated: Zone V currently has 50% grey, while Zone X has 100% grey. The only way to increase exposure for Zone X is therefore to change the definition for Zone V, where the exposure is connected to the way light meters are calibrated at 50% grey, or 18% reflection. Hence, if Zone X needs more exposure, the connection to how light meters are calibrated for Zone V is lost.

[praeburn]

If it’s working fine for you now, that’s great. As long as your getting what you need from the system then its right.

I your in Visual Effects for film (VFX) so I am looking at it from that perspective. We typically don’t use 8 or 16 bit ‘display referred’ colour space, meaning a value system pre-encoded for the display device, be it a screen or printer. sRGB gamma curves are designed to bias as much of the available numeric values (of 0-255) to the straight line portion of the curve.
We typically take whatever luminance curve the camera wrote (log encoding, raw camera data, and sometimes sRGB encoded) and un-wrap that luminance curve to a linear to light ‘working’ colour space. We then do ‘movie post production stuff’ to the image, and when we are finished it gets encoded to the display device (this is much simplified for clarity)

My point is the space we try and do the work in is ‘linear to light’. .18 is zone v, and all stops above and below are double or halved (pixel multiplied by pow(stops, 2)).

If you just look at these linear images with a gamma 2.2 applied they look terrible, like a cheap video camera from the 80 that did almost no colour science. Part of displaying it is a film emulation look up table, because we want to have it look like the cinema film s-curve that cinema print film used to do. It nicely encapulated the contrast control of the straight line portion of the curve, while crushing the blacks a little and rolling off the highlights for detail.

In my view you are skipping the linear to light colour space step and aiming straight for what we call ‘output referred colour space’. You are burning in the gamma response of the output medium (a screen) so any photo paper emulation to replicate the zone printing system would have to include the paper response, which has a strong rolloff. Paper doesn’t ‘clip’.

When you go to a digital print, there is a bunch of behind the scenes colour science in the print driver to go from a screen display referred colour space to the colour space of the printer / paper combination to look similar to the screen image.

Photographic paper does the same thing, has a nice roll off in the top while providing contrast control in the straight line portion. Ilford and Kodak both provide excellent data sheets showing this rolloff.
I worry you are letting the clip point of you highlights dictate the contrast of everything above nominal exposure (zone V). My instinct would be to retain a linear relationship between all stops 3 to 7 with 5 pinned at 0.18, adjusting the rolloff on the highlights and shadows to retain detail at those steps.

[ArnoG]

@praeburn

In my view you are skipping the linear to light colour space step and aiming straight for what we call ‘output referred colour space’. You are burning in the gamma response of the output medium (a screen)…

No, not at all. I have no idea what you are talking about. The method does not rely on color spaces, gamma’s in RGB, or output device non-linearities:

1. I start from defining light intensities with 1 stop changes from zone 0 to V, and 0.5 stop changes from zone V to X. This translates to light intensities dropping by 50% going down from zone V and by 70% going down from zone X (i.e., “linear to light” as you call this). I make 0.5 stops above zone V to force zone V to be at 50% grey or 18% reflection, since that is how most light meters are defined (some modern light meters are claimed to be defined at 12.5% reflection, which translates to 0.5 stop difference at a density of 0.9 as opposed to 0.75 for zone V).

2. I then use the definitions of Cielab (CIELAB color space - Wikipedia) to calculate the corresponding values of L* and use that to create the patches in PS.

3. These patches are then projected onto a calibrated display and the film is exposed by taking a shot of those projected patches.

4. To confirm that the patches as they are projected on the calibrated display indeed still correspond to 1 stop intervals below zone V and to 0.5 stop intervals above zone V, I used two methods:

4a. I used a light meter to measure the patches directly off the display, and their brightness indeed changes by 1 EV below zone V and by 0.5 EV above zone V.

4b. I used a digital camera to take a shot of the patches and confirmed the intervals to be 1 stop below zone V and 0.5 stop above zone V using Rawdigger (i.e., without raw conversion). I also converted this raw file using Adobe Camera Raw with linear settings, and confirmed that the patches as shot with the digital camera yield the same L* in PS as originally produced in PS (Fig. 5 in the long version of the writeup on my blog).

Hence, I confirmed that the brightnesses of the patches as projected on my display give indeed 1 EV intervals below zone V and 0.5 EV intervals above zone V in two ways: Hence, when I shoot the patches as projected on the display onto a frame on film, the film is exposed to linear steps in light.

5. After development I measure the densities of the patches with a densitometer in the usual fashion, and plot these as a function of LOG10 of the as-originally-defined light intensities.

6. To make objective film-to-film comparisons, I normalize the densities to the densities of zone 0 for each film, so as to be independent of film-base and fog, which differs between films, i.e., I compare the relative tonal responses with respect to the black point that would normally be set after scanning and inversion of photos.

As you can see from the above, I do not rely at all on a “color space” or an “RGB gamma”, and am not “burning in the gamma response of the output medium” as you suggest I do. I’m sorry, but I can’t explain it clearer than as written above, so I apologize in case you still do not fully comprehend it, and rest my case.

Arno

[praeburn]

Don’t get me wrong, I think your methodology and intent is fantastic. The fact that you are getting what you want out of your system is proof enough. I am really disagreeing on some terminology and discussing how I might approach it differently.

Let be clear about some terms. A colourspace is a definition of how a numeric value relates to either a brightness value in the scene (scene referred) or a display output colour (display referred). More often than not this is a combination of a luminance lookup and a gamut conversion. In your case you are only concerned with the luminance mapping. You are making your own colour space definition (exposure stop steps below zoneV map to certain values and contrast, and above zoneV a different contrast) and the values are referred to a display device.

You are “burning in the gamma response of the output medium” by directly relating code values to output luminaries without any ‘colour-science’ being used to covert from working space to display space. Or at least you haven’t described using any colour management systems.
If you didn’t have the game response of the monitor burnt into the image, the top ‘zone’ would account for half the possible code values (a stop) and it would look very dark and crushed. Generally the raw processing does this, as I’m sure you know.

Because your data values look ‘normal’ on a computer screen without a LUT means that they are in a colour space that is encoded for the display device already. You did that through your process, correlating numeric code values and display illumination.

From your process description, you are validating the screen display colours. I have not used ‘raw digger’ before. Reading the original 12/14 bit raw code values and mapping them through camera response would make an excellent ‘million’ spot meter.

Also, for clarity, a linear to light scene referred colourspace would be, well, linear, not changing arbitrarily changing contrast at a point. It generally is a float value (which Photoshop incorrectly labels 32bit) or High Dynamic Range. 0 means no light, but 1 has no meaning, as light can just keep increasing. We are not talking about reflectivity, but all possible scene light.

How I would differ in approach is to start with zoneV at .18 be linear around this. I understand you are using .5 stop steps for it to contain the full ‘range’, but it’s not really, it’s a 7 stop range with smaller steps. I would tend to replicate top end rolloff of photographic paper to account for the dynamic range of 10 stops. Switching contrast at zone 5 seems like an odd choice to me, but if it works for you, great. I would not assume even steps of reflectivity at the top few zones, because that is not the response that paper does, or as film does.

Your process is admirable for round trip validation, and I am surprised you’re getting a full 7 stops out of a calibrated monitors output. I can’t say I have ever tried it.

On a slightly different note, in your full write up you talked a little about box speed, and not feeling experience enough to mess with it. You have all the pieces in your testing. ANSI/ISO speed is defined as .1 log units about film base + fog to determine when the film crystals start to break. Most film tech data sheets note this in the fine print, even when the box says 3200 or whatever. You noted that forma400 doesn’t start to have detail till zone2. That suggests true speed is closer to 200. Film manufacturers are liars…

The lower contrast on the rest of the curve suggest if you pushed the film you could get closer to the apx densities. It doesn’t make the film faster, just more contrasty. Thus ‘expose for the shadows, develop for the highlights’. (and therefore contrast)

It sounds like you have the gear setup to to asses and calibrate your process for black and white darkroom assessment.

[ArnoG]

@praeburn

You are making your own colour space definition (exposure stop steps below zoneV map to certain values and contrast, and above zoneV a different contrast) and the values are referred to a display device.

@praeburn

Also, for clarity, a linear to light scene referred colourspace would be, well, linear, not changing arbitrarily changing contrast at a point.

I am not “changing contrast”. I am just make smaller steps in (linear) light intensity above zone V. This has nothing to do with contrast. It only effects the overall maximum achievable dynamic range, i.e., 7.5 EV vs. 10 EV (which could be, incorrectly, confused with contrast).

@praeburn

You are “burning in the gamma response of the output medium” by directly relating code values to output luminaries without any ‘colour-science’ being used to covert from working space to display space. Or at least you haven’t described using any colour management systems.

Of course I am obviously using a proper color managed system (and a calibrated display) or I would not obtain the correct brightness values on my display as confirmed by two independent methods. Yes, 7.5 EV is certainly achievable on a (good) display. Just look at the specs for dynamic range of monitors: 1:1000 is 10 EV. This is indeed far from achievable on paper. Hence my conclusion that a printed copy will not work. What could affect the measurement is the fact that a modern display is made from discreet R, G, and B LED’s with small bandwidth wavelengths, contrasting continuous daylight and possibly being affected by the spectral response of B&W film. So far, however, I do not see this being an issue.

[praeburn]

In ‘linear to light’ values, a stop increase would be a doubling of the code value. So
zone 0 is .005
zone 1 is .012
zone 2 is .0225
zone 3 is .045
zone 4 is .09
zone 5 is .18
zone 6 is .36
zone 7 is .72
zone 8 is 1.44
zone 9 is 2.88
zone 10 is 5.76

The relationship of doubling or halving the light by going up or down in EV always stays consistent. This relationship is locked to the scene physics and the amount of light hitting the sensor. It would be closest to the values you are seeing in rawdigger as 10 or 12 bit values before it gets gamma encoded.

You are choosing an arbitrary, but useful to you and therefor valid, relationship between the zone and a number that meaning to the output device. I say arbitrary because you have chosen to change the stepping relationship above zone5. These are not linear, because the slope changes (1 stop per step to .5 stops per step).
Changing the relationship, changes contrast. If you stretch 2 stops to fit between you brightest and darkest display value, that would be high contrast, 20 stops range would be low contrast, no?

1:1000 is 10 EV is a bit of an over simplification. Sure, the ‘ratio’ of zone 0 to zone 9 is 1:512, but percevable steps per code value are probably more relevant. Most calibration standards ask for between 80 and 120 nits peak brightness, opinions vary greatly, but the 6 to 8 stops I have heard from different industries as an agreed dynamic range for SDR displays does seem to agree with your 7.5 stop limit.

I apologise for insinuating that you are not using a ‘proper’ colour management system. I was saying - to your credit - you have developed something outside of ‘normal’ systems. Adobe, ICC, rec709, DCI P3, REC2020 provide a bunch of systems to do things like this. That what I meant by colour management systems.

I agree that narrow bandwidth would be an issue, but most ISP monitor are illuminated by a wide spectrum phosphors excited by leds (white leds) ranter than discreetly colour LEDs themselves, but the spectra of these are variable between brands. You may see more spectral spiking on a OLED, and discreet LED displays are still a little way away from desktop use. I also would be surprised if they had much effect on black and white images, but that said, the wider the cooler gamuts they are aiming for, metamerisms may be the cost, but I suspect in the more saturated colours rather than the monotones.

[ArnoG]

The original zone system is like so:

Where a doubling of light intensity (i.e., 1 stop) occurs between the zones. I changed this to:

since that corresponds better to light meter readings at zone V and other common definitions of zone V. It seems to me that you interpret a zone number as a variable to which something should be plotted, but it is simply a definition of a density range on the negative, and we can choose that definition as we see fit for our purpose. The “slope change” that you refer to likely stems from plotting brightnesses as a function of equidistant zone numbers, like so:

but there is no reason to make such a plot, since a zone number is not a variable, just a definition of a certain density (range). Note, btw, that the vertical axis is logarithmic to achieve “linearity in light” this way: A doubling or halving of the light intensity is not at all linear, but behaves like constant x 2^(zone number).

Nonetheless, the “slope change” in the above plot is due to plotting LOG(light) as a function of equidistant zone numbers, which works for how the zones are defined originally for a 10 EV range. Such a plot is wrong in the way I define the zones though: In my definition, the zones are defined at different light intensities/densities, and, hence, the zones on the LOG(exposure) axis are not equidistant but corresponding to their definition at half the distances on the LOG(exposure) axis above zone V, like so:

Hence, there is no “slope change” or “contrast change”: The H-D curve will still be linear for an ideal density response when the zones are shown at the LOG(exposure) value according to their upfront definition. A “slope change” only occurs if a plot is made as a function of the original definition of the zones with equidistant spacing on the LOG(exposure) axis, but such a plot is incorrect with the new definition. I could chop up the LOG(exposure) axis into an arbitrary number of sections and call these my zones, but I would have to label the zone numbers at the correct position on the LOG(exposure) axis before I can make a density vs. LOG(exposure) graph.

And BTW: A dynamic range of 1:1000 is exactly 10 EV, or a density range of 0 to 3, which follows from their definitions, and, hence, this is not an “oversimplification”…

I will now stop responding to possible further remarks along similar lines, since it is costing me too much time to repeatedly explain the same thing. For further information readers can look at my blog post that is linked to at the start of this thread.

[praeburn]

Yes, we seem to be going nowhere.

While a 1:1000 dynamic range is literally a 10 step range. Once you have defined a nominal exposure, or zone 5 or mid grey at a code value of 119 (or 116 if you prefer) then you have to contort the model to fit 10 stops, as you have done.

Like I keep saying, its an interpretation that works for you, then that’s exactly how right it needs to be.