Assignment, Week 13: Processing dynamic range, white

Theme: No theme this week

One of the biggest problems with cameras is their lack of dynamic range capability - while we are able to perceive a large dynamic range with our own eyes, our cameras are simply incapable of capturing the same range of brightness. The resulting out-of-camera images, however, are not the upper limit of a camera's dynamic range capability - employing a number of post-processing techniques, details can be recovered from the shadow and highlight regions of the image, effectively expanding the visible dynamic range in the images.

As an example, we'll take this grossly underexposed image:

This is an image that is mostly dominated by the "crushed shadows" mentioned last week - the dark pixels are so dark that they are visually indistinguishable and thus there is no semblance of detail at all - we can't make out any distinct features whatsoever in the bottom portion of the image, for example. However, even though the pixels are apparently indistinguishable, in reality they are very different and retain an incredible amount of detail. This is the same image with exposure compensation applied to it:

Even though the pixels all looked the same (indistinguishably dark) in the original image, brightening the image (in effect multiplying the differences between each pixel) reveals that the pixels are in fact distinguishable from one another, and possess a large amount of detail - we can see that the bottom half of the image retained detail that differentiates the subjects legs from the floor, and the floor from shards of glass on the right side, even though these differences were not visible in the original image. In fact, as long as the pixels are not clipped to pure black (0,0,0 for RGB values), detail still exists and can therefore be recovered.

The same is true for the highlight region. Take for example this image - portions like the sky at the top or the subject's shirt exhibit "blown highlights" - areas that are so bright that no detail can be distinguished:

However, applying an exposure adjustment downward, we get this image, which reveals that in fact plenty of detail still exists in the sky and and the subject's shirt.

Now that we've established that detail exists in the shadow and highlight regions of images, how do we go about actually restoring this detail? The first method is simple exposure adjustment - if the image is too dark or bright, simply brighten or darken the entire image, respectively. There are several tools for exposure adjustment, the most common ones being:

"Brightness" slider: A simplistic one-dimensional tool for exposure adjustment. Should not be used, as it gives little control.
Levels: A rudimentary dynamic range adjustment tool. Should not be used, as Curves already does everything Levels can, and more.
Curves: A versatile exposure adjustment too, with some selective adjustment ability. Probably the most powerful and practical tool for exposure adjustment.
Exposure adjustment: A one-dimensional exposure adjustment tool. Available in RAW converters. Doesn't give much control, but tends to mimic real-life exposure compensation very well (+1 EV exposure adjustment looks like an image that was taken at +1EV in real life), so useful for correcting incorrect exposure.

Exposure adjustment tools are useful for correcting incorrect exposures - images that are simply too bright or dark, like the two examples above. The weakness of overall exposure adjustment is that they act over the entire image. Thus, they don't work well for images with a large dynamic range. If I have a bright sky and a dark land in an image, for example, I couldn't use exposure adjustment to brighten the shadows without also brightening the highlights, which would destroy the detail in the highlights, and the same is true for trying to darken the highlight without darkening the ground. Curves has some control over what area of the image is affected, but to get the most out of shadow and highlight detail, true selective adjustment tools, like Photoshop's Shadows and Highlights or HDR blending, are needed. However, if you'd like to use curves for this assignment, see this writeup for an introduction on using the Curves tool.

As an example, we'll take this large dynamic range image - there are both very bright elements and very dark elements in the image.

Under normal exposure adjustment, we really wouldn't have much leeway in bringing out details in the shadow region without destroying all the detail in the highlight region, since overall exposure adjustments work on the entire image. By using a tool like Photoshop's Shadows and Highlights (Image->Adjustments->Shadows and Highlights, in Photoshop CS,CS2,CS3), however, we can selectively brighten only the dark pixels, leaving the bright pixels alone and preserving the detail in the highlight region.

And this is the result (mouseover to see the change):

Notice that the shadow region is brightened, but the area in the highlights remains unchanged. This is what makes the Shadows and Highlights tool so powerful - you can make strong adjustments to lift detail out of the shadow or highlight regions, without affecting the brightness/detail in other region of the image.

The Shadows and Highlights tool isn't perfect, however. Notice that, although we were able to extract a lot of detail from the image, it does look slightly dull and unnatural. The main subject in the shadows-adjusted image doesn't look anywhere as good as the subject in an image adjusted with regular exposure adjustment:

Of course, what we lose with overall exposure adjustment is all the detail in the highlights (the windows are now completely white). So images look better processed with overall exposure adjustment, but they destroy detail on one end or the other. Shadows and Highlights can make adjustments without destroying detail, but the images can only be pushed so far before they start looking really dull and desaturated. Is there any way to get the best of both worlds? There is, and it's called High Dynamic Range (HDR) blending.

With HDR blending, we process multiple photos where we only concern ourselves with a certain brightness range in the image. For our example image, I'll process two images - one is the original with full detail in the highlight area without regard to how the shadow area looks, and the other is an exposure-brightened version with full detail in the shadow area, without regard to how the highlight area looks:

Original, correct exposure for highlight area

Edited, correct exposure for shadow area

We combine the two images in a process called HDR blending. A manual, old-fashioned way to do HDR blending is to simply stack one image on top of the other, and use a layer mask to show the highlight areas from the image correctly exposed for the highlights, and show the shadow areas from the image correctly exposed for the shadows. You can download this .psd Photoshop file (96MB!) showing how this is done. However, the manual method is tedious and requires a lot of fine-tuning, and today there are quite a few automated HDR blending program. In any case, here is the resulting image, which you can see, retains the most detail from the highlights and the shadows, and doesn't look nearly as dull as the version processed with Shadows and Highlights.

HDR blended image

There are other ways to restore dynamic range as well. One tip is to shoot in RAW mode - RAW stores a lot more tonal information than a simple JPEG file does, and therefore there is more shadow and especially highlight headroom where detail can be recovered. Using your camera's flash in daylight situations, in a technique called "fill flash", also serves to lessen dynamic range in the scene, thus allowing your camera to capture the detail from the entire range of brightness. See the slides on the Week 13 powerpoint file for examples of this.

This week's assignment is to find ways to render as much of the dynamic range as possible from an image of a large dynamic range scene. Submit an original shot, and then show the "after" shot of the results of trying to expand the dynamic range captured, either by post-processing, shooting in RAW, or using fill flash.

Suggested programs:

GNU Image Manipulation Program (GIMP): Verstatile, free, open source image editing application. Not as powerful as Photoshop, and not as intuitive/easy to learn, but far more capable than most other free image editors
Windows download: http://gimp-win.sourceforge.net/stable.html
Mac download: http://darwingimp.sourceforge.net/download.html

Access curves by Colors->Curves

Adobe Photoshop
Access curves by Image->Adjustments->Curves
Access Shadows and Highlights by Image->Adjustments->Shadows and Highlights
Access HDR blending by opening files, File->Automate->Merge to HDR

Photomatix: Easy to use HDR blending program.
Windows/Mac download: http://www.hdrsoft.com/download.html

White Balance

An important but generally unknown characteristic of images is white balance - what exactly is a "neutral" color (greys, whites, blacks, etc), from which all the colors can be based upon. As we discussed before, the camera's sensor gets information about how much red, green, and blue there is in the picture, and from that, the camera derives the color. Normally, we would expect equal amounts of red, green, and blue to equal white light. However, what is actually "white" and what we perceive as "white" are two very differnet things. Take a typical indoor scene, for example:

This is what we see with our own eyes. However, the fluorescent lights in the room are actually emitting light that tend very heavily towards the red end of the spectrum - there is almost no blue spectrum light coming out of the fluorescent lamps at all. In actuality, the lighting in the room really looks like this:

Wow! This is quite a big difference, and nothing like what we see in real life! Yet if you actually measured the light in the room, this is what it would look like - lots and lots of red, and nearly zero blue. The reason we see the image above is because our brains interpret the signals from our eyes and performs a sort of "white balance". We actually do perceive the "true" color image on the bottom, but upon receiving this information, our brains go: "Woah! This color is all wrong! It should look like this..." and reinterprets the lighting so we get the much more natural-looking scene above that we're accustomed to.

Cameras however, don't know anything about what colors "should be". They simply receive and interpret the light for what it actually is, which results in images like the 2nd one. As our minds re-interpret the light information to give us a more natural-looking color balance, cameras must do the same "white balancing" to produce colors that are in sync with the human interpretation of light - essentially, turning the second image into the first image.

Cameras do this via the "white balance", where either a neutral point (a white/grey/black point that should have no color saturation) is defined, or the camera is given a preset interpretation ("interpret this light as sunny/cloudy/fluorescent"). This is one of the most critical aspects of creating an image, as interpreting color the wrong way can have catastrophic results:

However, if you happened to not set your white balance correctly to begin with, there are ways to correct the white balance afterwards. The most rudimentary tool for this is Color Balance. In Photoshop you can find this under Image->Adjustment->Color Balance. In GIMP this is found under Colors->Color Balance.

This is the example original image, which as you can see has a very strong warm tint, due to the nature of the fluorescent lights in the room:

Even though the color is extremely off, we can correct it by applying a very strong color balancing - we want to offset the yellowish and reddish tones, and shift them towards more cyan and blue tones. This is the result we get:

Assignment Summary:

You have four choices for this week's assignment:
- Take one of your photos that had a very high contrast (very bright and dark things in the same scene, where you couldn't see detail in both), and try to expand the shadow and highlight range using the Shadows/Highlights method described above. Only students who have access to Photoshop will be able to do this, although you can try to download the trial version of Photoshop from Adobe's website.
- Take pictures of the same scene, trying out the various white balance preset modes (daylight, cloudy, tungsten, etc) to see what presets do, and in what situations they actually improve pictures.
- Spend the entire week taking photos in the "daylight" white balance preset. For users with DSLRs, you can do this or set the color temperature to 5000K. Daylight or 5000K will interpret red, green, and blue spectrum light about equally - equal green, blue, and red parts combine to make white. Using this as a reference point, take photos and observe what the "true" color balance of different scenes and different lighting is.
- Take one of your photos with an incorrect white balance, and attempt to correct it using white balance. Be sure to record what color balance settings you used.

Resources:

For those interested, Cambridge in Colour has a fantastic article on color balance