Computational Photography

Giulio Alessandrini, WRI

Join the Conversation #WolframTechConf

Outline

Adjustments

Levels and curves

Color balance

Sharpening

Processing

Color conversion

Tone mapping

Combinations

Exposure and focus stacks

Flash+noFlash

Image layers

Adjustments: Levels

The histogram shows that the colors of this image does not span the full range:

img=Import[FileNameJoin[{NotebookDirectory[],"images","gull.tiff"}]]

ImageHistogram[img]

Stretch the channels automatically:

ImageHistogram[ImageAdjust[img]]

Stretch the channels using a custom interface:

DynamicModule[
{min=0,mid=1/2,max=1,int={0,1},
compareQ=False,
clippingQ=False,
image,n,
adjusted,res
},
With[{d=.05,framedims=ImageDimensions[img]},
Dynamic[
(*image preparation*)
image=img;
n=Times@@ImageDimensions[image];
Dynamic[
(*Adjust contrast, brightness and gamma*)
adjusted=ImageAdjust[image,{0,0,((Log[2])/(Log[(1/mid)]))}];
Dynamic[
(*final rescale (not to be shown in the histogram)*)
res=Image[ImageAdjust[adjusted,{0,0,1},{min,max},{0,1}]];

(*Visualization*)
Deploy@Panel[
Row[{
Grid[{
{
EventHandler[Button["View original",Dynamic[compareQ],ImageSize->First[framedims]/2],{"MouseUp":>(compareQ=False),"MouseDown":>(compareQ=True)}],
EventHandler[Button["Show clipping",ImageSize->First[framedims]/2],{"MouseDown":>(clippingQ=True),"MouseUp":>(clippingQ=False)}]
},
{
Dynamic[
Panel[Which[
clippingQ,
ColorCombine[Binarize[#,#==0||#==1&]&/@ColorSeparate[res,"RGB"],"RGB"],
compareQ,
image,
True,
res],
ImageSize->framedims,Alignment->Center,Background->White,FrameMargins->None],
TrackedSymbols:>{clippingQ,compareQ,res}
],
SpanFromLeft
}
},Alignment->Center],
"",
Column[{
Dynamic@ImageHistogram[adjusted,256,{-1/100,1+1/100},
ImageSize->300,
PlotRange->{All,{0,All}},
Appearance-> "Transparent",
Epilog->{Dashed,Black,
Line[{{min,0},{min,n}}],
Line[{{max,0},{max,n}}],
Dotted,GrayLevel[.3],
Line[{{mid,0},{mid,n}}]
},
Prolog->{GrayLevel[.95],Rectangle[{-d,0},{1+d,n}]},
Background->GrayLevel[.9]
],
IntervalSlider[Dynamic[{min,max}],{0,1,1/100},ImageSize->{300,20},Method->"Stop",MinIntervalSize->1/100],
Slider[Dynamic[mid],{d,1-d,1/100},ImageSize->{260,20}]
},
Alignment->Center
]
},Alignment->Center]
]]]]
]
]

(*img=Import["https://pixabay.com/static/uploads/photo/2013/10/08/20/57/gull-192909_640.jpg",ImageSize->{400}]*)

Adjustments: Curves

Gamma correction

Manipulate[
Deploy@Plot[x^Exp[γ],{x,0,1},ImageSize->ImageDimensions[img],Axes->False,PlotRange->All,PlotStyle->{RGBColor[1,0,0,.7],Thick},Prolog->Inset[ImageAdjust[img,{0,0,Exp[γ]}],Automatic,Automatic,{1,Automatic}],AspectRatio->Divide@@Reverse[ImageDimensions[img]],
PlotRangePadding->None
],{{γ,0},-2,2}]

Adjustments: Curves

Generic adjustment

With[{image = img}, 
DynamicModule[
	{pts = ConstantArray[{{0, 0}, {1, 1}}, 4], p, funcs, ch = 1, scheme, schemes, ord = 2}, 
	schemes = {
		{{0, 0}, {1, 1}} -> "linear", 
		{{0, .5}, {1, .5}} -> "flat", 
		{{0, 1}, {1, 0}} -> "negative", 
		{{0, 0}, {0.2, 0.1}, {0.5, 0.5}, {0.8, 0.9}, {1, 1}} -> "high contrast", 
		{{0, 0}, {0.2, 0.3}, {0.5, 0.5}, {0.8, 0.7}, {1, 1}} -> "low contrast"
	};
	With[
		{histograms = MapThread[
			ListPlot[#/Max[#], Frame -> True, FrameTicks -> None, 
				Joined -> True, PlotStyle -> #2, Filling -> Axis, DataRange -> {0, 1}, 
				PlotRange -> {{0, 1}, {0, 1}}, AspectRatio -> 1, ImageMargins -> 8, 
				ImageSize -> {280, 280}, Background -> White]&, 
			{Prepend[ImageLevels[lena][[All, All, 2]], ImageLevels[ColorSeparate[lena, "L"]][[All, 2]]], 
			{Gray, Red, Green, Blue}}]},

		Deploy@Panel[
		Row[{
			Dynamic[
			funcs = Block[{x}, Function[{x}, #]&/@Table[Min[Max[Quiet@Interpolation[Union@points, Method -> "Spline", InterpolationOrder -> ord][x], 0], 1], {points, pts}]];
			Image[
				ColorCombine[MapThread[
					ImageApply[First[funcs]@*#1, #2]&, 
					{Rest[funcs], ColorSeparate[image, "RGB"]}]
				], 
				ImageSize -> All]
			], 
			Spacer[20], 
			Dynamic@Column[{
			Row[{SetterBar[Dynamic[ch], Thread[Range[4] -> {"All", "R", "G", "B"}]], Spacer[30], PopupMenu[Dynamic[p], schemes], Spacer[30], PopupMenu[Dynamic[ord], Range[0, 4]]}], 
			p = pts[[ch]];
			LocatorPane[
			Dynamic[p], 
			Dynamic@Show[
			pts[[ch]] = Union[p, SameTest -> (#1[[1]] == #2[[1]]&)];
			histograms[[ch]], 
			Plot[Clip[Quiet@Interpolation[p, Method -> "Spline", InterpolationOrder -> ord][x], {0, 1}], {x, 0, 1}, PlotStyle -> Black, PlotRange -> All]
			], 
			{{0, 0}, {1, 1}}, 
			LocatorAutoCreate -> True]
			}]
			}
		]]
	]
	]
]

Adjustments: White Balance

A set of images with light or strong color cast:

sample=Import[FileNameJoin[{NotebookDirectory[],"images","sample_color_cast.tiff"}]];sample//Multicolumn[#,5]&

Adjustments: White Balance

Try different balancing procedures on different images:

Manipulate[
Row[Image[#,ImageSize->{300}]&/@{i,ColorBalance[i,Method->method]},Spacer[3]],
{{i,sample[[1]],"image"},#->Thumbnail[#]&/@sample},
{method,{"GrayScaling","RGBScaling","LMSScaling","ChromaticityScaling"}}
]

Adjustments: Unsharp Masking

You can enhance the detail level of an image by subtracting the Laplacian filtered version from the original:

img=Image[Import[FileNameJoin[{NotebookDirectory[],"images","gull.tiff"}]],ImageSize->All];

{img,ImageSubtract[img,LaplacianGaussianFilter[img,1]]}

Achieve a smoother, less noisy, sharpening by operating only on the lightness and using an unsharp mask:

{L,a,b}=ColorSeparate[img,"LAB"];
(unsharpmask=ImageClip@ImageMultiply[ImageDifference[L,Blur[L,1]],20])

highcontrastL=ImageSubtract[L,LaplacianGaussianFilter[L,1]];
newL=
ImageAdd[
ImageMultiply[highcontrastL,unsharpmask],
ImageMultiply[L,ColorNegate[unsharpmask]]
];
{img,Image[ColorCombine[{newL,a,b},"LAB"],ImageSize->All]}

Processing: Color conversion

img=Import[FileNameJoin[{NotebookDirectory[],"images","parrot.tiff"}]]

Convert an image to grayscale by picking a single channel:

TabView[Thread[{"R","G","B"}->ColorSeparate[img]]]

Convert an image to grayscale by mixing the channel:

ColorConvert[img,"Grayscale"]

Use arbitrary color mixing coefficients:

Manipulate[
With[{coeff=Normalize[List@@ColorConvert[color,"RGB"],Total]},Image[ImageApply[coeff.#&,img],ImageSize->All]
],
{color,RGBColor[.3,.59,.11],ImageSize->400}
]

(*img=Import["https://pixabay.com/static/uploads/photo/2015/03/23/13/57/rainbow-lorikeet-686100_640.jpg",ImageSize->{500}]*)

Processing: Tone Mapping

memorial=ImageResize[Import[FileNameJoin[{NotebookDirectory[],"images","memorial.exr"}]],{600},Resampling->"Linear"];

Manipulate[ImageMultiply[Image[memorial,ImageSize->All],Exp[exposure]],{{exposure,0},-4,4}]

ColorToneMapping[memorial]

ColorToneMapping[memorial,.6]

ColorToneMapping[memorial,Method->#]&/@{"RGB","Luminance","LogLuminance","DodgingAndBurning","BaseLuminance"}

(*memorial=ImageResize[Import["http://www.pauldebevec.com/Research/HDR/memorial.exr"],{500},Resampling->"Linear"];*)

Image Combinations: Exposure Stacking

Dynamic range blending

exposures=Import[FileNameJoin[{NotebookDirectory[],"images","stlouisarch.tiff"}]]

Blend[exposures]

ImageExposureCombine[exposures]

(*exposures = Import[#,ImageSize->{480}]&/@{
"http://upload.wikimedia.org/wikipedia/commons/thumb/0/09/StLouisArchMultExpEV-4.72.JPG/1024px-StLouisArchMultExpEV-4.72.JPG",
"http://upload.wikimedia.org/wikipedia/commons/thumb/c/c3/StLouisArchMultExpEV-1.82.JPG/1024px-StLouisArchMultExpEV-1.82.JPG",
"http://upload.wikimedia.org/wikipedia/commons/thumb/8/89/StLouisArchMultExpEV%2B1.51.JPG/1024px-StLouisArchMultExpEV%2B1.51.JPG",
"http://upload.wikimedia.org/wikipedia/commons/thumb/8/8f/StLouisArchMultExpEV%2B4.09.JPG/1024px-StLouisArchMultExpEV%2B4.09.JPG"
};*)

Image Combinations: Exposure Stacking

Correction of underexposed pictures

underexposed=ImageResize[Import[FileNameJoin[{NotebookDirectory[],"images","underexposed.tiff"}]],500]

img=underexposed[[2]];

Dynamically explore different brightness levels:

Manipulate[ImageMultiply[img,a],{a,1,10}]

Generate an artificial exposure stack:

ImageMultiply[img,2^#]&/@Range[0,3]

Combine the stack:

ColorToneMapping[img,Method->"RGB"]

ImageExposureCombine[%]

(*underexposed={"http://johnetherton.com/gallery/var/albums/2009/May/27th/Starbeck%20castle%20-%20under%20exposed.JPG?m=1295481475","http://christopherodonnellphotography.com/wp-content/uploads/2011/01/underexposed_sun.jpg"};*)

Image Combinations: Focus Stacking

Macro photography

fly=Import[FileNameJoin[{NotebookDirectory[],"images","fly.tiff"}]];

{fly1,fly2}=Image[#,ImageSize->All]&/@Most@Flatten@ImagePartition[fly,{Scaled[1/3],Scaled[1]}]

Combine multiple focus planes into one:

ImageFocusCombine[{fly1,fly2}]

Remember to check the image alignment before:

fly22=RemoveAlphaChannel@ImageAlign[fly1,fly2];

ImageFocusCombine[{fly1,fly22}]

(*fly=Import["http://goo.gl/xMThiZ"]*)

Image Combinations: Focus Stacking

Microscopy

cellStack=Import[FileNameJoin[{NotebookDirectory[],"images","cellStack.tiff"}]];

stack=ImagePartition[cellStack,{Scaled[1/2],Scaled[1/5]}][[;;-3,1]];
groundtruth=ImageAssemble[ImagePartition[cellStack,{Scaled[1/2],Scaled[1/5]}][[-2;;]]];
groundtruth=ImageResize[groundtruth,ImageDimensions[stack[[1]]],Resampling->{"OMOMS",7}];

Row[ImageResize[stack,250]]

speed=ImageFocusCombine[stack,PerformanceGoal->"Speed"];//AbsoluteTiming

quality=ImageFocusCombine[stack,PerformanceGoal->"Quality"];//AbsoluteTiming

TabView[Image[#,ImageSize->All]&/@Flatten[{stack,speed,quality,groundtruth}]]

(*cellStack= Import["https://upload.wikimedia.org/wikipedia/commons/5/50/FocusStack_BrightFieldLightMicroscopy_DiatomaceousEarth.jpg"];*)

Image Combinations: Flash+noFlash

Photograph taken with the a flash: high level of detail but skewed colors:

flash=Import[FileNameJoin[{NotebookDirectory[],"images","flash.tiff"}]]

Without flash the color are more faithful but much shot noise appears:

noflash=Import[FileNameJoin[{NotebookDirectory[],"images","noflash.tiff"}]]

flash=ImageTake[Import["http://research.microsoft.com/en-us/um/redmond/projects/FlashNoFlash/Applications/jbf/images/cave01_00_flash.jpg"],480];
noflash=ImageTake[Import["http://research.microsoft.com/en-us/um/redmond/projects/FlashNoFlash/Applications/jbf/images/cave01_01_noflash.jpg"],480];

A close-up showing the noise:

Image[ImageTrim[noflash,{{300,250}},80],ImageSize->400]

Take the best of both worlds by using the high detail image as a guide for the good colored one:

res=GuidedFilter[noflash,flash,2,10^-3]

Image[ImageTrim[res,{{300,250}},80],ImageSize->400]

{flash,noflash}=ImageTake[Import[#],480]&/@{"http://research.microsoft.com/en-us/um/redmond/projects/FlashNoFlash/Applications/jbf/images/cave01_00_flash.jpg",
"http://research.microsoft.com/en-us/um/redmond/projects/FlashNoFlash/Applications/jbf/images/cave01_01_noflash.jpg"};

Image Combinations: Layers

images=Import[FileNameJoin[{NotebookDirectory[],"images","layertest.tiff"}]];

Get["/Volumes/Files/Workspaces/Mathematica/Kernel/Source/Kernel/StartUp/ImageProcessing/CompositionOpsCompose.m"];

Manipulate[
ImageCompose[#1,{#2,opacity,mode}]&@@ConformImages[{Image[i1,"Real32"],Image[i2,"Real32"]}],
{{i1,images[[2]]},images},
{{i2,images[[2]]},images},
{{opacity,1},0,1},
{{mode,"Normal"},
{
"Normal","Dissolve",
Delimiter,
"Darken","DarkerColor","Multiply","ColorBurn","LinearBurn",
Delimiter,
"Lighten","LighterColor","Screen","ColorDodge","LinearDodge",
Delimiter,
"Overlay","SoftLight","HardLight","VividLight","LinearLight","PinLight","HardMix",
Delimiter,
"Difference","Exclusion",
Delimiter,
"Subtract","Divide",
Delimiter,
"Hue","Saturation","Color","Luminosity"
}}]

What’s coming

Image pyramids

Layers combination

Photo stitching, panorama creation

HDR creation

Image blending and cloning

...more to come!