Mathematica and Image Processing: A Teacher's Perspective

Han van Triest

Sino-Dutch School of Biomedical and Information Engineering
North-Eastern University, Shenyang, China
Han@bmie.neu.edu.cn

Overview

Background

Course "Image Processing"

Design Centered Learning (DCL)

DCL: Image Processing for Pathology

Concluding remarks

Background

Introduction

- Han van Triest, also known as 高汉

- Since September 2007 employed at the Sino-Dutch School of Biomedical and Information Engineering

- Studied BMT (Biomedical Engineering) at Technical University of Eindhoven (The Netherlands)

- Received BSc in 2005 and MSc in 2007

- Main focus: Medical Image Processing

- Group of Prof. Bart ter Haar Romenij

Background

Internship: "Towards a CAD system for Diffuse Lung Diseases"

- Goal: Locate Diffuse Lung Diseases in Computed Tomography (CT)

- Diffuse lung disorders are a group disorders affecting the interstitium of the lung

- Labor Intensive to locate and diagnose

- Approach: Automatically subdivide lung into smaller areas based on texture, next classify each region

Background

Graduation Project: "Tools for Shape Analysis of Vascular Response using Two Photon Laser Scanning Microscopy"

- A Two Photon Laser Scanning Microscopy can record 3D images from living tissue

- Goals:

- Determine the diameter of (life) murine (mice) arteries when only a shallow arc is available

- Segment and examine the cell-nuclei from a 3D Murine Arterial Data

Background

Responsible for several courses at BMIE

- Image Processing

- Design Centered Learning (DCL)

Course "Image Processing"

Image Processing

- Introductionary Course: Teaching the basics of Image Processing

- Topics include:

- Image Preprocessing and Enhancement Methods

- Segmentation Methods

- Features and Classification

Course "Image Processing"

Image Processing:

- Working with a lot of images (obviously)

- Many operators have parameters that should be explored

- Many filtering operators require moving a kernel over an image

Problem:

- Traditional courseware is very static

Course "Image Processing"

Mathematica and the course "Image Processing"

- Course completely given in Mathematica

- Students can experiment with the code, and get to know the image processing operations intimately

- An experimental class is given where the focus lies on "doing" and "exploring"

Course "Image Processing"

Example: Thresholding

- Simple segmentation algorithm

- Segmentation is the separation of foreground and background

- Thresholding uses intensity to separate foreground from background:

g(x,y)=(Piecewise[{{Foreground (1),  ,  ,  , if f(x,y) >T}, {Background (0),  ,  ,  , if f(x,y) ≤ T}}])

Examples:

SeedRandom[1];
SetOptions[ArrayPlot,{DataReversed->True,ColorFunction->GrayLevel, ClippingStyle->{Black,White}}];
img1=Table[If[(((x-32))^2)+(((y-32))^2)<(20^2),200,100]+Random[NormalDistribution[0,13]],{x,1,64},{y,1,64}];
Threshold[img_,t_]:=Map[If[#<t,0,1]&,img,{2}];
GraphicsRow[{ArrayPlot[img1],Histogram[Flatten[img1]],ArrayPlot[Threshold[img1,150],PlotLabel->"Threshold at 150"],ArrayPlot[Threshold[img1,170],PlotLabel->"Threshold at 170"]}]

Course "Image Processing"

Example: Thresholding

- Influence of Noise

Course "Image Processing"

Example: Thresholding

- Complicated to show like this. More convenient if you can play with the parameters!

SeedRandom[1];
SetOptions[ArrayPlot,{DataReversed->True,ColorFunction->GrayLevel, ClippingStyle->{Black,White}}];
Threshold[img_,t_]:=Map[If[#<t,0,1]&,img,{2}];
Manipulate[
img1=Table[If[(((x-32))^2)+(((y-32))^2)<(20^2),200,100],{x,1,64},{y,1,64}]+RandomReal[NormalDistribution[0,sd],{64,64}];
GraphicsRow[{ArrayPlot[img1],Histogram[Flatten[img1],Epilog->{Red,Thick,Line[{{t,0},{t,10000}}]}],ArrayPlot[Threshold[img1,t],PlotRange->{0,1}]},ImageSize->600],
{{t, 150,"Threshold value"},0,Max[img1]},
{{sd,10,"Standard Deviation of the Noise"},0,40},SynchronousUpdating->False,ControlPlacement->Top]

Course "Image Processing"

Example: Mathematical Morphology

Morphology normally denotes the branch of biology that deals with the form and structure of animals and plants

Mathematical Morphology: Tools for extracting image components that are useful in describing the objects

Input to Mathematical Morphology operators are binary images (pixels are either 0 or 1)

Mathematical Morphology can be described using operators from the set-theory

Course "Image Processing"

Basic Set Operations

Some basic operations:

Union: C=A⋃B

Intersection: D=A⋂B

Mirroring: (OverHat[B])={w|w=-b, for b∈B}

Translation: (Subscript[(A), z])={w|w=a+z, for a∈ A}

Course "Image Processing"

Example: Mathematical Morphology

Basic Mathematical Morphology operations

The basic Mathematical Morphology operations are Erosion and Dilation

Dilation:

Let A,B⊂ (ℤ^2), then the dilation of A by B, written as A⊕B is defined as:

A⊕B={z|(Subscript[((OverHat[B])), z])⋂A≠∅}

Some Examples:

BinaryDilation[im_,el_]:=ListConvolve[el,im,Ceiling[Dimensions[el]/2],0,BitAnd,BitOr];
(im_⊕el_):=BinaryDilation[im,el]
BinaryErosion[im_,el_]:=ListCorrelate[1-el,im,Ceiling[Dimensions[el]/2],0,BitOr,BitAnd];
(im_⊖el_):=BinaryErosion[im,el];
k1={{0,1,0},{1,1,1},{0,1,0}};
k2={{1,1,1},{1,1,1},{1,1,1}};
k3={{1,0,0},{0,1,0},{0,0,1}};
k4=Reverse[k3];
img14=(({{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}));
DotPlot[img1_,img2_,n_]:=ArrayPlot[img1,Epilog->{Red,Point@(Reverse/@(Position[img2,n]-0.5))},Mesh->False];
GraphicsGrid[{ArrayPlot[#,PlotRange->{0,1}]&/@{k1,k2,k3,k4},DotPlot[img14⊕#,img14,1]&/@{k1,k2,k3,k4}}]

Course "Image Processing"

Example: Mathematical Morphology

Basic Mathematical Morphology operations

The basic Mathematical Morphology operations are Erosion and Dilation

Erosion:

Let A,B⊂ (ℤ^2), then the erosion of A by B, written as A⊖B is defined as:

A⊖B={z|(Subscript[(B), z])⊆A}

Some Examples:

BinaryDilation[im_,el_]:=ListConvolve[el,im,Ceiling[Dimensions[el]/2],0,BitAnd,BitOr];
(im_⊕el_):=BinaryDilation[im,el]
BinaryErosion[im_,el_]:=ListCorrelate[1-el,im,Ceiling[Dimensions[el]/2],0,BitOr,BitAnd];
(im_⊖el_):=BinaryErosion[im,el];
k1={{0,1,0},{1,1,1},{0,1,0}};
k2={{1,1,1},{1,1,1},{1,1,1}};
k3={{1,0,0},{0,1,0},{0,0,1}};
k4=Reverse[k3];
img14=(({{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}));
DotPlot[img1_,img2_,n_]:=ArrayPlot[img1,Epilog->{Red,Point@(Reverse/@(Position[img2,n]-0.5))},Mesh->False];
GraphicsGrid[{ArrayPlot[#,PlotRange->{0,1}]&/@{k1,k2,k3,k4},DotPlot[img14⊖#,img14,1]&/@{k1,k2,k3,k4}}]

Course "Image Processing"

Example: Mathematical Morphology

Integrated in one simple applet, the process of Dilation and Erosion can be explained much easier

Clear[MorphologyHelper,StructElement];
StructElement[Dynamic[el_]]:=Grid[(({{Checkbox[Dynamic[el[[1,1]]],{0,1}], Checkbox[Dynamic[el[[1,2]]],{0,1}], Checkbox[Dynamic[el[[1,3]]],{0,1}]}, {Checkbox[Dynamic[el[[2,1]]],{0,1}], Checkbox[Dynamic[el[[2,2]]],{0,1}], Checkbox[Dynamic[el[[2,3]]],{0,1}]}, {Checkbox[Dynamic[el[[3,1]]],{0,1}], Checkbox[Dynamic[el[[3,2]]],{0,1}], Checkbox[Dynamic[el[[3,3]]],{0,1}]}}))];
BinaryDilation[im_,el_]:=ListConvolve[el,im,Ceiling[Dimensions[el]/2],0,BitAnd,BitOr];
(im_⊕el_):=BinaryDilation[im,el]
BinaryErosion[im_,el_]:=ListCorrelate[1-el,im,Ceiling[Dimensions[el]/2],0,BitOr,BitAnd];
(im_⊖el_):=BinaryErosion[im,el];
MorphologyHelper[img_,originalState_:False]:=
Block[{d4,ker={{0,1,0},{1,1,1},{0,1,0}}},
d4=Dimensions[img]-{3,3};
Manipulate[
Block[{res},
(*Erosion or Dilation*)
res=If[erOrDil=="Dilation",img⊕Reverse[ker],img⊖Reverse[ker]];
(*Displaying the results*)
Column[{
(*First Row: Text*)
Row[{"Original: ",Grid[Reverse[img[[y;;y+2,x;;x+2]]],Frame->All],"  Structuring Element : ",Grid[ker,Frame->All],If[erOrDil=="Dilation","  Dilated result : ","  Eroded result :"],Grid[{{If[showAnswer,res[[Round[y+1],Round[x+1]]],"?"]}},Frame->All]}],
(*Second Row: Images*)
GraphicsRow[{ArrayPlot[img,ImageSize->200,Epilog->{Thick,Red,Line[{{x-1,y-1},{x+2,y-1},{x+2,y+2},{x-1,y+2},{x-1,y-1}}]}],ArrayPlot[res,ImageSize->200,Epilog->{PointSize[.04],Red,Point[Round[{x+1,y+1}]-.5]}]}]
}]
]
,
(*Controllers*)
Style["Location",Bold,Medium],
{x,1,d4[[2]]+1,1},{y,1,d4[[1]]+1,1},
Style["Options",Bold,Medium],
{{erOrDil,"Dilation","Operation : "},{"Erosion","Dilation"}},
{{showAnswer,False,"Show the Answer : "},{True,False}},
Style["Structuring Element",Bold,Medium],
{{ker,{{0,1,0},{1,1,1},{0,1,0}},""},StructElement[##]&}
]]
MorphologyHelper[(({{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0}, {0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}}))]

Course "Image Processing"

Experimentation classes

- Doing experiments under supervision

- Exploring the functions introduced during the classes

- Learning by Doing!

Design Centered Learning (DCL)

Current Engineering Education

- Classical model: Teacher talks, student listens

- Learning is an individual experience

- Students are judged by their grades, not their potentials

- Little space for individuals

Design Centered Learning (DCL)

"Design"

Noun:

- A plan or drawing produced to show the look and function or workings of something before it is built or made.

- The art or action of producing such a plan or drawing.

- ...

Verb:

- Conceive and produce a design for

- Plan or intend for a purpose

(source: www.askoxford.com)

Design Centered Learning (DCL)

Design of:

- Experimental setup

- Model

- Algorithm

- Research Strategies

- Etc, etc, etc...

Focus of education: Skills!!

Design Centered Learning (DCL)

Origin

Developed at the Technical University of Eindhoven, The Netherlands, from 1998

Design Centered Learning (DCL)

What is DCL?

‘a concept of technical university education, in which students work co-operatively and actively on multidisciplinary design tasks, with the purpose of gaining qualifications as creative professionals capable of integrating all relevant aspects of education, in order to analyze existing technical systems, to assess their quality, functionality and cost price and with the purpose of designing new products  and systems with increased performance’.

(from: Towards Design-Based Learning, Prof. Dr. W. H. F. W. Wijnen, Technical University of Eindhoven)

Design Centered Learning (DCL)

Focus on Academic Skills such as:

- Reflection on personal activities

- Training of a sense of responsibility

- Critical analysis of design tasks

- Broad Interpretation of designing

- Incorporation of modern scientific views

Design Centered Learning (DCL)

Six main features of DCL

- Professionalizing

- Activating

- Co-operation

- Creating

- Integrating

- Multidisciplinar

Design Centered Learning (DCL)

A DCL is not a traditional course!

- Chain of Cases

- Opportunity to combine knowledge of  courses

- Give specific workshop or courses a context

- Opportunity to get experience of scientific practice

Design Centered Learning (DCL)

Once a Week, a Meeting: Under Supervision of a Tutor

Students tasks Involve:

- Formulate what to study

- Report back to the group

- Cooperate towards a final product

- Be responsible scientists

- Special roles

Design Centered Learning (DCL)

Rotating Special Roles:

- Minutes Secretary

- Writer

- Chairman

- Student Meeting Representative

- General Group member

Design Centered Learning (DCL)

Tutor

- Can steer students away from possible problems

- Is there to observe

- Does NOT participate in discussions

- Gives a personal evaluation for each student

DCL: Image Processing for Pathology

Pathology

"Pathology (from Greek πάθος, pathos, "fate, harm"; and -λογία, -logia) is the study and diagnosis of disease through examination of organs, tissues, bodily fluids and whole bodies (Autopsy)."

A pathologist is a specialist that diagnoses disease by means of studying microscopic samples of the affected organ

Samples are often taken during biopsies

Pathologists study many samples a day

Helping a pathologist can increase chances of correctly diagnosing a disease

DCL: Image Processing for Pathology

Learning objectives

- Carrying out a project where a quantitative clinical question on images is answered

- The design of creative and efficient methods for image analysis

- Get insight in the design process of a software solution by the critial analysis of the specific question

- Management of a software project, where a range of tasks have to be implemented in a coherent fashion; distribution of tasks, recognize strong points in group members and pass the expertise to each other

- Encounter real image analysis problems of a clinical department

- Reporting in an interactive document, where text, graphics, animaions, images etc. are integrated with the program code

DCL: Image Processing for Pathology

Approach

- A number of images are given to the students

- The students can decide by them selves which disease to tackle

- In the end of the project, a group should hand in a report written in Mathematica containing a working program

Conclusion

Advanced education requires advanced tools!

- The powerful dynamic structure enables for the creation of dynamic courseware, that can be adapted on the spot to answer specific questions by students

- The notebook structure gives the opportunity to play with courseware

- Mathematica enables students to experiment with state of the art and extremely powerful software

- New powerful Image Processing functions will enable students even more to explore the wonderful world of image processing