Seeing the Invisible: Quantitative Phase Imaging by Focus Variation
- Fig. 1: a,d,g) Three images of HeLa cells selected from a stack of images recorded for different z-positions (z-stack or focal series), ranging from z=-177 µm to z=+177µm. Partial spatial coherence of the illumination by the green LED (λ=0.524 µm) causes fine details to be blurred at large z. This is why it makes sense to change the z-position by small amounts at small z and large amounts at large z. For this particular series a geometric progression of z was used. b,e,h) Images simulated from a wave function (phase shown in fig. 2 f) reconstructed from this z-stack. The illumination was described as an incoherent set of plane waves, whose angular spread has the shape of a Gaussian with a width of 45 mrad . c,f,i) Differences between the experimental images and the corresponding simulations.
- Fig. 2: Phase recovered by a number of different algorithms developed by the authors from the z-stack, example images of which are shown in figure 1. a) Phase map recovered by the conventional TIE. b,c) Phase maps recovered by the MFTIE algorithm, including different defocus ranges (see text for further details). The Thikonov regularization parameter, an effective low-frequency cutoff, has been kept the same for all three TIE solutions. d) Conventional TIE in which the boundary conditions and low spatial frequency phase information have been improved by gradient flipping. e) Phase map recovered by the non-linear IDPS solver [9-11] which included regularization by minimizing, also the l1-norm of the phase filtered by the ring-filter shown in figure 3. f) Phase map recovered by the FRIH algorithm  combined with gradient flipping.
- Fig. 3: IDPS solution of the sum of absolute values for the data shown in figure 1, non-regularized. Below the kernel of the ringfilter which has been used to promote sparse solutions, as the one shown in figure 2e.
- Equation 1
- Equation 2
When detecting an image in the optical microscope the phase of the wave function forming it gets lost. This phase carries valuable information about variations in the refractive index of the transmitted material, or the topography of the object characterized in reflection. We provide an overview of different reconstruction algorithms developed in our lab to recover this phase information from a set of defocused images.
A Simple Experiment to Measure Phase
In a conventional light microscope photons emitted from a source are shot at a microscopic object and the transmitted or reflected photons are then collected on a detector. Photons may take different amounts of time for this trip, depending on the length of the optical path they have taken. Assuming that all these photons originate from a single point source and are thus highly spatially coherent, and ignoring the polarization of each of these photons, we can describe the light field in any plane normal to the optical axis of the microscope by a complex-valued wave function. The amplitude squared of this wave at any position describes the probability of a photon to be found at that place, and differences in the phase of the wave between two positions account for differences in the time those photons have travelled. These phase differences are thus directly proportional to local differences in refractive index or thickness of the transmitted medium, or, when operating in reflection, topographic information.
Realizing that the phase carries much of the relevant information about transparent objects Frits Zernike developed phase contrast microscopy in the early 1930s. Since then more quantitative techniques for sample-induced phase shifts in transmitted and reflected light waves have been developed, ranging from Mach-Zehnder interferometer-type setups to integrating differential phase contrast to various light-field detection schemes. Some of these schemes require high spatial coherence; others are less stringent on the coherence requirements, but most of them require either a specially designed microscope or some dedicated hardware attached to the microscope.
One very simple technique of measuring phase differences that requires neither high spatial coherence nor any additional hardware attached to the microscope is the reconstruction of the phase π from a measurement of the intensity in a given plane (I0) and the variation of the intensity along the optical axis (∂ZI) in that plane and solving the transport of intensity equation (TIE) : see equation 1 where k0=2π/λ is the wave number and λ the wavelength.
As figure 1 shows, when illuminating unstained live cells with photons emitted by a partially spatially coherent light source, such as an LED, the contrast in the amplitude of the transmitted wave is nearly vanishing - most features of the cell remain invisible. Only at some boundaries of cell constituents where the mass density and with that the refractive index changes abruptly, diffraction leads to some contrast in the focused image. Images recorded at some defocus (achieved by raising or lowering the specimen from the in-focus condition), however, will feature Fresnel fringes at places where the phase of the imaged wave changes. With increasing defocus these fringes expand, but, because of the partial spatial coherence of the illumination, also start to appear more blurred.
Improving on the Linear Approximation
Retrieving the phase via the TIE is straightforward, but there are several problems associated with it. Two very important issues are:
1) Solving this 2nd order elliptical partial differential equation requires the phase along the boundaries of the measured area to be known beforehand (Dirichlet boundary conditions), or at least its derivatives (Neuman boundary conditions). In lack of such information experimentally rarely realized periodic boundaries of the phase or flux-preserving boundary conditions (the component of the gradient of the phase normal to the boundary vanishes) are typically assumed.
2) Determining ∂ZI by a finite difference approach from intensity measurements at only two planes of focus makes the resulting phase reliable for just a narrow band of spatial frequencies.
Even without introducing known elements to the object, such as framing the field of view with a hard-edged aperture or so, we can, in many cases, make use of prior knowledge about the object to provide constraints that help fix the unknown boundary conditions. A widely applicable type of prior knowledge is sparseness in the domain of the gradient of the phase. Although they also contain amplitude information, the fact that differential phase contrast images of very many objects investigated under the optical microscope contain large fractions of zeros, shows that this is indeed the case. While compressed sensing implemented, for example, as a total variation minimization algorithm may seem to be the obvious choice for finding sparse solutions, we have shown that gradient flipping (iteratively flipping pixels in the gradient of the phase whose absolute value is below a certain threshold) while maintaining consistency with the experimental data)  is able to recover the correct phase in a much wider range of cases . Figure 2f gives an impression of how well this works in live cell imaging.
Regarding the accuracy of the measurement of ∂ZI several approaches exist that take into account images recorded at many different z-positions. These fit, for example, a higher order polynomial to the variation of the intensity in each pixel with defocus and keep only the linear component , or solve the TIE for individual frequency bands and add the results . Our own FFT-based multi-focus TIE (MFTIE) approach relies on a geometric progression of defocus values, together with the design of the inverse Laplacian (∇-2) operator as a filter in reciprocal space that accounts also for the effect of partial spatial coherence in a flux-preserving manner  as well as the fact that images defocused by a large amount encode low spatial frequency information more strongly than those defocused only slightly, while a small defocus provides access to high-frequency details in the phase. Figure 2 compares a result obtained from applying the conventional 3-plane TIE (i.e. one image recorded in-focus, and two out-of-focus images at focal planes
∆z = ± 1 µm with two phase maps obtained by the MFTIE approach, taking into account 15 (∆z = -4.7 ... 4.7 µm) and 41 (∆z = -177 ... 177 µm) planes. We have tested the MFTIE in both synthetic and experimental datasets, obtaining quantitative phase information that agrees with our expectations. Implementing this non-iterative algorithm for high-end graphics cards it delivers the phase in about the same time it takes to acquire the data, independent of the number of focal planes included.
While the simplicity of the linear TIE is very appealing and facilitates a unique solution (for known boundary conditions) by a non-iterative approach, it is based on a linearized approximation of the contribution of the defocus on the coherent contrast transfer function -see equation 2- where n is the refractive index, ∆z the defocus, and (kx,ky) the reciprocal space coordinates in the back-focal plane of the objective lens. This approximation starts to break down at numerical apertures greater than 0.3. at NA > 0.3 the correct expression (left hand side of ‘≈’) for the CTF should be used, and the non-linear nature of the resulting expression for the image intensity requires the use of iterative algorithms which optimize the estimate for amplitude and phase of the wave function by matching images simulated from this wave to the corresponding experimental data (see figure 1). A number of approaches aiming at accurate wave retrieval have been developed (limited space forbids to produce a representative list here). Some of them combine the power of the TIE to retrieve very low spatial frequency components of the phase with the accuracy of iterative approaches in recovering its high spatial frequency components. One of these is the full-resolution inline holography (FRIH) algorithm  which takes particular care to avoid low spatial frequency artefacts introduced by the TIE, accounts for partial spatial coherence, and changes in image magnification and distortions when changing the defocus. Figure 2f, and the phase image shown at the top of this article have been recovered using this algorithm.
In the inverse dynamical photon scattering (IDPS) algorithm, the propagation of the optical wave through the sample and the objective lens is taken into account via the beam propagation method (also called multislice approximation) [8-10]. Although it is capable of three-dimensional reconstructions , we will restrict ourselves to two dimensions here.
IDPS retrieves the phase by iteratively minimizing an error function through gradient descent optimization. These gradients are calculated efficiently by reformulating the image formation process as an artificial neural network. Furthermore, the error function can be chosen freely by the user, the advantage of which has been demonstrated in  where changing from a sum of least squares to a sum of least absolute values greatly increased the solution’s robustness.
IDPS’s explicit access to its error function also enables regularization by adding a regularization term, ER, and thus select for alternate solutions with enhanced lower spatial frequencies. In order to promote sparse edges, ER is set to the ℓ1-norm of the result of the so-called ring-filter applied to the solution.
In figure 3 the solution without regularization and the ring-filter are depicted, respectively. The regularized solution is shown in figure 2e. In all cases, only 13 defoci were used: 22 µm, 17 µm, 10 µm, 5 µm, 2 µm, 1 µm, 0 µm and their negatives. Note how the regularization brings out the lower frequencies.
In summary, we have presented several algorithms for recovering the phase shift induced by transparent objects in the optical microscope. While TIE-based solutions are faster than non-linear reconstruction algorithms, even when iteratively applying gradient flipping, the non-linear approaches promise higher resolution and accuracy at high NA and, in the case of IDPS access to 3D .
 M. Teague, Deterministic phase retrieval: a Green's function solution, J. Opt. Soc. Am. 73, 1434-1441 (1983) DOI: 10.1364/JOSA.73.001434
 A. Parvizi, W. Van den Broek, C.T. Koch, Recovering low spatial frequencies in wavefront sensing based on intensity measurements, Advanced Structural and Chemical Imaging 2, 3 (2016) DOI: 10.1186/s40679-016-0017-y
 A. Parvizi, W. Van den Broek, C.T. Koch, Gradient flipping algorithm: introducing non-convex constraints in wavefront reconstructions with the transport of intensity equation, Optics Express 24, 8344-8359 (2016) DOI: 10.1364/OE.24.008344
 T. Peterson, V.J. Keast, K. Johnson, and S. Duvall, TEM-based phase retrieval of p–n junction wafers using the transport of intensity equation, Philosophical Magazine 87, 3565-3578 (2007) DOI: 10.1080/14786430701361388
 J. Martínez-Carranza, K. Falaggis, and T. Kozacki, Multi-filter transport of intensity equation solver with equalized noise sensitivity, Optics Express 23, 23092-23107 (2015) DOI: 10.1364/OE.23.023092
 C.T. Koch, A flux-preserving non-linear inline holography reconstruction algorithm for partially coherent electrons, Ultramicroscopy 108, 141-150 (2008) DOI:10.1016/j.ultramic.2007.03.007
 C.T. Koch, Towards full-resolution inline electron holography, Micron 63, 69-75 (2014) DOI: 10.1016/j.micron.2013.10.009
 W. Van den Broek, C.T. Koch, Method for Retrieval of the Three-Dimensional Object Potential by Inversion of Dynamical Electron Scattering, Phys. Rev. Lett. 109, 245502 (2012) DOI: 10.1103/PhysRevLett.109.245502
 X. Jiang, W. Van den Broek, C.T. Koch, Inverse dynamical photon scattering (IDPS): an artificial neural network based algorithm for three-dimensional quantitative imaging in optical microscopy, Optics Express 24, 7006-7018 (2016) DOI: 10.1364/OE.24.007006
 W. Van den Broek, C.T. Koch, General framework for quantitative three-dimensional reconstruction from arbitrary detection geometries in TEM, Phys. Rev. B 87, 184108 (2013) DOI: 10.1103/PhysRevB.87.184108
Christoph T. Koch1, Johannes Müller1, Wouter Van den Broek1, Amin Parvizi1, Alberto Eljarrat1 and Katharina Blessing2
1 Humboldt University of Berlin, Department of Physics, Berlin, Germany
2 Ulm University, Institute for Biophysics, Ulm, Germany