III.D.1. Optical Diffraction

Optical diffraction is the simplest and most widely-practiced Fourier image processing technique, and is usually the initial step in many image processing studies. The main advantage of this technique is that it provides an objective way to assess and reveal periodic structural information. Klug and Berger (1964) were the first to use an optical bench to examine diffraction patterns of electron micrographs and thereby objectively analyze structural information in images of biological specimens.

a. Forming the diffraction pattern

Optical diffraction patterns are easily produced from selected (masked) regions of micrographs. A simple optical bench consists of a laser, to produce a parallel, monochromatic beam which illuminates a specific area of the micrograph, and a (diffraction) lens to focus the Fraunhofer diffraction pattern in the back focal plane of the lens. The pattern may be viewed directly, but cautiously to avoid focusing the bright, central spot formed from the unscattered rays on the retina, or it may be recorded on a standard photographic emulsion.

b. Experimental apparatus: the optical diffractometer

There are several diffractometer designs, which fall into one of two basic classes depending on whether the optical path is straight (linear diffractometer: Fig.III.58) or bent by use of optically flat mirrors (folded diffractometer: Figs.III.59 and 60). Which type of diffractometer to use depends, in part, on the intended use of the apparatus. The folded design is usually preferred for rapid screening of a large number of micrographs when image quality and specimen preservation need to be assessed in order to select images for subsequent optical (Sec. III.D.2) or digital (Sec. III.D.3) filtering operations. For high quality optical reconstruction work, the linear design is generally preferred since there are fewer optical components, and thus fewer aberrations.

A diffractometer of reasonable quality, suitable for simple experiments such as screening images or detecting and locating periodicities, can be built or purchased for a few thousand dollars or less. More expensive designs ($10,000 or more) are usually easier to use and align, and produce high quality diffraction patterns and reconstruction images. A quality diffractometer usually includes an image reconstruction system (with a high-quality, corrected, doublet lens), a pinhole spatial filtering system to remove noise in the illumination beam, a moderate-to-high power laser (1-50 mWatt), high-quality, high-reflectance mirrors (if the optical path is folded), fully adjustable, precision holders for all components, and an image and diffraction pattern recording system.

Fig.III.58. Simple optical diffractometer. The diagram shows the arrangement of the components used to construct a simple optical diffractometer. A. laser; B, shutter; C, beam expanding lens; D, pinhole; E, adjustable diaphragm; F3, diffraction lens; G, electron micrograph; and H, viewing screen or camera. (From Horne and Markham, p.336)

Fig.III.59. Schematic diagram of an optical diffractometer. (From Thompson (Lipson), p.48)

Fig.III.60. Diagram of the UCLA folded optical diffractometer, built in 1972.

Many laboratories prefer to construct their own diffractometers with specifications dictated by the intended use of the instrument. For example, if most of the image reconstruction work is to be performed on a computer, a simple and inexpensive linear diffractometer for surveying images will suffice. The use of liquid gates, in which micrographs are submerged in oil to iron out inhomogeneities in the micrograph emulsion (and glass or gelatin backing), produce diffraction patterns in which Friedel symmetry is nearly perfectly preserved (Table 1.I.C.2.e, Baker, 1981). However, such extreme measures prove inconvenient and, in any event, digital processing systems are both fast and reliable and are usually preferred over high-quality optical processing systems.

Details of the design, operation, components, alignment, and calibration of optical diffractometers can be found in several references (Horne and Markham, 1972; Mulvey, 1973; Johansen, 1975; Erickson, et al., 1978; Baker, 1981). The optical diffractometer, much like the electron microscope, needs to be carefully aligned and calibrated to perform optimally.

c. Applications of optical diffraction

Optical diffraction provides useful information about the geometrical arrangement of subunits in the specimen. Such structural detail often can not be discerned by simple, visual inspection of the original micrograph. For example, the presence of rotational screw axes or pseudo-symmetries may go undetected without the information provided by the optical diffraction pattern. These types of structural information are determined by correctly indexing the pattern, that is, defining a lattice (or lattices for multilayered or helical particles) which accurately defines the location of all diffraction spots.

Indexing is an essential step for correct application of optical or digital filtering, or 3D reconstruction techniques (Sec. III.D.2). With an exception for some helical and multilayered particles, the indexing of OD patterns from most planar specimens, is quite straightforward. Articles by Finch, Klug and Nermut (1967), Moody (1967), Kiselev and Klug (1969), Mikhailov and Belyaeva (1971), DeRosier and Klug (1972), Lake (1972b), Leonard, Kleindschmidt and Lake (1973), and Unwin and Taddei (1977) give excellent, examples of pattern indexing (these and additional examples are cited in Table 1.I.D.8, Baker, 1981). Misell's book (1978; pp.106-122) devotes an entire section to theoretical and practical problems of indexing. Artifacts in optical diffraction patterns can make indexing difficult (Table 1.I.E, Baker, 1981). The characteristic and often prominent "cross" observed in many optical diffraction patterns is mainly a consequence of strong diffraction caused by the edges of the mask used to select a region of interest in the micrograph. This feature is regarded as "noise" in the pattern and thus, should not influence the selection of a consistent indexing scheme. Fig.III.61 shows a typical example of an optical diffraction pattern recorded from an image of a negatively-stained 2D crystal of catalase. Other examples will be shown in class.

Several applications of optical diffraction include:

- Accurate measurement of lattice parameters (unit cell dimensions). - Detection of rotational and translational symmetry elements. - Determine relative orientation of multilayered specimens (e.g. stacked 2D sheets or opposite sides of two-sided structures). -Detect and measure specimen preservation (distortions, overall resolution, radiation damage) for selecting best images for further image analysis. - Assess short/long range order in periodic specimens. - Identification of signal vs. noise in images. - Ability to examine specific small areas. - Determine electron optical conditions, i.e. contrast transfer function (focus, drift, astigmatism, etc.) at time micrograph was recorded. - Determine the hand of 3D structures (from metal-shadowed or tilted specimens). - Superb device for teaching principles of diffraction, symmetry, and Fourier transforms.

Optical diffraction techniques have been successfully employed in fields outside electron microscopy, for example, in the study of small-molecule crystal structures. In fact, much of the original development of diffractometers was made by pioneers of crystallography. W. L. Bragg (1939) designed the first optical diffractometer, calling it a "new type of X-ray microscope".

Several crystallographers used optical diffraction methods as an aid in solving small molecule crystal structures. By comparing diffraction patterns produced by models of the crystal structure (using various size holes punched in sheets of metal at predicted positions to represent atoms) with the experimentally recorded x-ray diffraction patterns, it was often possible to rule out incorrect structures and thereby verify or solve a crystal structure.

Fig.III.61. (Left) Low magnification micrograph of negatively-stained bovine liver catalase. (Right) High magnification view of small portion of same crystal. (Right) Optical diffraction pattern recorded from the area outlined in the low magnification image.

III.D.2. Optical Filtering

Optical filtering, independently introduced by Klug and DeRosier (1966) and Bancroft, Hills and Markham (1967), is only suitable for the study of periodic specimens with translational symmetry. The main advantage of this method is it provides a simple way to remove the contributions from noise in micrographs and thereby reveal clearer images of specimen structure. In addition, it is a powerful method for separating Moiré images of multilayered specimens. Other applications are outlined Table 1.II.B of Baker (1981).

The review by Erickson, Voter, and Leonard (1978) and articles by Klug and DeRosier (1966) and Fraser and Millward (1970) give excellent introductions to the theory and techniques of optical filtration. The basic principle of the technique is straightforward, but, in practice, the method can easily lead to erroneous results. This is especially true for uninformed novices who are likely to be unaware of the types of artifacts that can occur.

a. Indexing the diffraction pattern

The first, and most important step in an optical filtering experiment is to correctly index the optical diffraction pattern of the specimen. A pattern is considered successfully indexed if it is possible to distinguish between spots arising from noise (aperiodic image details) and those attributed to the periodic nature of the specimen. Although it is unnecessary to attempt to identify all the noise components in the unprocessed image, a correct filtration experiment requires knowledge of how noise and signal components are distinguished. For most crystalline specimens, the diffraction pattern is a lattice of bright spots (reflections) against a weaker background of noise (Fig.III.62). Noise, or aperiodicities in the image, produce spots in all parts of the pattern. Note that noise (periodic noise) which appears at or close to the lattice points of the diffraction pattern, CANNOT be removed by filtering. Systematic specimen flattening or staining artifacts are examples of situations which produce periodic-type noise. Other major sources of noise are listed in Sec. III.B(p.178).

If the diffraction pattern is difficult to index, an incorrect lattice may have been identified (e.g. because a superlattice has been missed). Occasionally, strong, non-indexible spots may be attributed to multiple scattering (Table 1.I.D.7, Baker, 1981) or they might arise from strong, aperiodic features in the specimen. The temptation may be to ignore images with non-indexible patterns, but difficulties with indexing often clearly indicates that important structural information has been overlooked. Novices of image processing will benefit from study of the indexing examples in Misell's book (1978; pp.106-122) and cited in Table 1.I.D.8 of Baker (1981). Figures III.63-66 illustrate some aspects of the indexing of OD patterns.

Fig.III.62. Electron micrographs and optical diffraction patterns of four different kinds of T-even polyhead. All specimens were negatively-stained with 2% NaPT. (a) Coarse polyhead. (b) A-type polyhead. (c) B-type polyhead. (d) C-type polyhead. Differences in the OD patterns reflect differences (that can't be seen by naked eye) in the 'crystal' lattice structures. (From Steven et al., 1976a, p.192)

Fig.III.63. Optical diffraction of a portion of a plane layer of phosphorylase b particles and optical filtering of the image. (a) OD pattern (right hand part is indexed on the reciprocal lattice). (b) Portion of a 2D crystal before filtering experiment (stain here is white and protein is black). The particle in the circle is missing. (c) Filtered image. The missing particle shows up as a result of the averaging action of filtering. The unit cell shown on the image corresponds to the reciprocal lattice of (a). (From Kiselev et al., 1971, Plate III)

Fig.III.64. (a) Optical transform. (b) Micrograph of negatively-stained bacteriophage T4 polyhead (coarse). (c) One-sided optical reconstruction of the polyhead lattice. Image bar = 20 nm. DIffraction bar = 0.2nm-1. (From Misell, 1978, p.116)

Fig.III.65. Reciprocal lattice for the coarse polyhead shown in Fig.III.64. The first order of the hexagonal lattice is missing. Note also that these schematic representations of the OD pattern (a) and one side of the reciprocal lattice (b) are rotated 90 degrees with respect to the OD pattern depicted in Fig.III.64. (a) Original OD pattern with spots from one-side ringed. (b) Reciprocal lattice drawn through the spots resulting from one-sided diffraction (arrowed). (h,k) define the diffraction order; a* and b* are the reciprocal lattice constants. a* = b* for a hexagonal lattice. (From Misell, 1978, p.116).

Fig.III.66. A capsomere with 6-fold symmetry convolved over a hexagonal lattice generates a diffraction pattern in which the Fourier transform of the capsomere is sampled on a hexagonal reciprocal lattice. The 6-fold symmetry of the capsomere is reflected in its Fourier transform in that, in the absence of noise, each diffraction spot is related to five other hexagonally congugate spots. These sextets of equivalent spots lie on concentric circles which, in order of increasing radius we refer to as 1st, 2nd, 3rd, etc. orders, as shown in the schematic drawing. The radii of these circles bear fixed ratios to one another (R1 : R2 : R3 : R4 etc. = 1:~ 3:2:~7 etc.). In practice, sources of electron micrograph noise, as well as departures from exact symmetry, comprise the equivalence of hexagonally congugate spots, but indexation of the diffraction pattern is possible provided at least two orders are visible. (From Steven et al., 1976a, p.194)

Fig.III.67. Optical filtrations of the four major classes of T4 polyhead as shown in Fig.III.62. Upper left: coarse polyhead. Upper right: A-type polyhead. Lower left: B-type polyhead. Lower right: C-type polyhead. (From Steven et al., 1976a, p.200)

b. Filtering procedure

Once the diffraction spots are found to be consistent with a given lattice, a filter mask is designed with holes positioned to allow unobstructed passage of the diffraction spots at the lattice points (or lattice lines for helical particles). The mask is accurately positioned in the diffraction plane of the optical diffractometer so all spots at the lattice points are allowed through (Fig.III.68). Opaque regions of the mask block out most of the noise in the diffraction pattern arising from non-periodic image features. A reconstruction lens, placed behind the mask, refocuses the unobstructed rays and forms a filtered image. An unfiltered image is formed if the mask is removed. Optical reconstruction illustrates the Abbe double-diffraction phenomena of image formation (Sec. III.C.6.d): the diffraction pattern of the micrograph is formed in the first stage (forward transformation), and, in the second stage the reconstruction lens acts to rediffract the diffracted rays (back- or reverse transformation) to form an image (filtered or unfiltered). Thus, a filtered or unfiltered image is the result of rediffraction of the masked or unmasked diffraction pattern of the object (micrograph).

c. Filtering apparatus (Figs.III.68-70)

Filtration experiments are performed on an optical diffractometer equipped with a reconstruc-tion lens (or lenses). The reconstruction system must be of high optical quality to minimize image distortions (e.g. phase errors due to spherical aberration). Camera lenses often make suitable reconstruction lenses, although they are usually expensive and not ideally designed for the purposes of the optical reconstruction experiment (camera lenses are generally designed for optimum transmission of light, not for flatness of field). A high-quality, but inexpensive, corrected doublet, with a large usable aperture, can produce high resolution, reconstruction images.

Fig.III.68. Optical filtering: (a) recording of diffraction pattern; (b) recording of the filtered image. (From Slayter, p. 448)

Fig.III.69. An optical system typically used to diffract and filter electgron micrographs. (From Lake (Lipson), p. 63)

Fig.III.70. Schematic diagram of an optical diffractometer used to form a reconstructed image, I. C and S are the collimating and diffraction lenses, respectively and O is the reconstruction lens. (From Blundell and Johnson, p.110)

Most image processing laboratories use a folded diffractometer (e.g. FIg.III.60) both for survey and reconstruction work, mainly because it is more convenient to operate compared with a linear-type apparatus. The main disadvantage of the folded design is that mirrors are required to bend the optical light path. Mirrors add extra optical surfaces which collect dust or become scratched and thus can deteriorate the quality of the diffraction pattern or reconstruction image. Expensive, high-quality (high reflectance and optically flat) mirrors are recommended for optimum results. The quality of the optical bench is easily assessed by critically comparing an unfiltered reconstruction with the original image. The closer the match the better the reconstruction system.

Fig.III.71. (a) Electron micrograph of AB-type T4 polyheads, negatively-stained with 2% NaPT. (b) OD pattern of the marked region of the polyhead. (c) Indexation of the reciprocal lattice of the diffraction pattern generated by one side of the flattened polyhead bilayer. Visible diffraction spots are shown with solid circles, invisible ones with empty circles. For orientation, the dominant spots of the 4th radial order are shown with larger full circles. The other diffraction spots can be obtained from the Indexation by reflection of the given lattice through the meridian (vertical axis). (d) Optical filtration of this polyhead. (From Steven et al., 1976a, p.205)

d. Design of filter masks

Once a consistent indexing scheme is established, the design and fabrication of the filter mask (Table 1.II.E, Baker, 1981) usually constitute the rate limiting steps in a filtration experiment. Etching procedures are used to produce precise masks: but, more tedious and demanding manufacturing skills are required compared to those for preparing masks by punching or drilling holes. Erickson, Voter and Leonard (1978) describe a simple method for producing suitable masks within minutes. Their method has the additional advantage of using the original, recorded pattern as a template.

e. Image averaging

Optical filtering reduces image noise by averaging neighboring, periodically-repeated units in the array. As the size of holes in the filter mask are reduced, more noise in the diffraction pattern is removed and the extent of local averaging increases (Figs.III.72-74). That is, the image of a given unit in the array is averaged with more of its neighbors. If holes are made smaller than the diffraction spots, the signal-to-noise ratio may decrease (Table 1.II.F.1.d, Baker, 1981).

Fig.III.72. Optical filtering demonstration, Part 1. (a) 40 by 40 array of 'perfect' circular holes representing an idealized model of a crystal structure. A magnified portion appears directly above the complete array. The diffracting object is a copper foiled with holes etched in it. (b) The OD pattern of (a). Note that the transform exists only at discrete lattice points (reciprocal lattice) except for the subsidiary maxima which are shown more clearly in the enlarged view (c). Because (a) is the convolution of a circular hole with a 40 by 40 lattice of points, (b) is the transform of a single hole (Airy function) multiplied (or sampled) by a lattice which is the reciprocal of the lattice of (a). (c) Enlarged central region of (b) showing the subsidiary maxima between lattice points. The subsidiary maxima contain information about the overall shape of the diffracting object.. If n is the number of repeating units in a given direction, then the number of subsidiary maxima along the same direction in the transform is n-2. Thus, by counting the number (38 in this example) of maxima between two lattice points in (c), the number of repeating units (40) can be determined without seeing the object. (d) 50 by 50 array of imperfectly shaped holes representing a distorted crystal structure. The magnified region (above) also shows an extra hole which does not belong to the rest of the lattice. (e) Optical transform of (d). A large portion of the diffracted light falls between the lattice points, indicating the presence of aperiodic information in the object (d). (f) Enlarged central region of (e).

Fig.III.73. Optical filtering demonstration, Part 2. (a,e,i) Filtering masks (insets) and their transforms. The masks are identical 11 by 11 arrays except for the size of the holes. da*/a* = 0.43, 0.20, and 0.10 for the masks represented in (a), (e), and (i). The masks are designed to filter the central 11 by 11 array of the transform shown in FIg.III.72e. All three mask transforms have the same lattice parameters (which must be identical with the object lattice), but they are multiplied by the transform of the different size holes in each case. As the mask hole gets smaller, the area of the central maximum of the hole transform (Airy function) increases. The mask transform is the function that the object (Fig.III.72d) is convoluted with, thus the size of the central maximum is the area of local averaging in the filtered reconstructions (d,h,l). (b,f,j) Enlarged views of (a), (e), and (i), respectively. The number of lattice points contained in the central maximum is approximately the number of times each repeating unit of the object gets superimposed in the filtered reconstruction image. The numbers in these examples are approximately (b) 17, (f) 79, and (j) 314. (c,g,k) Same as Fig.III.72f with masks of (a), (e), and (i) positioned in the transform plane of the optical diffractometer. This shows what information is allowed to pass the transform plane of the diffractometer and recombine in the reconstruction plane. (Note: only the central 3 by 3 portion of the 11 by 11 array is shown here). (d,h,l) Filtered, reconstruction images of Fig.III.72d. The insets are from the identical region shown in the magnified portion of Fig.III.72d. Notice how the original 50 by 50 array of Fig.III.72d becomes larger in the filtered images.

Fig.III.74. Optical filtering demonstration, Part 3: Very 'distorted' structure. (a) 50 by 50 array of an "imperfect" crystal with some very large defects. (b) Enlarged portion of (a). (c,e,g) Filtered images of (a) obtained using the masks of Fig.III.73a,e,i respectively. (d,f,h) Enlarged views of (c), (e),and (g) from the same region as (b). Notice that the holes in the filter mask must be sufficiently small (da*/a* = 0.1) before the noise resulting from the major defects is averaged out. This also demonstrates how a periodicity can be forced on a structure by the mask.