Chapter 2: Autoindexing and Cell Refinement


Step 9: Lattice parameter refinement


Step 9: Lattice parameter refinement

Click the refine button to refine the lattice parameters. Check the image to make sure the predicted reflections sit on real spots. Make sure the direct beam position is correct and looks reasonable. Click the fit all button and refine again. Click the bravais lattice button, select the appropriate lattice for your crystal, and then click the apply & close bar.

Oops! The preds don’t line up with the spots anymore!

Occasionally the autoindexing routine does not give clear-cut results. There are four classes of data-reduction problems:


1.     Utter failure

2.     Misprediction

3.     Kind-of-failed

4.     The predicted positions agree perfectly with the peaks, but data scale poorly


1. Utter failure means that there is no reasonable predicted lattice, the crystal unit cell parameters are unreasonable, and the predicted diffraction pattern does not correspond to the observed image. In most cases failure can be ascribed to problems in three areas:

  experimental conditions   Problems with defining experimental conditions account for the vast majority of the problems with autoindexing single crystal data. If any of the following parameters is widely off, your autoindexing is also off (listed in order of importance):


1.  the x beam and y beam positions. These can be measured from the Diffraction Image using the cursor and reading off the position from the little green box in XdisplayF. A direct beam shot or a good educated guess (say based on the intersection of Laue streaks, the symmetry of the diffraction pattern, etc.) is often necessary. At synchrotrons, when the data is collected on IPs in removable cassettes and the beam wanders all over the place on the image, it is very helpful to have a beam stop which has been thinned sufficiently at its center so that a tiny amount of the direct beam passes through and makes a spot on each frame whose position can be measured. Bottom line: beam position error has to be smaller than half of the distance between diffraction spots. It is relatively easy to find beam position for test crystal with short unit cells and apply it to site.


Figure 29. The beam stop thinned in the center marks the direct beam position


2.  specification of the detector format (believe it, it's happened),


3.  the crystal-to-film distance The distance should be known accurately (within about 1%) and can be measured with a tape measure as a last resort


4.  the cassette rotx and roty values. Misset cassette rotx and roty values may be due to 2q angles other than zero.


5.  oscillation range. Very rarely autoindexing will fail because too large or too small an oscillation range was collected. In the case of large oscillations, the lunes may have severe overlap and the autoindexing program will have a problem with the non-uniqueness of the mapping from detector to reciprocal space. In the case of small oscillations or stills collected on non-mosaic crystals with a very parallel beam too few reflections may be present to describe a lune. As a rule of thumb the following types of oscillation ranges should be used: 

assuming well diffracting samples. There is a fairly wide margin on either side, at least a factor of 2.

Really, knowing the detector parameters only makes sense. If you want to deduce some unknowns (crystal lattice and cell, and orientation), you need some ‘knowns’ too.

   autoindexing parameters   Before autoindexing Denzo edits the peak-list produced by the Peak Search algorithm of the XdisplayF program. Problems with the Denzo autoindexing parameters account for a few of the remaining problems with autoindexing. These may have to be adjusted in the case of non-single crystals, very large or small unit cells, lots of noise peaks, or ice rings. These parameters are controlled by the keywords: longest vector, resolution limits, weak level, box and spot. They can be changed on Macros page or through the HKL-2000 window.

1.  The longest vector may need to be adjusted if the peaks are sparse due to a small unit cell or low mosaicity. In this case, it should be set to equal the longest vector of the reduced primitive real space unit cell. If you did not understand that last sentence, simply use the longest unit cell axis (which is the same in primitive space groups, anyway). Better to err on the side of larger rather than smaller values.

2.   resolution limits  can be adjusted, if you have:

3.  weak level If you have a number of spurious peaks in the image distributed randomly over the image, you can adjust the value of the weak level. This will affect the number of peaks accepted by the autoindexing routine. By increasing the weak level, you can exclude weak, spurious peaks, then run autoindexing, and examine the superimposition of the autoindexing predictions (green circles) on the image. If most of them cover real reflections and not noise, then you are succeeding.

4.  box and spot sizes may have to be adjusted if too many spots are being removed by the overlap or background rejection criteria. Autoindexing does not use spots which Denzo determines are overlaps.

   peak search   Finally, you can try changing the number of peaks you select in the peak picking routine. However, since Denzo edits the list of peaks anyway, this is unlikely to yield much improvement if the suggestions above do not work. In that case, do you have a better crystal?

2. Misprediction results in location of reflection masks not corresponding to the positions of the Bragg peaks. It is visibly obvious and is disastrous. It may be due to:

Misindexing Autoindexing is based on the assumption that the diffraction spots are correctly mapped from detector coordinates to diffraction (reciprocal) space. The origin of the diffraction space is defined by the position of the direct beam on the detector. A substantial error in the beam position can shift the indexing of the diffraction pattern by an integer vector. Such misindexing can be totally self-consistent until the stage when symmetry related reflections are compared. For any assumed (starting) value of the beam position, the origin of the diffraction space during indexing will be shifted to the nearest grid point of the best primitive lattice. An initial error in the direct beam position by 0.48 times the distance between reflections will lead to correct indexing, while an error of 0.52 times the same distance will cause a misindexing of the diffraction pattern by one index. Misindexing by one is never corrected by subsequent refinement of the crystal and detector parameters. Misindexing often produces poor agreement between the predicted and the observed positions of the reflections, but for some crystal orientations, the agreement between the predicted and the observed positions can be equally good for both correctly indexed and misindexed cases. This property of the diffraction geometry creates a potential trap for the unwary crystallographer.

Sometimes the crystal asymmetric unit may have molecules related by an approximate translation by a fraction (typically one half) of a unit cell edge or diagonal. The resulting diffraction pattern will have odd index reflections much weaker than even index reflections. Autoindexing may find one of the two possible solutions, the choice depending on whether odd reflections are weak enough to be assumed (within experimental error) systematically absent or not. This depends on what is the fraction of odd reflections in the peak search. If there are only a few odd reflections, then most of the peak search result may be explained with a smaller real-space unit cell. To prevent autoindexing from finding such a smaller cell, one should enhance the fraction of odd reflections used in autoindexing by changing the peak-search criteria or by using only high-resolution reflections in autoindexing. If one still cannot index odd reflections, then one should consider ignoring them all together. In such a case, one can solve the structure in a smaller unit cell and the resulting structural error will not be very significant if odd reflections are much weaker than even ones.

3. Kind-of-failed means that a lattice other than P1 was predicted, but it had a high distortion index, say 1-1.5%. This may not be a "failure" but simply an indication that further refinement of parameters is needed because the initial detector parameters in site.def were very far off. You should just run a few cycles of refinement in P1 and change Bravais Lattice. Unfortunately, misindexing may give similar results. A quick check of the superposition of the preds on the reflections should tell you if the refinement is likely to succeed. Note that refinement may converge to low χ2 values even with misindexed crystals but eventually the problem will surface and you will have to index correctly. This is now less of a problem with the 3D Window option because the requirement to integrate with consistent values over a range of frames is much more stringent a test. You are much less likely to succeed in misindexing to low χ2 values using the 3D Window.

4. Predicted positions agree perfectly with the peaks and no detector or diffraction artifacts are visible, but data scale poorly and produce many outliers. This problem may be: a simple mistake in data processing, like using the wrong file format, or wrong goniostat definitions, or a non-uniform exposure during crystal oscillation.

The non-uniform exposure may be caused by spindle motor backlash, shutter malfunction (opening too early or too late), ionization chamber electronics failure (if used), decay or variation of the X-ray beam intensity (if ionization chamber is not used), variable speed of the spindle motor etc. Non-uniform exposure is best diagnosed by exclusion of other problems that may affect data quality. Graphical feedback provides confidence that the problem cannot be at the indexing/integration stage.

Large variations in absorption of X-rays by the crystal will make data scale poorly and will produce visible variation of the background; however it will not affect positional agreement. The variation in the absorption can be avoided easily by a proper mounting of the crystal. The correction for absorption is a whole field in itself.

Errors in detector orientation will produce distorted mapping from detector to reciprocal space. A wrong crystal-to-detector distance will enlarge (or decrease) the apparent reciprocal space separation between Bragg peaks. This error will not be uniform in three directions; in the first approximation, along the beam direction, the error will be proportional to the square root of the error in the distance; in the other two directions the error will have linear dependence. In most cases a significantly wrong crystal-to-detector distance (say an error of 10%) will not make the autoindexing step fail immediately; however, the calculated unit cell will be quite wrong. The length of unit cell along the beam direction will be 5% shorter, in the perpendicular directions 10% shorter. If the crystal is diagonally oriented (no principal axis along the beam direction) than the apparent angles between axes will violate the lattice symmetry. Incorrectly defined angles between detector and the beam will result in incorrect angles between crystal axes. Therefore, the interpretation of the lattice symmetry is dependent, to some extent on how precisely the detector parameters are known a priori.

I think the direct Beam Position is wrong. How do I change it?

There are two ways to change the direct Beam Position: analytically and empirically. Analytically means that you just input the direct Beam Position as coordinates into the Site Configuration menu. Unless you have the password you will not be able to save this information to the site file, but it will be in use until you quit the program. If you do change the direct Beam Position this way, you may want to click Abort Indexing and re-run the autoindexing. HKL‑2000 also allows you to change the position of the direct beam empirically by moving a cursor around on the image and seeing how this changes the positions of the predicted reflections. To do this, click on the button that says set beam position. A dialog box will appear (Figure 30).


Figure 30. The Beam Position coordinates box and its position defined on the image by the red cross

Clicking on the big blue square will activate the cursor such that the coordinates of wherever you click in the main Image Display window will be entered into the fields for the new X and Y Beam Positions. At this time only the main Image Display window is active for this operation, not the Zoom window. These new positions do not take effect, however, until you click on the button that says apply current. Clicking here does two things. First, it enters new Beam Position coordinates into the refinement, and second, it gives a “go” command, which starts the refinement, equivalent to clicking the refine button. This allows you to see the effect of the new Beam Position on the predicted reflections right away. You can also enter these new values into the Site Configuration file by clicking the update site button.

What should I set at the 3D window?

A good rule of thumb is to set the 3D Window (after the resolution limits and lattice have been determined) to be at least twice the refined mosaicity. The default value of 5 is a reasonable compromise. Since all of the frames in the 3D Window are held in memory during the refinement, this may also restrict the number of frames used in the 3D Window on older computer systems. For small molecule structure determinations, where the reciprocal lattice is sparsely populated, a 3D Window of 30 frames is a good choice (Figure 31). The definition of the 3D window is in Step 12.


Figure 31. The 3D window in Controls box

What’s the procedure for choosing the correct lattice?

The lattice is best chosen after the first frames of the 3D Window have been refined in space group P1, but before the whole set of frames is integrated. Since primitive triclinic is the lowest possible symmetry for your crystal, if you can’t succeed in refining here choosing a higher symmetry lattice, which has fewer refineable parameters, will be even worse.

Clicking on the refine button starts the refinement of the autoindex lattice. Make sure that you are under the fit all regime – you do not want to be fixing refinement parameters. You should notice the number of spots used in the refinement go way up, and the χ2 X and Y values approach 1.0. Before proceeding further, go back to your diffraction Image Display window and examine the positions of the yellow or green circles with respect to the diffraction spots. If your indexing has worked, the predicted reflections (circles) should match up with the diffraction pattern.

If the preds do not line up with the diffraction spots, two key values to check are the Distance and the X and Y Beam Positions. THESE MUST BE CORRECT. If the preds do line up with the spots you have probably misindexed and you should examine experiment set-up parameters. Note that some instruments do not produce proper information in the image header and you have manually input some parameters.

 If the χ2 values are high (orange or red colored), things to examine are:

1.  The Spot Size. Too large a Spot Size will result in larger χ2 values. There will be more discussion of Spot Size later in this tutorial.  

2. The second thing to suspect is an error with the Site Configuration - for example, the spindle rotation direction, or even orientation, may be specified incorrectly – or the Frame Width. An easy way to test for this is to close down the 3D Window from 5 frames (the default for CCD detectors) to 1 frame, hit refine, and note the χ2 values again. If they improve (go “green”) this tells you that there is a problem with specifying the relationship between successive oscillation frames.

a. The most likely candidate is an incorrect Frame Width (perhaps the header was written incorrectly). This can be changed in the Frame Geometry panel of the Main page.

b. A second candidate, especially with data collected from customized goniostats, is that the spindle rotation direction is incorrectly specified. To change the goniostat misalignment parameters, go to the Site Configuration box and change the sign of the motor axis value. For example, if the axis is specified as 0 1 0, you could try changing it to 0 –1 0. Changing the order of the numbers, e.g. to 1 0 0 means that you are specifying a different rotation axis orientation, a severe error.

As in all things, the proof is in the pudding. If the χ2 values improve (after aborting the refinement and re-indexing to get more accurate values of the Distance and the crystal rotations), then continue on.

Assuming that the predicted reflections still match up with the diffraction pattern and the χ2 values are not red colored (i.e. above 5), now is the time to select the proper Bravais lattice. So click on the bravais lattice button and examine the table (Figure 32). If you know (really know, not kind-of know) the space group ahead of time, you can select the correct lattice. If not, select the lattice that has the highest symmetry and the lowest distortion index. Sometimes these are the same.

If a lower symmetry lattice (e.g. primitive monoclinic at 0.13%) has a lower distortion index than a higher symmetry lattice (e.g. primitive tetragonal at 0.57%) then sometimes this may be the better choice, even if the higher symmetry lattice (e.g. tetragonal) is colored green. Why is this? Thanks to the use of the 3D Window in the refinement process, the indexing now uses many more reflections and encompasses a thicker slice of diffraction space than the previous incarnation of autoindexing and refinement in Denzo. As a result, the distortion indices are much more accurate, and may reflect real deviations from crystallographic symmetry, rather than just an incomplete refinement of crystal parameters.

What’s the best way to proceed? A systematic way to choose the correct lattice is to march your way up the list of candidate lattices (candidate lattices being the green ones) and look for a significant change in the distortion index. For example let’s say you think you have a tetragonal crystal, and the candidate lattices are P1, C-centered monoclinic, primitive monoclinic, C-centered orthorhombic, primitive orthorhombic, and primitive tetragonal. After P1, you would select primitive monoclinic, refine, and examine the Bravais Lattice table again. The distortion index for monoclinic (and all lattices of lower symmetry – take note!) should be zero. Are the distortion indices for the higher symmetry lattices still close? If there is a significant jump (say > 0.2%) to go to a higher symmetry lattice, then perhaps those lattices are not correct and you should stick with the lower symmetry lattice.

Keep in mind that a crystal misindexed to a lower symmetry lattice will merely be characterized by a high degree of “non-crystallographic” symmetry and can reliably be solved and refined as long as complete data was collected, whereas a crystal misindexed to a higher symmetry lattice will cause you no end of grief as you try to force idealized symmetry onto a system that lacks it. That’s why it is important to accurately determine the lattice of your crystal early on in a data collection session, so that you will not be surprised and shocked when your best so-called “tetragonal” crystal turns out to be monoclinic and you’ve only collected half a data set and the crystal is lying in a puddle on the floor of the synchrotron hutch.

Once you have selected the lattice, by clicking on the little button by the one you chose, click the apply bar to input the new lattice.


Figure 32. The Bravais Lattice table

What if no lattice other than primitive triclinic comes up green?

This could mean that indeed you have a triclinic lattice and that you will have to go “around the world” to collect a complete data set.

Alternatively, you may be misindexed (see the discussion about misindexing above). If you are not able to see a particular pathology that would contribute to misindexing (e.g. incorrect direct beam position or incorrectly specified goniostat or detector parameters) then the best thing to do is to continue along the tutorial and integrate the frames. If the frames process well in P1 and the Bravais Lattice table continues to show high distortion indices for lattices other than primitive triclinic, then that is your lattice. If the frames do not process well in P1, then there is indeed something wrong with your indexing or with your data.

Summary: If P1 indexes and integrates well: lattice could be triclinic. If P1 does not index and integrate well: data or processing is bad and you cannot rule out other lattices yet.

What are the Positional and Partiality χ2 values and what should they be? What are the numbers in parentheses?

The χ2 values represent the average ratio, squared, of the error in the fitting divided by the expected error. In other words, it represents how close your observed errors are to those you predicted you would observe. χ2 analysis is the foundation of Bayesian statistics. The numbers in parentheses are the number of overlaps for each class of reflections.

Figure 33. The refinement information e.g. positional and partiality χ2

A good refinement will have χ2 values near 1.0. In the early stages of refinement these values can be high, but towards the end they should be below 2 or so. All other things being equal, χ2 values for x and y positions are sensitive to the Spot Size a larger Spot Size will result in larger χ2 values. It makes sense when you think about it: choosing a very large spot allows you considerable latitude when choosing the center of the reflection, since small changes in position won’t appear to have much effect on the integrated spot, and thus you will not have “pinned down” the center of the spot very well. A very small spot size, on the other hand, will be quite sensitive to the center of the reflection, since small changes here would have you integrating the sides of the reflection and not the center, and so you will detect that one place is good to be and another is worse.

The magnitude of the χ2 values is not a critical test for the success of Denzo, since they represent only the comparison of the spatial differences between the observed and predicted reflections to an error model that is typically quite strict in the prediction of small errors. Assuming default values of the error estimates, χ2 values of 2 or even 3 are acceptable, because the position of the predicted reflection, and hence the intensity, is still very accurate. Thus, the χ2 values may not reflect errors in the integration of the reflections.

*  Note that you can “improve” your χ2 values (get them closer to 1) by changing the error model. The most likely candidate for change is the value for “error positional.” This value represents the error, in mm, in the positional measurement of the spot center. The default values of Error Positional are set in the program for “typical” crystal for particular detector.

This can be changed by going to the Macros tab and entering the keyword “error positional [value]” under the field Immediate Execution, where [value] is replaced by your new error, e.g. 0.03 (Figure 34).

Figure 34. The Macros box

You should be cautious about increasing this value, especially for a well-tested and characterized detector, because the high χ2 values may reflect underlying problems with your crystal, rather than an actual error with the detector. For example, your crystal may be slightly cracked, with incomplete spot splitting, and the spot shape will be enlarged and distorted, reflecting the contribution from the multiple lattices. This will lead to ambiguities in determining the centers of the diffraction spots, which will be reflected in larger χ2 values.

It is a good idea to keep your error model roughly the same from crystal to crystal so that you can use the χ2’s to pick up any other variations in your data collection. For example if you are getting χ2’s in the range of 2 to 5 consistently and your crystal is good you have probably problems with detector calibration. Note that most people think that their crystals, like their children, are outstanding. At least for crystals, this is not always the case.

If the χ2 values are very high, (> 10 or so for default values of the error parameters) something is seriously wrong with the indexing, refinement, or the detector. This will be apparent from visual inspection of the displayed image, in any case.

Since the goal of Denzo is to produce a list of hkl’s and unscaled intensities, it does not matter that the errors in the positions differ a bit from the standard expected error. On the other hand, χ2 values substantially different from 1 in scaling should be investigated, because this directly affects a critical result of the scaling procedure, namely the s value assigned to each scaled intensity. So that's where you should pay attention to finer deviations of χ2 values from 1.

What are all those the Refinement Information values for, and what do they mean?

In fit all regime you can successfully refine all the parameters – crystal rotations, detector rotations, detector distance, unit cell dimensions, etc. – simultaneously, if their initial values are close enough. The fit basic options select the most critical values. These are usually the crystal rotations, unit cell parameters, and the beam position. The resolution limits must be set first (discussion in Step 8).

 If you are under the fit basic regime, then you have refined the parameters listed in the information panel (Figure 33):

 If you are under fit all regimes there are listed values for additionally refined parameters:

The detector and crystal parameters are refined by a least squares method that minimizes the deviation of the reflection centroids from their predicted positions.

Someone told me to gradually increase the resolution and alternate cycles of fitting a few parameters and fixing things. I’m patient, and I want to be careful. Why shouldn’t I do this?

This is the wrong way to go about indexing and integrating your images. For example, back in the olden days, people would get an initial index at low resolution, fixing all parameters except for, say, the crystal rotations. Then they would fix the rotations and refine the distance. Then they would fix the distance and refine the unit cell parameters. Then maybe they would increase the resolution a bit and repeat the process. The problem with this approach is that single parameter refinement may not allow correlated parameters to be sufficiently refined. In other words, you could get trapped into overfitting one variable and underfitting another, resulting in both variables being incorrectly determined. So unless you know with near-absolute certainty that a variable is correct, cycles of fixing and fitting is the wrong way to go.

People used to take these kinds of approaches because refinements were unstable and simultaneous fitting of multiple correlated variables led to instability in the refinement. High correlation makes it possible for the errors in one parameter to compensate partially for the errors in other parameters. If the compensation is 100%, the parameter would be undefined, but the error compensation by other parameters would make the predicted pattern correct.

HKL-2000 overcomes this problem in two main ways. First, eigenvalue filtering (described in Numerical Recipes as Singular Value Decomposition) is employed to remove the most correlated components from the refinement and make it numerically stable. Eigenvalue filtering works reliably when the starting parameters are close to the correct values but it may fail to correct large errors in the input parameters. Therefore, once the whole data set is integrated, the global refinement (sometimes called postrefinement) can refine crystal parameters more precisely. The unit cell used in further calculations should come from the global refinement (in Scalepack) and not from the Denzo refinement.

Second, refinements are generally carried out at the maximum resolution of the diffraction data, and, with the 3D Window, over multiple successive frames. More data allows more parameters to be fit simultaneously, and increasing the dimensionality (by adding a third dimension) further constrains variables that might be poorly fit in only 2 dimensions (like the distance, or the third unit cell dimension). These advances are made possible by improvements in the program as well as improvements in computer speed, data storage, and memory.



Displaying your image


Table of Contents


New lattice refinement