X4D Overview: Difference between revisions
Line 369: | Line 369: | ||
[[Image:X4DObjectTrackingWidget.png|Object Tracking Widget]] | [[Image:X4DObjectTrackingWidget.png|Object Tracking Widget]] | ||
{| class="wikitable" width="100%" | |||
! style="text-align:left;" | Controls | |||
{| class=" | |||
! style="text-align:left;" | | |||
|- | |- | ||
| | | | ||
<ul> | <ul> | ||
<li> | <li> | ||
Pixel Difference | '''Optimize / Move (Keyboard)''': These check boxes let you specify which objects are allowed to move during the tracking optimization and moved with keyboard commands. DRRs for all of the objects selected in the ''Object Configuration'' widget will be generated, but only the objects selected here will be allowed to move. This allows you to perform hierarchical tracking, which can be useful when bones overlap in the X-ray images. For example, you can track the femur, then fix it in its optimized pose while tracking the patella. | ||
</li> | |||
<li> | |||
'''Max Iterations''': The maximum number of iterations for one pass of the optimization algorithm. For single-frame optimization, this is the number of iterations for one frame (reporting time). For 4D optimization, this is the number of iterations for the first pass. For the second pass of 4D, half of this number is used as the maximum. | |||
</li> | |||
<li> | |||
'''Translation Ranges''': There is a slider for each of the X, Y, and Z translation ranges. Each one controls the amount that the DOF is allowed to change from its initial value during optimization. X, Y, and Z are the axes of the X-ray lab reference frame (defined by the calibration object). | |||
</li> | |||
<li> | |||
'''Rotation Ranges''': There is a slider for each of the X, Y, and Z rotation ranges. Each one controls the amount that the DOF is allowed to change from its initial value during optimization. X, Y, and Z are the axes of the X-ray lab reference frame (defined by the calibration object). | |||
</li> | |||
<li> | |||
'''Tracking''': The range of reporting times which will be optimized when using the ''Solve Range'', ''Solve Poses in Range'', or ''Solve Range 4D'' commands. | |||
</li> | |||
<li> | |||
'''Algorithm''': The type of optimization algorithm to use. ASA (adaptive simulated annealing) is currently the only recommended algorithm. SPAN is another type of simulated annealing. LBFGSB is a bounded, least-squares algorithm (local optimizer). | |||
</li> | |||
<li> | |||
'''Image Metric''': The type of image metric [[#Optimization Metrics|algorithm]] to use when comparing the processed X-ray images to the processed DRR images. Currently there are two algorithms implemented: ''Pixel Difference'' and ''Conditional Entropy.'' | |||
</li> | </li> | ||
<li> | <li> | ||
'''Solve Range''': Starts the frame-by-frame optimization process. The selected objects’ poses are optimized at each reporting time in the tracking range. It starts at the current time and proceeds towards the earliest, then goes back to the current time and proceeds towards the latest. Although each frame (reporting time) is optimized independently of the others, previously optimized frames will modify the pose map, which could affect the starting poses for subsequent frames. | |||
</li> | </li> | ||
</ | <li> | ||
< | '''Solve Current Frame''': Performs an optimization of the current frame. | ||
</li> | |||
<li> | |||
'''Solve Poses in Range''': Performs an optimization of each reporting time in the tracking range that already has a pose specified in the pose map. | |||
</li> | |||
: | <li> | ||
'''Solve Range 4D''': Starts the 4D optimization process. The entire tracking range is optimized at the same time. | |||
</li> | |||
<li> | |||
'''Stop''': Stops the currently running optimization. | |||
</li> | |||
<li> | |||
'''Evaluate''': Evaluates the poses of the currently selected bones using the selected image metric. The results are written to the output window. | |||
</li> | |||
<li> | |||
'''Poses Saved During Tracking''': Sometimes the global minimum of the image fitness function is not the correct pose of the bone being tracked. This is often because the X-ray and DRR image processing parameters are not ideal (see ''Image Optimization'', below). Other times, it is because the bone in the X-ray images is occluded by soft tissue or other bones, or because of inherent differences between X-ray images and DRRs (image resolution, X-ray scatter, CT thresholding, etc.). To account for these other times, the ASA tracking algorithm attempts to save bone poses that result in local minima of the cost function, while still searching for the global minimum. The idea is that if the correct bone pose is not the global minimum, it is likely to be a local minimum somewhere within the search range. The ASA algorithm thus attempts to save a set of local-minimum poses so that you can revisit them once optimization has finished. The ''Max Poses'' field specifies how many local minima to save. When the tracking optimization has finished, if the final bone pose is not the correct pose, you can re-apply the saved poses by dragging the slider for that bone. If one of them is correct (or at least better than the final pose), you can use the ''Add Current Poses to Maps'' command in the ''Pose Map'' widget to add that pose to the map, thus overwriting the final pose. | |||
:Note: When the ASA algorithm evaluates a particular bone pose during optimization, it cannot determine if it is actually a local minimum or not because it does not compute gradients. It estimates if the pose is a local minimum by examining its cost function value and its proximity to other poses with similar values. Here is an example that details the process. If you perform an ASA optimization of 1000 iterations, with ''Max Poses'' set to 20 and ''Distance (squared)'' set to 3.0, when the optimization is complete the 1000 bone poses that were evaluated are sorted from lowest to highest cost function value. The pose with the lowest value is marked as the global minimum and added to the list of saved poses. Then the pose with the next lowest value is checked to see how close it is to the first pose. The sum of the squares of the differences in the 6 DOF values is computed and compared to the ''Distance (squared)'' value (3.0 in this case). If the distance is less than 3.0, it is assumed that this pose is in the same valley of the solution space as the first pose, so it should not be saved as a local minimum. If it is greater than 3.0, the pose is added to the saved pose list. Then the next pose in the sorted list is checked. If it is not within 3.0 units of any pose already in the saved list, it is added to the saved list. This process proceeds until the saved list contains 20 poses. It will take some trial and error to determine the best value of the ''Distance (squared)'' parameter for a particular data set, but a value in the range of 1.0 to 3.0 is reasonable, as the solution spaces are often very bumpy with numerous local minima. The units for the DOF translations are model units (usually mm), and the units for the rotations are degrees. | |||
</li> | |||
: | |||
|} | |} | ||
Revision as of 17:46, 23 June 2020
Language: | English • français • italiano • português • español |
---|
X4D tracks 3D objects (bones, implants, etc.) in X-ray images by generating digitally reconstructed radiographs (DRRs) of the objects and matching them to the X-ray images. It requires that the 3D X-ray configuration parameters be defined in the subject file and that the X-ray images be corrected.
To match DRR images to X-ray images, both sets of images are processed first. X-ray images are smoothed with a convolution filter, and then an edge detection algorithm is performed on them. The final processed X-ray image that is used for tracking is the edge detection image times a weighting factor, plus the smoothed intensity image. DRR images are not smoothed, but are input to the edge detection algorithm. The final processed DRR image is the edge detection image time a weighting factor, plus the original DRR image.
Tutorials
How To: Get Started Tracking Bones
How To: Optimize Xray/DRR Settings
How To: Calculate Bone Poses from Mocap Data
Bone Tracking
4D Optimization |
---|
4D optimization tracks one or more objects by solving all frames in the tracking range at the same time. It works using a simulated annealing optimization process over two passes. In the first pass, a temporary pose map is created by evaluating the object’s pose map over the tracking range at equal time stamps. The object poses at these times are the control points that the optimizer varies to find the global solution. The spacing between the time stamps is determined by the Frame Spacing (Pass 1) parameter. For example, if the parameter is set to 4 and there are 20 reporting times in the tracking range, there will be 20 / 4 + 1 = 6 poses in the pose map-- a pose at each end of the range and 4 poses equally spaced between them. The algorithm then proceeds to optimize each control point independently of the others, while evaluating the DRR/X-ray matching for the X-ray frames in the neighborhood of the control point. The size of this neighborhood is determined by the Frame Envelope (Pass 1) parameter. For example, if the parameter is set to 2, each time the optimizer changes the pose at one of the time stamps, it will evaluate the DRR/X-ray matching at the closest X-ray frame to that time (in each view), and in the 2 frames on either side of that closest frame. In the second pass, a temporary pose map is created by evaluating the optimized pose map from the first pass at equal time stamps. This process is the same as described above, but uses the Frame Spacing (Pass 2) parameter to determine the spacing. The algorithm then optimizes each control point independently, evaluating a neighborhood of frames determined by the Frame Envelope (Pass 2) parameter. For this second pass, the number of simulated annealing iterations per control point is halved, the translation and rotation ranges are divided by four, and the initial temperature is divided by four. Once the second pass is complete, its final pose map is evaluated at the reporting times in the tracking range and those poses are inserted into the object’s pose map. |
Matching Xray and DRR Images |
---|
The process of matching the DRR images to the X-ray images is sensitive to the image processing parameters in the X-ray/DRR Settings widget. Once the DRR images have been generated for a particular set of bone poses the DRR images and the X-ray images are processed (using an identical method) and then compared to each other. The algorithm used to compare them, called the image metric, can be selected in the Object Tracking widget. The processing method consists of:
Pixels in the edge image whose values are greater than the Edge Capping maximum are set to zero. Very bright pixels (sharp edges) are usually inorganic objects like EMG electrodes or metal plates or wires. They can be removed from the edge image by lowering the Edge Capping maximum from 100. Every pixel in the edge image whose value is above the Edge Capping minimum is set to the Edge Capping minimum. This effectively strengthens weaker edges (those below the Edge Capping minimum) because the entire image is scaled later. For example, if the Edge Capping values are set to 5 and 30, every pixel in the edge image that is greater than 30 is set to 0. Every pixel that is between 5 and 30 is set to 5. The resulting edge image has all pixels between 0 and 5, with many of the edges of interest set to 5, making them all equally strong. After the edge image has been weighted and added to the original image, the result is thresholded using the Image Threshold values. All pixels above the Image Threshold maximum are set to the maximum, and all pixels below the Image Threshold minimum are set to 0. Much of the time these thresholds should be left at 100 and 0. However, there are times when it is useful to raise the minimum above zero to mask soft tissue regions, and lower the maximum from 100 to remove artificial edges, such as the end of a CT bone that is within the X-ray image. The success of the bone tracking algorithm is particularly sensitive to the Edge Capping minimum for the X-ray images. A change in this value of 2 or 3 (percent) can make a big difference in the solution found by the optimization. If you find that the optimization is locking into a bad pose with the edge capping minimum set to 10, try lowering the minimum to 7 or 8 (or even 5) and trying again. When you start processing a new data set, it can take some trial and error to figure out the set of image processing parameters that produce the best results. It is recommended that you start by tracking the bones (individually) in a single frame, trying different parameter values until the optimization algorithm can lock onto the correct pose for each bone. Then try solving a few adjacent frames to see if the same parameter values work on those. |
Motion Capture Data
Working with Motion Capture Data |
---|
If you have surface-marker-based motion capture data for an x-ray trial, you can use it to calculate initial poses for the bones you want to track in the trial. To use this data in X4D, you must have C3D files with labelled marker trajectories for the motion trial and its associated static trial, and Visual3D files defining a kinematic model and its motion constraints. In the DSX software, this model is referred to as a mocap model. Creating the model template and the constraint scripts requires a good working knowledge of Visual3D. The model template file (MDH) defines the segments and landmarks in the kinematic model, based on a specific marker set. The build script (V3S) creates the constraints that define the relative motions of the segments in the model. The recalc script (V3S) updates the segment poses to satisfy the constraints. Once the model is defined, and applied to a particular DSX trial, it can be used in three modes:
In addition to generating initial poses for the tracking objects, you can use the mocap model to help with manual positioning of the objects. In the Mocap Model widget in X4D, you can turn on the model's joint constraints, making it easier to move all of the objects together as a group, or to move them relative to each other in a realistic manner (e.g., flexing or twisting the vertebrae in a spine model). |
Optimization Metrics
The algorithm used to compare the X-ray and DRR images during object tracking.
Conditional Entropy | ||
---|---|---|
|
Menus
File Menu |
---|
![]()
|
View Menu |
---|
![]()
|
Options Menu |
---|
![]()
|
Help Menu |
---|
![]()
|
Widgets
DSX Configuration |
---|
![]()
|
X-ray and DRR Settings | ||
---|---|---|
![]() The matching of DRR images to X-ray images is sensitive to the image processing parameters in this widget. Once the DRR images have been generated for a particular set of bone poses, the DRR images and the X-ray images are processed (using an identical method), and then compared to each other using the selected image metric algorithm.
|
Pose Maps |
---|
A pose map is the set of poses for a tracked object. Each pose expresses the transform from the X-ray (lab) frame to the object’s anatomical frame (i.e., as if you were traveling from the lab frame to the object frame), and is at one of the trial’s reporting times. Each of the 6 DOFs in a pose map is interpolated (and possibly smoothed) with a GCV (generalized cross-validation) spline. Smoothing of pose map splines is controlled by the Pose Map Cutoff Frequency parameter. Pose maps are stored in CSV files. ![]() The pose map commands described below operate on the objects specified by the Objects Affected combobox at the bottom of the widget.
|
Object Tracking | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|