Locate3D Overview

Locate3D tracks radiopaque beads in X-ray trials. It is most often used to track objects (e.g., bones and implants) with implanted beads. If the object contains three or more beads in fixed locations, and these 3D locations are known relative to the object's CT coordinate system, then the beads' 2D locations in the X-ray images can be used to calculate the object's 3D pose in the X-ray lab frame. Locate3D is thus similar to X4D, except that it uses multiple beads to determine the pose of an object, instead of the contours of the object itself. Locate3D requires that the configuration of the X-ray equipment has been calculated (with CalibrateDSX), and that the locations of the beads in the object's local frame have been determined (with Surface3D).

Digitizing POIs

To digitize a POI for the current reporting time, select the POI by clicking on its name in the table in the Points of Interest widget. The table row will be highlighted in blue, and when you move the cursor into one of the X-ray windows, the cursor will change to a cyan circle inside a yellow square, with the POI name to the right. The circle represents the expected size of the POI on the X-ray image, which is dependent on the pixel size of the image and the radius of the POI, which is specified in the subject file. The center of the circle is also marked by a cyan point. The square represents the region of the image that will be searched for the center of the POI (some of the outer pixels of the POI can be outside the square). The size of the square is controlled by the Click Search Factor parameter. A larger number makes it quicker and easier to digitize POIs, but you don't want the square to contain more than one POI. When you click the left mouse button, Locate3D will search the X-ray pixels within the square to determine which one is the best candidate for the center of the POI. This is done by using the Centroid Threshold parameter to separate the POI pixels from the background, and the Edge Threshold parameter to determine if these pixels represent a circular POI. The POI is assumed to be a bright circle on a dark background, with the center of the POI being the brightest. The 2D coordinates of the POI center in each view are calculated as the weighted center of the pixels above the Centroid Threshold. Once you digitize the POI in one view (you can start with either view), a yellow line is drawn in the other view. This line is the projection of the point in the first view through the field of view of the second view, and its purpose is to help you identify the POI in the other view. You can then digitize the POI in the second view by clicking on a point near the line. If you make a mistake identifying the proper POI in the first view, you can press the Esc key to cancel the process. If you make a mistake after identifying the POI in the second view, you can remove the POI pose by unchecking the pose box in the POI table. Once you have digitized a POI, Locate3D automatically advances the current POI to the next one in the table that does not already have a pose (unless the Auto-advance box is not checked). Once you have identified the POIs for one or more reporting times, you can have Locate3D attempt to automatically identify them for other reporting times using the Propagate POIs command in the Tools menu. See the wiki entry for this command for more details. Note: If you hold down the Alt key while the POI cursor is displayed over an X-ray image, the cyan point inside the circle will display the location that the centroid-finding algorithm would calculate if you left-clicked with the cursor in its current position. This point will update in real-time as you move the cursor. If the point is not visible inside the yellow square, it means either there is no suitable location of a POI centroid, or that region of the image has been masked by a previously digitized POI. Note: To override the centroid-finding algorithm and digitize a POI at exactly the XY coordinates you click on, hold down the Ctrl key when clicking. While Ctrl is pressed, the yellow square will not be displayed, indicating that the centroid-finding algorithm has been deactivated. This feature can be helpful when digitizing non-circular or overlapping POIs. Note: When digitizing a POI, all previously digitized POIs on the image are masked out, so that no two POIs can be located at the same point. This is done by using the estimated size of each POI on the image to ignore the circular region of pixels around the POI location. This can be especially helpful when digitizing two adjacent POIs in a sequence of frames. If you digitize one of the POIs in all frames first, then propagating the second POI through the sequence is easier because it can't erroneously "lock onto" the first POI's location.

Menus

File Menu

Open Subject... (Ctrl + O) loads a subject file. Save Subject (Ctrl + S) saves the currently loaded subject to its existing subject file. Save Subject As... saves the currently loaded subject to a new subject file. Save POI Locations... saves three text files containing the coordinates of the current object’s POIs. The files are as follows: one with the 3D coordinates in the X-ray reference frame of all POIs for all frames, one for the first view with the 2D POI coordinates in the image frame of the view, and one for the second view with the 2D POI coordinates in the image frame of the view. If a name for any of these files is already specified in the subject file, it is used to write the data to. If the file already exists, it is overwritten without notification. For any that is not specified, a file browser is displayed, allowing you to select the file and folder. Save Pose Maps saves the pose map of each selected object in the current trial to a .csv file. This file contains 6 values for each pose: the XYZ translation and the XYZ Euler angles representing the transform from the X-ray lab frame to the object's local coordinate system (i.e., as if you were traveling from the lab frame to the object frame). If a name for the pose map file is not already specified in the subject file, you will be prompted to browse for the file to create. A link to this file will then be added to the subject file. Before the poses can be saved, they must be calculated using the Compute Object Poses from POIs command. Save Pose Maps As... saves the pose map for each selected object in the current trial to a new .csv file. You will be prompted to browse for a new file for each object. A link to this new file will then be added to the Subject file. Save Object Transforms... saves the pose map of each selected object in the current trial to a .txt file. This file contains a 4x4 transform for each pose, expressing the transform from the X-ray lab frame to the object's local coordinate system (i.e., as if you were traveling from the lab frame to the object frame). If a name for the transforms file is not already specified in the subject file, you will be prompted to browse for the file to create. A link to this file will then be added to the subject file. Because these transforms are calculated from the object poses, the Compute Object Poses from POIs command must be used before the transforms can be saved. Save Landmark Locations... saves a text file with the 3D coordinates of all landmarks on the current object for all frames that have poses. These locations are calculated using the object poses, so the Compute Object Poses from POIs command must be used before the landmarks can be saved. If a name for the landmarks file is not already specified in the subject file, you will be prompted to browse for the file to create. A link to this file will then be added to the subject file. Exit exits the program. The program can also be closed using the X in the top right corner of the program window.

View Menu

Show All POIs (Ctrl + A) toggles the display of all of the POIs for all objects in the current trial. Show Body Reference Frame (Alt+ R) toggles the display of the reference frame for the current object, as long as a local frame is defined in the subject file and the pose for the current frame has been calculated. Show Landmarks (Alt+ L) toggles the display of all of the landmarks for the current object. DSX Configuration (Ctrl + Shift + D) toggles the display of the dockable widget that shows the current session’s objects. Points of Interest (Ctrl + Shift + I) toggles the display of the dockable widget that shows the details of the points of interest. POI Parameters (Ctrl + Shift + P) toggles the display of the dockable POI Parameters widget. POI Graphs (Ctrl + Shift + G) toggles the display of the dockable widget that shows the XYZ coordinate graphs of the currently selected POIs.

Tools Menu

Propagate POIs (Ctrl + P) This command propagates all POIs in the current object through all of the frames selected in the Propagate Range slider. For each POI, Locate3D fits a GCV spline to the poses in that POI's pose map (as shown in the POI Graphs widget). For each reporting time in the range, if the POI does not already have a pose at that time, Locate3D interpolates the X, Y, and Z splines to calculate the expected location of the POI in each X-ray view. A region of the X-ray image around this point is then searched for a suitable location of the POI center. The size of this region is determined by the Propagation Search Factor parameter. If the POI moves a lot from frame to frame and this movement is not adequately predicted by the splines, you make need to increase the Propagation Search Factor. However, if this parameter is too large, or if several POIs are close to each other, Locate3D may misidentify POIs during propagation. Compute Object Poses from POIs (Ctrl + C) This command calculates the poses of the current object in all frames, using the POI locations. For each frame, if three or more POIs have poses, Locate3D will calculate the object pose using a least-squares optimization (fitting the POIs defined in the 3D voxel data to the POIs defined in the X-ray coordinate system). Recalculate 3D POIs (Ctrl + R) This command is enabled only for asynchronous trials. When the first pose for a POI is digitized, there is no valid pose map yet to extrapolate from the two frame times to the reporting time (the pose map splines are straight lines with slope=0, passing through the point being digitized). Thus, the 3D coordinates of the POI at the reporting time could have large errors. There is no way to account for this while digitizing the first POI because the shape of the pose map is not known yet. The work-around is to provide a function so the user can recalculate the 3D coordinates after the pose map contains more poses.

Options Menu
Settings opens the Settings dialog

Help Menu
Help (F1) opens the Locate3D wiki page in the default browser. About... displays a dialog with information about Locate3D, including the installed version number. It also contains the Deactivate button, for deactivating your DSX suite license.

Widgets

DSX Configuration

Subject This field is automatically populated with the ID of the subject when you load a subject file. Session This drop down menu is automatically populated with all of the sessions available for the current subject. Configuration This drop down menu is automatically populated with all of the configurations available for the current session. Trial This drop down menu is automatically populated with all of the trials available for the current configuration. Object This list is automatically populated with all of the objects in the current trial. When an object is selected, its POIs are displayed in the Points of Interest widget.

Points of Interest

Points of interest (POIs) are features of an object that you want to track in X-ray motion trials. They are most commonly used for radiopaque beads implanted in an object whose motion you want to measure. They are distinguished from landmarks, which are object features whose locations you want to calculate once the object pose is known. A common type of landmark is ligament attachment sites. Thus, POIs are used to determine the pose of an object and landmarks are used to calculate locations dependent on the pose. The recommended procedure for adding POIs to an object is to use xManager to create them and specify their names, radii, and colors (for display in the GUI). Then use Surface3D to specify their XYZ locations in the image data (usually CT). It is important that each POI have a correctly specified radius so that Locate3D's circle-finding algorithm can estimate its size in the X-ray images.      Propagate Range The Propagate Range slider lets you specify the range of frames over which the Propagate POIs command (in the Tools menu) operates. Once you have digitized a POI in one or more frames of a trial, the Propagate POIs command uses the spline through the poses (shown in the graphs) to attempt to digitize the POI in all frames in this range that do not already have a pose.      POIs There are six columns in the POI table: Name the name of the POI, which is used to identify the POI in the X-ray windows and in all 2D and 3D coordinate files. It can be changed by double clicking on the cell and typing in a new name. Pose a checkbox which is checked when the POI's 3D location is defined for the current frame. Deselecting it deletes its location for the current frame. Visibility toggles the visibility of the POI in the X-ray windows. It does not affect the presence or absence of locations in the pose map. Graph toggles the visibility of the POI in the POI Graphs Widget. Trail toggles the visibility of the POI's entire pose map in the X-ray windows. Color controls the color of the POI in both the X-ray windows and the POI Graphs Widget. Double clicking on the color cell opens a Select Color dialogue which allows you to change the color.

POI Parameters

This widget contains the parameters that control the digitization of POIs in the X-ray images. When you are digitizing POIs in the images, there are two methods of determining the XY coordinates in the 2D images, which are then used to calculate the 3D position in the X-ray lab frame. The recommended method is to left-click on any portion of the POI in the image and let Locate3D calculate the weighted center of pixels representing the POI. This process is described in detail in Digitizing POIs, and is governed by the parameters listed below. For cases in which this process does not correctly locate the POI center (e.g., non-circular POIs), you can use Ctrl-left-click to specify the point. In this case Locate3D will use the exact XY coordinates of where you clicked as the 2D location of the POI in that view. POI Scale used to calculate the expected radius, in pixels, of the POI in the X-ray images. The expected radius is the physical size of the POI multiplied by the scale factor, divided by the pixel size. The scale factor should increase with the distance between the POI and the image plane. Click Search Factor used when manually digitizing POIs in the X-ray images. If you click at point X,Y in an image, a region of the image around this point is searched for the best location of a grayscale "blob". This region is a square centered at X,Y with sides equal to two times the expected radius of the POI times the Click Search Factor. This factor should be set small enough so that the search region includes only the POI you are currently digitizing. Propagation Search Factor used when propagating POIs in the X-ray images. In order to propagate a POI with coordinates X,Y in one frame to the next frame, a region of the next frame is searched for the best location of a grayscale "blob". This region is a square centered at X,Y with sides equal to two times the expected radius of the POI times the propagation search factor. This factor should be set large enough so that the search region captures the movement of the POI from one frame to the next, but small enough so that it does not include other POIs. Centroid Threshold the relative threshold for identifying pixels that belong to the POI. Once a grayscale region of an X-ray image has been identified as a POI, its centroid is calculated by a weighted average of all the pixels that have an intensity value greater than this percentage (0.0 to 1.0) of the maximum intensity. Edge Threshold the relative threshold for identifying suitable POIs. For a grayscale region in an X-ray image to be considered a POI, the average edge value within the region must be at least this percentage (0.0 to 1.0) of the maximum edge value. Pose Map Cutoff Frequency the low-pass cut-off frequency for smoothing the cubic splines in the POI pose maps. Smoothing can be turned off by using a value of 0.0 or less.

POI Graphs

The POI Graphs widget displays graphs of the X, Y, and Z coordinates of the selected POIs for the current trial. As the cursor is moved around the graph a crosshair shows the current coordinates. If the cursor is close enough to one of the data points for a POI it will snap to it and display that point's coordinates in the POI's color. To move the view of a graph in the Y direction, press the left mouse button and drag the cursor up or down. The mouse wheel can also be used to zoom the graph in the Y direction. Press the R key to reset the graph to the default view. The X axis of the graphs cannot be changed. Data points cannot be added or moved via the graphs, but they can be deleted. To select one or more points, press the Ctrl key and then box-select them (hold the left mouse button down while sweeping the diagonal of a box with the cursor). Press the Delete key to delete the points. Press the Esc key to deselect them.

X-ray Windows

The X-ray windows display the X-rays of each view, as well as POIs, POI trails, reference frames, and landmarks. The Points of Interest widget lets you choose which POIs and POI trails to display on the X-rays. The View menu has commands for toggling the display of the reference frame and landmarks of the current object, and the POIs of all of the other objects. The following mouse and keyboard commands can be used in the X-ray windows: Left mouse key: digitize POI (weighted center). Ctrl-Left mouse key: digitize POI (exact coordinates). Esc: cancel POI digitization. Alt: show weighted center as calculated from current cursor position (prospective position of current POI). PgUp/PgDn: moves to the previous/next reporting time. Home/End: moves to the first/last reporting time. The mouse wheel zooms the X-ray image in and out. Arrow keys translate the X-ray image in image plane. Press and hold the middle mouse button to pan the X-ray image within the window. R resets the position of the X-ray image in the window. Press and hold the left mouse button to change the X-ray image contrast (move cursor left/right) and brightness (move cursor up/down). Shift-R resets the image brightness and contrast. I toggles the display of on-screen image info. H toggles the display of on-screen help text.

Settings
Euler the order of Euler rotations in the body pose maps. XYZ is the default, but in some situations it may be helpful to change the order to avoid gimbal lock. Color Theme you can choose between a light or a dark color theme for the GUI.