Surface3D Overview

Surface3D segments objects (bones, implants, etc.) from CT data for use in model-based tracking, and it creates surface models of these objects for subsequent kinematic analysis of the tracking results. After segmenting an object from the CT data, the CT data can be cropped and masked, and then saved to a RAW or TIFF file. These cropped image files with just one object in them, are used by X4D to generate DRRs for model-based tracking. It can also generate a polygonal surface model from the currently segmented object using a marching cubes algorithm, and save it to an OBJ file. The surface models are used by Orient3D to define anatomical references and regions of interest (for distance map calculations), and by Visual3D for kinematic analysis. Surface3D also allows you to identify landmarks (e.g., ligament attachments) and points of interest (e.g., implanted beads) in the image data.

Menus

File Menu

Load Subject... (Ctrl + O): loads a subject file. Save Subject (Ctrl + S): saves the currently loaded subject to its existing subject file. Save Subject As...: saves the currently loaded subject to a new subject file. Load Image... (Ctrl + I): loads 3D image data for segmentation. This command is meant to be used when you do not have a subject file, but just want to load image data and segment it. Currently, Surface3D supports DICOM, RAW, and TIFF formats. At least one of the image files needs to have the appropriate extension so that it can be selected in the file dialog. If a subject is currently loaded, it is closed. Load Supplemental Image...: loads 3D image data for an object that already has a segmented image file. This command is meant to be used for handling multiple DICOM stacks related to processing a single object. For example, when modeling the femur for a knee study, you might have one image stack for the distal femur and a second one with a few slices of the femoral head for locating the hip center. In this case you would load the distal femur stack for segmentation and surface model creation, then load the hip stack as a supplemental image for defining the hip center landmark. Load Label... (Ctrl + L): loads an earlier saved segmentation from file. Surface3D supports the TIFF and RAW formats. Loading a segmentation from file will erase the current label and label edits. The menu item becomes enabled after image data has been loaded. The size of the segmented image data in the selected file must be the same as the size of the currently loaded image data. Export Label... (Ctrl + E): saves the current segmentation to file. Surface3D supports the TIFF and RAW formats. Save Segmented Object: saves the currently loaded image data to a RAW or TIFF file. It is meant to be called after the data has been masked and cropped to a label. If a subject is loaded and the object in the subject already has a link to a segmented image file, this file will be overwritten without notification. If a link does not already exist, you will be prompted to select a new file name. You can then also choose the desired format (OBJ or IV) and units. Save Surface Model: saves the polygonal surface model to an OBJ or IV file. If a subject is loaded and the object in the subject already has a link to a surface model file, this file will be overwritten without notification. If a link does not already exist, you will be prompted to select a new file name. Exit: exits the program. The program can also be closed using the X in the top right corner of the program window.

View Menu

Object Configuration (Ctrl + Shift + O): toggles the appearance of the dockable Object Configuration widget. Landmarks (Ctrl + Shift + L): toggles the dockable Landmarks widget, which has a list of the landmarks in the selected object. This item appears in the menu only when a subject file is loaded and an object is selected. POIs (Ctrl + Shift + P): toggles the dockable POIs widget, which has a list of the points of interest in the selected object. This item appears in the menu only when a subject file is loaded and an object is selected. Segmentation (Ctrl + Shift + S): toggles the dockable Segmentation widget, which contains the editable list of object labels that can be used for segmentation. Histogram of Image (Ctrl + Shift + H): toggles the dockable widget with the histogram of the image data.

Segmentation Menu

Add Object Label (Ctrl + A): adds a segmentation label to the Segmentation widget. Remove Object Label (Ctrl + R): deletes the currently selected segmentation label. Undo (Ctrl + Z): undoes the last manual edit to the segmentation label. Redo (Ctrl + Y): redoes the last undone manual edit to the segmentation label. Select Thresholding Seed: allows the user to pick a point(pixel) in any of the 2d render windows, which will then be used as a seed for 3d thresholding using the threshold values set in the Histogram widget. Fill 3D Holes... (Ctrl + H): first opens a dialog with some descriptive text and the opportunity to change the value of the 3D Hole Threshold parameter (default: 2000). When the user decides to go ahead, the command automatically fills all 3D holes in the current segmentation. The algorithm finds each unlabelled voxel next to a labelled voxel and uses a flood-fill algorithm to find all unlabelled voxels that are connected to it. If the size of this region is less than the value of the 3D Hole Threshold parameter, then the region is labelled with the current label. Erode Label: automatically erodes the segmentation. The basic effect is to erode away the boundaries of regions of the currently selected label. Thus areas of label pixels shrink in size, and holes within those areas become larger. Dilate Label: automatically dilates the segmentation. The basic effect is to gradually enlarge the boundaries of regions of the currently selected label. Thus areas of label pixels grow in size, and holes within those areas become smaller. Unlabel Disconnected Regions: unlabels all voxels that are not connected to the selected voxel. It is useful for unlabelling large sections of the image data after you have labelled the bone of interest. For example, when segmenting a vertebra from spine images, the appropriate threshold settings often label all of the vertebrae because of their close proximity. Rather than manually erase the label from all of these unwanted voxels, you can use this command. Start by completely labelling the bone of interest. Then erase the label from all unwanted voxels bordering this bone, so that the bone is not connected to any other labelled regions. Then choose this command and select any voxel in the bone of interest.

Image Menu

Toggle Voxel Interpolation: toggles the method used to render the image data voxels and the segmentation label voxels. When this option is unchecked, each voxel is rendered with a constant color (flat shading). When this option is checked a smooth shading technique is used, in which a voxel's color is smoothly interpolated between its neighbors. Reverse Image Slice Order: reverses the slices of the loaded image, i.e., the first slice becomes the last one, and the last slice the first one. The segmentation and its edits are automatically adjusted, as are the locations of the POIs and landmarks. Mask Labeled Image Data: sets all unlabeled voxels in the current image to zero. This action cannot be undone. Crop Image Data to Labels: reduces the size of the image to the bounding box of the current segmentation label (i.e., the minimum dimensions that include all labeled voxels). This action cannot be undone.

Model Menu

Create Surface Model: creates a polygonal surface model from the current segmentation label. It uses a marching cubes algorithm to find an isosurface from the set of labelled voxels. For best results, the labelled voxels should not have any holes in the interior. Decimate Surface Model...: reduces the number of vertices and polygons in the surface model. Decimation is controlled by the Decimation Factor parameter, which specifies the approximate percentage of polygons that will be removed from the surface. For example, if the parameter is set to 0.2, approximately 20% of the polygons will be removed each time the decimate command is performed. Decimating too much may remove small details of the surface model that are needed for tracking. This action cannot be undone. Smooth Surface Model...: smooths the surface model by making small adjustments to the vertex positions using a Laplacian algorithm. This algorithm is controlled by the following three parameters. Relaxation Factor specifies the Laplacian relaxation factor. Smoothing Iterations is the number of iterations performed each time the smoothing command is executed. Feature Edge Smoothing turns on (true) and off (false) smoothing of sharp interior edges on the surface model. Smoothing will shrink the surface model towards its center. This action cannot be undone. Delete Surface Model: deletes the current surface model.

Options Menu
Settings...: opens the Settings dialog.

Help Menu
Help (F1): opens the Surface3D wiki page in the default browser. About...: displays a dialog with information about Surface3D, including the installed version number. It also contains the Deactivate button, for deactivating your DSX suite license.

Widgets

Object Configuration

The Object Configuration widget is available only if a subject file is currently loaded in Surface3D. Subject: This field is automatically populated with the ID of the subject when you load a subject file. Session: This drop down menu is automatically populated with all of the sessions available for the current subject. Object: This list is automatically populated with all of the objects in the current session. Each object is enabled if it either has a segmented image file specified for it, or if there is scan data specified for the session. Selecting one of the objects loads its segmented image file, if it has one. If it does not, a dialog is displayed containing all of the scan data files specified for the session, allowing you to select one of them for segmentation.

Histogram of Image
The histogram of the image shows the range of voxel intensities in the currently loaded image data. If a label is selected in the Segmentation widget, the upper and lower thresholds of the label are represented in the histogram as red and green vertical lines, respectively. You can left-click and drag these lines to change the respective threshold values.

Points of Interest

Points of interest (POIs) are features of an object that you are planning to track in X-ray motion trials using Locate3D. They are most commonly used for radiopaque beads implanted in an object whose motion you want to measure. They are distinguished from landmarks, which are object features whose locations you want to calculate once the object pose is known. A common type of landmark is ligament attachment sites. Thus, POIs are used to determine the pose of an object and landmarks are used to calculate locations dependent on the pose. Although the POI widget has commands for adding and deleting POIs, it is recommended that you use xManager to define them, where you can specify their radii and colors for display in the GUI. In order to track a POI in Locate3D, its radius must be specified so that the circle-finding algorithm can estimate its size in the X-ray images. Once they have been added to an object in xManager, you can use Surface3D to define their locations. If you do create new POIs in Surface3D, be sure to go back to xManager later to define their radii and colors. Add POI: adds a new POI to the object. After pressing the button, left-click in one of the 2D image data windows to set the location of the point. Surface3D will create a POI with a default name and add it to the table. To change the name, double-click on it and type in a new name. Remove POI: deletes the currently selected POI from the object. To select a POI, click on its row in the table. Show POIs: toggles the display of the POI spheres in the 2D and 3D image data windows. Show Labels: toggles the display of the POI labels (names) in the 2D and 3D image data windows. POI Table: The table shows all of the object's POIs. The columns are the name of the POI, and the X, Y, and Z coordinates in the image data's reference frame. A POI name can be changed by double-clicking on it and typing in a new name. To change the XYZ coordinates, select the POI by clicking on its row in the table, then click on the new location in one of the 2D image data windows. POI Digitizing: This control lets you choose how the coordinates of the POI will be determined when you left-click in one of the image data windows to add or modify a POI. If the Find Weighted Center checkbox is not checked, the POI is placed at the exact coordinates of the location you clicked on. If this box is checked, a marching cubes algorithm searches for all voxels connected to the one you clicked on that are within the specified Intensity Range. Surface3D then calculates the weighted center of these voxels and puts the POI at the center. For implanted beads, the recommended approach is to set the Intensity Range to the expected range for voxels containing beads, then left-click on any bead voxel to define a POI at the weighted center.

Landmarks

Landmarks are features of an object whose locations you want to calculate once the object's pose is known, such as ligament attachment sites. They are distinguished from points of interest, which are object features such as implanted beads that are used to calculate the pose. Thus, POIs are used to determine the pose of an object and landmarks are used to calculate locations dependent on the pose. Add Landmark: adds a new landmark to the object. After pressing the button, left-click in one of the 2D image data windows to set the location of the landmark. Surface3D will create a landmark with a default name and add it to the table. To change the name, double-click on it and type in a new name. Remove Landmark: deletes the currently selected landmark from the object. To select a landmark, click on its row in the table. Show Landmarks: toggles the display of the landmark spheres in the 2D and 3D image data windows. Show Labels: toggles the display of the landmark labels (names) in the 2D and 3D image data windows. Landmark Table: The table shows all of the object's landmarks. The columns are the name of the landmark, and the X, Y, and Z coordinates in the image data's reference frame. A landmark name can be changed by double-clicking on it and typing in a new name. To change the XYZ coordinates, select the landmark by clicking on its row in the table, then click on the new location in one of the 2D image data windows. Landmark Digitizing: This control lets you choose how the coordinates of the landmark will be determined when you left-click in one of the image data windows to add or modify a landmark. If the Find Weighted Center checkbox is not checked, the landmark is placed at the exact coordinates of the location you clicked on. If this box is checked, a marching cubes algorithm searches for all voxels connected to the one you clicked on that are within the specified Intensity Range. Surface3D then calculates the weighted center of these voxels and puts the landmark at the center.

Segmentation

The Segmentation widget provides access to the labels that are used to identify and segment objects (bones, implants, etc.) in the image data. The labels are contained in a table with their name, color, and upper and lower threshold values. The recommended approach for segmenting image data is described in How To: Segment Image Data and Create Surface Models. When you are finished with the first object, selecting the next object will delete the existing segmentation label. To create a label, use the command Segmentation -> Add Object Label. To remove a label, use the command Segmentation -> Remove Object Label. To change the name of a label, double-click on the name and type in a new one. To change its color, double-click on the color cell in the table and select a new color from the dialog. The upper and lower threshold values can be changed by dragging the vertical lines in the Histogram of Image widget, or by double-clicking on the cells in the table and typing in new values.

Settings

Undo Limit: maximum number of label editing actions that can be undone. The default value is 10. Paintbrush Radius: the radius (in voxels) of the circular paintbrush that is used for painting and erasing segmentation labels. The default value is 5. The radius can also be increased or decreased using the mouse wheel, while pressing 'p' to paint or 'e' to erase. Text Scale: the scale factor for POI and landmark text labels. The default value is 0.75. Color Theme: the user can choose between a light or a dark color theme.

Image Data Windows

2D Views

The three 2D views show cross sections of the image data along the X, Y, and Z axes. The upper-left view, with the red border around the image data, shows a slice perpendicular to the X axis. The lower-right view (green) shows a slice perpendicular to the Y axis, and the upper-right view (blue) shows a Z slice. When the mouse cursor hovers over a 2D view, its border turns yellow indicating the window has become active. The following interaction is possible in an active 2D view: zooming using the mouse wheel, panning using the right mouse button, image contrast can be adjusted by pressing the left mouse button and Shift key and dragging the cursor left and right, image brightness can be adjusted by pressing the left mouse button and Shift key and dragging the cursor up and down, and scrolling through the image slices using the Page Up/Down keys. pressing the ESC key will exit the interaction mode, i.e. cancel the selection of a segmentation seed, poi, or landmark location. pressing the R key will reset the camera view. pressing the HOME/END key will move to the first/last image slice. pressing the P key when a label is selected in the Segmentation widget activates paint mode. The current mode is displayed on the right side of the status bar at the bottom of the main window. In this mode you can label additional voxels in the segmentation by pressing the left mouse button. It is also possible to drag the mouse while keeping the left button depressed. The radius of the paintbrush can be changed while in paint mode using the mouse wheel. It can also be changed in the dialog accessible by selecting Options → Settings in the menu bar. pressing the E key when a label is selected in the Segmentation widget activates erase mode. The current mode is displayed on the right side of the status bar at the bottom of the main window. In this mode you can unlabel voxels in the segmentation by pressing the left mouse button. It is also possible to drag the mouse while keeping the left button depressed. The radius of the paintbrush can be changed while in erase mode using the mouse wheel. It can also be changed in the dialog accessible by selecting Options → Settings in the menu bar. pressing the M key when a label is selected in the Segmentation widget activates manual hole filling mode. The current mode is displayed on the right side of the status bar at the bottom of the main window. In this mode you can use the left mouse button to select holes in the segmentation that need to be filled. pressing the S key toggles the display of the segmentation labels. pressing the H key toggles the display of on-screen help text.

3D View

The 3D view in the lower-right shows the image slices from the 2D views as well as the current object's surface model, if one exists. It also shows the reference frame of the image data, with its origin in the lower-left corner of the first image slice in the data file. When the mouse cursor hovers over the 3D view, its border turns yellow indicating the window has become active. The following interaction is possible in an active 3D view: 3D rotation (virtual trackball) using the left mouse button, panning using the middle mouse button, zooming using the mouse wheel or right mouse button, and spinning (in-window-plane rotation) using the left mouse button while pressing the Ctrl key.