Orient3D prepares surface models created by Surface3D (or third-party software such as Mimics and ScanIP) for use in kinematic analyses after they have been tracked in X4D or Locate3D. With it you can define a local coordinate system (LCS) for each object, landmarks, such as ligament attachment sites, and regions of interest (ROIs), which are used in the calculation of distance maps. The local coordinate system of an object is usually, but not necessarily, its “anatomical” reference frame. When a local coordinate system is defined for an object, it is stored in the subject file as a transform from the frame of the object’s segmented image data to the local coordinate system. To analyze tracking results from X4D in Visual3D, the surface model, landmarks, and regions of interest must be saved in the local coordinate system. Even if you are not exporting your data for kinematic analysis in Visual3D, it is still recommended to define the anatomical coordinate system in Orient3D. X4D's tracking results are output using this coordinate system, and it is easier to manipulate the bone in X4D when an anatomical coordinate system is defined.
Load Subject (Ctrl + O): loads a subject file. Note: Only available with a DSX Suite license.
Save Subject (Ctrl + S): saves the currently loaded subject to its existing subject file. Note: Only available with a DSX Suite license.
Save Subject As: saves the currently loaded subject to a new subject file. Note: Only available with a DSX Suite license.
Load Surface...: loads a single surface model file. If you load a surface with this command, there is no way to save the local coordinate system or any landmarks you may define, because these items are saved in the subject file. Orient3D is able to load the following formats:
After the file is loaded the surface model is automatically triangulated to facilitate additional processing, such as decimation and smoothing. Note: Orient3D currently ignores color and texture definitions from MTL and COL files that sometimes accompany Wavefront OBJ files.
Save Surface (global frame): saves the current surface model in the global reference frame as an OBJ file. If a subject file is loaded, it will be updated with the name of the saved surface file. The file dialog contains an option for specifying the units of the vertex coordinates.
Save Surface (local frame): saves the current surface model in its local reference frame as an OBJ file. If a subject file is loaded, it will be updated with the name of the saved surface file. The file dialog contains an option for specifying the units of the vertex coordinates. Meters should be chosen if you plan to import the surface model into Visual3D.
Exit: exits the program. The program can also be closed using the X in the top right corner of the program window.
LCS Menu
Reset to Identity: resets the LCS to the identity matrix (i.e., aligned with the global frame).
Load from File...: loads a 4x4 transform matrix from a file and assigns it to the LCS. The transform must be specified as 16 comma-separated values defining the row-major transform from the global frame to the local frame.
Save to Subject Object: saves the current LCS to the subject object in the subject file. It is useful for saving the LCS when you have manually moved it on the surface model in the 3D window. If you use an algorithm to calculate the LCS, or load it from a file, the subject file is automatically updated, so you do not need to use this command.
Surface Menu
Erase ROI Polygons (Alt + E): Toggles in and out of Erase ROI Polygons mode. In this mode, the cursor changes to a crosshair with the label ROI eraser. If you press and hold the left mouse button and drag the cursor over the surface model, all polygons under the cursor will be removed from any ROIs. This is useful for "cleaning up" an ROI if it projected onto an undesired part of the surface model. Press Esc to exit this mode.
Note: There is no undo for this command.
Align with Cylinder: aligns and scales the oriented bounding box of the surface model with the "Visual3D cylinder". There is no undo for this command. Note: This command is not intended to be used in a DSX context.
View Menu
Display Global Axes (Alt + G): toggles the display of the global coordinate system in the 3D view.
'Display Local Axes (Alt + L): toggles the display of the local coordinate system in the 3D view.
Display Cylinder (Alt + C): toggles the display of a cylinder to which the surface model can be aligned when preparing the surface for use in Visual3D. The z-coordinates of the cylinder range between -1.0 and 0.0 and its radius is currently hard-coded to 0.2.
Note: This command is not intended to be used in a DSX context.
Display ROI Annotations (Alt + R): toggles the display of annotations on the subregions of each ROI.
Object Configuration (Ctrl + Shift + O): toggles the appearance of the dockable Object Configuration widget.
Note: Only available with a DSX Suite license.
Image/Surface Match (Ctrl + Shift + I): toggles the appearance of the dockable Image/Surface Match widget.
Note: Only available with a DSX Suite license.
Image Data Generator (Ctrl + Shift + G): toggles the appearance of the dockable Image Data Generator widget.
Note: Only available with a DSX Suite license.
Landmarks (Ctrl + Shift + L): toggles the appearance of the dockable Landmarks widget.
Local Coordinate System (Ctrl + Shift + C): toggles the appearance of the dockable Local Coordinate System widget.
Regions of Interest (Ctrl + Shift + R): toggles the appearance of the dockable Regions of Interest widget.
Options Menu
Settings: Opens the settings dialog.
Help Menu
Help (F1): opens the Orient3D wiki page in the default browser.
About...: displays a dialog with information about Orient3D, including the installed version number. It also contains the Deactivate button, for deactivating your license.
Widgets
Object Configuration
Subject: This field is automatically populated with the ID of the subject when you load a subject file.
Session: This drop down menu is automatically populated with all of the sessions available for the current subject.
Object: This list is automatically populated with all of the subject objects when you select a session. If the object has a Surface File or ROI Surface File specified for it, the object is enabled in the list so you can select it. The object that is selected is displayed in the 3D view. You can select only one object at a time.
Landmarks
Landmarks are 3D locations on a surface model, such as ligament attachment points, that you digitize for later use in other applications.
Their coordinates are expressed in the reference frame of the object’s segmented image data (not the “local coordinate system”), and are stored in the subject file.
Add Landmark
The Add Landmark button creates a new landmark. Click on the button and move the cursor into the 3D view. The cursor will change into a crosshair with the label "NEW LANDMARK." Click anywhere on the surface model to create a landmark at that location. Press Esc before clicking on the model to cancel the command. Once the landmark has been digitized and added to the table, you can double-click on its name and change it.
Table
The Table lists all of the landmarks that are defined in the object, with their names and X, Y, and Z coordinates. To change the name of a landmark, double-click on the name and type in a new one. To move an existing landmark, select the landmark in the list and then click on the new location on the surface model.
Remove Landmark
The Remove Landmark button deletes a landmark. Select the landmark in the table and then click this button to remove it.
Local Coordinate System (LCS)
This widget lets you create a local coordinate system (LCS) for each subject object in your data set. Before tracking objects in X4D or Locate3D, you should define LCSs for them because the tracking output for each xray frame is the transform from the xray lab frame to the object's LCS. The LCS is stored in the subject file, and is expressed as the transform from the object's segmented image frame to its local coordinate system. There are three methods of defining an LCS, as described in How To: Define the LCS of an Object.
The Visual3D panel can be opened by checking its checkbox and lets you create an LCS using Visual3D's method of defining proximal and distal joints with targets. Consult the Visual3D documentation for details on this method.
Regions of Interest (ROIs)
Regions of interest (ROIs) are portions of an object's surface model that have been labelled as belonging to a particular group. They are used to calculate distance maps for articulating bones by finding the shortest distance from each point in one ROI to the corresponding ROI on a neighboring bone. For example, on the distal femur it is common to define an ROI on the lateral condyle and another one on the medial condyle. On the proximal tibia, an ROI would cover the medial half of the plateau, and a second ROI would cover the lateral half. After both bones have been tracked for a motion trial, distance maps between the medial femur and medial tibia ROIs could be calculated, and the same for the lateral side. In Visual3D these maps can be displayed as color tables on the articulating bones, and used to graph shortest bone-to-bone distances during the trial. Before defining ROIs on an object, it is important to define the LCS first. When ROIs are created, they are aligned with the LCS so that it is easier to interpret distance calculations, and to ensure that all ROIs on an object align with each other. Like the LCS algorithms, the ROIs are implemented in a dll plugin framework, allowing you to integrate your own ROI types.
Currently implemented types of ROIs
Rectangle [NxM]
Rectangle [NxM] is a flat rectangular ROI whose normal vector is specified by the orientation parameter. It is divided into NxM subregions, and is projected onto the object along the negative of its orientation axis. Its initial placement on the object is specified by four landmarks: edge1, edge2, edge3, and edge4. These landmarks do not need to be placed in any specific order, or in any specific locations relative to each other. They are used to define the minimum and maximum extents of the ROI in the two non-principal dimensions in the object’s local coordinate system. For example, if the ROI’s orientation is +Z or -Z, then the minimum and maximum X and Y coordinates of the four landmarks define the ROI rectangle in the XY plane.
Swept Rectangle [NxM]
Swept Rectangle [NxM] is a curved rectangular surface divided into NxM subregions. It is useful for defining ROIs on curved bone surfaces such as the femoral condyles. Its placement is defined by four landmarks: curve1, curve2, curve3A, and curve3B. The first three landmarks define the arc along which the ROI is swept. They also define the direction with N subregions. Curve3B should be placed such that the vector from curve3A to curve3B is perpendicular to the arc, and along the orientation axis of the ROI. Curve3A and curve3B define the direction with M subregions and the distance between them defines the width of the ROI. When the ROI is projected, each subregion is projected along [the negative of] its normal. Because the normals of adjacent subregions along the length of the arc are not parallel, their projections on the object surface will overlap. In these cases the overlapping surface is bisected and divided equally between the two subregions.
Disk
Disk is a circular flat circular ROI whose normal vector is specified by the orientation parameter. It is divided into NxM subregions, and is projected onto the object along the negative of its orientation axis. Its placement is defined by two mandatory and two optional landmarks. The mandatory landmarks are Center and Outer Radius, which define the center and outer perimeter of the ROI, respectively. These mandatory landmarks also define the direction with N subregions. The optional Inner Radius defines the radius of the hole inside the ROI, and the Pie Angle allows you to remove a pie-shaped slice from the disk. The slice is defined by the angle Pie Angle-Center-Outer Radius (the slice between Pie Angle and Outer Radius is removed).
Parameters to control the placement of the ROI
ROI Offset Factor
ROI Offset Factor is the amount to offset the ROI surface along the principal axis, specified as a percentage (0.0 to 1.0) of the maximum dimension of the object. The default value is 0.03. The principal axis depends on the type of ROI, but is roughly equal to the direction of projection.
Principal ROI Expand
Principal ROI Expand is the scale factor for the search box in the principal direction of the ROI. The principal axis depends on the type of ROI, but is roughly equal to the direction of projection. The default value is 5.0.
Secondary ROI Expand
Secondary ROI Expand is the scale factor for the search box along the width of the ROI. The default value is 2.0.
Tertiary ROI Expand
Tertiary ROI Expand is the scale factor for the search box along the length of the ROI. The default value is 2.0.
Add
The Add button creates a new ROI and adds it to the list.
Remove
The Remove button deletes the selected ROI, even if it has been placed (but not projected) on the surface model. However, if the ROI has already been projected, it does not remove it from the model.
Place
The Place button displays the ROI floating above the surface model, based on the landmarks. You can then check and modify its positioning before actually projecting it onto the model. To assist with the positioning of the ROI, a yellow wire frame search box is displayed as well. The ROI is projected onto the surface model that is within this search box. The size of the search box can be adjusted using the values of the parameters in the widget.
Adjust
The Adjust button enters and exits manual ROI adjustment mode. In this mode the placed ROI can be moved manually using the mouse buttons. The following interactions are defined:
Rotating about roi center - moving mouse while pressing left button
Spinning - moving mouse while pressing left button and ctrl key
Translation - moving mouse while pressing left button and shift key or moving mouse while pressing middle button
Scaling - moving mouse while pressing right button
Note that the yellow search box adjusts itself with the ROI.
Pressing the Adjust button a second time exists adjustment mode.
Project
The Project button projects the placed ROI onto the surface model. Each ROI subregion is projected along its normal to determine the polygons of the surface that it intersects. These polygons are labelled with that subregion's label.
Clear Surface
The Clear Surface button clears all of the ROI labels from the surface model.
Image/Surface Match
This widget lets you check if an object's surface model is in the same reference frame as its image data. This is important in X4D so that the surface models are aligned with the DRRs. It works by creating an image surface from the image data, and displaying it in the 3D view with the object's surface model. If they are not aligned, you can use the widget controls to scale, rotate, and translate the image surface so that it aligns with the surface model. Then you can apply the inverse of this transformation to the surface model, so that it aligns with the initial pose of the image surface.
Create Image Data
This command creates a partially transparent surface (the image surface) directly from the object's image data and adds it to the 3D view. It loads the image file specified in the object definition in the loaded subject file. A surface model is created by running a marching cubes algorithm on the thresholded image. Before the surface is displayed it is smoothed and decimated. For the thresholding step the Render Threshold parameter (see: xManager->Objects->Complete Objects->File Data) is used if it is defined in the subject file. If the value of the Render Threshold parameter is not appropriate an error message is displayed.
Remove Image Surface
This command removes the image surface from the 3D view, which should be done after you have aligned it with the surface model and transformed the model.
Scale, Rotation, Translation
These number fields and sliders define the transformations that are applied to the image surface. They are applied in the order in which they are listed in the widget, and the rotations are body-fixed.
Store Transform
This command builds a 4x4 transform from the current scale, rotation, and translation values, and stores it internally. It then resets the scales, rotations, and translations to their initial values. The total transform that is applied to the image surface is this stored transform followed by the current scale, rotation, and translation values. Storing the transform and then specifying additional manipulations of the image surface allows you to perform more complicated transformations, such as an X rotation followed by a Y rotation followed by another X rotation. If you store the transform when there is already one stored, the new one is concatenated onto the stored one.
Clear Transform
This command resets the stored transform to the identity matrix.
ICP
This command runs an iterative closest point algorithm to try to automatically align the image surface with the object's surface model. It starts with the surfaces in their current positions, then translates the image surface so its centroid coincides with the object surface's centroid, then runs the ICP algorithm. This algorithm often fails to align the surfaces if a rotation of more than 60 degrees is required, so for best results you should manually align them more closely than that before running the algorithm.
Note: Another limitation of ICP is that it usually doesn't produce perfect alignment because it operates on the vertices of the surfaces, and the surfaces usually have different vertex sets. Because of this, the pose that the ICP algorithm calculates is output to the scale, rotation, and translation sliders in the widget. Thus you can manually adjust the values to achieve better/perfect alignment before transforming the object.
Transform Object
This command applies the inverse of the total transform to the object's surface model. The total transform is the stored transform followed by the scale, rotation, and translation specified by the number fields and sliders. The stored transform is then reset to the identity matrix, and the scales, rotations, and translations are reset to their initial values. After transforming the object you should save it to a file with File -> Save Surface (global frame).
Image Data Generator
This widget creates 3D image data from surface models of objects. X4D tracks objects in xray images by generating DRRs from 3D image data (usually CT) of the objects. In some cases CT data is not appropriate for a particular subject and only surface models are available. These surface models could be CAD files of metal implants, or surfaces made from MRI or some other imaging modality that does not represent bone density. This widget creates simulated CT data from the surface models so that the objects can be tracked in X4D. For implants you would typically start with a CAD model of the device, and create a solid object within the image data. For bones, it is recommended that you obtain an outer surface (the outer boundary of the cortical bone), and an inner surface (the inner boundary of the cortical bone), to create more realistic image data. If an inner surface is not available, you can specify the cortical thickness in order to create a shell of constant depth. Note: this widget is only available when the computer has a cuda-capable graphics card installed.
Voxel Size
These fields specify the XYZ dimensions of the voxels in model units (usually mm). As you change the values, the voxel count of the resulting image is updated.
Bone Densities
Cortical
The density of the voxels at the outer boundary of the cortical bone.
Cancellous
The density of the voxels in the center of the bone (either inside the inner surface, or more than cortical thickness from the outer surface).
Variation
The percent variation (noise) in densities in the cortical and cancellous regions of bone. Adding noise can make the image data look more realistic, and it helps reduce undesirable edge artifacts when performing edge detection on DRRs made from the image data.
Note: The voxels between the inner and outer surfaces are not all given the cortical density (+/- the variation), because this produces a double edge in the edge detected DRR that is not in the xray images. Instead the voxels are given the cortical density right at the outer boundary, and then ramp down to the cancellous density at the inner boundary. This produces a DRR edge image that better matches the xray edge image.
Inner Surface
If the current object has an OBJ file defining the inner surface of the cortical bone, the Use Inner Surface checkbox will be enabled. If the box is checked, this inner surface will be used when creating the image data. Voxels between the inner and outer surfaces will be given the specified cortical density, and voxels inside the inner surface will be given the specified cancellous density. If the box is not checked, the Cortical Thickness field is enabled. A cortical thickness of -1 means there is no inner boundary, so all voxels inside the outer surface will be given the cortical density. A value greater than zero will create an inner surface that is a constant distance (the specified thickness) from the outer surface. Voxels between the surfaces will have cortical density, and voxels inside the inner surface will have cancellous density.
Create Image Data
Pressing this button creates the image data. The resulting TIFF file will be added to object definition in the subject file, replacing any previously defined image file.
3D View
The 3D View shows the surface model of the currently loaded object. It can also display the global and local coordinate systems. You can change the view camera using these commands:
Rotation: left mouse button
Zooming: right mouse button or mouse wheel
Panning: Shift + left mouse button or middle mouse button
Spinning: Ctrl + left button (spinning is rotating about an axis perpendicular to the screen)
R: Resets the view camera such that the surface model is centered in the window.
Manual LCS manipulation.
The LCS is the only object that can be manipulated in the view. The LCS can be selected by clicking on it while holding the CTRL key. A trackball consisting of three axes and three circles is then displayed. Clicking near one of these axes or circles will select it and a mouse move will result in a constraint manipulation of the LCS (translation along selected axis or rotation in selected circle plane). Clicking anywhere else within the trackball will enable free manipulation: free-form rotation with the left mouse button and free-form translation with the left mouse button while the Shift key depressed. When the trackball is displayed, zooming of the view camera can be achieved with the mouse wheel.
Clicking anywhere outside of the trackball or pressing Esc will deselect the LCS and hide the trackball.
Note: Don't forget to use the Save to Subject Object command to save the manually manipulated LCS to the subject object in the subject file.
]
Settings
Color Theme: The user can choose between a light and dark color theme.
