DSX Overview

Purpose

Orthopaedic disorders are a leading cause of disability in the U.S., with arthritis and/or spine problems adversely affecting quality of life for more than 20% of adults. While advances in diagnostic imaging have improved our ability to detect structural changes in musculoskeletal tissues, they typically reveal little about joint function. There is evidence that abnormal mechanical joint function contributes significantly to the development and progression of many types of joint disease. Notably, joint translations of only a few millimeters are critical to estimating tissue stress or joint impingement during loaded functional movements. Dynamic Stereo X-ray (DSX) is the only currently available technology that can achieve sub-millimeter bone pose (position and orientation) estimation accuracy during a wide variety of functional movements.

Note: there are many names coined for DSX, include Röntgen Stereometric Analysis, Biplanar Videoradiography and X-ray Reconstruction of Moving Morphology (XROMM).

Biplane Videoradiography

A typical custom biplane videoradiography system. For those interested, contact Marty Kulis at Imaging Systems The X-ray tubes generate X-ray beams which pass through the joint of interest and enter the image intensifiers. Each image intensifier projects onto a phosphor screen an image that is subsequently captured by a camera mounted on the intensifier. The two cameras collect the images in a time synchronized manner. The purpose of the DSX software is to identify the pose (position and orientation) of the bones from the 2D X-ray images and subject specific model of the bones.

DSX Summary

The DSX Suite (DSX) processes data from biplanar videoradiography. The suite of applications allows processing of the X-ray data from collection through analysis and reporting. At the heart of the suite is the ability to track 3D objects (bones, implants, etc.) in X-ray images. DSX is based on a 3D-to-2D approach to markerless motion capture that generates digitally reconstructed radiographs (DRRs) of the objects and matches them to the X-ray images. These DRRs are generated from real or simulated CT data. Simulated CT data can be created from polygonal surface models, such as CAD models of implants or surfaces made from MRI. MRI surfaces track better if they are double-shelled, so if you are creating 3D bone models from MRI, it is recommended that you use a sequence like 3D CISS so you can extract the interior and exterior boundaries of the cortical bone. DSX also performs all of the calibration and image correction tasks needed to track objects in X-ray images. It calculates the 3D configuration parameters of the X-ray hardware, and uniformity- and distortion-corrects the X-ray images. It works with any 3D calibration object that is a collection of fixed beads, and any distortion grid that has regular spacing of beads or holes. DSX has a tool for segmenting CT images and creating 3D surface models for each bone. You can then define an anatomically meaningful local reference frame for the bone, as well as define landmarks (e.g., ligament attachments) and regions of interest (for calculating distance maps). This tool does not yet segment MRI data, but you can use a third-party program to segment MRI and then import the surface models into DSX for further processing (and for generating simulated CT data). One of the unique features of the DSX Suite is its integration with Visual3D and use of motion capture data. It can import surface-marker-based motion capture data to seed the bones for X-ray tracking. This can be a big time saver, even if the bones cannot be directly measured with surface markers. For example, a seven-segment lumbar spine model (pelvis->L5->L4->L3->L2->L1->torso) includes kinematic constraints that define normal motion of the seven components. It can be driven by three markers on the pelvis and three on the torso. Once the model is used to roughly position all of the bones in the X-ray images, it is deactivated so that the DRR-based tracking can track each bone independently. Once all of the bones have been tracked, the results can be exported to Visual3D for kinematic analysis and integration with motion capture data (e.g., whole-body motion, force-plate data, EMG). DSX also has a study planning tool, which helps you determine the best configuration of your X-ray hardware to capture the joint and motion of interest. It is a CAD tool that creates a virtual lab containing the subject, X-ray equipment, motion capture cameras, and other equipment such as treadmills and force plates. The subject skeleton can be animated with motion capture data, and the bones generate simulated X-rays as they pass in front of the virtual X-ray image planes.

DSX Software Objectives

Our objective was to develop commercial software for rapid, robust, and reliable bone pose estimation from radiographic image sequences with minimal operator intervention. The research applications of DSX are clearly established, and are leading to breakthroughs in nearly all orthopaedic specialties. The potential of DSX as a clinical tool for diagnosing joint and tissue disorders and directly improving patient treatment and outcomes has yet to be realized. The greatest challenge is robustness and the minimization of manual labor. Just as cine-angiography has revolutionized diagnosis and treatment of cardiovascular disorders, widespread availability of DSX could significantly improve treatment for a wide variety of orthopaedic disorders. We hope that the release of DSX will open the door for large clinical studies and the development of a myriad of diagnostic imaging applications. Mitigating concerns over the limited availability of appropriate imaging hardware and radiation exposure. The cost of a dynamic imaging system is actually similar to or less than that of many current diagnostic imaging systems (biplane cine-angiography, CT, MRI). The evolution of DSX should expand the applications to simpler, lower-cost equipment by removing the requirement for synchronous imaging. The system provides dynamic data during functional activities that is unavailable from any other modality (including MRI), and should have unique clinical applications for evaluating movement-related musculoskeletal disorders.

Summary of Markerless Pose Estimation Algorithm

The 3D pose of a bone is quantified by a position (3 degrees of freedom) plus an orientation (3 degrees of freedom). A pose map is a series of contiguous poses for a bone represented as either discrete poses at each X-ray frame or by a spline across all frames. Given a 3D representation of a bone extracted from a high-resolution CT scan of the subject, a local reference frame assigned to the bone, and a time series of X-ray images containing the bone, a pose map is the solution of the DSX across all frames. The DSX algorithm solves for the 3D pose by registering two non-coplanar X-ray images of a bone to two digitally reconstructed radiographs (DRRs) (Figure). Given the position and orientation of an X-ray source, an X-ray image plane, and volumetric CT bone, a DRR is the projection of the CT bone onto a virtual X-ray image using a simplified X-ray generation model. In other words, rays from the X-ray source are cast through the bone to generate a simulated X-ray with the same size and resolution of the actual X-ray. Figure: 3D representation of a biplane X-ray configuration when the two X-rays are synchronized. The distal femur, reconstructed from the CT data, is shown in the middle. The inline X-ray image (in line with the X-axis of the lab reference frame) is shown in the red frame; the red line is the perpendicular from the center of the X-ray image plane to the X-ray source. The offset X-ray image (offset from the X-axis of the lab reference frame) is shown in the green frame; the green line is the perpendicular from the center of the X-ray image plane to the X-ray source. For illustration, the inline X-ray image is shown after processing (smoothing and edge detection) and the offset image is shown unprocessed.

Summary of Marker-Based (Implanted Beads) Pose Estimation Algorithm

The gold standard for Biplanar Videoragiography is based on tracking beads implanted onto the bones. Locate3D can be used for tracking the beads, and Visual3D can be used to estimate the pose. Unlike Marker-based optical motion capture the beads do not move relative to the bones (i.e. there is no soft tissue artifact), and the resulting pose estimates are accurate to less than 1 mm. If beads are implanted, all of the flexibility and power of Visual3D comes into play to analyze and report the biomechanical results.

Scientific Significance

Orthopaedic disorders are a leading cause of disability in the U.S., with arthritis and/or spine problems adversely affecting quality of life for more than 20% of adults. With an aging population, the rate of disability from orthopaedic disorders has been increasing steadily. While the majority of tools for clinical assessment of orthopaedic conditions rely upon static measures, joints must function properly in a range of postures and complex loading conditions. Pain and functional limitations are often activity-specific, and can defy reliable diagnosis with conventional clinical tests. Abnormal structural findings on MRI can be poorly correlated with clinical symptoms (Carragee et al, 2006; Djurasovic et al 2012). Joint instability is a common diagnosis for a variety of joint disorders, but there are no diagnostic tests that actually evaluate functional joint stability. Widely-used clinical laxity tests for assessing the knee after ACL injury are insufficient for identifying functional status and ineffective for determining which individuals might be able to cope with the injury without surgery (Eastlack et al, 1999)). By evaluating joint function during activities that are important to the patient (e.g., lifting for the factory worker with low back pain, running/jumping for the athlete with a knee injury, or climbing stairs for an individual with patello-femoral pain or osteoarthritis) the relationship between structural abnormalities and dysfunction can be assessed directly. By combining high-accuracy bone motion with patient-specific soft-tissue geometry from MRI, it is possible to characterize soft-tissue behavior directly. Requirements for a dynamic imaging system with these capabilities include sample rates high enough to capture dynamic movements and sub-millimeter spatial accuracy to characterize tissue deformation, identify joint instability or impingement, and/or identify early signs of tissue degeneration. DSX is the only currently available technology that can achieve this requisite level of performance during a wide variety of functional movements [Tashman and Anderst, 2003; Bey et al, 2006; Bey et al, 2008; Anderst et al, 2009; Martin et al, 2011; Li et al, 2012; Aiyangar at al, 2014). Thus, it is not surprising that the popularity of dynamic radiographic imaging has grown rapidly over the last decade. DSX has been used to characterize a variety of joint disorders, including changes in joint contact kinematics with knee injuries (ACL,PCL, meniscus) (Tashman et al., 2004; Gill et al., 2009; Van de Velde et al, 2009; Hoshino et al., 2013; Goyal et al., 2012; Marsh et al., 2014), dynamic aspects of patello-femoral disorders (Fernandez et al., 2008; Bey et al., 2008), femoro-acetabular impingement of the hip (Martin et al., 2011; Kapron et al., 2014), shoulder function after rotator cuff injury (Bey et al, 2011) and arthroplasty (Massimini et al., 2010), changes in intervertebral kinematics with lumbar disc degeneration (Anderst et al., 2008; Li et al, 2011), and deformation of the joint capsule and intervertebral discs with cervical spine disc fusion (Anderst et al., 2013; Anderst et al., 2014). References: Aiyangar AK, Zheng L, Tashman S, William JA, Xudong Z (2014). Capturing three-dimensional in vivo lumbar intervertebral joint kinematics using dynamic stereo-X-ray imaging. J Biomech Eng; vol 136(1): 011004. PMID: 24149991. Anderst WJ, Vaidya R, Tashman S (2008). A technique to measure three-dimensional in vivo rotation of fused and adjacent lumbar vertebrae. Spine J. 2008; vol 8: 991-7. PMID: 17919983. Anderst W, Zauel R, Bishop J, Demps E, Tashman S (2009). Validation of three-dimensional model based tibio-femoral tracking during running. Med Eng Phys; vol 31: 10–6. PMID: 18434230. Anderst, W. J., E. Baillargeon, W. F. Donaldson, 3rd, J. Y. Lee and J. D. Kang (2011). Validation of a noninvasive technique to precisely measure in vivo three-dimensional cervical spine movement. Spine; vol 36: 393-400. PMID 21376550. Anderst WJ, Donaldson WF, Lee JY, Kang JD (2014). In vivo cervical facet joint capsule deformation during flexion-extension. Spine; vol 39(8): 514-20. PMID: 24480943. Anderst W, Donaldson W, Lee J, Kang J (2013). Cervical disc deformation during flexion-extension in asymptomatic controls and single-level arthrodesis patients. J Orthop Res; vol 31(12):1881-9. PMID: 23861160. Bey MJ, Peltz CD, Ciarelli K, Kline SK, Divine GW, van Holsbeeck M, MU S, Kolowich P, Lock T, Moutzouros (2011). In Vivo Shoulder Function After Surgical Repair of a Torn Rotator Cuff: Glenohumeral Joint Mechanics, Shoulder Strength, Clinical Outcomes, and Their Interaction. Am J Sports Med; vol 10: 2117-2129. PMID: 21737834. Bey MJ, Zauel R, Brock SK, Tashman S (2006). Validation of a new model-based tracking technique for measuring three-dimensional, in vivo glenohumeral joint kinematics. J Biomech Eng; vol (128): 604–9. PMID: 16813452. Bey MJ, Kline SK, Tashman S, Zauel R (2008). Accuracy of biplane X-ray imaging combined with model-based tracking for measuring in-vivo patellofemoral joint motion. J. Orthop Surg Res; vol 3: 38. PMID: PMC2538511. Brainerd EL, Baier DB, Gatesy SM, Hedrick TL, Metzger KA, Crisco JJ (2010). X-ray reconstruction of moving morphology (XROMM): Precision, accuracy and applications in comparative biomechanics research. Journal of Experimental Zoology; vol (313A): 262–79. PMID 20095029. Carragee EJ, Alamin TF, Miller JL, Carragee JM (2005). Discographic, MRI and psychosocial determinants of low back pain disability and remission: a prospective study in subjects with benign persistent back pain. Spine J; vol 5: 24-35. PubMed PMID: 15653082. Djurasovic M, Carreon LY, Crawford CH, 3rd, Zook JD, Bratcher KR, Glassman SD (2012). The influence of preoperative MRI findings on lumbar fusion clinical outcomes. Eur Spine J; vol 8: 1616-23. PubMed PMID: 22388983. Eastlack ME, Axe MJ, Snyder-Mackler L (1999). Laxity, instability, and functional outcome after ACL injury: copers versus noncopers. Med Sci Sports Exerc; vol 31(2), 210-5. PMID: 10063808. Fernandez JW, Akbarshahi M, Kim HJ, Pandy MG (2008). Integrating modelling, motion capture and x-ray fluoroscopy to investigate patellofemoral function during dynamic activity. Comput Methods Biomech Biomed Engin; vol 11(1): 41-53. PMID: 17943487. Gill TJ, Van de Velde SK, Wing DW, Oh LS, Hosseini A, Li G (2009). Tibiofemoral and Patellofemoral Kinematics Following Reconstruction of an Isolated Posterior Cruciate Ligament Injury: In Vivo Analysis During Lunge. Am J Sports Med; vol 37(12): 2388-85. PMID: 19726621. Goyal K, Tashman S, Wang JH, Li K, Zhang X, Harner C (2012). In vivo analysis of the isolated posterior cruciate ligament-deficient knee during functional activities. Am J Sports Med; vol 40(4): 777-85. PMID: 22328708. Haque MA, Anderst W, Tashman S, Mari GE (2013). Hierarchical model-based tracking of cervical vertebrae from dynamic biplane radiographs. Med Eng Phys; vol 35(7): 994-1004. PMID: 23122602. Hoshino Y, Fu FH, Irrgang JJ, Tashman S (2013). Can joint contact dynamics be restored by anterior cruciate ligament reconstruction? Clin Orthop Relat Res; vol 471(9): 2924-31. PubMed PMID: 23283673. Kapron AL, Aoki SK, Peters CL, Maas SA, Bey MJ, Zauel R, Andersen A (2014). Accuracy and feasibility of dual fluoroscopy and model-based tracking to quantify in vivo hip kinematics during clinical exams. J Appl Biomech; vol 30(3): 461-70. PMID: 24584728. Lee J, Baillargeon E, Anderst W (2010). Lumbar Spine Motion During Functional Movement: In Vivo Validation of Flexion/Extension Movement Tracking. Lumbar Spine Research Society; Chicago, Il, Li K, Tashman S, Fu F, Harner C, Zhang X (2010). Automating analyses of the distal femur articular geometry based on three-dimensional surface data. Ann Biomed Eng; vol 38(9): 2928-36. PMID: 20496005. Li K, Zheng L, Tashman S, Zhang X (2012). The inaccuracy of surface-measured model-derived tibiofemoral kinematics. J Biomech; vol 45(15): 2719-23. PubMed PMID: 22964018. Martin DE, Greco NJ, Klatt BA, Wright VJ, Anderst WJ, Tashman S (2011). Model-based tracking of the hip: implications for novel analyses of hip pathology. J Arthroplasty; 26(1): 88-97. PubMed PMID: 20347253. Marsh C, Tashman S (2014). Dynamic Changes in Subchondral Tibiofemoral Joint Space as a Predictor of Early Cartilage Damage. Presented at the 2014 Annual Meeting of the Orthopaedic Research Society, March 2014. PDF available at: http://www.ors.org/Transactions/60/1963.pdf Marsh C, Martin DE, Harner C, Tashman S (2014). Effect of Posterior Horn Medial Meniscus Root Tear on In Vivo Knee Kinematics. Orthop J Sports Med; vol 2(7): 1-7. Massimini DF, Li G, Warner JP (2010). Glenohumeral contact kinematics in patients after total shoulder arthroplasty. J Bone Joint Surg AM; vol 92(4): 916-26. PMID: 20360516. Tashman S, Anderst W (2003). In-vivo measurement of dynamic joint motion using high speed biplane radiography and CT: application to canine ACL deficiency. J Biomech Eng; vol 125(2): 238-45. PMID: 12751286. Tashman S, Collon D, Anderson K, Kolowich P, Anderst W (2004). Abnormal rotational knee motion during running after anterior cruciate ligament reconstruction. Am J Sports Med; vol 32(4): 975-83. PMID: 15150046. Tomazevic D, Likar B, Slivnik T, Pernus F (2003). 3-d/2-d registration of CT and MR to X-ray images. IEEE Trans Med Imag; vol (22): 1407–16. PMID: 14606674. Thorhauer E, Abebe E, Araki D, Ma T, Tashman S (2015). Can Clinical-Grade MRI Scans Reliably Detect Native Anterior Cruciate Ligament Insertion Sites? . Presented at the 2014 Annual Meeting of the Orthopaedic Research Society, March 2014.. PDF available at: www.ors.org/Transactions/60/2000.pdf Van de Velde SK, Gill TJ, Li G (2009). Evaluation of kinematics of anterior cruciate ligament-deficient knees with use of advanced imaging techniques, three-dimensional modeling techniques, and robotics. J Bone Joint Surg AM; vol 91 Suppl: 108-14. PubMed PMID: 19182035. Zhang X, Aiyangar A, Zheng L, Tashman S, Anderst W (2013). Capturing Three-dimensional In Vivo Lumbar Intervertebral Joint Kinematics Using Dynamic Stereo-X-ray Imaging. J Biomech Eng; vol 136(1): 011004. PMID: 24149991. Zhua Z, Massimini DF, Wanga G, Warner JJP, Li G (2012). The accuracy and repeatability of an automatic 2D–3D fluoroscopic image-model registration technique for determining shoulder joint kinematics. Med Eng Phys; vol (34): 1303-1309. PMID 22285714.

DSX_Workflow

The DSX workflow is complicated given the number of files and programs that are used. The graphic below shows the flow of data between the different programs in the DSX suite. Some of the data (e.g. bead and landmark locations) are stored in the subject file, other data (e.g., large image files, and data required for processing in Visual3D) are stored in separate files. All DSX programs store the file path of the files they create in the subject file. It is not required to use all programs in the DSX suite. You may prefer to use a third party application to generate a surface model from the subject's scan data. In this situation you need to manually update the subject file in xManager with the path to the generated bone surface model (obj file), the labeled (and potentially cropped) CT data, and with landmark locations.

Image quality requirements (resolution, contrast, depth).

Specific recommendations for optimal image resolution depend on the other hardware in your system and the types of motions and joints you'll be investigating. But we have some general advice. Always go with the highest quality camera you can afford, because that way you do not limit your options. Pixel size, fill factor, sensitivity (which depends largely on the previous two factors) and dynamic range/noise are probably more important than maximum resolution. Dynamic range is linked to bit depth, but it is really the noise level of the camera that ultimately determines how many of the bits are actually useful. A high-quality 10-bit camera can easily outperform a noisy 12-bit one. The optimal resolution depends on the size of the image intensifiers and the joints being imaged. For knees, hips, etc., a resolution such as 600x600 is often enough, but for smaller joints higher resolution can be better. In many cases (especially cervical and lumbar spine), edge-detection on the high-resolution images (e.g., 1920x1920) does not yield edges strong enough to track, so you end up downsampling to 640x640 anyway. Also, if your high-resolution X-ray images have 0.2mm pixels, but your CT scans have 0.5-1.0mm voxels, then you end up "wasting" some of the X-ray resolution by comparing them to relatively coarse DRRs. In other words, high-resolution cameras are good to have in case you need them, but in many applications you will want or need to reduce the resolution for processing. If you have some sample images from your current setup, you can load them into ImageJ and see how they look at various downsampled resolutions. Then you can use the "Find Edges" command (which is similar to how DSX performs edge detection) to see how the edges compare at the different resolutions. This will give you some idea of the ideal target resolution for that study. Sensitivity of each type of camera sensor varies with frequency of the light, so the published “white light” sensitivity may not apply at the green wavelengths the image intensifiers output (520 - 560 nm). Lastly, certain features are necessary in the camera to provide proper synchronization and control, so these need to be considered in addition to image quality.

DSX Suite of Applications

DSX Suite is designed to process data from biplanar videoradiography. The suite of applications allows processing of the X-ray data from collection through analysis and reporting. The documentation contains some technical terms that are specific to the program and technology. Definitions of these terms can be found here. The release notes for DSX, with a list of enhancements and bug fixes in every release since the first can be found here.

There are seven applications in the DSX suite, plus Visual3D:

XManager

XManager is the graphical interface to a subject file, which contains all of the information about one DSX subject, including links to all of the data files. This information is read and written by the other DSX applications (except PlanDSX).

PlanDSX

PlanDSX is a design tool that helps you determine the appropriate configuration of the X-ray equipment to capture high-quality images of the joint[s] of interest when planning a new study. It is essentially a CAD tool that creates a virtual lab containing the subject, X-ray equipment, motion capture cameras, and other equipment such as treadmills and force plates. The subject skeleton can be animated with motion capture data, and the bones generate simulated X-rays as they pass in front of the virtual X-ray image planes.

Surface3D

Surface3D segments objects (bones, implants, etc.) from CT data for use in model-based tracking, and it creates surface models of these objects for subsequent kinematic analysis of the tracking results. It works by labelling and segmenting CT data for each object, then saving the segmented objects to RAW or TIFF files. Then it produces polygonal surface models from the segmented objects using a marching cubes algorithm, and saves them to OBJ files. The segmented objects are used by X4D to generate DRRs for model-based tracking. The surface models are used by Orient3D to define anatomical reference frames and regions of interest (for distance map calculations), and by Visual3D for kinematic analysis. Surface3D also allows you to identify landmarks (e.g., ligament attachments) and points of interest (e.g., implanted beads) in the image data.

Orient3D

Orient3D prepares surface models created by Surface3D (or third-party software such as Mimics and ScanIP) for use in kinematic analyses after they have been tracked in X4D or Locate3D. With it you can define a local (anatomical) coordinate system (LCS) for each object, landmarks, such as ligament attachment sites, and regions of interest (ROIs), which are used in the calculation of distance maps.

CalibrateDSX

CalibrateDSX prepares the X-ray images and calculates the configuration of the X-ray hardware. It performs three important tasks:

1. correcting the X-ray images (uniformity, distortion, smoothing, resizing)
2. calculating the 3D configuration of the X-ray hardware (the pose of the X-ray sources and camera image planes)
3. calculating the transformation between the X-ray lab frame and the motion capture frame

Locate3D

Locate3D tracks radiopaque beads in X-ray trials. It is most often used to track bones with implanted beads. It is similar to X4D, except that it uses multiple beads to determine the pose of an object, instead of the contours of the object itself. Locate3D requires that the configuration of the X-ray equipment has been calculated (with CalibrateDSX), and that the locations of the beads in the object's CT frame have been determined (with Surface3D).

X4D

X4D tracks 3D objects (bones, implants, etc.) in X-ray images by generating digitally reconstructed radiographs (DRRs) of the objects and matching them to the contours in the X-ray images. It requires that the configuration of the X-ray equipment has been calculated (with CalibrateDSX). It is recommended that the LCSs of the objects to be tracked have been determined (with Orient3D), so there is no need to apply an additional transform, which could have rounding errors. For a validation study, it is required that the implanted beads have been tracked (with Locate3D).

Visual3D

Visual3D is not part of the DSX suite of applications but can be used for:

1. defining the kinematic models used to process the motion capture data.
2. viewing the tracking results.
3. performing kinematic analyses including: joint animations, distance maps, and ligament lengths.