The Math of Principal Component Analysis (PCA)

note: this page is under construction

New technology allows researchers to generate large quantities of data, whether it be time-series data, or any other waveform data that we commonly see in biomechanics research. And what is common amongst all these forms of data is that they are highly dimensional [1]. With new technologies like [Markerless], we are collecting so much more data that trying to make meaningful comparisons is difficult to understand and to do computationally.

In the research methods in biomechanics text, it is emphasized that the basic point of looking at methods of processing waveform data is that no matter the type of research questions that a biomechanist is trying to answer, transforming our data into smaller forms makes it more manageable and useful for analysis. Prior to being able to make meaningful comparisons, we want to reduce the amount of data and not lose any important discriminatory information that may be crucial to our comparisons [1].

Traditional approaches of extracting features from waveform data (max, min, range, etc.) result in a lose of so much temporal information in the waveform, leading to such a large reduction in data and discarded information. Additionally, varying definitions of these discrete parameters across the field lead to inconsistent conclusions, and these measures may not always be definitive in certain populations [1]. One such example mentioned in the text found that knee adduction angles in patients with OA don’t have definitive peaks, and thus finding max and min angles is meaningless [2].

PCA

PCA is an orthogonal decomposition technique that computes and extracts a unique set of basis functions from the waveforms based on the variation that is present in the waveform data [1].

Waveform Data

Waveform data can be represented in matrix form , where the rows are the time series waveforms for each subject, and the columns are particular points within the waveform (n = subjects, p = points) [1].

Covariance Matrix

To define variation in the waveform matrix, X, based on how the data changes over time, and between subjects (n) at each point in time (p), we look at the covariance matrix, S. The diagonal elements of the covariance matrix are the variation at each instance in the data. The off-diagonal elements represent the variation in each pair of instances. If the covariance values are non-zero, this means there is a linear relationship between the variables. If the off-diagonal values of S are non-zero, this means that the columns of the original data X are correlated. However, principal components are orthogonal, independent features of the original data, and thus we look for principal components that have a covariance matrix with off-diagonal components equal to zero [1].

To transform the original data set covariance matrix, S, into a principal component covariance matrix, D, diagonalization is used, where U is an orthogonal transformation matrix. U transforms the data into the new coordinate system, which are the coordinates of the principal components [1].

U'SU = D

Resources

[1] Robertson G., Caldwell G., Hamill J., Kamen G., Whittlesey S., Research Methods in Biomechanics Textbook, 2nd edition ([1])

[2] Deluzio KJ and Astephen JL (2007) Biomechanical features of gait waveform data associated with knee osteoarthritis. An application of principal component anslysis. Gait & Posture 23. 86-93 (pdf)