Differences

This shows you the differences between two versions of the page.

--- sift:tutorials:using_directory_watchers [2025/02/06 17:52] – created sgranger
+++ sift:tutorials:using_directory_watchers [2025/02/06 19:11] (current) – sgranger
@@ Line 1: / Line 1: @@
 ===== Automating Work Flow With Directory Watchers =====
 In this tutorial we will walk you through the steps of setting up Sift to automatically process your data when it is loaded. You will learn how to create a directory watcher and point it to a folder your data will be dropped in, then set it to execute all your V3D and Sift processing automatically
+===== The Task =====
+Before we can automate something we need to have an idea of what we are trying to do in the first place, in this example we will take a treadmill running trial captured with Theia, and ultimately run SPM (Statistical Parametric Mapping) to note any significant differences between the left and right knee angles.
+To start download this zip file, it contains everything needed to get this example working.
+===== Setting Up =====
+==== V3D Pipeline ====
+Before we create the watchers themselves we need to inspect the scripts that will be fired by them, first up is the V3D pipeline script "V3D_Pipeline.v3s" open this up. This script is very basic, it opens the c3d included calculates the left and right knee angles, then manually sets some events before saving as a CMZ.
+We do have to make one small alteration, we must set the paths within the pipeline to match our computer replace all occurrences of [[REPLACE]] with the path to the unzipped "Watcher Demo" folder in my case "C:\Users\shane\OneDrive\Desktop\Testing\Watcher Demo", this needs to be changed in the File_Open and File_Save commands
+<code>
+File_Open
+/FILE_NAME=[[REPLACE]]\Watcher Demo\Data\DE3_1_pose_0.c3d
+/FILE_PATH=[[REPLACE]]\Watcher Demo\Data\
+! /SEARCH_SUBFOLDERS=FALSE
+! /SUFFIX=
+! /SET_PROMPT=File_Open
+! /ON_FILE_NOT_FOUND=PROMPT
+! /FILE_TYPES_ON_PROMPT=
+;
+Compute_Model_Based_Data
+/RESULT_NAME=RKneeAngle
+/SUBJECT_TAG=ALL_SUBJECTS
+/FUNCTION=JOINT_ANGLE
+/SEGMENT=RTH
+/REFERENCE_SEGMENT=RSK
+/RESOLUTION_COORDINATE_SYSTEM=
+! /USE_CARDAN_SEQUENCE=FALSE
+! /NORMALIZATION=FALSE
+! /NORMALIZATION_METHOD=
+! /NORMALIZATION_METRIC=
+/NEGATEX=TRUE
+/NEGATEY=TRUE
+/NEGATEZ=TRUE
+! /AXIS1=X
+! /AXIS2=Y
+! /AXIS3=Z
+! /TREADMILL_DATA=FALSE
+! /TREADMILL_DIRECTION=UNIT_VECTOR(0,1,0)
+! /TREADMILL_SPEED=0.0
+;
+Compute_Model_Based_Data
+/RESULT_NAME=LKneeAngle
+/SUBJECT_TAG=ALL_SUBJECTS
+/FUNCTION=JOINT_ANGLE
+/SEGMENT=LTH
+/REFERENCE_SEGMENT=LSK
+/RESOLUTION_COORDINATE_SYSTEM=
+! /USE_CARDAN_SEQUENCE=FALSE
+! /NORMALIZATION=FALSE
+! /NORMALIZATION_METHOD=
+! /NORMALIZATION_METRIC=
+! /NEGATEX=FALSE
+! /NEGATEY=FALSE
+! /NEGATEZ=FALSE
+! /AXIS1=X
+! /AXIS2=Y
+! /AXIS3=Z
+! /TREADMILL_DATA=FALSE
+! /TREADMILL_DIRECTION=UNIT_VECTOR(0,1,0)
+! /TREADMILL_SPEED=0.0
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=200
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=350
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=505
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=650
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=810
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=950
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=1110
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=120
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=RHS
+/FRAME=50
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=260
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=425
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=580
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=730
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=880
+! /TIME=
+;
+Event_Explicit
+/EVENT_NAME=LHS
+/FRAME=1030
+! /TIME=
+;
+File_Save_As
+/FILE_NAME=DE3
+/FOLDER=[[REPLACE]]\Watcher Demo\CMZs
+! /SET_PROMPT=Save CMZ file as
+! /SAVE_EMBEDDED_GRAPHICS=FALSE
+! /CREATE_FOLDER_PATH=FALSE
+;
+</code>
+==== Sift BATCH Script ====
+The BATCH script "Sift_Script.bat" does not need to be updated, the script gets a reference to its location itself (the %~dp0 you see in the script).
+There are a couple things to note about the script, the first line is ::SIFT, a .bat file will only be executed if this line is present, this prevents a unintended script from being executed by accident, you will also notice that the entire sift command is on one line. This is because they all need to be passed at once via a command line call and new line characters will execute everything preceding it and treat everything remaining as a brand new command to make this easier to manage you can enable wordwrap on your text editor of choice.
+This script will query the left and right knee angles, conduct some outlier detection to clean the data, register signals based on their maximum value, and then perform SPM.
+For more details on creating a .bat file for command line processing in Sift check out the [[sift:tutorials:command_line|Batch Processing Tutorial]].
+<code>
+::SIFT
+"%ProgramFiles%\Sift\Sift.exe" -LoadLib "%~dp0\CMZs" -LoadQuery "%~dp0\Sift_Query.q3d" -SelectSignals -RunPCA "KneeAnglePCA" -RunLOF grouping "group" -exportPCA file "%~dp0\Exports\Knee Angle - LOF Outliers.txt" lof lofThreshold exportFormat "transposed" -SelectSignals group "RKneeAngleX" -RegisterCurves name "RKneeAngleX_Reg" max -SelectSignals group "LKneeAngleX" -RegisterCurves name "LKneeAngleX_Reg" max -SelectSignals group "RKneeAngleY" -RegisterCurves name "RKneeAngleY_Reg" max -SelectSignals group "LKneeAngleY" -RegisterCurves name "LKneeAngleY_Reg" max -SelectSignals group "RKneeAngleZ" -RegisterCurves name "RKneeAngleZ_Reg" max -SelectSignals group "LKneeAngleZ" -RegisterCurves name "LKneeAngleZ_Reg" max -SelectSignals -CreateGLM name "KneeAngle_GLM" -RunSPM name "KneeAngleX_SPM" glm "KneeAngle_GLM" group1 "RKneeAngleX_Reg" group2 "LKneeAngleX_Reg" -RunSPM name "KneeAngleY_SPM" glm "KneeAngle_GLM" group1 "RKneeAngleY_Reg" group2 "LKneeAngleY_Reg" -RunSPM name "KneeAngleZ_SPM" glm "KneeAngle_GLM" group1 "RKneeAngleZ_Reg" group2 "LKneeAngleZ_Reg" -ExportSPM file "%~dp0\Exports\KneeAngle_SPM.txt" spm exportFormat "transposed" -SaveProject "%~dp0\Exports\Knee Angle.i3d" -Exit
+</code>
+==== Creating Watchers ====
+To create the watchers themselves we will need to do so within the Sift Command Line, to launch Sift on the command line, first open a command line tutorial you can do this by pressing **Windows Key + ** on your keyboard and typing **cmd** into the window that pops up \\
+{{:windows_run_dialog.png}} \\
+then navigate to the Sift install directory by typing **cd C:\Program Files\Sift** this path may be different depending on your specific computer \\
+{{:change_directory.png}} \\
+Now that we are in the install directory we type **Sift.exe -NoGUI** this will open a command line instance of Sift \\
+{{:launch_sift.png}}
+In the Sift Command Line we will create our first watcher this one will point to the V3D Pipeline \\
+**-CreateWatcher name "V3D_Watcher" directory "C:\Users\shane\OneDrive\Desktop\Testing\Watcher Demo\Watcher Demo\Data" script "C:\Users\shane\OneDrive\Desktop\Testing\Watcher Demo\Watcher Demo\V3D_Pipeline.v3s"** \\
+Lets break down the command the **name** parameter is how we will refer to the watcher the **directory** parameter will be the one watched, in this case we want to watch the Data folder, and finally the **script** parameter is what we want to execute when the watcher is triggered in this case the V3D pipeline " <sub>Tip: if you want to quickly get the path to a file or folder click on it and press **Ctrl + Shift + C**</sub> \\
+{{watcher_v3d.png}} \\
+Now we will create the second watcher, this time for the Sift script \\
+**-CreateWatcher name "Sift_Watcher" directory "C:\Users\shane\OneDrive\Desktop\Testing\Watcher Demo\Watcher Demo\CMZs" script "C:\Users\shane\OneDrive\Desktop\Testing\Watcher Demo\Watcher Demo\Sift_Script.bat"** \\
+{{watcher_sift.png}} \\
+now that both watchers have been created lets run **-ListWatchers** just to be sure that everything is correct, the output should look something like this: \\
+{{sift_rename_watchers.png}} \\
+With our watchers made we must set Sift to run in the background (Watchers will only trigger when Sift is running in the background) enter **-LaunchBackgroundInstance** this will immediately launch an instance of Sift in the background, we can interact with it in the system tray icon. Before moving on be sure to exit the open Sift command line console, only one instance can be open at a time and the watcher script will need to open one when executing the BATCH file.\\
+{{sift_launch_bg.png}} \\
+Here we can see what watchers we have made and even enable or disable them
+<sub>Tip: You can set Sift the launch to the system tray by default if you pass the command **-LaunchMinimized**</sub> \\
+{{sift_tray.png}}
+===== Using The Watchers =====
+Now that everything is set up for us all that is left to do is trigger the watchers! To do this simply drag and drop the C3D "DE3_1_pose_0.c3d" into the Data folder, this will trigger the **V3D_Watcher** which will output the CMZ to the CMZs folder, the **Sift_Watcher** is watching this folder, so when the CMZ is placed within it it will begin to execute the Sift_Script. This will ultimatly export everything to the Exports folder. \\
+\\
+That's everything, with some simple set up, you can fully automate the entirety of you processing pipeline!