Differences

This shows you the differences between two versions of the page.

--- sift:tutorials:command_line [2024/09/19 13:31] – sgranger
+++ sift:tutorials:command_line [2025/02/06 16:33] (current) – sgranger
@@ Line 1: / Line 1: @@
 ======Batch Command Line Processing Tutorial======
-In this tutorial we will learn how to run sift on the command line, loading a library, running queries then running PCA and using that for outlier detection. once we run through the commands step by step we will use them
+In this tutorial, we will guide you through running Sift on the command line. You'll learn how to load a library, execute queries, run PCA (Principal Component Analysis), and use it for outlier detection. After walking through each command step by step, we’ll create a batch script to automate these tasks. We'll also include some logic to further streamline the process. To get started, download the sample data folder. {{:automation_tutorial.zip}}
-to create a batch script that can be used to execute them automatically, we will include so logic to automate the process even further.
 =====Running on Command Line=====
-  - First we need to run Sift on the command line, on your keyboard press the **Windows Key + R** in the run window type in cmd and press enter, this will open a command line terminal. \\ {{:windows_run_dialog.png}}
+  - First, we need to run Sift via the command line. On your keyboard, press Windows Key + R, type "cmd" into the run window, and press Enter to open a command prompt. \\ {{:windows_run_dialog.png}}
-  - Then change the directory to Sift's install location \\ {{:change_directory.png}}
+  - Next, navigate to Sift's installation directory. \\ {{:change_directory.png}}
-  - Next to launch sift we need to call the .exe and pass the argument --NoGUI \\ {{:launch_sift.png}}
+  - To launch Sift, call the .exe file and pass the -NoGUI argument \\ {{:launch_sift.png}}
-  - This will open another console that we can use to interact with Sift
+  - This will open new console where we can interact with Sift
-  - In the new console we will load a library, enter the command //--LoadLib directory "**Path to downloaded library**"// \\ {{:cl_load_lib.png}}
+  - In this console, load a library by entering the command: //-LoadLib directory "**Path to downloaded library**"// \\ {{:cl_load_lib.png}}
-  - Next load and run a predefined query by entering the command //--LoadQuery file "**Path to query definition file**"// \\ {{:cl_load_query.png}}
+  - Next load and run a predefined query by entering: //-LoadQuery file "**Path to query definition file**"// \\ {{:cl_load_query.png}}
-  - Select all signals by running //--SelectSignals// (Ensure all signal groups and work-spaces have been selected) \\ {{:cl_select_signals.png}}
+  - Select all signals by running //-SelectSignals// (Ensure all signal groups and work-spaces have been selected) \\ {{:cl_select_signals.png}}
-  - Run a PCA with the name "Outlier_PCA" //--RunPCA name "Outlier_PCA"// \\ {{:cl_run_pca.png}}
+  - Run a PCA named "Outlier_PCA" //-RunPCA name "Outlier_PCA"// \\ {{:cl_run_pca.png}}
-  - Now we will run our first round of outlier detection run an Local Outlier Factor test with the command //--RunLOF grouping "group" autoExclude// This will find any group level outliers and exclude them automatically \\ {{:cl_lof_one.png}}
+  - Now, let's perform the first round of outlier detection. Run a Local Outlier Factor (LOF) test with the command: //-RunLOF grouping "group" autoExclude// This will find any group level outliers and exclude them automatically \\ {{:cl_lof_one.png}}
-  - Now lets export the results of this LOF test run the command: //--ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThrehsold// \\ {{:cl_export_group_outliers.png}}
+  - To export the results of this LOF test, run: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThrehsold// \\ {{:cl_export_group_outliers.png}}
+  - Next, run another LOF test, this time on workspaces: //-RunLOF grouping "workspace" autoExclude// \\ {{:cl_lof_workspace.png}}
+  - Export the results of this LOF test as well: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThreshold// \\ {{:cl_export_workspace_outliers.png}}
+  - Finally, save the project to allow for future review in the GUI, if needed: //-SaveProject file "Path to saved project"// \\ {{:cl_save_project.png}}
+Now that processing is complete, we have two exports detailing group and workspace-level outliers, along with a saved project file that excludes all detected outliers. This can serve as a baseline for future processing.
+=====Batch Processing=====
+Now that we have a clear understanding of our baseline processing, it's time to automate the workflow!
+====The Scenario====
+To effectively automate a process, we need to understand all the details. Let’s create a scenario: \\
+Imagine we're working with a baseball team, and each day at practice, we gather data that needs to be processed. This data is stored in date-labeled folders within a root directory, and inside each folder are three trials containing relevant C3D files.
+====The Plan====
+Now that we know what we're dealing with, we can plan how to build our batch script. First, we’ll need to check each folder within the root directory. If a folder doesn’t already contain our exports (such as .txt files and .i3d files), we know that folder still needs to be processed. Below is a batch script that accomplishes this. You’ll need to modify the path in **targetDir**, as well as the path pointing to **Sift.exe**, to match your computer's setup. Afterward, save the file as a .bat. To do this, create a .txt file and rename it, changing the extension to .bat.
+<code>
+@echo off
+setlocal
+:: Set the target directory where the practice data is stored
+set "targetDir=C:\Users\shane\OneDrive\Documents\practice"
+:: Loop through each folder (%%d) in the target directory
+for /D %%d in ("%targetDir%\*") do (
+    :: Reset the flag to check if an i3d file exists
+    set "i3dFileFound="
+    :: Loop through each .i3d file in the current folder
+    for %%f in ("%%d\*.i3d") do (
+        :: If an i3d file is found, set the flag to true
+        if exist "%%f" (
+            set "i3dFileFound=true"
+        )
+    )
+    :: If no i3d file was found, process the folder
+    if not defined i3dFileFound (
+        :: Set the folder path that needs processing
+        set "folderWithoutI3d=%%d"
+        :: Run Sift with the necessary commands to process the data
+        "C:\Program Files\Sift\Sift.exe" -NoGui -LoadLib directory "%%d" -LoadQuery file "%%d\query_definition.q3d" -selectSignals -RunPCA name "Outlier_PCA" -RunLOF grouping "group" autoExclude -ExportPCA file "%%d\Group Outliers.txt" exportFormat "transposed" lof lofThreshold -RunLOF grouping "workspace" autoExclude -ExportPCA file "%%d\Workspace Outliers.txt" exportFormat "transposed" lof lofThreshold -SaveProject file "%%d\Outliers.i3d" -Exit
+    )
+)
+:: End the script and clean up
+endlocal
+</code>
+====The Execution====
+Now that we have our script ready, running the processing is as simple as double-clicking the .bat file. But we can take the automation a step further! By using Windows Task Scheduler, we can set the batch script to run at a specific time each day automatically.
+  - Open the start menu and type Task Scheduler into the search bar \\ {{:cl_search_bar.png?500}}
+  - Open Task Scheduler from the search results. \\ {{:cl_search_scheduler.png?500}}
+  - In Task Scheduler, select **Create Basic Task** from the top-right of the screen. \\ {{:cl_scheduler.png?700}}
+  - In the popup dialog, enter a meaningful name (in this case, we used "Baseline Processing") and press **Next**. \\ {{:cl_name_task.png?700}}
+  - On the next screen, select **Daily** and press **Next**. \\ {{:cl_daily_task.png?700}}
+  - Now, choose the time of day you'd like the task to run. We selected 12:00 AM. \\ {{:cl_task_time.png?700}}
+  - Ensure Start a Program is selected on the next screen and press **Next**. \\ {{:cl_start_program.png?700}}
+  - Browse and select the batch script you created earlier. \\ {{:cl_pick_script.png?700}}
+  - On the final screen, review your settings and press **Finish**. \\ {{:cl_finish_task.png?700}}
+  - Lastly, check the list of existing tasks to confirm it was created successfully. \\ {{:cl_all_done.png?700}}
+That's it! Your script is now set to run every night at 12:00 AM automatically, with no further input required from you.