HAS-Motion Software Documentation

User Tools

Site Tools

You are here: Main Page » sift » tutorials » Batch Command Line Processing Tutorial
sift:tutorials:command_line

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
sift:tutorials:command_line [2024/09/19 16:16] – sgrangersift:tutorials:command_line [2025/02/06 16:33] (current) – sgranger
Line 1: Line 1:
 ======Batch Command Line Processing Tutorial====== ======Batch Command Line Processing Tutorial======
-In this tutorial we will learn how to run sift on the command line, loading a library, running queries then running PCA and using that for outlier detection. once we run through the commands step by step we will use them +In this tutorial, we will guide you through running Sift on the command line. You'll learn how to load a library, execute queries, run PCA (Principal Component Analysis), and use it for outlier detection. After walking through each command step by step, we’ll create a batch script to automate these tasks. We'll also include some logic to further streamline the process. To get started, download the sample data folder. {{:automation_tutorial.zip}}
-to create a batch script that can be used to execute them automatically, we will include so logic to automate the process even further.+
  
 =====Running on Command Line===== =====Running on Command Line=====
-  - First we need to run Sift on the command line, on your keyboard press the **Windows Key + R** in the run window type in cmd and press enter, this will open a command line terminal. \\ {{:windows_run_dialog.png}} +  - First, we need to run Sift via the command line. On your keyboard, press Windows Key + R, type "cmd" into the run window, and press Enter to open a command prompt. \\ {{:windows_run_dialog.png}} 
-  - Then change the directory to Sift's install location \\ {{:change_directory.png}} +  - Next, navigate to Sift's installation directory. \\ {{:change_directory.png}} 
-  - Next to launch sift we need to call the .exe and pass the argument --NoGUI \\ {{:launch_sift.png}} +  - To launch Sift, call the .exe file and pass the -NoGUI argument \\ {{:launch_sift.png}} 
-  - This will open another console that we can use to interact with Sift +  - This will open new console where we can interact with Sift 
-  - In the new console we will load a library, enter the command //--LoadLib directory "**Path to downloaded library**"// \\ {{:cl_load_lib.png}} +  - In this console, load a library by entering the command: //-LoadLib directory "**Path to downloaded library**"// \\ {{:cl_load_lib.png}} 
-  - Next load and run a predefined query by entering the command //--LoadQuery file "**Path to query definition file**"// \\ {{:cl_load_query.png}} +  - Next load and run a predefined query by entering: //-LoadQuery file "**Path to query definition file**"// \\ {{:cl_load_query.png}} 
-  - Select all signals by running //--SelectSignals// (Ensure all signal groups and work-spaces have been selected) \\ {{:cl_select_signals.png}} +  - Select all signals by running //-SelectSignals// (Ensure all signal groups and work-spaces have been selected) \\ {{:cl_select_signals.png}} 
-  - Run a PCA with the name "Outlier_PCA" //--RunPCA name "Outlier_PCA"// \\ {{:cl_run_pca.png}} +  - Run a PCA named "Outlier_PCA" //-RunPCA name "Outlier_PCA"// \\ {{:cl_run_pca.png}} 
-  - Now we will run our first round of outlier detection run an Local Outlier Factor test with the command //--RunLOF grouping "group" autoExclude// This will find any group level outliers and exclude them automatically \\ {{:cl_lof_one.png}} +  - Now, let's perform the first round of outlier detection. Run a Local Outlier Factor (LOF) test with the command: //-RunLOF grouping "group" autoExclude// This will find any group level outliers and exclude them automatically \\ {{:cl_lof_one.png}} 
-  - Now lets export the results of this LOF test run the command: //--ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThrehsold// \\ {{:cl_export_group_outliers.png}} +  - To export the results of this LOF test, run: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThrehsold// \\ {{:cl_export_group_outliers.png}} 
-  - Next we will run another LOF test, this time on workspaces: //--RunLOF grouping "workspace" autoExclude// \\ {{:cl_lof_workspace.png}} +  - Next, run another LOF test, this time on workspaces: //-RunLOF grouping "workspace" autoExclude// \\ {{:cl_lof_workspace.png}} 
-  - Now export the results of this LOF test as well: //--ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThreshold// \\ {{:cl_export_workspace_outliers.png}} +  - Export the results of this LOF test as well: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThreshold// \\ {{:cl_export_workspace_outliers.png}} 
-  - Finally, lets save the project so we can view it in the GUI if needed: //--SaveProject file "Path to saved project"// \\ {{:cl_save_project.png}} +  - Finally, save the project to allow for future review in the GUI, if needed: //-SaveProject file "Path to saved project"// \\ {{:cl_save_project.png}} 
-Now our processing is complete, we have two exports detailing group and workspace level outliers and a saved project file that has excluded all of the found outliers. We can use this as a baseline for future processing.+Now that processing is complete, we have two exports detailing group and workspace-level outliers, along with a saved project file that excludes all detected outliers. This can serve as a baseline for future processing.
  
 =====Batch Processing===== =====Batch Processing=====
-Now that we have an understanding of what we want as a baseline for processing, we can take the steps to automate it! +Now that we have a clear understanding of our baseline processing, it's time to automate the workflow!
  
 ====The Scenario==== ====The Scenario====
-In order to fully understand how we should automate a process we need to understand all the details, so lets create a scenario. \\ +To effectively automate a process, we need to understand all the details. Let’s create a scenario: \\ 
-Lets say we are working with a baseball team, and every day at practice we get some data that needs to be processed, that data will be stored in dated folders in a root directory something like this: \\ +Imagine we're working with a baseball team, and each day at practice, we gather data that needs to be processed. This data is stored in date-labeled folders within a root directory, and inside each folder are three trials containing relevant C3D files.
-Within each of these folders are three trials which contain relevant C3Ds \\+
  
 ====The Plan==== ====The Plan====
-Now that we know what we are working with we can think of how to build our batch script, first we will need to check each folder within the root directory, and if the folder does not already contain our exports (.txt files and .i3d) then we know this folder needs to be processed. Below is a batch script that will do just that, you will need to change the path in targetDir as well as the path pointing to the Sift.exe to match your own computer, then save the file as a .bat, to do so you can create a .txt file and then rename it to change the file extension to .bat+Now that we know what we're dealing with, we can plan how to build our batch script. First, we’ll need to check each folder within the root directory. If a folder doesn’t already contain our exports (such as .txt files and .i3d files), we know that folder still needs to be processed. Below is a batch script that accomplishes this. You’ll need to modify the path in **targetDir**, as well as the path pointing to **Sift.exe**, to match your computer's setup. Afterward, save the file as a .bat. To do this, create a .txt file and rename it, changing the extension to .bat.
 <code> <code>
 @echo off @echo off
 setlocal setlocal
  
 +:: Set the target directory where the practice data is stored
 set "targetDir=C:\Users\shane\OneDrive\Documents\practice" set "targetDir=C:\Users\shane\OneDrive\Documents\practice"
  
 +:: Loop through each folder (%%d) in the target directory
 for /D %%d in ("%targetDir%\*") do ( for /D %%d in ("%targetDir%\*") do (
 +    
 +    :: Reset the flag to check if an i3d file exists
     set "i3dFileFound="     set "i3dFileFound="
 +    
 +    :: Loop through each .i3d file in the current folder
     for %%f in ("%%d\*.i3d") do (     for %%f in ("%%d\*.i3d") do (
 +        
 +        :: If an i3d file is found, set the flag to true
         if exist "%%f" (         if exist "%%f" (
             set "i3dFileFound=true"             set "i3dFileFound=true"
Line 43: Line 49:
     )     )
          
 +    :: If no i3d file was found, process the folder
     if not defined i3dFileFound (     if not defined i3dFileFound (
 +        
 +        :: Set the folder path that needs processing
         set "folderWithoutI3d=%%d"         set "folderWithoutI3d=%%d"
-        "C:\Program Files (x86)\Sift\Sift.exe" --NoGui --LoadLib directory "%%d" --LoadQuery file "%%d\query_definition.q3d" --selectSignals --RunPCA name "Outlier_PCA" --RunLOF grouping "group" autoExclude --ExportPCA file "%%d\Group Outliers.txt" exportFormat "transposed" lof lofThreshold --RunLOF grouping "workspace" autoExclude --ExportPCA file "%%d\Workspace Outliers.txt" exportFormat "transposed" lof lofThreshold --SaveProject file "%%d\Outliers.i3d" --Exit+         
 +        :: Run Sift with the necessary commands to process the data 
 +        "C:\Program Files\Sift\Sift.exe" -NoGui -LoadLib directory "%%d" -LoadQuery file "%%d\query_definition.q3d" -selectSignals -RunPCA name "Outlier_PCA" -RunLOF grouping "group" autoExclude -ExportPCA file "%%d\Group Outliers.txt" exportFormat "transposed" lof lofThreshold -RunLOF grouping "workspace" autoExclude -ExportPCA file "%%d\Workspace Outliers.txt" exportFormat "transposed" lof lofThreshold -SaveProject file "%%d\Outliers.i3d" -Exit
     )     )
 ) )
  
 +:: End the script and clean up
 endlocal endlocal
 </code> </code>
  
 ====The Execution==== ====The Execution====
-Now that we have a script ready to go, executing our processing will be as easy as double clicking the .bat file, but we can take the automation even further! Leveraging windows Task Scheduler we can set the batch script to execute at a certain time each day+Now that we have our script ready, running the processing is as simple as double-clicking the .bat file. But we can take the automation a step further! By using Windows Task Scheduler, we can set the batch script to run at a specific time each day automatically.
   - Open the start menu and type Task Scheduler into the search bar \\ {{:cl_search_bar.png?500}}   - Open the start menu and type Task Scheduler into the search bar \\ {{:cl_search_bar.png?500}}
-  - Open Task Scheduler \\ {{:cl_search_scheduler.png?500}} +  - Open Task Scheduler from the search results. \\ {{:cl_search_scheduler.png?500}} 
-  - In the Task Scheduler select create basic task in the top right of the screen \\ {{:cl_scheduler.png?700}}  +  - In Task Scheduler, select **Create Basic Task** from the top-right of the screen. \\ {{:cl_scheduler.png?700}}  
-  - In the popup dialog enter a meaningful name in this case we went with "Baseline Processing" press next \\ {{:cl_name_task.png?700}} +  - In the popup dialog, enter a meaningful name (in this case, we used "Baseline Processing") and press **Next**. \\ {{:cl_name_task.png?700}} 
-  - In the next step ensure daily is selected and press next \\ {{:cl_daily_task.png?700}} +  - On the next screen, select **Daily** and press **Next**. \\ {{:cl_daily_task.png?700}} 
-  - Next select what time of day you would like the task to run, we picked 12:00AM \\ {{:cl_task_time.png?700}} +  - Now, choose the time of day you'd like the task to run. We selected 12:00 AM. \\ {{:cl_task_time.png?700}} 
-  - In the next step ensure start a program is selected and press next \\ {{:cl_start_program.png?700}} +  - Ensure Start a Program is selected on the next screen and press **Next**. \\ {{:cl_start_program.png?700}} 
-  - Now we will browse and select the batch script we created \\ {{:cl_pick_script.png?700}} +  - Browse and select the batch script you created earlier. \\ {{:cl_pick_script.png?700}} 
-  - On the final screen review and press finish \\ {{:cl_finish_task.png?700}} +  - On the final screen, review your settings and press **Finish**. \\ {{:cl_finish_task.png?700}} 
-  - Finally check the existing task to ensure it has been created successfully \\ {{:cl_all_done.png?700}} +  - Lastly, check the list of existing tasks to confirm it was created successfully. \\ {{:cl_all_done.png?700}} 
-Now we are finished! We have set it up so our script will execute every night at 12:00AM with no need for any user input at all! +That's it! Your script is now set to run every night at 12:00 AM automatically, with no further input required from you.
sift/tutorials/command_line.1726762585.txt.gz · Last modified: 2024/09/19 16:16 by sgranger