======Batch Command Line Processing Tutorial======
In this tutorial, we will guide you through running Sift on the command line. You'll learn how to load a library, execute queries, run PCA (Principal Component Analysis), and use it for outlier detection. After walking through each command step by step, we’ll create a batch script to automate these tasks. We'll also include some logic to further streamline the process. To get started, download the sample data folder. {{:automation_tutorial.zip}}

=====Running on Command Line=====
  - First, we need to run Sift via the command line. On your keyboard, press Windows Key + R, type "cmd" into the run window, and press Enter to open a command prompt. \\ {{:windows_run_dialog.png}}
  - Next, navigate to Sift's installation directory. \\ {{:change_directory.png}}
  - To launch Sift, call the .exe file and pass the -NoGUI argument \\ {{:launch_sift.png}}
  - This will open new console where we can interact with Sift
  - In this console, load a library by entering the command: //-LoadLib directory "**Path to downloaded library**"// \\ {{:cl_load_lib.png}}
  - Next load and run a predefined query by entering: //-LoadQuery file "**Path to query definition file**"// \\ {{:cl_load_query.png}}
  - Select all signals by running //-SelectSignals// (Ensure all signal groups and work-spaces have been selected) \\ {{:cl_select_signals.png}}
  - Run a PCA named "Outlier_PCA" //-RunPCA name "Outlier_PCA"// \\ {{:cl_run_pca.png}}
  - Now, let's perform the first round of outlier detection. Run a Local Outlier Factor (LOF) test with the command: //-RunLOF grouping "group" autoExclude// This will find any group level outliers and exclude them automatically \\ {{:cl_lof_one.png}}
  - To export the results of this LOF test, run: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThrehsold// \\ {{:cl_export_group_outliers.png}}
  - Next, run another LOF test, this time on workspaces: //-RunLOF grouping "workspace" autoExclude// \\ {{:cl_lof_workspace.png}}
  - Export the results of this LOF test as well: //-ExportPCA file "Path to exported txt file" exportFormat "transposed" LOF LOFThreshold// \\ {{:cl_export_workspace_outliers.png}}
  - Finally, save the project to allow for future review in the GUI, if needed: //-SaveProject file "Path to saved project"// \\ {{:cl_save_project.png}}
Now that processing is complete, we have two exports detailing group and workspace-level outliers, along with a saved project file that excludes all detected outliers. This can serve as a baseline for future processing.

=====Batch Processing=====
Now that we have a clear understanding of our baseline processing, it's time to automate the workflow!

====The Scenario====
To effectively automate a process, we need to understand all the details. Let’s create a scenario: \\
Imagine we're working with a baseball team, and each day at practice, we gather data that needs to be processed. This data is stored in date-labeled folders within a root directory, and inside each folder are three trials containing relevant C3D files.

====The Plan====
Now that we know what we're dealing with, we can plan how to build our batch script. First, we’ll need to check each folder within the root directory. If a folder doesn’t already contain our exports (such as .txt files and .i3d files), we know that folder still needs to be processed. Below is a batch script that accomplishes this. You’ll need to modify the path in **targetDir**, as well as the path pointing to **Sift.exe**, to match your computer's setup. Afterward, save the file as a .bat. To do this, create a .txt file and rename it, changing the extension to .bat.
<code>
@echo off
setlocal

:: Set the target directory where the practice data is stored
set "targetDir=C:\Users\shane\OneDrive\Documents\practice"

:: Loop through each folder (%%d) in the target directory
for /D %%d in ("%targetDir%\*") do (
    
    :: Reset the flag to check if an i3d file exists
    set "i3dFileFound="
    
    :: Loop through each .i3d file in the current folder
    for %%f in ("%%d\*.i3d") do (
        
        :: If an i3d file is found, set the flag to true
        if exist "%%f" (
            set "i3dFileFound=true"
        )
    )
    
    :: If no i3d file was found, process the folder
    if not defined i3dFileFound (
        
        :: Set the folder path that needs processing
        set "folderWithoutI3d=%%d"
        
        :: Run Sift with the necessary commands to process the data
        "C:\Program Files (x86)\Sift\Sift.exe" -NoGui -LoadLib directory "%%d" -LoadQuery file "%%d\query_definition.q3d" -selectSignals -RunPCA name "Outlier_PCA" -RunLOF grouping "group" autoExclude -ExportPCA file "%%d\Group Outliers.txt" exportFormat "transposed" lof lofThreshold -RunLOF grouping "workspace" autoExclude -ExportPCA file "%%d\Workspace Outliers.txt" exportFormat "transposed" lof lofThreshold -SaveProject file "%%d\Outliers.i3d" -Exit
    )
)

:: End the script and clean up
endlocal
</code>

====The Execution====
Now that we have our script ready, running the processing is as simple as double-clicking the .bat file. But we can take the automation a step further! By using Windows Task Scheduler, we can set the batch script to run at a specific time each day automatically.
  - Open the start menu and type Task Scheduler into the search bar \\ {{:cl_search_bar.png?500}}
  - Open Task Scheduler from the search results. \\ {{:cl_search_scheduler.png?500}}
  - In Task Scheduler, select **Create Basic Task** from the top-right of the screen. \\ {{:cl_scheduler.png?700}} 
  - In the popup dialog, enter a meaningful name (in this case, we used "Baseline Processing") and press **Next**. \\ {{:cl_name_task.png?700}}
  - On the next screen, select **Daily** and press **Next**. \\ {{:cl_daily_task.png?700}}
  - Now, choose the time of day you'd like the task to run. We selected 12:00 AM. \\ {{:cl_task_time.png?700}}
  - Ensure Start a Program is selected on the next screen and press **Next**. \\ {{:cl_start_program.png?700}}
  - Browse and select the batch script you created earlier. \\ {{:cl_pick_script.png?700}}
  - On the final screen, review your settings and press **Finish**. \\ {{:cl_finish_task.png?700}}
  - Lastly, check the list of existing tasks to confirm it was created successfully. \\ {{:cl_all_done.png?700}}
That's it! Your script is now set to run every night at 12:00 AM automatically, with no further input required from you.