===== Pipeline Overview =====

The Pipeline is a set of Visual3D commands that are processed in sequence. The Pipeline processor provides access to the core of Visual3D functionality through commands with each of Visual3D's functions represented by a command.

**Note:** The pipeline is a **COMMAND** language, not a **SCRIPTING language**.

The Pipeline is typically used to automate processing steps. It has the ability to manage files, define events, execute signal processing computations, create and edit modes, create and modify reports, and generated statistics. Edit boxes are usually provided for configuring each process step. The pipeline processor (or any text processor) can be used to create the pipeline. Any pipeline command that can be run interactively through the pipeline processor may be saved to a text command file. This file may be reloaded later, or combined with other command files to build a more complex pipeline. The pipeline processor can be launched from the Visual3D tool bar or from the Pipeline menu.

{{:Tutorial4_1.jpg}}

A common use case for the Visual3D Pipeline is to define the processing steps for a given experimental data analysis protocol. This allows, for example, the director of the laboratory to control the processing that is done by a member of the laboratory. This may be essential for some experimental studies that require that all of the data analysis be performed consistently.

If this is your first introduction to the Visual3D Pipeline then we recommend that you work through our [[Visual3D:Tutorials:Pipeline:Command_Pipeline_|detailed, introductory tutorial]] to teach you how to use the Pipeline and introduce you to pipeline commands.

==== Pipeline Commands ====

Pipeline commands are divided into various categories: File, Signal, Event, etc. Commands under these folders are added to the pipeline by expanding the folder for a category, and selecting the function to be added by clicking the "Add" button. The selected function will be added to the pipeline immediately after the command that is currently highlighted in the pipeline. Commands can be repositioned in the pipeline using the arrow keys to the right of the list box. For details of specific commands that can be used through the Visual3D pipeline, see: **[[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Commands_Reference|Pipeline Commands Reference]]**

=== Pipeline Commands: Active Files ===

Pipeline commands usually process the [[Visual3D:Documentation:Definitions:Active_Files|Active Files]]; e.g. those files active in the Visual3D Workspace. If only one file is selected in the file selection box, the pipeline, when executed, will perform actions only on the data from that file. If ALL_FILES is selected, the script will perform actions on every file individually, as if the script had been run sequentially for each file.

=== Pipeline Commands: Syntax ===

Each Pipeline command consists of two parts - the command and its parameters.

<code>
Command_Name 
/Parameter1= something 
/Parameter2= this_value+another_value 
/Parameter3= 3.14 
!/Parameter4= default_value_for_Parameter4
;
</code>

^Special Character ^ Meaning ^
| / | Each parameter begins with a forward slash. |
| ; | The semi-colon character terminates a command. |
| ! | The exclamation point indicates that the remainder of the line is a comment. |
| + | The plus sign is a delimeter separating entries. For example, 1+2 indicates that the variable has two values (1 and 2). |
| & | The ampersand indicates a concatenation. For example, this&that will be interpreted in the pipeline as thisthat. |

If the "!" character is at the beginning of a line, the line is ignored, and the default values for the parameter are used in the processing. The script text is provided so that users are aware of the default values and can easily uncomment these optional parameters and pass non-default values.

To pass more than one set of parameters to the command, list the values of each parameter, separated by "+" signs. The generalized command above passes one set of three parameters to the command "Command_Name".

=== Pipeline Commands: Specifying Signals in a Command ===

Signals in Visual3D must be identified by 3 names, according to that signal's location in the [[visual3d:documentation:visual3d_signal_types:data_tree|data tree]]:
  - **Signal_Type - Target, Analog, etc.**
  - **Signal_Folder - Original, Processed, etc.**
  - **Signal_Name - Signal Name**

Signals can be explicitly represented by the following three parameters:

<code>
/Signal_Types= The types of the signals
/Signal_Names= The names of the signals
/Signal_Folder= The name of the signal folder
</code>

If only the signal type is specified then all signals of that type (and the default folder) are implicitly selected for use in the command. For legacy support (e.g. Visual3D version 1.0), if the SIGNAL_FOLDER is empty it defaults to ORIGINAL. For example: to apply a lowpass filter to all TARGETS. This is accomplished by specifying the SIGNAL_TYPE and SIGNAL_FOLDER but leaving SIGNAL_NAMES empty.

<code>
Lowpass_Filter
/SIGNAL_TYPES=TARGET
/SIGNAL_FOLDER=ORIGINAL
/SIGNAL_NAMES=
/RESULT_FOLDER=PROCESSED
/RESULT_SUFFIX=
/FILTER_CLASS=BUTTERWORTH
/FREQUENCY_CUTOFF=6.0
/NUM_REFLECTED=6
/NUM_EXTRAPOLATED=0
/TOTAL_BUFFER_SIZE=6
/NUM_BIDIRECTIONAL_PASSES=1
;
</code>

Some commands act on signals (Target, Analog, etc). You can select a signal name for the command to work on by checking the signals in the data tree (**Signal and Event** processing mode) and pressing the **Import Signals from Data Tree** button. This will add the signal information (Type,Name,Folder) to the command. Many of the commands that act on signals will not show the signals in the dialog. This can be a bit of an annoyance when there are only a few signals selected, but often commands contain so many signals that the list box in the dialog will become meaningless because of all the signals. To make it easier to enter the signal types, names, and folders into a command, there is a button to add all "checked" signals from the data tree.

Alternatively, for some commands signals are represented using the pipeline parameter syntax from the [[Visual3D:Documentation:Pipeline:Expressions:Expressions_Overview|Evaluate_Expression]] command. For example, to identify the RFT1 signal we use the syntax TYPE::FOLDER::NAME

<code>
TARGET::PROCESSED::RFT1
</code>

==== Pipeline Parameters ====

An important feature of the pipeline is the ability to create and use [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Parameters|Pipeline Parameters]], which is a way to store a text string for use in pipeline commands.

In one sense it is similar to specifying a global variable, such as body weight, that could be used in computations. Within the Visual3D pipeline, however, it is actually much more flexible since pipeline commands permit multiple entries on a single line. Since the entire line can be represented as a string, a single Pipeline Parameter can represent multiple entries.

==== Editing Pipeline Scripts ====

Pipeline commands in a script may be edited in 3 ways:

  - Using the built-in text editor (select the **TEXT** Button)
  - Using a dialog (select the **EDIT button**). Most, but not all Visual3D commands have an edit dialog. If no dialog exists, then the text editor appears.
  - Using an independent text editor such as [[https://notepad-plus-plus.org/|Notepad++]].

==== Pipeline Control ====

=== RECALC Pipeline ===

The [[Visual3D:Documentation:Pipeline:General_Information:RECALC_Pipeline|RECALC]] button updates the Visual3D Workspace by performing the following:

  - If "use processed analog" is set, re-process force platforms.
  - Rebuild all models in the Workspace
  - Compute landmarks and model pose (e.g. position and orientation of all segments) for all #files in the Workspace
  - Assign Force Platforms to Segments
  - Compute Assigned Forces
  - Compute Kinematics
  - Compute Kinetics
  - Compute Center of Mass.
  - Execute the RECALC pipeline
  - Update the display dialogs for any new data
  - Recompute the Report graphs

=== Pipeline Favorites ===

The [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Favorites|Pipeline Commands:Pipeline Favorites]] feature allows you to list your pipeline scripts as a Table of Contents. Your list of pipeline scripts can be viewed or run from the 'Pipeline Favorites' list box.

=== Launching Visual3D & Pipeline Externally ===

Visual3D can be launched from a [[visual3d:getting_started:commandline|DOS command line]] (or an external program).

If you have a DOS PATH set to the Visual3D directory you can type

<code>
Visual3D.exe
</code>

If not, you must specify the full path to Visual3D.

<code>
c:\Program Files\Visual3D\Visual3D.exe
</code>

It is possible to have a pipeline executed immediately on launching Visual3D using the following syntax.

<code>
Visual3D.exe /s c:\exampleData\test.v3s
</code>

Note that the full path to the pipeline file is required unless the DOS PATH to this folder is specified.

==== Creating a Pipeline ====

The .v3s file is an ASCII file that can be saved to disk and can be edited with any common word processing program such as [[visual3d:documentation:pipeline:general_information:notepad|Notepad++]]. When Visual3D loads the .v3s file it parses this ASCII file into the individual commands that are displayed in the pipeline dialog.

Many of the pages in this wiki pages contain pipelines or snippets of pipelines, with a note saying that the text can be copied and pasted into a file. For example,

<code>
! Starting Fresh - Clear Workspace
File_New 
;

! Create a Hybrid model.
! Prompt for the Standing Trial to use... 
Create_Hybrid_Model 
! /CALIBRATION_FILE= 
; 

! Load the Model Template (Segment Definitions)
Apply_Model_Template 
! /MODEL_TEMPLATE= 
! /CALIBRATION_FILE= 
;

! Prompt for the Movement data files. 
! Multiple files can be selected in the dialog file listing using CTRL-Click 
Open_File
! /FILE_NAME= 
;

! Assigning the Movement files to the model 
! Pop up the dialog box...  
Assign_Model_File
! /CALIBRATION_FILE= 
! /MOTION_FILE_NAMES= 
;
</code>

**The Process for creating a pipeline from the above text.**

  - Select the pipeline text above, with the right mouse button on the highlighted text select copy.
  - Open Notepad
  - Paste into Notepad
  - Save the Notepad file as TestPipeline.v3s
  - This command can now be opened in the Visual3D pipeline dialog

==== Pipeline Examples ====

See the [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Examples|Pipeline Examples]] page for a collection of pipeline processing examples.