Differences

This shows you the differences between two versions of the page.

--- visual3d:documentation:pipeline:pipeline_overview [2024/06/19 12:49] – sgranger
+++ visual3d:documentation:pipeline:pipeline_overview [2024/10/30 14:19] (current) – Cleaned up page as part of review and added internal links. wikisysop
@@ Line 1: / Line 1: @@
-{{===== overview =====
+===== Pipeline Overview =====
-**the pipeline** is a set of visual3d commands that are processed in sequence.
+The Pipeline is a set of Visual3D commands that are processed in sequence. The Pipeline processor provides access to the core of Visual3D functionality through commands with each of Visual3D's functions represented by a command.
-**note:** the pipeline is a **command** language, not a **scripting language**.
+**Note:** The pipeline is a **COMMAND** language, not a **SCRIPTING language**.
-the pipeline processor provides access to the core of visual3d functionality through commands. each function is represented by a command. the pipeline is typically used to automate processing steps.
+The Pipeline is typically used to automate processing steps. It has the ability to manage files, define events, execute signal processing computations, create and edit modes, create and modify reports, and generated statistics. Edit boxes are usually provided for configuring each process step. The pipeline processor (or any text processor) can be used to create the pipeline. Any pipeline command that can be run interactively through the pipeline processor may be saved to a text command file. This file may be reloaded later, or combined with other command files to build a more complex pipeline. The pipeline processor can be launched from the Visual3D tool bar or from the Pipeline menu.
-the pipeline has the ability to manage files, define events, execute signal processing computations, create and edit modes, create and modify reports, and generated statistics. edit boxes are usually provided for configuring each process step.
+{{:Tutorial4_1.jpg}}
-the pipeline processor (or any text processor) can be used to create the pipeline. any pipeline command that can be run interactively through the pipeline processor may be saved to a text command file. this file may be reloaded later, or combined with other command files to build a more complex pipeline. the pipeline processor can be launched from the visual3d tool bar or from the pipeline menu.
+A common use case for the Visual3D Pipeline is to define the processing steps for a given experimental data analysis protocol. This allows, for example, the director of the laboratory to control the processing that is done by a member of the laboratory. This may be essential for some experimental studies that require that all of the data analysis be performed consistently.
-tutorial4_1.jpg
+If this is your first introduction to the Visual3D Pipeline then we recommend that you work through our [[Visual3D:Tutorials:Pipeline:Command_Pipeline_|detailed, introductory tutorial]] to teach you how to use the Pipeline and introduce you to pipeline commands.
-in special cases, c-motion creates custom pipeline commands for users. these commands are usually fully functional but hidden from the interface. users requiring these special commands will need to remember the syntax for the commands.
+==== Pipeline Commands ====
-a detailed **tutorial** to teach you how to use the command processing pipeline and introduce you to pipeline commands can be found here: [[visual3d:tutorials:pipeline:command_pipeline_|**tutorial: the command pipeline**]]
+Pipeline commands are divided into various categories: File, Signal, Event, etc. Commands under these folders are added to the pipeline by expanding the folder for a category, and selecting the function to be added by clicking the "Add" button. The selected function will be added to the pipeline immediately after the command that is currently highlighted in the pipeline. Commands can be repositioned in the pipeline using the arrow keys to the right of the list box. For details of specific commands that can be used through the Visual3D pipeline, see: **[[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Commands_Reference|Pipeline Commands Reference]]**
-===== pipeline commands =====
+=== Pipeline Commands: Active Files ===
-pipeline commands are divided into various categories: file, signal, event, etc. commands under these folders are added to the pipeline by expanding the folder for a category, and selecting the function to be added by clicking the "add" button. the selected function will be added to the pipeline immediately after the command that is currently highlighted in the pipeline. commands can be repositioned in the pipeline using the arrow keys to the right of the list box.
+Pipeline commands usually process the [[Visual3D:Documentation:Definitions:Active_Files|Active Files]]; e.g. those files active in the Visual3D Workspace. If only one file is selected in the file selection box, the pipeline, when executed, will perform actions only on the data from that file. If ALL_FILES is selected, the script will perform actions on every file individually, as if the script had been run sequentially for each file.
-a pipeline can be used to define the processing steps for a given experimental data analysis protocol. this allows, for example, the director of the laboratory to control the processing that is done by a member of the laboratory. this may be essential for some experimental studies that require that all of the data analysis be performed consistently.
+=== Pipeline Commands: Syntax ===
-visual3d plug ins integrate seamlessly into visual3d and are accessed through the pipeline processor. the pipeline can be saved to disk as an ascii file ([[#pipeline_v3s_file|the pipeline (.v3s) file]]) that can be edited with any common word processing program.
+Each Pipeline command consists of two parts - the command and its parameters.
-for details of specific commands that can be used through the visual3d pipeline, see: **[[visual3d:documentation:pipeline:general_information:pipeline_commands_reference|pipeline commands reference]]**
-==== pipeline commands: active files ====
-pipeline commands usually process the "active files"; e.g. those files active in the visual3d workspace. if only one file is selected in the file selection box, the pipeline, when executed, will perform actions only on the data from that file. if all_files is selected, the script will perform actions on every file individually, as if the script had been run sequentially for each file. more information on these active files can be found here: **[[visual3d:documentation:definitions:active_files|active files]]**.
-==== pipeline commands: syntax ====
-each pipeline command consists of two parts - the command and its parameters.
 <code>
-command_name
+Command_Name
-/parameter1= something
+/Parameter1= something
-/parameter2= this_value+another_value
+/Parameter2= this_value+another_value
-/parameter3= 3.14
+/Parameter3= 3.14
-!/parameter4= default_value_for_parameter4
+!/Parameter4= default_value_for_Parameter4
 ;
 </code>
-**note:** that each parameter begins with a slash and that the command is terminated with a semicolon.
+^Special Character ^ Meaning ^
+| / | Each parameter begins with a forward slash. |
+| ; | The semi-colon character terminates a command. |
+| ! | The exclamation point indicates that the remainder of the line is a comment. |
+| + | The plus sign is a delimeter separating entries. For example, 1+2 indicates that the variable has two values (1 and 2). |
+| & | The ampersand indicates a concatenation. For example, this&that will be interpreted in the pipeline as thisthat. |
-the "!" character is used to comment out a parameter. if the "!" character is at the beginning of a line, the line is ignored, and the default values for the parameter are used in the processing. the script text is provided so that these optional parameters may be un-commented and additional parameter values added.
+If the "!" character is at the beginning of a line, the line is ignored, and the default values for the parameter are used in the processing. The script text is provided so that users are aware of the default values and can easily uncomment these optional parameters and pass non-default values.
-to pass more than one set of parameters to the command, list the values of each parameter, separated by "+" signs. the generalized command above passes one set of three parameters to the command "command_name".
+To pass more than one set of parameters to the command, list the values of each parameter, separated by "+" signs. The generalized command above passes one set of three parameters to the command "Command_Name".
-optional parameter are commented out - thus they start with an exclamation mark. when a command is first added to the pipeline the default values are displayed on the line, so that the user is aware of the default values.
+=== Pipeline Commands: Specifying Signals in a Command ===
-==== pipeline commands: special command characters ====
+Signals in Visual3D must be identified by 3 names, according to that signal's location in the [[visual3d:documentation:visual3d_signal_types:data_tree|data tree]]:
+  - **Signal_Type - Target, Analog, etc.**
+  - **Signal_Folder - Original, Processed, etc.**
+  - **Signal_Name - Signal Name**
-the pipeline commands reserve 4 special characters
+Signals can be explicitly represented by the following three parameters:
-  * **!** - if this is the first character of a line, the line is considered a comment. if the ! is at the beginning of a variable name, the default value of the variable is used
+<code>
-  * **/** - indicates a command variable
+/Signal_Types= The types of the signals
-  * **+** - is a delimeter separating entries. for example, 1+2 indicates that the variable as two values (1 and 2).
+/Signal_Names= The names of the signals
-  * **&** - indicates a concatenation. for example, this&that will be interpreted in the pipeline as thisthat
+/Signal_Folder= The name of the signal folder
+</code>
-==== pipeline commands: specifying signals in a command ====
+If only the signal type is specified then all signals of that type (and the default folder) are implicitly selected for use in the command. For legacy support (e.g. Visual3D version 1.0), if the SIGNAL_FOLDER is empty it defaults to ORIGINAL. For example: to apply a lowpass filter to all TARGETS. This is accomplished by specifying the SIGNAL_TYPE and SIGNAL_FOLDER but leaving SIGNAL_NAMES empty.
-signals in visual3d must be identified by 3 components
+<code>
+Lowpass_Filter
-  - **signal_type - target, analog, etc**
+/SIGNAL_TYPES=TARGET
-  - **signal_name - signal name**
+/SIGNAL_FOLDER=ORIGINAL
-  - **signal_folder - the specific folder**
+/SIGNAL_NAMES=
+/RESULT_FOLDER=PROCESSED
+/RESULT_SUFFIX=
+/FILTER_CLASS=BUTTERWORTH
+/FREQUENCY_CUTOFF=6.0
+/NUM_REFLECTED=6
+/NUM_EXTRAPOLATED=0
+/TOTAL_BUFFER_SIZE=6
+/NUM_BIDIRECTIONAL_PASSES=1
+;
+</code>
-signals can be represented in two different ways depending on the command.
+Some commands act on signals (Target, Analog, etc). You can select a signal name for the command to work on by checking the signals in the data tree (**Signal and Event** processing mode) and pressing the **Import Signals from Data Tree** button. This will add the signal information (Type,Name,Folder) to the command. Many of the commands that act on signals will not show the signals in the dialog. This can be a bit of an annoyance when there are only a few signals selected, but often commands contain so many signals that the list box in the dialog will become meaningless because of all the signals. To make it easier to enter the signal types, names, and folders into a command, there is a button to add all "checked" signals from the data tree.
-many commands contain the following three parameters:
+Alternatively, for some commands signals are represented using the pipeline parameter syntax from the [[Visual3D:Documentation:Pipeline:Expressions:Expressions_Overview|Evaluate_Expression]] command. For example, to identify the RFT1 signal we use the syntax TYPE::FOLDER::NAME
 <code>
-/signal_types= the types of the signals
+TARGET::PROCESSED::RFT1
-/signal_names= the names of the signals
-/signal_folder= the name of the signal folder
 </code>
-these commands allow the user to import the checked signals in the data tree, so the syntax need not be remembered.
+==== Pipeline Parameters ====
-=== implicit behaviors ===
+An important feature of the pipeline is the ability to create and use [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Parameters|Pipeline Parameters]], which is a way to store a text string for use in pipeline commands.
-if only the signal type is specified. all signals of that type (and the default folder) are implicitly selected for use in the command.
+In one sense it is similar to specifying a global variable, such as body weight, that could be used in computations. Within the Visual3D pipeline, however, it is actually much more flexible since pipeline commands permit multiple entries on a single line. Since the entire line can be represented as a string, a single Pipeline Parameter can represent multiple entries.
-for example: to apply a lowpass filter to all targets. this is accomplished by specifying the signal_type and signal_folder but leaving signal_names empty.
+==== Editing Pipeline Scripts ====
-**lowpass_filter**
+Pipeline commands in a script may be edited in 3 ways:
-/signal_types=target
-/signal_folder=original
-/signal_names=
-/result_folder=processed
-/result_suffix=
-/filter_class=butterworth
-/frequency_cutoff=6.0
-/num_reflected=6
-/num_extrapolated=0
-/total_buffer_size=6
-/num_bidirectional_passes=1
-**;**
-for legacy support (e.g. visual3d version 1.0), if the signal_folder is empty it defaults to original. in other words
-/signal_folder=
+  - Using the built-in text editor (select the **TEXT** Button)
-is equivalent to
+  - Using a dialog (select the **EDIT button**). Most, but not all Visual3D commands have an edit dialog. If no dialog exists, then the text editor appears.
-/signal_folder=original
+  - Using an independent text editor such as [[https://notepad-plus-plus.org/|Notepad++]].
-=== entering signal names into a command ===
-some commands act on signals (target, analog, etc). you can select a signal name for the command to work on by checking the signals in the data tree (**signal and event** processing mode) and pressing the **import signals from data tree** button. this will add the signal information (type,name,folder) to the command.
+==== Pipeline Control ====
-many of the commands that act on signals will not show the signals in the dialog. this can be a bit of an annoyance when there are only a few signals selected, but often commands contain so many signals that the list box in the dialog will become meaningless because of all the signals.
+=== RECALC Pipeline ===
-to make it easier to enter the signal types, names, and folders into a command, there is a button to add all "checked" signals from the data tree.
+The [[Visual3D:Documentation:Pipeline:General_Information:RECALC_Pipeline|RECALC]] button updates the Visual3D Workspace by performing the following:
-==== pipeline commands: expressions ====
+  - If "use processed analog" is set, re-process force platforms.
+  - Rebuild all models in the Workspace
+  - Compute landmarks and model pose (e.g. position and orientation of all segments) for all #files in the Workspace
+  - Assign Force Platforms to Segments
+  - Compute Assigned Forces
+  - Compute Kinematics
+  - Compute Kinetics
+  - Compute Center of Mass.
+  - Execute the RECALC pipeline
+  - Update the display dialogs for any new data
+  - Recompute the Report graphs
-for some commands, signals are represented using the a pipeline parameter syntax. this type of syntax is used in the evaluate_expression command. for example, to identify the rft1 signal we use the syntax type::folder::name
+=== Pipeline Favorites ===
-target::processed::rft1
+The [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Favorites|Pipeline Commands:Pipeline Favorites]] feature allows you to list your pipeline scripts as a Table of Contents. Your list of pipeline scripts can be viewed or run from the 'Pipeline Favorites' list box.
-more details on expressions and evaluate_expression can be found here: **[[visual3d:documentation:pipeline:expressions:overview|expressions]]**
-===== pipeline parameters =====
+=== Launching Visual3D & Pipeline Externally ===
-an important feature of the pipeline is the ability to create and use pipeline parameters. a **pipeline parameter** is a way to store a text string for use in pipeline commands.
+Visual3D can be launched from a [[visual3d:getting_started:commandline|DOS command line]] (or an external program).
-in one sense it is similar to specifying a global variable, such as body weight, that could be used in computations. it is actually much more flexible than that in the pipeline. the visual3d pipeline commands permit multiple entries on a single line, and since the entire line can be represented as a string, a single pipeline parameter can represent multiple entries.
+If you have a DOS PATH set to the Visual3D directory you can type
-more details on naming and using parameters in pipelines can be found here: **[[visual3d:documentation:pipeline:general_information:pipeline_parameters|pipeline parameters]]**
+<code>
+Visual3D.exe
+</code>
-==== editing pipeline parameters ====
+If not, you must specify the full path to Visual3D.
-parameters in pipeline commands may be edited in 3 ways:
+<code>
+c:\Program Files\Visual3D\Visual3D.exe
+</code>
-  - using the built-in text editor (select the **text** button)
+It is possible to have a pipeline executed immediately on launching Visual3D using the following syntax.
-  - using a dialog (select the **edit button**). most, but not all visual3d commands have an edit dialog. if no dialog exists, then the text editor appears.
-  - using an independent [[#pipeline_v3s_file|text]] editor such as [[https://www.c-motion.com/free-downloads/#did1|notepad++]].
-===== pipeline control =====
+<code>
+Visual3D.exe /s c:\exampleData\test.v3s
+</code>
-==== recalc pipeline ====
+Note that the full path to the pipeline file is required unless the DOS PATH to this folder is specified.
-the recalc button updates the visual3d workspace by performing the following:
+==== Creating a Pipeline ====
-  - if "use processed analog" is set, re-process force platforms.
+The .v3s file is an ASCII file that can be saved to disk and can be edited with any common word processing program such as [[visual3d:documentation:pipeline:general_information:notepad|Notepad++]]. When Visual3D loads the .v3s file it parses this ASCII file into the individual commands that are displayed in the pipeline dialog.
-  - rebuild all models in the workspace
-  - compute landmarks and model pose (e.g. position and orientation of all segments) for all #files in the workspace
-  - assign force platforms to segments
-  - compute assigned forces
-  - compute kinematics
-  - compute kinetics
-  - compute center of mass.
-  - execute the recalc pipeline
-  - update the display dialogs for any new data
-  - recompute the report graphs
-more details on recalc in the visual3d workspace and pipelines can be found here: **[[visual3d:documentation:pipeline:general_information:recalc_pipeline|pipeline commands:recalc pipeline]]**
+Many of the pages in this wiki pages contain pipelines or snippets of pipelines, with a note saying that the text can be copied and pasted into a file. For example,
-==== pipeline favorites ====
-pipeline favorites is a very powerful way for you to list your pipeline scripts as a table of contents. your list of pipeline scripts can be viewed or run from the 'pipeline favorites' list box. more detail on pipeline favorites can be found here: **[[visual3d:documentation:pipeline:general_information:pipeline_favorites|pipeline commands:pipeline favorites]]**
-==== launching visual3d & pipeline externally ====
-launching visual3d from a dos command line (or an external program)
-if you have a dos path set to the visual3d directory you can type
-**visual3d.exe**
-if not, you must specify the full path to visual3d.
-**c:\program files\visual3d\visual3d.exe**
-it is possible to have a pipeline executed immediately on launching visual3d using the following syntax.
-**visual3d.exe /s c:\exampledata\test.v3s**
-note that the full path to the pipeline file is required unless the dos path to this folder is specified.
-===== creating a pipeline =====
-==== pipeline v3s file ====
-the .v3s file is an ascii file that can be saved to disk and can be edited with any common word processing program such as [[notepad%2b%2b|notepad++]]. when visual3d loads the .v3s file it parses this ascii file into the individual commands that are displayed in the pipeline dialog.
-many of the topics in the wiki pages contain pipelines or snippets of pipelines, with a note saying that the text can be copied and pasted into a file.
-for example,
 <code>
-! starting fresh - clear workspace
+! Starting Fresh - Clear Workspace
-file_new
+File_New
 ;
-! create a hybrid model.
+! Create a Hybrid model.
-! prompt for the standing trial to use...
+! Prompt for the Standing Trial to use...
-create_hybrid_model
+Create_Hybrid_Model
-! /calibration_file=
+! /CALIBRATION_FILE=
 ;
-! load the model template (segment definitions)
+! Load the Model Template (Segment Definitions)
-apply_model_template
+Apply_Model_Template
-! /model_template=
+! /MODEL_TEMPLATE=
-! /calibration_file=
+! /CALIBRATION_FILE=
 ;
-! prompt for the movement data files.
+! Prompt for the Movement data files.
-! multiple files can be selected in the dialog file listing using ctrl-click
+! Multiple files can be selected in the dialog file listing using CTRL-Click
-open_file
+Open_File
-! /file_name=
+! /FILE_NAME=
 ;
-! assigning the movement files to the model
+! Assigning the Movement files to the model
-! pop up the dialog box...
+! Pop up the dialog box...
-assign_model_file
+Assign_Model_File
-! /calibration_file=
+! /CALIBRATION_FILE=
-! /motion_file_names=
+! /MOTION_FILE_NAMES=
 ;
 </code>
-**the process for creating a pipeline from the above text.**
+**The Process for creating a pipeline from the above text.**
-  - select the pipeline text above, with the right mouse button on the highlighted text select copy.
-  - open notepad
-  - paste into notepad
-  - save the notepad file as testpipeline.v3s
-  - this command can now be opened in the visual3d pipeline dialog
-\\
-===== pipeline examples =====
-see the pipeline examples page for a collection of pipeline processing examples, found here: [[visual3d:documentation:pipeline:general_information:pipeline_examples|pipeline_examples]]
+  - Select the pipeline text above, with the right mouse button on the highlighted text select copy.
+  - Open Notepad
+  - Paste into Notepad
+  - Save the Notepad file as TestPipeline.v3s
+  - This command can now be opened in the Visual3D pipeline dialog
+==== Pipeline Examples ====
-}}
+See the [[Visual3D:Documentation:Pipeline:General_Information:Pipeline_Examples|Pipeline Examples]] page for a collection of pipeline processing examples.