Differences

This shows you the differences between two versions of the page.

--- visual3d:documentation:pipeline:general_information:string_data [2024/06/19 12:51] – sgranger
+++ visual3d:documentation:pipeline:general_information:string_data [2024/10/04 15:36] (current) – Initial rework - the page is still missing string function descriptions. wikisysop
@@ Line 1: / Line 1: @@
-===== contents =====
+===== String Data =====
+String data is fundamentally different from time-varying signals and metrics so we have created a collection of string-specific functions within [[visual3d:documentation:pipeline:expressions:expressions_overview|Evaluate_Expression]] to allow common string manipulations to be accomplished within your pipeline scripts.
+==== Syntax ====
-  * [[#syntax|1 syntax]]
+Within Evaluate_Expression, the expression parser reads through the provided expression as one large string and parses out the specific variables, data and operators. Because Visual3D has no way to recognize the difference between a literal string and a variable name, it is necessary to use quotes around string data, e.g., subject_name="John Smith". String parameters in the pipeline are not quoted.
-  * [[#modify|2 modify]]
-  * [[#parsing|3 parsing]]
-    * [[#parsing_string_data_(more...)|3.1 parsing string data (more...)]]
-    * [[#string_left|3.2 string_left]]
-    * [[#string_right|3.3 string_right]]
-    * [[#string_mid|3.4 string_mid]]
-    * [[#string_length|3.5 string_length]]
-    * [[#string_find|3.6 string_find]]
-    * [[#string_reverse_find|3.7 string_reverse_find]]
-    * [[#string_to_lower|3.8 string_to_lower]]
-    * [[#string_to_upper|3.9 string_to_upper]]
-    * [[#to_string|3.10 to_string]]
-    * [[#example:_extract_text_(more...)|3.11 example: extract text (more...)]]
+=== Literal String Data ===
-==== syntax ====
+Strings can be explicitly created within a pipeline script without having to be stored as a pipeline variable. To accomplish this, all that needs to be done is to use quotes.
-quotes are to be used around string data in an expression to differentiate it from variable names. ie; subject_name="john smith". parameters in the pipeline that are strings, are not quoted (i.e.; expression is a string, but you don't need quotes around it). within expression, our expression parser parses the variables, data and operators out of the expression. data can be doubles or strings. if it is a string, visual3d has no way to recognize the difference between a string and a variable name reference, except to quote the string data.
+<code>
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_STRING
+/EXPRESSION="A123456789"
+/AS_INTEGER=FALSE
+;
+</code>
-for example, there is a problem with the following evaluate_expression:
+The result of this command is
-**evaluate_expression**
+<code>
-/expression=(::handchoice="left")
+::TEST_STRING = A123456789
-/result_name=test_4
+</code>
-! /result_type=derived
-! /result_folder=processed
-**;**
-this uses a pipeline parameter in the middle of a string parameter (the expression). global parameters cannot be evaluated in the middle of a string, so to recognize that a pipeline parameter, you need to concatenate it into the expression as follows:
-**evaluate_expression**
+=== Concatenation ===
-/expression=(&::handchoice&="left")
+Literal string data must be explicitly concatenated with values from parameters. For example, there is a problem with the following Evaluate_Expression syntax:
-/result_name=test_4
-! /result_type=derived
-! /result_folder=processed
-**;**
-currently, the operators are only parsed as a single character, so the use of "/=" (not equals) cannot be used yet. you can do not(item1=item2) in order to process a not equals currently, but this will change shortly to allow "/=", "!=" cannot be used because the "!" is our start comment character for the pipeline commands.
-==== modify ====
+<code>
+Evaluate_Expression
+/EXPRESSION=(::HandChoice="LEFT")
+/RESULT_NAME=TEST_4
+! /RESULT_TYPE=DERIVED
+! /RESULT_FOLDER=PROCESSED
+;
+</code>
-==== parsing ====
+This uses a pipeline parameter in the middle of a string parameter (the EXPRESSION). Global parameters cannot be evaluated in the middle of a string, so to recognize that a pipeline parameter, you need to concatenate it into the expression as follows:
-given a pipeline parameter that is a string. for internal reasons, the string cannot be a number, or it will be interpreted as a number.
+<code>
+Evaluate_Expression
+/EXPRESSION=(&::HandChoice&="LEFT")
+/RESULT_NAME=TEST_4
+! /RESULT_TYPE=DERIVED
+! /RESULT_FOLDER=PROCESSED
+;
+</code>
-there are 3 commands that extract a portion of a string.
+=== Negation ===
-string_left(string,index)
+Operators are only parsed as a single character, so the "/=" notation for "not equals" cannot be used. It is also not possible to use the "!=" notation for "not equals" either because the "!" character is dedicated to indicating when comments begin within pipeline scripts.
-string_right(string,index)
-string_mid(string,index1,index2)
-string_length(string)
-string_find(string, substring, start)
-**note:** when referencing strings, the string should be surrounded by quotes. this identifies to visual3d that you are referencing a string, and not a file name etc.
-\\
+In order to negate an expression you can use the syntax "NOT(item1=item2)".
+==== Modify ====
-=== parsing string data (more...) ===
+==== Parsing ====
-**set_pipeline_parameter_from_expression**
+Given a pipeline parameter that is a string, there are four commands that extract a portion of a string:
-/parameter_name=test_string
-/expression="a123456789"
-/as_integer=false
-**;**
-the result is
-::test_string = a123456789
+STRING_LEFT(string,index)
-=== string_left ===
+STRING_RIGHT(string,index)
+STRING_MID(string,index1,index2)
+STRING_FIND(string, substring, start)
-**set_pipeline_parameter_from_expression**
+and one command for measuring the string:
-/parameter_name=test_left
-/expression=string_left("&::test_string&",3)
-/as_integer=false
-**;**
-the result is
-::test_left = a12
+STRING_LENGTH(string)
-=== string_right ===
-**set_pipeline_parameter_from_expression**
+=== String_Left ===
-/parameter_name=test_right
-/expression=string_right("&::test_string&",3)
-/as_integer=false
-**;**
-the result is
-::test_right = 789
+<code>
-=== string_mid ===
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_LEFT
+/EXPRESSION=STRING_LEFT("&::TEST_STRING&",3)
+/AS_INTEGER=FALSE
+;
+</code>
-**set_pipeline_parameter_from_expression**
+The result is:
-/parameter_name=test_mid
-/expression=string_mid("&::test_string&",3,3)
-/as_integer=false
-**;**
-the result is
-::test_mid = 345
+<code>
-//note: for string_mid the number values are zero based//
+::TEST_LEFT = A12
+</code>
-=== string_length ===
+=== String_Right ===
-**set_pipeline_parameter_from_expression**
+<code>
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_RIGHT
+/EXPRESSION=STRING_RIGHT("&::TEST_STRING&",3)
+/AS_INTEGER=FALSE
+;
+</code>
-/parameter_name=test_length
+The result is:
-/expression=string_length("&::test_string&")
-/as_integer=true
-**;**
-the result is
-::test_length = 10
+<code>
-**set_pipeline_parameter_from_expression**
+::TEST_RIGHT = 789
+</code>
-/parameter_name=test_index
+=== String_Mid ===
-/expression=string_find("&::test_string&",234,0)
-/as_integer=true
-**;**
-the result is
-::test_index = 2
+Note that the two index parameters are zero-based for the STRING_MID function.
-\\
+<code>
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_MID
+/EXPRESSION=STRING_MID("&::TEST_STRING&",3,3)
+/AS_INTEGER=FALSE
+;
+</code>
-=== string_find ===
+The result is:
-**set_pipeline_parameter_from_expression**
+<code>
+::TEST_MID = 345
+</code>
-/parameter_name=test_index
+=== String_Find ===
-/expression=string_find("&::test_string&",234,5)
-/as_integer=true
-**;**
-the result is
-::test_index = -1
+<code>
-=== string_reverse_find ===
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_INDEX
+/EXPRESSION=STRING_FIND("&::TEST_STRING&",234,5)
+/AS_INTEGER=TRUE
+;
+</code>
-=== string_to_lower ===
+Because the string we are searching for "5" is not in the string being searched, "234", the result is:
-!converting to_upper and to_lower string data (more...)
+<code>
+::TEST_INDEX = -1
+</code>
-**set_pipeline_parameter_from_expression**
+=== String_Reverse_Find ===
-/parameter_name=test
-/expression=string_to_lower("xxxxxx")
-/as_integer=false
-**;**
-**set_pipeline_parameter_from_expression**
-/parameter_name=test
-/expression=string_to_upper("yyyyyyy")
-/as_integer=false
-**;**
-**select_active_file**
-/file_name=global
-! /query=
-**;**
-**create_text_data**
-/signal_folder=scott
-/signal_names=name
-/text_data=hello world
-! /text_file_name=
-! /prompt_on_empty=true
-**;**
-**evaluate_expression**
-/expression=string_to_upper(text_data::scott::name)
-! /signal_types=
-! /signal_folder=original
-! /signal_names=
-/result_types=text_data
-/result_folders=scott
-/result_name=scott_upper
-! /apply_as_suffix_to_signal_name=false
-**;**
-**evaluate_expression**
-/expression=string_to_lower(text_data::scott::name)
-! /signal_types=
-! /signal_folder=original
-/signal_names=
-/result_types=text_data
-/result_folders=scott
-/result_name=scott_lower
-! /apply_as_suffix_to_signal_name=false
-**;**
-=== string_to_upper ===
-=== to_string ===
+To be completed...
-=== example: extract text (more...) ===
+=== String_Length ===
-given the following text_data loaded from a file store in a signal text_data::scott:test
+<code>
-random stuff <subject>scott</subject> more random stuff
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_LENGTH
+/EXPRESSION=STRING_LENGTH("&::TEST_STRING&")
+/AS_INTEGER=TRUE
+;
+</code>
-extract the name of the subject.
+The result is:
-**set_pipeline_parameter_from_expression**
-/parameter_name=f1
-/expression=9+string_find(text_data::scott::test, "<subject>", 0)
-/as_integer=true
-**;**
-**set_pipeline_parameter_from_expression**
-/parameter_name=f2
-/expression=string_find(text_data::scott::test, "</subject>", 0)-&::f1
-/as_integer=true
-**;**
-**set_pipeline_parameter_from_expression**
-/parameter_name=subject
-/expression=string_mid(text_data::scott::test,&::f1&,&::f2&)
-/as_integer=false
-**;**
-the resulting pipeline parameter contains "scott"
+<code>
+::TEST_LENGTH = 10
+</code>
+<code>
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST_INDEX
+/EXPRESSION=STRING_FIND("&::TEST_STRING&",234,0)
+/AS_INTEGER=TRUE
+;
+</code>
+The result is:
+<code>
+::TEST_INDEX = 2
+</code>
+=== String_To_Lower ===
+<code>
+!Converting To_Upper and To_Lower String Data (more...)
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST
+/EXPRESSION=STRING_TO_LOWER("XXXXXX")
+/AS_INTEGER=FALSE
+;
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=TEST
+/EXPRESSION=STRING_TO_UPPER("yyyyyyy")
+/AS_INTEGER=FALSE
+;
+Select_Active_File
+/FILE_NAME=GLOBAL
+! /QUERY=
+;
+Create_Text_Data
+/SIGNAL_FOLDER=SCOTT
+/SIGNAL_NAMES=NAME
+/TEXT_DATA=Hello World
+! /TEXT_FILE_NAME=
+! /PROMPT_ON_EMPTY=TRUE
+;
+Evaluate_Expression
+/EXPRESSION=STRING_TO_UPPER(TEXT_DATA::SCOTT::NAME)
+! /SIGNAL_TYPES=
+! /SIGNAL_FOLDER=ORIGINAL
+! /SIGNAL_NAMES=
+/RESULT_TYPES=TEXT_DATA
+/RESULT_FOLDERS=SCOTT
+/RESULT_NAME=SCOTT_UPPER
+! /APPLY_AS_SUFFIX_TO_SIGNAL_NAME=FALSE
+;
+Evaluate_Expression
+/EXPRESSION=STRING_TO_LOWER(TEXT_DATA::SCOTT::NAME)
+! /SIGNAL_TYPES=
+! /SIGNAL_FOLDER=ORIGINAL
+/SIGNAL_NAMES=
+/RESULT_TYPES=TEXT_DATA
+/RESULT_FOLDERS=SCOTT
+/RESULT_NAME=SCOTT_LOWER
+! /APPLY_AS_SUFFIX_TO_SIGNAL_NAME=FALSE
+;
+</code>
+=== String_To_Upper ===
+To be completed...
+=== To_String ===
+To be completed
+==== Example: Extracting text ====
+Given the following Text_Data loaded from a file,
+<code>
+Random Stuff <Subject>Scott</Subject> More Random Stuff
+</code>
+this example demonstrates how to store the desired string "<Subject>Scott</Subject>" into the [[visual3d:documentation:visual3d_signal_types:data_tree|data tree]] at TEXT_DATA::SCOTT:TEST.
+<code>
+!Extract the name of the subject.
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=F1
+/EXPRESSION=9+STRING_FIND(TEXT_DATA::SCOTT::TEST, "<Subject>", 0)
+/AS_INTEGER=TRUE
+;
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=F2
+/EXPRESSION=STRING_FIND(TEXT_DATA::SCOTT::TEST, "</Subject>", 0)-&::F1
+/AS_INTEGER=TRUE
+;
+Set_Pipeline_Parameter_From_Expression
+/PARAMETER_NAME=SUBJECT
+/EXPRESSION=String_Mid(TEXT_DATA::SCOTT::TEST,&::F1&,&::F2&)
+/AS_INTEGER=FALSE
+;
+</code>