This is a base package of Splicer containing a CLI wrapper implemented in splicer.py source file and splicing logic implemented in TM1SpliceExecutor class. The class is described in detail below.
TM1SpliceExecutor (class in TM1SpliceExecutor.py)
The class implements splicing and desplicing logic in two entry point methods that are called from splicer.py when splicing or desplicing is required from the command line. Below you may find in-depth description of the most important methods provided by the class.
Initialization (constructor __init__)
The class is initialized before splicing or desplicing from parser.py and when the init is finished, the instance of TM1SpliceExecutor will have TM1SpliceConfigContainer associated with itself and all configuration files will be already loaded into memory. The configuration data is provided to TM1SpliceExecutor transparently through the TM1SpliceConfigContainer instance (more about the configuration container can be found in Containers (package)
The connection with the TM1 instance is established during the initialization as well.
Splicing Cubes (method splice_cube_rules)
Splicing logic is implemented in splice_cube_rules method and contains following steps.
First, the method will determine and call TI processes to be run before commencing splicing. The list of processes is configurable and might be added if required to kpi_template.json configuration file to preprocess_TI::Splice object. The list is retrieved from TM1SpliceConfigContainer instance associated with the TM1SpliceExecutor.
Then the method will establish a list of cubes for splicing. The list can be either all cubes as configured in fpm.json in Cubes object (--all option), or a list supplied from command line (--cubes option) or a list of cubes containing supplied list of dimensions to a command line (--dimension option) within their directives.
Then it will loop all the cubes in the list and will call individual cube splicing for each cube.
After the loop is finished, the method will determine and call TI processes to run after splicing. The list of processes is configured in kpi_template.json in postprocess_TI::Splice object.
Finally the method will check status of each cube that was spliced to see if any errors have occurred during splicing. If an error was detected during splicing, the changes that have been committed by Splicer will be rolled back.
Splicing Individual Cube (method splice_cube_rule)
The method implements actual splicing of the rule file and is phased to following steps.
First the internal store cubes_status for cube splicing status will be set to False. The status store is used to store success or error flag per each cube during splicing and file name of associated backup file that is created before the rule is modified by the Splicer. The False value is indicating failure - the status will need to be set to True when the method finishes successfully.
Then the method will establish a new TM1DirectiveResolver instance (Containers (package) ) and will parse the rule. It will get a parse tree of the rule as a result of a call to TM1RuleParser.get_parse_tree method. The parse tree is in-memory representation of the parsed rule composed by interlinked objects from Objects package (Objects (package) ). These objects are allocated by the parser implemented by TM1RuleGrammar (Parser/Grammar (package) ) through methods make_ or add_ defined in TM1RuleParser (Parser (package) ) according to the PEG grammar (Parser/Grammar (package) ).
Then it will determine if there is overlap between dimensions extracted from all directives defined in the rule and supplied list of dimensions if Splicer was run with --dimension option. This feature was implemented to optimize run time of Splicer in cases when there were changes provided to certain dimension in FPM that in turn required splicing of cubes that contain the changed dimension.
Once finished, the method continues by backing up the rule file to a backup folder as configured from command line (default location is a subfolder rule).
After that the method will determine if the rule contains region FPM-KPI. This region is intended to store all automatically generated KPI calculations and is regenerated by Splicer. If the region is available in the rule, its parse subtree will be excluded from the parse tree. This will result in in-memory removal of all lines contained within the #Region and #EndRegion tags for FPM-KPI. Then new contents of the region will be generated by calling generate_kpi method of TM1SpliceExecutor, which will return a parse subtree containing all the generated lines with KPI calculations. The subtree is then injected into the parse tree obtained before.
The method then calls TM1Parser.get_rule to splice the rule by calling convert method of each object in the parse tree and get the text representation of the modified parse tree. This string will be used to patch the cube rule in TM1. Before it is attempted, the method will first check if there are any changes by comparing original rule with a rule retrieved from the modified parse tree. If the rules are not identical, it will save the string into a proposed file in the proposed rule folder (configured from command line, default location is rule), otherwise it will skip further steps and the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.
After the changes are saved in the proposed file, the method will patch the cube with updated rule and will run check of the rule on the TM1 server. An exception will be generated if the check fails, the cube status will not be updated in cubes_status store (initial value was already set to False to indicate failure) and rollback of changes will be run. Otherwise the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.
Desplicing Cubes (method desplice_cube_rules)
Desplicing is a reversal of splicing and is implemented very similarly as above described splicing method. The steps are following.
First, the method will determine and call TI processes to be run before commencing desplicing. The list of processes is configurable and might be added if required to kpi_template.json configuration file to preprocess_TI::Desplice object. The list is retrieved from TM1SpliceConfigContainer instance associated with the TM1SpliceExecutor.
Then the method will establish a list of cubes for desplicing. The list can be either all cubes as configured in fpm.json in Cubes object (--all option), or a list supplied from command line (--cubes option). Option to desplice cubes containing supplied list of dimensions to a command line (--dimension option) is not yet implemented.
Then it will loop all the cubes in the list and will call individual cube desplicing for each cube.
After the loop is finished, the method will determine and call TI processes to run after desplicing. The list of processes is configured in kpi_template.json in postprocess_TI::Desplice object.
Finally the method will check status of each cube that was spliced to see if any errors have occurred during desplicing. If an error was detected during desplicing, the changes that have been committed by Splicer will be rolled back.
Desplicing Individual Cube (method desplice_cube_rule)
The method implements reversal of splicing of a cube rule, the algorithm is similar to cube splicing - but differs in metadata used for element names substitution. These come from fpm.json and are based on QueryRollback MDX query definitions for each applicable directive. The desplicing consists of following steps.
First the internal store cubes_status for cube splicing status will be set to False. The status store is used to store success or error flag per each cube during splicing and file name of associated backup file that is created before the rule is modified by the Splicer. The False value is indicating failure - the status will need to be set to True when the method finishes successfully.
The method will then back up the rule file to a backup folder as configured from command line (default location is a subfolder rule).
Then the method will establish a new TM1ReverseDirectiveResolver instance (Containers (package) ) and will parse the rule. It will get a parse tree of the rule as a result of a call to TM1RuleParser.get_parse_tree method. The parse tree is in-memory representation of the parsed rule composed by list of objects from Objects package (Objects (package)) representing individual rule lines (line is represented by a token finished by ;). These objects are allocated by the parser implemented by TM1RuleGrammar (Parser/Grammar (package) ) through methods make_ or add_ defined in TM1RuleParser (Parser (package) ) according to the PEG grammar (Parser/Grammar (package) ).
After that the method will determine if the rule contains region FPM-KPI. If the region is available in the rule, its parse subtree will be excluded from the parse tree. This will result in in-memory removal of all lines contained within the #Region and #EndRegion tags for FPM-KPI.
The method then calls TM1Parser.splice_rule to splice the rule by calling convert method of each object in the parse tree and get the text representation of the modified parse tree. The method then will use the string to patch the cube rule in TM1. After that the method will run check of the rule on the TM1 server. An exception will be generated if the check fails, the cube status will not be updated in cubes_status store (initial value was already set to False to indicate failure) and rollback of changes will be run. In such a case the method will save the patch string into a proposed file in the proposed rule folder (configured from command line, default location is rule).
If the patch action and rule check were successful the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.
