TM1Splice (package)

This is a base package of Splicer containing a CLI wrapper implemented in splicer.py source file and splicing logic implemented in TM1SpliceExecutor class. The class is described in detail below.

`TM1SpliceExecutor` (class in `TM1SpliceExecutor.py`)

The class implements splicing and desplicing logic in two entry point methods that are called from splicer.py when splicing or desplicing is required from the command line. Below you may find in-depth description of the most important methods provided by the class.

Initialization (constructor `init`)

The class is initialized before splicing or desplicing from parser.py and when the init is finished, the instance of TM1SpliceExecutor will have TM1SpliceConfigContainer associated with itself and all configuration files will be already loaded into memory. The configuration data is provided to TM1SpliceExecutor transparently through the TM1SpliceConfigContainer instance (more about the configuration container can be found in https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540863497

The connection with the TM1 instance is established during the initialization as well.

Splicing Cubes (method `splice_cube_rules`)

Splicing logic is implemented in splice_cube_rules method and contains following steps.

First, the method will determine and call TI processes to be run before commencing splicing. The list of processes is configurable and might be added if required to kpi_template.json configuration file to preprocess_TI::Splice object. The list is retrieved from TM1SpliceConfigContainer instance associated with the TM1SpliceExecutor.

Then the method will establish a list of cubes for splicing. The list can be either all cubes as configured in fpm.json in Cubes object (--all option), or a list supplied from command line (--cubes option) or a list of cubes containing supplied list of dimensions to a command line (--dimension option) within their directives.

Then it will loop all the cubes in the list and will call individual cube splicing for each cube.

After the loop is finished, the method will determine and call TI processes to run after splicing. The list of processes is configured in kpi_template.json in postprocess_TI::Splice object.

Finally the method will check status of each cube that was spliced to see if any errors have occurred during splicing. If an error was detected during splicing, the changes that have been committed by Splicer will be rolled back.

Splicing Individual Cube (method `splice_cube_rule`)

The method implements actual splicing of the rule file and is phased to following steps.

First the internal store cubes_status for cube splicing status will be set to False. The status store is used to store success or error flag per each cube during splicing and file name of associated backup file that is created before the rule is modified by the Splicer. The False value is indicating failure - the status will need to be set to True when the method finishes successfully.

Then the method will establish a new TM1DirectiveResolver instance (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540863497 ) and will parse the rule. It will get a parse tree of the rule as a result of a call to TM1RuleParser.get_parse_tree method. The parse tree is in-memory representation of the parsed rule composed by interlinked objects from Objects package (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540929025 ). These objects are allocated by the parser implemented by TM1RuleGrammar (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2539913225 ) through methods make_ or add_ defined in TM1RuleParser (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540240903 ) according to the PEG grammar (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2539913225 ).

Then it will determine if there is overlap between dimensions extracted from all directives defined in the rule and supplied list of dimensions if Splicer was run with --dimension option. This feature was implemented to optimize run time of Splicer in cases when there were changes provided to certain dimension in FPM that in turn required splicing of cubes that contain the changed dimension.

Once finished, the method continues by backing up the rule file to a backup folder as configured from command line (default location is a subfolder rule).

After that the method will determine if the rule contains region FPM-KPI. This region is intended to store all automatically generated KPI calculations and is regenerated by Splicer. If the region is available in the rule, its parse subtree will be excluded from the parse tree. This will result in in-memory removal of all lines contained within the #Region and #EndRegion tags for FPM-KPI. Then new contents of the region will be generated by calling generate_kpi method of TM1SpliceExecutor, which will return a parse subtree containing all the generated lines with KPI calculations. The subtree is then injected into the parse tree obtained before.

The method then calls TM1Parser.get_rule to splice the rule by calling convert method of each object in the parse tree and get the text representation of the modified parse tree. This string will be used to patch the cube rule in TM1. Before it is attempted, the method will first check if there are any changes by comparing original rule with a rule retrieved from the modified parse tree. If the rules are not identical, it will save the string into a proposed file in the proposed rule folder (configured from command line, default location is rule), otherwise it will skip further steps and the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.

After the changes are saved in the proposed file, the method will patch the cube with updated rule and will run check of the rule on the TM1 server. An exception will be generated if the check fails, the cube status will not be updated in cubes_status store (initial value was already set to False to indicate failure) and rollback of changes will be run. Otherwise the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.

Desplicing Cubes (method `desplice_cube_rules`)

Desplicing is a reversal of splicing and is implemented very similarly as above described splicing method. The steps are following.

First, the method will determine and call TI processes to be run before commencing desplicing. The list of processes is configurable and might be added if required to kpi_template.json configuration file to preprocess_TI::Desplice object. The list is retrieved from TM1SpliceConfigContainer instance associated with the TM1SpliceExecutor.

Then the method will establish a list of cubes for desplicing. The list can be either all cubes as configured in fpm.json in Cubes object (--all option), or a list supplied from command line (--cubes option). Option to desplice cubes containing supplied list of dimensions to a command line (--dimension option) is not yet implemented.

Then it will loop all the cubes in the list and will call individual cube desplicing for each cube.

After the loop is finished, the method will determine and call TI processes to run after desplicing. The list of processes is configured in kpi_template.json in postprocess_TI::Desplice object.

Finally the method will check status of each cube that was spliced to see if any errors have occurred during desplicing. If an error was detected during desplicing, the changes that have been committed by Splicer will be rolled back.

Desplicing Individual Cube (method `desplice_cube_rule`)

The method implements reversal of splicing of a cube rule, the algorithm is similar to cube splicing - but differs in metadata used for element names substitution. These come from fpm.json and are based on QueryRollback MDX query definitions for each applicable directive. The desplicing consists of following steps.

First the internal store cubes_status for cube splicing status will be set to False. The status store is used to store success or error flag per each cube during splicing and file name of associated backup file that is created before the rule is modified by the Splicer. The False value is indicating failure - the status will need to be set to True when the method finishes successfully.

The method will then back up the rule file to a backup folder as configured from command line (default location is a subfolder rule).

Then the method will establish a new TM1ReverseDirectiveResolver instance (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540863497 ) and will parse the rule. It will get a parse tree of the rule as a result of a call to TM1RuleParser.get_parse_tree method. The parse tree is in-memory representation of the parsed rule composed by list of objects from Objects package (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540929025) representing individual rule lines (line is represented by a token finished by ;). These objects are allocated by the parser implemented by TM1RuleGrammar (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2539913225 ) through methods make_ or add_ defined in TM1RuleParser (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540240903 ) according to the PEG grammar (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2539913225 ).

After that the method will determine if the rule contains region FPM-KPI. If the region is available in the rule, its parse subtree will be excluded from the parse tree. This will result in in-memory removal of all lines contained within the #Region and #EndRegion tags for FPM-KPI.

The method then calls TM1Parser.splice_rule to splice the rule by calling convert method of each object in the parse tree and get the text representation of the modified parse tree. The method then will use the string to patch the cube rule in TM1. After that the method will run check of the rule on the TM1 server. An exception will be generated if the check fails, the cube status will not be updated in cubes_status store (initial value was already set to False to indicate failure) and rollback of changes will be run. In such a case the method will save the patch string into a proposed file in the proposed rule folder (configured from command line, default location is rule).

If the patch action and rule check were successful the method will set the cube status in cubes_status store to True and will return without errors signalling to the caller to continue with a following cube rule.

TM1SpliceExecutor Properties and Methods

The class defines following attributes:

Attribute	Type	Usage

Attribute	Type	Usage
`config`	`TM1SpliceConfigContainer`	Instance of configuration container to retrieve config data from (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2540863497 ).
`tm1`	`TM1Service`	TM1Py connection object to access target TM1 instance.
`parser`	`Parser`	Instance of parser (https://apliqoux.atlassian.net/wiki/spaces/AFD/pages/2539913225) to be used to parse the rule.
`kpi`	`Dict[str, str]`	Dictionary of KPI calculation definitions retrieved from `}APQ C3 FIN CoA Map` to be used to generate KPI calculations in `FPM-KPI` region.
`log`	`Logger`	Logger to use to log into `application.log`, console and TM1 instance through `TM1LoggingProvider`

The class exposes following methods:

Method	Usage

Method	Usage
`get_tm1_connection`	Creates a connection with target TM1 instance.
`execute_mdx`	A helper method to run a set MDX query, returns a list of elements.
`execute_rollback_mdx_rule`	A helper method to run a set MDX query stored with each directive in `QueryRollback` object in `fpm.json`, returns a list of elements.
`execute_mdx_rule`	A helper method to run a set MDX query stored with each directive in `Query` object in `fpm.json`, returns a list of elements.
`rollback_rule_changes`	Recovers a rule from a backup file located in rule subfolder and saves it on TM1 server.
`desplice_cube_rule`	Desplices an individual cube rule - see details provided above.
`splice_cube_rule`	Splices an individual cube rule - see details provided above.
`get_parse_tree`	Returns a parse tree from `TM1RuleParser`, the parse tree corresponds to the unchanged rule.
`parse_text`	Returns a spliced rule as string, the rule has been spliced.
`run_ti_processes`	A helper method to run a TI process, used to run TI processes before and after splicing.
`get_cube_names`	A helper method to compile a list of cubes depending on Splicer CLI settings.
`get_element_parents`	A helper method to return a list of parents related to a supplied element.
`get_immediate_descendants`	A helper method to return a set of immediate descendants of a supplied element.
`clear_dirty_flag`	A helper method to clear `dirty` flag values for all KPIs in `}APQ C3 FIN CoA Map`.
`check_cubes_status`	A helper method to check cube status of all cubes that were processed. If there is an error indicated for any of the cubes, the method will initiate rollback of the rule changes.
`prepare_desplice_results`	A helper method to prepare data by running `QueryRollback` MDX queries for desplicing.
`desplice_cube_rules`	Desplices all cubes depending on Splicer CLI settings.
`splice_cube_rules`	Splices all cubes depending on Splicer CLI settings.
`register_results`	A helper method to register results of splicing for a directive and dimension.
`get_kpi_set`	A method to retrieve a dictionary of KPI calculation definitions from `}APQ C3 FIN CoA Map` to be used to generate KPI calculations in `FPM-KPI` region.
`generate_kpi`	A method to generate content of FPM-KPI region based on `kpi`, returns a parse subtree.
`disconnect`	Disconnects a TM1 connection session.

TM1SpliceExecutor (class in TM1SpliceExecutor.py)

Initialization (constructor __init__)

Splicing Cubes (method splice_cube_rules)

Splicing Individual Cube (method splice_cube_rule)

Desplicing Cubes (method desplice_cube_rules)

Desplicing Individual Cube (method desplice_cube_rule)