Page Comparison

...

• TM1RuleParser - implements custom actions that get called from the generated parser to create object tree representing parsed TM1 rule.

• ArgumentParser - implements CLI parsing and is used by Splicer.py to run required actions by instantiating TM1SpliceExecutor and running its method corresponding to the action required from CLI.

TM1RuleParser (class in TM1RuleParser.py)

This class is responsible to translate parsed rule into objects in memory that represent the parsed syntax. Methods with name starting with make_ or add_match custom actions as defined in TM1RuleGrammar.peg (described later). These methods are called during parsing of the TM1 rule and are fed with data from the parser defined in TM1RuleGrammar.py.

...

This rule will be parsed by the TM1RuleParser method get_parse_tree and the parsing result will be returned in an instance of TM1Splice.Parser.Grammar.TM1RuleGrammar.TreeNode class. The class contains elements property, which is a list of parsed tokens. You may see a simplified representation of the parsed rule on below picture.

...

It is important to understand how the parser accesses the parsed tokens during the custom actions. Let’s demonstrate it on a following example:

Following simplified code represents a PEG rule that we will further refer to in the custom action methods implemented in TM1RuleParser.

DirDimElemPairList      <- DirDimElemPair DirDimElemPairs* %add_dim_to_map
DirDimElemPairs         <- _ "," _ DirDimElemPair %add_dim_to_map2
DirDimElemPair          <- DimIdentifier _ ":" _ DirElementList %make_dim_elem_pair
DimIdentifier           <- "'" [^\'\"\@\<\!\[\]\^\*\>\=\\\|\?\/\;\,\~\%\&\:\n\r]+ "'" %make_ident

The parser generated by Canopy interfaces all tokens trough an element list. This list is always passed as an argument to a custom action method and contains tokens indexed according to a PEG parsing rule that activated the custom action.

For better understanding let’s explain how the first parsing rule will affect content of the element list when add_dim_to_map method is called. Note: This method will called once the parser has parsed a dimension/element pair or pairs.

    def add_dim_to_map(self, input, start, end, elements):
        dim_map = dict()
        if elements[0] and elements[1]:
            for dim, elem in [*[elements[0]], *(elements[1].elements)]:
                dim_map[dim] = elem
        return dim_map

As you can see from the method header and implementation, the elements list is used in the code logic to access DirDimElemPair content by accessing elements[0]. Index 0 corresponds with position of DirDimElemPair non-terminal symbol in the rule. It is first symbol that has been parsed and since the elements list is zero based, the index of the symbol is 0.

Similarly for DirDimElemPairs, the symbol is second in the rule and this its index will be 1. Note that DirDimElemPairs is followed by * in the PEG rule, which indicates zero or n repetitions of the non-terminal symbol. That's the reason why we access elements[1].elements as we intend to read all occurrences of the DirDimElemPairs non-terminal symbol as have been parsed.

Non-terminal symbol as per definition needs to be further parsed, so we can follow the parsing logic of DirDimElemPair until let's say DimIndetifier rule. You may see from the rule itself, once it will be parsed, the parser will call a custom method make_dim_elem_pair.

    def make_dim_elem_pair(self, input, start, end, elements):
        if type(elements[4]) is list:
            pairs = []
            for (hierarchy, element) in elements[4]:
                pairs.append((hierarchy if hierarchy else elements[0], element))
            return (elements[0], pairs)
        else:
            if not elements[4][0]:
                elements[4] = (elements[0], elements[4][1])
            return (elements[0], [elements[4]])

Again you may notice the custom action method header is following the same pattern as in previous case. We have elements list from which we can access all tokens as parsed. In this case we can find DimIdentifer contents in elements[0], elements[1] would contain all whitespace characters that could follow a dimension identifier (or if there are none, elements[1] will be empty). Next, elements[2] would contain a single character : that is used to divide an element name from the dimension name. And similarly, elements[3] would contain all whitespace characters that could follow a : separator. Now, elements[4] refers to tokens that were collected when parsing DirElementList non-terminal symbol.

We followed expansions of each non-terminal symbol in above examples, but the parser needs to use terminal symbols to be able to properly select which parsing rules apply or if the input doesn’t follow the rule logic (in that case the parser would output a parse error). Let’s demonstrate this on the last rule the parser would use when reading a dimension name. The rule that applies in this case is expansion of DimIdentifier. This rule looks complicated, but can be interpreted as a string enclosed in single quotes consisting of at least one allowed character. If the parser reads such a token, it will call custom action method make_ident. Let’s have a look how the method is implemented:

The class defines following attributes:

...

ArgumentParser (class in ArgumentParser.py)

The class implements command line parsing and is used by main application defined in Splicer.py.

...