Page Comparison

...

This class is responsible to translate parsed rule into objects in memory that represent the parsed syntax. Methods with name starting with make_ or add_match custom actions as defined in TM1RuleGrammar.peg (described later). These methods are called during parsing of the TM1 rule and are fed with data from the parser defined in TM1RuleGrammar.py.

Representation of Parse Tree

Let’s explain how the parser stores the parsed rule in memory on following example.

...

The parser generated by Canopy interfaces all tokens trough an element elements list. This list is always passed as an argument to a custom action method and contains tokens indexed according to a PEG parsing rule that activated the custom action.

For a better understanding let’s explain how the first parsing rule will affect content of the element list when add_dim_to_map method is called. Note: This method will called once the parser has parsed a dimension/element pair or pairs.

...

As you can see from the method header and implementation, the elements list is used in the code logic to access DirDimElemPair content by accessing elements[0]. Index 0 corresponds with position of DirDimElemPair non-terminal symbol in the rule. It is first symbol that has been parsed and since the elements list is zero based, the index of the symbol is 0. It is important to understand that content will be available after full expansion of the DirDimElemPair non-terminal symbol. That means the parser has to expand all non-terminal symbols until it reads terminal symbols from the input rule.

Similarly for DirDimElemPairs, the symbol is second in the rule and this thus its index will be 1. Note that DirDimElemPairs is followed by * in the PEG rule, which indicates zero or n repetitions of the non-terminal symbol. That's the reason why we access elements[1].elements as we intend to read content of all occurrences of the DirDimElemPairs non-terminal symbol as have been parsed.

Non-terminal symbol as per definition needs to be further parsedexpanded, so we can follow the parsing logic of DirDimElemPair until let's say DimIndetifier rule. You may see from the rule itself, once it will be parsed, the parser will call a custom method make_dim_elem_pair.

...

Again you may notice the custom action method header is following the same pattern as in previous case. We have elements list from which we can access all tokens as parsed during expansion of DirDimElemPair. In this case we can find DimIdentifer contents in elements[0], elements[1] would contain all whitespace characters that could follow a dimension identifier (or if there are none, elements[1] will be empty). Next, elements[2] would contain a single character : that is used to divide an element name from the dimension name. And similarly, elements[3] would contain all whitespace characters that could follow a : separator. Now, elements[4] refers to tokens that were collected when parsing DirElementList non-terminal symbol.

We followed expansions of each non-terminal symbol in above examples, but the parser needs to use terminal symbols to be able to properly select which parsing rules apply or if the input doesn’t follow the rule logic (in that case the parser would output a parse error). Let’s demonstrate this on the last rule the parser would use when reading a dimension name. The rule that applies in this case is expansion of DimIdentifier. This rule looks complicated, but can be interpreted as follows: DimIdentifier is a string enclosed in single quotes consisting of at least one allowed character. If the parser reads such a token, it will call custom action method make_ident. Let’s have a look how the method is implemented:

def make_ident(self, input, start, end, elements):
    ident = ""
    for char in elements:
        ident = ident + char.text
    # ignore leading/trailing apostrophes
    ident = ident.strip("'")
    return ident

The method has the same header as in above cases, elements list will now contain only terminal symbols as the rule for DimIdentifier suggests. Note: Terminal symbols are basically characters that are read from the source text being parsed.

That means the parser will “consume” the terminal symbols until the last matching character, which is a single quote that concludes a dimension name used in an area statement or a directive definition. The consumed characters will be available in elements list. Again we must refer to the matching PEG grammar rule to identify how the characters read from source will map to individual elements in the list. In our case, let’s say we have parsed the following string:

'T Year'

The elements list will consist of terminal symbols as in below table.

Since terminal symbols are represented as TreeNode objects by the parser, the actual content is stored in text property of each TreeNode object contained in the elements list. Therefore to get the identifier, we must concatenate the text property of each member. As a very last step we will strip leading and trailing single quotes as they have no value for splicing.

TM1RuleParser Properties and Methods

The class defines following attributes:

...