...
TM1RuleParser
- implements custom actions that get called from the generated parser to create object tree representing parsed TM1 rule.ArgumentParser
- implements CLI parsing and is used bySplicer.py
to run required actions by instantiatingTM1SpliceExecutor
and running its method corresponding to the action required from CLI.
TM1RuleParser
(class in TM1RuleParser.py
)
This class is responsible to translate parsed rule into objects in memory that represent the parsed syntax. Methods with name starting with make_
or add_
match custom actions as defined in TM1RuleGrammar.peg
(described later). These methods are called during parsing of the TM1 rule and are fed with data from the parser defined in TM1RuleGrammar.py
.
...
This rule will be parsed by the TM1RuleParser
method get_parse_tree
and the parsing result will be returned in an instance of TM1Splice.Parser.Grammar.TM1RuleGrammar.TreeNode
class. The class contains elements
property, which is a list of parsed tokens. You may see a simplified representation of the parsed rule on below picture.
...
It is important to understand how the parser accesses the parsed tokens during the custom actions. Let’s demonstrate it on a following example:
Following simplified code represents a PEG rule that we will further refer to in the custom action methods implemented in TM1RuleParser
.
Code Block | ||
---|---|---|
| ||
DirDimElemPairList <- DirDimElemPair DirDimElemPairs* %add_dim_to_map
DirDimElemPairs <- _ "," _ DirDimElemPair %add_dim_to_map2
DirDimElemPair <- DimIdentifier _ ":" _ DirElementList %make_dim_elem_pair
DimIdentifier <- "'" [^\'\"\@\<\!\[\]\^\*\>\=\\\|\?\/\;\,\~\%\&\:\n\r]+ "'" %make_ident |
The parser generated by Canopy interfaces all tokens trough an element list. This list is always passed as an argument to a custom action method and contains tokens indexed according to a PEG parsing rule that activated the custom action.
For better understanding let’s explain how the first parsing rule will affect content of the element list when add_dim_to_map
method is called. Note: This method will called once the parser has parsed a dimension/element pair or pairs.
Code Block |
---|
def add_dim_to_map(self, input, start, end, elements):
dim_map = dict()
if elements[0] and elements[1]:
for dim, elem in [*[elements[0]], *(elements[1].elements)]:
dim_map[dim] = elem
return dim_map |
As you can see from the method header and implementation, the elements
list is used in the code logic to access DirDimElemPair
content by accessing elements[0]
. Index 0
corresponds with position of DirDimElemPair
non-terminal symbol in the rule. It is first symbol that has been parsed and since the elements list is zero based, the index of the symbol is 0
.
Similarly for DirDimElemPairs
, the symbol is second in the rule and this its index will be 1
. Note that DirDimElemPairs
is followed by *
in the PEG rule, which indicates zero or n
repetitions of the non-terminal symbol. That's the reason why we access elements[1].elements
as we intend to read all occurrences of the DirDimElemPairs
non-terminal symbol as have been parsed.
Non-terminal symbol as per definition needs to be further parsed, so we can follow the parsing logic of DirDimElemPair
until let's say DimIndetifier
rule. You may see from the rule itself, once it will be parsed, the parser will call a custom method make_dim_elem_pair
.
Code Block |
---|
def make_dim_elem_pair(self, input, start, end, elements):
if type(elements[4]) is list:
pairs = []
for (hierarchy, element) in elements[4]:
pairs.append((hierarchy if hierarchy else elements[0], element))
return (elements[0], pairs)
else:
if not elements[4][0]:
elements[4] = (elements[0], elements[4][1])
return (elements[0], [elements[4]]) |
Again you may notice the custom action method header is following the same pattern as in previous case. We have elements
list from which we can access all tokens as parsed. In this case we can find DimIdentifer
contents in elements[0]
, elements[1]
would contain all whitespace characters that could follow a dimension identifier (or if there are none, elements[1]
will be empty). Next, elements[2] would contain a single character : that is used to divide an element name from the dimension name. And similarly, elements[3]
would contain all whitespace characters that could follow a :
separator. Now, elements[4]
refers to tokens that were collected when parsing DirElementList
non-terminal symbol.
We followed expansions of each non-terminal symbol in above examples, but the parser needs to use terminal symbols to be able to properly select which parsing rules apply or if the input doesn’t follow the rule logic (in that case the parser would output a parse error). Let’s demonstrate this on the last rule the parser would use when reading a dimension name. The rule that applies in this case is expansion of DimIdentifier
. This rule looks complicated, but can be interpreted as a string enclosed in single quotes consisting of at least one allowed character. If the parser reads such a token, it will call custom action method make_ident
. Let’s have a look how the method is implemented:
Code Block |
---|
The class defines following attributes:
...
Method | Usage |
---|---|
| Custom action: Create a |
| Custom action: Creates a |
| Custom action: Creates a |
| Custom action: Creates instance of |
| Custom action: Creates instance of |
| Custom action: Will call |
| Custom action: Creates a |
| Custom action: Creates a |
| Custom action: Creates a |
| Custom action: Returns a string representing an identifier parsed by the parser - for example it may be a dimension, hierarchy or element name. |
| Custom action: Returns a string representing an ID to use to retrieve MDX query from |
| Custom action: Returns a string representing name of a region after |
| Custom action: Returns a dictionary of elements that follow a dimension name in a directive or in an area statement indexed by the dimension name. |
| Custom action: Returns a tuple containing a dimension and an element name. |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
|
ArgumentParser
(class in ArgumentParser.py
)
The class implements command line parsing and is used by main application defined in Splicer.py
.
...