Metadata syncing service

The “metadata syncing service” is a mechanism to synchronize metadata between reporting instances and the Apliqo UX Content Store. The scope of the synchronization is quite limited and includes only:

• Principal name of all elements in the }Clients dimension and the }TM1_DefaultDisplay value alias

• Principal name of all elements in the }Groups dimension and the }TM1_DefaultDisplay value alias

• Members belonging to each group

• Principal name of all elements in the }Cubes dimension

• Principal name of all elements in the }Dimensions dimension

• The Default_Member of each hierarchy as defined in the }HierarchyProperties cube

Benefits

It is possible to have a fully functioning Apliqo UX application without ever using the metadata syncing service. However, if you do use the service you will get up and running with your application faster, avoid manual maintenance tasks, and be able to take advantage of more advanced features.

The primary reason for being of the metadata syncing service is to reduce manual

maintenance! Specifically to eliminate the need to set up users and groups in the Content Store when all users, and presumably all needed role definitions and group memberships are already maintained elsewhere in your primary TM1 server instances (“reporting instances” from the perspective of Apliqo UX).

The metadata syncing service can completely automate user and group creation for you, including group memberships, providing that the definition of the group memberships exist on another server which is accessible to Apliqo UX. In addition, by syncing dimension and cube metadata you can define default context elements for key “point of view” dimensions which will be applied to the setting service when users log in, and you can apply default row and column layouts to cubes which facilitates faster report building by your power users.

Technical

The syncing service is provided by the UXToolkit utility (uxtoolkit.exe) which is included with the installation of the Apliqo Application Server. The utility is a windows command line executable based on tm1Py and is part of the UX installation since version 2.5.2. The utility is also available in earlier versions of Apliqo UX as a python script. This requires significantly more setup as the correct version of python, tm1Py and other module dependencies must be installed manually.

Inside the root directory of the apliqo server you should see a “py” folder which contains

1. uxtoolkit.exe

2. setings.json

3. readme.pdf

Open

The ReadMe document contains a change log for the utility and detailed technical documentation of the parameters which can be passed via command line. This document should contain sufficient detail for regular usage and standard use cases. However, please refer to the ReadMe document in the installation directory for the definitive list of available features and available parameters with your version.

The settings.json file is very important. This file contains the instructions for the utility to log into the ContentStore and the location of the instances.json file associated with your Apliqo UX application.

Open

The password field can contain plain or encrypted text. We recommend encrypting the stored password using the utility’s “store” command (described below).

pathToInstances can be either a full path or a relative path based on the location of the settings.json file.

» Basic parameters

There are a number of command line arguments which can be passed to the UX Toolkit regardless of what kind of operation it is performing. Arguments which are specific to a defined "task mode" are explained in the following section on each mode.

• path : the path to the settings.json file. If omitted then the toolkit assumes the settings file is located in the same directory as the uxtoolkit.exe itself e.g.

• instances_path : the path to the instances.json file. If omitted then the toolkit will use the pathToInstances value from the settings.json file

• cs_cam : the CAM Namespace for authentication to the ContentStore. If omitted then the toolkit will use the value from the settings.json file

• cs_user : the user ID for authentication to the ContentStore. If omitted then the toolkit will use the value from the settings.json file. If using CAM then the user ID must be the user ID used when authenticating to CAM. For all other authentication modes it should be the element name from the }Clients dimension

• cs_password : the password for the specified user for authentication to the ContentStore. If omitted then the toolkit will use the value from the settings.json file. Can be passed either as plain text or encrypted string

• key : the encryption key for password encryption/decryption. If omitted then the toolkit will use a built-in default key

• debug : instructs the utility to run in debug mode and write debug log to the console

The above parameters apply to all operating modes (store, transfer, createattrs, clean) and must be placed BEFORE the positional arguments in the command line string.

» Positional arguments (modes of operation)

In addition to the basic parameters, the utility accepts one of 4 alternate positional arguments which control the mode of operation. These are:

• store : stores username, password, CAM Namespace, pathToInstances into the specified settings.json file

• clean : performs a "clean up" of a specified instance or all instances available in the instances.json file. Objects that are deleted from the system are all redundant objects from Apliqo UX 1.x versions (e.g. objects with }APQ C3 Canvas* naming)

• transfer : transfer metadata to the Content Store. Which data is transferred depends on supplied modifying parameters (-initialize, -security, -data, -all)

• createattrs : create attributes in a selected instance or in all instances based on Create flag in }APQ UX Instance Dimension Attribute cube

The modes of operation and additional modifying parameters for each more are described in the next section.

Operating Modes

The utility has 4 different modes of operation or tasks which it can perform.

» Store

Store mode is designed as an assistance to initial configuration and writes (stores) values to the settings.json file. In all other modes the settings file is read by the utility and provides default parameters for connecting to the Content Store and the reporting instances. The Toolkit will also attempt to authenticate to the Content Store with the credentials in (or being written to) the settings.json file. If connection is unsuccessful the routine will abort with error and not update the file. If successful then in addition to the settings file being updated the authentication credentials will also be stored in the }APQ UX Instance Attributes cube in the Content Store.

This is most likely a once off configuration task which will be run once manually. There would only be a need to run again if the password changes for the background admin account or the location of the webapp is changed.

Of course you can edit the settings.json file directly in Notepad++ or Visual Studio Code but to store the connection password as an encrypted string then you must use the store function. Store mode accepts the following modifying parameters:

• cam : the CAM Namespace for authentication to the ContentStore. If omitted then the toolkit will not update the CAMNamespace value in the settings.json file

• user : the user ID for authentication to the ContentStore. If omitted then the toolkit will not update the userName value in the settings.json file. If using CAM then the user ID must be the user ID used when authenticating to CAM. For all other authentication modes it should be the element name from the }Clients dimension

• password : the password for the specified user for authentication to the ContentStore. If omitted then the toolkit will not update the password value in the settings.json file. Can be passed either as plain text or encrypted string

• instances_path : the path to the instances.json file for the webapp. If omitted then the toolkit will not update the pathToInstances value in the settings.json file

• instance : a specific instance from instances.json. If an instance is specified then the connection credentials for the instance will be updated in the }APQ UX Instance Attributes cube only and the settings.json file will not be edited. If omitted then the toolkit will update ContentStore credentials in the settings.json file

E.g. to encrypt the password and store directly into settings.json file:

From UX version 2.5.2 there is now an administration app provided to make this process easier (you don’t have to use command line unless you really want to!)

In the background the built-in content Store process }APQ.UX.Toolkit.Run.Store provides a wrapper to call the uxtoolkit.exe program via ExecuteCommand.

» Transfer

Transfer mode is the main work of the UX Toolkit utility. Transfer mode connects to each specified reporting instances and reads the }Clients, }Groups, }Dimensions and }Cubes metadata and transfers this information to staging cubes in the Content Store.

Transfer mode accepts the following modifying parameters:

• initialize : reads the instances.json file and checks all instances are set up in the }APQ UX Instance dimension. Any instance in the logonInstances array will have the credentials for ContentStore connection duplicated to the instance entry in the }APQ UX Instance Attributes cube

• security : reads the instances.json file and authenticates to each instance and writes the contents of the }Clients and }Groups dimensions as well as the }ClientGroups cube back to the ContentStore

• data : reads the instances.json file and authenticates to each instance and transfers security and then writes the contents of the }Dimensions and }Cubes dimensions back to the ContentStore. (E.g. performs security step, then also copies cube & dimension names)

• all : performs the initialize and data steps

• cs_lead : Boolean default=false. If true then only groups previously marked with the ContentStoreImport flag will be queried for their members to update to the Content Store

Positional parameters dictating the mode of operation can of course be combined with global parameters which need to be specified before the mode n the command line. For example:

• instance : a specific instance from instances.json. If an instance is specified then the Toolkit will connect to and copy metadata only for the specified instance. If omitted then the toolkit will cycle through all instances in the instances.json file (including the Content Store itself)

Typically "initialize" is a once off operation. Likewise, once the list of dimensions and cubes has been synced with the "data" step this would not usually need to be run again. However the "security" sync would typically be done on a regularly scheduled basis (e.g. nightly) in order to keep users and (synced) group memberships in sync between the reporting instance and Content Store and moreover to keep the Content Store maintenance free.

Scheduling such a task is a simple process with the built-in content Store process }APQ.UX.Toolkit.Run.Transfer which provides a wrapper to call the uxtoolkit.exe program via ExecuteCommand.

From UX version 2.5.2 there is now an administration app provided to make administering the metadata syncing easy.

In the application below the "Fetch Users & Groups" button (marked step #1) runs the process }APQ.UX.Toolkit.Run.Transfer as described above, which calls uxtoolkit.exe in transfer -security mode

This syncs the reporting instance security model to the }APQ UX Instance ClientGroups cube. The UX administrator then has the option to flag whether to automatically import users and groups into the content store. Typically you would import all users (step #2 left). However, you would probably opt to only import groups to the content store which are relevant to Content Store security (step #2 right). (You may decide not to import any groups.)

The button "Sync to Content Store" (step #3) will run the process }APQ.UX.Security.ClientGroups.Import.FromInstances which will create all users marked to be imported and likewise for all groups marked to be imported will match the group memberships. Note that this process is already include in the daily maintenance chore. However you could also run this directly after any scheduled UX Toolkit sync in order to fully sync the Content Store security to the reporting instance in the lead.

» Create Attributes

Many advanced features in Apliqo UX can be easily controlled via attributes. Some attributes which control features in Apliqo UX are standard attributes which have universal properties in other Planning Analytics UIs like Caption or Format. However, there are other attributes which have a special purpose in Apliqo UX. For example:

• Color : if an attribute is set with a standard color name or HexCode this will be recognized by Apliqo UX and all data series will be automatically plotted in the selected color consistently in all reports

• ibcs-class : this attribute can be used in version or scenario dimensions but also in continuous time dimensions for consistent formatting in both table column headers and chart series. The use case is similar to the Color attribute, for example to ensure that Budget, Actual and Last Year are always formatted consistently.

• CellType : This attribute is most commonly applied to measure dimensions used on column or rows in Apliqo UX applications. Common cell types are "Checkbox" which applies a table cell class of checkbox (as used on this screen) and "Date" which signals the cell value should be entered via data picker modal.

• Group : If a group attribute is set this is used on column dimensions of views to automatically "nest" elements and allow whole groups to be displayed or hidden with a single click.

• Renderer : The renderer attribute is to support "Scorecard Metrics" KPI reports with the same formatting as supported by standard IBM Cognos interfaces.

Using the createattrs mode switch in the UX Toolkit we can easily create such attributes on the target reporting instance.

In the background the }APQ.UX.Toolkit.Run.CreateAttrs wrapper process calls UX Toolkit. See the section on the built-in administration app for more information.

» Clean

The UX Toolkit has one final function which is to assist in removing redundant objects on reporting instances from Apliqo UX 1.x applications. (That is pre-Content Store versions of UX).

By running the Toolkit in clean mode all "C3 Canvas" objects will be automatically removed from the system.