FPM contains several parallel enabled processes to provide faster way of data copy between FIN General Ledger TB, FIN General Ledger and planning cubes. The processes are mainly part of roundtrip data processing.
Parallelization Mechanism Introduction
Parallel processing in FPM is initiated by the process itself from its main thread by spawning child threads and once the child threads are running, main thread is monitoring their status and waiting till all child threads finish. Once finished, the main thread continues. It’s important to note that all parallel enabled processes in FPM share common code for both the main thread and child threads. This means that both are implemented by a single TI process.
Parallelization Mechanism Details
Main Thread Role
Main thread serves two purposes. It spawns child threads and once spawned it starts to monitor the child threads and wait until all child threads have finished.
Monitoring of the threads runs in two phases, first during the initiation of the child threads and second during their execution. The main thread watches for timeouts during both phases to prevent zombie threads running on the server.
The main thread has the option to participate in data processing. However, doing so might compromise its capability to identify time out scenarios. This is because the main thread itself could time out, preventing it from accessing the code that oversees the child threads.
Spawning Threads
Data Allocation to Threads
no mapping between source and target cube dimensions
mapping between source and target cube dimensions
Thread Polling Mechanism
As of TM1 version 11 there aren’t any specific functions for synchronizing threads in parallel processes. The current solution depends on checking the thread status in the }APQ C3 Thread Control cube. The main process always performs this check, while the child threads only need to update their status in the }APQ C3 Thread Control cube.
The checking mechanism is based on the periodic execution of the CubeSaveData command on the }APQ C3 Thread Control cube. The main process runs this command at short intervals, allowing it to detect any changes in the data from the child threads. This is crucial because TM1's commit model typically doesn't allow other processes to see data changes in a cube until the process that's modifying the data has finished.
This method ensures that the main thread can identify changes in the control cube while the child threads are running. Although this approach has its drawbacks, it doesn’t depend on semaphore files or external software and is implemented entirely as TI code.
Note: The situation will change with TM1 version 12, which will no longer support CubeSaveData command, but will support thread synchronization out of the box. As a possible solution to this problem, there could be two versions of the parallel enabled processes in the FPM model - one for version 11 and second for version 12 and above. Switching between these can be simply provided by modified roundtrip configurations or by assigning proper process version to respective roundtrip step.
Thread Initiation Polling
The polling interval is a configuration variable that can be set in }APQ Settings cube and should be set to a reasonable value. Setting the value too low can lead to various locking situations, setting the value too high will cause delay the main process execution as the signal to continue with main process execution comes at periodic intervals.
Since there are no tools to control child threads from the main thread, the parallel processing algorithm implements two different timeouts for child thread monitoring to prevent main process stalling in the polling loops and thus having zombie processes running on the TM1 server.
First timeout is used to monitor if all child threads have initiated correctly within a specified time period. Two types of exception can happen during this period:
The child thread will not initiate and will exit before it started processing data (for example due to a data source error).
The child thread has time related issues during its prolog initialization (for example due to infinite loop in the prolog code or a source view that calculates slowly).
Thread Completion Polling
Messages Collection and Final Message
The polling interval is a configuration variable that can be set in }APQ Settings cube and should be set to a reasonable value. Setting the value too low can lead to various locking situations, setting the value too high will cause delay the main process execution as the signal to continue with main process execution comes at periodic intervals.
Since there are no tools to control child threads from the main thread, the parallel processing algorithm implements two different timeouts for child thread monitoring to prevent main process stalling in the polling loops and thus having zombie processes running on the TM1 server.
First timeout is used to monitor if all child threads have initiated correctly within a specified time period. Two types of exception can happen during this period:
The child thread will not initiate and will exit before it started processing data (for example due to a data source error).
The child thread has time related issues during its prolog initialization (for example due to infinite loop in the prolog code or a source view that calculates slowly).
The main process will monitor each thread initiation status in the }APQ C3 Thread Control cube and if any of the child threads