Automation of 'Maintain FDMEE Data Tables' System Maintenance Task

For those who are new to my blog, we have already covered all the Oracle Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks with the step-by-step process to manually execute them. If you have not already gone through those articles, I would suggest you click on the below links and read them first to get a comprehensive understanding of all the Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks.

FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables
FDMEE: System Maintenance Tasks: Maintain ODI Session Data
FDMEE: System Maintenance Tasks: Maintain Application Folder
FDMEE: System Maintenance Tasks: Maintain Process Tables
FDMEE: System Maintenance Tasks: Maintain EBS GL Balances Table

Running 'FDMEE System Maintenance Tasks' manually for multiple combinations of Target Applications and Categories requires a lot of manual effort causing it to be a tiresome process because for each maintenance task you need to manually select all the combinations of target applications and categories one by one in Data Management (FDMEE) console for a particular set of start and end period. So why not have a script which does all of these tasks for us and make life easier.

But now using below given automated script you no more need to do all these activities manually These automated jobs can be scheduled in Windows Task Scheduler to run at any chosen day/time. Also, these automated scripts don’t require any outage on FDMEE application and run smoothly using only the inbuilt functionality of FDMEE.

Note:

• The demonstrating Hyperion environment has the 'Windows Server 2012 R2 Standard' operating system. 
• The demonstrating Hyperion environment has 'Oracle database server 12.2.0.2 (18c)' as backend database. 
• This post has been written and associated activities have been demonstrated on Oracle FDMEE version 11.1.2.4.220.
• There is no outage required on the FDMEE application to execute these scripts. But it is advisable not to run any data loads during the activity.
• Don't forget to take complete FDMEE schema backup before attempting to run these scripts. 
• Different paths and folder names mentioned in this post may slightly vary for different Hyperion setups but at large it should be the same.  

In this blog series, we are going to learn how can we automate these 'FDMEE System Maintenance Tasks' one-by-one and schedule it via Windows Task Scheduler to run whenever we want to do FDMEE System maintenance activities.

FDMEE batch utility executescript.bat
------------------------------------------------------------
Oracle provides a utility called executescript.bat to run each of these System Maintenance Tasks as well as other custom scripts from the command line. This command-line utility allows us to execute custom scripts either from Windows or Unix/Linux systems. Unfortunately, this topic is not well documented in the FDMEE admin guide.

You will find the executescript.bat (among others) in the folder E:\apps\OracleEPM\Middleware\user_projects\epmsystem_fdm\FinancialDataQuality.

Using utility executescript.bat you can automate all the FDMEE System Maintenance Tasks.

Command syntax of FDMEE batch utility executescript.bat:
----------------------------------------------------------------------------------
executescript <username> <password> "<script name>" "<parameters>" <execution mode>

Where:

• <username>: is your FDMEE user running the script
• <password>: can be either the hard-coded password or a file with encoded password (-f:filename)
• <script name>: is the name of the script registered in Script Registration page (not the python script filename). You find the script name here: Navigate-Administer-Data Management-Setup tab-Script Registration
• <parameters>: are the script parameters in the form "Display Name=Parameter Value"
• <execution mode>: is either SYNC (synchronous) or ASYNC (asynchronous). SYNC—Process runs immediately and control returns when process completes. ASYNC—When the ODI job is submitted, control returns. The load process continues to execute in ODI.

Now the command syntax in more detailed form will look like below:

executescript <username> <password> "<script name>" "<parameterdisplay1=value1>" "<parameterdisplay2=value2>" <SYNC | ASYNC>

If you are going to use an encrypted password (which is the recommended best practice), following will be the command syntax:

executescript <username> <password | -f:password.txt> "<script name>" "<parameterdisplay1=value1>" "<parameterdisplay2=value2>" <SYNC | ASYNC>

You will find the configuration steps for password encryption in the FDMEE admin guide as shown below:

You need to ensure that your encrypted password.txt file is placed in the location defined in Navigate-Administer-Data Management-Setup tab-System Settings-Encrypted Password Folder.

Automating 'Maintain FDMEE Data Tables' System Maintenance Task:
--------------------------------------------------------------------------------------------
In this post, we will see how to automate 'Maintain FDMEE Data Tables' System Maintenance Task. I suggest you to first read this post FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables for theoretical understanding, manual implementation, and post-execution activities of 'Maintain FDMEE Data Tables' System Maintenance Task.

When you execute ‘Maintain FDMEE Data Tables’ purge script manually from Data Management console, it seeks the following input parameters:

• Target Application
• Start Period
• End Period
• Category

To run ‘Maintain FDMEE Data Tables’ purge script using executescript.bat command-line utility below is the command syntax:

executescript username password "MaintainFDMEEDataTables" "Target Application=TargetAppName" "Start Period=YYYY-MM-DD" "End Period=YYYY-MM-DD" "Select Category=CategoryKey" SYNC

Description:

• TargetAppName should be entered the same as what you get in the output by running below query against FDMEE schema:

          SELECT APPLICATION_NAME FROM AIF_TARGET_APPLICATIONS; 

Above query-output is same as what you see in Data Management console Target Applications list values as shown below:

• Start Period and End Period can be chosen from the values what you get in the output by running below query against FDMEE schema:

          SELECT PERIODDESC, PERIODKEY FROM TPOVPERIOD;

Above query-output is same as what you see in Data Management console Period list values as shown below:

If you want to quickly check what is the min and max value of period to help you choose your Start Period and End Period, you can run below query against FDMEE schema:

           SELECT MIN(PERIODKEY), MAX(PERIODKEY) FROM TPOVPERIOD; 

Note: Start Period and End Period values must be entered in the 'YYYY-MM-DD' format only.

• In Category segment, you need to enter the 'Category Key (number)' what you get in the output by running below query against FDMEE schema:

          SELECT CATNAME, CATKEY FROM TPOVCATEGORY;

Above query-output is same as what you see in Data Management console Category list values as shown below:

Most of the time, we need to run 'MaintainFDMEEDataTables' purge script for many combinations of various Target applications and Categories for a selected Start Period and End Period range.

Let's understand the automation script for 'MaintainFDMEEDataTables' with the help of an example.

Suppose using FDMEE Admin user, you want to run 'MaintainFDMEEDataTables' purge script for 4 Target applications name TargetApp1, TargetApp2, TargetApp3, TargetApp4, and 3 Categories keys name CategoryKey1, CategoryKey2, CategoryKey3 for Start Period Oct 2017 to End Period Apr 2018.

In order to achieve the above goal, create a batch script name 'MaintainFDMEEDataTables.bat' under folder E:\Admin\scripts having the following content:

MaintainFDMEEDataTables.bat:
--------------------------------------------------
@echo off

REM Set the value for the variables 
set "username=Admin"
set "password=YourAdminPassword"
set "startperiod=2017-10-01"
set "endperiod=2018-04-01"
set "logfile=E:\Admin\scripts\output.log"

REM Clear the content of existing logfile
rem.> %logfile%

REM Navigate the executescript.bat utility folder
cd "E:\apps\OracleEPM\Middleware\user_projects\epmsystem_fdm\FinancialDataQuality"

REM Run executescript to trigger "MaintainFDMEEDataTables" for all combinations of 4 Target applications and 3 Category Keys for the mentioned Start and End Period range

for %%f in (TargetApp1 TargetApp2 TargetApp3 TargetApp4) do (
    for %%i in (CategoryKey1 CategoryKey2 CategoryKey3) do (
      echo call executescript %username% %password% "MaintainFDMEEDataTables" "Target Application=%%f" "Start Period=%startperiod%" "End Period=%endperiod%" "Select  Category=%%i" SYNC
call executescript %username% %password% "MaintainFDMEEDataTables" "Target Application=%%f" "Start Period=%startperiod%" "End Period=%endperiod%" "Select Category=%%i" SYNC
)
echo ------------------------------------------------------------------------------
echo ------------------------------------------------------------------------------
) >>%logfile%

cd E:\Admin\scripts

Change the paths and variables mentioned in the script as per your requirement and FDMEE application setup.

You can put your Target Application names in the segment (TargetApp1 TargetApp2 TargetApp3 TargetApp4) and Category Keys in (CategoryKey1 CategoryKey2 CategoryKey3) separated by single space. 

Now login to your FDMEE server, open an Administrator Command prompt, and run 'MaintainFDMEEDataTables.bat'.

After completion of the script your output.log will be generated in below format:

To verify whether your process IDs 8237, 8238... have successfully completed or not, we can navigate to Process Details page (In Data Management console, click on the Process Details link under the Workflow tab—Monitor—Process Details):

We can see our process IDs 8237, 8238 have executed successfully. Click on the 'Show' button for Process IDs 8237 to open the corresponding log file to see more details. Similarly, you can verify other process IDs recorded in output.log file for various combinations of Target Applications and Categories.

That's all for this post.

I hope this article has helped you.
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!