FDMEE: System Maintenance Tasks: Maintain Process Tables

Hi Friends,

This is the fourth post under Oracle Hyperion Financial Data Quality Management (FDMEE): System Maintenance Tasks blog series where we are going to cover the Maintain Process Tables script.

If you have not yet read the previous three posts of this blog series, I would suggest you to go through them on below links to get a comprehensive picture of Oracle Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks:

FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables
FDMEE: System Maintenance Tasks: Maintain ODI Session Data

FDMEE: System Maintenance Tasks: Maintain Application Folder

In this fourth post, we will focus on the ‘Maintain Process Tables’ task and will see how to manually execute ‘Maintain Process Tables’ purge script.  

Important Note:

• This post has been written and associated activities have been demonstrated on Oracle Hyperion Financial Data Quality Management (FDMEE) version 11.1.2.4.210.
• The demonstrating Hyperion environment has Oracle database server 12.2.0.2 (18c) as backend database. 
• There is no outage required on the Oracle Hyperion Financial Data Quality Management (FDMEE) application to execute these purge scripts. But it is advisable not to run any data loads during the purge activity.
• Based on the data retention policy of your organization, you should pre-decide how many days of data you want to retain for Oracle Hyperion Financial Data Quality Management (FDMEE) Process Tables.
• Don't forget to take complete Oracle Hyperion Financial Data Quality Management (FDMEE) schema backup before attempting to run these purge scripts.

Maintain Process Tables

The purpose of running Maintain Process Tables script is to purge old/historical Process IDs and related details from associated execution tables in Oracle Hyperion Financial Data Quality Management (FDMEE). This will not only reduce the overall FDMEE schema size but also help improve the performance of your FDMEE application. 

The “Maintain Process Tables” script cleans up the Process IDs in the Process details related FDMEE tables by the number of days you enter as input parameter while running the script.

It deletes entries from following Oracle Hyperion Financial Data Quality Management (FDMEE) execution tables (AIF_ tables):

• AIF_PROCESSES
• AIF_PROCESS_DETAILS
• AIF_PROCESS_LOGS
• AIF_PROCESS_PARAMETERS
• AIF_PROCESS_PERIODS
• AIF_PROCESS_STEPS
• AIF_BAL_RULE_LOADS
• AIF_BAL_RULE_LOAD_PARAMS
• AIF_BATCH_JOBS
• AIF_BATCH_LOAD_AUDIT
• AIF_TEMP

Before proceeding further, with purging steps, I would suggest you to check and note down the pre-purge FDMEE schema size to see how much space has been freed up post purge activity.

Step-by-Step process to execute ‘Maintain Process Tables’ purge script

1- Log in to Workspace with Admin credentials.

2- Open Oracle Hyperion Financial Data Quality Management (FDMEE) console (Data Management) from Workspace and on the Workflow tab, under Other, select Script Execution:

3- In Script Execution, and then in Custom Script Group, select System Maintenance Tasks.

4- From the Scripts grid, select Maintain Process Tables script and Click Execute.

5- It will prompt to enter the following parameter value on the Execute Script screen.

Days to keep records

Enter the number of days you want to retain entries for in FDMEE Process Tables from today’s date. For example, if you want to delete all the entries except all 3 months (90 days) entries in FDMEE Process Tables, enter 90 in Days to keep records.

Note: Maintain Process Tables scrip takes some time to execute so it’s advisable not to purge too much in one go.

6- Keep Execution mode selected as Online. The online method processes the report immediately.

Execution modes:

online  — ODI processes the data in sync mode (immediate processing).

offline — ODI processes the data in async mode (runs in background).

7- Optionally, If you want to schedule the purge script run rather than running it instantly, Click on Schedule option and enter the scheduling details as per your requirement.

8- If you want to instantly run the purge script, then directly click on OK (not on the Schedule option). 

You will see a message informing the process ID of your script run.

9- To see the status of your script execution, click on the Process Details link under the Workflow tab—Monitor—Process Details. 

Click on the Show button for the Process ID of your script run to open the corresponding log file.

In the log file, look for the message "Maintain Process Tables: Completed" to confirm that script has run successfully without any error. If you encounter any errors, troubleshoot them based on the cause mentioned in the log file. In our case, we see no issue in the script execution.

Once you are done with running the Maintain Process Tables purge script, you can check your Hyperion Financial Data Quality Management (FDMEE) schema size to see how much space has been cleared as a result of FDMEE Process Tables purging.

Here it’s important to note that after running the FDMEE Process Tables purge script, your DBA needs to do FDMEE tables reorganization activity in order to reclaim freed space. 

Actually, when you delete the data from the Oracle tables, the data blocks will go onto the free list for that table. That free space will be used for the subsequent insert and update operations into the table. You need to perform table reorganization to reclaim that free space.

Note: Reorganization of tables requires outage in your Hyperion environment so you have to stop all your application services prior to the reorganization activity to be performed by your application DBA.

Don't forget to ask your DBA team to 'Rebuild Indexes' in your Hyperion Financial Data Quality Management (FDMEE) database after performing the 'Tables Reorganisation' activity.

If you miss rebuilding indexes after tables reorganization activity, your FDMEE data load job will be taking longer than normal to complete.

Index rebuild becomes absolutely necessary especially when you delete millions of records from FDMEE tables as part of running FDMEE purge scripts because the corresponding indexes of those deleted records go in the unusable state after tables reorganization activity. Due to this bulk deletion of records, indexes become fragmented causing indexes taking more time to give the output and hence delaying your FDMEE data load job.

You will observe the following type of error during your FDMEE data load run if you do not rebuild database indexes after tables reorganization activity:

ORA-01502: index 'FDMEE_ODI.PK_AIF_*' or partition of such index is in unusable state.

Once both the DBA tasks (Tables reorganization + Rebuilding Indexes) are completed, compare pre and post purge size of your Hyperion Financial Data Quality Management (FDMEE) schema to see how much has the schema size reduced. 

Maintain Process Tables purge script can also be automated to run using Executescript batch file located in the folder: 

<drive>\Oracle\Middleware\user_projects\epmsystem1_fdm\FinancialDataQuality.

We will cover the automation of Maintain Process Tables purge script in a separate post of this blog series. 

That's all for this post.

I hope this article has helped you.
Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!