Automation of 'Maintain FDMEE Data Tables' System Maintenance Task

For those who are new to my blog, we have already covered all the Oracle Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks with the step-by-step process to manually execute them. If you have not already gone through those articles, I would suggest you click on the below links and read them first to get a comprehensive understanding of all the Hyperion Financial Data Quality Management (FDMEE) System Maintenance Tasks.

FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables
FDMEE: System Maintenance Tasks: Maintain ODI Session Data
FDMEE: System Maintenance Tasks: Maintain Application Folder
FDMEE: System Maintenance Tasks: Maintain Process Tables
FDMEE: System Maintenance Tasks: Maintain EBS GL Balances Table

Running 'FDMEE System Maintenance Tasks' manually for multiple combinations of Target Applications and Categories requires a lot of manual effort causing it to be a tiresome process because for each maintenance task you need to manually select all the combinations of target applications and categories one by one in Data Management (FDMEE) console for a particular set of start and end period. So why not have a script which does all of these tasks for us and make life easier.

But now using below given automated script you no more need to do all these activities manually These automated jobs can be scheduled in Windows Task Scheduler to run at any chosen day/time. Also, these automated scripts don’t require any outage on FDMEE application and run smoothly using only the inbuilt functionality of FDMEE.

Note:

• The demonstrating Hyperion environment has the 'Windows Server 2012 R2 Standard' operating system. 
• The demonstrating Hyperion environment has 'Oracle database server 12.2.0.2 (18c)' as backend database. 
• This post has been written and associated activities have been demonstrated on Oracle FDMEE version 11.1.2.4.220.
• There is no outage required on the FDMEE application to execute these scripts. But it is advisable not to run any data loads during the activity.
• Don't forget to take complete FDMEE schema backup before attempting to run these scripts. 
• Different paths and folder names mentioned in this post may slightly vary for different Hyperion setups but at large it should be the same.  

In this blog series, we are going to learn how can we automate these 'FDMEE System Maintenance Tasks' one-by-one and schedule it via Windows Task Scheduler to run whenever we want to do FDMEE System maintenance activities.

FDMEE batch utility executescript.bat
------------------------------------------------------------
Oracle provides a utility called executescript.bat to run each of these System Maintenance Tasks as well as other custom scripts from the command line. This command-line utility allows us to execute custom scripts either from Windows or Unix/Linux systems. Unfortunately, this topic is not well documented in the FDMEE admin guide.

You will find the executescript.bat (among others) in the folder E:\apps\OracleEPM\Middleware\user_projects\epmsystem_fdm\FinancialDataQuality.

Using utility executescript.bat you can automate all the FDMEE System Maintenance Tasks.

Command syntax of FDMEE batch utility executescript.bat:
----------------------------------------------------------------------------------
executescript <username> <password> "<script name>" "<parameters>" <execution mode>

Where:

• <username>: is your FDMEE user running the script
• <password>: can be either the hard-coded password or a file with encoded password (-f:filename)
• <script name>: is the name of the script registered in Script Registration page (not the python script filename). You find the script name here: Navigate-Administer-Data Management-Setup tab-Script Registration
• <parameters>: are the script parameters in the form "Display Name=Parameter Value"
• <execution mode>: is either SYNC (synchronous) or ASYNC (asynchronous). SYNC—Process runs immediately and control returns when process completes. ASYNC—When the ODI job is submitted, control returns. The load process continues to execute in ODI.

Now the command syntax in more detailed form will look like below:

executescript <username> <password> "<script name>" "<parameterdisplay1=value1>" "<parameterdisplay2=value2>" <SYNC | ASYNC>

If you are going to use an encrypted password (which is the recommended best practice), following will be the command syntax:

executescript <username> <password | -f:password.txt> "<script name>" "<parameterdisplay1=value1>" "<parameterdisplay2=value2>" <SYNC | ASYNC>

You will find the configuration steps for password encryption in the FDMEE admin guide as shown below:

You need to ensure that your encrypted password.txt file is placed in the location defined in Navigate-Administer-Data Management-Setup tab-System Settings-Encrypted Password Folder.

Automating 'Maintain FDMEE Data Tables' System Maintenance Task:
--------------------------------------------------------------------------------------------
In this post, we will see how to automate 'Maintain FDMEE Data Tables' System Maintenance Task. I suggest you to first read this post FDMEE: System Maintenance Tasks: Maintain FDMEE Data Tables for theoretical understanding, manual implementation, and post-execution activities of 'Maintain FDMEE Data Tables' System Maintenance Task.

When you execute ‘Maintain FDMEE Data Tables’ purge script manually from Data Management console, it seeks the following input parameters:

• Target Application
• Start Period
• End Period
• Category

To run ‘Maintain FDMEE Data Tables’ purge script using executescript.bat command-line utility below is the command syntax:

executescript username password "MaintainFDMEEDataTables" "Target Application=TargetAppName" "Start Period=YYYY-MM-DD" "End Period=YYYY-MM-DD" "Select Category=CategoryKey" SYNC

Description:

• TargetAppName should be entered the same as what you get in the output by running below query against FDMEE schema:

          SELECT APPLICATION_NAME FROM AIF_TARGET_APPLICATIONS; 

Above query-output is same as what you see in Data Management console Target Applications list values as shown below:

• Start Period and End Period can be chosen from the values what you get in the output by running below query against FDMEE schema:

          SELECT PERIODDESC, PERIODKEY FROM TPOVPERIOD;

Above query-output is same as what you see in Data Management console Period list values as shown below:

If you want to quickly check what is the min and max value of period to help you choose your Start Period and End Period, you can run below query against FDMEE schema:

           SELECT MIN(PERIODKEY), MAX(PERIODKEY) FROM TPOVPERIOD; 

Note: Start Period and End Period values must be entered in the 'YYYY-MM-DD' format only.

• In Category segment, you need to enter the 'Category Key (number)' what you get in the output by running below query against FDMEE schema:

          SELECT CATNAME, CATKEY FROM TPOVCATEGORY;

Above query-output is same as what you see in Data Management console Category list values as shown below:

Most of the time, we need to run 'MaintainFDMEEDataTables' purge script for many combinations of various Target applications and Categories for a selected Start Period and End Period range.

Let's understand the automation script for 'MaintainFDMEEDataTables' with the help of an example.

Suppose using FDMEE Admin user, you want to run 'MaintainFDMEEDataTables' purge script for 4 Target applications name TargetApp1, TargetApp2, TargetApp3, TargetApp4, and 3 Categories keys name CategoryKey1, CategoryKey2, CategoryKey3 for Start Period Oct 2017 to End Period Apr 2018.

In order to achieve the above goal, create a batch script name 'MaintainFDMEEDataTables.bat' under folder E:\Admin\scripts having the following content:

MaintainFDMEEDataTables.bat:
--------------------------------------------------
@echo off

REM Set the value for the variables 
set "username=Admin"
set "password=YourAdminPassword"
set "startperiod=2017-10-01"
set "endperiod=2018-04-01"
set "logfile=E:\Admin\scripts\output.log"

REM Clear the content of existing logfile
rem.> %logfile%

REM Navigate the executescript.bat utility folder
cd "E:\apps\OracleEPM\Middleware\user_projects\epmsystem_fdm\FinancialDataQuality"

REM Run executescript to trigger "MaintainFDMEEDataTables" for all combinations of 4 Target applications and 3 Category Keys for the mentioned Start and End Period range

for %%f in (TargetApp1 TargetApp2 TargetApp3 TargetApp4) do (
    for %%i in (CategoryKey1 CategoryKey2 CategoryKey3) do (
      echo call executescript %username% %password% "MaintainFDMEEDataTables" "Target Application=%%f" "Start Period=%startperiod%" "End Period=%endperiod%" "Select  Category=%%i" SYNC
call executescript %username% %password% "MaintainFDMEEDataTables" "Target Application=%%f" "Start Period=%startperiod%" "End Period=%endperiod%" "Select Category=%%i" SYNC
)
echo ------------------------------------------------------------------------------
echo ------------------------------------------------------------------------------
) >>%logfile%

cd E:\Admin\scripts

Change the paths and variables mentioned in the script as per your requirement and FDMEE application setup.

You can put your Target Application names in the segment (TargetApp1 TargetApp2 TargetApp3 TargetApp4) and Category Keys in (CategoryKey1 CategoryKey2 CategoryKey3) separated by single space. 

Now login to your FDMEE server, open an Administrator Command prompt, and run 'MaintainFDMEEDataTables.bat'.

After completion of the script your output.log will be generated in below format:

To verify whether your process IDs 8237, 8238... have successfully completed or not, we can navigate to Process Details page (In Data Management console, click on the Process Details link under the Workflow tab—Monitor—Process Details):

We can see our process IDs 8237, 8238 have executed successfully. Click on the 'Show' button for Process IDs 8237 to open the corresponding log file to see more details. Similarly, you can verify other process IDs recorded in output.log file for various combinations of Target Applications and Categories.

That's all for this post.

I hope this article has helped you.
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!

Read More

HFM Patch Set Update 11.1.2.4.209 (Patch 29343616)

In April 2020 CPU (Critical Patch Update) alert, Oracle had released HFM (Hyperion Financial Management) PSU 11.1.2.4.209 (Patch 29343616). You can find the patch details on this link: https://blogs.oracle.com/proactivesupportepm/psu_hfm_11124209

I thought let's upgrade my Hyperion Financial Management (HFM) application from the existing version 11.1.2.4.204 to this latest patch level to be up to date with HFM bug fixes. So let's see how it goes.

Note: The demonstrating Hyperion Financial Management (HFM) environment has the 'Windows Server 2012 R2 Standard' operating system. 

As usual, the first thing to check was the prerequisites. Going through the README document of the patch, we come to know that the following prerequisites patches need to be installed before installing HFM PSU 11.1.2.4.209 (Patch 29343616):

As I was already having HFM 11.1.2.4.204 version running in my Hyperion environment so I was fully confident that following 2 prerequisites patches must have been already installed on my HFM servers as part of HFM 11.1.2.4.204 (Patch 25192566) patching:

1. Fusion Middleware (FMW) - WebServices  Patch 13866584
2. Fusion Middleware (FMW) - WebLogic  Patch 16810628

Why is it so? because these two patches are required to be installed as prerequisite patches even for HFM 11.1.2.4.204 (Patch 25192566) patching. You can check the same on this link: https://blogs.oracle.com/proactivesupportepm/patch-set-update-for-oracle-hyperion-financial-management-11124204-is-available

Let's cross-check from the HFM server itself. 

As Fusion Middleware (FMW) - WebServices  Patch 13866584 is a file-replacement kind of patch, so it will not be registered under home E:\apps\OracleEPM\Middleware\oracle_common. But if you know your last HFM patching date, you can match it with the modification date of the file E:\apps\OracleEPM\Middleware\oracle_common\modules\oracle.webservices_11.1.1\oracle.webservices.standalone.client.jar to confirm the Patch 13866584 applied.

Fusion Middleware (FMW) - WebLogic  Patch 16810628 can be checked using a command as shown below:

Now coming to the third prerequisite patch Fusion Middleware (FMW) Patch 30378046, its README document suggests:

Ensure the patch for bug 16964825 has already been applied before you proceed with the current patch.

If you remember, Oracle JDeveloper ADF Patch 16964825 is already included in HFM 11.1.2.4.100 installation itself. When upgrading Hyperion environment from 11.1.2.4 to 11.1.2.4.100 one of the prerequisites is to “Install the ADF patch 21240419”. The prerequisite to 21240419 is “patch 16964825(mandatory)”. 

So it must be there on my HFM server. Let's verify with opatch lsinventory in E:\apps\OracleEPM\Middleware\oracle_common that patch 16964825 is already present.

Now its turn to install Fusion Middleware (FMW) Patch 30378046.

When I ran the OPatch command to install Fusion Middleware (FMW) Patch 30378046, it was throwing the following issue:

Actually, Patch 30378046 was conflicting with the patch 20326778.  

In such cases Oracle recommendation is:

Obtain a copy of each patch and compare the 'inventory' file or the 'readme' file. These files contain a list of bugs fixed by each patch. Compare the bugs listed. Make sure that the patch being applied has all the bugs mentioned that are fixed in the existing patch.

If all the patches listed in the existing patch are included in the patch being applied you can safely continue the patch.

In some cases, the patch being applied may not contain all the bug fixes that are included with the existing patch. If you do not require the missing patches you can safely continue the new patch installation.

If you do require the missing patches then you should stop the patch installation and log a service request with Oracle Support requesting a "Merge Label Request, MLR" of the existing patch and the patch being applied.

Following the recommendation, I compared the two patches and noticed that following three bugs fixed in EXISTING Patch 20326778, are not included in the NEW Patch 30378046:

• 17890124: BLR BACKPORT 17776065 TO 11.1.1.7.1: FF25, ALL RTL TEXT IS MISALIGNED FOR DVT HT
• 18601128: BLR BACKPORT OF BUG 18233049 ON TOP OF 11.1.1.7.1 (BLR #3572140)
• 18954287: BLR BACKPORT OF BUG 18820382 ON TOP OF 11.1.1.7.1 (BLR #3789756)

But, if you dig deep you will notice that these are BACKPORT Bugs and their sources (17776065, 18233049, 18820382) are included in the new Patch 30378046. So it's safe to move with installing over Patch 20326778. It was also making sense as Patch 30378046 is the latest patch available for JDeveloper ADF and the other Patch 20326778 was outdated by 5 years at least.

I did the same as shown below:

As you can see OPatch itself will first uninstall the Patch 20326778 and then install the new Patch 30378046. 

For the fourth prerequisite ADF Patch 25113405, it was self-explanatory:

If you are applying this HFM PSU 11.1.2.4.209 Patch 29343616 to version 11.1.2.4.203 or an earlier version, this Financial Management patch requires Oracle Application Development Framework (ADF)  Patch 25113405.

Finally, its time to install the main Hyperion Financial Management PSU 11.1.2.4.209 Patch 29343616.  As all the prerequisites were clear, It got installed successfully without any issue:

After applying the HFM Patch 29343616, don't forget to execute the sql commands as per your database type mentioned in the README document of HFM PSU 11.1.2.4.209 Patch 29343616. 

As I have HFM version 11.1.2.4.204 and Oracle Database 18c, so I ran the below commands by logging to HFM schema (Refer the README document of Patch 29343616 for more details):

1- Executed the following sql command to upgrade the schema against the Financial Management database:

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\Oracle_from_11.1.2.4.208_to_11.1.2.4.209_SchemaUpgrade.sql

2- For the 'Importing Applications' feature, executed the following sql commands:

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\Oracle_Create_ImportApp.sql

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\Oracle_Create_InsightDetails.sql

Make sure these sql commands are executed without any error message.

Post Hyperion services restart, all users should clear cached files from their browsers before login to Workspace.

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

How to Extract Oracle DRM (Data Relationship Management) Users List and Access Attributes

Tracking changes regarding application users and their accesses in the Production environment is a very important part of your Hyperion application auditing. DRM is one such application where the Business team performs regular user auditing because your critical Hyperion metadata is managed here.

Here we are going to explore some of the very handy and important Oracle database queries using which you can check your Data Relationship Management (DRM) users, their assigned roles, roles permissions, Property Categories access, Node Access Groups Access, Object Access Groups Access etc.

These queries are also helpful for your DRM Licensing based on the DRM user base.

Using these queries, you can automate Data Relationship Management (DRM) users tracking process in your Production environment. For automation, you just need to put your chosen query in a .sql file, add SQL output formatting options as per your need, call that .sql file from a batch script and send the formatted output to the recipients using an email script.

In DRM_DB schema/repository where your Oracle Data Relationship Management (DRM) application is configured, there are many tables that contain information regarding your DRM application users, their roles, etc. but in a distributed manner, i.e. there is no single table, which consists of everything. Therefore, you need to join multiple tables to fetch the required users' information.

What will you learn in this ebook?:

• 30+ very handy and important DRM database queries with outputs and actual screenshots that you can directly use not only to check your Oracle DRM Users and their access tracking but also to automate them. 
• Various DRM database tables, which consist of information regarding DRM users and their various access levels.
• DRM Security concept and different levels of  DRM security options under the following headings:

Queries to extract DRM Users and their assigned DRM Roles:  

• Query to list out all the DRM users including those users who do not have any DRM ROLE assigned.
• Query to list out all your DRM business users (Active Directory users) who are externally authenticated through Hyperion Shared Services.
• Query to list out all the DRM Roles and their full list of permissions.
• Query to list out all the DRM users with all of their assigned roles (concatenated in one row), who have been assigned at least one DRM role. 
• Query to list out only those DRM users with all their assigned roles who are externally authenticated through Hyperion Shared Services.
• Query to list out all those DRM users who have been assigned 'Application Administrator' role in your DRM application.
• Query to list out all those DRM users who have been assigned 'Application Administrator' role in your DRM application and who are externally authenticated through Hyperion Shared Services.
• Query to list out all those DRM users who have been assigned EITHER 'Interactive User' OR 'Application Administrator' role in your DRM application. 
• Query to list out all those DRM users who have been assigned EITHER 'Interactive User' OR 'Application Administrator' role in your DRM application and who are externally authenticated through Hyperion Shared Services. 
• Query to list out all those DRM users who have been assigned BOTH 'Access Manager' and 'Application Administrator' role in your DRM application. 
• Query to list out all those DRM users who have been assigned BOTH 'Access Manager' and 'Application Administrator' role in your DRM application and who are externally authenticated through Hyperion Shared Services. 

Queries to extract DRM Users and their assigned Property Categories Access:

• Query to list out all the DRM Users with their assigned Property Category names and Access Levels (Read/Edit).
• Query to list out only those DRM users with their assigned Property Category names who have READ level access.
• Query to list out only those DRM users with their assigned Property Category names who have EDIT level access.
• Query to list out only those DRM users with their assigned Property Category names and Access Levels (Read/Edit) who are externally authenticated through Hyperion Shared Services.
• Query to list out user detail with his/her assigned Property Category names and Access Levels (Read/Edit) for a particular DRM user. 

Queries to extract DRM Users and their assigned Node Access Groups:

• Query to list out all the DRM Users with their assigned Node Access Groups (in one row). 
• Query to list out only those DRM Users with their assigned Node Access Groups who are externally authenticated through Hyperion Shared Services.

Queries to extract DRM Users and their assigned Object Access Groups:

• Query to list out all the DRM Users with their assigned Object Access Group and Object access level.
• Query to list out only those DRM Users with their assigned Object Access Group and Object access level who are externally authenticated through Hyperion Shared Services.

To purchase the ebook, choose any of the below given payment options:

Option-1 : Buy on PayPal ( Price = $4 ):

Option-2 : Buy in INR on Instamojo ( Price = INR 295 ):

Option-3 : To pay in INR via UPI (GooglePay or Paytm), please use below given links. Post payment, drop an email to skp.1world@gmail.com with payment screenshot. After verifying the payment, we will share the pdf ebook with you on the same email.

Paytm Payment Link --> Paytm (INR 295)

# For any query/issue regarding the ebook purchase, please feel free to drop an email at skp.1world@gmail.com. Thank you!

Read More

Essbase Client MSI (EssbaseClient.exe) installation and version check

Hi Friends,

Topic: How to install Essbase client MSI and check the installed Essbase client version 

Essbase Client MSI (EssbaseClient.exe) is required to be installed when you need to use MaxL or ESSCMD interpreters on the Windows server. Most importantly Essbase Client MSI should be installed on HPCM and FDMEE Windows servers as you may face connection issues between HPCM-Essbase and FDMEE-Essbase, if Essbase Client is not installed on these servers.

When you first time install and configure Hyperion Essbase, EPM System Installer installs both 32-bit and 64-bit Essbase Client on a machine with a 64-bit operating system. EPM System Installer installs only 32-bit Essbase Client on a machine with a 32-bit operating system.

Once you have an already running Hyperion Essbase setup, then to apply the latest patch set update (PSU) released for Essbase, for a 32-bit Windows operating system, download and install the 32-bit version of Essbase Client and for a 64-bit Windows operating system, download and install the 64-bit version of Essbase Client.

In this post, we will see how you can install and uninstall Essbase client MSI on the Windows server and check its version post-installation.

Important Note:

1. The demonstrating Hyperion environment has the 'Windows Server 2012 R2 Standard' 64-bit operating system. 
2. This post has been written and associated activities have been demonstrated on Oracle Hyperion Essbase Client MSI  11.1.2.4.021 (Patch 26835581). The latest version of Essbase Client MSI (released till today 26th June 2020) is 11.1.2.4.039 that you can read more about here https://blogs.oracle.com/proactivesupportepm/psu_essbase_11124039 
3. Caution: The Essbase client version should match the version of the Essbase Server. Oracle recommends using the same version of all Essbase portfolio products (Essbase, Essbase Administration Services, Hyperion Provider Services, and Essbase Studio) and components (server, client, runtime client, API, and JAPI). When only some Essbase portfolio products are included in a patch release, the last released versions of the products that are not included in the patch are supported. For example, Essbase Administration Services 11.1.2.4.037, Provider Services 11.1.2.4.039, and Essbase Studio 11.1.2.4.031 are supported for use with Essbase 11.1.2.4.039.
4. Refer to the Readme files prior to proceeding with these PSU implementations for important information that includes the supported paths per component, along with a full list of the defects fixed, additional support information, prerequisites, details for applying patch, and troubleshooting FAQ's. It is important to ensure that the requirements and support paths to this patch are met as outlined within the Readme file. 
5. Before installing the new version of Essbase Client, you first need to uninstall the already installed version of Essbase Client on your Windows server.

How to uninstall Essbase Client MSI (EssbaseClient.exe):

To uninstall the already installed Essbase Client on Windows server:

1.    In Control Panel of the Windows server, navigate to Programs and select 'Uninstall a program'.
2.    Select 'Oracle(R) Hyperion Essbase Client', right-click, and then select Uninstall.

Once you have successfully uninstalled the already running Essbase Client, you can now install your new Essbase Client version.

How to install the new Essbase Client MSI (EssbaseClient.exe):

Note: You can download the required version of Essbase Client from the Oracle Support portal. For example, here we have downloaded the Patch 26835581 which consists of EssbaseClient.exe of version 11.1.2.4.021. After download, extract the zip file to your desired folder. In this case, we have extracted it to OPatch folder E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch.

1. In the extracted folder, right-click on EssbaseClient.exe file and select 'Run as administrator'.
   
2. Select your choice of language.
   
3. Pick a destination folder (make a note of the destination directory). Mostly it's installed under the path E:\apps\OracleEPM\Middleware\EPMSystem11R1\products\Essbase\EssbaseClient\
   
4. You can de-select Essbase Client C API and Essbase Client Samples, if you want to use Essbase Client only for Hyperion applications connections within a Hyperion environment.  #NOTE: Essbase Client C API is used to connect to Essbase server from non-Hyperion application setups like Spotfire server, MicroStrategy products, etc. The API is an interface between your custom client program and Essbase, and manages the transfer of data between client and server. Your program makes calls to functions within the API, and data is returned from the Essbase servers you connect to. You can also run custom programs on the server machine, using the same API functions as on the client. You don't have to be concerned about where the Essbase Server computer is located on the network when writing a custom API program. Locating the server and transferring data is handled by the API.
   
5. Click Install.
   
6. Your installation is completed.
   
7. In the Installed Programs list of the Control Panel, you can find a listing for Oracle(R) Hyperion Essbase Client.
   

Note: After applying the Essbase Client patch, you might need to set the ESSLANG environment variable to your local language, if not already done. Also set the library path and PATH environment variables to point to the Essbase client bin directory, if not already done.

How to verify the installed version of Essbase Client on the Windows server:

Method-1:

1. Goto the path E:\apps\OracleEPM\Middleware\EPMSystem11R1\products\Essbase\EssbaseClient\bin
2. Right-click on ESSCMD.exe file and select Properties.
3. You will notice your Essbase Client Product version in the properties tab.

Method-2:

1. Open a command prompt and navigate to the path E:\apps\OracleEPM\Middleware\EPMSystem11R1\products\Essbase\EssbaseClient\bin.
2. Enter the command startMaxl.cmd. 
3. In the output, you will notice your Essbase Client version i.e. 11.1.2.4.021

That's all for this post.

I hope this article has helped you.
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!

Read More

Automation: Batch Script to delete Hyperion backup folders older than N months/days

Hi Friends,

Topic: How to delete Oracle Hyperion backup folders older than N months/days

Maintaining a backup of various applications is an important part of any Oracle Hyperion Production system. In most of the cases, we have a network drive connected and accessible from all the servers of a Hyperion Production environment where we save the backup of Hyperion application data, metadata, artifacts, LCM exports, etc. either manually or through automated scripts like Lifecycle Management (LCM) automation, FDMEE data load jobs, Maxl scripts, etc. on a daily or weekly basis.

Over the course of time, your network drive is filled with past many months’ backup folders consuming a lot of space and thus causing a requirement for network drive housekeeping. In most of the cases, maintaining last 3-6 months application backup folders actually serve the purpose.   

We too faced the similar case where we had many Hyperion application LCM backup folders over a network shared drive starting from the year 2017, which were consuming a huge amount of space raising critical space alerts. To housekeep the network drive when we tried to delete some of these LCM backup folders manually, we encountered the notorious ‘Source Path Too Long’ error to make matters worse.

Therefore, to fix this issue permanently, we have created a batch script called “Network_Housekeeper” which works perfectly fine and is based on the following logic:

1- We have a LCM automated scheduled job which adds two huge sized LCM backup folders every day to the network drive path: \\NetworkServerAddress\HyperionBackupFolder\AutomatedLCMExport with the following folder naming convention:

HYP_PROD_YYYYMMDDhhmm

Note: To learn how you can add current date and time in your backup folders' name, kindly refer this link: Batch script to add today's date and time in file/folder name

2- The script maps the network drive path as a local drive on the server from where we trigger this script.

3- It identifies the date 4 months (120 days) before today’s date.

4- It lists out all the LCM backup folders with the name HYP_PROD_* having the oldest date folder first and compares the date of each folder with the date 4 months before the current date.

5- If the folder date is less than the date 4 months before the current date, it passes that folder name to ROBOCOPY purge command section as an input to delete that folder.

Note: ROBOCOPY purge command is used to troubleshoot ‘Source Path Too Long’ error while deleting the folder.

6- This loop continues until all the folders older than 4 months (120 days) are deleted.

7- The script saves the names of the deleted folders in a log file for your record: D:\Data\Scripts\NETWORK_HOUSEKEEPER\Deleted_Folder.log 

Network_Housekeeper Batch Script:

@echo off

REM Connect to the network drive path

pushd \\NetworkServerAddress\HyperionBackupFolder\AutomatedLCMExport

REM Set the number of days to retain the folders for. You need to change the day=-120 to the relevant number of days you want. 

set day=-120

echo >"%temp%\%~n0.vbs" s=DateAdd("d",%day%,now) : d=weekday(s)

echo>>"%temp%\%~n0.vbs" WScript.Echo year(s)^& right(100+month(s),2)^& right(100+day(s),2)

for /f %%a in ('cscript /nologo "%temp%\%~n0.vbs"') do set "result=%%a"

del "%temp%\*%~n0.vbs"

set "yyyy=%result:~0,4%"

set "mm=%result:~4,2%"

set "dd=%result:~6,2%"

set "final=%yyyy%%mm%%dd%"

setlocal enabledelayedexpansion

REM List out all the folders with name HYP_PROD_* having the oldest folder first and then extract the date from each folder name 

for /f "delims=" %%a in ('dir "HYP_PROD_*" /a:d /o:d /b') do (

set "folder=%%a"

set folddate=!folder:~9,8!

REM If folder date is less than the date 4 months before the current date, trigger the 'work' section of the script

if !folddate! LSS !final! call :work

)

goto :EOF

REM Delete the folder using ROBOCOPY command route 

:work

echo !folder! >> D:\Data\Scripts\NETWORK_HOUSEKEEPER\Deleted_Folder.log

rmdir emptyfolder

mkdir emptyfolder

robocopy emptyfolder "!folder!" /purge                  

rmdir !folder!

rmdir emptyfolder

You can schedule this batch script through Windows Task Scheduler to run at any time of the day. I have tested this script on my system and it takes 80 seconds to delete a 2 GB LCM backup folder. That is quite ok! 

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

LCM Import timeout for HPCM artifacts:EPMLCM-13000:Service currently not available

Hi Friends,

Topic: How to fix LCM timeout issue while importing HPCM (Hyperion Profitability and Cost Management) artifacts

In this post, we will see LCM (Lifecycle Management) timeout error (EPMLCM-13000: Service currently not available) and how to fix it while we import HPCM (Hyperion Profitability and Cost Management) artifacts into any Oracle Hyperion environment.

Important Note: 

• This post has been written and associated activities have been demonstrated on Oracle HPCM (Hyperion Profitability and Cost Management) version 11.1.2.4.127.
• Different paths and folders' names mentioned in this blog may slightly vary for different Oracle Hyperion setups but at large it should be the same. 
• It's very important to test any workaround/fix first in a non-Prod environment, and move/apply it to PROD only when you are fully sure about its workability. 
• Never forget to take a complete backup of EVERYTHING before deleting or changing anything.

Issue:

In our Production environment, when we were trying to import HPCM application models, POVs etc. components using LCM, it was getting aborted after 5 minutes throwing following error in migration details page:

EPMLCM-13000: Service currently not available.  

We tried to run the LCM import two more times, but every time it was getting failed after 5 mins.

Analysis:

When we checked SharedService_LCM.log (MIDDLEWARE_HOME/user_projects/domains/EPMSystem/servers/FoundationServices0/logs), we found following error message recorded over there:

2020-05-16T09:04:44.587-06:00] [FoundationServices2] [NOTIFICATION] [EPMLCM-13000] [oracle.EPMLCM] [tid: 815] [userId: <anonymous>] [ecid: 00j15bZ53w2EoItDwfV4CW0003po001Y3p,0:1:4:3:4:3:3:3:3:4:3:3:3:3:4:3:4:4:4:3:3:3:3:3:4:3:4:3:3:3:3:3:3:3:3:3:3:3:3:3:3:4:4:4:3:4:3:3:3:3:3:4:3:3:3:3:3:3:4:3:3:3:4:3:4:4:4:3:3:4:4:4:3:4:3:3:3:4:3:3:3:4:3:3:3:3:3:3:3:4:4:4:3:3:3:3:3:4:3:3:4:4:4:4:3:3:4:4:3:4:4:3:4:4:3:4:4:3:3:3:4:3:3:4:3:4:4:3:4:4:4:4:4:4:3:4:4:3:3:3:3:3:3:3:3:4:4:3:4:3:3:3:3:3:3:3:3:3:3:4:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:4:4:3:3:3:4:3:3:3:3:4:3:3:3:3:3:3:3:4:4:4:4:4:4:4:4:4:4:3:4:4:4:4:3:4:3:3:3:3:4:4:3:4:3:4:3:3:3:4:3:4:3:3:3:3:3:3:3:3:4:3:1:1:1] [APP: SHAREDSERVICES#11.1.2.0] Service currently not available.

[2020-05-16T09:04:44.587-06:00] [FoundationServices2] [ERROR] [EPMLCM-30052] [oracle.EPMLCM] [tid: 815] [userId: <anonymous>] [ecid: 00j15bZ53w2EoItDwfV4CW0003po001Y3p,0:1:4:3:4:3:3:3:3:4:3:3:3:3:4:3:4:4:4:3:3:3:3:3:4:3:4:3:3:3:3:3:3:3:3:3:3:3:3:3:3:4:4:4:3:4:3:3:3:3:3:4:3:3:3:3:3:3:4:3:3:3:4:3:4:4:4:3:3:4:4:4:3:4:3:3:3:4:3:3:3:4:3:3:3:3:3:3:3:4:4:4:3:3:3:3:3:4:3:3:4:4:4:4:3:3:4:4:3:4:4:3:4:4:3:4:4:3:3:3:4:3:3:4:3:4:4:3:4:4:4:4:4:4:3:4:4:3:3:3:3:3:3:3:3:4:4:3:4:3:3:3:3:3:3:3:3:3:3:4:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:3:4:4:3:3:3:4:3:3:3:3:4:3:3:3:3:3:3:3:4:4:4:4:4:4:4:4:4:4:3:4:4:4:4:3:4:3:3:3:3:4:4:3:4:3:4:3:3:3:4:3:4:3:3:3:3:3:3:3:3:4:3:1:1:1] [APP: SHAREDSERVICES#11.1.2.0] [SRC_CLASS: ?] [SRC_METHOD: ?:?] Failed to connect to "HPCMPRD" while performing import for application. Received status code - "503" with error message - "Weblogic Bridge Message Failure of server APACHE bridge: No backend server available for connection: timed out after 10 seconds or idempotent set to OFF or method not idempotent. ". Possible cause of error no backend server available for connection.

We first tried to restart Hyperion Foundation and HPCM related services but no luck. As per the SharedService_LCM.log, there seems to be an issue while connecting to the HPCM web application server during the LCM import process. As we noticed the LCM import process is getting aborted every time exactly after five mins. So we thought it to be a time out issue probably at the OHS layer. 

Solution:

As LCM import was getting aborted after 5 mins so we thought to check and modify (increase or add) the timeout setting for profitability application in the OHS (Oracle HTTP Server) configuration file i.e. mod_wl_ohs.conf. The Oracle HTTP Web server can be configured to time out if a job takes longer than a predefined period. 

Note: If your OHS server has not been configured to a specific shared location and has been kept as default then your mod_wl_ohs.conf file will be stored at <EPM_ORACLE_INSTANCE\httpConfig\ohs\config\OHS\ohs_component\mod_wl_ohs.conf. But If your OHS server is load balanced and has been configured to a shared location and you don't know that location then you can run a registry report and search for that shared location. 

How to modify LCM timeout default settings for HPCM (Profitability) application in Oracle HTTP Server server configuration file (mod_wl_ohs.conf ):

If you are using Hyperion LCM to import large HPCM models or POVs, the LCM import process may take longer to process than the time specified in the default timeout settings on the Oracle WebLogic Server. To fix this issue, you need to increase or add a timeout setting for Profitability application in OHS configuration file i.e. mod_wl_ohs.conf. 

Steps:

1- Stop your Oracle Hyperion services (Foundation and OHS service must be stopped).

2- On your OHS Web server, navigate to the path:

%Middleware_HOME%\user_projects\epmsystem1\httpConfig\ohs\config\OHS\ohs_component\mod_wl_ohs.conf.

3- First take a backup of mod_wl_ohs.conf file. Then open it and locate the below section: 

<LocationMatch ^/profitability>

4- Add the following lines within the <LocationMatch ^/profitability> section (modify or add the WLIOTimeoutSecs property with a value that will encompass the duration of typical HPCM migration tasks):

WLIOTimeoutSecs 3000

Idempotent OFF

5- Now on OHS web server, navigate to the path:

%Middleware_HOME%\user_projects\epmsystem1\httpConfig\ohs\config\OHS\ohs_component\htppd.conf

6- First take a backup of htppd.conf file. Then open it and locate the below section:

# Timeout: The number of seconds before receives and sends time out. 

7- Add or modify the Timeout value to 3000, as shown below:

# Timeout: The number of seconds before receives and sends time out. 

Timeout 3000

Note: The server timeout shown above is a suggested limit, and may be modified to suit the specific timeout settings for your application server.

8- Start your Oracle Hyperion services and try to re-import your HPCM application artifacts. It should work fine now without any timeout issue.

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

Shell Command/Script to extract Oracle EPM/Hyperion already applied patches in Linux/Unix server

Hi Friends,

Topic: How to extract Oracle Hyperion patches currently applied in your Hyperion environment using shell/bash script

In this post, we will see how you can extract all the Oracle Hyperion patches currently applied in your Hyperion environment Linux servers. I am sure you would have seen Oracle Hyperion Essbase application installed and configured on Linux/Unix server in many Hyperion setups. 

When you plan to apply any newly released Oracle patches to upgrade your Hyperion applications patch level, you may need to first know the currently applied Oracle patches for that particular Hyperion application. 

For the Windows Servers, to know how to extract Oracle Hyperion patches currently applied in your Hyperion environment using a batch script, you may like to read this post: Batch script to extract Oracle Hyperion patches currently applied

Note: 

• This post has been written and associated activities have been demonstrated on Oracle Hyperion version 11.1.2.4.
• The demonstrating Hyperion Essbase application server has the 'Red Hat Enterprise Linux Server release 6.10' operating system.

Concept:

As you know, we all use ./opatch lsinventory command in order to see details of all the patches that are currently installed on a Hyperion Linux/Unix server.

The lsinventory command reports what has been installed on the system for a particular Oracle home directory, or for all installations. 

The following syntax is used for this command (on Linux/Unix machine):

./opatch lsinventory -oh /apps/OracleEPM/Middleware/EPMSystem11R1

where -oh option is your Oracle Home location. 

The result of ./opatch lsinventory command contains a lot of data like various versions, paths, bugs fixed, overlaying patch details, etc as shown in the below image. So in order to list out only the Oracle patch numbers and when they were applied you need to go through all the details generated as a result of ./opatch lsinventory command then extract the required details manually.

The below-mentioned shell/bash script does all these things for you automatically by extracting out only the Oracle patch numbers with the date when they were applied. It saves the output into a log file for your record along with saving your manual effort. 

On Command prompt:

If you want to see all the applied patch list directly on your EPM/Hyperion Linux/Unix server's command prompt, run the below commands:

To see all installed patches:

----------------------------------------

cd /apps/oracle/epm/Middleware/EPMSystem11R1/OPatch

For EPMSystem11R1 patches:

./opatch lsinventory -oh /apps/oracle/epm/Middleware/EPMSystem11R1 -jdk /apps/oracle/epm/Middleware/jdk160_35 -invPtrLoc /apps/oracle/epm/Middleware/EPMSystem11R1/oraInst.loc | grep -i applied

For oracle_common patches:

./opatch lsinventory -oh /apps/oracle/epm/Middleware/oracle_common -jdk /apps/oracle/epm/Middleware/jdk160_35 -invPtrLoc /apps/oracle/epm/Middleware/oracle_common/oraInst.loc | grep -i applied

To see a particular installed patch:

---------------------------------------------

cd /apps/oracle/epm/Middleware/EPMSystem11R1/OPatch

For EPMSystem11R1 patches:

./opatch lsinventory -oh /apps/oracle/epm/Middleware/EPMSystem11R1 -jdk /apps/oracle/epm/Middleware/jdk160_35 -invPtrLoc /apps/oracle/epm/Middleware/EPMSystem11R1/oraInst.loc | grep -i "applied" | grep -i "28314691"

For oracle_common patches:

./opatch lsinventory -oh /apps/oracle/epm/Middleware/oracle_common -jdk /apps/oracle/epm/Middleware/jdk160_35 -invPtrLoc /apps/oracle/epm/Middleware/oracle_common/oraInst.loc | grep -i "applied" | grep -i "18514458"

Shell/Bash Script:

Create a folder (/apps/Admin) in the Hyperion Linux/Unix server where you want to extract the installed Oracle patch list and then create a shell/bash script (/apps/Admin/test.sh) with the below codes under that folder.

#!/bin/bash

cd /apps/OracleEPM/Middleware/EPMSystem11R1/OPatch

./opatch lsinventory -oh /apps/OracleEPM/Middleware/EPMSystem11R1 > /apps/Admin/Patch_History.txt

cd /apps/Admin

truncate -s 0 /apps/Admin/Installed_Patches_$HOSTNAME.txt

grep -w "Interim patches" /apps/Admin/Patch_History.txt >>/apps/Admin/Installed_Patches_$HOSTNAME.txt

grep -w "applied" /apps/Admin/Patch_History.txt >>/apps/Admin/Installed_Patches_$HOSTNAME.txt

When you run this test.sh script via bash command line, it will extract out the following things for you:

1. Total number of Oracle patches
2. Patch numbers of all the currently applied Oracle Hyperion patches
3. Date of applying these patches

It will also save the output in a log file with your Linux/Unix server HOSTNAME appended in the log file name.

Below is the directory structure and the output format of the above-given script:

You can run this script across all the Linux/Unix servers of your Hyperion environment to extract the list of applied Oracle Hyperion patches. 

In order to see which Oracle Hyperion applications your extracted patch numbers belong to, simply do the following:  

• Go to https://support.oracle.com.
• Select the Patches & Updates tab.
• In Patch Search, enter the patch number. 
• Make sure you are searching for a Patch Name or Number and select Search.

The corresponding Hyperion application name with patch details will be displayed. 

That's all for this post.

I hope this article has helped you. 
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!

Read More