Oracle DRM (Data Relationship Management) Application Logs

To access Oracle DRM (Data Relationship Management) application related activities log, there are two methods:

1. Auditing Transactions, Jobs, and Requests through DRM Web Client
2. Extracting DRM (Data Relationship Management) application log from backend DRM application database schema

Auditing Transactions, Jobs, and Requests through DRM (Data Relationship Management) Web Client

Oracle Data Relationship Management records a history of different activities performed within an application for audit purposes. 

• The transaction history logs all operations performed including changes to version data, application metadata, and user security. 
• The job history tracks the completion of long running processes such as imports, blenders, and exports. 
• The request history maintains a record of change requests submitted from external applications.
  

For further details you can read DRM (Oracle Data Relationship Management) Suite User's Guide.

You can query and view transactions, requests, and job history using the Audit task wizard in DRM Web Client navigating through Home --> Audit as shown below:

The audit wizard is used to define the parameters and criteria for the query. The wizard content is organized across Source, Filters, and Columns tabs to guide input. Although the wizard tabs are ordered to gather input in a sequence, you can navigate between any tabs.

Extracting DRM (Data Relationship Management) application log from backend DRM application database schema

Extracting DRM Transaction History:

In DRM application database schema, there is a table name RM_TRANSACTION_HISTORY that contains a row for each transactional update made to your DRM repository.

Below is all the columns listed what RM_TRANSACTION_HISTORY table consists of: 

Name                                       Null                     Type            

-------------------------- -------- --------                    ------- 

I_TRANSACTION_ID           NOT NULL         NUMBER(38)      

I_DETAIL_ID                         NOT NULL         NUMBER(38)      

D_TIMESTAMP                                                 TIMESTAMP(6)    

I_SESSION_ID                                                   NUMBER(38)      

C_USER_NAME                    NOT NULL        NVARCHAR2(100)  

C_ACTION                                                         NVARCHAR2(40)   

C_ACTION_LEVEL                                          NVARCHAR2(40)   

C_VERSION_ABBREV                                    NVARCHAR2(255)  

I_VERSION_ID                                                  NUMBER(38)      

C_HIERARCHY_ABBREV                              NVARCHAR2(255)  

I_HIERARCHY_ID                                           NUMBER(38)      

C_NODE_ABBREV                                           NVARCHAR2(255)  

I_NODE_ID                                                       NUMBER(38)      

C_NODE_DESCR                                             NVARCHAR2(255)  

C_PROPERTY_ABBREV                                  NVARCHAR2(255)  

I_PROPERTY_ID                                               NUMBER(38)      

B_LEAF                                     NOT NULL     NUMBER(38)      

C_ACTION_DESCRIPTION                             NVARCHAR2(1024) 

X_FROM_VALUE                                                NCLOB           

C_FROM_ORIGIN                                               NVARCHAR2(1024) 

X_TO_VALUE                                                      NCLOB           

C_TO_ORIGIN                                                     NVARCHAR2(1024) 

I_ORIGINATING_ID                                             NUMBER(38)      

I_ORIGINATING_DETAIL_ID                             NUMBER(38)      

C_OBJECT_NAME                                                NVARCHAR2(255)  

I_DOMAIN_ID                                                       NUMBER(38)      

C_DOMAIN_ABBREV                                          NVARCHAR2(255)  

I_WORKFLOW_REQUEST_ID                              NUMBER(38)      

I_WORKFLOW_REQUEST_ITEM_ID                  NUMBER(38)      

C_NODE_TYPE                                                       NVARCHAR2(255)  

As we see, RM_TRANSACTION_HISTORY columns include: Action (e.g. Add Node, Insert Node, Delete Node, and ChangeProp), Node, Hierarchy, Version and Property Name values as well as Node Description, Leaf flag, to and from Property values for ChangeProp operations.

You can query this RM_TRANSACTION_HISTORY table to extract DRM application related logs for a particular time period.

For example, if you want to extract DRM Application log from 13-Mar-2021 to 19-Mar-2021, run following query:

select * from RM_Transaction_History where D_TIMESTAMP between to_date('2021-03-13 00:00:01', 'YYYY-MM-DD HH24:MI:SS') and to_date('2021-03-19 23:59:59', 'YYYY-MM-DD HH24:MI:SS') order by D_TIMESTAMP;

Output:

Note: Its preferable to use SQL Developer to extract DRM application logs from backend schema. Make sure in SQL Developer, you have set the date and time format same as mentioned in above query.

Extracting DRM Job History:

DRM Job history is stored in a table name RM_JOB_HISTORY that consists of following columns:

You can query this table RM_JOB_HISTORY to extract all job run details.

For example, if you want to extract DRM Job History log from 13-Mar-2021 to 19-Mar-2021, run following query:

select * from RM_Job_History where D_TIME_STARTED between to_date('2021-03-13 00:00:01', 'YYYY-MM-DD HH24:MI:SS') and to_date('2021-03-19 23:59:59', 'YYYY-MM-DD HH24:MI:SS') order by D_TIME_STARTED;

Output:

Extracting DRM Request History:

DRM Request history is stored in a table name RM_REQUEST that consists of following columns:

You can query this table RM_REQUEST to extract all change requests submitted from external applications.

select * from RM_REQUEST;

That's all for this post.
I hope this article has helped you.
Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!

Read More

EPM 11.1.2.4: Microsoft Edge and Google Chrome browsers support patching : Part-2

You may like to read Part-1 first to understand the background of Microsoft Edge and Google Chrome browsers support patching in Hyperion/EPM 11.1.2.4 setup. 

In this post, I will tell how it worked for us. We have recently applied these browsers (Microsoft Edge and Google Chrome) compatibility patches in our EPM/Hyperion 11.1.2.4 environment. 

As discussed in last post, below is what we need to apply in order to make your EPM/Hyperion 11.1.2.4 environment working on Microsoft Edge and Google Chrome browsers:

Important Notes:

• Above list provides the minimum patch version for each Hyperion application you need to install, If you want, you can go for higher version too. 
• I assume your EPM/Hyperion 11.1.2.4 environment is up-to-date in terms of patches and you have all the prerequisite patches already applied.
• Its recommended to have EPM/Hyperion application schemas backup and if possible, backup of the existing installation/configuration directory to handle unforeseen critical issues.
• You should apply these patches in below given order. 
• I have all these Hyperion applications installed and configured on Windows server 2012 R2, therefore below given steps are applicable for Windows machines. For other operating systems, you can refer 'README' document of the respective patch number.
• Stop all the Hyperion services and WebLogic Admin server of your environment before proceeding further.

Lets start the patching now. 

Hyperion Workspace 11.1.2.4.825 (Patch 31124100)

----------------------------------------------------------------------

This patch needs to be applied on all your EPM Foundation/Workspace servers of the Hyperion environment. 

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch 

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\31124100 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

Hyperion Shared Services 11.1.2.4.100 (Patch 31319089)

-------------------------------------------------------------------------

This patch needs to be applied on all your EPM Foundation/Workspace servers of the Hyperion environment. 

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch 

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\31319089 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

Oracle JDeveloper / Application Development Framework 11.1.1.7.1 (Patch 31246831)

-----------------------------------------------------------------------------------------------------------------

This patch needs to be applied on all your EPM Foundation/Workspace servers of the Hyperion environment. 

After downloading the patch zip file, extract it to C:\Patch_Top. There should now be a 31246831 directory in the C:\Patch_Top directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\oracle_common\OPatch\ 

opatch.bat apply C:\Patch_Top\31246831\oui -oh E:\apps\OracleEPM\Middleware\oracle_common -jre E:\apps\OracleEPM\Middleware\jdk160_35

Hyperion Planning, Hyperion Capital Asset Planning, Hyperion Workforce Planning, Hyperion Project Financial Planning 11.1.2.4.009+ (Patch 29889455)

---------------------------------------------------------------------------

This patch needs to be applied on all your Hyperion Planning application servers and Web servers of the environment. 

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch 

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\29889455 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

Delete the tmp directory for the Planning server in: 

<MIDDLEWARE_HOME>/user_projects/domains/EPMSystem/servers/<server> 

where <server> is the Planning server (for example, Planning0).

The full path for tmp is: <MIDDLEWARE_HOME>/user_projects/domains/EPMSystem/servers/<server>/tmp.

Hyperion Calculation Manager 11.1.2.4.014  (Patch 28557058)

-------------------------------------------------------------------------------

This patch needs to be applied on all your Calculation Manager servers and Web servers of the Hyperion environment. 

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch 

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\28557058 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

On the server where Oracle Hyperion Calculation Manager is installed, delete the cached files in the following directories:

<Middleware_Home>/user_projects/domains/<DOMAIN_NAME>/servers/<CalcMgr0>/tmp 

<Middleware_Home>/user_projects/domains/<DOMAIN_NAME>/servers/<CalcMgr0>/cache

Hyperion Financial Reporting 11.1.2.4.712  (Patch 30670918)

----------------------------------------------------------------------------------

This patch needs to be applied on all your Financial Reporting servers, Web servers and also on HFM application server (if you use HFR for reporting) of the Hyperion environment. 

To apply this patch first Uninstall the Financial Reporting Studio by going to Control Panel, Add or Remove Programs, and then uninstall Financial Reporting Studio. 

On the Financial Reporting server machines, download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\30670918 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

Delete the Financial Reporting precompiled jsp files located at:

<ORACLE_HOME>\Middleware\user_projects\domains\EPMSystem\precompiled\hr

Use the EPM System configurator to redeploy the Financial Reporting Web Application.  If RA Framework, OHS, or IIS are installed on a separate host, it must be patched also.

In order to run the new Oracle Hyperion Financial Reporting Web Studio, you must enable proxying. I am sure it would have already been enabled in your environment too. If not, kindly refer 'README' document of the patch.

At the end, when you start your EPM services, remember to connect to the EPM Workspace machine on patched Financial Reporting server machine. Download the Financial Reporting Studio installer from the EPM Workspace menu: From the Tools menu, select Install and then select Financial Reporting Studio and then launch the installer. Follow the installation guide to install the Financial Reporting Studio to the location you want.

Hyperion Financial Management PSU 11.1.2.4.209 (Patch 29343616)

----------------------------------------------------------------------------------------

This patch needs to be applied on all your Hyperion Financial Management application servers, HFM Web servers and also on HFR application server (if you use HFR for reporting) of the Hyperion environment.  

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\29343616 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

If you are applying this patch to version 11.1.2.4.208 or an earlier version, also execute the following sql command to upgrade the HFM schema against the Financial Management database. Use appropriate sql as per your database type.

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\<DB Type>_from_11.1.2.4.208_to_11.1.2.4.209_SchemaUpgrade.sql

E:\apps\OracleEPM\Middleware\EPMSystem11R1\products\FinancialManagement\Server\conf\Oracle_from_11.1.2.4.208_to_11.1.2.4.209_SchemaUpgrade.sql

In addition, you must run below by login to HFM schema:

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\Oracle_Create_ImportApp.sql

<EPM_ORACLE_HOME>\products\FinancialManagement\Server\conf\Oracle_Create_InsightDetails.sql

This step is required for 'Importing Application' feature. 

If you are using the extract data to database functionality with Oracle database as the target database, you will need to reconfigure DSN. To reconfigure:

1. Select Navigate, then Administer, and then Consolidation Administration.
2. From Admin Tasks, click Configure DSN. 
3. Select any data source that has Oracle database as database type, and then click on Edit.
4. Enter the database connection information, including the database server network address as Host.
5. Click Test Connection. If successful, click on Save.

Hyperion Financial Close Management and Hyperion Tax Governance 11.1.2.4.253+ (Patch 29060830)

Hyperion Tax Provision 11.1.2.4.202+ (Patch 25316913)

---------------------------------------------------------------------------

We don't use these two components in our EPM setup. You can follow README document to apply these patches, if there components are there in your EPM/Hyperion environment.

Hyperion Data Relationship Management 11.1.2.4.350  (Patch 31420887)

---------------------------------------------------------------------------------------------

This patch needs to be applied on all your DRM application server and DRM Web services servers of the Hyperion environment.  

Make sure you have stopped the DRM services. 

Take a backup of DRM schema (Must!) before applying the DRM patch.

Save a backup copy of the drm-config.xml file found under directory

E:\apps\OracleEPM\Middleware\EPMSystem11R1\products\DataRelationshipManagement\server\config

Uninstall the existing 11.1.2.4 release:

a.    Stop the Data Relationship Management service.

b.    Verify all processes are shut down.

c.    Run the installer from that release and choose the Remove option.

Run the new setup.exe to install this patch release.

If a multiple-machine configuration is being used, repeat the uninstall and install steps above for each machine.

When using an Oracle repository and upgrading from a Data Relationship Management release prior to 11.1.2.4.330, run the following statement for the current schema owner for each application to be updated: GRANT CREATE TYPE, CREATE SEQUENCE TO <user>;

In the Configuration Console on the application controller machine, select Application, then 'Apply Updates' for all existing applications.

Make sure to save any configuration changes made in the Data Relationship Management Console. Click on save configuration to do so. 

Redeploy the Data Relationship Management Web service application (oracle-epm-drm-webservices.ear) to Oracle Weblogic. The oracle-epm-drm-webservices.ear file is located in the %EPM_ORACLE_HOME%\products\DataRelationshipManagement\api directory. This step is only required if applying this patch to an installation where the Web service application has been previously deployed. See the “Redeploying a Web Services Application” section in the Oracle® Fusion Middleware Security and Administrator's Guide for Web Services for instructions to redeploy using Oracle Enterprise Manager. 

FDMEE (Hyperion Financial Data Management) Patch Set Update 11.1.2.4.220 (Patch 25312033)

----------------------------------------------------------------------------------------------------------------------

This patch needs to be applied on all your Hyperion Financial Data Management application servers (FDMEE) of the Hyperion environment.  

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\25312033 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jre E:\apps\OracleEPM\Middleware\jdk160_35

Additional Installation Instructions:

1.    Copy the following script files.

For Windows, copy from:

EPM_ORACLE_HOME/FinancialDataQuality/bin/executescript.bat.template 

to:

EPM_INSTANCE_HOME/FinancialDataQuality/executescript.bat

2.    If your deployment is a distributed environment where Hyperion Financial Management and Planning are on different servers than FDMEE, copy the following jar files:

Copy files from the Hyperion Financial Management server folder EPM_ORACLE_HOME\common\hfm\11.1.2.0\lib to the same directory on the FDMEE server.

Copy files from the Planning server on the EPM_ORACLE_HOME\common\planning\11.1.2.0\lib to same directory on the FDMEE server.

3. Now, on the servers where FDMEE is installed, you must delete tmp and cache files in the following directories:

<MIDDLEWARE_HOME>/user_projects/domains/<DOMAIN_NAME>/servers/<MANAGED_SERVER_NAME/tmp/

<MIDDLEWARE_HOME>/user_projects/domains/<DOMAIN_NAME>/servers/<MANAGED_SERVER_NAME/cache

Hyperion Profitability and Cost Management (HPCM) 11.1.2.4.131 (Patch 31159372)

-----------------------------------------------------

This patch needs to be applied on all your Hyperion Profitability and Cost Management (HPCM) of the Hyperion environment. 

Perform a full system backup, including EPM Architect and the Profitability and Cost Management databases. If you have Management Ledger applications, you should back them up using Life Cycle Management (LCM) in Shared Services. 

This patch must automatically re-register your Profitability and Cost Management applications, which will delete native non-Admin user provisioning.  Use Life cycle Management (LCM) in the Shared Services console to export the Native Directory under Foundation, usually located at Foundation => Shared Services => Native Directory, so you can restore it later.  

 1)  From the Shared Services console, open the Application Groups folder.

 2)  Open the Foundation sub-folder and select Shared Services.  

 3)  Export the Native Directory artifact and all content under it.  

You will need the resulting LCM export file later to restore the user provisioning.

Download the patch and place it under apps\OracleEPM\Middleware\EPMSystem11R1\OPatch directory.

Open command prompt and apply the patch using below commands:

cd E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch

opatch.bat apply E:\apps\OracleEPM\Middleware\EPMSystem11R1\OPatch\31159372 -oh E:\apps\OracleEPM\Middleware\EPMSystem11R1 -jdk E:\apps\OracleEPM\Middleware\jdk160_35

Clear the WebLogic cache under

E:\apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\servers\Profitability0

When you will start the Profitability service (Hyperion Profitability - Java Web Application) it automatically performs any required database updates after applying a new patch - there is no need to run a manual SQL script.

Invoke Profitability and Cost Management, open an application, and check in the System Information tab under Model Summary to verify that the Schema Version is 11124.153. If instead you get a Schema Version Mismatch error, report this to Oracle Support.  You can also search the hpcm.log file for SQL errors following the first recent occurrence of the string '11124.153'. If you find an error that you can resolve, the database upgrade can be performed manually after resolving the error, to complete the installation (see the Performing Manual Database Updates section below).

From the Shared Services console, import the LCM file you created earlier, to restore the native user provisioning.

Note also that any changes made previously to the hpm_ws_client.properties file to customize either of the web services sample programs will be overwritten by the patch and must be reapplied.

Optional – Performing Manual Database Updates (only when there are errors in the automatic update):

--------------------------------------------------------------------------------------------------------

If errors occurred during the automatic database updates performed during service startup, and you have identified and corrected the problem, you can follow these instructions to complete the required database updates manually.

1.    Shut down the Profitability service (Hyperion Profitability – Web Application).

Run the appropriate database update patch script for your database type.  Run the script from the Profitability and Cost Management product database/schema/user.  Capture the SQL script log output and save it.  If there are any errors, report them to Oracle Support.

Database Type: MS SQL Server 

Database Patch Files: Run the following database update script using Microsoft SQL Server Management Studio against the Profitability and Cost Management product database: <EPM_ORACLE_HOME>\products\Profitability\database\Common\MSSQLServer\create_11.1.2.4.131.sql

Database Type: Oracle

Database Patch Files: Run the following database update script in the Profitability and Cost Management product schema/user using Oracle SQL Developer:<EPM_ORACLE_HOME>\products\Profitability\database\Common\Oracle\create_11.1.2.4.131.sql

2.    Start the Profitability service (Hyperion Profitability - Java Web Application).

3.    Invoke Profitability and Cost Management, open an application, and check in the System Information tab under Model Summary to verify that the Schema Version is 11124.142.  

4.    For each Detailed application, you must re-deploy the Reporting Tables and Views.

5.    Complete any remaining steps you have not performed in the Applying this Patch section above.

Once all the patching steps are completed, do the following:

--------------------------------------------------------------------------------------------

1.    Stop all EPM services.

2.    For each of the following folders, create a backup copy of the folder, then delete all the contained files within the original folders:

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices$\cache

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\servers\FoundationServices$\tmp

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\servers\RaFramework$\cache

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\servers\RaFramework$\tmp

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\precompiled\raframework

apps\OracleEPM\Middleware\user_projects\domains\EPMSystem\precompiled\workspace

[NOTE: The value of the above strings will actually be a digit in your environment (e.g., FoundationServices0). Depending on the deployment, the number could be 0, 1, 2, etc.]

3.    You need to clear browser's cache and temporary files on all the client/users machines.

4.    Start all EPM services and WebLogic Admin server of your EPM environment.

5.    Validate the Hyperion applications on Microsoft Edge and Google Chrome 80+ browsers.

Post patching, all EPM applications like Workspace, Essbase, HFM, HPCM, DRM, FDMEE are working fine in our environments (DEV/TEST/PROD). We haven't faced any showstopper issue/error in these applications as such. 

There is one issue observed in EPMA that you can read about in below article:

EPM 11.1.2.4: EPMA is not compatible with Google Chrome and Microsoft Edge browsers

That's all for this post. 

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

ROBOCOPY Backup files over Network paths and ERROR 5 (0x00000005) Copying NTFS Security to Destination Directory-Access denied

As part of the annual EPM DR Rehearsal, we got a requirement to move Prod Application backup files from PROD Network location to DR network path.

In this post, we will see how we can RoboCopy EPM backup files from one Network location to another network path.

Why ROBOCOPY?

If you are copying files from one network path to another and you have Windows Vista or later operating systems then Robocopy is a better choice than any other option. Because you don't need to bother with drive mappings, since it handles UNC paths just fine.

Robocopy is not a third party software. It's native (built-in) to all versions of Windows Vista and later.

It is usually far more reliable than xcopy command, and provides a lot more options.

Robocopy is tolerant of interrupts during copying i.e. it can pick up where it left off if it gets stopped for some reason. It has the ability to recover from certain types of network hiccups automatically.

Read Link1 and Link2 for more details about ROBOCOPY and its various options.

Script using ROBOCOPY to copy EPM backups from PROD to DR network path:

@echo off

for /f "delims=" %%a in ('wmic OS Get localdatetime ^| find "."') do set "dt=%%a"

::Format the WMIC command output in DDMMYYYY format

set "YY=%dt:~0,4%"

set "MM=%dt:~4,2%"

set "DD=%dt:~6,2%"

set "today_date=%DD%%MM%%YY%"

::Define Source path

set sourcepath1=\\PROD_Network_Share\data\EPM_Backups\%today_date%

set sourcepath2=\\PROD_Network_Share\data\EPM_Backups\DataZip

set sourcepath3=\\PROD_Network_Share\data\EPM_Backups\Scripts

::Define Destination path

set destinationpath1=\\DR_Network_Share\data\EPM_Backups_Copy\%today_date%

set destinationpath2=\\DR_Network_Share\data\EPM_Backups_Copy\DataZipCopy

set destinationpath3=\\DR_Network_Share\data\EPM_Backups_Copy\ScriptsCopy\%today_date%

::Define Log path

set logfile=E:\Admin\Prod_To_DR_Copy.log

::Run RoboCopy commands

REM Copy all the files, folders and sub-folders from source to destination

robocopy %sourcepath1% %destinationpath1% /E /COPY:DAT /NP /LOG+:"%logfile%"

REM Copy today'sdate(DDMMYY).zip file from source to destination

robocopy %sourcepath2% %destinationpath2% %today_date%.zip /COPY:DAT /NP /LOG+:"%logfile%"

REM Copy all the files with extensions .sh, .mxl, .ksh, .scr from source to destination

robocopy %sourcepath3% %destinationpath3% *.sh *.mxl *.ksh *.scr /COPY:DAT /NP /LOG+:"%logfile%"

Below is what each ROBOCOPY command options used above means:

• /E = Copy files including subfolders (even empty ones)
• /COPY:copyflag[s] = what to COPY for files. Here we have selected DAT: D=Data, A=Attributes, T=Timestamps 
• /NP = No Progress - don’t display % copied text in logfile; this keeps filesize down. 
• /LOG+:logfile = Output status to LOG file (+= append to existing log).

ERROR 5 (0x00000005) Copying NTFS Security to Destination Directory. Access denied

When I was trying to figure out the right set of options for ROBOCOPY command, I encountered this error multiple times.

This error is usually caused by RoboCopy trying to copy the security settings of the files, and this causes some mismatch regarding the file permissions. 

There is a /B switch in RoboCopy for copying in backup mode but Backup mode cannot circumvent explicit NTFS deny ACL’s if the copier isn’t the objects’ owner.

Solution: Use /COPY:DAT only

Option /COPY:copyflag[s] can take multiple values based on what you want to copy for files. To Copy ALL file info (equivalent to /COPY:DATSOU), there is an option /COPYALL.

To overcome the above-mentioned error, you should use /COPY:DAT instead of the /COPYALL option, because /COPY:DAT  ignores the NTFS access control lists (the COPY:S parameter) of the files you're copying. 

This works because /COPYALL is equivalent to /COPY:DATSOU, D=Data, A=Attributes, T=Timestamps, S=Security=NTFS ACLs, O=Owner info, U=aUditing info. While we mainly need Data and Timestamps of the files for EPM backups.

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

Send PowerShell email from Windows batch file without creating separate email.ps1 file

In this article, we will see how we can send a PowerShell email from the Windows batch file without creating any separate email.ps1 file.

Suppose you have a batch script to perform a certain task and once that task is completed you want to be notified through an email alert.

One way to achieve this is by creating a separate email.ps1 file and calling that email.ps1 from your batch file. You can find multiple solutions on Google for this approach. But if the purpose is just to send an email alert with an attachment like a log file or with some text in the email body then why to go for creating another email.ps1 file.

You can simply insert the below code at the end of your batch script or with some conditional statements to send an email alert within the batch script:

:: Send Email using PowerShell

Powershell.exe -command "& {Send-MailMessage -From Admin@company.com -To users@company.com -Subject 'Prod Hyperion Backup Copy to DR for DR Rehearsal' -SmtpServer 'smtpserver.company.com' -Body 'PFA log file consisting of details of PROD Hyperion Backup files copied to DR Network path' -Attachments 'E:\Data\Prod_DR_Copy.log'}"

Here:

• '&' is the call operator. The call operator (&) allows you to execute a command, script, or function.
• Change the parameter values (To, From, Subject, Body, SmtpServer, Attachments) as per your requirement.  

There are many other parameters that you can add to the above command as per your need. Read this Link1 and Link2 for more details.

Make sure you copy and paste the above code as it is. Because on some occasions, I have noticed that using double quotes ("") in place of single quotes ('') will throw PowerShell error:

Send-MailMessage :  A positional parameter cannot be found that accepts arguement "SomeWord"...

It happens because the PowerShell command processor strips your double quotes and your "SomeWord" appears as a parameter to Send-MailMessage. So to avoid this error, you need to escape double quotes in your command.

The best way is to use single quotes ('') in your command as given above but If you really want to use double quotes for whatsoever reason, then use it like below by escaping double quotes ("") in -Subject and -Body parameters:

Powershell.exe -command "& {Send-MailMessage -From Admin@company.com -To users@company.com -Subject \"Prod Hyperion Backup Copy to DR for DR Rehearsal\" -SmtpServer "smtpserver.company.com" -Body \"PFA log file consisting of details of PROD Hyperion Backup files copied to DR Network path\" -Attachments "E:\Data\Prod_DR_Copy.log"}"

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

EPM 11.1.2.4: Microsoft Edge and Google Chrome browsers support patching : Part-1

Note: This is Part-1 of 'Google Chrome and Microsoft Edge Browser Support' patching blogs . You can read Part-2 here.

In April 2020, one of our clients (on EPM 11.1.2.4) informed us that effective immediately, their IT team would begin supporting Microsoft’s new Edge* web browser making it the default browser on user machines. Users were recommended to use Edge for day-to-day browser activities. However, they did mention that there are several apps that are not yet compatible. So if you use one of those apps that have known compatibility issues, then you can continue using Internet Explorer (IE11) in these cases.

*Microsoft Edge is based on the Chromium open-source engine, just like Chrome. Compared to IE, with Edge, you can look forward to significantly faster browsing performance and compatibility, better overall security, and a slew of helpful new features.

We checked the EPM 11.1.2.4 certification matrix and informed the client that currently (as of May 2020), Google Chrome and Microsoft Edge Browsers are not supported by all the EPM applications so we should use IE11 only to smoothly access all the Hyperion apps.

But this whole picture got changed when in June 2020, Oracle notified that Google Chrome and Microsoft Edge Browser Support has been added to Enterprise Performance Management (EPM) 11.1.2.4 (click here to read).

Oracle updated the EPM 11.1.2.4 certification matrix to reflect the new changes, as shown below:

In order to make your EPM 11.1.2.4 environment compatible with Google Chrome and Microsoft Edge Browsers, you must update to the following supported releases:

• Hyperion Shared Services 11.1.2.4.100+ (Patch 31319089. Use Patch 31574562 for release 11.1.2.4.9xx) 
• Hyperion Planning, Hyperion Capital Asset Planning, Hyperion Workforce Planning, Hyperion Project Financial Planning 11.1.2.4.009+ (Patch 29889455) 
• Hyperion Calculation Manager 11.1.2.4.014+ (Patch 28557058) 
• Hyperion Financial Reporting 11.1.2.4.712+ (Patch 30670918. Use Patch 30671119 for release 11.1.2.4.9xx) 
• Hyperion Financial Management 11.1.2.4.208+ (Patch 28511735) 
• Hyperion Financial Close Management and Hyperion Tax Governance 11.1.2.4.253+ (Patch 29060830) 
• Hyperion Tax Provision 11.1.2.4.202+ (Patch 25316913) 
• Hyperion Financial Data Management 11.1.2.4.220+ (Patch 25312033) 
• Hyperion Profitability and Cost Management 11.1.2.4.130+ (Patch 29461894) 
• Hyperion Workspace 11.1.2.4.825+ (Patch 31124100. Use Patch 31486872 for release 11.1.2.4.9xx) 
• Hyperion Data Relationship Management 11.1.2.4.350+ (Patch 31420887)
• Additionally, you must also update Oracle JDeveloper / Application Development Framework 11.1.1.7.1 (Patch 31246831).

EXCEPTIONS

• Oracle JDeveloper / Application Development Framework does not apply to Hyperion Data Relationship Management or Hyperion Workspace.

For more information, read the following knowledge article: Browser Support Added for Google Chrome and Microsoft Edge with EPM System Releases Beginning with Release 11.1.2.4 - Doc ID 2675883.1

So we planned to apply these compatibility patches in our EPM 11.1.2.4 environment starting with DEV to make it work with Google Chrome and Microsoft Edge browsers.

As an obvious next step, we started looking for known issues/errors faced by any users after applying these compatibility patches.

Then we came to know about one technical issue reported as Hyperion Shared Services not working in Google Chrome and Microsoft Edge (Doc ID 2698535.1).

Notably, as per the Doc ID 2698535.1, this issue is reported not only in EPM 11.1.2.4 but also in the very latest EPM 11.2.1 and EPM 11.2.2 releases (Let me know in the comment section if you have a functional EPM 11.2.1 or 11.2.2 environment and face this issue on Microsoft Edge or Google Chrome browser.) 

As per Doc ID 2698535.1, after applying these compatibility patches mentioned in Doc ID 2675883.1, from Shared Services Console, attempting the following tasks results in an "empty" grey screen or grey pop-up:

• create new Native User
• provision existing user
• LCM import/export
• Migration Status Report

In short, you will be having issues while using Hyperion Shared Services Console on Google Chrome or Microsoft Edge browsers post applying these patches.

Oracle has already raised two bugs for this problem as mentioned in Doc ID 2698535.1:

1. Unpublished bug created for 11.1.2.4 Bug 31686588
2. Unpublished bug created for 11.2.1 Bug 31546643

As a workaround for this issue in Hyperion Shared Services console, the following is suggested in Doc ID 2698535.1:

• For 11.1.2.4, workaround is to use Internet Explorer 11.x or Firefox 31+ ESR
• For 11.2.1, workaround is to use Internet Explorer 11.x or Firefox ESR
• For 11.2.2, workaround is to use Firefox ESR 

We decided to first clear the air about this compatibility patches and reported bugs before we patch our systems.

To that effect, we recently had a call with Oracle, and below is what we can summarize based on our discussion:

As far as these two bugs are concerned, there is still no solution provided for EPM 11.1.2.4, EPM 11.2.1, or EPM 11.2.2. Most probably, these bugs will be fixed in the EPM 11.2.3 release (as of Oct 2020). 

For EPM 11.1.2.4, you should go-ahead and patch your EPM 11.1.2.4 environment with these compatibility patches. As post patching there will be no impact on EPM 11.2.4 functioning with IE11 browser i.e. all the features/functionalities will continue to work fine on IE11 browser regardless of whether you have applied the new compatibility patches or not to make your EPM 11.2.4 environment compatible with Google Chrome and Microsoft Edge.

Post compatibility patching, the reported issue of "empty" grey screen in Doc ID 2698535.1 is limited to Hyperion Shared Services Console only for the tasks like create new Native User, provision existing user, LCM import/export, Migration Status Report. Other functionalities/features/consoles in EPM 11.1.2.4 work fine on Google Chrome and Microsoft Edge browsers. 

For the end-users, there are hardly any issues because Hyperion Shared services Console is mostly used by Hyperion Admins for user management and LCM related tasks.

The above summary gives a sense of security that at least there is nothing to lose after applying these compatibility patches mentioned in Doc ID 2675883.1 in your EPM 11.1.2.4 environment. After applying these patches in your EPM 11.1.2.4 environment, you can use:

• Google Chrome or Microsoft Edge browser for all the activities except on Hyperion Shared Services console
• IE11 for Hyperion Shared Services console activities only (mostly by Hyperion Admins)

As for the client, upgrading the current EPM version 11.1.2.4 to EPM 11.2 is still a good 6 months away (as of Oct 2020) and their IT team has already made Microsoft Edge the default browser, we are going to apply these compatibility patches in our 11.1.2.4 DEV environment for Microsoft Edge and Google Chrome compatibility. 

That's all for this post. 

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

FDMEE 11.1.2.4.220: Essbase Export data file has changed naming convention

It has not been long since we applied FDMEE (Financial Data Quality Management, Enterprise Edition) Patch Set Update 11.1.2.4.220 (Patch 25312033) in one of our TEST environments. Earlier it was on version FDMEE 11.1.2.4.210. The patch got successfully applied but during environment validation, we observed that for data load to Essbase target applications, the data export files (.dat) coming into the FDMEE outbox folder have changed its naming convention. 

This naming convention change is there only for Hyperion Essbase data files. HFM (Hyperion Financial Management) application data files continue to be using the same old naming convention. 

On investigation, we noticed it's happening exactly after we applied FDMEE (Financial Data Quality Management, Enterprise Edition) PSU 11.1.2.4.220.

What is the issue?

Earlier, in FDMEE 11.1.2.4.210, for Essbase Target applications data load, the export data file names were in the format: 

<Essbase Application Name>_<Process ID>.dat. 

But now in FDMEE 11.1.2.4.220, the data file naming convention has changed to the format: 

<Essbase Application Name>-<Essbase Database Name>_<Process ID>.dat

We see Essbase Database Name has been added with Essbase Application Name in the .dat file name.

You can see below how all the Essbase export .dat files had started to generate with the new naming format after a particular date:

This was also confirmed by comparing the data load process logs of FDMEE 11.1.2.4.220 and 11.1.2.4.210 for the same FDMEE Location or Essbase target application, as shown below:

FDMEE 11.1.2.4.210 process log:

FDMEE 11.1.2.4.220 process log:

What is the impact?

This naming convention change for Essbase target applications had broken one of our automation processes for Essbase application data load. One impacted script identified was AftLoad.py which was called after the data is loaded to the Essbase target application. In that script, it was hardcoded to read the generated data file in format <Essbase Application Name>_<Process ID>.dat.

Investigation?

As per Oracle documentation, we know that data is written to a data file in the outbox directory of the application (defined in System Settings) in the format <APPLICATION NAME>_<PROCESS ID>.dat. It is then loaded to your Essbase target applications. 

Details of our target applications (Data Management > Setup > Register > Target Application) are stored in AIF_TARGET_APPLICATIONS table in the FDMEE database. 

When we query this table for the below columns:

SELECT TARGET_APPLICATION_TYPE, TARGET_APPLICATION_NAME, ESSBASE_DB_NAME, APPLICATION_NAME FROM AIF_TARGET_APPLICATIONS WHERE TARGET_APPLICATION_TYPE = 'ESSBASE';

We get the following output:

Values in the three columns come as: 

• TARGET_APPLICATION_NAME  = Essbase Application Name
• ESSBASE_DB_NAME                    = Essbase Database Name
• APPLICATION_NAME                   = Essbase Application Name-Essbase Database Name

So apparently, to generate the Essbase export data file, FDMEE 11.1.2.4.220 is taking the value of column APPLICATION_NAME while FDMEE 11.1.2.4.210 was taking the value of column TARGET_APPLICATION_NAME. That's why the naming convention of Essbase export data files has been changed in FDMEE 11.1.2.4.220. 

HFM export data files names are unaffected because, for HFM, both APPLICATION_NAME and TARGET_APPLICATION_NAME columns values are the same as obvious:

What Oracle said?

We raised an Oracle SR and below is what they updated:

We have cross verified this with Product Development Manager and they said that this is an accepted behavior in FDMEE (Financial Data Quality Management, Enterprise Edition) 11.1.2.4.220 PSU as the Developer has changed the backend code for FDMEE functionality with Essbase. So you need to modify your automated scripts accordingly. 

So what is the workaround?

Well, as advised by Oracle we have modified our automated scripts like AftLoad.py and related batch scripts to incorporate the new naming convention of Essbase export data files. 

That's all for this post. 

I hope this article has helped you. 

Your suggestions/feedback are most welcome.

Keep learning and Have a great day!!!

Read More

EPM 11.2 servers and HyperThreading

A few days ago, we had a discussion with our server build team for acquiring new hardware for EPM 11.2 installation & configuration. To put it in context, one of our clients has EPM 11.1.2.4 on-premise setup consisting of multiple Hyperion instances running across IBM AIX 7.1 (Oracle Database), RedHat Linux 6.10 x86_64 (Essbase app), and Windows Server 2012 R2 64-bit (all other apps like HFM, FDMEE, DRM, HPCM, etc.) in a distributed environment.

Recently, the client wished to move the complete setup to the latest EPM 11.2.x release having Windows Servers 2019 and Linux 7 (why EPM 11.2 and not EPM Cloud for this setup? Well its a matter we will discuss some other time.)

In the current 11.1.2.4 setup, we have both kinds of servers -Physical Servers + VM Servers. To be precise, only HFM and Essbase apps are hosted on Physical machines in both PROD and TEST environments rest all are on VM servers.

All Physical servers (HFM + Essbase App servers) have HyperThreading** ENABLED while there is no HyperThreading on VM machines. 

**HyperThreading: HyperThreading is Intel’s term for simultaneous multithreading (SMT). This is a process where a CPU splits each of its physical cores into virtual cores, which are known as threads. For example, most of Intel's CPUs with two cores use hyper-threading to provide four threads, and Intel CPUs with four cores use hyper-threading to provide eight threads. Therefore, when HyperThreading is enabled, every CPU core has two CPU threads instead of one. A server with 8 cores would appear to have 16 CPU threads. If one thread is idle or stalls on a cache miss, the other thread can continue to execute. This is a potential throughput advantage, especially for multithreaded workloads that frequently have cache misses. HyperThreading (HT) permits the CPU to continue useful processing by running the other thread. On the other hand, both threads compete for the core’s resources, and each thread may run slower than if it owned an entire core. Each thread potentially displaces the other thread’s level 1 cache contents, causing both threads to run slower. This is especially visible for compute-intensive workloads that might normally fit in the cache without HT. The result is that it's reasonable to enable HyperThreading by default, but it can be valuable to test performance to see whether it adds performance for a specific application or not.

The current EPM 11.1.2.4 setup is working perfectly fine for past many years having an optimum performance for all the apps.

Now, while we have decided to upgrade it to EPM 11.2, it was quite apparent for us to get the answers of the following queries:

• Is there any recommendation for HyperThreading in EPM 11.2 version (Windows server 2019, Linux 7)?
• Can we continue with having the existing 11.1.2.4 HyperThreading CPU configuration even in EPM 11.2?
• Is there any known issue or performance improvement if we continue to run Essbase and HFM on HyperThreading enabled servers even in EPM 11.2?

Let's try to find out.......

Oracle and HyperThreading:

Oracle will work just fine on any OS that is running and recognizes a HyperThreading enabled system and will take advantage of the logical CPUs to their fullest extent (assuming the OS reports that it recognizes that hyper-threading is enabled). All Oracle versions can take advantage of hyper-threading and is considered a supported configuration since no support code has been added for it on the Oracle end to take advantage of HyperThreading; all the changes were in the OS, the bios, and the hardware. When Oracle asks the OS how many CPUs are in the system, the OS just reports the total number of logical CPUs.

Oracle Hyperion and HyperThreading:

I googled and read some blogs and what I found is that the experience of Oracle EPM apps with HyperThreading is quite mixed. For some, it worked well while others didn’t. I will leave it up to you to google and read those observations.

I remember there used to be a line in the Essbase 11.1.2.1 dbag document -"Enabling hyperthreading on the computer on which Essbase Server runs is not recommended." But now this line is no more there in Essbase dbag Release 11.1.2.4, implying the statement is no more valid.

My current EPM 11.1.2.4 configuration was done much before I joined this project so not sure about the exact reason to have HyperThreading enabled physical servers for Essbase and HFM apps and not for other apps which were installed on VM machines (please let me know if you know any technical correlation). But I am sure there must have been some idea behind opting the current 11.1.2.4 servers' CPU configuration (on Windows Server 2012 R2 and Linux 6).

However seeing the good performance of the current 11.1.2.4 PROD environment (Essbase, HFM, FDMEE, DRM, and HPCM) over the past many years and in the absence of any conclusive stats about HyperThreading impact in EPM servers, I was tempted to take the current PROD 11.1.2.4 HyperThreading setup as a reference for the EPM 11.2 hardware configuration too.

But, notably, EPM 11.2 version will be running on Windows servers 2019 compared to currently used Windows servers 2012 R2, so I thought better to consult Oracle to make a final recommendation on HyperThreading for EPM 11.2 Servers' CPU configuration.

And below is what Oracle said:

There are no specific Recommendations on HyperThreading for EPM 11.2 version (Windows server 2019) and we didn't see any issues reported by other customers as well. However, you can enable HyperThreading in your 11.2 setup and let us know if you face any issues post the implementation.

So it’s clear that there is no clear-cut YES/NO from the Oracle side as far as Enabling/Disabling HyperThreading in EPM 11.2 servers are concerned. Oracle leaves it to the discretion of your Hyperion technical team and Servers build team to decide whether you want to have servers built with HyperThreading enabled or disabled. Of course, it will vary with different Hyperion environments, OS, applications and most importantly it should be guided by the time-tested performance of your Hyperion applications for example in terms of HFM consolidation, Essbase calculation, aggregation, data load, and report script run, FDMEE data load run time, etc.

But yes! Oracle will come to your rescue if you face any issues post EPM 11.2 implementation (irrespective of what you chose for your CPU configuration).

And for those who are interested to know how we have decided to go for HyperThreading in EPM 11.2 servers. Well, it’s still under discussion (as of Sep 2020)....(but its almost certain that our EPM 11.2 servers will be having the same Hardware configuration what we have in our existing 11.1.2.4 setup being a time-tested reference for our current set of Essbase, HFM, HPCM, FDMEE, DRM, etc. applications).

Do leave a comment to let me know how you have built your EPM 11.2 servers (with/without HyperThreading).

That's all for this post.

I hope this article has helped you. 

Your suggestions/feedback are most welcome.
Keep learning and Have a great day!!!

Read More