Friday, July 22, 2011

Where can I find the most common log files for Middleware (including FMW, ADP, JVMD) monitoring?

While working on middleware (including OEM/FMW/ADP/JVMD) monitoring, when I run into an issue, I need to access logs. OEM logs can be found at various locations. Different components do logging in different files.
As per OEM documentation, there are various log files (documented at

The most common log files that I often access are:

OMS logs
OMS logs can be found in various directories and all contain information respective to that tier.

1. OMS logs are under
<EM_INSTANCE_BASE>/em/<OMS_NAME>/sysman/log
e.g. /u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log

a. emoms.trc – this is most important trace file that contains oms actions and errors.
b. emoms.log - It has oms starting/stopping/any error when it generates an error.
c. emctl.log - emctl commands are logged in this file

2. Weblogic Server - Since em application run inside weblogic server. So these logs are often very important and contain useful info.
<EM_INSTANCE_BASE>/user_projects/domains/<domain_name>/servers/<SERVER_NAME>/logs/<SERVER_NAME>.out
e.g.
/u01/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

3. WebTier logs can be found under
<EM_INSTANCE_BASE>/<webtier_instance_name>/diagnostics/logs/OHS/<ohs_name>
e.g.
/u01/app/oracle/gc_inst/WebTierIH1/diagnostics/logs/OHS/ohs1

4. OPMN log
<EM_INSTANCE_BASE>/<webtier_instance_name>/diagnostics/logs/OPMN/<opmn_name>
e.g.
/u01/app/oracle/gc_inst/WebTierIH1/diagnostics/logs/OPMN/opmn1

EMAgent logs
EMAgent logs can be found in $AGEN_HOME/sysman/log directory

a. emagent.log - Log file used by the Agent process.  Contains all informational messages in local language.
b. emagent.nohup - Log file for the Agent monitor. This contains all actions the monitor has performed.
c. emagent.trc - Trace file used by the Agent process. Contains all the trace messages in English.
d. emagent_memdump_<time>.trc - Optional trace file, generated by an 'emctl status agent memory' command. Contains the overview of the memory usage of the Agent at that point in time.
e. emagentfetchlet.log - The fetchlet log files used by the EM Agent for certain data-gathering tasks
f. emagentfetchlet.trc - The fetchlet trace file used by the EM Agent for certain data-gathering tasks.
g. emagent_perl.trc - Trace file for the PERL scripts. This includes the PERL metrics and the discovery
h. emdctl.log - Agent control utility log file
i. emdctl.trc - Agent control utility trace file

ADP logs
When ADP is registered on same OEM UI
1. ADP Manager logs
Would be present under
$ADP_HOME/log/manager-log*.csv

2. ADP Agent logs
Would be present under target Application server directory
<APPSERVER_HOME>/AcseraAgent/* and
<APPSERVER_HOME>/AcseraLogs/*

In addition to these directories, adp agent also marks its foot print in application server specific log files for deploy/undeploy/start/stop/configuration files etc.

3. ADP UI logs
<MW_HOME>/sysman/ocamm/Log/*
e.g.
/u01/app/oracle/middleware/sysman/ocamm/Log/*

JVMD logs
When JVMD is installed with OEM
1. JVMD Manager logs
<EM_INSTANCE_BASE>/user_projects/domains/<domain_name>/servers/<SERVER_NAME>/logs/<SERVER_NAME>.out
e.g.
/u01/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMAD4JMANAGER1/logs/EMAD4JMANAGER1.out

2. JVMD Agent logs
JVMD agent logs its output on stdout. So all its logs get logged in container output file e.g. if
jamagent is deployed on OMS weblogic server, you can find jamagent logs in
<EM_INSTANCE_BASE>/user_projects/domains/<domain_name>/servers/<SERVER_NAME>/logs/<SERVER_NAME>.out
e.g.
/u01/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

3. JVMD UI logs
Some logs goto <EM_INSTANCE_BASE>/em/<OMS_NAME>/sysman/log/emoms.log
e.g. /u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log/emoms.log

and sometimes, you will find errors under
<EM_INSTANCE_BASE>/user_projects/domains/<domain_name>/servers/<SERVER_NAME>/logs/<SERVER_NAME>.out
e.g.
/u01/app/oracle/gc_inst/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.out

5 comments:

Romain Salmon said...

Hi,
Thank you for this interesting post. Do you know where we can reduce the retention of logs ? specially Webtier logs ? It seems to be never purged and i try to avoid cron purge !

Neelima Bawa said...

Pl. look at section 7.1.3 on http://download.oracle.com/docs/cd/E21764_01/web.1111/e10144/man_logs.htm#BACGIDHJ
Hope it should help.

Anonymous said...

Is there a way to determine the path for EM_INSTANCE_BASE if it's not the default path?

Neelima Bawa said...

Lot of times with upgrade, you may multiple directories or multiple copies of same files. Best way might be to do a "ps -ef|grep OMS" and then look for value of ORACLE_HOME specified there.

Anonymous said...

Heyas!
Thanks so much for this. Logged onto a newly installed, newly broken oem and your log file locations immediately pointed me to the problem. Appreciate your help on this.
Tony