EM12c: How to deploy JVMD agent on EBiz 12.1 (oc4j based instance) ?

E-Business 12.1 has Oracle iAS based tech stack. It has oc4j as its application server.
EM12c doesn't have any automated UI flow for deploying JVMD agent on oc4j server.
That deployment has to be manual.

Manual JVMD deployment is referred in this doc

Download JVMD agent file from EM12c UI as given here

1. In EM Cloud Control, Select Setup menu-> Middleware Management-> Application Performance Management.

2. On the Application Performance Management page, select JVM Diagnostics Engine.

3. Click Configure. The 'JVM Diagnostics Setup' page appears.

4. On the 'JVM Diagnostics Setup' page, click JVMs and Pools, then click Download.

5. From the JVMD Component menu, to download jamagent.war. From the JVMD Component menu, select JVMD Agent, then click OK. The JVM Diagnostics Agent web.xml parameters dialog box appears.

6. From the Available Engines menu, select a JVMD engine from the list, then click Download.
  
7. Click Download to download jamagent.war.


Due to some changes in EBiz 12.1 iAS, it requires some additional steps for JVMD agent deployment. EBiz12.1 oc4j doesn't have its own admin web console, so one needs to deploy JVMD agent using command line utility like admin_client.jar.  That requires resetting oc4jadmin password for oafm and oacore  etc.

Given steps can be used as workaround for JVMD agent deployment on oc4j. Not sure if one would like to use these steps in a Production env. Take your call.

a. To reset oc4jadmin password for oafm and oacore
edit the following files:
/oracle/r12/VIS12/apps/tech_st/10.1.3/j2ee/oafm/config/system-jazn-data.xml
/oracle/r12/VIS12/apps/tech_st/10.1.3/j2ee/oacore/config/system-jazn-data.xml
look for the <credentials> section under OC4J Administrator
replace the encrypted password e.g. {903}xxxxxxxxxx with a new plaintext e.g. !welcome1
(new plaintext must begin with a !)

Also modify the opmn config file, to disable SSL and allow remote connections
edit:
/oracle/r12/VIS12/apps/tech_st/10.1.3/opmn/conf/opmn.xml
change the <notification-server> section so it looks something like this
   <notification-server interface="ipv4">
      <ipaddr remote="0.0.0.0" request="0.0.0.0"/>
      <port local="17006" remote="17007" request="17005"/>
      <ssl enabled="false" wallet-file="$ORACLE_HOME/opmn/conf/ssl.wlt/default"/>
   </notification-server>
(add the line with <ipaddr...> and change <ssl enabled="true"...> to false)

b.  Use following command to deploy JVMD agent
java  -jar /oracle/r12/VIS12/apps/tech_st/10.1.3/j2ee/home/admin_client.jar
 deployer:oc4j:opmn://linux-ebs-r12-pc.us.oracle.com:6501/oafm oc4jadmin welcome1
-deploy -file /tmp/jamagent.war -deploymentName jamagent -bindAllWebApps

or
 
java  -Djava.util.logging.config.file=logging.properties   -jar
/oracle/r12/VIS12/apps/tech_st/10.1.3/j2ee/home/admin_client.jar
 deployer:oc4j:opmn://linux-ebs-r12-pc.us.oracle.com:6501/oafm oc4jadmin welcome1 -deploy -file /tmp/jamagent.war -deploymentName jamagent -bindAllWebApps

EM12c: How to deploy JVMD agent on weblogic 12c manually?

Assumption: User has EM12cR2 with FMW 12.1.0.3 plugin

Simple option would be to deploy jamagent via Diagnostics Agent Deployment wizard. In case when deployment job fails for some reason then jamagent can be deployed manually. Instructions are given below.

1. Goto Setup Menu->Application Performance Management->Select JVM Diagnostics Engine->Click on Configure button as shown here

2. Select 'JVMs and Pools' tab on 'JVM Diagnostics Setup' page

3. Click on Download button

4. It will show you a Dialogue box that will list options to download JVMD Agent/DB Agent/JVMD Engine. Select JVMD Agent.

5. For a weblogic server, do provide the WLS server EM target name as Target Name and WLS Domain name as Pool Name. Click on Download button.

6. Download jamagent.war

7. Take this jamagent.war and deploy it via weblogic console UI.

Note: One can find more info on http://docs.oracle.com/cd/E24628_01/install.121/e24215/jvmd_using.htm#BCGCHEAG

EM12c: How to fix java.lang.UnsatisfiedLinkError: /tmp/libjamagent.so.1: failed to map segment from shared object: Operation not permitted error while deploying Jamagent?

Problem: While deploying jamagent, following error is seen in jamagent log:
12/07/04 07:58:34 JAM Agent: jaminit.init()
12/07/04 07:58:34 JAM Agent: jaminit.init() Instantiating jamagent
12/07/04 07:58:34 OS is Linux
12/07/04 07:58:34 arch is i386
12/07/04 07:58:34 JAM Agent loadNative() Wrote to [/tmp/libjamagent.so.1]
12/07/04 07:58:34 >>java.library.path: /u01/usr/apps/abc/10.1.3/appsutil/jdk/jre/lib/i386/server:/u01/usr/apps/abc/10.1.3/appsutil/jdk/jre/lib/i386:/u01/usr/apps/abc/10.1.3/appsutil/jdk/
12/07/04 07:58:34 JAM Agent: loadNative Exception loading [/tmp/libjamagent.so.1] /tmp/libjamagent.so.1: /tmp/libjamagent.so.1: failed to map segment from shared object: Operation not permitted
12/07/04 07:58:34 java.lang.UnsatisfiedLinkError: /tmp/libjamagent.so.1: /tmp/libjamagent.so.1: failed to map segment from shared object: Operation not permitted
12/07/04 07:58:34       at java.lang.ClassLoader$NativeLibrary.load(Native Method)
12/07/04 07:58:34       at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1803)
12/07/04 07:58:34       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1699)
12/07/04 07:58:34       at java.lang.Runtime.load0(Runtime.java:770)
12/07/04 07:58:34       at java.lang.System.load(System.java:1003)
12/07/04 07:58:34       at jamagent.jamagent.loadNative(jamagent.java:688)
12/07/04 07:58:34       at jamagent.jamagent.<init>(jamagent.java:995)
12/07/04 07:58:34       at jamagent.jaminit.init(jaminit.java:52)

Possibility is that error is encountered when /tmp directory is mounted with -noexec option.

Solution: Make /tmp directory executable using following command:
mount -o remount,exec /tmp

Retry deploying jamagent.

Jam Agent : Could not find required Ocamm agent class file - What does it mean?

When one deploys jamagent, one often sees this error.
"Jam Agent : Could not find required Ocamm agent class file"
What does it mean?

JVMD and ADP(CAMM) have been integrated for some enhanced functionality
called Request Monitoring in EM11g. Only If you are using Request
Monitoring then this error matters, but in general this error has
no impact on jamagent functionality and can be ignored safely.

What is Cross Tier (xtier) functionality in JVM Diagnostics (AD4J)?

Using JVM Diagnostics(AD4J), one can diagnose an issue across 3 tiers i.e. Java application<-> JVM <-> Database. User can relate an application activity to database activity. As of now, only oracle database is supported.

If cross tier is configured correctly, in real time user can see DB Wait link for active JVM threads that are busy in db activity. While in monitoring data, data appears in Top DB States and Top SQLs. One can use filters on them. By looking at Database pages, one can see JVM activity inside db.

Cross tier configuration requires: 

• Java application and JVM with jamagent running and registered with console
• Database to be monitored needs to be registered with console
• Dbagent needs to be running on machine where database is running as same user who is running oracle database.

To ensure that cross tier is configured correctly:

1.            Navigate to Targets->Middleware->JVM Diagnostics->Setup->Databases page and ensure that database to be monitored is registered correctly. Verify following:

•             OS User field should specify the user who installed the database

•             DB User field should specify the DB application user credentials (Should have access to GV$ and V$ views)

•             DB User (Explain Plan) field should specify DB system user credentials

2.            DB Agent should be running on the machine on which DB is installed with correct manager IP and port

./dbagent jamconshost=<ManagerIP> jamconsport=<ManagerPort>

Note: On aix, either dbagent needs to run as root user or set the suid to root for dbagent binary.

3.            To verify cross tier functionality, go to Targets->Middleware->JVM Diagnostics. Select JVM on left Panel and click JVM->Threads->Real Time Analysis page for the JVM if you encounter a thread that is in the DB Wait state, it should be a hyperlink so you can follow the thread into the Database pages and see its database session information including the SQL that it executes.

In addition, if monitoring is started, data would appear in Top DBStates and Top SQLs table and charts.