How to Use RDA to Generate WLS Thread Dumps At Specified Intervals?
- by Daniel Mortimer
Introduction
There are many ways to generate a thread dump of a WebLogic Managed Server. For example, take a look at:
Taking Thread Dumps - [an excellent blog post on the Middleware Magic site]or Different ways to take thread dumps in WebLogic Server (Document 1098691.1)
There is another method - use Remote Diagnostic Agent! The solution described below is not documented, but it is relatively straightforward to execute. One advantage of using RDA to collect the thread dumps is RDA will also collect configuration, log files, network, system, performance information at the same time.
Instructions
1. Not familiar with Remote Diagnostic Agent? Take a look at my previous blog "Resolve SRs Faster Using RDA - Find the Right Profile"
2. Choose a profile, which includes the WebLogic Server data collection modules (for example the profile "WebLogicServer"). At RDA setup time you should see the prompt below:
-------------------------------------------------------------------------------
S301WLS: Collects Oracle WebLogic Server Information
-------------------------------------------------------------------------------
Enter the location of the directory where the domains to analyze are located
(For example in UNIX, <BEA Home>/user_projects/domains or <Middleware
Home>/user_projects/domains)
Hit 'Return' to accept the default
(/oracle/11AS/Middleware/user_projects/domains)
>
For a successful WLS connection, ensure that the domain Admin Server is up and
running.
Data Collection Type:
1 Collect for a single server (offline mode)
2 Collect for a single server (using WLS connection)
3 Collect for multiple servers (using WLS connection)
Enter the item number
Hit 'Return' to accept the default (1)
> 2
Choose option 2 or 3.
Note: Collect for a single server or multiple servers using WLS connection means that RDA will attempt to connect to execute online WLST commands against the targeted server(s). The thread dumps are collected using the WLST function - "threadDumps()". If WLST cannot connect to the managed server, RDA will proceed to collect other data and ignore the request to collect thread dumps. If in the final output you see no Thread Dump menu item, then it's likely that the managed server is in a state which prevents new connections to it. If faced with this scenario, you would have to employ alternative methods for collecting thread dumps.
3. The RDA setup will create a setup.cfg file in the RDA_HOME directory. Open this file in an editor. You will find the following parameters which govern the number of thread dumps and thread dump interval.
#N.Number of thread dumps to capture
WREQ_THREAD_DUMP=10
#N.Thread dump interval
WREQ_THREAD_DUMP_INTERVAL=5000
The example lines above show the default settings. In other words, RDA will collect 10 thread dumps at 5000 millisecond (5 second) intervals. You may want to change this to something like:
#N.Number of thread dumps to capture
WREQ_THREAD_DUMP=10
#N.Thread dump interval
WREQ_THREAD_DUMP_INTERVAL=30000
However, bear in mind, that such change will increase the total amount of time it takes for RDA to complete its run.
4. Once you are happy with the setup.cfg, run RDA. RDA will collect, render, generate and package all files in the output directory.
5. For ease of viewing, open up the RDA Start html file - "xxxx__start.htm". The thread dumps can be found under the WLST Collections for the target managed server(s). See screenshots belowScreenshot 1:RDA Start Page - Main Index
Screenshot 2: Managed Server Sub Index
Screenshot 3: WLST Collections
Screenshot 4: Thread Dump Page - List of dump file links
Screenshot 5: Thread Dump Dat File Link
Additional Comment:
A) You can view the thread dump files within the RDA Start Page framework, but most likely you will want to download the dat files for in-depth analysis via thread dump analysis tools such as:
Thread Dump Analyzer -
Samurai - a GUI based tail , thread dump analysis tool
If you are new to thread dump analysis - take a look at this recorded Support Advisor Webcast Oracle WebLogic Server: Diagnosing Performance Issues through Java Thread Dumps[Slidedeck from webcast in PDF format]B) I have logged a couple of enhancement requests for the RDA Development Team to consider:
Add timestamp to dump file links, dat filename and at the top of the body of the dat file
Package the individual thread dumps in a zip so all dump files can be conveniently downloaded in one go.
