Questions and Answers from the Oracle Enterprise Management Online Forum

By Scott McNeil

We received tons of questions from our recent Oracle Enterprise Manager 12c Online Forum, we wanted to continue to provide answers to the most popular ones. In this installment, we'll cover questions from the DBA and developer track. Q. What kind of testing do you recommend for upgrading from Oracle Database 11gR1 to 11gR2? A. Oracle Real Application Testing using Database Replay and SQL Performance Analyzer's capabilities are recommended for any Oracle Database upgrade. Check out these resources for more details: Oracle Real Application Testing datasheet and OTN for details on upgrading your database. Q. Can I manage Oracle Exadata and traditional Oracle Databases from the same console? A. Yes, you can manage Oracle Exadata and single instance and/or RAC databases from the same Oracle Enterprise Manager Cloud Control console, as well as WebLogic and many other targets. Check out this demo to see how. Q. I thought Active Session History (ASH) was intended for real-time? A. ASH can be use for both real-time and historical analysis. It is a black box that records session activities and helps to analyze across several performance dimensions. Click here to see a quick demo. Q. What is the difference between Oracle Enterprise Manager 12c and Oracle Database Control? A. Database Control is a subset which runs off a single database. Oracle Enterprise Manager 12c is a central repository which allows administrators to manage and monitor from a single console. Q. How does Real-Time Automatic Database Diagnostics Monitor (ADDM) work – the database is hung right? A. Real-Time ADDM, included in the Oracle Diagnostic Pack for Oracle Database, uses 2 different modes of connection to the database. A normal connection and a diagnostic mode that is a lock less, latch less connection which allows only few actions. Using the diagnostic mode connection Real-Time ADDM performs a hang analysis and determines any blockers in the systems. Check out this demo to see how Real-Time ADDM works. Q. Can we achieve all the new functionality in Oracle Enterprise Manager 12c using terminal? A. No. Features like ASH Analytics, Real-Time ADDM and Compare Period ADDM are only available using Oracle Enterprise Manager 12c's console. Q. Is Real-Time ADDM available only in Oracle Enterprise Manager 12c? A. Yes, Real-Time ADDM is a new feature in Oracle Enterprise Manager 12c. Q. Can you explain the difference between Oracle Database 11g Monitor in Memory Access Mode vs. Real-Time ADDM direct access to SGA? A. Real-Time ADDM does not use Memory attach mode. We use a proprietary connection method which connects using a lock less, latch less connection bypassing the SQL Access layer. Q. Is there any limit on the number of days on which ASH can be used for analysis? A. The in memory ASH data is typically available for 1 hour or till the point the memory buffer is flushed to disk. You can find out ASH retention by using: select min(sample_time), max(sample_time) from sys.WRH$_ACTIVE_SESSION_HISTORY;

Applying database and CRS patches on Exadata

A friend asked: "Could you share what is the best practise for appling rolling patches

on database, grid infrastructure and Storage cells on EXADATA environment, we

are using half rack node in our environment?"

My reply:

You can have a look at Enterprise Manager's Provisioning and Patch Automation pack,

which can automate the application of rolling patchs on database and Grid Infrastructure.

It is worthwhile to spend a few extra dollars on Enterprise Manager packs to aid

with managment of complicated Exadata machines - and EM managment of Exadata is

going to get better and better! Get the customer to agree on the management

software cost (EM). It will definitely help them in the long run. EM will be

able to centrally manage Exadata machines in the organization, find performance

issues, patch and upgrade the database, etc.

In reality, CRS/RAC patching is used extensively by Enterprise Manager Grid Control customers for RAC databases. We have had good success stories in the real world. In Exadata, there are mostly 8 nodes in the RAC database so It makes sense to use the automated patch deployment procedures in the Provisioning and Patch automation pack of Grid Control when we can have rolling patch upgrades automatically handled by the deployment procedure. It is not rocket science.

Remember to apply the OMS cumulative patches first, of course. Core DBAs should remember that this is a way to ease their manual work and their late nights, especially if CPU/PSU patches are to be applied every three months (as per Oracle recommendations) to the RAC databases. Fundamentally, give it a whirl. If the pre-reqs pass, the patching will work. If not, you can either fix the issues or do the patching manually if you prefer.

11g OCM certification achieved

Friends,

I am pleased to announce that I am now an Oracle 11g OCM (Oracle Certified Master), after passing the 1 day upgrade exam, in addition to my Oracle 10g OCM certification.

Regards,

Porus.

Publisher says Outstanding Orders to be filled soon

Friends,

I have heard back from the publisher today. They have printed enough copies to fill the demand. All outstanding orders should be filled within the next two weeks. The demand was higher than expected and caught them off guard. That is actually good news that the book is selling better than expected. Moving forward there will be sufficient copies available to satisfy everyone.

Thanks for the great support from all my readers.

You can place an order for the book by clicking on the title below. An out of stock warning may appear right now, but you can safely ignore that. A description of the book is as follows.

Oracle Enterprise Manager Grid Control: Advanced OEM Techniques for the Real World

Oracle’s Enterprise Manager Grid Control is recognized as the IT Industry’s leading Oracle stack (application to disk) administration and management tool. It is unrivalled in its ability to monitor, manage, maintain and report on entire enterprise grids that comprise hundreds (if not thousands) of Oracle databases, middle-ware and servers following an approach that is consistent and repeatable, as well as the Oracle Applications such as Oracle E-Business Suite, PeopleSoft, Siebel, and so on.

However, Enterprise Manager Grid Control may seem daunting even to the most advanced Oracle Administrator. The problem is you know about the power of Enterprise Manager but how do you unleash that power amongst what initially appears to be a maze of GUI-based screens that feature a myriad of links to reports and management tasks that in turn lead you to even more reports and management tasks?

This book shows you how to unleash that power.

Based on the Author’s considerable and practical Oracle database and Enterprise Manager Grid Control experience you will learn through illustrated examples how to create and schedule RMAN backups, generate Data Guard Standbys, clone databases and Oracle Homes and patch databases across hundreds and thousands of databases. You will learn how you can unlock the power of the Enterprise Manager Grid Control Packs, PlugIns and Connectors to simplify your database administration across your company’s database network, as also the management and monitoring of important Service Level Agreements (SLAs), and the nuances of all important real-time change control using Enterprise Manager.

There are other books on the market that describe how to install and configure Enterprise Manager but until now they haven’t explained using a simple and illustrated approach how to get the most out of your Enterprise Manager. This book does just that.

Regards,

Porus.

New White Paper Published "Managing Oracle Applications with EM"

Friends,

I am pleased to announce that a brand new white paper of mine has been published this month (June 2011) on the Oracle Technical Network (OTN):

White Paper: Managing Oracle Applications with Oracle Enterprise Manager 11g

This white paper delves into the four Oracle Application Management Suites for Oracle’s E-Business Suite, Siebel, PeopleSoft and JD Edwards EnterpriseOne products and examines the many management capabilities that they offer for Oracle Applications. It is geared towards Oracle application managers and application administrators, with a goal to provide a clear understanding of the immense advantages and benefits the Application Management suites have to offer. It is a fantastic introduction to these powerful Application Management Suites of Oracle Enterprise Manager. Enjoy the world of Oracle EM !

Some comments in LinkedIn groups:

"Many congrats and really appreciate the efforts for the documentation on E-Biz related features. Highly informative. Its indeed a powerful EM :-)."

"Nice improvement over previous versions of EM! "

"This white paper is very useful for our jobs!"

"Hey .. Great white paper very valuable."

"Thanks for sharing this, very useful."

Thanks people for appreciating the white paper.

Regards,

Porus.

Agent Performance

A friend writes:

"Do you have any tips for keeping the emagent from making a bad situation worse. For instance when everything is working well emagent hardly takes any CPU or memory, but once the host or db's start getting a high load emagent seems to start using more CPU than just about anything else. It's kind of counter intuitive to me, that something designed to monitor and notify you about problems actually becomes the problem when a server becomes short on resources."

My response:

One point to remember is that you should always upgrade to the latest version of the agent. For eg. If your central OMS is on EM11g, then your agents should also be EM11g agents and not EM10g agents. Upgrading to the latest version at the agent level will also fix some performance issues for the agent.

The issue you describe happens when there is a contention for resources at both sides. Make sure the agent is properly configured, it has enough free disk space to store its temporary work, etc. Remember the agent pre-requisites should always be followed - for eg, if you install the EM 10g agent on a box running Oracle 9i, you must ensure the OS is patched etc upto the level required if you were installing an Oracle 10g database on the same box. All such pre-requisites for an Oracle 10g database should be followed in such a case.

Many times the prerequisites are not followed, and the agent then struggles to perform on the older boxes since the OS is not patched to the level required, some packages are older than the requirement, swap space is lower than required, etc. Then, there are performance issues. So please check the setup first.

My New EM11g Book is in stock now

I am pleased to say my first EM11g book is in stock and is being delivered now.

Thanks to all my readers for their patience. The book is not very expensive,

only $19.95 USD (30% off if ordered now).

http://www.rampant-books.com/book_1001_advanced_techniques_oem_grid_control.htm

Oracle Enterprise Manager Grid Control: Advanced OEM Techniques for the Real World

Oracle’s Enterprise Manager Grid Control is recognized as the IT Industry’s leading Oracle database administration and management tool. It is unrivalled in its ability to monitor, manage, maintain and report on entire enterprise grids that comprise hundreds (if not thousands) of Oracle databases and servers following an approach that is consistent and repeatable.

However, Enterprise Manager Grid Control may seem daunting even to the most advanced Oracle Administrator. The problem is you know about the power of Enterprise Manager but how do you unleash that power amongst what initially appears to be a maze of GUI-based screens that feature a myriad of links to reports and management tasks that in turn lead you to even more reports and management tasks?

This book shows you how to unleash that power.

Based on the Author’s considerable and practical Oracle database and Enterprise Manager Grid Control experience you will learn through illustrated examples how to create and schedule RMAN backups, generate Data Guard Standbys, clone databases and Oracle Homes and patch databases across hundreds and thousands of databases. You will learn how you can unlock the power of the Enterprise Manager Grid Control Packs, PlugIns and Connectors to simplify your database administration across your company’s database network, as also the management and monitoring of important Service Level Agreements (SLAs), and the nuances of all important real-time change control using Enterprise Manager.

There are other books on the market that describe how to install and configure Enterprise Manager but until now they haven’t explained using a simple and illustrated approach how to get the most out of your Enterprise Manager. This book does just that.

Covers the NEW Enterprise Manager Grid Control 11g.

Regards,

Porus.

News from my publisher

News from my publisher: Due to unforeseen printer-related delays for my EM Book, orders placed for the book may not have been delivered so far. The publisher is now using a new printer and expects to start delivery of my EM book very soon. If you have already placed an order, please wait another week or so for delivery. Thanks for your patience.

Regards,

Porus.

Latest List of my Published White Papers and Technical Articles on OTN

Many of you have asked me for a full list of all my published technical articles/white papers on the Oracle Technical Network (OTN), for easy reference. Here is the latest list as of Oct 2012. I have placed the white papers at the top, and the articles are in chronological order in the article list. The latest article on Enterprise Manager Cloud Control 12c, published in October 2012, is right at the bottom of the list. A couple of these articles were even in the most popular OTN article list in 2009.

My Published White Papers on Oracle Technology Network (OTN)

White Paper: Advanced Uses of Oracle Enterprise Manager 11g
White Paper: Managing Oracle Applications with Oracle Enterprise Manager 11g

My Published Articles on Oracle Technology Network (OTN)
Using Grid Control with Filer Snapshotting
Oracle Enterprise Manager Grid Control Architecture for Very Large Sites
Oracle RMAN Backups: Pushing the "Easy" Button
Patch a Thousand Databases, Using Oracle Enterprise Manager Grid Control
Easy Disaster Proof Production with Grid Control
Using Oracle GoldenGate for Real-Time Data Integration
Mask Your Secrets Using Oracle Enterprise Manager
Manage Mass Provisioning Using Oracle Enterprise Manager Grid Control

Overview of Oracle EM Management Packs

Provision Your Oracle RAC Systems Using Oracle Enterprise Manager
NEW!! Ease the Chaos with Automated Patching: Oracle Enterprise Manager Cloud Control 12c

Happy Reading

No one manages Oracle technology better than Oracle

There are several features in Oracle EM that third-parties do not have.

For eg., the EM Tuning pack has the SQL Profile. The SQL Profile contains auxiliary information collected by the ATO for a SQL statement and compensates for missing or stale statistics or errors in optimizer estimates, and is able to improve performance of even packaged application SQL without changing the code. No other non-Oracle software has this.

Oracle EM is also integrated with My Oracle Support in a way no other software can match and this is a big plus point.

The Application Management Suite for Oracle EBS can manage cloning, patching and customizations, in a way no other software can match. And there are App Mgt Suites for EBS, PeopleSoft and Siebel.

No one manages Oracle technology better than Oracle, and that is a fact. Management has to decide if they want to bypass all that EM has to offer, or they want to use it. It makes sense to use EM. The Oracle stack is best managed by Oracle Enterprise Manager.

For more information, please refer to my OTN article:

Overview of Oracle Enterprise Manager Management Packs

http://www.oracle.com/technetwork/articles/oem-packs-overview-195704.html

Enterprise Manager Creating Oracle Data Guard Standby for 11.2 Database with Grid user role separation

We recently had a LinkedIn discussion in which one of my friends had written:

We went down the role separation path with a grid user for storage/cluster admin that we were going to handover to our unix team and the oracle user for normal DB admin.....It turns out our Unix admins didn't want anything to do with managing ASM or GI so we'd have been better staying with it all under the oracle account. The role separation does cause it's fair share of issues. The listener being owned by the grid user is a particular problem as the oracle user can no longer reconfigure/restart the listener. This breaks a lot of the OEM provisioning functionality and causes problems when attempting to create standby databases...... a lot of manaul work arounds are needed to resolve.

My response to this was:

What he said about "Keep it simple" is right - that principle applies in any case. However just want to clarify on the issue mentioned. A little later after the comment was made, this issue was investigated further in his "My Oracle Support" SR and it was found it was a listener.ora file parsing issue - some spaces were missing in the sid_list_listener entry in the listener.ora file probably because the entry had been manually made. The person concerned himself found this out and fixed the issue. When the extra spaces were added in this line, the file parsed ok. So, there were no real issues with the EM11g Create Data Guard Standby database wizard which he was talking about (this is actually separate from EM11g Provisioning procedures). The EM Create Data Guard Standby database wizard works ok even with role separation with a grid user for the Grid Infrastructure home.

My latest OTN Article: Provision Your Oracle RAC Systems Using Oracle Enterprise Manager

Friends,

A new technical article written by myself has just been published on the

Oracle Technology Network (OTN):

Provision Your Oracle RAC Systems Using Oracle Enterprise Manager

http://www.oracle.com/technetwork/articles/havewala-rac-provisioning-253418.html

(Published January 2011)

Learn how to seamlessly provision multi-node Oracle RAC systems in a

totally automated manner - including provisioning of RAC databases.

Have a look and spread the word.

Regards,

Porus.

Blog Post on New book

Thanks to Aman Sharma for his blog post on my new book:

http://blog.aristadba.com/?p=167

Shared loader directory, RAC and WebLogic Clustering

A friend wrote:

I was reading your paper on OTN http://www.oracle.com/technetwork/articles/havewala-gridcontrol-088685.html Its a very interesting paper and well written. I am doing a similar implemention using OEM 11g on a RAC environment and had a few questions..

1. In the paper you reference about using an IP configuration to load balance between a RAC environment. In this option, uses will round robin between the two management servers running on the two instances. In this case, what happens to the feeds that come from the managed agents to the EM cluster. the feed from the agent is only received by one instance and the certain data such as the log details are available only on one instance.. how does the data get over to the other instances. Can i use ACFS to locate these files so that they are visible to both servers.

2. Have you tried to cluster the weblogic servers so that they make the OMS highly available? the Weblogic server that comes with the GC download is not complete and many of the features are not available. however if the organization has weblogic can this be used, what are the implications with use this option, specifically with respect to future OEM upgrades etc.Is weblogic clustering supported for OEM implementation?

My reply:

Thanks for your kind words. RAC is optional. Even the load balancer is optional. The feed from the agents also goes to the load balancer on a different port and it is routed to the available management server. In normal case, this is ok. If there is a failure, that is where the concept of the shared loader directory is used - so other management servers can be used to pick up the load. This concept is explained in the advanced EM manual. You should be able to use ACFS for the shared loader directory, however it is not certified or supported for use for this purpose. See the manual for the additional steps to set up the shared loader directory.

WebLogic Clustering will not be used by EM GC 11g Release 1.

Good Luck for the implementation. My tip is, keep it as simple as possible. My idea would be to first put this on a single database with Data Guard, and then upgrade to RAC after EM GC has settled down.

Regards,

Porus.

Another News Article on my new book

Another News Article on my new book. Thanks to Fahd Mirza for this story.

http://itnewscast.com/database/oracle-grid-control-book-porus-homi-havewala-maestro-work

New EM Provisioning Article published on Oracle Technical Network

Dear Friends,

Happy Deepavali (Indian Festival of Lights).

I am pleased to announce that a new article of mine has been published on the Oracle Technical Network in November 2010:

Manage Mass Provisioning Using Oracle Enterprise Manager Grid Control

http://www.oracle.com/technetwork/articles/mass-provisioning-177500.html

Learn how to provision literally hundreds or even thousands of systems using the mass provisioning techniques described here. The Deployment Procedures supplied by Oracle for provisioning, really make it a simple process.

Regards,

Porus.

New Application Management Suites

Blog Entry by Chung Wu

Introducing Oracle Application Management Suites, the most complete and integrated solutions designed specifically for managing Oracle E-Business Suite, Siebel, PeopleSoft and JD Edwards EnterpriseOne applications.

Four management suites are now available. They include:

- Application Management Suite for Oracle E-Business Suite
- Application Management Suite for Siebel
- Application Management Suite for PeopleSoft
- Application Management Suite for JD Edwards EnterpriseOne

Built on Oracle Enterprise Manager, these management suites enable you to manage your mission critical Oracle Applications more effectively so that you can achieve better application performance and availability, improve agility while maintaining control, and lowering the costs and risks that are associated with operating these software systems.


The management suites provides you with a different and better approach of managing your applications. Applications are different from other information technologies such as database, servers, and storage arrays in that applications are used directly by end users. Consequently, proper management of applications need to begin with a clear understanding on end user experience of the applications. Traditional component centric approach of managing application does not provide this kind of insight. A key new addition to each of the four application management suites is user experience management. Using the suites, one could gain insights on the identity of the users, when they use the apps, where they log in from, and click-by-click information on what they did and the application experience that they received, including response time measurement and records of errors and warnings. These information help you focus your resource on making improvements to the parts of the application and the underlying IT infrastructure that matter most to the users.

The management suites also provides full visibility into the internal workings of application specific artifacts beyond what typical management tools provide. Typical generic management tools monitor applications from the operating system level. However, each of this packaged Oracle applications include "containers" that are execution engines of various services. Application Management Suite's system monitoring and diagnostic capabilities enable you to look inside these containers, exposing a full set of metrics that you can set alerts on and visualize trends vs dashboards and reports. In additional, specialize tools for troubleshooting transactions, analyzing log files and diagnosing JVM problems are also built into the suites to provide you with a full range of options to address the toughest production problems. These tools help you proactively manage your application infrastructure in order to prevent problems from occurring, and when things do go wrong, troubleshoot problems quickly in order to minimize impacts to business operations.

In addition to user experience management and system monitoring and diagnostics, the suites also support configuration management. When something goes wrong with an application system, the natural thing to ask is what changed, as the changes are often the root causes of problems. Application Management Suites include configuration analysis tools that enable you to track changes to system configuration settings of the applications, compare settings across environments and create snapshot of good configurations so that they can be referenced later on. The new management suites added specific support to automate IT compliance management tasks as well, providing real time change detection and reconcilation of changes against change management authorizations so that unauthorized changes can be detected, and presenting compliance status on a set of compliance dashboards and reports, making it much easier to manage configuration related IT compliance tasks.

The fourth set of capabilities that the management suites provide is lifecycle automation. Using the suites, administrators can automate manual and error prone procedures by defining multi-step batch jobs and schedule them to run unattended, leaving the tasks of monitoring to be handled by the management suites automatically. For Oracle E-Business Suite, additional support for patching, cloning, customization deployment and functional setup migration are also included in the Application Management Suite for Oracle E-Business Suite product.

Additional information about these products is available on Oracle Technology Network. Click here for more information.

- Chung Wu.

My Oracle White Paper published

Friends,

The good news is that I have recently written an Oracle White paper that has been published. This is my first white paper in addition to my articles already published on the Oracle Technical Network.

The white paper is on Advanced Uses of Oracle Enterprise Manager 11g. The paper has been announced in the Enterprise Manager Newsletter of September 2010.

Please read and recommend to your friends! Thanks very much.

Regards,

Porus.

Oracle Live Webcasts on "Improving Business Performance with Oracle Core Technology"

Friends, If you and your colleagues are interested, you can attend Oracle Live Webcasts on "Improving Business Performance with Oracle Core Technology". This is to show you how to improve business performance with small changes to your database. The results are rapid and can make a real difference to the performance of all your applications.

Check out the EMEA live Webcast in English, French, German, Italian and Spanish here. You learn about how to transform your database with a big impact on the business. The contents of the Webcasts also cover topics relevant for Oracle Enterprise Manager (business-driven Application management, integrated application-to-disk management, performance, analytics, availability etc.).

Good Post by Werner De Gruyter on Differences

Good post by Werner De Gruyter on differences between Enterprise Manager 10g Grid Control and 11g Grid Control:

http://blogs.oracle.com/oem/2010/09/differences_between_em10g_and.html