The Hunger Site

Wednesday, February 22, 2012

Oracle Exadata Management — Diary of an Exadata DBA

I recently had the chance to sit down with Deba Chatterjee, Senior Product Manager, Database Manageability to talk about Oracle Enterprise Manager 12c and how it's making Oracle Exadata management easier for DBAs. He had a great analogy that I wanted to share. "To describe how Enterprise Manager is making Exadata management simple and less complex, is by using an approach we call, 'Exadata Aware Database Performance Analysis'. The best way to illustrate how this works, is to use real-world scenarios," Deba added. Here is one of those scenarios—an excerpt from the diary of an Exadata DBA.

November 15th, Late Evening

Database performance has suddenly dropped becoming very poor. The same reports that were running fine a couple of hours ago are now taking forever to complete. The boss just left me a message that the CRM analytics director was pretty upset and wants to know why we have a performance problem.

I checked the queries and the execution plans both look the same with lots of smart scans and cell offloading. Suddenly, the response time has gone south and I cannot figure out why?

My first step was to look at the database performance page in Oracle Enterprise Manager 12c, which now comes with built-in Oracle Exadata support. Hey, what do I have here in the I/O tab? The Exadata system health button is red, that's not good. What's happening? I clicked the button and it took me to the Exadata health triage page, this is where I can see if I have potential issues such as;

  • Load imbalance
  • ASM related problems
  • Cell software or hardware failures
  • Cell configuration issues
  • Network related failures


The red Oracle Exadata system health button indicates potential issues impacting database performance.

Next, I noticed that the network status was red, it must be a network issue, right? I quickly checked the administration screen in the Infiniband Target Home page, and saw we had a degraded port in the Port Details status section. I disabled the bad port and allowed the network traffic to automatically fail over to its peer. I'll have to look into this later, for now I just wanted to get the system up and running.


View of the degraded port in Infiniband Switch Schematic.

I quickly re-checked the Exadata system health status again, it showed all green—looks like everything was working, problem solved. I called my boss back to give him the good news that the database performance was back up. I sure am glad all these things are integrated; otherwise I would have been here all night trying to figure this out.

No comments:

Disclaimer

Opinions expressed in this blog are entirely the opinions of the writers of this blog, and do not reflect the position of Oracle corporation. No responsiblity will be taken for any resulting effects if any of the instructions or notes in the blog are followed. It is at the reader's own risk and liability.

Blog Archive