How to resolve a Windows locking issue that prevents patching via Grid Control

We ran into a peculiar issue when patching an 11g Windows database using Grid Control, and I want to describe the peculiar way we used to resolve this issue in Windows.

As a background, Grid Control enables the DBA to patch an Oracle Database using the Provisioning Pack. Oracle supplies deployment procedures that can be executed out of the box, or copied and modified, to patch a single database, or a RAC database, and so on. The deployment procedure, when it reaches the step of actually patching the database, ultimately uses opatch at the database server level. 

Now, the problem was, that when trying to patch an 11g database on Windows using Grid control,  opatch was failing since the "oci.dll" file in the 11g db home was locked. 

We first used a third party utility ProcessExplorer.zip to find the file, and this found it was locked by windows process svchost (Service Host). (There are multiple svchost processes running. To see which copy of svchost is running what service, use "tasklist /svc" in a command window.)

As per a hint found in Google search, we set the windows service "Distributed Link Tracking  Client" to manual and stopped this service. This made no difference and the file stayed locked.  As per another hint found in Google search, we renamed oci file to oci.bak then killed the scvhost process using process explorer. The scvhost process restarts automatically but cant find the oci file. But this effected the working of Grid Control (because the listner cannot really work without oci.dll being used) so do not use this renaming technique.

We then found that when the listener process is stopped, and scvhost is killed, scvhost restarts without the lock. So the issue obviously happens because during the patch process, when the listener is stopped, the lock is not released by scvhost. This was the main cause of the issue.

We then edited a custom deployment procedure and enabled the extra step to stop all Windows services in that home. Normally this step is not enabled. But this had no effect on the error.

The solution that ultimately worked was that we used unlocker (another third-party utility) to unlock oci.dll just after the databases and listeners were stopped by the deployment procedure. Make sure you exit process manager before doing the unlocking, because process manager itself locks other dlls. So the steps are:

1. Start the patching process.
   
2. Dont start the third-party process manager at all. 
   
3. In windows explorer go to the 11g Oracle Home/bin and find oci.dll. 
   
4. Wait till the grid control patching reaches the step of stopping databases.
   
5. Right click on oci.dll and unlock the lock of scvhost on this file, using third-party unlocker. 
   
6. Patching then continues successfully since the file is not locked any more.
   

Managing SAP R/3 with Enterprise Manager

Managing SAP R/3 with Enterprise Manager - would you have thought it possible? Doubtful to say the least. 

I checked with one of the top Oracle experts here in Singapore, Alivin, who surprised me by saying SAP R/3 can definitely be managed using a plug-in to Enterprise Manager known as NimBUS SAP Plug-In. This is a third-party plugin. 

This is described at http://www.nimsoft.com/oem/ if you are using SAP and want to use Enterprise Manager to

manage and monitor SAP. 

Grid Control Sessions in DBA Round Tables by Oracle OTN

Fellow DBAs,

The Oracle Developer Program team (which is the same as the Oracle Technical Network ie. OTN) is holding an Oracle DBA Round Table in Kuala Lumpur on 25th November and in Singapore on 27 November. Oracle DBA Round Table is a half-day session and the purpose of the round table is to create a sense of community, share experiences and provide DBAs with the opportunity to network with Oracle experts, Oracle ACE/ACE Directors and other DBAs.

As the only Oracle ACE Director in this region, I will be conducting a session at both events on "Patching Databases with Grid Control". The scheduled agenda is as follows:

Kuala Lumpur, 25 November
Time	Activity	Speaker
10.00a.m.	Introduction	Blair Layton
10.15a.m.	Flashback Data Archive	Rajender Singh, Oracle ACE
11.15a.m.	Backup and Recovery with Oracle Database
12.15p.m.	Lunch
1.00p.m.	Patching Oracle Databases via Grid Control	Porus Homi Havewala, Oracle ACE Director, S&I Systems Pte Ltd
2.00p.m.	Customer Presentation
3.00p.m.	Break & General Discussion	Blair Layton
4.00p.m.	Closing Remark	Blair Layton
Singapore, 27 November
Time	Activity	Speaker
10.00a.m.	Introduction	Blair Layton
10.15a.m.	Creating an Ideal Database Infrastructure	Ravishankar Buddha
11.15a.m.	Backup and Recovery with Oracle Database
12.15p.m.	Lunch
1.00p.m.	Patching Oracle Databases via Grid Control	Porus Homi Havewala, Oracle ACE Director, S&I Systems Pte Ltd
2.00p.m.	Customer Presentation - Olam 11g Reference
3.00p.m.	Break & General Discussion	Blair Layton
4.00p.m.	Closing Remark	Blair Layton

The events will be held in the Oracle offices:

Oracle KL address:
Langkawi Room
Level 38 Menara Citibank, 165 Jalan Ampang, 50450 Kuala Lumpur, Malaysia

Oracle Singapore address:
Hong Kong Room
6 Temasek Boulevard #18-01 Suntec Tower 4 Singapore 038986

Please let me know if any of you will be attending the round tables in either Kuala Lumpur or Singapore, and I will let Oracle know. You can write to oracleacedirector@si-asia.com , thanks.