Monday, October 22, 2007

How to get details about patch applied in Oracle Applications by OAM & SQL?

There are some tables in oracle apps (AD tables especially) involved when applying patches.
Some of them are very useful when we need specific information about patch already applied.

I will show the main tables and afterwards some handy related SQL’s to retrieve patch applied details and how we can also get all this information via OAM.

AD_APPLIED_PATCHES – The main table when we are talking about patches that applied in Oracle Apps.
This table holds information about the "distinct" Oracle Applications patches that have been applied.
If 2 patches happen to have the same name but are different in content (e.g. "merged" patches), then they are considered distinct and this table will therefore hold 2 records (eTRM).
I also found that if the applications tier node is separate from the concurrent manager node, and the patch applied on both nodes, this table will hold 2 records, one for each node.

AD_PATCH_DRIVERS – This table holds information about all patch drivers included in specific patch.
For example if patch contain only one unified driver like u[patch_name].drv then ad_patch_drivers will hold 1 record.
On the other hand, if patch contain more than 1 driver, for example d[patch_name].drv and c[patch_name].drv, this table will hold 2 records.

AD_PATCH_RUNS – holds information about each execution of adpatch for a specific patch driver.
In case a patch contains more than one driver, this table will hold a record for each driver.
This table also holds one record for each node the patch driver has been applied on (column APPL_TOP_ID).

AD_PATCH_RUN_BUGS – holds information about all the bugs fixed as a part of specific run of adpatch.

AD_BUGS – this table holds information about all bug fixes that have been applied.


We have 2 options to view applied patch information:
1) via OAM – Oracle Applications Manager
2) Via SQL queries


With OAM it’s easy and very intuitive, from OAM site map -> “Maintenance” tab -> “Applied Patches” under Patching and Utilities.

Search by Patch ID will get all information about this patch; In addition, drill down by clicking on details will show the driver details.


For each driver we can use the buttons (Timing Details, Files Copied, etc.) to get more detailed information.

With SQL we can retrieve all the above information, sometimes more easily.

For example: How to know which modules affected by specific patch?

With OAM:
1) search patch by Patch ID
2) click on Details
3) For each driver click on “Bug Fixes” and look on product column.

With SQL:
Run the following query, it will show you all modules affected by specific patch in one click…

select distinct aprb.application_short_name as "Affected Modules"
from ad_applied_patches aap,
ad_patch_drivers apd,
ad_patch_runs apr,
ad_patch_run_bugs aprb
where aap.applied_patch_id = apd.applied_patch_id
and apd.patch_driver_id = apr.patch_driver_id
and apr.patch_run_id = aprb.patch_run_id
and aprb.applied_flag = 'Y'
and aap.patch_name = '&PatchName';


Another SQL will retrieve basic information regarding patch applied, useful when you need to know when and where (node) you applied specific patch:

select aap.patch_name, aat.name, apr.end_date
from ad_applied_patches aap,
ad_patch_drivers apd,
ad_patch_runs apr,
ad_appl_tops aat
where aap.applied_patch_id = apd.applied_patch_id
and apd.patch_driver_id = apr.patch_driver_id
and aat.appl_top_id = apr.appl_top_id
and aap.patch_name = '&PatchName';


To check if specific bug fix is applied, you need to query the AD_BUGS table only.
This table contains all patches and all superseded patches ever applied:

select ab.bug_number, ab.creation_date
from ad_bugs ab
where ab.bug_number = '&BugNumber';


For any question or additional information you are welcome to leave a comment...

Aviad

Wednesday, October 10, 2007

Troubleshooting after Upgrade Oracle Applications 11i to 10g

Following the last post on Troubleshooting during Upgrade Oracle Applications 11i to 10g, in this post I'm going to write about troubleshooting AFTER upgrade Oracle Applications 11i to 10g.

Case 1: ORA-29877: FAILED IN THE EXECUTION OF THE ODCIINDEXPDATE ROUTINE

Symptom:

When trying to save in "Find Service Request" Form, an error message shows up:

ORA-29877: failed in the execution of the ODCIINDEXUPDATE routine
ORA-20000: Oracle Text error:
DBG-50857: oracle error in textindexmethods. ODCIIndexUpdate
ORA-20000: Oracle Text error:
DBG-10602: failed to queue DML change to column for in Package
CS_ServiceRequest_PVT Procedure Update_ServiceRequest

Cause:

KOREAN_LEXER was deprecated in release 9.0.1 and desupported in release 10.2.0.1.
It is replaced by KOREAN_MORPH_LEXER and Oracle Text indexes need to be migrated from the obsolete KOREAN_LEXER to the new KOREAN_MORPH_LEXER.
This was not done during the database upgrade.

Solution:

Run the following script as sysdba:
sqlplus "/ as sysdba" @$ORACLE_HOME/ctx/sample/script/drkorean.sql

If you get this error (or similar):

$ sqlplus "/as sysdba" @drkorean.sql
SQL*Plus: Release 10.2.0.3.0 - Production on …
Copyright (c) 1982, 2006, Oracle. All Rights Reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - Production
With the Partitioning, OLAP and Data Mining options

List of indexes that use KOREAN_LEXER as top level lexer:
List of indexes that use KOREAN_LEXER as a sub lexer:
"CS"."CS_FORUM_MESSAGES_TL_N4"
"CS"."CS_INCIDENTS_ALL_TL_N1"
"CS"."CS_KB_SOLN_CAT_TL_N1"
"CS"."SUMMARY_CTX_INDEX"
"ICX"."ICX_QUES_CTX"
Migrate KOREAN_LEXER to KOREAN_MORPH_LEXER
Rebuild all indexes that use korean lexer as top level lexer:
Reindex all documents that use KOREAN_LEXER as sub lexer
reindexing : "CS"."CS_FORUM_MESSAGES_TL" finished.
reindexing : "CS"."CS_INCIDENTS_ALL_TL" finished.
declare
*
ERROR at line 1:
ORA-29861: domain index is marked LOADING/FAILED/UNUSABLE
ORA-06512: at line 316

It is mean that one or more Text indexes are invalid so we have to check which indexes are affected.

Run this query to check which indexes are invalid:

select owner, index_name, index_type, status, domidx_status, domidx_opstatus
from dba_indexes
where ityp_owner='CTXSYS'
order by 1;

select idx_owner, idx_name, idx_type, idx_status
from ctxsys.ctx_indexes
order by 1;


Drop the affected indexes (e.g. drop index cs.CS_KB_SOLN_CAT_TL_N1 force;)

Recreate dropped indexes as described in Metalink Note 312640.1 - Oracle Text: Re-installation of Applications 11i (11.5.10) Oracle Text Indexes
Section 2: Create the CS Text index preferences and indexes (7, 8, 9, 10 & 11)

Rerun script $ORACLE_HOME/ctx/sample/script/drkorean.sql to complete the lexer conversion.


Case 2: FRM-40735: ON-ERROR TRIGGER ORA-01001 FRM-42100 WHEN SEARCHING PARTIALLY PEOPLE

Symptom:

In Users form, when trying to query by person field like ‘%name%’ an error message shows up:

FRM-40735: ON-ERROR TRIGGER RAISED UNHANDLED EXCEPTION ORA-01001

At alert_SID.log:
ORA-07445: exception encountered: core dump [qerixGetKey()+562] [SIGSEGV] [Address not mapped to object] [0x10] [] []

Cause:

Initialization parameters didn’t set properly for 10g.

Solution:

Add the follwing to initSID.ora file:
_b_tree_bitmap_plans = FALSE

For additional information see note 216205.1 - Database Initialization Parameters for Oracle Applications 11i


Case 3: Error in iAS Cache: java.io.NotSerializableException: oracle.apps.qot.util.LookupValuesKey

Symptom:

When trying to access the Quote tab from sales online, an error message is returned:

Error in iAS Cache: java.io.NotSerializableException: oracle.apps.qot.util.LookupValuesKey

Restart Apache resolve the issue for a short time but after some clicks it’s happens again.

Solution:

Should apply patch 6024690.
Note 422499.1 - LookupValuesKey Error Accessing Quote Tab


Case 4: Oracle Forms Web CGI: Error detected

Symptom:

After login to applications and choose a responsibility, just before java applet startup, an error message shows up:

Oracle Forms Web CGI: Error detected
Your request cannot be serviced at this time due to the following error:
ERROR: URL Contains One or More Restricted Characters

Solution:

1) Comment out the value FORMS60_BLOCK_URL_CHARACTERS in apps.conf.
2) Bounce Apache.

Follow Note:395628.1 - 11.5.10 Error: URL Contains One Or More Restricted Characters

You are welcom to leave a comment...

Aviad

Wednesday, October 3, 2007

Troubleshooting during Upgrade Oracle Applications 11i to 10g

During the Oracle Apps Database upgrade to 10g that I wrote about in the previous post, I ran into some errors.
In this post I will write about the errors symptoms, causes and the solutions that worked for me. :-)

Case 1: XDB (Oracle XML Database) schema is invalid

Symptom:

During the upgrade process an error message shows up:
ORA-06576: not a valid function or procedure name

In Oracle_Server.log: ($ORACLE_HOME/cfgtoollogs/dbua/[SID]/upgrade/)
call dbms_xdb.CleanSGAForUpgrade();
call dbms_xdb.CleanSGAForUpgrade();
*
ERROR at line 1:
ORA-06576: not a valid function or procedure name

When click on “Ignore” an error message shows up:


ORA-31159: XML DB is in an invalid state
ORA-00600: internal error code, arguments:
[qmtb_init_len], [], [], [], [], [],

Cause:

Oracle XML Database (XDB) is in an Invalid state.

Execute this query to check the XDB state:
select comp_name, status, version
from DBA_REGISTRY
where comp_name= 'Oracle XML Database’;

COMP_NAME STATUS VERSION
Oracle XML Database INVALID 9.2.0.6.0

Solution:

The XDB schema should be reinstalled.
I’ve Followed Note 243554.1 - How to Deinstall and Reinstall XML Database (XDB)

Step by Step Action Plan:
1) Restart the DB
2) Connect to DB as sysdba
3) Run $ORACLE_HOME/rdbms/admin/catnoqm.sql to remove XDB from database.
4) Update init.ora with the following (at least):
Shared_pool_size=150MB
Java_pool_size=150MB
5) Shutdown immediate
6) Startup
7) Run $ORACLE_HOME/rdbms/admin/catqm.sql XDB XDB TEMP
to create the XDB schema with all required objects
8) Run $ORACLE_HOME/rdbms/admin/catxdbj.sql
to register XDB java api jar files.
9) Check invalid objects for xdb users
select count(*)
from dba_objects
where owner='XDB'
and status='INVALID';
10) Check dba_registry (should be valid now…)
select comp_name, status, version
from DBA_REGISTRY
where comp_name= 'Oracle XML Database’;


Case 2: ORA-06512 AT SYS.UTL_RECOMP

Symptoms:

During the upgrade process, an error message shows up:

ORA-01003: no statement parsed
ORA-06512: at "SYS.UTL_RECOMP", line 464
ORA-06512: at "SYS.UTL_RECOMP", line 467
ORA-06512: at "SYS.UTL_RECOMP", line 607
ORA-01003: no statement parsed
ORA-06512: at line 4


I’ve take a look on log file $ORACLE_HOME/cfgtoollogs/dbua/[SID]/upgrade/PostUpgrade.log:

DECLARE
2 threads pls_integer := &&1;
3 BEGIN
4 utl_recomp.recomp_parallel(threads);
5 END;
6 /
DECLARE
*
ERROR at line 1:
ORA-01003: no statement parsed
ORA-06512: at "SYS.UTL_RECOMP", line 464
ORA-06512: at "SYS.UTL_RECOMP", line 467
ORA-06512: at "SYS.UTL_RECOMP", line 607
ORA-01003: no statement parsed
ORA-06512: at line 4

And those are the last lines of the related trace file:

[Thread-23] [18:12:10:580] [SQLEngine.setSpool:1744] Setting Spool =
/oracle/tst10db/10.2/cfgtoollogs/dbua/TST10/upgrade/PostUpgrade.log
[Thread-23] [18:12:10:581] [SQLEngine.setSpool:1745] Is spool appendable? -->
true
[Thread-23] [18:12:10:584] [SQLEngine.reInitialize:602] Reinitializing SQLEngine...
[Thread-23] [18:12:10:591] [SQLEngine.initialize:241] Execing SQLPLUS/SVRMGR process...
[Thread-23] [18:12:10:609] [SQLEngine.initialize:269] m_bReaderStarted: false
[Thread-23] [18:12:10:610] [SQLEngine.initialize:273] Starting Reader Thread...
[Thread-23] [18:12:10:617] [SQLEngine.initialize:322] Waiting for m_bReaderStarted to be true
[Thread-57] [18:58:44:357] [BasicStep.handleNonIgnorableError:430]
oracle.sysman.assistants.util.UIMessageHandler@c8e4de:messageHandler
[Thread-57] [18:58:44:361] [BasicStep.handleNonIgnorableError:431] ORA-01003: no
statement parsed
ORA-06512: at "SYS.UTL_RECOMP", line 464
ORA-06512: at "SYS.UTL_RECOMP", line 467
ORA-06512: at "SYS.UTL_RECOMP", line 607
ORA-01003: no statement parsed
ORA-06512: at line 4

Cause:

Database initialization parameters are not set properly.

Solution:

Set the following initialization parameters:

SHARED_POOL_SIZE = 250 Mb or greater
LARGE_POOL_SIZE = 150 Mb or greater
JAVA_POOL_SIZE = 150 Mb or greater

Then rerunning:

sqlplus "/ as sysdba"
SQL> shutdown immediate
SQL> startup upgrade
SQL> spool my_upgrade.log
SQL> @/rdbms/admin/catpupgrd.sql
SQL> @/rdbms/admin/utlrp.sql
SQL> spool off


Case 3: Shutdown Database Error ORA-600 [Librarycachenotemptyonclose]

Symptom:

The first shutdown after upgrade to 10g shows an error message:

SQL> shutdown immediate
Database closed.
Database dismounted.
ORA-00600: internal error code, arguments: [LibraryCacheNotEmptyOnClose], [], [], [], [], [], [], []

Cause:

Related note 435926.1 - Shutdown Database Erroring ORA-600 [Librarycachenotemptyonclose]

An unpublished bug: 4483084 say that this particular ORA-600 error can be ignored as it means nothing - actually there is no error.

Solution:

Do nothing…
When it will be available, upgrade to 10.2.0.4.0.


Case 4: OLAP Catalog component (AMD) is invalid after upgrade

Symptom:

Check OLAP catalog state:
select comp_name, status, version
from DBA_REGISTRY
where comp_name= 'OLAP Catalog';


COMP_NAME STATUS VERSION
OLAP Catalog INVALID 10.2.0.3.0

Solution:

Recreate OLAP Catalog.

ORACLE_HOME=/oracle/livedb/10.2
ORACLE_SID=TST10
PATH=$ORACLE_HOME/bin:$PATH
sqlplus /nolog
SQL>spool olap_spool.log
SQL>connect SYS as SYSDBA
SQL>@?/olap/admin/olap.sql SYSAUX TEMP;
-- Recompile everything
SQL>@?/rdbms/admin/utlrp.sql
SQL>spool off

Check status of AMD again – should be VALID:
select comp_name,version,status
from dba_registry
where comp_name like '%OLAP%';



Case 5: A lot of invalid objects in DB after upgrade

Symptom:

A lot of invalid objects in DB which cannot be compiled because of this error: (examples)

Compilation errors for PACKAGE BODY APPS.HR_DM_LIBRARY
#13#10Error: PLS-00907: cannot load library unit PUBLIC.DBMS_SQL (referenced by APPS.HR_DM_LIBRARY)
Line: 0
Compilation errors for PACKAGE BODY APPS.HR_DU_DI_INSERT
#13#10Error: PLS-00907: cannot load library unit PUBLIC.UTL_FILE (referenced by APPS.HR_DU_DI_INSERT)
Line: 0
Compilation errors for PACKAGE BODY APPS.IBE_QUOTE_SAVE_PVT
#13#10Error: PLS-00907: cannot load library unit PUBLIC.UTL_HTTP (referenced by
APPS.CZ_CF_API)
Line: 0

Cause:

Existing PL/SQL modules should recompiled in the FORMAT required by the NEW DATABASE.
The utlirp.sql hasn't been run after the upgrade.

Solution:

conn /as sysdba
shutdown
startup upgrade
@$ORACLE_HOME/rdbms/admin/utlirp.sql
shutdown
startup
@$ORACLE_HOME/rdbms/admin/utlrp.sql

About the utlirp.sql script:
The utlirp.sql script invalidate all existing PL/SQL modules (procedures, functions, packages, types, triggers, views) in a database.
This script must be run when it is necessary to regenerate the
compiled code because the PL/SQL code format is inconsistent with the Oracle executable.
Please note that this script does not recompile invalid objects automatically so you must restart the database and explicitly invoke utlrp.sql to recompile invalid objects.

Hope you won't get into this errors, but if so, you can use this post as a reference... :-)

Feel free to leave a comment or to share your experience with 10g upgrade!

You can also register for email subscription to this blog using the box at the right side menu bar.

Aviad