Sunday 22 January 2012

R12 Cloning onto a New Server/Machine.

R12 Environment cloning to a new box, thought will share my experience..Good if it's going to be useful to some one..

Objective  : Clone/Create an R12.1.3 Environment onto a new box.

Source: Linux 64 bit Servers.
ERP : 12.1.3
DB : 11g R2.

Target Version : Should be same as Source.

1. Complete installing the OS pre requisites as documented in the note:761566.1 [For any other OS, Please refer to 405565.1].


2. Setup your environment varaibles and PATH to the appropriate values.
3. Apply the required patches.
- 9171651
- 9833058
- 12404574 - has a prerequisite patch - 9171651.
- 12598630.
These were the patches required in our environment, please complete a careful analysis before concluding the patch list.

3. Auto config on DB Tier and Apps Tier was completed earlier and were not required in our case.
4. Updated Current view snapshot (Complete APPL_TOP).
5. Prepare the source system for cloning.
5a. $ORACLE_HOME/appsutil/scripts/[CONTEXT_NAME]
perl adpreclone.pl dbTier
5b. $INST_TOP/admin/scripts
perl adpreclone.pl appsTier

6. Copy the application tier.
- Copy APPL_TOP
- COMMON_TOP
- Apps Tech stack
-  ORACLE_HOME
- IAS_ORACLE_HOME

In our environment, I created the tarzip of the root folder like /u01/app/R12... completely and then copied them over to the target.
at the target instance after  untaring the required files, the directory structure was layed and all that I had to do was to change the permissions and rename the INST-TOP directory

Note: INST_TOP directory  is not required to be copied from PROD as this directory will be created during cloning.

Configure the target system.


-On the Database tier, log on to $ORACLE_HOME/appsutil/clone/bin
perl ./adcfgclone.pl dbTier
Provided below are some of the options that you would be prompted with.
Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA
                          Oracle Applications Rapid Clone
                                     Version 12.0.0
                        adcfgclone Version 120.31.12010000.8
Enter the APPS password :
Running:.........
perl adcfgclone.pl dbTier
Log file located at <CloneContext_1125160841>.log
Provide the values required for creation of the new Database Context file.
Target System Hostname (virtual or normal) [machine_name] :
Target Instance is RAC (y/n) [n] :
Target System Database SID : TEST
Target System Base Directory : /u01/oracle
Target System utl_file_dir Directory List : /usr/tmp
Number of DATA_TOP's on the Target System [1] :
Target System DATA_TOP Directory 1 [/u01/app/SRC1/srcdata] : /u01/app/test/data
Target System RDBMS ORACLE_HOME Directory [/u01/oracle/db/tech_st/11.1.0] : <targe_O_H> location.
Do you want to preserve the Display [87.0] (y/n)  : n
Target System Display [machine1:0.0] : machine:1.0
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
 Target System Port Pool [0-99] : 6
Checking the port pool 6
done: Port Pool 6 is free
Report file located at ....../portpool.lst
Complete port information available at ......./portpool.lst


On the Application tier as appltest user


$COMMON_TOP/clone/bin
perl adcfgclone.pl appsTier
Provided below are some of the options that you would be prompted with.
perl adcfgclone.pl appsTier
prompt entries.
Log file located at <CloneContext_1126140313>.log
Provide the values required for creation of the new APPL_TOP Context file.
Target System Hostname (virtual or normal) [machine1] :
Target System Database SID : TEST
Target System Database Server Node [machine] :
Target System Database Domain Name [domain.com] :
Target System Base Directory : /u01/oracle
Target System Tools ORACLE_HOME Directory [/u01/oracle/apps/tech_st/10.1.2] : /u01/oracle/TEST/apps/tech_st/10.1.2
Target System Web ORACLE_HOME Directory [/u01/oracle/apps/tech_st/10.1.3] : /u01/oracle/TEST/apps/tech_st/10.1.3
Target System APPL_TOP Directory [/u01/oracle/apps/apps_st/appl] : /u01/oracle/TEST/apps/apps_st/appl
Target System COMMON_TOP Directory [/u01/oracle/apps/apps_st/comn] : /u01/oracle/TEST/apps/apps_st/comn
Target System Instance Home Directory [/u01/oracle/inst] : /u01/oracle/TEST/inst
Target System Root Service [enabled] :
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [disabled] :
Do you want to preserve the Display [machine:0.0] (y/n)  : n
Target System Display [machine-ebs-test:0.0] : machine:1.0
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 6
Checking the port pool 6
done: Port Pool 6 is free
Report file located at ............./portpool.lst
Complete port information available at ............../portpool.lst
UTL_FILE_DIR on database tier consists of the following directories.
1. /usr/tmp
2. /usr/tmp
3. /u01/oracle/TEST/db/tech_st/11.2.0/appsutil/outbound/TEST_machine-ebs-test
4. /usr/tmp
Choose a value which will be set as APPLPTMP value on the target node [1] : 1
Do you want to reset custom variable settings(y/n) [n] :


after the adfgclone.pl completes,  do the following to set the environment in your new server.
- Source the below environment file in the profile file with complete path.
./u01.../../APPSTEST_machine-ebs-test.env
Also set the environment file at the dbTier. 


Post Configuration steps,

1. Updating profile options.
2. Enabling/Disabling Responsibilities
3. Configuring workflow (please be very careful to set the email override address as appropriate, before enabling the workflow services)
4. Other environment specific configurations like enabling OID/SSO, Directory patchs etc.

Cloning process is very much similar to earlier 11i version, except that the $INST_TOP needs some attention until we get used to this type cloning.


Errors/Solutions.
------->
Error 
The report format is:
      <filename>  <phase>  <return code where appropriate>


  [APPLY PHASE]
  AutoConfig could not successfully execute the following scripts:
    Directory:................../db/tech_st/11.2.0/perl/lib/5.8.3 -I /u01/oracle/ECDEV/db/tech_st/11.2.0/perl/lib/site_perl/5.8.3 -I ........................................................../db/tech_st/11.2.0/appsutil/clone
      ouicli.pl               INSTE8_APPLY       255
AutoConfig is exiting with status 1
---->Solution : perl issue.
PATH = /u01/oracle/TEST/db/tech_st/11.2.0/bin/:/usr/kerberos/bin:/usr/local/bin:/home/oraecdev/bin:/home/oraecdev/bin
set the ORACLE_HOME/bin/perl initially at the PATH for this to succeed.

Happy Reading/Sharing folks.


RELATED POSTS::

E-business suite Autoconfig and instance customization.

RMAN Errors during DUPLICATE/Clone database.





















Monday 16 January 2012

ORA-20001,ORA-20202,ORA-06512 errors during R12.1.1 upgrade.

During one of the R12 Upgrades, This below error was encountered and it's not a generic error and very few have encountered this scenario or May be I'm the only unlucky one, which has triggered me to write this blog. I'm sharing this information over this blog, In case some one upgrading to R12 make sure that these settings/requirements are in place. Believe me, the solution is very simple but to get to the root cause and finding a solution during upgrade down time is very very stressful. May be this information would be useful to some one. Wish, no one to get into such situation especially during PROD upgrade.

Background:
Upgrade Iteration 1:
- We cloned 11.5.10.2 environment to a new server.
- Completed functional steps to complete/enable multi org etc.
- Copied over the DB to another machine,
-Converted the DB to 64 bit and upgraded to 11.2.0.3 with Examples.
-Installed R12.1.1 techstack.
-Applied R12 upgrade patch, and then to R12.1.3 RUP.
- OAS patching etc.
- Some issues we encountered, but all went as planned.

Upgrade Iteration 2:
The same steps were followed, went ahead as planned.

PROD Upgrade 3 :
No cloning required, as we are doing this time on PROD.
All the steps were executed successfully until R12.1.1 upgrade patch, the job ran for about 14 hours approximately and then some of the sql's were failing with the below error message.

ERROR at line 1:
ORA-20001: Please specify a valid printer.
Either you have not specified a printer for your report, or the printer name
you have given does not match that in the program definition.
Please specify a printer in the program definition, report set definition, run
reports form or as a profile option.
ORA-06512: at "APPS.HR_UTILITY", line 939
ORA-06512: at line 46

and


SQL> @bomcpbom.sql
declare
*
ERROR at line 1:
ORA-20202: Please specify a valid printer.
Either you have not specified a printer for your report, or the printer name
you have given does not match that in the program definition.
Please specify a printer in the program definition, report set definition, run
reports form or as a profile option.
ORA-06512: at line 38 


Initially, these were the steps taken.


1. Checked the worker log files --> Same error.
2. Tried executing the same sql from the sql> prompt --> Resulted in the same error.
3. No relevant hits in the Metalink/Google.
4. As usual raised an OTN thread, of course Hussein Sawwan was the first and only one to respond.
<I've done some more steps here to check the printer settings at the OS level, tried printing a job to a printer at OS level etc> 
5. Parallely raised an SR. 

After working round the clock in an SEV1 SR,  it was concluded to be a bug. Details are as below.

Bug : 8337047. product version 11.5.10.2.
SR analyst suggested me to update the profile option "Printer"  at the site level to "noprint" value and redo the upgrade from scratch,i.e starting from migrating to 64 bit...

Observation: When I checked with the "Printer" profile option in PROD the vaue was blank, where as in cloned UAT and SIT test environments the value was set to "no print" with Record history as "Autoinstall". 
Created  by "AUTO INSTALL"
Updated by "AUTO INSTALL" 
DATE: Pointed to the date of cloning.

So this gave me confidence or a hope to expect light at the end of tunnel.


Updated the "Print" profile option to "noprint" as suggested and upgrade went smooth with only expected issues/errors.


Happy Reading/Sharing folks.







Thursday 12 January 2012

Configuring SSO using Kerberos/SPNEGO for Weblogic Server for Oracle Fusion middleware products.

Purpose : To Configure Single sign on for Oracle fusion middleware products like SOA, UCM, Web Center spaces etc.
This method of enabling SSO using the Microsoft active directory - Key Distribution center has the following configuration steps. Suggest a detailed analysis of every step marked below before executing them on to your servers.

Summary of Configuration tasks.
·         Collect Environment details for
-          KDC/AD  Machine
-          Windows client
-          Weblogic Server
·         Active Directory Setup.
·         Setting up keytab files.
·         Configuration setup steps to access MS-AD and to check the key tab files.
·         Configure Weblogic server to access KDC.
·         Create and Identity asserter.
·         Enable SSO for UCM.
·         Enable SSO for Web Center spaces.
·         Setups in OHS server.
·         Set up the client IE browser settings.
·         Test the SSO after deploying the required applications.
Environment Details.

a. KDC/AD machine
    OS: Windows 2003 Server
    Domain: DOMAIN.COM
    Machine: win2003 R2.

b. Windows Client
   OS: Windows XP SP3
   Domain: DOMAIN.COM
   Machine: win-xp
   Client: IE 8

                  c. Weblogic Server 10.3.5.0
                      OS: Oracle Enterprise Linux 5 64 bit.
                      Machine: node1 and node2


Active Directory Setup.

Create an user in Active Directory with below details.
User Name : kerberosadm(Name this user anything that suits your situation)
Pw : Password1 ( An example to reiterate the same password is referenced in later sections during password reset)

The user account’s encryption type must be DES and the account must require Kerberos pre-authentication.

Please do not select any other option than to keep the option "use DES encryption types for this account" checked.


Node1 :

ktpass -princ HTTP/node1.domain.com.au@DOMAINABC.COM -pass Password1 -mapuser kerberosadm -pType   KRB5_NT_PRINCIPAL -crypto DES-CBC-CRC -kvno 0 -out node1.HTTP.keytab

Node2:
ktpass -princ HTTP/node2.domain.com.au@DOMAINABC.COM -pass Password1 -mapuser kerberosadm -pType   KRB5_NT_PRINCIPAL -crypto DES-CBC-CRC -kvno 0 -out node2.HTTP.keytab

URL:
ktpass -princ HTTP/fusiontest.domain.com.au@DOMAINABC.COM -pass Password1 -mapuser kerberosadm -pType KRB5_NT_PRINCIPAL -crypto DES-CBC-CRC -kvno 0 -out fusiontest.HTTP.keytab

AD User:

ktpass -princ HTTP/kerberosadm.domain.com.au@DOMAINABC.COM -pass Password1 -mapuser kerberosadm -pType KRB5_NT_PRINCIPAL -crypto DES-CBC-CRC -kvno 0 -out kerberosadm.HTTP.keytab 

Note: Here I'm configuring SSO for 2 node weblogic cluster and F5 load balancer. So the steps may slightly alter depending on your configuration.

è Now there would be four keytab files under the root directory of the user logged into the command prompt.
“C:\Documents and Settings\redrockadm”
1.       Node1.HTTP.keytab
2.       Node2.HTTP.keytab
3.       fusiontest.HTTP.keytab
4.       kerberosadm.HTTP.keytab

Note: This is a very important step, please make sure that the user kerberosadm has four SPNs set, this can be confirmd with the command “setspn –L kerberosadm”
Copy/FTP all the four key tab files to Node1 of the weblogic server machine.

Create krb5.conf file to specify the KDC/AD machine details for the weblogic server connectivity. This file will be present under /etc/krb5.conf, copy this file and edit it as per the requirement mentioned below.
weblogic_domain = /apps/oracle/product/middleware/user_projects/domains/weblogic_domain/

[logging]
 default = FILE:/apps/oracle/product/middleware/user_projects/domains/web_domain/kerb_logs/krb5libs.log
 kdc = FILE:/apps/oracle/product/middleware/user_projects/domains/web_domain/kerb_logs/krb5kdc.log
 admin_server = FILE:/apps/oracle/product/middleware/user_projects/domains/web_domain/kerb_logs/kadmind.log

[libdefaults]
 default_realm = DIOMAINABC.LOCAL
 default_tkt_enctypes = des-cbc-crc
 default_tgs_enctypes = des-cbc-crc
 ticket_lifetime = 600
 forwardable = yes

[realms]
 DOMAINABC.COM = {
  kdc = <AD-server-ip>
  admin_server = <AD-server-Machine>
  default_domain = DOMAINABC.COM
 }

[domain_realm]
.domainabc.local = DOMAINABC.COM
.domainabc.au = DOMAINABC.COM
domainabc = DOMAINABC.COM

[appdefaults]
   autologin = true
   forward = true
   encrypt = true
   forwardable = true

Copy the keytab files generated in step III above to your weblogic sso_domain.


As we have four keytab files we need to merge these files into one, In order to do this please follow below steps.

Ktutil
Ktutil: rkt Node1.HTTP.keytab
Ktutil: rkt Node2.HTTP.keytab
Ktutil: rkt fusiontest.HTTP.keytab
Ktutil: rkt kerberosadm.HTTP.keytab
Ktutil : wkt sso_all.keytab
Ktutil : q

Note : Source the /usr/kerberos/sbin for ktutil command.

Run the following command to test the keytab and generate a cache for the kerberos ticket:

kinit -V -k -t /apps/oracle/product/middleware/user_projects/domains/weblogic_domain/sso_all.keytab HTTP/kerberosadm.domain.com.au@DOMAINABC.COM

The output should be “ Authenticated to Kerberos v5”.

klist command at $ prompt should have the details of the cahced ticket.

This confirms that the handshake between Linux box and the Microsoft AD/KDC is successful.



Create JAAS.login file as below to specify the login modules.


com.sun.security.jgss.krb5.initiate {
     com.sun.security.auth.module.Krb5LoginModule required
     principal="HTTP/domain.com.au@DOMAINDEV.LOCAL"
     keyTab=sso_all.keytab
     useKeyTab=true
     storeKey=true
     debug=true;
};
com.sun.security.jgss.krb5.accept {
     com.sun.security.auth.module.Krb5LoginModule required
     principal="HTTP/domain.com.au@DOMAINDEV.LOCAL"
     keyTab=sso_all.keytab
     useKeyTab=true
     storeKey=true
     debug=true;
};
















So, by now your domain root folder sso_domain should have 3 files,

1.       JAAS.login.
2.       Krb5.conf
3.       *.sso_all.keytab files.

create a file by name “setSSODomainEnv.sh” with below details in it.


##
# Environment file for SSO Configuration.
##
export KRB5_CONFIG=${DOMAIN_HOME}/krb5.conf

JAVA_OPTIONS="${JAVA_OPTIONS} -Dsun.security.krb5.debug=true -Dweblogic.security.enableNegotiate=true -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=${DOMAIN_HOME}/jaas.login -Djava.security.krb5.conf=${KRB5_CONFIG} -Djava.security.krb5.realm=DOMAINDEV.LOCAL -Djava.security.krb5.kdc=135.101.82.36:88 -Djava.security.debug=configfile,configparser,gssloginconfig"

export JAVA_OPTIONS

-where
${JAVA_OPTIONS} are referenced from previous settings in  weblogic server + setting above;
${DOMAIN_HOME} - : /apps/oracle/product/middleware/user_projects/domains/web_domain
${KRB5_CONFIG} : Set above to ${DOMAIN_NAME}/krb5.conf file

Note: Here we are setting up the environment for Single Sign On in weblogic server.

 Reference the above file from “setDomainEnv.sh”, which in turn will be referenced by startweblogic.sh Restart the entire domain and all the managed servers involved.

a.       Start weblogic server console and create an identity asserter with the navigation as provided below.
1.       Login to weblogic admin console.
2.       Click “Security Realms”
3.       Click “My realms”
 









4. Click “Providers”










5. Click “Authentication subtab”.

6.       Click New to create a new “ Identity Asserter” and Reorder using the button provided so that the Newly created Identity asserter is the first in the order as provided in the above screen.
7.       Click on the authentication provider you just created from the list of providers.
8.       Open the configuration tab and Provider Specific  tab and check “Forms Based Negotiation Enabled” .

Deploy an application to the weblogic server, This will be useful in testing the application for SSO.

Deploy a test application to weblogic server similar to “Sample SSO application” under the note : 779272.1


Note: we are currently deploying only two applications 1. UCM and 2. Inbound refinery (IBR).

  • Log in to the Oracle WebLogic Server Administration Console.
  • Click Deployments in the Domain Structure navigation tree.
  • On the Control tab, click Next until you see the Oracle UCM deployment you want to change:
    • Oracle Universal Content Management - Content Server
    • Oracle Universal Content Management - Inbound Refinery
  • Select the checkbox to the left of the deployment to be changed.
  • Click Update.
  • Under the Deployment plan path, select Change Path.
  • Navigate to and select the appropriate plan file:
    • cs-deployment-plan.xml (for Content Server)
    • ibr-deployment-plan.xml (for Inbound Refinery)
  • Verify that Redeploy this application using the following deployment files is selected.
  • Click Next.
  • Click Finish.
Use the application “cs-deployment-plan.xml” after FTPing it onto the server.

<?xml version='1.0' encoding='UTF-8'?>
<deployment-plan xmlns="http://xmlns.oracle.com/weblogic/deployment-plan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation= "http://xmlns.oracle.com/weblogic/deployment-plan http://xmlns.oracle.com/weblogic/deployment-plan/1.0/deployment-plan.xsd" global-variables="false">
  <application-name>ibr.ear</application-name>
  <variable-definition>
   <variable>
      <name>http-only</name>
      <value>false</value>
    </variable>
  </variable-definition>
  <module-override>
    <module-name>ibr.war</module-name>
    <module-type>war</module-type>
    <module-descriptor external="false">
      <root-element>weblogic-web-app</root-element>
      <uri>WEB-INF/weblogic.xml</uri>
      <variable-assignment>
        <name>http-only</name>
        <xpath>/weblogic-web-app/session-descriptor/cookie-http-only</xpath>
      </variable-assignment>
    </module-descriptor>
  </module-override>
</deployment-plan>


Enable SSO for Web Centre Spaces.


Once you have completed the steps for configuring the Negotiate Identity Assertion Provider and Active Directory Authenticator, and all applications on your WebLogic domain are configured for single sign-on with Microsoft clients in the required domain, a final step is required to provide a seamless single-sign-on experience for your users when accessing WebCenter Spaces. There are two options for doing this:

•Turn off public access, by logging in to WebCenter Spaces as an administrator and removing View access from the Public-User role. When public access is turned off, accessing the URL http://host:port/webcenter takes the user directly to the authenticated view rather than the default public page which has a login section. This is recommended when users are accessing WebCenter Spaces only using Internet Explorer, and are confined to the domain where WNA is set up.

•If you must retain public access to WebCenter Spaces, then the recommendation is to use the oracle.webcenter.osso.enabled flag when starting the WLS_Spaces server. This flag tells WebCenter Spaces that SSO is being used and no login form should be displayed on the default landing page. A Login link is displayed instead that the user can click to invoke the SSO authentication where the user will be automatically logged in. If Firefox is used to access WebCenter Spaces within the Windows network configured for WNA, or any browser is used to access WebCenter Spaces from outside the Windows network domain, users see the login page after clicking the Login link.

Note : Contents here are references from one of the oracle sites. Please use them for referential purposes only.



Local Intranet Domains.
  • In the IE  Tools à Internet Options.
  • Select the security tab.
  • Select local intranet and click sites
In the local intranet popup, ensure that the “Include all sites that bypass the proxy server" and "Include all local (intranet) sites not listed in other zones" options are checked.
  • Click Advanced.
In the local intranet(Advanced) dialog box, add all the relative domain names that will be used for weblogic server instances participating in the SSO configuration ex: myhost.example.com and click OK.
Configure Intranet Authentication
  • Tools àInternet options.
  • Security tab
  • Local intranet and click custom level.
  • In the security settings dialog box, scroll to the user authentication section.
  • Select Automatic logon only in Intranet zone (This option prevents users from having to re-enter the login credentials, which is a key piece to this solution.
  • Click OK.
Verify the Proxy Settings.(Only if the proxy server is enabled).
  • Tools à Internet options.
  • LAN Settings.
  • Verify the proxy server address and port numbers are correct.
exceptions
  • Click Advanced.
  • In the Proxy Settings dialog box, ensure that all desired domain names are entered in the Exceptions field.
  • Click OK.
Set Integrated Authentication for Internet Explorer.
  • Tools àInternet options.
  • Advanced.
  • Scroll to the security section
  • Verify that the Enable Integrated Windows Authentication option is checked. If not, check it, click Ok and restart the computer.
  • click OK.

Please log in and test your SSO using any of the deployed applications. 

References:

http://docs.oracle.com/cd/E24329_01/web.1211/e24422/sso.htm#i1106670
http://technet.microsoft.com/en-us/library/cc780469(WS.10).aspx
Please let me know if we can any way improve the contents of this blog, I'll be more than willing, with a comment referring you name on it.

Happy Reading/Happy sharing.

Errors/Solutions
kinit(v5): Cannot find KDC for requested realm while getting initial credentials
Soln:setting the environment variable KRB5_CONFIG=<absolute path of the krb5.conf file> fixes this issue

kinit(v5): Cannot contact any KDC for realm <DOMIAN NAME> while getting initial credentials.

 I had assumed that since I was just working through the local host it wouldn't need it, but it turns out you need to associate a real network address with the realm name and to make sure those ports are available. so in /etc/hosts, I had to have this line:
<ip address> example.com
Solution 2:
Check the firewall settings for getting the tickets from KDC.
Solution 3: In my case the issue was resolved by choosing port :88.

Setting up WINDOWS7 for DES kerberos authentication:
WINDOWS-7, by default does not support the DES(Data Encryption standards) encryption as that's older than AES(Advanced Encryption standards).
So please carry out the following configuration options at the WINDOWS-7 client machine in order for the kerberos authentication to work.


Go to Local Security Policies(By typing “Local Security Policies” in run dialog)->Local Policies->Security Options->Network security: Configure encryption types allowed for Kerberos:
Here select checkboxes against DES_CBC_CRC, DES_CBC_MD5 and RC4_HMAC_MD5.

I would update with some more relevant error messages and solution that we encountered during implementation,
to be continued....