Oracle Application Cloning (11i)

Cloning is the process of creating an identical copy of the oracle application system.There are two types of cloning methods available for Oracle application one is adclone and another one is rapid clone.
Ad clone is an oracle provided utility to clone application system. This utility is used to clone application system for release 11.5.1 to 11.5.5 for systems which are not autoconfig  enabled, Whereas rapidclone is used for those which are autoconfig enabled.
We can find our application is autoconfig enabled or not by opening env files from APPL_TOP APPLSYS.env and APPSORA.env. If the top of the file says that it is maintained by autoconfig, then your system probably using auotconfig.

What happens when you run adpreclone.pl?
adpreclone.pl prepares source system to be cloned by collecting information about the database and creates various templates of files containing source specific hard core values.It collects all the information about the source application system, the port number of the source etc.these templates are stored in the appsutil/templae directory.

Different kinds of Cloning.

1)Single node to multi node clone

Sorce Node-PROD                                                                  Tartget-Dev1

DB Tier                                                                                    DB Tier 
prodb                                                                                       proddb
proddata                                                                                   proddata

                                                                                                Tartget-Dev2
Apps Tier
prodcomn                                                                                Apps tier
prodappl                                                                                  prodcomn,prodappl,prodora
prodora

Some of the reasons to do a cloning are:

• To create a test environment from an existing production environment to test some patches or to reproduce any production issues.
• To keep a test environment with the most current information of a production environment.
•  To move any existing environment to other servers

Cloning prerequisite steps:

Make sure target servers have same component verson and OS as source servers.

• Login as Applications file user & set the environment file on source node.

           -bash-3.2$ su - applmgr
           -bash-3.2$ cd /maildata_test/mcgm/R12/apptier/apps/apps_st/appl
          -bash-3.2$. APPSrdb_mailserver2.env

• Login to database tier as oracle user and set the environment on source node.

            -bash-3.2$ su - oracle
            -bash-3.2$ cd /maildata_test/mcgm/R12/dbtier/db/tech_st/11.1.0/
            -bash-3.2$ . rdb_mailserver2.env

Prepare the source server

1.      Prepare the source system dbtier for cloning 

           -bash-3.2$ cd /maildata_test/mcgm/R12/dbtier/db/tech_st/11.1.0/appsutil/scripts/rdb_mailserver2/
            -bash-3.2$.perl /adpreclone.pl dbTier

     2       Prepare the source system application tier for cloning
               -bash-3.2$  cd $COMMON_TOP/admin/scripts
                                in R12(cd $ADMIN_SCRIPTS_HOME)
              -bash-3.2$cd  /maildata_test/mcgm/R12/apptier/apps/rdb_mailserver2/admin/scripts(R12)
              -bash-3.2$perl adpreclone.pl appsTier

Copy the Source Node File System

1. Log on to the source system application tier nodes as the APPLMGR user
2. Shut down the application tier server processes as shown below

             -bash-3.2$cd $COMMON_TOP/admin/scripts/rdb_mailserver2(11i)
             -bash-3.2$cd  /maildata_test/mcgm/R12/apptier/apps/rdb_mailserver2/admin/scripts(R12)
              -bash-3.2$./adstpall.sh apps/apps

Copy the following application tier directories from the source node to the target application tier node:
copy appltop and commontop and oratop to target servers
scp –pr appl_top applmgr@target_server:/d01/oracle
scp –pr comn_top applmgr@target_server:/d01/oracle
scp –pr ora_top applmgr@target_server:/d01/oracle
Once copied –

Check the ownership as required

Rename the directory on target server
cd /d01/oracle
mv appl_top devappl
mv comn_top devcomn
mv ora_top devora


Copy the database tier file system

stop all database services
Perform a normal shutdown of the source system database
-bash-3.2$cd $RDBMS_ORACLE_HOME/appsutil/scripts/rdb_mailserver2
-bash-3.2$./addbctl.sh stop
Copy the database (DBF) files from the source to the target system
Copy the source database ORACLE_HOME to the target system as shown below:
cd /d01/oracle
scp –pr proaddb oracle@target_server:/d01/oracle
scp –pr proaddata oracle@target_server:/d01/oracle
Once copied –
- Check the ownership as required
- Rename the directory
cd /d01/oracle
mv proddb devdb
mv proddata devdata
Start up the source Applications system database and application tier processes

Configure the Target System

Operating System of Target should be same as Source. Operating system should have all pre-requisite packages required for Oracle R11i before configuring the Target System. Execute the following commands to configure the target system. You will be prompted for the target system specific values (SID, Paths, Ports, etc).
Log on to target node as oracle user and run the following command and input your values to each prompt as shown below
-bash-3.2$ cd /maildata_test/mcgm/R12/dbtier/db/tech_st/11.1.0/appsutil/clone/bin
-bash-3.2$ perl adcfgclone.pl dbTier
Enter APPS Password :
 Configure the target system application tier server nodes
Log on to the target system as the APPLMGR user and type the following commands and
specify your values to each prompt as shown below.
-bash-3.2$ cd /maildata_test/mcgm/R12/apptier/apps/apps_st/comn/clone/bin
$ perl adcfgclone.pl appsTier
Enter the APPS password
All application services are started successfully. Thatmeans, we have done cloning successfully.
Finishing Tasks
Post clone steps vary from client to client, here is the basic change.
Profile Option Name Changes at Site Level after Cloning
Site Name-> PROD, Change it to “DEV – Clone of PROD as of Dec 14 2010”
DEV – Clone of PROD as of 14-Dec-10
Login to Oracle Apps as SYSADMIN
Select System Administrator responsibility


Multinode Sharedappltop to same architechture

If you having shared APPL_TOP then apart from running the adcfgclone.pl you need to run "adclonectx.pl sharedappltop" for sharing appltop from all nodes. In this case you need to run adcfgclone.pl only form the first node and you can create the context file using adclonectx.pl in  other nodes using context file refernece as first node. Then you need to run the txkSOHM.pl from $FND_TOP/patch/115/bin which will create 8.0.6 and IAS config homes for your application system.

Multinode to single node(multinode appltop to single appltop)

1)Maintain snapshot information

Log in as the applications user on each application tier node and run ‘Maintain Snapshot Information’ by using adadmin.

2) Merge existing APPL_TOPs
Log in to the primary node of your application tier as the application user user and run:
$ cd [COMMON_TOP]/admin/scripts/[CONTEXT_NAME]
$ perl adpreclone.pl appsTier merge

This will prompt you with option to merge secondary nodes

Now log in as the applications user to each of the secondary application tier nodes being merged and run:

$ cd [COMMON_TOP]/admin/scripts/[CONTEXT_NAME]
$ perl adpreclone.pl appltop merge

4) Prepare the source system database tier.

Log on to the database tier of the source system as the oracle user and run the following commands:

$ cd [RDBMS ORACLE_HOME]/appsutil/scripts/[CONTEXT_NAME]
$ perl adpreclone.pl dbTier

With this all the pre clone tasks on the source system have been completed.

Steps to be performed on target system

5)Create OS user accounts

Create a OS user account for your applications

$ useradd -g dba -d [home_directory] -p password username

Similarly create a OS user account for your database

copy the required tops from 

Copy the following application tier directories from the primary node of your source system to the target application tier node, retaining the original directory structure:

– [APPL_TOP]
– [OA_HTML]
– [OA_JAVA]
– [COMMON_TOP/util]
– [COMMON_TOP/clone>
– [806 ORACLE_HOME]
– [iAS ORACLE_HOME]  

6. Copy the required files for merging

Log in as the applications user to each of the secondary source nodes and recursively copy:

directory [COMMON_TOP]/clone/appl
- to -
directory [COMMMON_TOP]/clone/appl on the target system node

Before proceeding with the next steps you must shutdown your oracle applications services and the database on the source system

7. Copy the database tier file system

Log on to the source system as the database user

Copy the database (DBF) files from the source to the target system
Copy the source database ORACLE_HOME to the target system

After this you can now startup the database and applications services on your source system and release it for use.

8. Configure the target system database server

Log on to the target system as the database user and type the following commands to configure and start the database:

$ cd [RDBMS ORACLE_HOME]/appsutil/clone/bin
$ perl adcfgclone.pl dbTier

This will prompt for new ORACLE_SID,ORACLE_HOME,Port Pool,JAVA_TOP and DATA_TOP give the appropriate values matching your target system

Once successful this should start your database and listener

9. Configure the application tier server nodes

The database and its listener should remain up before executing the next set of commands.
Log in to the merged APPL_TOP node as the applications user and execute the following commands:

$ cd [COMMON_TOP]/clone/bin
$ perl adcfgclone.pl appsTier

This will prompt you new port pool for applictaion tier services as well as new APPLTOP,COMMON_TOP,ORACLE_HOME and IAS_TOP

Successful completion of this task will bring up your application tier services on the target or the cloned node.

Post-cloning Tasks

Log in to the target system application tier node as the APPLMGR user.
Run the following tasks in adadmin for all products:
- generate JAR files
- generate message files
- relink executables
- copy files to destination

- Remove the temporary directory [COMMON_TOP]/clone/appl to reduce disk space usage.

Note: Rapid clone takes care of the site level profile options all other profile options need to be manually updated.

Unified APPL_TOP in R12

Unified APPL_TOP is different from separate APPL_TOP in Oracle Applications 11i. Unified APPL_TOP make more sense if you are using multi node Oracle Applications R12.

In Oracle Applications 11i , in multi node installation each APPL_TOP have its different name and files in each APPL_TOP will depend on type of Node (i.e. Forms Node will have fmx or forms related files where as APPL_TOP belonging to CM only node will have rdf or files required to run CM node)

Starting from R12, it will use Unified APPL_TOP which means all files required for all middle tier services are included in all nodes of Multi Node installation (irrespective of services running on that node)

What are few changes because of Unified APPL_TOP?

During Cloning of multi node Oracle Application install, only one copy (any one appl_top) of Application Node files need to copied to target instance.

During Cloning of Multi Node to Single Node you don’t have to merge APPL_TOP’s in R12 as required in 11i .

Services start/stop If you are using adstrtall.sh to start services on a node then it will start services which were configured to start during install on that node (You can still start a specific service even though it was not suppose to configure/start by calling startup script of that specific services).
To explain this better, lets assume you installed multi node R12 instance with Node 1 as Forms & Web Server where as Node 2 was installed with Concurrent Manager. Now on Node1 when you use adstrtall.sh script to start services as expected it will start Forms and Web services but unlike 11i, You can still start Concurrent Manager on Node1 by running adcmctl.sh
–FND_NODES From R12 all nodes will have Y against all services (For multi-node) in FND_NODES table.

Adutilities in Oracle Apps

QWhat are the Ad utilities?
Ad utilities stand for application DBA utilities. These are set of tools used for installing, upgrading, maintaining and patching Oracle application. There are around 15 Ad utilities.

QWhat is maintenance mode and how it enable/disable the same?
Maintenance mode is a feature which is introduced in 11.5.10. For doing any maintenance activity in the application system like applying patch the maintenance mode need to be enable in the application system.It can be disable and enable using adadmin utility .

QWhat is the log file location for Adutilities?
$APPL_TOP/admin/$TWO_TASK/log. format <adutility name>.log

QWhat is adadmin why it is used?

Adadmin is used to perform a number of administrative tasks to maintain the Oracle Applications.Adadmin ensure that Oracle Application run smoothly.
Adadmin perform two types of works -one which is performed at database level and other which is performed at the file system level.
Adadmin can be scheduled to run at a later time in a non-interactive mode.

QHow Adadmin run in a non-interactive mode?

To run Adadmin in a non-interactive mode, you must at first create defaults file. Once the defaults file is created you can run adadmin in non-interactive mode using this file.The defaults file must be located under
APPL_TOP/admin/SID
$adadmin defaultsfile=APPL_TOP/admin/SID/defaults.txt
$adadmin defaultsfile=APPL_TOP/admin/SID/defaults.txt
 logfile-adadamin.log
workers=10
interactive=no
menu_option=DISABLE_MAINT_MODE
In order to choose which task defaults file run you add menu option.

QHow can you regenerate all the forms files using adadmin and what does it do internally?
The generate forms files options of adadmin take  care of generating form files(extension .fmx) from binary forms defination file (extension .fmb). These binary form files are located under AU_TOP and the executable files are stored under each products directory.

QYou have accidentally deleted the environment file how can u recreate it?
You can recreate the environment file by using the options create applications file from adadmin.This options create environment file that your system configuration.

QWhat is the significance of the DUAL table? Who owns this table and how many rows it should have?
The DUAL table is created automatically by Oracle along with the data dictionary. It is located in the schema of SYS. It has one column named DUMMY of type VARCHAR2 and contain one row with the value of 'X'. In case it has more than one row application system may not function properly.

QWhat is ad splice? what does it do?
Oracle often release new products after the base release of Oracle applications. This products are known as off cycle products. Adsplice is a utility which take care of incorporating and off cycle products in to Oracle applications so that it is recognized by the ad utilities as a valid oracle application product.

QWhat are the advantages of distributed AD?
With distributed AD the workers can utilize the additional resources of the remote nodes where they are running apart form the primary node. This capability improves scalability, performance and resource utilization and completion of worker in a faster time.

Distributed AD

Distributed AD

Distributed AD is a parallel processing feature that can reduce downtime by efficiently utilizing all the available resources on a shared application file system. AD Administration and Auto Patch run on the primary node and direct workers running on that node and other nodes in the system. The AD Controller utility controls and monitors the actions of the workers that you specify.
This parallel processing makes use of managers, which direct the actions of worker processes. The manager assigns each worker a processing job and monitors its progress. When a worker completes a job, the manager assigns it another until all jobs are complete. AD Administration and AutoPatch can be directed to distribute processing tasks across multiple remote machines in a multi-node system. This type of parallel processing operation is called Distributed AD. It further reduces the time to complete a maintenance task by utilizing the processing capabilities of all of the nodes in the system.
Distributed AD can only be used on systems that are using a Shared application Tier File System to ensure that files are maintained in a single, centralized location.

Using Distributed AD:

Start AutoPatch or AD Controller with Distributed AD worker options
On one of your shared application tier file system nodes, start your AutoPatch or AD Administration session with the following command line options:

localworkers= workers=For example, to run an AutoPatch session with a total of eight workers (three workers on the local node and five workers on a remote node):

$ adpatch workers=8 localworkers=3 Start AD Controller on remote node(s)
On each of the additional shared application tier file system nodes, start an AD Controller session with the additional distributed command line option:

$ adctrl distributed=yAfter providing basic information, AD Controller will prompt for the worker number(s) to be started. For example, to start workers 4 through 8 on a second node, enter “4 5 6 7 8″ or “4-8″.

The following is an example of starting a three-node session with a total of five workers:

Node 1:
$ adpatch localworkers=3 workers=5Node 2:
$ adctrl distributed=y
Enter the worker range: 4Node 3:
$ adctrl distributed=y
Enter the worker range: 5

During execution of AutoPatch or AD Administration, you can start a normal AD Controller session (without distributed=y) from any of the nodes in the shared application tier file system environment to perform any of the standard AD Controller operations. All of the standard AD Controller options have the same effect on both local and non-local workers, with the following exception:

option 6: Tell manager to start a worker that has shutdown this option will always result in a worker being started on the same node that the AutoPatch or AD Administration utility is running on. This means that if an AutoPatch worker exited on a distributed node, choosing this option will start the worker on the node that is currently running AutoPatch, rather than the node that was originally running the worker.

AD Controller Log Files

The log file created by AD Controller is created on whichever node the AD Controller session is started. This is to prevent file locking issues on certain platforms. It is therefore recommended that the AD Controller log file include the name of the node from which the AD Controller session is being invoked.
Managing Distributed Workers and Nodes

If a worker has exited on a distributed node 
Make sure that the worker is not running. 
Start AD Controller on the same distributed node using: 
$ adctrl distributed=y force_restart=yEnter the worker number(s) to start when prompted. AD Controller will start those workers and then exit. 

AdConfig in Oracle Apps

Autoconfig is an 11i configuration tool (set of java classes run from perl/shell scripts) introduced in 11.5.4, which eliminates the need to manually maintain these configuration files and profile options. Autoconfig maintains the environment specific values for the 11i environment in an xml file (stored in $APPL_TOP/admin). When autoconfig run it overwrites the existing configuration files with new ones that it creates by merging the templates and the context file, it also updates the database by running the script it instantiates (merging a template with the values in the context file is termed as instantiating the template in Oracle parlance) from the script templates.

That’s essentially what autoconfig does i.e. Instantiating configuration files from templates and updating database with values from the xml file. It does not move directories, check for availability of ports, update DNS entries and so on. These have to be done manually.

How Auto config works..

Each configuration file has one corresponding template file (provided by autoconfig patches, stored in $PROD_TOP/admin/template directory). Profile options and other instance specific information in the database is maintained by many sql scripts, called from shell/perl scripts. These scripts also have corresponding template files (also provided by autoconfig patches, in $PROD_TOP/admin/templates).In these template files all the environment specific values are replaced by placeholders.

Configuration files can be easily created from the template file by just replacing all the placeholders with the corresponding values from the context file. This process is termed as “instantiating the template” in Oracle documentation. 

Driver files (stored in $PROD_TOP/admin/driver) store information about what to do with each template (e.g. instantiate it and replace the existing configuration file with it, instantiate it and run it). These files are named as <PROD_TOP>tmpl.drv (e.g. adtmpl.drv, fndtmpl.drv etc.) They contain one line for each template they manage.

When autoconfig (adconfig.pl or adconfig.sh) runs it just processes the driver file for each product, line by line doing what the driver file instructs it to do. The order of execution of each line is not sequential as we shall see later

How to run AutoConfig:

Step 1 : Stop all services 

$ $COMMON_TOP/admin/scripts/adstpall.sh APPS/ APPSPASSWORD

Step 2:Run the autoconfig script, $adautocfg.sh & respond to the prompts. 

Step 3:Check log files for any errors.log file for autoconfig is adconfig.log & is 

located in /admin//log/MMDDhhmm/ 

Step 4: Restart all services : 

$ $COMMON_TOP/admin/scripts/adstrtal.sh APPS/APPSPASSWORD

Roll back autoconfig :

Each execution of AutoConfig creates a roll back script in case we want to revert 

back to the previous configuration settings. For this we perform the following 

steps:

Step 1 :Shut down all services: 

$ cd $COMMON_TOP/admin/scripts/ 

$ adstpall.sh APPS/ 

Step 2:Restore AutoConfig session. 

$ cd $APPL_TOP>/admin//out/MMDDhhmm 

$ restore.sh 

Step 3 : Restart all services: 

$ $COMMON_TOP/admin/scripts//adstrtal.sh APPS/

Restoring the previous configuration 

Log file locations:(11i)

Dbtier

$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/MM_DD_YYYY/adconfig.log

Appstier:

$APPL_TOP/admin/$CONTEXT_NAME/log/MMDDYYYY/adconfig.log

R12:
Appstier
$INST_TOP/apps/$CONTEXT_NAME/admin/log/$MMDDHHMM/adconfig.log
Dbtier
$ORACLE_HOME/appsutil/log/$CONTEXT_NAME/<MMDDHHMM>/adconfig.log

Some important Question about Concurrent manager(11i)

Q.In administer concurrent manager form there are two columns target and actual. What are these columns.

  Target columns lists the number of processes that are running for each manager for a particular work shift.

Actual column that is currently how many process are running. If actual column is zero then there are no process are running for that particular manager.

If the Target column is zero then there is no work shift assigned for that manager or the current work-shift does not specify any target processes. If the target column not zero then the manger goes down or manager processes have either failed.

Q.How do i run/schedule a concurrent request from operating system level without logging into the applications.?

A Concurrent request can be scheduled/ru from the operating system using the CONCSUB utility.CONCSUB means concurrent submit.

QWhat are the different ways to check Concurrent manger is running or not?

 By using FNDLIBR fom os level, from the forms it can be check from navigatio>concurrent>Manager>Administer. It can be also checked by using script adcmctl.sh status finally it can be checked from OAM also.

QWaht is the default location of concurrent manager?

If environment variable set
 $APPLCSF/$APPLLOG
$APPL_TOP/admin/SID/log(11i)
$/maildata_test/mcgm/R12/apptier/apps/rdb_mailserver2/logs/appl/conc/log   (R12)
If environment variable not set
$FND_TOP/$APPLLOG

Q i have submitted a request and its showing the status inactive no manager

If the concurrent manager is up and running and the request goes to inactive no manager  for some time it means the cache cycle is less. Cache size set on the concurrent >manager> Define form..
Basically this regulates how many requests a manager will pick up for each sleep cycle.the solution is either increase the sleep cycle or increase the number of manager process. The manager should be either slandered manager or any other manager for which the issue is coming.

Q I have submitted a request  it has gone to pending standby status for a long time whereas other request are getting completed normally with out any issues. what could be the reason.?

 If any particular request is going pending standby status and other request are getting completed it means that either is waiting for other request output or it is conflicting with some other request. If the request is conflicting check the  queue of the conflict resolution manager for troubleshooting.

Q How do i process more concurrent request in parallel?

If you want to process more requests simultaneously there are two way by increasing target process for the manager and other one increasing cache size.


Q When do the tables FND_CONCURRENT_REQUESTS and FND_CONCURRENT_PROCESS need to be purged?

when the tables reached 20000 rows the performance being to diminish. You may want to run purge concurrent request on a regular basis depending on the amount of request being run.

Q What if the internal concurrent manager dies? All the managers also killed immediately after it?

No if the ICM dies the request continue to run normally except queue control requests.

Q Does the ICM run or schedule any request for itself?

No the ICM does not run or schedule any request. It has nothing to do with scheduling requests, or deciding which manager will run a particular request. Its function is only to run queue control requests which are requests to start up or shutdown other manager.It is responsible for start-up and shutdown of the whole concurrent processing facility and it also monitors the other managers periodically and restart them when they goes down. It can also take over the conflict resolution manager  job and resolve incompatibilities. 

Q If the ICM goes down do i need to kill all other managers before starting ICM?

No, If the ICM goes down no need to kill all other managers just simply start ICM

Q Can I delete Concurrent Manager?
Yes you can delete any concurrent manager for deleting , query for the manager in the defined concurrent manager form and delete the row.
Deleting the predefined  manager not recommended and it should never done.

Qwhat is internal monitor?

This manager is used to implement parllel concurrent processing . It monitors whether the ICM is still running and if ICM crashes it will restart it on another node.


QHow do i clean out concurrent manager tables?
For this Oracle provide a script called cmclean.sql.

QI hit the restart button to start the standard manager but it still not started?
Asking a manager to restart sets the status to restart. The ICM will start in the next process monitor session or next time ICM starts. Use activate to start the manager immediately. Also when a manager deactivated manually the ICM will not start it you need to set it to restart or activate it manually.

Q What are the circumstances in which you need to bounce the concurrent manager?

• when you modify the definition of the printer
• when  you modify the environment variables say you have changed the APPLTMP and APPLPTMP variable
• when all the requests are pending or hanging

Q What are the reasons a CM (concurrent manager) hangs?

• Long running jobs
• The Internal manager was activated by some one other than owner of the application system
• The OS file system is full.
• Its not able to create log files
• You've down the Internal Manager but actual has number on it.
• The database hanging may be because of archive logs full
• Pending/standby requests are too many

QThe Different parameter of startmgr utility

Parameters       Description                                                                         Default

sysmgr            sqlplus sername/pwd that owns the foundation tables           Applsys/<pwd>

Mgrname        the name of the manager                                                      Internel Manager

Log file           the logfile of the manager                                                     $FND_TOP/$APPLLOG

                                                                                                                                or

                                                                                                                  $APPLCSF/$APPLLOG

Sleep              the number of seconds the ICM should wait

                       before checking for the new request from the table                         60sec

                       FND_CONCURRENT_REQUESTS                                    

Restart            If the CM goes down abnormally it will automatically
                       restart the manager Y=the  number of minutes the ICM           N=no to restart after abnormal
                       waits before restarting the manager                                         termination

Mailto           MAILTO is a list of users who should receive mail whenever      Current user
                     Manager terminates

Printer           The default printer for sending the output files

Diag               This is used for diagnosis If the CM is started with parameter              N
                      diag=y then full diagnostic output is produced in the log file.

Pmon               The number of sleep cycle ICM will wait before checking                20
                        Failed manager

Quesiz             Number of pmon cycle the ICM waits between times it checks for      1
                        normal changes in concurrent manager operation


Q. What exactly happens when a concurrent request is submitted?
   
     Once a concurrent request is submitted by the user, the table FND_CONCURRENT_REQUEST is automatically updated with the details of the request. The table also updated with the information about schedule of the concurrent whether its immediately scheduled or scheduled at a fixed time. Once the request is scheduled to run the concurrent manager checks the FND_CONCURRENT_REQUEST table to find the request is incompatible with any other request . If the request is incompatible then the Conflict resolution manager take care of the request and find out what are the incompatibilities and resolve them. If there are incompatibilities then it checks for any other special manager assigned to that particular request. If there are any special manager to take care of this request then it goes to the queue of that manager else the standard manager takes care of the same.Once the requset is processed FND_CONCURRENT_REQUEST updated with that status.

Q.In the administer concurrent manager form, what is the significance of the terminate button?

  The terminate button used to terminate any concurrent manager. When you terminate internal manager all the manager automatically ge deactivated and all the running request are terminated. If you want to terminate a particular manager, select that manager and click terminate. The status of the manager changes to deactivate after a few seconds and all the requests are processed by that manager are immediately terminated. Once the manager terminated it does not starts  automatically. You have to manually restart it using the restart button.

Q In administer concurrent manager form what is the significance of the deactivate button and how can you deactivate a manager from there?

For deactivating a particular manager select that manager and press the deactivate button. In case of deactivation, all the requests are processed before shutdown manger does shutdown. If the internal manager goes deactivated all other manager goes deactivated and all requests are processed before manger going down.This is the main difference between terminate and deactivation. In termination all running request are terminated immediately whereas in case of deactivation all the running requests are allowed to complete first.

QIn administer concurrent manager form, what is the significance of the verify button and for which manager its available?

 The verify button enable for only internal manager. One of the function of the internal manager is to monitor the processes of the each concurrent manager. The process of monitoring the other manager process by IM is known as a PMON cycle.When you click the verify button you can force the process monitoring or the PMON activity to occur.The verify button also available for conflict resolution manager.

Q What is parallel processing and what is the significance of the same?
 
   Parallel concurrent processing is the way to distribute concurrent manager across multiple nodes in a cluster, massively parallel or networked environment. It helps in distributing the load across multiple nodes thereby fully utilizing the hardware resource

In case of parallel concurrent processing all the managers are assigned a primary and secondary node. The managers are started on primary node by default. In case of failure or Oracle instance failure all the concurrent manager get into secondary node. If the primary node available all concurrent manger get back to the primary node.During that migration process a manger spread across on both nodes .
In case of parallel concurrent processing it may happen that in a node where parallel co current processing is configured, the Oracle instance may or may not be running. The node which s not running Oracle, the concurrent manager connect via Net8 to a node where is running oracle.

The internal concurrent manager can  run on any node and can active and reactive concurrent managers on all nodes. Since the internal concurrent managers must be active at all times, it needs high fault tolerance. To provide this fault tolerance parallel concuncurent processing uses internal monitor processes. The job of Internal monitor is to monitor constantly the internal manager and start it when it is fails.Only one monitor process can be active on a single node. You have to decide which nodes have an internal monitor process when you configured system. You can also assign each internal monitor process a primary and a secondary node to ensure fail over protection. Internal monitor process like concurrent manager assigned a work shifts and are activated and deactivated by the internal concurrent manager.

Q I have defined  for nodes of the concurrent manager Now do i need to start the concurrent manager from all the nodes?
No, even if you have defined the concurrent manager in four different nodes you need not start the concurrent manager from all the nodes. You just need to start the concurrent on primary node GSM will take care of starting the concurrent manager form all other nodes.

QThe user comes to you saying that the request are taking a lot of time to complete. what is the debugging approach ?

• You can run trace on the request id to find the expensive sqls and then tell the developer to fix the same.
• you can check the program incompatibilities in the concurrent request
• you can check the query which the concurrent program is executing and see if it is creating nay locks in the database 
• many times the users schedule the request to run at a later time.
• You cab check the parameters with which the request is run.

Q Waht are the things take care when you defining a concurrent program.

• selecting an executable file to run the program
• Choosing the execution method for program
• Defining the parameter for the program
• defining the printing information

Oracle Concurrent Manager

The concurrent managers in the Oracle e-Business suite serve several important administrative functions. Foremost, the concurrent managers ensure that the applications are not overwhelmed with requests, and the second areas of functions are the management of batch processing and report generation.

User interaction with oracle application data can be conducted using with HTML application or more traditional form based application. How ever there are also reporting programs and data updating programs that runs periodically or an ad-hoc basis.

These programs will run in background to user continue there work on other tasks. Concurrent processing is an oracle application future that allows these non-interactive and potentially long running functions to be continued. A process that runs on concurrent processing server is call concurrent request.

When you submit a request through the html or form based application a row is inserted into the database table specifying a program to be run. Then concurrent manager reads the request from the table and starts the associate concurrent program. Acting as job scheduling and execution system concurrent manager will run in its own operating system process this operating system is call target process(often called workers) each worker can start one concurrent program at a time.

Whats a Concurrent Program
A program that implements a business functionality and needs to be executed again and again at regular interval or as per business needs is called concurrent program. They can be implement in PL/SQL, Shell Script, C/C++ etc..

Concurrent Manager
Now when a concurrent program is written, it needs to be executed daily at particular time. If we do it manual, there might be chances of delays or it might happen that two different people run the same program at the same time which might lead problems. So we need a manager which can do all this tasks for us. might lead problems. So we need a manager which can do all this tasks for us. Manager, which ensures that each concurrent program can run successfully without any conflicts. They also ensures that the applications are not overwhelmed with requests. They also manages the batch processing and report generation.
The default installation of Oracle Applications comes with a number of pre defined concurrent managers however you can create your custom concurrent managers to spread out the load of your job processing. Apart from taking care of the load of your jobs the concurrent managers can also schedule the jobs periodically. Also we can assign specific priority and specific times to the different programs, so that the concurrent managers can run them in specific workshifts.
Concurrent managers also allows you to tweak the number of concurrent process that it can handle concurrently. If any request exceed this prescribed limit they are automatically put on pending state. The processing of a request takes place based on the time of request submission and priority of the request submitted.

The Oracle e-Business suite has three important master Concurrent Managers:

Internal Concurrent Manager:

The master manager is called the Internal Concurrent Manager (ICM) because it controls the behavior of all of the other managers, and because the ICM is the boss, it must be running before any other managers can be activated. The main functions of the ICM are to start up and shutdown the individual concurrent managers, and reset the other managers after one them has a failure.

Standard Manager
Another important master Concurrent Manager is called the Standard Manager (SM). The SM functions to run any reports and batch jobs that have not been defined to run in any specific product manager. Examples of specific concurrent managers include the Inventory Manager, CRP Inquiry Manager, and the Receivables Tax Manager.

Conflict Resolution Manager
The Conflict Resolution Manager (CRM) functions to check concurrent program definitions for incompatibility rules. However, the ICM can be configured to take over the CRM's job to resolve incompatibilities.

When a concurrent program is incompatible with another program, the two programs cannot access or update the same data simultaneously.When you define a concurrent program, you can list those programs you want it to be incompatible with. You can also list the program as incompatible with itself, which means that two instances of the program cannot run simultaneously. You can also make a program incompatible with all other concurrent programs by defining the program to be run-alone.
There are two types of program incompatibilities, "Global" incompatibilities, and "Domain-specific" incompatibilities.
Incompatibility rules is used to govern your request set so that the request set does not run at the same time as other reports or concurrent programs.When you list a program as incompatible with your request set, the program will not run simultaneously within the same conflict domain as the request set or any of the reports within the set.

Transaction Manager
Apart from these three concurrent manages there is another type of concurrent manager known as the Transaction Manager also exists. The transaction manager is responsible for taking the load off the concurrent request table for pooling the request submitted by the user.The transaction manager takes care of these requests and sends it to standard manager directly.In a RACenvironment the Transaction manager is required to be activated on each node of the RAC environment.

From the front end you could view the status of your concurrent manager by logging with the System Administration responsibility and going to the Concurrent Manager administer screen.

Concurrent Manager Processes
The concurrent managers are like other process which run on the oracle applications executable FNDLIBR. The FNDLIBR executable is located at $FND_TOP/bin.

You could also grep the FNDLIBR executable to check if any concurrent manager process are running
$ ps -ef|grep FNDLIBR
The $FND_TOP/sql/afcmstat.sql script gives you a list of concurrent managers and their respective status.

All Concurrent Mangers log files are located in the $APPLCSF/$APPLLOG location.The following bellow options are available to get the latest log files.

Log in R12:/maildata_test/mcgm/R12/apptier/apps/rdb_mailserver2/logs/appl/conc/log

Different Ways to start Concurrent Manager

1)adcmctl.sh--$COMMON_TOP/admin/scripts/ (11i) ,$INST_TOP/admin/scripts(R12)
2)stratmgr.sh --$FND_TOP/bin

Different ways to stop Concurrent Manager

1)adcmctl.sh
2)by using concsub utility, fro the opertating system we can kill the process by checking with FNDLIBR


In Administer concurrent manager form there are two columns labeled as actual and target 

Target:
Target column lists the number of processes that are running for each manager for this particular shift.
Actual:
Actual column list the number processes that are actually running

11i different Patching Options

By default adpatch does not take any clauses but there are some clauses that you could use with adpatch

Running a patch in test mode

You can use apply clause with adpatch to specify weather to run the patch in TEST mode or not, when you run the patch in the test mode it does not do any changes but run generates a log file with all the actions it would have performed

$adpatch apply=n/y  default is apply=y

Pre-Install mode

You can also run a patch in pre install mode this would be done normally during an upgrade or consolidated update. When a patch is applied in a preinstall mode the all AD utilities are updated before the upgrade or update.

$adpatch preinstall=y   default preinstall=n

Other options with adpatch

 You can use options clause to specify some of the other options available with adpatch.

Autoconfig

You can use the options=noautoconfig top specify autopatch that you do not wish to run autoconfig as a part of patch application. This can be useful when applying a large number of patches when they are not merged. By default autoconfig is run as a part of adpatch.

$adpatch options=noautoconfig Checkfile

The checkfile option of adpatch tells adpatch to check for already executed exec,SQL, and exectier commands. You can use options=nocheckfile skips this check, however this can cause performance overhead so should be used only when specified.

$adpatch options=nocheckfile

Compile Database
By default autopatch compile invalid objects after patch application, in casuse you wish not do so you can specify options=nocompiledb along with auotpatch

$adpatch options=nocompiledb

Compile JSP
By default autopatch compile jsp pages after patch applyed, in case you dont wat to do so 
optios=nocompilejsp along with autopatch.
$adpatch options=nocompilejsp

Copy Portion

If you wish adpatch not to execute the commands present in the copy driver portion of the patch you can use the options=nocopyportion

$adpatch options=nocopyportion

Database Portion

If you wish adpatch not to execute the commands present in the Database driver portion of the patch you can use the options=nodatabaseportion.

$adpatch options=nodatabaseportion

Generate Portions

If you wish adpatch not to execute the commands present in the Generate driver portion of the patch you can use the options=nogenrateportion.

$adpatch options=nogenerateportion


Maintenance

If you wish to apply patch regardless of the system being in maintenance mode you can use options=hotpatch

$adptach options=hotpatch

Integrity

If  you wish to check the integrity of the patch you can use the options=integrity

default is noitegrity
$adpatch options=integrity

Maintain MRC

You can use the maintainmrc option to specify weather you wish adpatch to execute the maintainmrc schema as part oth patch application or not. By default maintainMRC is done for standard patches and is disabled for translation ad documentation patches.

$adpatch options=nomaintainmrc

Pre requisite Patch Check

If you wish to adpatch not to check Pre requisite patches before application  of the mai patch you can use options=noprereq by default Pre requisite checking is enable.

Validate Schemas

If you wish adpatch to explicitly validate all the registed schemas by makig a connection you can use options=validate. bye default this validation is not performed.

$adpatch options=validate

Java Classes
if you wish adpatch not to copy new java classes from the patch you can use the options
adpatch options=nocopy

Force Copy

by default adpatch copies the file with out check the version of the existing giles already present on the system. If you do no wish the newer version of the file to be replaced by the older version contained in the patch use.

options=noforcecopy

Relinking

if you wish adpatch not do perform relinking you can use options=nolink

$adpatch option=nolink

Generate forms
if you wish adpatch not to generate forms files you can specify 
options=nogenform

$adpatch options=nogenform

Generate Reports
If you wish adpatch not to generate reports files you can specify
options=nogenrep

You can specify multiple options at the command line using as below

$adpatch options=hotpatch,nocopy,nogenereate

Merging Patches:

admrgph utility is used to merge two or more patches in oracle apps. The advantge of the merging patches is that it reduces downtime as the repetitive task of compiling invalid database objects, generae forms and reports ,jar files etc

How to use Admrgpch to merge patches

Down laod the patches in /patch directory. Now create 2 sumdirectories in /patch say mergsource-- which contains the unzipped pathes to be merged and mergedest-- which contains the merged patch. Please note that both mergesource and mergedest should be created as immidiate child of same parent directory say /patch. Now you can execue the fallowing command to merge the patches

admrgpch -s <source> -d <dest> -merge_name <mergename>

For Example:
admrgpch -s </patch/mergesource> -d </patch/mergedest> -merge_name <amebrup2>
Please make sure the merge path log file admrgpch.log does not contain any error.

Now go to destination merge patch directory say "mergedest". You can see that the admrgpch already created tbe driver with name "u_<mergename>.drv" when prompted.

Restriction of admrgpch


It will not merge patches of different releases,platforms,different parallel modes, Also do not use admrgpch to mrg AD and Non-AD patches.