IAS 10gR2 (10.1.2.0.0) on SUSE LINUX Enterprise Server 9

The purpose of this document is not really a description of how to install and application server on SLES9.

Even if I didn't understand all the options I had to provide in order to install this product, the procedure was not hard at all and I had to solve only minor issues. “Boring and long” I would define it.

Before starting I have to add two things: the main goal of this document is to understand something more about the application server (which I don't know) and to show my findings on the variable LD_ASSUME_KERNEL.

<>Another important thing: for some of the features provided by the application server (such as single sign on and portal) you need a metadata repository: i.e. a database where the needed schema will be created. Check in the compatibility matrix of metalink before using an already existing database of yours. Personally I created a 10.1.0.3 DB only to be my “infrastructure database”.
 

I'm starting with the usual recommendation on what you need in order to be able to successfully install the IAS.

I like the minimal operating system installation. Then I add the following packages:

make
gcc-3.3.3
compat
XFree86-libs
libaio-devel
libaio
openmotif-libs
openmotif

The oracle installer will check for other packages which are not really necessary:

libstdc++
libstdc++-devel
gcc-c++
gnome-libs
gnome-libs-devel

In particular to solve the dependences of the last two a lot of rpms are going to be added. You are warned.

Install the package db1 (if you have installed the additional components then this package is already installed) which contains libdb.so.2 or create a link beetween libdb.so.3 and libdb.so.2 (ln -s libdb.so.3 libdb.so.2) in /usr/lib; otherwise you will get this error:

• error while loading shared libraries: libdb.so.2: cannot open shared object file: No such file or directory

Orarun is not required but I prefer to install it.

linux: # rpm -Uvh orarun-1.8-109.5.i686.rpm
 

Now create you ORACLE_HOME:

linux: # mkdir -p /u01/app/oracle/product/10.1/ias_1
linux: # chown -R oracle:oinstall /u01/app/oracle
 

Go in /etc/profile.d, modify the entries in oracle.sh to meet your system (have a look at the other HOWTO on this website) and rename all the shell file giving a .bak extension otherwise you'll get:

oracle@breoraasls02:~> /media/cdrom/runInstaller
***************************************************************
The following files should not be present :
/etc/profile.d/oracle.csh
/etc/profile.d/oracle.sh
/etc/profile.d/alljava.csh
/etc/profile.d/alljava.sh
Please remove the above files or move them to .bak
Remove the ". ./.oracle" entries from /opt/oracle/.profile
Logout and login to reset the Shell environment.
***************************************************************
Do you want to continue anyway: [y/n]

breoraasls02:/etc/profile.d # mv oracle.csh oracle.csh.bak
breoraasls02:/etc/profile.d # mv oracle.sh oracle.sh.bak
breoraasls02:/etc/profile.d # mv alljava.sh alljava.sh.bak
breoraasls02:/etc/profile.d # mv alljava.csh alljava.csh.bak

Start a new session without a set profile (you mustn't have oracle variable set).
You will be back on the environment later.

Mount the cdrom (subfs should mount it for you automatically).
For this step I met an issue. The cdrom wasn't properly recognized by subfs so I had to mount it manually using iso9660.

You should get this warning:

oracle@breoraasls02:~> /media/cdrom/runInstaller
For SuSE, you are required to login as root and run the patch present in
the utils/3167528/ and utils/4015045 directory
Enter y if you have allready installed the patch
Enter n to exit the installer and run the patch
Is the patch for bug # 3167528 and 4015045 run : [y/n] :

These two patches are simple. As root executes:

breoraasls02:~ # cd /media/cdrom/utils/3167528
breoraasls02:/media/cdrom/utils/3167528 # perl commentport.pl
Modifications to /etc/services successful
Backup saved as /etc/services.PRE_ORACLE

and:

breoraasls02:/media/cdrom/utils/4015045 # perl commentipv6.pl
Modifications to /etc/hosts successful
Backup saved as /etc/hosts.PRE_ORACLE

The sooner will comment some reserved ports in /etc/services while the latter will comment a ipv6 entry in /etc/hosts which otherwise will give you the error:

TaskMaster sysInit failed for /u01/app/oracle/product/oem10g
     java.net.ConnectException: Connection refused
         at java.net.PlainSocketImpl.socketConnect(Native Method)
         at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:305)
         at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:171)
         at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:158)
         at java.net.Socket.connect(Socket.java:452)
         at java.net.Socket.connect(Socket.java:402)
         at java.net.Socket.<init>(Socket.java:309)
         at java.net.Socket.<init>(Socket.java:124)

when configuring the webcache.

Now you can go on answering “y” to the runInstaller.

Insert the path of the ORACLE_HOME you created.

The choice depends by your needs.

On SLES9 you always have a fail on openmotif. Simply check the box and click "next".

Again: depends by your needs.

Live and let die.

That's important for the administration.

The autoconfiguration. It could take several minutes.

To stop and start you need the command opmnctl.

If you wish to use your env variable (I find them useful for administration) then rename oracle.sh.bak in oraccle.sh and modify the content.

Here is an example taken from my configuration:

  ORACLE_SID=IASPT

  ORACLE_BASE=/u01/app/oracle
  ORACLE_HOME=$ORACLE_BASE/product/10.1/ias_1

  #TNS_ADMIN=$ORACLE_HOME/network/admin
  #ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
  PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/opmn/bin
  LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$ORACLE_HOME/lib
  #CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib

  #export ORACLE_BASE ORACLE_HOME ORA_NLS33 ORACLE_SID PATH LD_LIBRARY_PATH CLASSPATH TNS_ADMIN
  export ORACLE_BASE ORACLE_HOME ORACLE_SID PATH LD_LIBRARY_PATH

  ORACLE_TERM=xterm; export ORACLE_TERM
  NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P15; export NLS_LANG

Even these two lines should be commented:

  #test -d /lib/i686 && export LD_ASSUME_KERNEL=2.4.21

  #test -f /usr/lib/libInternalSymbols.so && export LD_PRELOAD=/usr/lib/libInternalSymbols.so

Then insert in /etc/init.d/oracle lines similar to these ones:

        echo "Starting up IAS"
        su - $ORACLE_OWNER -c "export ORACLE_HOME=$ORACLE_HOME; $ORACLE_HOME/opmn/bin/opmnctl startall > /dev/null"
       rc_status -v -r

You can start single components with:

opmnctl startproc ias-component=dcm-daemon

This part is the more time consuming and the one with the highest number of issues I met so far.
First of all the installation of the metadata repository with repca (repository creation assistant) is still undocumented on tahiti.oracle.com and secondarily you can met a couple of bugs.

However this installation is needed if you want to register you IAS in a farm using an infrastructure database instead of a filesystem.

Here the steps needed:
 

• install a 10g database and make sure you have oracle text and ultrasearch,
• take the database compenion CD and install it (text knowledge based is the part needed),
• launch as sysdba the script $ORACLE_HOME/md/admin/mdinst.sql;
• issue the command "alter user mdsys identified by mdsys account unlock;" (there is a bug here),
• connect a couple of time to mdsys to check if the password is expired (the password needs to be "mdsys");
• patch the system at least at 10.1.0.4 (make sure to issue catpatch),
• compile all the INVALID objects,
• some DB parameters should be adjasted (most memory and aq_tm_processes > 1),
• now you can install the "application server metadata repository creation assistant" (the Undocumented One): I prefer to install it in the machine with the repository DB;
• run runRepca from the repca home but make sure the NLS_LANG is unset!!! (you can set it later after the process is terminated),
• now you can decide if to "load" a repository, register it (you need an OID) or remove a previous installed repository (useful if anything goes wrong),
• "load" the repository and provide the DB information,
• you should be able to terminate the installation.

The major issues I met where due to NLS_LANG (mine was set on WEISO8859P15) and the infamouse:
 

Error: uploading country.xml failed.
[ExternalProcessAction] Process Error: java.sql.SQLException: ORA-29855: error occurred in the
execution of ODCIINDEXCREATE routine
ORA-06550: line 1, column 8:
PLS-00201: identifier 'MDSYS.MDPRVT_IDX' must be declared
ORA-06550: line 1, column 8:
PL/SQL: Statement ignored
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 10
ORA-06512: at line 1

solved by unlocking the mdsys account and creating the MDPRVT_IDX with mdinst.sql.

Now you can set back you environment (for you DB) and your DB memory parameters to have a SGA_TARGET properly confugurated.

Time to connect to your IAS and register them in the new FARM using the DB connection string.

If your IAS is already registered you can deregister it with the command:

$ORACLE_HOME/dcm/bin/dcmctl leavefarm

I'm still fighting to create a cluster using IAS + webcaches. The webcaches seems unwilling to work in the cluster.

This session is, in my opinion, the most interesting.

If you have followed the installation procedure you have seen that it is required that no env variable should be set so even LD_ASSUME_KERNEL which is usually an important part of other product installation.

What does it mean?

LD_ASSUME_KERNEL specify which implementation of the libc.so.6 (glibc) to use.
Normally three different version are installed on your system:

/lib/i686/libc.so.6
/lib/libc.so.6
/lib/tls/libc.so.6

this is made for compatibility with older system.
so if you set LD_ASSUME_KERNEL to 2.4.21 (the usual set for SUSE) you are going to use /lib/i686/libc.so.6

Let's go to a system where I installed a database and issue the command:

oracle@brealmdbls01:~> lsof|grep oracle|grep libc.so.6|head -n 5
tnslsnr    2612  oracle  mem    REG      104,2    1321972    275270 /lib/i686/libc.so.6
tnslsnr    2627  oracle  mem    REG      104,2    1321972    275270 /lib/i686/libc.so.6
tnslsnr    2628  oracle  mem    REG      104,2    1321972    275270 /lib/i686/libc.so.6
tnslsnr    2629  oracle  mem    REG      104,2    1321972    275270 /lib/i686/libc.so.6
oracle     2784  oracle  mem    REG      104,2    1321972    275270 /lib/i686/libc.so.6

As you can see all the processes of the orcle user are employing /lib/i686/libc.so.6

Doing the same test on the machine where you have just installed the application server you'll get:

oracle@breoraasls02:~> lsof|grep oracle|grep libc.so.6|head -n 5
sshd       3356  oracle  mem    REG    8,2  1345545     950307 /lib/tls/libc.so.6
bash       3357  oracle  mem    REG    8,2  1345545     950307 /lib/tls/libc.so.6
opmn       6980  oracle  mem    REG    8,2  1345545     950307 /lib/tls/libc.so.6
java       7004  oracle  mem    REG    8,2  1345545     950307 /lib/tls/libc.so.6
opmn       8509  oracle  mem    REG    8,2  1345545     950307 /lib/tls/libc.so.6

The result is, as expected, not the same.

Then what are the differences between the two libc.so.6?

If you use them as a command you'll discover:

oracle@breoraasls02:~> /lib/i686/libc.so.6
GNU C Library stable release version 2.3.3 (20040412), by Roland McGrath et al.
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Configured for i686-suse-linux.
Compiled by GNU CC version 3.3.3 (SuSE Linux).
Compiled on a Linux 2.6.5 system on 2004-06-30.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        linuxthreads-0.10 by Xavier Leroy
        GNU Libidn by Simon Josefsson
        NoVersion patch for broken glibc 2.0 binaries
        BIND-8.2.3-T5B
        libthread_db work sponsored by Alpha Processor Inc
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Thread-local storage support included.
pthread library is compiled with floating stack support enabled.
Report bugs using the `glibcbug' script to <bugs@gnu.org>.

oracle@breoraasls02:~> /lib/tls/libc.so.6
GNU C Library stable release version 2.3.3 (20040412), by Roland McGrath et al.
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Configured for i686-suse-linux.
Compiled by GNU CC version 3.3.3 (SuSE Linux).
Compiled on a Linux 2.6.5 system on 2004-06-30.
Available extensions:
        GNU libio by Per Bothner
        crypt add-on version 2.1 by Michael Glad and others
        NPTL 0.61 by Ulrich Drepper
        GNU Libidn by Simon Josefsson
        NoVersion patch for broken glibc 2.0 binaries
        RT using linux kernel aio
        BIND-8.2.3-T5B
        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Thread-local storage support included.
Report bugs using the `glibcbug' script to <bugs@gnu.org>.

Mainly NPTL 0.61 by Ulrich Drepper are used instead of the older linuxthreads-0.10 by Xavier Leroy.

In 10g we can see how heavily the development as been shifted from processes to threads. Even the listener changed:

oracle@brealmdbls01:~> ps -fe|grep tnsl
oracle    2612     1  0 Mar20 ?        00:01:59 /u01/app/oracle/product/10.1.0.3/db_1/bin/tnslsnr LISTENER -inherit
oracle    2627  2612  0 Mar20 ?        00:00:00 /u01/app/oracle/product/10.1.0.3/db_1/bin/tnslsnr LISTENER -inherit
oracle    2628  2627  0 Mar20 ?        00:00:00 /u01/app/oracle/product/10.1.0.3/db_1/bin/tnslsnr LISTENER -inherit
oracle    2629  2627  0 Mar20 ?        00:00:00 /u01/app/oracle/product/10.1.0.3/db_1/bin/tnslsnr LISTENER -inherit

Four threads (I have 4 CPU) instead of a process.

Since I had serious problems with SLES7 threads (linuxthreads-0.09) and RAC I'm wondering if the newer native posix thread libraries should be used instead of the older. I can't answer now since I need to test the behaviour of the systems using different libc.so.6 but I feel I contributed somehow to a possible misconfiguration of several DBs in the SUSE community.
 

Another worry concerns async I/O (RT using linux kernel aio) but even here I need to investigate further.

<>What to do now?
With this document I'd like to warn people: official documentation and installation process can be flawed!

Contact information:
fabrizio.magni _at_ gmail.com