Oracle Grid Infrastructure – Part 5

Dear Readers,

My name is Franz Devantier, creator of this blog.  I am an Oracle Certified Professional (OCP DBA 11g) Security DBA.  I will be sharing with you the basic duties of an Oracle DBA, and also some of the undocumented, and not so well known tasks. 

Oracle Grid Infrastructure – Part 5

Configuring Disk Devices Manually for Oracle ASM

Kernel version 2.6 has a default device file naming scheme call “udev”.  The 2.6 kernel will dynamically create device files names when you restart the server.  Also the devices will be assigned to root ownership.  If udev is allowed to apply the default settings, then it will change device file names and owners for the disks, which will corrupt them, if an Oracle ASM instance is restarted.  Using ASMLIB, means that you alleviate this problem, and you will not have to ensure device path consistency, and permissions in “udev”.

If you are not using ASMLIB, then you can create a custom rules file.  After a restart udev will sequentially carry out rules configuration directives, that are defined in the rules files.  The rules files are stored in the path /etc/udev/rules.d/  The rule files are read in lexical order from the directories.  In the case were the rules files would describe the same device more than once: On Asianux, Red Hat, and Oracle, the last file read would be read and applied.  On SUSE Enterprise Server, with the 2.6 kernel, the first file read is the one that is applied.

Configuring a permissions file for disk devices.

=> Configure the SCSI devices as trusted devices. Edit /etc/scsi_id.config and add “options=ig”

=> Create a udev file for your Oracle ASM devices, using a text editor.  The permissions should be set to 0660 for the installation owner, as well as for the group whose members are administrators of the grid infrastructure software.  So the installation owner could be “grid”, and the OSASM group could be “asmadmin”.

$ vi /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id",

RESULT=="14f70656e66696c00000000", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd?2", BUS=="scsi", PROGRAM=="/sbin/scsi_id",

RESULT=="14f70656e66696c00000000", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd?3", BUS=="scsi", PROGRAM=="/sbin/scsi_id",

RESULT=="14f70656e66696c00000000", OWNER="grid", GROUP="asmadmin", MODE="0660"

=> Now you can load the updated block device partition tables on the server as the root user:

$ /sbin/partprobe <devicename>

=> Give the commands to restart the udev service.

For Asianux, Oracle Linux 5, Red Hat Enterprise Linux 5:

$ /sbin/udevcontrol reload_rules

$ /sbin/start_udev

For SUSE Linux Enterprise server 10 / 11

$ /etc/init.d boot.udev restart

Installing Oracle Grid Infrastructure Using a Software-Only Installation

Only the binaries are copied to the specified location.  In this case you must manually configure Oracle Grid Infrastructure for a standalone server, and then Oracle ASM must be done manually after the installation is finished.

With this type of installation, you will then also need to manually configure Oracle Restart after you have installed the software.

The software-only installation method provides no validation of the installation, and this type of installation will also require a number of post-installation steps before the Oracle Grid Infrastructure Software is enabled.

Here is a breakdown of the Software-only installation:

Installing the software binaries, and configuring the software binaries.

=> Installing the software binaries

- You start of by running “runInstaller” from the relevant location on the Oracle 11g release 2 installation media or download directory.

- Complete a software only installation of Oracle Grid Infrastructure.

- Verify that the server meets the installation requirements.

$ runcluvfy.sh stage pre hacfg

Ensure that all the storage and server pre-installation steps are completed and successful.

=> Configuring the software binaries.

To configure and activate a software-only Oracle Grid Infrastructure installation for Oracle Restart, you will need to complete the following tasks.

- # $GRID_HOME/perl/bin/perl –l $GRID_HOME/perl lib –l $GRID_HOME/crs/install $GRID_HOME/crs/install/roothas.pl

- Change to $GRID_HOME/oui/bin   $GRID_HOME is the path of the Oracle Grid Infrastructure home.

- # ./runInstaller –updateNodeList ORACLE_HOME=$GRID_HOME -defaultHomeName CLUSTER_NODES=CRS=TRUE

- Now you need to add the listener, the Oracle ASM instance, and all the Oracle ASM disk groups to the Oracle Restart configuration.  To achieve this you can use the SVRCTL utility, Network Configuration Assistant, and Oracle ASMCA.

Installing and Configuring Oracle Grid Infrastructure for a Standalone Server

When you install Oracle Grid Infrastructure first, and then you continue to create your database, the result is that the database is automatically added to the Oracle Grid infrastructure configuration, and will be automatically restarted when required.

However if there is already an Oracle database on a host, and you install Oracle Grid Infrastructure, then you will have to manually add the database, the listener, and Oracle ASM instance, as well as other components to the Oracle Grid Infrastructure configuration.

Oracle Grid Infrastructure will accommodate multiple single instance databases on a single host computer.

Installing Oracle Grid Infrastructure with a New Database Installation

In this case we are going to first install the Oracle Grid Infrastructure, and then create a database, that will be managed by Oracle Restart.  When you install Oracle Grid Infrastructure, then you will be installing Oracle Restart and Oracle ASM at the same time.  You will need to configure ASM with at least one disk group, before you install an Oracle database, that stores database files in the Oracle ASM disk groups.

Before you start with the Oracle Grid Infrastructure installation, make sure that there are no existing Oracle processes running.  Assuming that this is an installation for Oracle Grid Infrastructure for a standalone server, and that we are going to install a new Oracle database on it afterwards.

-> switch user to the Oracle Grid Infrastructure software owner.  Start the Oracle Universal Installer.

- If the installation files are on an installation media, then firstly you must install Oracle Grid Infrastructure for a standalone server in this scenario.

# $DIRECTORY_PATH/runInstaller

- If the installation files are on a hard disk, then you need to change directory to the “Oracle Grid Infrastructure” directory, which is the clusterware directory.

# ./runInstaller

Downloading Updates Before Installation

From Oracle Database 11.2.0.2 onwards, you can download the updates, before starting the installation, provided that the system that you are on, has internet access.

# ./runInstaller –downloadUpdates

You will need to supply your “My Oracle Support”, username and password, and proxy settings if required.  When the download of the updates has finished, then you can transfer the update file to the directory on the host or server where you are going to run Oracle Universal Installer from.  Remember to start the Oracle Universal Installer as the Oracle Grid Infrastructure software owner.  Make sure that your X Window Display is working properly before you start.

-> From Oracle Database 11.2.0.2 onwards, you have the option when you run Universal Installer, to dynamically download and apply the latest updates.  When you start Universal Installer, you will come to the “Download Software Updates” screen.  On this screen you will be presented with three options.  Select one of them and click on the “Next” button.

Option 1: “Use My Oracle Software Support credentials for download:”

This option will download and apply the latest software updates.  You must click on the “Proxy Settings”, to configure a proxy for Oracle Universal Installer to use, for the connection to the internet.  You will also need to give the details of a user account that has access to the local area network, through which the server is connecting.

From Oracle database 11.2.0.3, you can enter the Proxy realm information, such as the Proxy Username, Proxy Password, and Proxy Realm fields.

Before you leave this screen you should press the “Test Connection” button, to make sure that Oracle Universal Installer can access the downloads.

Option 2: “Use pre-downloaded software updates:”

If you had previously downloaded the latest downloads, then you can choose this option.

# ./runInstaller –downloadUpdates

Option 3: “Skip software updates:”.

If you don’t want to download and or apply any updates, then you can select this option.

=> If you had selected to download or apply updates from the previous screen, then you will be brought to the “Apply Software Updates” screen.

- If you had selected “Use My Oracle Support Credentials for Download”, then you should select the “Download and apply all updates” option and click on the “Next” button to apply the updates.

OUI places the downloads in the home directory of the Oracle Installation Owner, that is running the installation.  However if you would prefer the downloads to go to another location, then you can click on the “Browse” button, and select the location on the server that you want the updates to go to.

- If you had selected “Use pre-downloaded software updates”, then you can select “Apply all updates”, and click on the “Next” button to apply the downloads, before the installation starts.

=> Next you come to the “Select Installation Option” screen.  On this screen you need to select the “Configure Oracle Grid Infrastructure for a Standalone Server” option.  Selecting this option will then install and configure Oracle Restart and Oracle ASM.  When you are ready click the “Next” button.

=> “Product Languages” screen.  You can select one or more languages, and move the languages from the Available languages list to the Selected Languages list.  When you are ready, you can click on the “Next” button.

=> “Create ASM Disk Group” screen will display.  On this screen you will see a list of all the Oracle ASM disks.  You may like to change the devices that are to be used by Oracle ASM.  You can click on the “Change Discovery Path” button.  In the “Change Discovery Path” window, you can enter a string for Oracle to use to search for devices that Oracle ASM will use.  The disk string may be set to “” for example, or to “ORACL:*”.  During installation Oracle with look for disks mounted on Oracle ASM, and registered on ASMLIB with the string “ORCL:*”, such disks will be listed by default as database storage candidate disks.

After you have finished selecting the disks on the “Create ASM Disk Group” screen to be used by Oracle ASM, then before you click the “Next” button, consider this:

- The default Disk Group Name is “DATA”.  You can use this default name or enter a new name for the disk group.

- The disk devices must be owned by the user performing the grid installation.

- Verify with your system administrator, if the disks to be used by Oracle ASM are mirrored at the storage level.  If they are, then you must select “External” for the redundancy, otherwise you must select “Normal” for the redundancy.  Keep in mind that for normal redundancy you will require roughly double the storage or disk space to hold the same amount of data.  So if you are allowing 200GB for your database, you will need 400GB of storage.

- Every Oracle ASM disk is divided into allocation units (AU).  Allocation Units are the fundamental unit of allocation within disk groups.  From Oracle 11.2.0.3 onwards, you can select the AU sizes.  By default they are set to 1MB, but you can also select 2,4,8,or 16MB.  This will depend on the disk group compatibility level and so is not guaranteed.

When you are satisfied you can click on the “Next button” to get to the next screen.

=> “Specify ASM Password” screen.  You must enter the SYSASM password, that will be required to connect to the Oracle ASM instance.

The Oracle ASM instance is managed by a privileged role “SYSASM”.  This role grants full access to the Oracle ASM disk groups.  You should create a less privileged user called ASMSNMP with the SYSDBA privileges, that can be used to monitor the Oracle ASM instance.

Now you can enter the passwords for the SYS and ASMSNMP user accounts.  There are rules for the passwords, for example, they should be a minimal of 8 characters long.  The password should include at least one alphabetic character, one numeric character and one of the following: hyphen “-“, underscore “_”, or the number sign “#”.  These three are the only special characters allowed.  You do have the option of using the same password for all the accounts that you are defining here.  Oracle recommends that you should specify a different password for each account.  Perhaps a good idea to write them on a piece of paper, put it in an envelope and lock it up in a safe, because you really need to be able to remember these passwords.

When you are ready you can click on the “Next” button.

=> Privileged Operating System Groups” screen.  On this screen you must select the name of the operating system group that you created for the OSDBA group, as well as the OSASM group, as well as the OSOPER group.  If you were lazy and created only the OSDBA group, then you can use this group for all three privileged groups.  If you did create a separate “asmadmin” group, then use that group for the OSASM group.

When you are satisfied, then click on the “Next” button.

=> “Specify Installation Location” screen, you need to specify the following information:

- Oracle Base: No spaces in the path name, and as short as possible.

- Software Location: You can either accept the default path given, or you can enter the directory path in which you want to install the software.  No spaces in the path name.

When you are satisfied you can click on the “Next” button.   

=> “Create Inventory” screen.  This screen will only appear if you have not previously installed Oracle software on the server.  If you are not happy with the default Inventory Directory, you can change the Directory path.  Remember that all the Oracle products will be using this same directory path.

Select “oinstall” for the “oraInventory” group name, if you are not satisfied with the default.

When you are satisfied you can click on the next button.

=> “Perform Prerequisite Checks” screen.  On this screen oracle will perform a check to see if the minimal requirements are met to perform the Oracle Grid Infrastructure installation.

- If all the system requirements are met, then you are directed to the Summary Sreen.

- If there are some components that don’t meet the requirements, then there will be a note about them.  You can try to manually fix whatever the problem is, and then click on the “Check Again” button.  You can also click on the “Fix & Check Again” button, which will cause the installer to try to remedy the problem, and then to check again.  The Fix & check again will generate a fix script, which you must run as the root user.  This script may set some system and kernel parameters.  It is not a good idea to modify the contents of this script.  Once you have run the script, you are ready to check again.

You may decide to get a list of the failed requirements.  You can select the “Show Failed” button.  To see the successful pre-requisites select “Show Successful”, and to see all the pre-requisites select “Show All”.

You should consider a problem on this screen to be a show-stopper as far as the installation is concerned.  If you select the “Ignore All” option and continue with the installation, then you must remember that Oracle Universal Installer was not able to confirm that the server is configured in such a way that you will be able to install an Oracle database on it.  So you may go ahead with the installation, but the chances are that you will have huge problems down the line.

When you are satisfied, you can click on the “Next” button.

=> You arrive at the Summary screen.  You must review everything on the Summary screen, to make sure that the configuration is setup in the way that you intended.  Once you are happy you can click on the “Install” button.

From 11.2 onwards, you can click the “Save Response File” option, which will save all the installation steps into a response file for you.  You can then use this response file for a silent installation.

=> “Install Product” screen will display the progress of the Oracle Grid Infrastructure installation.

During the installation another window will appear “Execute Configuration Scripts”.  You must first successfully verify that these scripts have been run as the root user, before you click on the “Ok” button.  The scripts mentioned to be run may be the “root.sh”, and the “orainstRoot.sh”.

=> “Finish” screen will finally display the installation status.  If all your pre-requisites were met, then there is a good chance of having success on this screen.  If you did encounter problems, then you must make a note of the location of the configuration log, to search for more information.  You can click on the “Close” button to end the installation.  Then you click on the “Yes” button to exit Oracle Universal Installer (OUI).   

=> Oracle ASMCA will be installed as part of the Oracle Grid Infrastructure for a standalone server installation.  Now if you want to crate additional disk groups, you can run the Oracle ASMCA utility.  You may like to create a disk group called “RECOVERY” to store or house the fast recovery area.

You will want to verify that Oracle High Availability Service has been installed properly.

Change directory to the $GRID_HOME/bin directory.  $GRID_HOME is the path to the Oracle Grid Infrastructure home for a standalone server.

# ./crsctl check has

Ohasd is a software daemon that gets installed with Oracle Grid Infrastructure, that will start the software services such as Oracle ASM.

=> Now you are ready to install the Oracle database.

Remember that if the database is installed after a Grid Infrastructure installation, then the listener will run from the Oracle Grid Infrastructure home.  When you subsequently install a database, then the database must use the same listener that was created during the Oracle Grid Infrastructure installation.

If you are using Oracle Restart then the default listener, as well as any additional listeners have to run from the Oracle Grid Infrastructure home.

Franz Devantier,

Need a database health check?

devantierf@gmail.com