Oracle RAC | Electric Monk

Archive for Oracle RAC

Installing Oracle Linux 7 and Oracle Grid Infrastructure and Database 12c RAC cluster on vSphere 6 (Lab)

January 3, 2016 Oracle RAC 15 comments

What is Oracle RAC?

The Oracle Database with the Oracle Real Application Clusters (RAC) option allows the running of multiple database instances on different servers in the cluster against a shared set of data files, also known as the database. The database spans multiple hardware systems and yet appears as a single unified database to the application.

This enables the utilization of commodity hardware to reduce total cost of ownership and to provide a scalable computing environment that supports various application workloads. Oracle RAC is Oracle’s premier shared disk database clustering technology

Pre-Requisites

vSphere 5 or 6 or VirtualBox also works very well
2 x Oracle Linux 7 VMs (2 vCPUs and 6GB RAM)
Oracle Grid Infrastructure 12c Release 1 (12.1.0.2)
Oracle Database Enterprise Edition 12.1.0.2.0 for Linux x86-64
1 x 50GB O/S disk on each node
3 x 20GB CSR disks (Multi-writer enabled on Disk 2/3/4) on node 1 only and added to node 2 as existing disks
1 x 25GB DB Disk (Multi-writer enabled on Disk 5) on node 1 only and added to node 2 as an existing disk
1 x 25GB Redo Disk (Multi-writer enabled on Disk 6) on node 1 only and added to node 2 as an existing disk

IP Addresses

2 x LAN IP addresses
2 x Private IP addresses
2 x VIP IP addresses
3 x SCAN IP addresses

Note: I had to add 3 A records in DNS pointing to 192.168.1.132, 192.168.1.133 and 192.168.1.133 for the scan IPs
See this link for more info on DNS and SCAN names http://www.oracle.com/technetwork/products/clustering/overview/scan-129069.pdf

Software

Download the following software.

Oracle Linux; Once in the Oracle Cloud service, select “Oracle Linux” and “x86 64-bit”.
VirtualBox (Use if you don’t have a vSphere 5/6 installation)
Oracle Database 12c Release 1 (12.1.0.1.0) (use Linux x86-64, then “See All” link).
Oracle Database 12c Release 1 Grid Infrastructure (12.1.0.1.0) for Linux x86-64 (use Linux x86-64, then “See All” link).

Instructions for the first VM

Note: There may be points where it is good to snapshot your VMs if you need a point to go back to if anything goes wrong. It took me a while to work all this out and snapshots were very useful 🙂

Download Oracle Linux from http://edelivery.oracle.com/linux

Create 2 VMs
Add 6 disks to the first VM and just the O/S disk on the second VM
1 x O/S disk on each node
3 x CSR disk (Multi-writer enabled on Disk 2/3/4) on node 1 only and added to node 2 as an existing disks. (Must be independent persistent)
1 x DB Disk (Multi-writer enabled on Disk 5) on node 1 only and added to node 2 as an existing disk (Must be independent persistent)
1 x Redo Disk (Multi-writer enabled on Disk 6) on node 1 only and added to node 2 as an existing disk (Must be independent persistent)

Log into the web client and go to the second VM
You will need to add all the extra disks as existing disks to the second VM
Next you need to set the multi-writer flag on both VMs – The multi-writer has to be set on the non O/S disks in the vSphere Web Client not the c# client in vSphere 6 or the settings do not stay and you will find you cannot power on the second VM. In vSphere 5 you can still use the c# client to set the multi-writer flag in the Configuration parameters of the VM as seen below.
Link for Information here

VMFS is a clustered file system that disables (by default) multiple virtual machines from opening and writing to the same virtual disk (.vmdk file). This prevents more than one virtual machine from inadvertently accessing the same .vmdk file. The multi-writer option allows VMFS-backed disks to be shared by multiple virtual machines. This option is used to support VMware fault tolerance along with Oracle RAC, which allows a primary virtual machine and a standby virtual machine to simultaneously access a .vmdk file.
You will also need to add the configuration parameter disk.enableUUID and set the value to true. This step is necessary so that the VMDK always presents a consistent UUID to the VM, thus allowing the disk to be mounted properly. For each of the virtual machine nodes (VMs) that will be participating in the cluster, follow the steps below from the vSphere client. VMware by default does not pass through the device UUID’s for the disks so the Oracle documentation (the udev rules) will fail to work because they depend on the UUID’s to create the ASM symlinks in DEV
Attach the Oracle Linux iso to the first VM
This link is useful – https://oracle-base.com/articles/linux/oracle-linux-7-installation

Let it run through bootup then select your language

You will now see the below screen

Click on System and check the hard drive layout
I left it on automatic partitioning

Click on network and put in host and network name settings
Click Configure to put in IPv4 IP Address, Netmask and Gateway settings

Click Save then move the network connection slider to On

Click Done
Click on Software Selection > Select Server with GUI
You may want to choose to install a GUI desktop. This is never recommended for a production environment. For ease of maintenance, you can click on the SOFTWARE SELECTION button and choose to install a “Server with GUI” option and choose the KDE desktop.
I also selected Java Platform / Performance Tools and Development Tools

Click on the Begin Installation button after you select all the options that you want installed

You will now see the screen below

Set a root password

Next enter a username who will be the administrator

It will now install

Once installation is complete, reboot your machine
You should now see the below screen

Accept the license agreement

Check the network configuration and click Finish Configuration

You will now see the login page
Log in using your admin account which you setup

You will now see the below screen

On the typing screen select your language

Skip the email account setup

You should now see the screen below

Install Oracle on the second node

Next update VMware Tools on both nodes – Attach the iso to the VM

Log in to Putty and type the below commands
ls media to check nothing is previously mounted to /media
mount /dev/cdrom /media/
cd /media
ls
cp VMwareTools9.10.5.tar.gz /tmp/
cd /tmp
ls
tar xvzf VMwareTools9.10.5.tar.gz
cd vmware-tools-distrib
./vmware-tools.pl -default
rm -rf vmware-tools-distrib
rm -rf VMwareTools9.10.5.tar.gz
If there is an issue with VMware Tools saying VMware Tools not installed on the system then type
rm -f /etc/vmware-tools/locations
Next Open Putty and run yum update -y on both nodes

Next type yum install ntp -y on both nodes

Next Oracle have provided a pre-requisite checker
Type yum install oracle-rdbms-server-12cR1-preinstall -y on both nodes

Note: If you get an message saying No package oracle-rdms-server-12cR1-preinstall available then try typing
cd /etc/yum.repos.d/
wget http://public-yum.oracle.com/public-yum-ol7.repo
yum repolist
yum install oracle-rdbms-server-12cR1-preinstall -y

WINSCP into the VM and modify the host file to reflect both nodenames and IP addresses on both nodes
The SCAN address is commented out of the hosts file because it must be resolved using a DNS, so it can round-robin between 3 addresses on the same subnet as the public IPs. The DNS can be configured on the host machine using BIND or Dnsmasq, which is much simpler. If you are using Dnsmasq, put the RAC-specific entries in the hosts machines “/etc/host” file, with the SCAN entries uncommented, and restart Dnsmasq.

Set the password for the Oracle user on both nodes
Type passwd oracle and change password

Make sure the “/etc/resolv.conf” file on both nodes includes a nameserver entry that points to the correct nameserver. Also, if the “domain” and “search” entries are both present, comment out one of them. For this installation my “/etc/resolv.conf” looked like this.

On both nodes amend the “/etc/security/limits.d/20-nproc.conf” file from 4096 to 16384

Change the setting of SELinux on both nodes to disabled by editing the “/etc/selinux/config” file, making sure the SELINUX flag is set as follows.

Next we are going to disable the Linux firewall on both nodes, which will otherwise interfere with the RAC install. First let’s make sure the firewall service is stopped
Type systemctl stop firewalld
Type systemctl disable firewalld

Either configure NTP, or make sure it is not configured so the Oracle Cluster Time Synchronization Service (ctssd) can synchronize the times of the RAC nodes. If you want to deconfigure NTP do the following, which is what I did for this installation.

Add the following lines to the “/etc/security/limits.conf” file.

oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768
grid     soft   nofile   1024
grid     hard   nofile   65536
grid     soft   nproc    2047
grid     hard   nproc    16384
grid     soft   stack    10240
grid     hard   stack    32768

Add or amend the following lines to the “/etc/sysctl.conf” file on both nodes. Note this file is probably OK… I didn’t need to adjust it.

fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500

Run /sbin/sysctl -p
Create the directories in which the Oracle software will be installed on both nodes
mkdir -p /u01/app/12.1.0.2/grid
mkdir -p /u01/app/oracle/product/12.1.0.2/db_1
chown -R oracle:oinstall /u01
chown oracle:oinstall /u01/app/oracle
chmod -R 775 /u01/

Add in groups on both nodes
Note you will see from the screenprint below that the pre-requisites installer from Oracle creates 2 groups but not the rest so we still need to create them
groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper
groupadd -g 54324 backupdba
groupadd -g 54325 dgdba
groupadd -g 54326 kmdba
groupadd -g 54327 asmdba
groupadd -g 54328 asmoper
groupadd -g 54329 asmadmin

The user oracle was already created by the rdbms preinstall package with oinstall and dba groups. But it is not member of all newly created groups. Also, it has no rights for ASM, and is not able to see ASM disks to create a database. For that, the oracle user needs the asmdba group. You can drop and recreate the user, or just alter its parameters with this command:
/usr/sbin/usermod -g oinstall -G dba,backupdba,dgdba,kmdba,oper,asmdba,asmadmin oracle
Grid Infrastructure needs a software owner, member of oinstall, which is not created by the rdbms preinstall package. It is advised to create a dedicated user:
/usr/sbin/useradd -u 54322 -g oinstall -G asmadmin,asmdba,asmoper grid
Type passwd grid and change the password
These are the groups which are set up

You now need to edit the following lines at the end of the “/home/oracle/.bash_profile” file
To do this I opened WinSCP and navigated to /home/oracle
Click on Hidden at the bottom right hand corner
You should now see the hidden .bash_profile file

Copy the file to your desktop and you now need to add the following lines to the end of the file

# Oracle Settings
export TMP=/tmp
export TMPDIR=$TMP

export ORACLE_HOSTNAME=techlabora001.techlab.local
export ORACLE_UNQNAME=cdbrac
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.1.0.2/grid
export DB_HOME=$ORACLE_BASE/product/12.1.0.2/db_1
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=cdbrac1
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

if [ $USER = "oracle" ]; then
  if [ $SHELL = "/bin/ksh" ]; then
    ulimit -p 16384
    ulimit -n 65536
  else
    ulimit -u 16384 -n 65536
  fi
fi

alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'
umask 022

Next Create a file called “/home/oracle/grid_env” with the following contents.

export ORACLE_SID=+ASM1
export ORACLE_HOME=$GRID_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

Next Create a file called “/home/oracle/db_env” with the following contents.

export ORACLE_SID=cdbrac1
export ORACLE_HOME=$DB_HOME
export PATH=$ORACLE_HOME/bin:$BASE_PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

We now need to edit the “/home/oracle/.bash_profile” file on the “techlabora002” node to correct the ORACLE_SID and ORACLE_HOSTNAME values.
Note: At the bottom of the file it says unmask and meant to say umask

Create and edit the /home/oracle/grid_env file and edit as below

Create and edit the /home/oracle/db_env file and edit as below

Once the “/home/oracle/.bash_profile” has been run, you will be able to switch between environments as follows.
I logged in as oracle to both nodes and ran the following

Follow this link below or my instructions underneath to set a swap file space on both nodes.

http://it-toolkit.blogspot.co.uk/2015/07/increase-swapfile-space-on-oracle.html

Putty into each node and run the following
dd if=/dev/zero of=/home/swapfile bs=6048 count=1048576
mkswap /home/swapfile
swapon /home/swapfile
swapon -a
swapon -s

WinSCP into each node and update /etc/fstab (Important)

Next do the following
Open /etc/fstab/ and add the highlighted line to the bottom
tmpfs /dev/shm /tmpfs rw,exec 0 0

Then run in Putty the following command
mount -o remount,size=4g /dev/shm

The reason we do this is during the installer pre-requisite checker, it will sometimes error falsely. This error of /dev/shm is because of a bug in cluvfy which comes with 12.1.0.2 Grid Infrastructure. I don’t get any errors after doing the above
Next Type chkconfig avahi-daemon off on both nodes
On each node add the following to /etc/pam.d/login file on both nodes
session required /lib64/security/pam_limits.so session required pam_limits.so
session required pam_limits.so
Next on both nodes unzip the grid files
cd /tmp/grid
unzip linuxamd64_12102_grid_1of2.zip -d /tmp/grid
unzip linuxamd64_12102_grid_2of2.zip -d /tmp/grid
Restart both nodes

When both nodes have started, check they can both ping all the public and private IP addresses

At this point I shut both servers down and took a snapshot
Power both nodes back on again
Next we need to format the disks originally added to Node 1 only on node 1
Format the 5 disks which will be sdb, sdc, sdd, sde, and sdf
As an example for the first disk, type fdisk /dev/sdb
Type n for new
Type p for primary
Type 1 for partitionPress enter
Press enter
Press enter
Press w

SSH

The OUI (Oracle Universal Installer) is run on one node, and uses SSH and scp commands to run command and copy files on remote nodes during the install. SSH must be configured on all members of the cluster to avoid each command to prompt for a password. This configuration can be done during install process, with the OUI, or with a script provided by Oracle and available in /software/grid/sshsetup.

You need to run the sshUserSetup.sh script for both oracle and grid accounts:

Log into Node 1 using Putty and navigate to the grid folder you placed in the /tmp folder followed by navigating to the sshsetup folder
Note I checked this in WinSCP before logging in to Putty. You can see the sshUserSetup.sh script below

Within Putty, you need to enable execute permissons (chmod +x sshUserSetup.sh)
Then type ./sshUserSetup.sh -user grid -hosts “techlabora001 techlabora002” -advanced -exverify –noPromptPassphrase

You will be prompted to enter some inputs
Do you want to continue and let the script make the changes Say Yes
Say No to creating a passphrase
Put in the grid password several times
You should now see the below

Next you need to run the same command but change the user to oracle
Type ./sshUserSetup.sh -user oracle -hosts “techlabora001 techlabora002” -advanced -exverify –noPromptPassphrase
Putty into the second node
Within Putty, you need to enable execute permissons (chmod +x sshUserSetup.sh)
Then type ./sshUserSetup.sh -user grid -hosts “techlabora001 techlabora002” -advanced -exverify –noPromptPassphrase
Then type ./sshUserSetup.sh -user oracle -hosts “techlabora001 techlabora002” -advanced -exverify –noPromptPassphrase
Once you have set up the ssh, test it in Putty by typing ssh techlabora001 or techlabora002 from one node to another to test connectivity

ASM

The installation and configuration procedures in this section should be performed on both of the Oracle RAC nodes in the cluster. Creating the Automatic Storage Management (ASM) disks, however, will only need to be performed on a single node within the cluster (Node1).

ASM will be used as the shared file system and volume manager for Oracle Clusterware files (OCR and voting disk), Oracle Database files (data, online redo logs, control files, archived redo logs), and the Fast Recovery Area.

ASM distributes the I/O load across all available resources to optimize performance, while removing the need for manual I/O tuning. ASM helps DBAs manage a dynamic database environment by allowing them to grow the database size without any database downtime for storage allocation modification. Oracle ASM is easier to manage than conventional file systems, and is designed to provide optimal performance for Oracle database deployments.

ASM provides redundancy by mirroring against disk failures. ASM stripes data evenly across a number of disks in the disk group, and maintains a uniform distribution of extents as the storage configuration changes

With the introduction of Oracle ASM, administrators can expand storage capacity without having to take the database offline; additional ASM disks can be added to an existing ASM disk group for increased storage capacity. It is a best practice that all ASM disks belonging to the same disk group are similarly sized. Therefore, when adding new ASM disks to an existing disk group, it is recommended to configure the new disks with similar sizes as the existing ASM disks of the same disk group.

Adding or dropping ASM disks will automatically trigger ASM to rebalance the I/O that spreads data across all available disks in the disk group for optimal performance and utilization; this rebalancing involves physical movement of file extents. The impact to the online database activity is generally low, because the rebalance is done at one allocation unit at a time. However, Oracle recommends that you schedule the rebalance operation during off‐peak hours.

To use Oracle ASM, two or three packages are required, depending on the situation. Here, we consider using ASMLib instead of Raw devices. So three packages will be required:

oracleasm-support
oracleasmlib
The driver oracle asm (For this package, it depends of your kernel. If your server is configured with the default Unbreakable Enterprise Kernel (UEK), the driver is already compiled in the kernel. If you are using the Red Hat compatible kernel, you must install the following package (for Oracle Linux 6)

Instructions

Log into Putty
Install ASM Support
Type yum install oracleasm-support

Next we need ASMLIB
ASMLib is a support library for the Automatic Storage Management feature of the Oracle Database. Oracle provides a Linux specific implementation of this library
Type wget http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.12-1.el7.x86_64.rpm
Type rpm -iv oracleasmlib-2.0.12-1.el7.x86_64.rpm

Type yum install kmod-oracleasm (May not need this. Oracle Linux 7 has the unbreakable kernel)

Next do the same on the second node
Next we need to configure and load the asm kernel module on both nodes
Type /usr/sbin/oracleasm configure -i
grid
asmadmin
y

The script completes the following tasks:

Creates the /etc/sysconfig/oracleasm configuration file
Creates the /dev/oracleasm mount point
Mounts the ASMLib driver file system
Next, initialize ASM module on both nodes:
Type /usr/sbin/oracleasm init

Next add the disks to the first node only

Next go to the second node and you can type
/usr/sbin/oracleasm scandisks
You can also type /usr/sbin/oracleasm status

If you want to check the disks type
/usr/sbin/oracleasm listdisks

Now we can start to install our Grid Infrastructure
Previously we downloaded and zipped the relevant grid files into the /tmp/grid/grid folder
Install cvuqdisk. The file you are looking for to start with is cvuqdisk in the /tmp/grid/grid/rpm folder

Type rpm -Uvh cvuqdisk*

Next there is a tool in the Grid software called runcluvfy. This is a tool provided by Oracle. It checks all software and hardware pre requisites on each node to make sure the install can be done. This tool is called before installing grid infrastructure, before adding a node, or before adding a database.

Run the following as the Oracle user
Type cd /tmp/grid/grid
Type ./runcluvfy.sh stage -pre crsinst -osdba asmdba -orainv oinstall -n techlabora001,techlabora002 -r 12.1 -fixup -verbose
Check the output for any tests which didn’t pass. I initially had an issue where it wouldn’t start at all in the below screenprint

Fixed by adding the hostname in /etc/hostname. for some reason it has unknown:00:50:56 etc
So a correct view is below

Next log into your Node 1 as the grid user
Open a terminal session
Navigate to /tmp/grid/
Type ./runInstaller

The Oracle Installer will now run
Select the “Install and Configure Oracle Grid Infrastructure for a Cluster” option, then click the “Next” button.

Choose Configure a Standard cluster

Choose Typical

Put in the correct SCAN name and public and virtual addresses

Click Add and put in your second node details

You should now see both your nodes

Click SSH Connectivity and test it. It should come back with the below screen

Check the public and private networks are specified correctly. If the NAT interface is displayed, remember to mark it as “Do Not Use”. Click the “Next” buttonCheck the network interfaces

Enter /u01/app/oracle as the Oracle Base
Enter “/u01/app/12.1.0.2/grid” as the software location and “Automatic Storage Manager” as the cluster registry storage type. Enter the ASM password, select “dba” or “asmadmin” as the group and click the “Next” button.

I left my disk group name as CRS
Set the redundancy to “External” and select your disks

NORMAL REDUNDANCY – Two-way mirroring, requiring two failure groups.
HIGH REDUNDANCY – Three-way mirroring, requiring three failure groups.
EXTERNAL REDUNDANCY – No mirroring for disks that are already protected using hardware RAID or mirroring

If you don’t see your disks straight away then click the “Change Discovery Path” button and set the path to “/dev/oracleasm/disks/*” OR ORCL:*. (See a few screenprints down after the troubleshooting)
I selected just my CRS disks as I will create another disk group later

Also I found in one installation I tried the disks just wouldn’t appear because I had run through the installer once and it had failed but used the disks, I needed to delete and recreate my disks as per below which was a pain but at least easy to rectify

Back to the ASM Disk Group wizard
Accept the default inventory

If you want the root scripts to run automatically, enter the relevant credentials
Or run them manually
run the /u01/app/oraInventory/orainstRoot.sh
and /u01/app/11.2.0/grid/root.sh scripts

You will now see the pre-requisites being checked

You will see I have some errors
Some are fixable, some will just need to be investigated and then you can run a check again (Note: After taking this screenprint I have added all the steps to stop these messages in the steps previously!)

Once the pre-requisites are complete, click Next and check the summary

It should then run through the install

It will stop and ask you about running scripts – Say yes

You can click Details on the installer screen to see what is happening
Highlighted in yellow is the install log which you can use in the event of any issues

Once it has completed, you should see the below screen

Next we can run some commands to check the cluster

CRSCTL

CRSCTL is an interface between you and Oracle Clusterware, parsing and calling Oracle Clusterware APIs for Oracle Clusterware objects.

Oracle Clusterware 11g release 2 (11.2) introduced cluster-aware commands with which you can perform check, start, and stop operations on the cluster. You can run these commands from any node in the cluster on another node in the cluster, or on all nodes in the cluster, depending on the operation.

You can use CRSCTL commands to perform several operations on Oracle Clusterware, such as:

Starting and stopping Oracle Clusterware resources
Enabling and disabling Oracle Clusterware daemons
Checking the health of the cluster
Managing resources that represent third-party applications
Integrating Intelligent Platform Management Interface (IPMI) with Oracle Clusterware to provide failure isolation support and to ensure cluster integrity
Debugging Oracle Clusterware components

Testing the cluster

type /u01/app/12.1.0.2/grid/bin/crsctl status resource -t

Check for cluster nodes
Type /u01/app/12.1.0.2/grid/bin/olsnodes -n

Check the CRS status on the first node as the grid user
Type /u01/app/12.1.0.2/grid/bin/crsctl check crs

Check for cluster registry (OCR)
Type /u01/app/12.1.0.2/grid/bin/ocrcheck

Check voting disk
Type /u01/app/12.1.0.2/grid/bin/crsctl query css votedisk

The number of voting disks in ASM is derived by the redundancy of the diskgroup.
1 Vote disk with external redundancy, 3 with normal and 5 with high redundancy.

Usefu Cluster Troubleshooting Link

https://www.toadworld.com/platforms/oracle/b/weblog/archive/2014/01/28/troubleshooting-oracle-clusterware-common-startup-failures

Next Oracle Database 12c software installation

Useful video link

http://www.oracle.com/webfolder/technetwork/tutorials/demos/db/11g/r2/grid_rac/05_asm_asmca/asm_asmca_viewlet_swf.html

Copy the database files into the Oracle Linux /tmp folder and unzip them

cd /tmp/database
unzip V46095-01_1of2.zip -d /tmp/database
unzip V46095-01_2of2.zip -d /tmp/database

In this section, we will create two additional ASM disk groups using the ASM Configuration Assistant ( asmca). These new ASM disk groups will be used later in this guide when creating the clustered database
Log into Node 1 as grid
Type cd /u01/app/12.1.0.2/grid/bin/asmca and you should now get the below box

From the disk group tab click Create
Put in a name for your Disk Group
Select disk/disks

You should see the confirmation message

Next create the FRA disk group
From the disk group tab click Create
Put in a name for your Disk Group – FRA
Select disk/disks

Make sure you get a confirmation message

You should now see all your disks

Check the cluster services are up before starting the installer
Type /u01/app/12.1.0.2/grid/bin/crsctl status resource -t

Log into the first RAC node as the oracle user and start a terminal session
Type cd /tmp/database/database
./runInstaller

The installer should start

Uncheck the security updates checkbox and click the “Next” button and “Yes” on the subsequent warning dialog.

Select Install Database software only

Accept the “Oracle Real Application Clusters database installation” option by clicking the “Next” button.

Make sure both nodes are selected

Choose your language

Choose your database edition

Enter “/u01/app/oracle” as the Oracle base and “/u01/app/oracle/product/12.1.0.2/db_1” as the software location, then click the “Next” button

It will then run through pre-requisite checking

I got one error to fix

I clicked fix and check again and got the below screen

So I ran the below on both nodes

Then clicked OK on the message and it worked fine and took me to the next screen below

It will then run through the installer
You can click on details to see where it up to

During the install the following message will pop up

I putty’d into each node as root and ran /u01/app/oracle/product/12.1.0.2/db_1/root.sh

Go back to the installer and click on OK
Hopefully now your installation should be successful

Creating a new database

Oracle Link to follow and for explanations and descriptions

http://docs.oracle.com/cd/B16276_01/doc/server.102/b14196/install003.htm

Log into Putty as oracle
You can type dbca or type the path to the runInstaller (instructions paths are below but I used dbca)

If you get the below message you will need to modify the permissions on the /u01/app/oracle/cfgtoollogs/dbca folder

At this point I have found you also need to change the permissions on the /u01/app/oracle/admin folder on both nodes or you will get to the end of the wizard and it will fail straightaway with permissions

You will now see the below screen
Choose Advanced Mode then we can see all the options you get

You then see the below screen

Administrator-managed: Database administrators define on which servers a database resource should run, and place resources manually as needed. This is the management strategy used in previous releases.
Policy managed: Database administrators specify in which server pool (excluding generic or free) the database resource will run. Oracle Clusterware is responsible for placing the database resource on a server.
Put in cdbrac for the name

Good Link for understanding Server Pools

An introduction to Policy Managed Databases in 11.2 RAC

You can run the following command to check what server pools there are
/u01/app/oracle/product/12.1.0.2/db_1/bin/srvctl config srvpool

You will now be at the below screen

Put in a Global Database Name – In my case cdbrac.techlab.local
The SID prefix is cdbrac
Select Create as Container Database
Select Create a Container database with one or more PDBs and name it pdb1
On the Database placement, select Nodes

Next choose your management options

Next set your database credentials
I chose to use the same for all accounts

Click Next
The next screen sets Storage Locations

If you click on File Location variables you will see the below screen

On the Database Schemas page, just leave this as it is

In the Initialisation Parameters window for memory, select Automatic memory management

In the Sizing tab, keep the default

For character sets, just keep the defaults although read the below
Oracle recommends using Unicode for all new system deployments. Migrating legacy systems to Unicode is also recommended. Deploying your systems today in Unicode offers many advantages in usability, compatibility, and extensibility. Oracle Database enables you to deploy high-performing systems faster and more easily while utilizing the advantages of Unicode. Even if you do not need to support multilingual data today, nor have any requirement for Unicode, it is still likely to be the best choice for a new system in the long run and will ultimately save you time and money as well as give you competitive advantages in the long term