Starting an Oracle instance & Hugepages
(No Ratings Yet)
Loading ...Hugepages are a feature of Linux to increase the memory page size so instead of allocating memory in 4Kb blocks it gets done in 2MB chunks, other OS’s have similiar features. In theory this should enable you to have better scaling with large memory allocations. It is relatively straightforward to set up, just add an entry like the following in the /etc/sysctl.conf file:
vm.nr_hugepages = 5000
This allocates 10GB of memory to be allocated via hugepages. However to get oracle to utilise this memory, I find I have to startup a 10gR2 RAC instance using srvctl:
srvctl start instance -d DATABASE -i INSTANCE
To be clear, when starting up the RAC instance via sqlplus the memory for the instance DOES NOT get allocated from the hugepages pool, only when you startup with srvctl. The trouble starts when you are doing an upgrade, say installing a patchset, you have to startup the database with the migration option to apply the changes to the data dictionary. Except you cannot use srvctl to startup upgrade, though it does accept various options, like mount & nomount.
Be careful if you are using hugepages that if you startup your instance with just using sqlplus you may find yourself out of memory, I found I had to change the memory & the number of processes to get the instance to start for the upgrade.
April 6th, 2007 at 7:47 pm
Is there a technical reason why srvctl causes Oracle to use hugepages and sqlplus does not?
April 10th, 2007 at 9:41 am
hello,
no, I have attempted to use strace to try and find out what difference is happening between the two but to no avail. I really wish I knew, i have also not tested this with a non-rac installation. Maybe sqlplus works for non-rac?
jason.
May 16th, 2007 at 11:20 pm
I have seen the same behaviour on 10g R2 RAC / Redhat 4 on Itanium. We noticed that the huge pages were not being used on this particular instance, and that excessive swapping was occuring.The huge pages could be allocated by a simple C program with a shmget call with the correct flags. We suspected perhaps a privilege problem. It may be that srvctl runs as root or has a suid root executable that can create the required shared memory area, whereas sqlplus may not call any suid executables and is run as oracle, and hence not make the required shared areas. I further suspect that the call to create the shared memory area with huge pages fails silently, and is retried without the hugepages flag. Thanks for the hint
May 18th, 2007 at 2:49 pm
well, i’m not really sure about suid as looking at srvctl executable it’s a shell script that calls some interminable java stuff.
I also tried setting the group of hugepages shm to be the following in sysctl.conf:
vm.hugetlb_shm_group = 2000
this don’t seem to help!
jason.
August 1st, 2007 at 4:53 pm
Definitely a permissions issue - works in sqlplus as OS user oracle but not any other user.
August 2nd, 2007 at 10:59 am
hello Rich,
Not sure about that. I have attempted to start as user oracle connected as sysdba and it does not use hugepages - only with srvctl. This is a RAC install on RedHat 4 U3. using Oracle 10.2.
jason.
August 2nd, 2007 at 6:34 pm
64-bit 10.2.0.3 RAC on RH 4 U3 x86_64 works through SQL*Plus as OS user oracle [group dba].
I don’t have any 32-bit installations that use hugepages…