oracle 11g ASM磁盘组扩容实验：
该主机为我经常用的测试机，因为为了顺便学习ASM，底层存储使用的是ASM的方式，但是以前规划的时候磁盘组只有4G，由于数据文件增加，导致没有可用的空间。所以，模拟生产环境做了ASM在线扩容的实验。
建议在做之前，对数据库进行备份。
---------------------------------------1.主机和数据库环境---------------------------------------
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
[root@ray ~]# lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: EnterpriseEnterpriseServer
Description: Enterprise Linux Enterprise Linux Server release 5.6 (Carthage)
Release: 5.6
Codename: Carthage
[root@ray ~]#
--------------------------------------2. 存储环境-------------------------------------------------
由于我用的是虚拟机，所以在虚拟机添加硬盘是很容易的，我们添加一块5G大小的磁盘，重启主机，通过下面查询可以看到我们在/dev/sde就是我们所添加的硬盘。
1. [root@ray ~]# ls -l /dev/sd*
brw-r----- 1 root disk 8, 0 Dec 18 21:17 /dev/sda
brw-r----- 1 root disk 8, 1 Dec 18 21:18 /dev/sda1
brw-r----- 1 root disk 8, 2 Dec 18 21:18 /dev/sda2
brw-r----- 1 root disk 8, 3 Dec 18 21:17 /dev/sda3
brw-r----- 1 root disk 8, 4 Dec 18 21:17 /dev/sda4
brw-r----- 1 root disk 8, 5 Dec 18 21:18 /dev/sda5
brw-r----- 1 root disk 8, 16 Dec 18 21:17 /dev/sdb
brw-r----- 1 root disk 8, 32 Dec 18 21:17 /dev/sdc
brw-r----- 1 root disk 8, 48 Dec 18 21:17 /dev/sdd
brw-r----- 1 root disk 8, 49 Dec 18 21:17 /dev/sdd1
brw-r----- 1 root disk 8, 64 Dec 18 21:17 /dev/sde
[root@ray ~]# fdisk /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): q
[root@ray ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 25 200781 83 Linux
/dev/sda2 26 286 2096482+ 83 Linux
/dev/sda3 287 547 2096482+ 82 Linux swap / Solaris
/dev/sda4 548 2610 16571047+ 5 Extended
/dev/sda5 548 2610 16571016 83 Linux
Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 623 5004216 83 Linux
Disk /dev/sde: 5368 MB, 5368709120 bytes ------------------可以看到是5G大小
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
--------------------------------------------3. 绑定物理设备----------------------------------
[root@ray ~]# /bin/raw /dev/raw/raw3 /dev/sde
/dev/raw/raw3: bound to major 8, minor 64
--------------------------------------------4. 配置UDEV规则---------------------------------
[root@ray ~]# vi /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
#
ACTION=="add",KERNEL=="/dev/sdb", RUN+="/bin/raw /dev/raw/raw1 %N",OWNER="grid", GROUP="asmadmin", MODE="0660"
ACTION=="add",KERNEL=="/dev/sdc", RUN+="/bin/raw /dev/raw/raw2 %N",OWNER="grid", GROUP="asmadmin", MODE="0660
#add:
ACTION=="add",KERNEL=="/dev/sde", RUN+="/bin/raw /dev/raw/raw3 %N",OWNER="grid", GROUP="asmadmin", MODE="0660
KERNEL=="raw*", WNER="grid", GROUP="asmadmin", MODE="0660"
chown grid:oinstall /dev/raw/raw*"
~
~
------------------------------------------5. 配置RAW的控制文件---------------------------------
[root@ray ~]# vi /etc/sysconfig/rawdevices
# raw device bindings
# format: <rawdev> <major> <minor>
# <rawdev> <blockdev>
# example: /dev/raw/raw1 /dev/sda1
# /dev/raw/raw2 8 5
/dev/raw/raw1 /dev/sdb
/dev/raw/raw2 /dev/sdc
/dev/raw/raw3 /dev/sde
~
[root@ray ~]# ls -l /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 1 Aug 21 17:28 /dev/raw/raw1
crw-rw---- 1 grid asmadmin 162, 2 Aug 21 17:28 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 3 Aug 21 17:27 /dev/raw/raw3
在上面，我们在屋里层面上的操作已经做完了。下面我们要在数据库层面做。
----------------------------------------DATA扩容----------------------------------
1. 登录到数据库服务器节点1，以grid用户查看ASM磁盘组空间情况：
[root@ray ~]# su - grid
grid@ray:/home/grid>asmcmd
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 4096 1048576 4096 174 0 87 0 N DATA/
注：可以看到，我们的DATA磁盘只有174M的可用容量。
2. 如果在生产环境，磁盘组添加磁盘需要很长的平衡时间，所以，我们先查询asm_power_limit的值，并调整为10；
grid@ray:/home/grid>sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 21 17:30:36 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
SQL> show parameter asm_power_limit;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_power_limit integer 1
SQL> alter system set asm_power_limit=10 scope=both;
System altered.
SQL> show parameter asm_power_limit
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_power_limit integer 10
3. 查看磁盘组使用情况及磁盘名称和路径
SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;
GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ------------------------------ ----------- ---------- ----------
1 DATA MOUNTED 4096 174
SQL> col name for a10
SQL> col path for a30
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
4. 将新增加的磁盘添加到DATA磁盘组：
grid@ray:/home/grid>sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 21 17:31:59 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
注：此处必须以sysasm身份操作，否则报错ORA-15032和ORA-15260，这是oracle 11G ASM磁盘管理上的一个改进。
SQL> alter diskgroup DATA add disk '/dev/raw/raw3';
Diskgroup altered.
5. 查看是否添加完成
SQL> col name for a10
SQL> col path for a30
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
6. 将数据进行冗余
将当前的diskgroup组的rebalance速度修改为10：
SQL> alter diskgroup DATA rebalance power 10;
Diskgroup altered.
查看重新平衡需要的时间：这里已经完了，因为磁盘只有5G。
SQL> select operation,est_minutes from v$asm_operation;
OPERA EST_MINUTES
----- -----------
REBAL 0
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
7. 将参数改回去
SQL> alter diskgroup DATA rebalance power 1;
Diskgroup altered.
SQL> alter system set asm_power_limit=1 scope=both;
System altered.
查询DATA磁盘组容量，可以看到，我们已经增加了5G的容量。
SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;
GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ---------- ----------- ---------- ----------
1 DATA MOUNTED 9216 5233
欢迎大家批评指正：
QQ交流群：300392987
论 坛：www.oraclefreebase.com
该主机为我经常用的测试机，因为为了顺便学习ASM，底层存储使用的是ASM的方式，但是以前规划的时候磁盘组只有4G，由于数据文件增加，导致没有可用的空间。所以，模拟生产环境做了ASM在线扩容的实验。
建议在做之前，对数据库进行备份。
---------------------------------------1.主机和数据库环境---------------------------------------
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
[root@ray ~]# lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: EnterpriseEnterpriseServer
Description: Enterprise Linux Enterprise Linux Server release 5.6 (Carthage)
Release: 5.6
Codename: Carthage
[root@ray ~]#
--------------------------------------2. 存储环境-------------------------------------------------
由于我用的是虚拟机，所以在虚拟机添加硬盘是很容易的，我们添加一块5G大小的磁盘，重启主机，通过下面查询可以看到我们在/dev/sde就是我们所添加的硬盘。
1. [root@ray ~]# ls -l /dev/sd*
brw-r----- 1 root disk 8, 0 Dec 18 21:17 /dev/sda
brw-r----- 1 root disk 8, 1 Dec 18 21:18 /dev/sda1
brw-r----- 1 root disk 8, 2 Dec 18 21:18 /dev/sda2
brw-r----- 1 root disk 8, 3 Dec 18 21:17 /dev/sda3
brw-r----- 1 root disk 8, 4 Dec 18 21:17 /dev/sda4
brw-r----- 1 root disk 8, 5 Dec 18 21:18 /dev/sda5
brw-r----- 1 root disk 8, 16 Dec 18 21:17 /dev/sdb
brw-r----- 1 root disk 8, 32 Dec 18 21:17 /dev/sdc
brw-r----- 1 root disk 8, 48 Dec 18 21:17 /dev/sdd
brw-r----- 1 root disk 8, 49 Dec 18 21:17 /dev/sdd1
brw-r----- 1 root disk 8, 64 Dec 18 21:17 /dev/sde
[root@ray ~]# fdisk /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): q
[root@ray ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 25 200781 83 Linux
/dev/sda2 26 286 2096482+ 83 Linux
/dev/sda3 287 547 2096482+ 82 Linux swap / Solaris
/dev/sda4 548 2610 16571047+ 5 Extended
/dev/sda5 548 2610 16571016 83 Linux
Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 623 5004216 83 Linux
Disk /dev/sde: 5368 MB, 5368709120 bytes ------------------可以看到是5G大小
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
--------------------------------------------3. 绑定物理设备----------------------------------
[root@ray ~]# /bin/raw /dev/raw/raw3 /dev/sde
/dev/raw/raw3: bound to major 8, minor 64
--------------------------------------------4. 配置UDEV规则---------------------------------
[root@ray ~]# vi /etc/udev/rules.d/60-raw.rules
# Enter raw device bindings here.
#
# An example would be:
# ACTION=="add", KERNEL=="sda", RUN+="/bin/raw /dev/raw/raw1 %N"
# to bind /dev/raw/raw1 to /dev/sda, or
# ACTION=="add", ENV{MAJOR}=="8", ENV{MINOR}=="1", RUN+="/bin/raw /dev/raw/raw2 %M %m"
# to bind /dev/raw/raw2 to the device with major 8, minor 1.
#
ACTION=="add",KERNEL=="/dev/sdb", RUN+="/bin/raw /dev/raw/raw1 %N",OWNER="grid", GROUP="asmadmin", MODE="0660"
ACTION=="add",KERNEL=="/dev/sdc", RUN+="/bin/raw /dev/raw/raw2 %N",OWNER="grid", GROUP="asmadmin", MODE="0660
#add:
ACTION=="add",KERNEL=="/dev/sde", RUN+="/bin/raw /dev/raw/raw3 %N",OWNER="grid", GROUP="asmadmin", MODE="0660
KERNEL=="raw*", WNER="grid", GROUP="asmadmin", MODE="0660"
chown grid:oinstall /dev/raw/raw*"
~
~
------------------------------------------5. 配置RAW的控制文件---------------------------------
[root@ray ~]# vi /etc/sysconfig/rawdevices
# raw device bindings
# format: <rawdev> <major> <minor>
# <rawdev> <blockdev>
# example: /dev/raw/raw1 /dev/sda1
# /dev/raw/raw2 8 5
/dev/raw/raw1 /dev/sdb
/dev/raw/raw2 /dev/sdc
/dev/raw/raw3 /dev/sde
~
[root@ray ~]# ls -l /dev/raw/raw*
crw-rw---- 1 grid asmadmin 162, 1 Aug 21 17:28 /dev/raw/raw1
crw-rw---- 1 grid asmadmin 162, 2 Aug 21 17:28 /dev/raw/raw2
crw-rw---- 1 grid asmadmin 162, 3 Aug 21 17:27 /dev/raw/raw3
在上面，我们在屋里层面上的操作已经做完了。下面我们要在数据库层面做。
----------------------------------------DATA扩容----------------------------------
1. 登录到数据库服务器节点1，以grid用户查看ASM磁盘组空间情况：
[root@ray ~]# su - grid
grid@ray:/home/grid>asmcmd
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 4096 1048576 4096 174 0 87 0 N DATA/
注：可以看到，我们的DATA磁盘只有174M的可用容量。
2. 如果在生产环境，磁盘组添加磁盘需要很长的平衡时间，所以，我们先查询asm_power_limit的值，并调整为10；
grid@ray:/home/grid>sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 21 17:30:36 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
SQL> show parameter asm_power_limit;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_power_limit integer 1
SQL> alter system set asm_power_limit=10 scope=both;
System altered.
SQL> show parameter asm_power_limit
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_power_limit integer 10
3. 查看磁盘组使用情况及磁盘名称和路径
SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;
GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ------------------------------ ----------- ---------- ----------
1 DATA MOUNTED 4096 174
SQL> col name for a10
SQL> col path for a30
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
4. 将新增加的磁盘添加到DATA磁盘组：
grid@ray:/home/grid>sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Thu Aug 21 17:31:59 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Automatic Storage Management option
注：此处必须以sysasm身份操作，否则报错ORA-15032和ORA-15260，这是oracle 11G ASM磁盘管理上的一个改进。
SQL> alter diskgroup DATA add disk '/dev/raw/raw3';
Diskgroup altered.
5. 查看是否添加完成
SQL> col name for a10
SQL> col path for a30
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
6. 将数据进行冗余
将当前的diskgroup组的rebalance速度修改为10：
SQL> alter diskgroup DATA rebalance power 10;
Diskgroup altered.
查看重新平衡需要的时间：这里已经完了，因为磁盘只有5G。
SQL> select operation,est_minutes from v$asm_operation;
OPERA EST_MINUTES
----- -----------
REBAL 0
SQL> select name,path from v$asm_disk;
NAME PATH
---------- ------------------------------
DATA_0001 /dev/raw/raw2
DATA_0000 /dev/raw/raw1
DATA_0002 /dev/raw/raw3
7. 将参数改回去
SQL> alter diskgroup DATA rebalance power 1;
Diskgroup altered.
SQL> alter system set asm_power_limit=1 scope=both;
System altered.
查询DATA磁盘组容量，可以看到，我们已经增加了5G的容量。
SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;
GROUP_NUMBER NAME STATE TOTAL_MB FREE_MB
------------ ---------- ----------- ---------- ----------
1 DATA MOUNTED 9216 5233
欢迎大家批评指正：
QQ交流群：300392987
论 坛：www.oraclefreebase.com
