OCFS2 self-fencing during cluster failover on SUSE Linux Enterprise

During a clustered failover or network disruption, all paths to the storage system can become unavailable. As a result, the nodes might fence by panicking. Set the O2CB_HEARTBEAT_THRESHOLD value in /etc/sysconfig/o2cb to a value large enough to ensure that the nodes do not self-fence.

Oracle Cluster File System (OCFS2) has a fencing mechanism that relies on each node being able to write a heartbeat value to a specific area on each OCFS2 volume. Every two seconds, each node writes a new heartbeat value, and reads the value of the other node. If a node is unable to write or read the heartbeat value, it retries the operation a specific number of times, and then self-fences by panicking the Linux kernel as it fails to obtain quorum.

During a clustered failover or network disruption, all paths to the storage system can become unavailable. None of the nodes in the OCFS2 cluster is able to write or read the heartbeat values. As a result, the nodes might fence by panicking.

With a dedicated TCP/IP network for the iSCSI link between Linux hosts and the storage system, a reasonable value usually is 181. This value corresponds to a total of 360 seconds before a node self-fences.

Depending on the load in the iSCSI network, you should raise the value. It is best to use a value in the range of 91 to 201 for a configuration with OCFS2 on multipath, and iSCSI devices.

Clusterizar SQUID con HeartBeat - OpenAis

Preparando el entorno

Doy por supuesto que esta instalado el pattern Cluster en Suse y todos los componentes de Heartbeat.

Preparamos el fichero /etc/hosts con los nodos
Es esencial que todos los miembros del cluster tengan resolución de nombres entre ellos. Si la resolución no funciona, la comunicación interna del cluster fallará.

Doy por supuesto que el provisionamiento de disco iscsi está configurado y accesible desde los nodos.

Configuramos OpenAIS

mediante el fichero /etc/ais/openais.conf en suse con yast cluster

aisexec {
group: root
user: root
}

service {
use_mgmtd: yes
use_logd: yes
ver: 0
name: pacemaker
}

totem {
rrp_mode: none
token_retransmits_before_loss_const: 10
join: 1000
max_messages: 20
vsftype: none
token: 5000
nodeid: 2
consensus: 2500
send_join: 45
secauth: off
version: 2

interface {
bindnetaddr: 192.168.20.0
mcastaddr: 226.94.1.1
mcastport: 5405
ringnumber: 0
}
clear_node_high_bit: no
}

logging {
to_file: no
to_syslog: yes
debug: off
timestamp: on
to_stderr: off
fileline: off
syslog_facility: daemon
}
amf {
mode: disabled
}

rcopenais start

Copiar el archivo de configuración /etc/ais/openais.conf en los demas nodos del cluster, sin olvidar de modificar el nodeID. Para arrancar OpenAIS desde la linea de comando, se puede usar rcopenais start.

Configurar el usuario hacluster.

Abrir YAST e ir a la sección “Security and Users”.
Abrir “User Management”; en la ventana “User and Group Administration”, hacer click en la lista “Set Filter” y seleccionar “System Users”.
Seleccionar el usuario“hacluster” e editarlo. En la nueva ventana indicar la contraseña de su elección e confirmarla.
Cerrar la ventana de “User Management”.

Configuracion del LVM

Editamos /etc/lvm/lvm.conf :

locking_type = 3

primitive dlm ocf:pacemaker:controld
primitive clvm ocf:lvm2:clvmd \
params daemon_timeout="30"
clone dlm-clone dlm \
meta target-role="Started" interleave="true" ordered="true"
clone clvm-clone clvm \
meta target-role="Started" interleave="true" ordered="true"
colocation colo-clvm inf: clvm-clone dlm-clone
order order-clvm inf: dlm-clone clvm-clone


Prepararamos el volumen físico:
pvcreate <physical volume path>

Creamos el volumen logico para el cluster:
vgcreate --clustered y <volume group name> <physical volume path>
lvcreate --name testlv -L 4G <volume group name>

Configuramos el recurso en el Cluster:

primitive vg1 ocf:heartbeat:LVM
params volgrpname="<volume group name>"
clone vg1-clone vg1
meta interleave="true" ordered="true"
colocation colo-vg1 inf: vg1-clone clvm-clone
order order-vg1 inf: clvm-clone vg1-clone

Configuramos el recurso IP:

Configuramos el recurso SQUID:

primitive RECURS_SQUID lsb:squid
operations $id="RECURS_SQUID-operations"
op monitor interval="10s" timeout="30s" start-delay="10s"
params squid_exe="/usr/sbin/squid" squid_conf="/cluster/etc/squid/squid.conf" squid_pidfile="/cluster/var/run/squid.
pid" squid_port="3128"
meta target-role="started"

Vemos como queda:
============
Last updated: Tue Feb 16 12:38:51 2010
Current DC: smc-nid1 - partition with quorum
Version: 1.0.3-0080ec086ae9c20ad5c4c3562000c0ad68374f0a
2 Nodes configured, 2 expected votes
4 Resources configured.
============

Online: [ smc-nid1 smc-nid2 ]

Resource Group: Cluster_SVPROXADM
Cluster_FS (ocf::heartbeat:Filesystem): Started smc-nid1
RECURS_IP (ocf::heartbeat:IPaddr): Started smc-nid1
RECURS_SQUID (lsb:squid): Started smc-nid1
RECURS_DANSGUARDIAN (lsb:dansguardian): Stopped
Clone Set: dlm-clone
Started: [ smc-nid1 smc-nid2 ]
Clone Set: clvm-clone
Started: [ smc-nid1 smc-nid2 ]
Clone Set: vg1-clone
Started: [ smc-nid2 smc-nid1 ]

Referencias: http://www.novell.com/documentation/sle_ha/

Proxima misión... DANSGUARDIAN