gmane.linux.highavailability.user

Problem with crm shadow CIB's

Tony Stocker — 2013-05-22T13:20:06

Version Info:
OS:                     CentOS 6.4
Kernel (current):       2.6.32-358.6.2.el6.x86_64
Pacemaker:              1.1.8-7.el6
Corosync:               1.4.1-15.el6_4
CRMSH:                  1.2.5-55.4


I was attempting to create a shadow CIB so that I could test changes and 
I am unsuccessful getting it to work:

# crm cib
crm(live)cib# new test-conf
INFO: 8: test-conf shadow CIB created
ERROR: 8: test-conf: no such shadow CIB
crm(live)cib# reset test-conf
INFO: 9: copied live CIB to test-conf
crm(live)cib# list
crm(live)cib# new test-conf
A shadow instance 'test-conf' already exists.
   To prevent accidental destruction of the cluster, the --force flag is 
required in order to proceed.
crm(live)cib# use test-conf
ERROR: 12: test-conf: no such shadow CIB
crm(live)cib# delete test-conf
INFO: 13: test-conf shadow CIB deleted

So it appears to be there at some level since I can copy the live CIB, and 
it complains if I try to create the same name again, and I can delete it. 
But I cannot see it via 'l

Confirmation about resource group ordering

Tony Stocker — 2013-05-22T12:15:00

I just want to make sure that my understanding of resource groups is 
correct before I proceed too far.  Are the following true statements 
regarding resource groups:

 1. The order in which resource primitives are listed in a resource 
group is the order in which they will be started?

 2. The order in which resource primitives are listed in a resource
group is the inverse of the order in which they will be stopped?

 3. Resource primitives in a resource group are started 
sequentially, not in parallel?

 4. Even if one of the primitives in the group fails to start, 
subsequent primitives will still be launched?

 5. Ordering constraints outside of the group are not applied to 
resource group members?


For example, given the following resource group definition:

group HACMASTER HACMASTER-JOBFILE HACMASTER-PWFILE HACMASTER-JAVA 
HACMASTER-SSHKEYS HACMASTER-EXT-IP HACMASTER-BAK-IP HACMASTER-COMM-IP 
HACMASTER-STOR-IP \
    meta target-role="Started"

The 8 resource primitives listed will always be star

Re: Antw: Using crm to configure a rule

Tony Stocker — 2013-05-22T11:34:08


Lars,

Thanks so much!  I was using the documentation as a guide so that's why I 
had the hash mark since I didn't realize that is was reserved for special 
attributes.  Combined with upping the scores for the constraints 
for the resource group, this seems to be working at the moment.

Thanks again very much.

Tony

Re: Unable to Configure Heartbeat on RHEL-6.4 64 Bit

Lars Marowsky-Bree — 2013-05-22T08:16:54


Just for completeness, both DRBD and Pacemaker+corosync are fully
supported on SUSE Linux Enterprise Server, and have been for years.

That boils down to a question of support priorities - is the base
enterprise distribution more important than the specific HA features.


Regards,
    Lars

Re: Unable to Configure Heartbeat on RHEL-6.4 64 Bit

Andrew Beekhof — 2013-05-22T04:58:22

On 22/05/2013, at 2:23 PM, Digimer <lists< at >alteeve.ca> wrote:


For RHEL6, "2." should be "corosync + cman + pacemaker"


Since 6.4 there is a reasonable chance that fixes to serious bugs affecting Pacemaker + CMAN will qualify for a z-stream.


This was only true for those not using Pacemaker with CMAN.


_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Re: Unable to Configure Heartbeat on RHEL-6.4 64 Bit

Digimer — 2013-05-22T04:23:35

Heartbeat is deprecated. It's not been developed in some time (though 
Linbit supports existing installs).

You have two options here;

1. Red Hat Cluster Suite (corosync + cman + rgmanager)

2. Pacemaker (corosync + pacemaker)

There are pros and cons to both, and you need to choose which is best 
for you.

1. Is the only* officially supported (by Red Hat) HA stack. This means 
that, through the life of RHEL (and thus CentOS), upgrades to the OS 
will never break your cluster.

2. Pacemaker is the future of HA clustering. It is expected that, in 
RHEL 7, Pacemaker will replace RHCS. So there is a strong argument to 
start with pacemaker now.

The main problem that might push you away from Pacemaker (and what keeps 
me personally on RHCS) is that Pacemaker is in "Tech Preview" mode in 
RHEL. What this means is two fold;

a) The pacemaker package does not get updated except with y-stream 
releases (6.0 -> 6.1 -> 6.2 -> etc). Even if bugs are found between 
releases, no updated package will be shipped.

b) Ea

Unable to Configure Heartbeat on RHEL-6.4 64 Bit

Deepak Yadav — 2013-05-22T03:59:21

Dear Team


I want  to configure the DRBD and Heartbeat on RHEL-6 OS but I am not able
to find out exact package of heartbeat for the installation.
Please help me out to resolve this issue !

Re: LVM Resource agent, "exclusive" activation

David Vossel — 2013-05-22T02:36:59

----- Original Message -----

yes, we are on the same page now.  I am in favor of this approach.

I am still confused about Jonathan's comment though when you proposed this solution...
"The above would not quite be sufficient.  You would still have to change the 'volume_list' field in lvm.conf (and update the initrd)."

Why would this be necessary?  I think I'm missing something here.

Re: Sockets Missing from /var/run/heartbeat

Wilson, Christopher (IT — 2013-05-21T23:47:36

Lars,

I found a post from someone with almost the same problem as myself. They claim to have tracked it down to a multicast problem and a shutdown and reboot of both systems resolved the problem. As this system is in production I am unable to try this until either a scheduled outage in the coming month or an unexpected failure of the current and only active node ;-) I will update this thread at the end of June to let everyone know how it went 

Thanks

Chris Wilson

On May 21, 2013, at 6:14 PM, "Lars Ellenberg" <lars.ellenberg< at >linbit.com> wrote:

_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Re: LVM Resource agent, "exclusive" activation

Lars Ellenberg — 2013-05-21T22:58:05

No.  I suggested to NOT set that "pacemaker" tag in the config (lvm.conf),
but only ever explicitly set that tag from the command line as used from
the resource agent ( --config "tags { pacemaker {} }" )

That would also mean to either override volume_list with the same
command line, or to have the tag mentioned in the volume_list in
lvm.conf (but not set it in the tags {} section).


You are right, we only have to make sure we don't break existing setup
by rolling out a new version of the RA.  So if the resource agent
won't "accidentally" use a code path where support of a new feature
(of LVM) would be required, that's "good enough" compatibility.

Still it won't hurt to pick the most "compatible" implementation
of several possible "equivalent" ones (RA-feature wise).

I think the proposed --config "tags { pacemaker {} }"
is simpler (no retagging, no re-writing of lvm meta data),
and will work for any setup that knows about tags.

(and I still think the RA should not try to second guess pacemaker and
doubl

Re: Sockets Missing from /var/run/heartbeat

Lars Ellenberg — 2013-05-21T22:14:20

 :-(


:-((


If you start heartbeat,
after some initial timeouts,
it should log "Comm_now_up(): updating status to active",
immediately after that it will try to unlink -- if exist --
and recreate those sockets.
If it fails to create the sockets, it will abort.

So if you have a running heartbeat master control process,
it has been able to create those sockets during its startup.

Unless, well, "your" heartbeat is different than "my" heartbeat.
In which case you need to either use my heartbeat,
or go to those that screwed up yours ;-/


Maybe you have masked the sockets by mounting a tmpfs over?

If you start heartbeat first, then mount -t tmpfs tmpfs /var/run/
later, obviously the sockets will no longer be found...

Or a later "cleanup" by some init script or misguided daemon
did rm -rf /var/run/* ?

What does "lsof -p <pid of heartbeat master control process>" say?

Lars

Re: LVM Resource agent, "exclusive" activation

David Vossel — 2013-05-21T21:52:39

----- Original Message -----

wait, did we just go around in a circle.  If we add "pacemaker" to the volume list, and use that in every cluster node's config, then we've by-passed the exclusive activation part have we not?!

Also, we're not happy with the auto_activate list because it won't work with old distros?!  It's a new feature, why do we have to work with old distros that don't support it?

Re: Antw: Using crm to configure a rule

Lars Ellenberg — 2013-05-21T21:49:11

You use "class" above.
So lose the hash mark:
           rule $id="hacmaster_prefer_b-rule" 1000: class eq B

The "#uname" attribute is a special, implicit one...

Re: Virtual interface (eth0:0) disappeared

DaveW — 2013-05-21T16:12:15

On 05/21/2013 12:13 AM, Alan Robertson wrote:
Exactly what happened.  You're right about human-induced problems. This 
server ran for about a year and a half without so much as a reboot 
before its first failover, which was when I accidentally turned off the 
UPS on the main system.  The failover was so smooth and fast that by the 
time I hit refresh on the browser that was already displaying a database 
query, it was already there.  No one in the factory even knew it failed 
over except me.  I really hate to mess with that kind of stability.  
Upgrading at this time is not in the cards.  I already have monitoring 
programs that watch all my services.  I naively assumed this was an 
integral part of the system and that heartbeat would have handled it, 
but now I see that ipfail is just another service that needs to be 
monitored and I will add that to my system.  Thanks for the info.


_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org

Re: Antw: Using crm to configure a rule

Tony Stocker — 2013-05-21T14:20:46



Okay, let me back up a bit.  Maybe you can help me answer the following 
questions which sort of underlie what I'm trying to do:


 1. Can a location constraint apply to a resource group?  Or must 
it apply only to resource primitives?


 2. Can a location constraint rule apply to a node parameter other 
than #uname?



I've tried creating one using crmsh but it doesn't seem to be working as I 
anticipate:

# crm configure show
node gpmhac01 \
         attributes class="A" kernel="2.6.32-358.6.2.el6.x86_64"
node gpmhac06 \
         attributes class="A" kernel="2.6.32-358.6.2.el6.x86_64"
node gpmhac09 \
         attributes class="B" kernel="2.6.32-358.6.2.el6.x86_64"
node gpmhac10 \
         attributes class="B" kernel="2.6.32-358.6.2.el6.x86_64"
node gpmhac11 \
         attributes class="B" kernel="2.6.32-358.6.2.el6.x86_64"
node gpmhac12 \
         attributes class="B" kernel="2.6.32-358.6.2.el6.x86_64"
primitive HACMASTER-BAK-IP ocf:heartbeat:IPaddr2 \
         params ip="192.168.1.179" \
         op

Re: Antw: Using crm to configure a rule

Ulrich Windl — 2013-05-21T13:51:00

Nachricht
<alpine.LRH.2.03.1305211310530.10201< at >tf6102xuryqne.ccf.rbfqvf.anfn.tbi>:

Hi!

Maybe that's my fault: "system health" is the term that you should look for ("info HealthSMART", "node-health-strategy").
I don't think the crm can write corresponding rules without using XML.

Regards,
Ulrich



_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Re: Antw: Using crm to configure a rule

Tony Stocker — 2013-05-21T13:21:07


I can't find any references to "node coloring" in the documentation that I 
have, or through a quick search online so I don't know if that applies to 
my situation.


What I am trying to do is to have a resource group prefer to run on a 
specific class of node over other classes of nodes by using a special node 
attribute that I define for each node.  I want to do this so that I can 
avoid setting a location constraint score for each individual node, and 
instead provide a location constraint score for each 'class' of nodes - 
based on the node attribute value.  But I do NOT want the resource group 
excluded from running on other classes of nodes, I just want it to prefer 
a specific class.

I **think** what I need would look like this in XML, based on Example 8.9 
from the document previously mentioned in my original post:

<rsc_location id="hacmaster-prefers-b" rsc="HACMASTER">
    <rule id="prefer-b-rule" score="1000">
       <expression id="prefer-b-expr" attribute="#class"
        operation="eq" value

Antw: Using crm to configure a rule

Ulrich Windl — 2013-05-21T13:04:15

Hi!

I don't know what you are trying to do, but could "node coloring" do what you want (run resources on nodes with a specific color (red/green/yellow))?

Regards,
Ulrich

Nachricht
<alpine.LRH.2.03.1305211212370.10201< at >tf6102xuryqne.ccf.rbfqvf.anfn.tbi>:



_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Using crm to configure a rule

Tony Stocker — 2013-05-21T12:25:24

Version Info:
OS:                     CentOS 6.4
Kernel (current):       2.6.32-358.6.2.el6.x86_64
Pacemaker:              1.1.8-7.el6
Corosync:               1.4.1-15.el6_4
CRMSH:                  1.2.5-55.4

I'm using a number of documents from clusterlabs.org, specific to this 
question I'm using "Pacemaker-1.1-Pacemaker_Explained-en-US.pdf" and I'm 
in Chapter 8, section 8.4 regarding the use of node parameters to control 
resource location.  This particular document appears to rely almost solely 
on XML snippets for its examples, and in this case these snippets are the 
only examples.  In an online archive of the mailing list for Pacemaker I 
found a similar question from Jan 11, 2010 where the questioner was 
referred to:
 "Or read the crm_cli doc at clusterlabs.org.
 Thanks,
 Dejan "

However the document referred to, "crm_cli doc", no longer appears to be 
hosted on clusterlabs.org.  Does anyone know of its new location?  Or the 
location of a similar document which will explain the correct synta

Re: Virtual interface (eth0:0) disappeared

Alan Robertson — 2013-05-21T07:13:01

This was some kind of human-induced problem.  They don't go away on 
their own.  Doing an ifdown/ifup on the main interface would do it.  If 
you're using DHCP (a really bad idea for an HA server) and it issued a 
new netmask, for the IP then that would probably do it too.

My guess is that someone did the ifdown/ifup to fix the netmask - from 
what you said that would be necessary.  And, that would definitely do it.

Pacemaker would have brought it back up again.  The haresources 
configuration doesn't monitor any services - so it doesn't know if 
they're working or not -- it only monitors servers for up/down status.

It does what it does quite well - and it's very simple to set up.  But 
it doesn't do everything that Pacemaker does.

_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems

Re: Virtual interface (eth0:0) disappeared

Andrew Beekhof — 2013-05-21T06:34:24

On 21/05/2013, at 4:19 PM, Nikita Michalko <michalko.system< at >a-i-p.com> wrote:


In haresources mode, there is very little difference


_______________________________________________
Linux-HA mailing list
Linux-HA< at >lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems