gmane.os.openbsd.pf

Re: Simultaneous CARP failover for multiple interfaces

Kyle Lanclos — 2012-04-27T18:05:47

I quoted from the OpenBSD FAQ:

I re-did my tests this morning, and yes indeed, both CARP interfaces in
the CARP group switch automatically between the BACKUP and MASTER states
when I do abusive things to one of the connections. I say "automatically"
instead of "simultaneously" because there was a bit of a delay while the
reconnected link negotiated speed. I blame Cisco for that.

Thanks for all the assistance, I found this exchange to be quite helpful.

--Kyle

Re: Simultaneous CARP failover for multiple interfaces

Daniel Hartmeier — 2012-04-24T06:57:36


Without any active probing, like with ifstated, there is no way to
distinguish which uplink is "up but not forwarding". It could be either
the master's, or the backup's, or both. Statistically, for every time you
improve the situation by failing over, there is a time you shoot yourself
in the foot doing the same. If you do nothing, you have the same chances,
and things remain simpler.

With ifstated, you only need to change one side's advskew so their order
reverses, then rely on carp's election process. For instance, run
ifstated only on the master, pinging next hops on all sides, and

 - when any ping fails (the first time), demote:
     increase own advskew above the backup's

 - when all pings succeed (again), promote:
     reset own advskew to the original value (below the backup's)

In your example above, ifstated on the master would detect a ping
failure on one next hop and demote by increasing its advskew above the
backup's.

With preempt enabled, the master would lose election on the other
interfa

Re: Simultaneous CARP failover for multiple interfaces

Kyle Lanclos — 2012-04-23T21:34:31

I originally wrote:

It sounds like using 'defer' may allow pf + pfsync to handle the issues
resulting from asymmetric routing of packets, as long as the asymmetry
is fully contained within the pfsync'd hosts.

I apologize if I gave too much airtime to the pf + pfsync aspects of
what I was trying to resolve, we largely worked around those by enabling
carp preemption.

--Kyle

Re: Simultaneous CARP failover for multiple interfaces

Kyle Lanclos — 2012-04-23T21:23:20

Whoops, thanks for pointing that out. I will re-do my tests (the setup is
at a remote site), and make sure that I'm done making things up.

I'm not sure where your quote came from; I see similar text in NetBSD docs,
but this is what I (now) find in the OpenBSD FAQ:

net.inet.carp.preemptAllow hosts within a redundancy group
that have a better advbase and advskew
to preempt the master. In addition, this
option also enables failing over a group
of interfaces together in the event that
one interface goes down. If one physical
CARP-enabled interface goes down, CARP
will increase the demotion counter,
carpdemote, by 1 on interface groups that
the carp(4) interface is a member of, in
effect causing all group members to fail-
over together. 

http://www.openbsd.org/faq/pf/carp.html

I did establish an interface group for my two CARP interfaces, but I did
not do my failover tests while it was in that state. As I said, I clearly
need to re-do my tests.


At least for our

Re: Simultaneous CARP failover for multiple interfaces

Karl O. Pinc — 2012-04-23T21:20:56


I didn't notice _any_ reference to pfsync in the original
post.  Perhaps this is part of the problem?



Karl <kop< at >meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: Simultaneous CARP failover for multiple interfaces

Stuart Henderson — 2012-04-23T20:19:44

pfsync(4)'s "defer" option might help. there is a penalty but it might
be acceptable for your use case.

     Where more than one firewall might actively handle packets, e.g. with
     certain ospfd(8), bgpd(8) or carp(4) configurations, it is beneficial to
     defer transmission of the initial packet of a connection.  The pfsync
     state insert message is sent immediately; the packet is queued until
     either this message is acknowledged by another system, or a timeout has
     expired.  This behaviour is enabled with the defer parameter to
     ifconfig(8).

Re: Simultaneous CARP failover for multiple interfaces

Daniel Hartmeier — 2012-04-23T20:56:32


Yes, it will:

     net.inet.carp.preempt       Allow virtual hosts to preempt each other.
                                 It is also used to failover carp interfaces
                                 as a group.  When the option is enabled and
                                 one of the carp enabled physical interfaces
                                 goes down, advskew is changed to 240 on all
                                 carp interfaces.  See also the first example.
                                 Disabled by default.

(i.e. this single sysctl knob enables both group failover and failback)

This covers link state change (unplugged cable) as well as
administrative down of the physical interface.

It does not cover the case where the link remains up, but the uplink
switch stops forwarding, for instance, but...


.. see ifstated(8), which can ping uplink hops and issue ifconfig advskew
changes to demote the master when appropriate.

Daniel

Simultaneous CARP failover for multiple interfaces

Kyle Lanclos — 2012-04-23T18:49:14

I have a pair of OpenBSD firewall/routers in a reasonably vanilla
pf + pfsync + CARP configuration, each straddling two routed networks.
The CARP interface on the internal network is the default gateway for
that subnet. The CARP interface on the external network is the default
destination for traffic aimed at the internal network.

It all works splendidly, with one exception.

In order for our firewall to operate effectively, we use 'keep state'
pf rules. We empirically determined that we must have CARP preemption
enabled, otherwise pf cannot properly establish state for new TCP
connections. If pfsync could be told to synchronize incomplete states,
this issue might go away.

Example: firewall1 is the master on the carp1 interface, and firewall2
is the master on the carp2 interface. Inbound traffic to an internal
host arrives via the carp1 interface, and return traffic arrives via
the carp2 interface. pf will not establish state for this new connection
since the inbound and return traffic are not handled by t

Re: inbound queueing on external interface due to multiple internal interfaces

Karl O. Pinc — 2012-04-12T14:38:01

I am all wet then.  Isn't the first time.  Sorry to distract you.



Karl <kop< at >meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: inbound queueing on external interface due to multiple internal interfaces

Kelley Reynolds — 2012-04-12T12:56:35


I don't know if your setup will support it but you could make a trunk interface with queues then use VLANs instead of physical interfaces, works fine.

Kelley Reynolds
CTO / Cofounder, RubyScale

Re: inbound queueing on external interface due to multiple internal interfaces

Stuart Henderson — 2012-04-12T12:40:47

Yes, that thread is talking about creating multiple queues with the
same name, this is good for traffic classification but it is not a
single queue shared between multiple interfaces.


Currently two machines is the cleanest (and maybe only) way to do this,
I have a feeling you'll get better results making these route rather than
bridge.

altq's queues are all per-interface. Anything to change this would
be major work. Plan is for the queue code to be completely replaced
so working on this before that happens would get in the way.


lo(4) is a bit of a specialist interface to be trying this with.

You might get somewhere with vether(4), the uncommitted diff "virtual
crossover link with vether(4)" on tech< at >openbsd.org could be of interest.

Another option might be to route things to a userland daemon via tun(4)
that reinjects the packets then you can queue on the tun interface
instead. I have some old code from kjc sitting in my inbox that might
be useful here, I'll try and find time to dust it off. Higher ov

Re: inbound queueing on external interface due to multiple internal interfaces

Andy Lemin — 2012-04-12T09:48:52

Hi, thanks very much for your suggestion.


I read through that archive but the problem it seems to be discussing seems=
 pretty trivial as its discussing semantics of queuing downstream traffic w=
hich you just do by queuing on the internal interface (which works perfectl=
y if you only have one internal interface).

It does suggest creating a queue structure without assigning a specific int=
erface as you suggest, however when you do this I believe PF simply creates=
 copy's of the queue structure and creates instances per interfaces? I.e. t=
he different interfaces are not sharing the same physical queue in memory. =
Is this true?

I need to be able to perform downstream queuing pre-routing effectively so =
that all the downstream traffic (regardless of the internal zone) is all qu=
eued equally as one, allowing all internal interfaces to share the total do=
wnstream bandwidth.



Sadly we don't have enough ports/slots on our firewall to add a physical lo=
op back and so it has to be a virtual interface :

Re: inbound queueing on external interface due to multiple internal interfaces

Karl O. Pinc — 2012-04-11T16:48:46

You will want to see this thread:
Working example of bi-directional asymmetric ALTQ + NAT ruleset?
http://marc.info/?t=129472965800001&r=1&w=2

It talks about being able to have a single queue
on more than one interface, so you'd use a single
outbound queue on all your internal interfaces to
effectively rate-limit your inbound wan traffic.
You'll want to use the hfsc scheduler because
you're trading bandwidth for latency.  And hfsc has
sub-queues too so that might help allocate the
traffic per internal interface.  I haven't thought 
about this in quite some time but I think that this approach
will work.  But because I've not thought about
it I could be all wrong.  :-)

It's not a perfect solution; it won't work in the
general case where you've more than one interface
you want to limit inbound traffic on.

If sharing a queue on your internal 
interfaces does not do it you could get ugly and
use an extra 2 real interfaces (instead of the loopback interface
as you suggest) and a separate routing table and phys

inbound queueing on external interface due to multiple internal interfaces

Andy Lemin — 2012-04-11T13:02:41

Hello,
I know this has been discussed before, but I am not a developer (I
wish I did have the skills as I would code this myself otherwise).
I have a huge respect for all that the OpenBSD community does and this
is not a winge or a moan, please hear our woes.


Yes, of course downstream queuing is only of benefit for TCP to get
the sender to slow down etc etc, but that is exactly what we need.
IF we don't queue downstream traffic, our downstream link gets
saturated and our ISP starts to drop random packets. If we do the
downstream queuing here, we pre-emptively drop low priority packets
before saturation thus slowing the sender down before saturating the
WAN's downstream leaving headroom for VoIP etc.


We have a firewall with 4 internal interfaces for various different
subnets and 1 external WAN interface like many users have. Queuing
upload traffic is easy on the WAN interface, but downstream traffic
needs to be queued on each of the internal interfaces.
Initially this would seem fine, until you realise th

CARP ip balancing on ExtremeWare

Kevin Bowling — 2012-02-07T10:03:36

I'm having a hell of a time using Extreme Networks Summit 400-24t
switches with IP balancing of any type.

I've tried OpenBSD 5.0 and a -current snapshot from Feb 02.  I've
tried all the modes, but none of them work.  There's not a good way
I'm aware of to do port mirroring for ip-unicast, but I don't
understand why ip-stealth isn't working.  I manually clear the
forwarding database after activating ip-stealth.

I'm just about to relegate these to dumb switch duty and try and find
some other vendor that just works.  Any chance someone else has
cracked the code on these with pf in the past?

Regards,
Kevin

Re: handling local traffic

Justin Murdock — 2012-01-30T13:23:47

I've been thinking about this for months (on and off) but as soon as I 
send the email a solution occurs to me. Typical.

On 30/01/2012 12:31, Justin Murdock wrote:
I can, however, catch this traffic using
     match out log user != unknown

similarly becomes:
     pass on $dmz to port {http, https, ssh}
     block in user != unknown
     pass in from <trusted> to port ssh user != unknown

I'm not entirely comfortable with this - especially as the "... in ... 
user != unknown" construction depends on there being a listening socket 
- which could be quite interesting with the rdr-to interactions.

handling local traffic

Justin Murdock — 2012-01-30T12:31:55

OpenBSD 4.9 GENERIC.MP#819 amd64

I'm not quite sure when things changed, but I can no longer apply rules 
to locally originating traffic:
     match in log on lo

now only logs local->local traffic and
     match out log received-on lo

logs nothing. The best I can do, it seems, is to
     match in tag "EXTERNAL"
     match out log tagged ""

More worrying for me, however, is the inability to control traffic being 
received by the host. I want to be able to

     pass on $dmz to port {http, https, ssh}
     block out on lo
     pass out on lo from <trusted> to port ssh

I feel I must be missing something, I'm just not sure what.

Re: Matter with transparent proxy

Stuart Henderson — 2012-01-12T15:52:37

There are quite clear instructions for transparent proxy configuration in the
pkg-readme file for the squid package (see /usr/local/share/doc/pkg-readmes).

Matter with transparent proxy

pizzahut — 2012-01-12T14:15:54

Hello,

I try now to create a transparent proxy using squid and using OpenBSD 5.0
Packet Filter all by passing a bridge.

The squid run I tested the bridge walking machines located on the other side
can access the outside.

The problem is that when I try to redirect traffic using packet filter to
127.0.0.1 on port squid listening (port 3128) nothing happens the machine is
on the other side can access the Internet and suffers no restrictions
previously configured in the squid.

If anyone has an idea of the problem please.

thank you

Re: I want copy pf.conf from FreeBSD 8.2 to OpenBSD 5 and use it

Karl O. Pinc — 2011-11-07T14:59:27

Have you looked at the pf faq at www.openbsd.org or the man page,
especially the "Translation" section?

I seem to recall simple nat examples that you could use
once you change the IP numbers/networks to the ones you're
using.

I don't understand your english but it sounds like you may
want "match binat-to" to get a 1-to-1 mapping between internal
and external IP numbers -- either assigned per individual ip
number, or with "bitmask" or perhaps with some other natting
method and "sticky-address".

Karl <kop< at >meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Re: I want copy pf.conf from FreeBSD 8.2 to OpenBSD 5 and use it

Gary — 2011-11-07T16:28:37


If all you want is one-to-one NAT and no packet inspection or filtering, why do you need a firewall at all? It's not good practice for a bunch of desktop Windows OS systems but you might as well just assign them each a public IP. If it were me, I'd start looking for a different chat service or set up my own XMPP server and keep the Windows hosts out of public IP space..

-Gary