gmane.os.freebsd.jail

Current problem reports assigned to freebsd-jail-HZy0K5TPuP5AfugRpC6u6w< at >public.gmane.org

FreeBSD bugmaster — 2013-06-17T11:06:46

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.


S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o bin/178302   jail       jail(8): unknown parameter: ip6.addr when kernel compi
o kern/176112  jail       [jail] [panic] kernel panic when starting jails
o kern/176092  jail       [jail] [panic] Starting a jail on my releng/9.1 kernel
o kern/174902  jail       [jail] jail should provide validator for jail names
o kern/174436  jail       [jail] Jails with numbers as names don't work
o bin/173469   jail       [jail] regression: security.jail.sysvipc_allowed=1 no 
o kern/169751  jail       [jail] reading routing information does not work in ja
o bin/167911   jail       new jail(8) problem with removal, ifconfg -

Re: Big problem about ipc on 8.4

Albert Shih — 2013-06-11T14:07:31

 Le 11/06/2013 ? 15:23:00+0200, Albert Shih a écrit

OK I find the solution. 

Just in case some have the same problem.

In FreeBSD 8.4 some changes make you need to add 

    jail_NAME_OF_JAIL_parameters="allow.sysvipc"

in the rc.conf of the host.

Regards.

Big problem about ipc on 8.4

Albert Shih — 2013-06-11T13:23:00

Hi everybody

After I upgrade my server from 8.2 to 8.4 I've got a big problem about
ipcs.

In one of my jail I'm running 

    uwsgi

this software need a security.jail.sysvipc_allowed to be true, so since 8.2 I using

    jail_sysvipc_allow="YES"

inside /etc/rc.conf (on the host). And everything work fine.

After upgrading to 8.4 uwsgi stop working I got this message in the log of
uwsgi
(inside the jail) : 

    emget(): Function not implemented [core/lock.c line 507]
    semctl(): Function not implemented [core/lock.c line 602]

generally meaning some problem with systemvipc. 

I've check on the host : 

    security.jail.sysvipc_allowed: 1

Anyone have a idea ? 

Regards.

Current problem reports assigned to freebsd-jail-HZy0K5TPuP5AfugRpC6u6w< at >public.gmane.org

FreeBSD bugmaster — 2013-06-10T11:06:50

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.


S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o bin/178302   jail       jail(8): unknown parameter: ip6.addr when kernel compi
o kern/176112  jail       [jail] [panic] kernel panic when starting jails
o kern/176092  jail       [jail] [panic] Starting a jail on my releng/9.1 kernel
o kern/174902  jail       [jail] jail should provide validator for jail names
o kern/174436  jail       [jail] Jails with numbers as names don't work
o bin/173469   jail       [jail] regression: security.jail.sysvipc_allowed=1 no 
o kern/169751  jail       [jail] reading routing information does not work in ja
o bin/167911   jail       new jail(8) problem with removal, ifconfg -

Re: Per-IP Bandwidth Monitoring

Bernt Hansson — 2013-06-09T08:28:15

IPFW with pipes. No graphs.

man ipfw

TRAFFIC SHAPER CONFIGURATION
      The ipfw pipe, queue and sched commands are used to configure the 
traffic
      shaper and packet scheduler.  See the TRAFFIC SHAPER (DUMMYNET)
      CONFIGURATION Section below for details.

      If the world and the kernel get out of sync the ipfw ABI may 
break, pre-
      venting you from being able to add any rules.  This can adversely 
effect
      the booting process.  You can use ipfw disable firewall to temporarily
      disable the firewall to regain access to the network, allowing you 
to fix
      the problem.

Re: Per-IP Bandwidth Monitoring

Luke S. Crawford — 2013-06-09T03:10:43

if you only care about IPv4, I've used bandwidthd[1] to great effect; 
it's trivial to setup and easy to use.

But, I need IPv6 support, so this won't work for me anymore.

I've been talking about trying to do something with pmacct[2] but never 
quite got it working properly.




[1]http://bandwidthd.sourceforge.net/
[2]http://www.pmacct.net/


On 06/08/2013 07:32 PM, Kenta Suzumoto wrote:

Re: Per-IP Bandwidth Monitoring

Charles Sprickman — 2013-06-09T03:09:46


This exposes lots of per-jail information via snmp, which makes it fairly easy to graph or query from your monitoring/billing/etc:

http://thewalter.net/stef/software/bsnmp-jails/

It hasn't been updated in some time, so I'm not sure if it will work beyond 8.3.

Charles

Per-IP Bandwidth Monitoring

Kenta Suzumoto — 2013-06-09T02:32:11

Hello. I'm running a FreeBSD machine with 5 IP addresses, each of them attached to a specific jail. I'm wondering if there is an easy way to monitor the bandwidth usage of each of them individually. Upon googling, I ran into a lot of suggestions like bandwidthd. I gave it a try and it seemed very broken and basically didn't work at all. I'm basically looking for a "vnstat that works per IP instead of per interface" kind of thing. jnettop wasn't what I was looking for. It doesn't have to make pretty graphs(but that's nice too), just human-readable text is fine. Anyone have a recommendation?

Some links I came across that were unhelpful: 
http://freebsd.1045724.n5.nabble.com/how-to-measure-bandwidth-per-jail-td5797422.html

http://forum.pfsense.org/index.php?&topic=32256.0

http://www.daemonforums.org/showthread.php?t=1199

Thanks

Re: pf + vimage patch

Nikos Vassiliadis — 2013-06-06T16:47:01

Unloading should be simple in the non-vimage case as well - I think.


Good point. Thank you both for your comments.

Nikos

Re: pf + vimage patch

Nikos Vassiliadis — 2013-06-06T10:35:33

Hi,

Comments below.

On 06/05/2013 10:52 AM, Mikolaj Golub wrote:

I'll try to break it in parts. It should be easy now to break it.
Getting familiar with git will need some time so I'll handle it myself 
this time.


Yes, I wrongly assumed that pf didn't follow style(9) strictly.
Will fix that.


Yes indeed, thanks!


Per VNET variables are not useful for pf_hashsize and pf_srchashsize
since these values are RO and cannot be modified at runtime.


It is indeed ugly. I was trying not to diverse to much from the original
code. I will do it properly.


module unload is broken:( Maybe it can be fixed at a (bit) later date?


will fix these.


Thank you for your comments. I was informed by bz< at > that there
is a security issue with pf being exposed in a jail, so I'll
start from there and then will continue with your comments.

Nikos

Re: pf + vimage patch

Mikolaj Golub — 2013-06-05T08:52:20

Hi,

On Mon, Jun 03, 2013 at 01:58:38PM +0200, Nikos Vassiliadis wrote:

Thank you for working on this. I'd really like to see pf and vimage
integrated, as it looks like one of the main stoppers to have vimage
in GENERIC.

I hoped more knowledgeable people would comment on this
though. Anyway, not being familiar with pf, here is a couple of things
I would recommend to make your patch more attractive for potential
reviewers:

1) It looks like the patch can be split on several parts. A log
message to every change describing why it is needed and what problem
solves would be very helpful. As a tool to maintain such changes I
personally prefer git.

2) You use spaces for indentation, where tabs should be. This adds
unnecessary noise and makes the patch less readable. Also someone
will need to fix this when eventually committing to the tree.

3) When generating diff from svn, adding -x-pu options will make the
diff more readable.

Also see some questions/comments below.


Why do you have to devirtualize these vari

Current problem reports assigned to freebsd-jail-HZy0K5TPuP5AfugRpC6u6w< at >public.gmane.org

FreeBSD bugmaster — 2013-06-03T11:06:47

Note: to view an individual PR, use:
  http://www.freebsd.org/cgi/query-pr.cgi?pr=(number).

The following is a listing of current problems submitted by FreeBSD users.
These represent problem reports covering all versions including
experimental development code and obsolete releases.


S Tracker      Resp.      Description
--------------------------------------------------------------------------------
o bin/178302   jail       jail(8): unknown parameter: ip6.addr when kernel compi
o kern/176112  jail       [jail] [panic] kernel panic when starting jails
o kern/176092  jail       [jail] [panic] Starting a jail on my releng/9.1 kernel
o kern/174902  jail       [jail] jail should provide validator for jail names
o kern/174436  jail       [jail] Jails with numbers as names don't work
o bin/173469   jail       [jail] regression: security.jail.sysvipc_allowed=1 no 
o kern/169751  jail       [jail] reading routing information does not work in ja
o bin/167911   jail       new jail(8) problem with removal, ifconfg -

Re: Cant reach Jailed services from internet.

Fbsd8 — 2013-06-02T17:34:46

Mogamat Abrahams
It's customary to post your solution as the last post in this thread. 
Since you have so many kernel options included it would be nice to know 
which one really made the difference.

BY process of limitation
  nooptions  sctp  problem was fixed in 8.1-release
  device     pf    your not using this firewall
  device     pflog

  options    ACCEPT_FILTER_DATA   These 4 have never been talked about
  options    ACCEPT_FILTER_DNS    before in vnet context. Not likely
  options    ACCEPT_FILTER_HTTP   to have any bearing on your problem.
  options    ZERO_COPY_SOCKETS

Since your problem was happening with both if_bridge/epair and netgraph
vnet networks seems unlikely that
  device   epair
  device   if_bridge
compiled into the kernel has any bearing on your problem.

My money is on
options   MROUTING   # Multicast routing

May I suggest you remove the above kernel options and recompile with 
modules. If it works then you know what kernel option is the solution to 
  a vnet jail receiving inbound

Re: Cant reach Jailed services from internet.

Mogamat Abrahams — 2013-06-02T12:51:54


This address was provided and I manually configured the nic. 


I had no firewall installed on the machine as we were still setting up and 
usually only add firewalling last. Here is something interesting though, 
since compiling a custom kernel and 
including:

device<><------>pf
device<><------>pflog
nooptions<----->sctp
options><------>VIMAGE
device ><------>epair
device ><------>if_bridge
options><------>NULLFS

#firewall

options         MROUTING                # Multicast routing

options         IPFIREWALL              #firewall
options         IPFIREWALL_VERBOSE      #enable logging to syslogd(8)
options         IPFIREWALL_VERBOSE_LIMIT=100    #limit verbosity
options         IPFIREWALL_DEFAULT_TO_ACCEPT    #allow everything by default
options         IPFIREWALL_FORWARD      #packet destination changes

options         ACCEPT_FILTER_DATA
options         ACCEPT_FILTER_DNS
options         ACCEPT_FILTER_HTTP
options         ZERO_COPY_SOCKETS


My JAILS now both receive and respond to traffic! This was

Re: Cant reach Jailed services from internet.

Joe — 2013-05-30T13:49:51

Lets find out about those jail ip addresses. You stated those ip address 
prefixed with 174 were provided by you colo provider.

Questions to ask them. Are those 174.x.x.x ip addresses provisioned or 
said a different way are they true static ip addresses? Read up on the 
difference.

Your 67.205.xx.xx ip address looks like a dynamic ip address that you 
use dhcp to automatically obtain all the network configuration 
information needed by your host. Static ip addresses don't work that 
way. You have to manually configure the static network. If I remember 
correctly, for a block of 3 assignable ip addresses you need a block of 
5 from your provider. The first and last ip address are used to config 
the network.

Best you talk to your provider to find out how those ip addresses are 
configured at their end and how you should config them at your end.


You never said if you have a firewall on your host. The firewall rules 
maybe dropping unsolicited inbound traffic for those 174 prefixed ip 
addresses. Try put

Re: Cant reach Jailed services from internet.

Mogamat Abrahams — 2013-05-30T12:50:56

Added it and not difference.
Yes, name resolution works ok - i can reach out from the jail to other 
services on the internet.

I only have sshd configured on the host, that on the 67. ip address. So I 
assume those listening ports are coming from the jail as its on the same IP 
and ports 80 and 81 

Any other suggestions?

M

Re: Cant reach Jailed services from internet.

Ian Smith — 2013-05-29T14:33:02

 >> That's a worry .. you won't do any good trying to use the broadcast 
 >> address.  Mats is right, you only get 2 usable addresses with a /30.

 > Assigning a /30 for four jails is perfectly valid, if it's an 
 > aggregate of four /32s. I would configure a static route on the 
 > default gateway for 174.x.x.76/30 -> 67.x.x.x, then on the host I'd 
 > assign the four /32s to lo1..lo4. Packets arrive to the jails because 
 > of the /30 static route in the neighbouring router, packets leave the 
 > jail because of the host's already existing default route, and of 
 > course traffic between the jails and the host are OK because the 
 > kernel knows its own interfaces. (Actually that's how I run my 
 > FreeBSD jails.)

 > Regards,
 > András

Ok, thanks, that's interesting.  Maybe I can squeeze more from my /29 ..

cheers, Ian

Re: Cant reach Jailed services from internet.

Joe — 2013-05-29T12:40:35

Do you have   gateway_enable="YES"  statement in the host's rc.conf?

Is the jails /etc/resolv.conf populated with the correct info?

You said "Netstat on the host and jail also show services
listening on those addresses on the correct ports."

If what you mean is the host has processes listening on the SAME
ip address / ports as the jails are listening on, then your jails
will never get any unsolicited traffic because the host always gets
access to that traffic first and processes it without the jail ever 
knowing about it.

Re: Cant reach Jailed services from internet.

Mark Felder — 2013-05-29T11:39:49

On Wed, 29 May 2013 02:18:43 -0500, Mogamat Abrahams <lists-rTf2jDP+83gQZ7m9OUuf1w< at >public.gmane.org>  
wrote:


VIMAGE is still considered experimental and may or may not cause you  
grief. I believe it has major issues with PF as well unless some fixes  
made it into 9.1-RELEASE which I don't believe they did. In fact, the  
fixes someone was working on might only be in HEAD.

YMMV

Re: Cant reach Jailed services from internet.

Mogamat Abrahams — 2013-05-29T07:18:43

The plot thickens!

Running tcpdump on the host, I can see that the packets are received at the 
host on the ip address. Netstat on the host and jail also show services 
listening on those addresses on the correct ports. 
But for some reason the jails are not responding to the packets....... and 
tcpdump does not work inside jails. Are their any other tools that can be used 
to diagnose this?

Compiling a kernel a VIMAGE in the meantime, just in case...

M

Re: Cant reach Jailed services from internet.

Mogamat Abrahams — 2013-05-28T16:25:26

Hi

Thanks for the help thus far. 


From the internet I can reach services on the host which are bound to these 
addresses. Still no luck with the jails.... is there anything else I can to 
to diagnose this?


Talking about routes, i take it these are configured by the kernel?

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default         67.x.x.33          UGS         0     2319    em0
67.x.x.32/27    link#1             U           0        0    em0
67.x.x.57       link#1             UHS         0        0    lo0
127.0.0.1       link#7             UH          0       94    lo0
174.x.x.76      link#1             UHS         0        0    lo0 =>
174.x.x.76/32   link#1             U           0        0    em0 =>
174.x.x.76/30   link#1             U           0        0    em0
174.x.x.77      link#1             UHS         0       28    lo0 =>
174.x.x.77/32   link#1             U           0        0    em0
174.x.x.78      link#1             UHS         0        0    lo0