gmane.network.tipc.general

[PATCH 0/2] Add IP/UDP bearer support

2013-05-17T09:45:36

From: Erik Hugne <erik.hugne< at >ericsson.com>

I've had this laying around for quite some time now and i figured
it's time to do a push.
The functionality is fairly isolated, the changes done in the core parts and
userspace API is to allow for longer link names that contains a ':' character.
IPv6 support is not yet possible, as that would violate the current protocol
spec (bearer level originating address field is too small).

An updated tipcutils required to configure UDP bearers can be found here:
git clone -b udp_bearer git< at >github.com:Hugne/tipc-config-dev.git

Erik Hugne (2):
  tipc: allow longer link names containing ':' character
  tipc: add ip/udp media type

 include/uapi/linux/tipc_config.h |    9 +-
 net/tipc/Makefile                |    2 +-
 net/tipc/bearer.h                |    3 +
 net/tipc/core.c                  |    4 +
 net/tipc/link.c                  |    5 +-
 net/tipc/udp_media.c             |  477 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 494 insertions(+), 6 deletions(-)

TIPC 1.7.7 question.

Azeez Mohammad — 2013-05-16T19:48:35

Hi

I sent out emails before about a leak in TIPC 1.7.7 using Windriver 2.6.27.57-WR3.0.3bc_standard while doing a performance test.

I was able to narrow it down to this code. Specifically to lines 1382, 1383 in tipc_socket.c file below.

The TIPC_ERR_OVERLOAD condition is not being checked in 1.7.7 while it was being checked in 1.7.6
Was it intentional?

When I added the code for checking the TIPC_ERR_OVERLOAD condition in tipc_socket.c, the slab is not growing. I added statistics and there were a lot of messages that met the overload condition.

Please let me know if this is a bug or if it is per design.

Thanks
Azeez



tipc_socket.c from TIPC 1.7.7

   1360 static u32 dispatch(struct tipc_port *tport, struct sk_buff *buf)
   1361 {
   1362         struct sock *sk = (struct sock *)tport->usr_handle;
   1363         u32 res;
   1364
   1365         /*
   1366  *          * Process message if socket is unlocked; otherwise add to backlog queue
   1367  *                   *
   1368  *

Getting link events from topology server?

Erik Hugne — 2013-05-13T15:30:17

A common problem in cluster environment is how to detect/
handle the loss of a logical connection between two
nodes. This loss can be caused by NIC failure, cable unplugged,
switch failure etc. 
TIPC handles this quite well, and it should be quite simple
to expose this information to userland.
I'd like to push for a feature that would allow an 
application to subscribe to link layer events through 
the topology server.

Some examples of applications that this would be useful for is
traffic loadbalancers or routing daemons.

Agree/Disagree?

//E

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and 
their applications. This 200-page book is written by three acclaimed 
leaders in the field. The early access version is available now. 
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may

[PATCH net-next 0/8] tipc: removing remnants ofnative API

Jon Maloy — 2013-05-07T03:57:15

We remove the last remnants of the TIPC native API, to make it
possible to simplify locking policy and solve a problem with lost
topology events.
 
First, we introduce a socket-based alternative to the native API.

Second, we convert the two remaining users of the native API, the 
TIPC internal topology server and the configuarion server, to use the
new API. 

Third, we remove the remaining code pertaining to the native API.

Fourth we change the spinlock used in configuration server int mutexes

We also make some cosmetic changes that have become possible due to the
removed code.


Ying Xue (8):
  tipc: introduce new TIPC server infrastructure
  tipc: convert topology server to use new server facility
  tipc: convert configuration server to use new sever facility
  tipc: delete the truly orphaned code
  tipc: remove user_port instance from tipc_port structure
  tipc: rename tipc_createport_raw to tipc_createport
  tipc: convert config_lock from spinlock to mutex
  tipc: save sock structure pointer instead o

[PATCH net-next 1/1] tipc: allow implicit connectfor stream sockets

Jon Maloy — 2013-05-06T04:11:05

From: Erik Hugne <erik.hugne< at >ericsson.com>

(Paul: This one seems to be urgent, as we must have this accepeted
in net-next before we can convince SUSE to allow it in their TIPC package.
Ericsson is using SUSE extensively as our Linux distribution provider)

TIPC's implied connect feature, aka piggyback connect, allows
applications to save one syscall and all SYN/SYN-ACK signalling
overhead when setting up a connection.
Until now this has only been supported for SEQPACKET sockets.
Make it possible to use this feature even with stream sockets.

At the connecting side, the connection is completed when the
first data message arrives from the accepting peer.
This means that we must allow the connecting user to call blocking
recv() before the socket has reached state SS_CONNECTED.
So we must must relax the state machine check at recv_stream(), and
allow the recv() call even if socket is in state SS_CONNECTING.

Signed-off-by: Erik Hugne <erik.hugne< at >ericsson.com>
Signed-off-by: Jon Maloy <jon.maloy< at >ericsson.com>
--

RTT link statistics?

Erik Hugne — 2013-05-03T08:23:48

Now that we're starting to get more bearer types in TIPC,
maybe it would be a good idea to add latency to the
link statistics?

There is already a (32bit) unused timestamp field in the link protocol
header, so this is likely not a new idea :)

T0   - calculated when a link state probe message is transmitted
T1   - set in the packet when a link state response message is transmitted
T2   - calculated when a link state response (probe reply) message is received

T0 is a new u32 field in the link struct, T2 is only used when doing the RTT 
calculations.

The nodes can then get:
RTT  = T2-T0
A->B = T1-T0//Latency from the local node to the link peer
B->A = T2-T1//Latency from the peer to the local node

It could be represented like this in the link statistics (last line)

Link <1.1.2:eth0-1.1.1:eth0>
  ACTIVE  MTU:1500  Priority:10  Tolerance:1500 ms  Window:192 packets
  RX packets:100526 fragments:0/0 bundles:0/0
  TX packets:586321 fragments:0/0 bundles:0/0
  TX profile sample:158 packets  average:1350 octet

[PATCH net-next 0/8] tipc: removing remnants ofnative API

Ying Xue — 2013-04-28T10:16:27

Hi All,

In this version, below changes are made:

- rebase patches on the latest net-next branch
- add restriction on rx patch of server
- removal of top_srv structure
- revise some error log
- revise incorrect comment style
- update patch #2, #3, and #7 commit header
- divide patch #4 into three small patches
- add another patch #8 to avoid unnecessary cast.

Ying Xue (8):
  tipc: introduce new TIPC server infrastructure
  tipc: convert topology server to use new server facility
  tipc: convert configuration server to use new sever facility
  tipc: delete the truly orphaned code
  tipc: remove user_port instance from tipc_port structure
  tipc: rename tipc_createport_raw to tipc_createport
  tipc: convert config_lock from spin lock to mutex
  tipc: save sock structure pointer instead of void pointer to
    tipc_port

 net/tipc/Makefile |    2 +-
 net/tipc/config.c |  119 +++++------
 net/tipc/core.c   |   13 +-
 net/tipc/core.h   |    9 +-
 net/tipc/link.c   |   54 +----
 net/tipc/msg.c    |   15 +-
 net/t

[PATCH] tipcutils: Fix a precedence bug and gccwarning

Rodrigo Campos — 2013-04-27T20:58:47

When compiling with gcc 4.7.2, this warning is thrown:
tipc-pipe.c: In function ‘pipe_start’:
tipc-pipe.c:290:3: warning: suggest parentheses around ‘&&’ within ‘||’

This patch just fixes it by adding the parentheses in a way that if either of
the fds receive a POLLHUP and no data_in_len, then it breaks and exits the
function.

Without parentheses "a || b && !c" is evaluated as "a || (b && !c)", so without
parentheses it will be true if "a" is true, no matter "c". And IIUC that is not
what we want, as we want to make sure it exits when no data is received.

Now it seems to work "by chance" and the bug is never hit. Because if we close
then "a" (the local POLLHUP) is true, and then the whole if is true (although
"c" is 0 in that case too) and exits. And when the other end closes, then "b" is
true (the other end POLLHUP) and "c" is 0 in that case (as it is initialized to
0 at the beggining of the loop) so b && !c is true, and the whole if is true.

But it seems that the right thing is to check a

[PATCH] tipcutils: Fix a precedence bug and gccwarning

Rodrigo Campos — 2013-04-27T21:06:18

When compiling with gcc 4.7.2, this warning is thrown:
tipc-pipe.c: In function ‘pipe_start’:
tipc-pipe.c:290:3: warning: suggest parentheses around ‘&&’ within ‘||’

This patch just fixes it by adding the parentheses in a way that if either of
the fds receive a POLLHUP and no data_in_len, then it breaks and exits the
function.

Without parentheses "a || b && !c" is evaluated as "a || (b && !c)", so without
parentheses it will be true if "a" is true, no matter "c". And IIUC that is not
what we want, as we want to make sure it exits when no data is received.

Now it seems to work "by chance" and the bug is never hit. Because if we close
then "a" (the local POLLHUP) is true, and then the whole if is true (although
"c" is 0 in that case too) and exits. And when the other end closes, then "b" is
true (the other end POLLHUP) and "c" is 0 in that case (as it is initialized to
0 at the beggining of the loop) so b && !c is true, and the whole if is true.

But it seems that the right thing is to check a

[RFC PATCH] GRO for TIPC

2013-04-26T14:57:47

From: Erik Hugne <erik.hugne< at >ericsson.com>

Extensive testing in larger clusters are needed, so i'm marking it RFC
for now.
We should also discuss if/how RDM GRO can be done, and what should trigger 
a flush for that traffic.
Also, how about fragmented traffic?
I'd like to avoid having separate *_gro_receive/complete routines for each
traffic type.. maybe we can do something smart here?

If you want to build/try it out, the patch is based on v3.8-9166-g998f7e6 
with the following extra patches:
tipc: increase the max allowed link windowsize
tipc: select default bearer window from interface txq len
tipc: pass desired link window in ndisc request/response exchange

Erik Hugne (1):
  tipc: add basic GRO support

 include/linux/skbuff.h |    2 +
 net/tipc/eth_media.c   |  127 +++++++++++++++++++++++++++++++++++++++++++++++-
 net/tipc/link.c        |    8 ++-
 net/tipc/port.c        |    6 ++-
 net/tipc/socket.c      |   15 +++++-
 5 files changed, 151 insertions(+), 7 deletions(-)

Bug filed for CONFIG_TIPC=m on RedHat / Fedora

Lokesh Mandvekar — 2013-04-24T20:22:50

I filed a bug on RedHat's bugzilla to enable TIPC in the default config.

Progress can be tracked here:
https://bugzilla.redhat.com/show_bug.cgi?id=956365

Thanks,

Is 2.0 backward-compatible with 1.7 within acluster?

Peter Butler — 2013-04-24T15:37:56

If nodes running 1.7 coexist with nodes running 2.0, all within a given 
network cluster, are any compatibility issues expected?  We have just 
recently set up such a network cluster and are seeing some strange 
behaviour, such as links dropping and re-establishing, etc.
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service 
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr

[PATCH v2 0/5] tipc: add TIPC over Infinibandsupport]

Erik Hugne — 2013-04-18T08:23:16

Infiniband bearer support is now in net-next!

----- Forwarded message from David Miller <davem< at >davemloft.net> -----

Date: Wed, 17 Apr 2013 14:19:06 -0400
From: David Miller <davem< at >davemloft.net>
To: kaber< at >trash.net
CC: jon.maloy< at >ericsson.com, allan.stephens< at >windriver.com, netdev< at >vger.kernel.org, roland< at >kernel.org, sean.hefty< at >intel.com, hal.rosenstock< at >gmail.com, linux-rdma< at >vger.kernel.org
Subject: Re: [PATCH v2 0/5] tipc: add TIPC over Infiniband support
X-Mailer: Mew version 6.5 on Emacs 24.1 / Mule 6.0 (HANACHIRUSATO)

From: Patrick McHardy <kaber< at >trash.net>
Date: Wed, 17 Apr 2013 18:18:24 +0200


All applied to net-next, thanks!
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

----- End forwarded message -----

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics

FW: FW: Question about TIPC 1.7.7 slab.

Felix Nayman — 2013-04-17T01:51:26

All,

Any idea why we would be seeing the leak in 1.7.7, but from what appears to be the same code in 1.7.6 we don't see it?

Here's some additional information that Azeez noticed from his testing:

After I enabled slub_debug, I noticed that the buf_acquire is allocating the memory and skb_release_data is freeing it (see below). The numbers (buf_acquire, skb_release_data) seem to match but the # of active objects keeps on increasing until there is no memory left. How can I proceed further as to why the slab is not being freed?

Thanks for your help.

from slabtop: 349594 281787 80% 4.07K 49942 7 1598144K kmalloc-4096

sort -n /sys/kernel/slab/kmalloc-4096/alloc_calls 259945 buf_acquire+0x30/0x70 [tipc] age=251/2199820/3285324 pid=3789 cpus=0-7

from /sys/kernel/slab/kmalloc-4096/free_calls 260208 skb_release_data +0x85/0xd0 age=1499/2291151/4037734 pid=4722-31618 cpus=0-7



Thanks a lot for your help,
Felix


-----Original Message-----
From: Azeez Mohammad
Sent: Wednesday, April 10, 2013 10:08 AM
To: 'Ying

tipcutils-2.0.3 rpm packaging for Fedora

Lokesh Mandvekar — 2013-04-15T02:34:57

I noticed Erik's package review request on Red Hat's bugzilla for tipcutils-2.0.3.
I did a review for it, but I'm not sure if that route is still being pursued.

If Erik et.al. are still interested in getting it into Fedora (but find the
packaging to be a chore), I would like to be the maintainer / co-maintainer
of the package. I have recently gotten into RPM packaging, and should be getting my
first package accepted soon (and thus get upload rights).

Let me know what you guys think.

Thanks,
--
Lokesh

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

[PATCH] tipc: skb_copy() buffers for multicastsending

Gerlando Falauto — 2013-04-11T14:27:32

When sending packets, TIPC bearers use skb_clone() before writing their
hardware header. This will however NOT copy the data buffer.
So when the same packet is sent over multiple bearers (to reach multiple
nodes), the same socket buffer data will be treated by multiple
tipc_media drivers which will write their own hardware header through
dev_hard_header().
Most of the time this is not a problem, because by the time the
packet is processed by the second media, it has already been sent over the
first one. However, when the first transmission is delayed (e.g. because
of insufficient bandwidth or through a shaper), the next bearer will
overwrite the hardware header, resulting in the packet being sent:
a) with the wrong source address, when bearers of the same type,
e.g. ethernet, are involved
b) with a completely corrupt header, or even dropped, when bearers of
different types are involved.

So when the same socket buffer is to be sent multiple times, send a
skb_copy() instead (from the second instance on), and

Question about TIPC 1.7.7 slab.

Azeez Mohammad — 2013-04-10T19:27:22

Hi,

One more data point.

I ran the tipc client server program from tipc_utils. When I did that, all the 30+ gigs of memory has been used. This program used the buf_acquire from tipc_msg_build() of tipc_msg.c


Data has been pasted below. Please let me know if you have any suggestions.

Thanks
Azeez


bash-3.2# free -m
             total       used       free     shared    buffers     cached
Mem:         32238      31396        842          0          3        479
-/+ buffers/cache:      30912       1325


slabtop
4337316 4333986  99%    4.07K 632904        7  20252928K kmalloc-4096


sort -n /sys/kernel/slab/kmalloc-4096/alloc_calls
4255380 buf_acquire12+0x30/0x70 [tipc] age=124/1419661/4610844 pid=4866-23199 cpus=0-7  <--- tipc client/server






mpug< at >ATCA37_pl0_1:/sys/kernel/slab/kmalloc-4096$ cat /proc/meminfo
MemTotal:     33011820 kB
MemFree:       1135196 kB
Buffers:          3376 kB
Cached:         285780 kB
SwapCached:          0 kB
Active:        6950792 kB
Inactive:       203852 kB
SwapTotal:

FW: FW: Question about TIPC 1.7.7 slab.

Azeez Mohammad — 2013-04-10T15:08:10

Hi,

About the slab leak in TIPC 1.7.7 :

   2259 buf_acquire11+0x30/0x70 [tipc] age=4747/219692/530647 pid=0-23199 cpus=6


Well, It is the buf_acquire11 < at > line # 2922 below.


I don't see a buf_discard(pbuf) call corresponding to "pbuf = buf_acquire11(msg_size(imsg))". How is pbuf freed?

And the code is the same in TIPC1.7.6 . I am not sure why it doesn't leak in TIPC 1.7.6

Thanks for your help.
Azeez


2883 /*
   2884  * tipc_link_recv_fragment(): Called with node lock on. Returns
   2885  * the reassembled buffer if message is complete.
   2886  */
   2887 int tipc_link_recv_fragment(struct sk_buff **pending, struct sk_buff **fb,
   2888                 struct tipc_msg **m)
   2889 {
   2890     struct sk_buff *pbuf = *pending;
   2891     struct sk_buff *fbuf = *fb;
   2892     struct sk_buff *prev = NULL;
   2893     struct tipc_msg *fragm = buf_msg(fbuf);
   2894     u32 long_msg_orig = msg_orignode(fragm);
   2895     u32 long_msg_seq_no = msg_fragm_msg_no(fragm);
   2896
   2897     *fb = NULL;

[RFC PATCH net-next 0/5] introduce linkcongestion control algorithm

Ying Xue — 2013-04-10T10:38:18

In Erik's intial patches, several important link improvements have been
involved. But these changes still might not make full use of bandwidth
throughput, and flexibly adjust link congestion window with the change
of network condition. For example, Erik's patch may work effectively with
one or two links attached to one bearer. But if ten or even hundred of
links are attached to the same bearer, maybe this would bring some big
challenge to it. So it's necessary for us to introduce congestion
control algorithm to link layer.

Thus, I modify Erik's patches as belows:

Patch #2, the default window is configured to the interface transmit
queue length instead of its 75%.

Patch #3, the desired initial *bearer* window size, derived from the rx
ring buffer length instead of its 75% is passed on to the peer in the
ndisc request/response messages. And the default bearer window is set
to slow start threshold.

Patch #4, introduce congestion control algorithm.

Patch #5, just one debug patch which can let us observe how

[PATCH net-next 0/5] tipc: removing remnants ofnative API

Ying Xue — 2013-04-07T08:25:49

Revise patch #1 by Paul's comments.

PS: Jon, I add your name in the patch #1. If you see some problem,
please directly change them.

Ying Xue (5):
  tipc: introduce new TIPC server infrastructure
  tipc: convert topology server to use new server facility
  tipc: convert configuration server to use new sever facility
  tipc: remove facilities of user port
  tipc: convert config_lock from spin lock to mutex

 net/tipc/Makefile |    2 +-
 net/tipc/config.c |  116 +++++------
 net/tipc/core.c   |   10 +-
 net/tipc/core.h   |    7 +-
 net/tipc/link.c   |   18 +-
 net/tipc/msg.c    |   13 +-
 net/tipc/msg.h    |    2 +-
 net/tipc/port.c   |  309 +---------------------------
 net/tipc/port.h   |   65 +-----
 net/tipc/server.c |  591 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 net/tipc/server.h |   92 +++++++++
 net/tipc/socket.c |  104 +++++++++-
 net/tipc/subscr.c |  311 +++++++++-------------------
 net/tipc/subscr.h |    8 +-
 14 files changed, 960 insertions(+), 688 deletions(-)
 create mode 100644

TIPC multicast sending over multiple bearers

Gerlando Falauto — 2013-04-05T16:08:30

Hi everyone,

we've been facing some very weird problems with how TIPC handles (sends) 
multicast packets.
More to the detail, our scenario includes two different bearers (of two 
different tipc_media types: one is ethernet and the other one is our 
proprietary HDLC).
What happens is that multicast packets would be sent over both bearers, 
and this of course normally works.
However, we noticed that when the HDLC bearer (which is much slower than 
the ethernet one) is somehow in use (e.g. we are performing some stress 
test over the channel), these multicast packets are NOT sent over HDLC, 
and this will of course trigger retransmissions at a later point.
Simply disabling the ethernet bearer seemed to magically fix the 
situation. As an alternative, disabling sending over ethernet by hacking 
into the code somehow leads to the same results.

After several hours of debugging, I noticed that on eth_media.c we have 
the following (our proprietary tipc_media does exactly the same thing):

static int send_msg(stru