gmane.linux.drivers.openib

Re: [ofa-general] SPAM git http url for ofed_1_3/management.git

Hal Rosenstock — 2008-12-02T00:05:56

git://git.openfabrics.org/ofed_1_3/management.git

[ofa-general] SPAM git http url for ofed_1_3/management.git

Todd Bowman — 2008-12-01T23:28:43

[ofa-general] Re: [PATCH] opensm/osm_prefix_route.h: prefix and guidare in network rather than host endian order

Sasha Khapyorsky — 2008-12-01T20:51:48

Applied. Thanks. Sasha

RE: [ofa-general] [RMDA CM IPv6 support. PATCHv5 2/2] RDMA CMsupport

Sean Hefty — 2008-12-01T19:52:57

I missed this before, but I think we also need to change cma_loopback_addr() (called from rdma_bind_addr() through cma_any_addr()) to handle IPv6. - Sean

[ofa-general] RE: [RMDA CM IPv6 support. PATCHv5 1/2]IPv6 IB addrresolution

Sean Hefty — 2008-12-01T19:38:29

Thanks for adding this. Acked-by: Sean Hefty intel.com> with one nit... Is this assignment needed/used? dst_ip; *)dst_in)->sin6_addr; addr)-

[ofa-general] [PATCH] opensm/osm_prefix_route.h: prefix and guid are in network rather than host endian order

Hal Rosenstock — 2008-12-01T18:43:47

Sasha, Minor patch to declare prefix and guid in network rather than host endian order.

Re: [ofa-general] GitWeb really slow

Jeff Becker — 2008-12-01T18:21:46

Hi Nicolas Nicolas Morey Chaisemartin wrote: I have noticed this as well. OFA will be moving to a new server after OFED 1.4 releases (soon). We will make sure the problem does not occur there. Please bear with us for now. Thanks. Jeff Becker OFA server admin

[ofa-general] [GIT PULL] please pull infiniband.git

Roland Dreier — 2008-12-01T18:16:10

Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get two fixes to the ehca driver for problems in patches added in the 2.6.28 cycle, and two fixes for mlx4, one for a regression introduced in the 2.6.28 cycle, and one for a resource leak that is serious enough and has a simple enough fix to target -stable. Jack Morgenstein (2): mlx4_core: Save/restore default port IB capability mask IB/mlx4: Fix MTT leakage in resize CQ Joachim Fenkes (1): IB/ehca: Change misleading error message on memory hotplug Roland Dreier (1): Merge branches 'ehca' and 'mlx4' into for-linus Stefan Roscher (1): IB/ehca: Fix problem with generated flush work completions drivers/infiniband/hw/ehca/ehca_classes.h | 4 ++- drivers/infiniband/hw/ehca/ehca_main.c | 3 +- drivers/infiniband/hw/ehca/ehca_qp.c | 26 +++++++++++--- drivers/infiniband/hw/ehca/ehca_reqs.c | 51 +++++++++++++++++------------ drivers/infiniband/hw/mlx4/cq.c | 5 +++ drivers/net/mlx4/main.c | 8 ++++ drivers/net/mlx4/mlx4.h | 1 + drivers/net/mlx4/port.c | 39 +++++++++++++++++++++- include/linux/mlx4/device.h | 1 + 9 files changed, 107 insertions(+), 31 deletions(-) diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h index 4df887a..7fc35cf 100644 --- a/drivers/infiniband/hw/ehca/ehca_classes.h +++ b/drivers/infiniband/hw/ehca/ehca_classes.h < at >< at > -163,7 +163,8 < at >< at > struct ehca_mod_qp_parm { /* struct for tracking if cqes have been reported to the application */ struct ehca_qmap_entry { u16 app_wr_id; -u16 reported; +u8 reported; +u8 cqe_req; }; struct ehca_queue_map { < at >< at > -171,6 +172,7 < at >< at > struct ehca_queue_map { unsigned int entries; unsigned int tail; unsigned int left_to_poll; +unsigned int next_wqe_idx; /* Idx to first wqe to be flushed */ }; struct ehca_qp { diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c index bb02a86..bec7e02 100644 --- a/drivers/infiniband/hw/ehca/ehca_main.c +++ b/drivers/infiniband/hw/ehca/ehca_main.c < at >< at > -994,8 +994,7 < at >< at > static int ehca_mem_notifier(struct notifier_block *nb, if (printk_timed_ratelimit(&ehca_dmem_warn_time, 30 * 1000)) ehca_gen_err("DMEM operations are not allowed" - "as long as an ehca adapter is" - "attached to the LPAR"); + "in conjunction with eHCA"); return NOTIFY_BAD; } } diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c index 9e05ee2..cadbf0c 100644 --- a/drivers/infiniband/hw/ehca/ehca_qp.c +++ b/drivers/infiniband/hw/ehca/ehca_qp.c < at >< at > -435,9 +435,13 < at >< at > static void reset_queue_map(struct ehca_queue_map *qmap) { int i; -qmap->tail = 0; -for (i = 0; i < qmap->entries; i++) +qmap->tail = qmap->entries - 1; +qmap->left_to_poll = 0; +qmap->next_wqe_idx = 0; +for (i = 0; i < qmap->entries; i++) { qmap->map[i].reported = 1; +qmap->map[i].cqe_req = 0; +} } /* < at >< at > -1121,6 +1125,7 < at >< at > static int calc_left_cqes(u64 wqe_p, struct ipz_queue *ipz_queue, void *wqe_v; u64 q_ofs; u32 wqe_idx; +unsigned int tail_idx; /* convert real to abs address */ wqe_p = wqe_p & (~(1UL << 63)); < at >< at > -1133,12 +1138,17 < at >< at > static int calc_left_cqes(u64 wqe_p, struct ipz_queue *ipz_queue, return -EFAULT; } +tail_idx = (qmap->tail + 1) % qmap->entries; wqe_idx = q_ofs / ipz_queue->qe_size; -if (wqe_idx < qmap->tail) -qmap->left_to_poll = (qmap->entries - qmap->tail) + wqe_idx; -else -qmap->left_to_poll = wqe_idx - qmap->tail; +/* check all processed wqes, whether a cqe is requested or not */ +while (tail_idx != wqe_idx) { +if (qmap->map[tail_idx].cqe_req) +qmap->left_to_poll++; +tail_idx = (tail_idx + 1) % qmap->entries; +} +/* save index in queue, where we have to start flushing */ +qmap->next_wqe_idx = wqe_idx; return 0; } < at >< at > -1185,10 +1195,14 < at >< at > static int check_for_left_cqes(struct ehca_qp *my_qp, struct ehca_shca *shca) } else { spin_lock_irqsave(&my_qp->send_cq->spinlock, flags); my_qp->sq_map.left_to_poll = 0; +my_qp->sq_map.next_wqe_idx = (my_qp->sq_map.tail + 1) % +my_qp->sq_map.entries; spin_unlock_irqrestore(&my_qp->send_cq->spinlock, flags); spin_lock_irqsave(&my_qp->recv_cq->spinlock, flags); my_qp->rq_map.left_to_poll = 0; +my_qp->rq_map.next_wqe_idx = (my_qp->rq_map.tail + 1) % +my_qp->rq_map.entries; spin_unlock_irqrestore(&my_qp->recv_cq->spinlock, flags); } diff --git a/drivers/infiniband/hw/ehca/ehca_reqs.c b/drivers/infiniband/hw/ehca/ehca_reqs.c index 6492807..00a648f 100644 --- a/drivers/infiniband/hw/ehca/ehca_reqs.c +++ b/drivers/infiniband/hw/ehca/ehca_reqs.c < at >< at > -179,6 +179,7 < at >< at > static inline int ehca_write_swqe(struct ehca_qp *qp, qmap_entry->app_wr_id = get_app_wr_id(send_wr->wr_id); qmap_entry->reported = 0; +qmap_entry->cqe_req = 0; switch (send_wr->opcode) { case IB_WR_SEND: < at >< at > -203,8 +204,10 < at >< at > static inline int ehca_write_swqe(struct ehca_qp *qp, if ((send_wr->send_flags & IB_SEND_SIGNALED || qp->init_attr.sq_sig_type == IB_SIGNAL_ALL_WR) - && !hidden) + && !hidden) { wqe_p->wr_flag |= WQE_WRFLAG_REQ_SIGNAL_COM; +qmap_entry->cqe_req = 1; +} if (send_wr->opcode == IB_WR_SEND_WITH_IMM || send_wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) { < at >< at > -569,6 +572,7 < at >< at > static int internal_post_recv(struct ehca_qp *my_qp, qmap_entry = &my_qp->rq_map.map[rq_map_idx]; qmap_entry->app_wr_id = get_app_wr_id(cur_recv_wr->wr_id); qmap_entry->reported = 0; +qmap_entry->cqe_req = 1; wqe_cnt++; } /* eof for cur_recv_wr */ < at >< at > -706,27 +710,34 < at >< at > repoll: goto repoll; wc->qp = &my_qp->ib_qp; +qmap_tail_idx = get_app_wr_id(cqe->work_request_id); +if (!(cqe->w_completion_flags & WC_SEND_RECEIVE_BIT)) +/* We got a send completion. */ +qmap = &my_qp->sq_map; +else +/* We got a receive completion. */ +qmap = &my_qp->rq_map; + +/* advance the tail pointer */ +qmap->tail = qmap_tail_idx; + if (is_error) { /* * set left_to_poll to 0 because in error state, we will not * get any additional CQEs */ -ehca_add_to_err_list(my_qp, 1); +my_qp->sq_map.next_wqe_idx = (my_qp->sq_map.tail + 1) % +my_qp->sq_map.entries; my_qp->sq_map.left_to_poll = 0; +ehca_add_to_err_list(my_qp, 1); +my_qp->rq_map.next_wqe_idx = (my_qp->rq_map.tail + 1) % +my_qp->rq_map.entries; +my_qp->rq_map.left_to_poll = 0; if (HAS_RQ(my_qp)) ehca_add_to_err_list(my_qp, 0); -my_qp->rq_map.left_to_poll = 0; } -qmap_tail_idx = get_app_wr_id(cqe->work_request_id); -if (!(cqe->w_completion_flags & WC_SEND_RECEIVE_BIT)) -/* We got a send completion. */ -qmap = &my_qp->sq_map; -else -/* We got a receive completion. */ -qmap = &my_qp->rq_map; - qmap_entry = &qmap->map[qmap_tail_idx]; if (qmap_entry->reported) { ehca_warn(cq->device, "Double cqe on qp_num=%#x", < at >< at > -738,10 +749,6 < at >< at > repoll: wc->wr_id = replace_wr_id(cqe->work_request_id, qmap_entry->app_wr_id); qmap_entry->reported = 1; -/* this is a proper completion, we need to advance the tail pointer */ -if (++qmap->tail == qmap->entries) -qmap->tail = 0; - /* if left_to_poll is decremented to 0, add the QP to the error list */ if (qmap->left_to_poll > 0) { qmap->left_to_poll--; < at >< at > -805,13 +812,14 < at >< at > static int generate_flush_cqes(struct ehca_qp *my_qp, struct ib_cq *cq, else qmap = &my_qp->rq_map; -qmap_entry = &qmap->map[qmap->tail]; +qmap_entry = &qmap->map[qmap->next_wqe_idx]; while ((nr < num_entries) && (qmap_entry->reported == 0)) { /* generate flush CQE */ + memset(wc, 0, sizeof(*wc)); -offset = qmap->tail * ipz_queue->qe_size; +offset = qmap->next_wqe_idx * ipz_queue->qe_size; wqe = (struct ehca_wqe *)ipz_qeit_calc(ipz_queue, offset); if (!wqe) { ehca_err(cq->device, "Invalid wqe offset=%#lx on " < at >< at > -850,11 +858,12 < at >< at > static int generate_flush_cqes(struct ehca_qp *my_qp, struct ib_cq *cq, wc->qp = &my_qp->ib_qp; -/* mark as reported and advance tail pointer */ +/* mark as reported and advance next_wqe pointer */ qmap_entry->reported = 1; -if (++qmap->tail == qmap->entries) -qmap->tail = 0; -qmap_entry = &qmap->map[qmap->tail]; +qmap->next_wqe_idx++; +if (qmap->next_wqe_idx == qmap->entries) +qmap->next_wqe_idx = 0; +qmap_entry = &qmap->map[qmap->next_wqe_idx]; wc++; nr++; } diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d0866a3..1830849 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c < at >< at > -343,6 +343,7 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) { struct mlx4_ib_dev *dev = to_mdev(ibcq->device); struct mlx4_ib_cq *cq = to_mcq(ibcq); +struct mlx4_mtt mtt; int outst_cqe; int err; < at >< at > -376,10 +377,13 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) goto out; } +mtt = cq->buf.mtt; + err = mlx4_cq_resize(dev->dev, &cq->mcq, entries, &cq->resize_buf->buf.mtt); if (err) goto err_buf; +mlx4_mtt_cleanup(dev->dev, &mtt); if (ibcq->uobject) { cq->buf = cq->resize_buf->buf; cq->ibcq.cqe = cq->resize_buf->cqe; < at >< at > -406,6 +410,7 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) goto out; err_buf: +mlx4_mtt_cleanup(dev->dev, &cq->resize_buf->buf.mtt); if (!ibcq->uobject) mlx4_ib_free_cq_buf(dev, &cq->resize_buf->buf, cq->resize_buf->cqe); diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c index 468921b..90a0281 100644 --- a/drivers/net/mlx4/main.c +++ b/drivers/net/mlx4/main.c < at >< at > -753,6 +753,7 < at >< at > static int mlx4_setup_hca(struct mlx4_dev *dev) struct mlx4_priv *priv = mlx4_priv(dev); int err; int port; +__be32 ib_port_default_caps; err = mlx4_init_uar_table(dev); if (err) { < at >< at > -852,6 +853,13 < at >< at > static int mlx4_setup_hca(struct mlx4_dev *dev) } for (port = 1; port <= dev->caps.num_ports; port++) { +ib_port_default_caps = 0; +err = mlx4_get_port_ib_caps(dev, port, &ib_port_default_caps); +if (err) +mlx4_warn(dev, "failed to get port %d default " + "ib capabilities (%d). Continuing with " + "caps = 0\n", port, err); +dev->caps.ib_port_def_cap[port] = ib_port_default_caps; err = mlx4_SET_PORT(dev, port); if (err) { mlx4_err(dev, "Failed to set port %d, aborting\n", diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h index 56a2e21..34c909d 100644 --- a/drivers/net/mlx4/mlx4.h +++ b/drivers/net/mlx4/mlx4.h < at >< at > -385,5 +385,6 < at >< at > void mlx4_init_mac_table(struct mlx4_dev *dev, struct mlx4_mac_table *table); void mlx4_init_vlan_table(struct mlx4_dev *dev, struct mlx4_vlan_table *table); int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port); +int mlx4_get_port_ib_caps(struct mlx4_dev *dev, u8 port, __be32 *caps); #endif /* MLX4_H */ diff --git a/drivers/net/mlx4/port.c b/drivers/net/mlx4/port.c index e2fdab4..0a057e5 100644 --- a/drivers/net/mlx4/port.c +++ b/drivers/net/mlx4/port.c < at >< at > -258,6 +258,42 < at >< at > out: } EXPORT_SYMBOL_GPL(mlx4_unregister_vlan); +int mlx4_get_port_ib_caps(struct mlx4_dev *dev, u8 port, __be32 *caps) +{ +struct mlx4_cmd_mailbox *inmailbox, *outmailbox; +u8 *inbuf, *outbuf; +int err; + +inmailbox = mlx4_alloc_cmd_mailbox(dev); +if (IS_ERR(inmailbox)) +return PTR_ERR(inmailbox); + +outmailbox = mlx4_alloc_cmd_mailbox(dev); +if (IS_ERR(outmailbox)) { +mlx4_free_cmd_mailbox(dev, inmailbox); +return PTR_ERR(outmailbox); +} + +inbuf = inmailbox->buf; +outbuf = outmailbox->buf; +memset(inbuf, 0, 256); +memset(outbuf, 0, 256); +inbuf[0] = 1; +inbuf[1] = 1; +inbuf[2] = 1; +inbuf[3] = 1; +*(__be16 *) (&inbuf[16]) = cpu_to_be16(0x0015); +*(__be32 *) (&inbuf[20]) = cpu_to_be32(port); + +err = mlx4_cmd_box(dev, inmailbox->dma, outmailbox->dma, port, 3, + MLX4_CMD_MAD_IFC, MLX4_CMD_TIME_CLASS_C); +if (!err) +*caps = *(__be32 *) (outbuf + 84); +mlx4_free_cmd_mailbox(dev, inmailbox); +mlx4_free_cmd_mailbox(dev, outmailbox); +return err; +} + int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port) { struct mlx4_cmd_mailbox *mailbox; < at >< at > -273,7 +309,8 < at >< at > int mlx4_SET_PORT(struct mlx4_dev *dev, u8 port) ((u8 *) mailbox->buf)[3] = 6; ((__be16 *) mailbox->buf)[4] = cpu_to_be16(1 << 15); ((__be16 *) mailbox->buf)[6] = cpu_to_be16(1 << 15); -} +} else +((__be32 *) mailbox->buf)[1] = dev->caps.ib_port_def_cap[port]; err = mlx4_cmd(dev, mailbox->dma, port, is_eth, MLX4_CMD_SET_PORT, MLX4_CMD_TIME_CLASS_B); diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h index bd9977b..371086f 100644 --- a/include/linux/mlx4/device.h +++ b/include/linux/mlx4/device.h < at >< at > -179,6 +179,7 < at >< at > struct mlx4_caps { intnum_ports; intvl_cap[MLX4_MAX_PORTS + 1]; intib_mtu_cap[MLX4_MAX_PORTS + 1]; +__be32ib_port_def_cap[MLX4_MAX_PORTS + 1]; u64def_mac[MLX4_MAX_PORTS + 1]; inteth_mtu_cap[MLX4_MAX_PORTS + 1]; intgid_table_len[MLX4_MAX_PORTS + 1];

[ofa-general] Agenda for OFED meeting today (Dec 1) on OFED 1.4

Tziporet Koren — 2008-12-01T16:21:17

This is the Agenda for the OFED meeting today: 1. Bug list review: 1421 blo raisch< at >de.ibm.com ipoib/ehca crash when bouncing network 1413 cri jackm< at >mellanox.co.il rmmod mlx4_ib deadlock - on work 1384 maj eli< at >mellanox.co.il netperf latency small messages increase 5% - UD mode issue, seems caused by LRO 1386 maj eli< at >mellanox.co.il ofed 1.4 - iperf tcp connected mode BW large messages dec... - RC mode - not reproduced for now 1395 maj vu< at >mellanox.com kernel panic during SRP HA test - on work 1419 maj vlad< at >mellanox.co.il Iperf-2.0.4 fails: page allocation failure. order:5 1423 maj jackm< at >mellanox.co.il RDMA_Write RC small message bw degradation jitter of 6% - doing more measurements 2. UNH Logo testing update 3. Issue with new Intel MPI Benchmark (IMB, known as Pallas). In OFED 1.3 we used IMB 3.0 and in OFED 1.4 we upgraded the IMB to 3.1 version. During our performance verification we found 5% latency degradation with IMB 3.1 multi PingPong benchmark (we compare the result to IMB 3.0 version). Do we wish to go back to IMB 3.0? 4. Open discussion Tziporet

[ofa-general] RESEND: net.ipv4.tcp_timestamps

John Russo — 2008-12-01T15:33:27

Does anyone know why this value is being altered by OFED? "net.ipv4.tcp_timestamps" is being set to 0 during OFED installation. Default value of this parameter is set to 1 on standard RHEL/SLES distros which OFED installation script modifies to 0. Also, when OFED is uninstalled, it does not reset these sysctl parameters to their original values. This parameter is specifically recommended to be turned ON for High performance network. This is a TCP option that can be used to calculate the Round Trip Measurement in a better and more accurate way than the retransmission timeout method can. Accurate value of retransmission timeout should be determined to avoid unnecessary retransmissions and hence to improve TCP performance. RFC 1323 talks about this TCP extension for High Performance. When the parameter net.ipv4.tcp_timestamps=1, then it adds extra 12 bytes into TCP header increasing its size. This has an obvious effect of decrease in bandwidth as we have some extra data flowing. Is this the reason why OFED turns it OFF to net.ipv4.tcp_timestamps=0.

[ofa-general] net.ipv4.tcp_timestamps

John Russo — 2008-12-01T15:29:03

Re: [ofa-general] [RMDA CM IPv6 support. PATCHv5 2/2] RDMA CM support

Aleksey Senin — 2008-12-01T13:23:26

IPv6 RDMA CM support From: Aleksey Senin voltaire.com> Date: Mon, 1 Dec 2008 14:01:10 +0200 Subject: [PATCH] RDMA CM IPv6 support AF_INET6 case in rdma_bind_addr added AF_INET6 support in cma_format_hdr function Use sockaddr_storage structure in cma_bind_any function Signed-off-by: Aleksey Senin voltaire.com> :100644 100644 d951896... df22c5c... Mdrivers/infiniband/core/cma.c diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index d951896..df22c5c 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c < at >< at > -1467,10 +1467,10 < at >< at > static void cma_listen_on_all(struct rdma_id_private *id_priv) static int cma_bind_any(struct rdma_cm_id *id, sa_family_t af) { -struct sockaddr_in addr_in; +struct sockaddr_storage addr_in; memset(&addr_in, 0, sizeof addr_in); -addr_in.sin_family = af; +addr_in.ss_family = af; return rdma_bind_addr(id, (struct sockaddr *) &addr_in); } < at >< at > -2073,7 +2073,7 < at >< at > int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr) struct rdma_id_private *id_priv; int ret; -if (addr->sa_family != AF_INET) +if (addr->sa_family != AF_INET && addr->sa_family != AF_INET6) return -EAFNOSUPPORT; id_priv = container_of(id, struct rdma_id_private, id); < at >< at > -2113,32 +2113,61 < at >< at > EXPORT_SYMBOL(rdma_bind_addr); static int cma_format_hdr(void *hdr, enum rdma_port_space ps, struct rdma_route *route) { -struct sockaddr_in *src4, *dst4; struct cma_hdr *cma_hdr; struct sdp_hh *sdp_hdr; -src4 = (struct sockaddr_in *) &route->addr.src_addr; -dst4 = (struct sockaddr_in *) &route->addr.dst_addr; - -switch (ps) { -case RDMA_PS_SDP: -sdp_hdr = hdr; -if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) -return -EINVAL; -sdp_set_ip_ver(sdp_hdr, 4); -sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; -sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; -sdp_hdr->port = src4->sin_port; -break; -default: -cma_hdr = hdr; -cma_hdr->cma_version = CMA_VERSION; -cma_set_ip_ver(cma_hdr, 4); -cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; -cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; -cma_hdr->port = src4->sin_port; -break; +if (route->addr.src_addr.ss_family == AF_INET) { +struct sockaddr_in *src4, *dst4; + +src4 = (struct sockaddr_in *) &route->addr.src_addr; +dst4 = (struct sockaddr_in *) &route->addr.dst_addr; + +switch (ps) { +case RDMA_PS_SDP: +sdp_hdr = hdr; +if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) +return -EINVAL; +sdp_set_ip_ver(sdp_hdr, 4); +sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; +sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; +sdp_hdr->port = src4->sin_port; +break; +default: +cma_hdr = hdr; +cma_hdr->cma_version = CMA_VERSION; +cma_set_ip_ver(cma_hdr, 4); +cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr; +cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr; +cma_hdr->port = src4->sin_port; +break; +} +} else { +struct sockaddr_in6 *src6, *dst6; + +src6 = (struct sockaddr_in6 *) &route->addr.src_addr; +dst6 = (struct sockaddr_in6 *) &route->addr.dst_addr; + +switch (ps) { +case RDMA_PS_SDP: +sdp_hdr = hdr; +if (sdp_get_majv(sdp_hdr->sdp_version) != SDP_MAJ_VERSION) +return -EINVAL; +sdp_set_ip_ver(sdp_hdr, 6); +sdp_hdr->src_addr.ip6 = src6->sin6_addr; +sdp_hdr->dst_addr.ip6 = dst6->sin6_addr; +sdp_hdr->port = src6->sin6_port; +break; +default: +cma_hdr = hdr; +cma_hdr->cma_version = CMA_VERSION; +cma_set_ip_ver(cma_hdr, 6); +cma_hdr->src_addr.ip6 = src6->sin6_addr; +cma_hdr->dst_addr.ip6 = dst6->sin6_addr; +cma_hdr->port = src6->sin6_port; +break; +} } + return 0; }

[ofa-general] [RMDA CM IPv6 support. PATCHv5 1/2]IPv6 IB addrresolution

Aleksey Senin — 2008-12-01T13:16:44

Set from two patches for support IPv6 protocol in RMDA CM Changes: Use two patches instead of six. This patch adds IPv6 support for IB address resolution. From: Aleksey Senin voltaire.com> Date: Mon, 1 Dec 2008 13:55:01 +0200 Subject: [PATCH] IB addr IPv6 support Support for network discovery in addr_send_arp function. Local IPv6 address resolution. Added remote IPv6 address resolusion for RDMA CM. Function addr_resolve_remote used as wrapper for two other functions: addr4_resolve_remote ( original addr_resolve_remote ) addr6_resolve_remote ( new function ) Signed-off-by: Aleksey Senin voltaire.com> :100644 100644 f95d21f... d9170c6... Mdrivers/infiniband/core/addr.c diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c index f95d21f..d9170c6 100644 --- a/drivers/infiniband/core/addr.c +++ b/drivers/infiniband/core/addr.c < at >< at > -43,6 +43,7 < at >< at > #include #include #include +#include MODULE_AUTHOR("Sean Hefty"); MODULE_DESCRIPTION("IB Address Translation"); < at >< at > -172,27 +173,42 < at >< at > static void queue_req(struct addr_req *req) mutex_unlock(&lock); } -static void addr_send_arp(struct sockaddr_in *dst_in) +static void addr_send_arp(struct sockaddr *dst_in) { struct rtable *rt; struct flowi fl; -__be32 dst_ip = dst_in->sin_addr.s_addr; +struct dst_entry *dst; memset(&fl, 0, sizeof fl); -fl.nl_u.ip4_u.daddr = dst_ip; -if (ip_route_output_key(&init_net, &rt, &fl)) -return; +if (dst_in->sa_family == AF_INET) { +fl.nl_u.ip4_u.daddr = +((struct sockaddr_in *)dst_in)->sin_addr.s_addr; -neigh_event_send(rt->u.dst.neighbour, NULL); -ip_rt_put(rt); +if (ip_route_output_key(&init_net, &rt, &fl)) +return; + +neigh_event_send(rt->u.dst.neighbour, NULL); +ip_rt_put(rt); + +} else { +fl.nl_u.ip6_u.daddr = +((struct sockaddr_in6 *)dst_in)->sin6_addr; + +dst = ip6_route_output(&init_net, NULL, &fl); +if (!dst) +return; + +neigh_event_send(dst->neighbour, NULL); +dst_release(dst); +} } -static int addr_resolve_remote(struct sockaddr *src_in, - struct sockaddr *dst_in, +static int addr4_resolve_remote(struct sockaddr_in *src_in, + struct sockaddr_in *dst_in, struct rdma_dev_addr *addr) { -__be32 src_ip = ((struct sockaddr_in *)src_in)->sin_addr.s_addr; -__be32 dst_ip = ((struct sockaddr_in *)dst_in)->sin_addr.s_addr; +__be32 src_ip = src_in->sin_addr.s_addr; +__be32 dst_ip = dst_in->sin_addr.s_addr; struct flowi fl; struct rtable *rt; struct neighbour *neigh; < at >< at > -223,8 +239,8 < at >< at > static int addr_resolve_remote(struct sockaddr *src_in, } if (!src_ip) { -src_in->sa_family = dst_in->sa_family; -((struct sockaddr_in *)src_in)->sin_addr.s_addr = rt->rt_src; +src_in->sin_family = dst_in->sin_family; +src_in->sin_addr.s_addr = rt->rt_src; } ret = rdma_copy_addr(addr, neigh->dev, neigh->ha); < at >< at > -236,6 +252,47 < at >< at > out: return ret; } +static int addr6_resolve_remote(struct sockaddr_in6 *src_in, + struct sockaddr_in6 *dst_in, + struct rdma_dev_addr *addr) +{ +struct flowi fl; +struct neighbour *neigh; +struct dst_entry *dst; +int ret = -ENODATA; + +memset(&fl, 0, sizeof fl); +fl.nl_u.ip6_u.daddr = dst_in->sin6_addr; +fl.nl_u.ip6_u.saddr = src_in->sin6_addr; + +dst = ip6_route_output(&init_net, NULL, &fl); +if (!dst) +return ret; + +if (dst->dev->flags & IFF_NOARP) { +ret = rdma_copy_addr(addr, dst->dev, NULL); +} else { +neigh = dst->neighbour; +if (neigh && (neigh->nud_state & NUD_VALID)) +ret = rdma_copy_addr(addr, neigh->dev, neigh->ha); +} + +dst_release(dst); +return ret; +} + +static int addr_resolve_remote(struct sockaddr *src_in, +struct sockaddr *dst_in, +struct rdma_dev_addr *addr) +{ +if (src_in->sa_family == AF_INET) { +return addr4_resolve_remote((struct sockaddr_in *)src_in, +(struct sockaddr_in *)dst_in, addr); +} else +return addr6_resolve_remote((struct sockaddr_in6 *)src_in, +(struct sockaddr_in6 *)dst_in, addr); +} + static void process_req(struct work_struct *work) { struct addr_req *req, *temp_req; < at >< at > -279,29 +336,58 < at >< at > static int addr_resolve_local(struct sockaddr *src_in, struct rdma_dev_addr *addr) { struct net_device *dev; -__be32 src_ip = ((struct sockaddr_in *)src_in)->sin_addr.s_addr; -__be32 dst_ip = ((struct sockaddr_in *)dst_in)->sin_addr.s_addr; -int ret; +int ret = -EADDRNOTAVAIL; -dev = ip_dev_find(&init_net, dst_ip); -if (!dev) -return -EADDRNOTAVAIL; +if (dst_in->sa_family == AF_INET) { +__be32 src_ip = ((struct sockaddr_in *)src_in)->sin_addr.s_addr; +__be32 dst_ip = ((struct sockaddr_in *)dst_in)->sin_addr.s_addr; + +dev = ip_dev_find(&init_net, dst_ip); +if (!dev) +return -EADDRNOTAVAIL; -if (ipv4_is_zeronet(src_ip)) { -src_in->sa_family = dst_in->sa_family; -((struct sockaddr_in *)src_in)->sin_addr.s_addr = dst_ip; -ret = rdma_copy_addr(addr, dev, dev->dev_addr); -} else if (ipv4_is_loopback(src_ip)) { -ret = rdma_translate_ip(dst_in, addr); -if (!ret) -memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +if (ipv4_is_zeronet(src_ip)) { +src_in->sa_family = dst_in->sa_family; +((struct sockaddr_in *)src_in)->sin_addr.s_addr = dst_ip; +ret = rdma_copy_addr(addr, dev, dev->dev_addr); +} else if (ipv4_is_loopback(src_ip)) { +ret = rdma_translate_ip(dst_in, addr); +if (!ret) +memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +} else { +ret = rdma_translate_ip(src_in, addr); +if (!ret) +memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +} +dev_put(dev); } else { -ret = rdma_translate_ip(src_in, addr); -if (!ret) -memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +struct in6_addr *a = &((struct sockaddr_in6 *)dst_in)->sin6_addr; + +for_each_netdev(&init_net, dev) +if (ipv6_chk_addr(&init_net, &((struct sockaddr_in6 *) addr)->sin6_addr, dev, 1)) +break; + +if (!dev) +return -EADDRNOTAVAIL; + +a = &((struct sockaddr_in6 *)src_in)->sin6_addr; + +if (ipv6_addr_any(a)) { +src_in->sa_family = dst_in->sa_family; +((struct sockaddr_in6 *)src_in)->sin6_addr = +((struct sockaddr_in6 *)dst_in)->sin6_addr; +ret = rdma_copy_addr(addr, dev, dev->dev_addr); +} else if (ipv6_addr_loopback(a)) { +ret = rdma_translate_ip(dst_in, addr); +if (!ret) +memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +} else { +ret = rdma_translate_ip(src_in, addr); +if (!ret) +memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN); +} } -dev_put(dev); return ret; } < at >< at > -344,7 +430,7 < at >< at > int rdma_resolve_ip(struct rdma_addr_client *client, case -ENODATA: req->timeout = msecs_to_jiffies(timeout_ms) + jiffies; queue_req(req); -addr_send_arp((struct sockaddr_in *)dst_in); +addr_send_arp(dst_in); break; default: ret = req->status;

[ofa-general] [PATCH] mlx4_ib: Fix MTT leakage in resize_cq

Jack Morgenstein — 2008-12-01T12:32:38

mlx4_ib: Fix MTT leakage in resize_cq. MTTs associated with the old CQE buffer were not deallocated (i.e., returned to the free pool). As a result, the MTT free pool was eventually completely emptied (for users of resize_cq). Once the resize_cq command returns successfully from FW, FW no longer accesses the old CQE buffer, so it is safe to deallocate the MTT entries used by the old CQE buffer. Finally, if the resize_cq command fails, the MTTs allocated for the new CQEs buffer also need to be de-allocated. Signed-off-by: Jack Morgenstein dev.mellanox.co.il> --- Roland, Resize_cq is already in kernel 2.6.27, so I think this fix should go into a "latest stable" kernel version (say, 2.6.27.8). For heavy resize_cq users, such as MPI, this fix is significant. There is no need to call mlx4_mtt_cleanup separately for userspace and kernel-space paths -- this way the call is made in only one place. Actually, we cannot call mlx4_mtt_cleanup inside a spinlock anyway, because it may sleep (it indirectly invokes mlx4_UNMAP_ICM). There is also no need to protect the mlx4_mtt_cleanup call against parallel access (spinlock), since the cleanup only is relevant to FW access to the buffer (MTT entries); the buffer itself is still available (or not!) to the driver. Finally, I save the mtt for cleanup because the switchover to the new buffer may have already occurred (in mlx4_ib_poll_one) when the resize_cq call returns (in which case cq->buf.mtt has already been overwritten with the new CQE buffer info). diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index d0866a3..20d6833 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c < at >< at > -343,6 +343,7 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) { struct mlx4_ib_dev *dev = to_mdev(ibcq->device); struct mlx4_ib_cq *cq = to_mcq(ibcq); +struct mlx4_mtt mtt; int outst_cqe; int err; < at >< at > -376,10 +377,12 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) goto out; } +mtt = cq->buf.mtt; err = mlx4_cq_resize(dev->dev, &cq->mcq, entries, &cq->resize_buf->buf.mtt); if (err) goto err_buf; +mlx4_mtt_cleanup(dev->dev, &mtt); if (ibcq->uobject) { cq->buf = cq->resize_buf->buf; cq->ibcq.cqe = cq->resize_buf->cqe; < at >< at > -406,6 +409,7 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) goto out; err_buf: +mlx4_mtt_cleanup(dev->dev, &cq->resize_buf->buf.mtt); if (!ibcq->uobject) mlx4_ib_free_cq_buf(dev, &cq->resize_buf->buf, cq->resize_buf->cqe); diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c diff --git a/drivers/infiniband/hw/mthca/mthca_mad.c b/drivers/infiniband/hw/mthca/mthca_mad.c

[ofa-general] ofa_1_4_kernel 20081201-0200 daily build status

Vladimir Sokolovsky (Mellanox — 2008-12-01T11:20:34

This email was generated automatically, please do not reply git_url: git://git.openfabrics.org/ofed_1_4/linux-2.6.git git_branch: ofed_kernel Common build parameters: Passed: Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.24 Passed on i686 with linux-2.6.22 Passed on i686 with linux-2.6.21.1 Passed on i686 with linux-2.6.26 Passed on i686 with linux-2.6.27 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on x86_64 with linux-2.6.16.43-0.3-smp Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.60-0.21-smp Passed on x86_64 with linux-2.6.18-1.2798.fc6 Passed on x86_64 with linux-2.6.18-8.el5 Passed on x86_64 with linux-2.6.18-53.el5 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on x86_64 with linux-2.6.18-93.el5 Passed on x86_64 with linux-2.6.22 Passed on x86_64 with linux-2.6.21.1 Passed on x86_64 with linux-2.6.22.5-31-default Passed on x86_64 with linux-2.6.25 Passed on x86_64 with linux-2.6.24 Passed on x86_64 with linux-2.6.26 Passed on x86_64 with linux-2.6.9-55.ELsmp Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on x86_64 with linux-2.6.27 Passed on x86_64 with linux-2.6.9-67.ELsmp Passed on x86_64 with linux-2.6.9-78.ELsmp Passed on ia64 with linux-2.6.16 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.16.21-0.8-default Passed on ia64 with linux-2.6.18 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.21.1 Passed on ia64 with linux-2.6.22 Passed on ia64 with linux-2.6.23 Passed on ia64 with linux-2.6.24 Passed on ia64 with linux-2.6.25 Passed on ia64 with linux-2.6.26 Passed on ppc64 with linux-2.6.16 Passed on ppc64 with linux-2.6.17 Passed on ppc64 with linux-2.6.18 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.18-8.el5 Failed:

Re: [ofa-general] opensm support for toroidal meshes

Sven-Arne Reinemo — 2008-12-01T08:43:02

Hi all, I just thought I should share some simulation results with you. I just did some simulations to test Bob's suggested changes, and I see that the number VLs required for both 2d and 3d tori is either _reduced_ or _equal_ to that of the existing implementation. Moreover, the port reordering seems to work very well on tori that is not cabled regularly with regards to port numbering. As state by Bob it makes LASH route them as if they where regularly cabled, which is the most optimal for LASH. Below are some numbers for the VLs required for various 2d and 3d tori. Be aware that the number of VLs required would be different if the torus has a different size along each dimension. Tori Current Patch 4x4 2 2 5x5 3 3 6x6 4 3 7x7 3 3 8x8 6 4 9x9 4 4 10x10 9 6 11x11 5 5 12x12 9 4 13x13 7 7 14x14 12 8 15x15 10 10 4x4x4 5 5 5x5x5 5 4 6x6x6 10 6 7x7x7 10 10 8x8x8 12 9 9x9x9 14 14 Regards, Sven-Arne On ma., 2008-11-10 at 15:47 -0600, Robert Pearson wrote:

RE: [ofa-general] [RMDA CM IPv6 support. PATCHv4 1/6]AF_INET6support for rdma_bind_addr

Sean Hefty — 2008-12-01T07:41:37

The patch set taken collectively looks good to me. I think it makes sense to view the series as 2 patches, one for ib_addr (patches 4-6), and one for rdma_cm (patches 1-3). The ib_addr patch should come first. - Sean

[ofa-general] GitWeb really slow

Nicolas Morey Chaisemartin — 2008-12-01T07:11:31

Hi, This is not necessary the best place to post it but I was wondering why is ofed's gitweb so slow on the main page? It takes only a few seconds to display all the repository on kernel.org (and there's a lot more) but it takes nearly a minute to display the OFED git main page... I know it's probably not the most critical issue you have to work on but I connect quite often on this page and it starts to be really bugging me. And I'm probably not the only one ;) Thanks in advance Nicolas Morey-Chaisemartin

Re: [ofa-general] receiving wrong data after trying to allocationa too large memory chunk

Asmund Ostvold — 2008-12-01T07:01:09

I apologize for my dyslectic subject. This should be better. We would very much like to know if anybody else can reproduce the results? If you need more info please contact us. Regards, Asmund (dyslectic programmer) Asmund Ostvold wrote:

[ofa-general] Re: {PATCH] [2] opensm: per mesh data

Sasha Khapyorsky — 2008-11-30T23:54:14

lash_cleanup() is called at start of LASH processor, so mesh will keep allocated data between routing calculation cycles. But as far as I can see it is not used there. Also osm_mesh_cleanup() is not called on lash deletion and we have a memory leak. Maybe osm_mesh_cleanup() should be static function (mesh_cleanup()) and be called somewhere at end of osm_do_mesh_analysis()? Sasha

[ofa-general] Re: [PATCH][10] opensm: hook mesh code into lash(updated)

Sasha Khapyorsky — 2008-11-30T21:34:02

I think it should be fixed in related patch. Nice finding. Would snprintf() be more suitable here in order to prevent potential overflow? (This is a nit - dimension value is limited now in mesh_info structure). This 'if (sw1 == sw2)' is related for non mesh case too, right? Sasha