gmane.linux.drivers.openib

Re: [ofa-general] Re: [ewg] rhel 5.2 iSER support?

Jesse Butler — 2008-12-04T04:33:06

For me, this ended up being that the CM service was not yet configured at the time that I was attempting to login. So, it is possible that you need to set the port settling time attribute to ensure that the port are configured properly. Sameer, ping me directly if you need further assistance. /jb On Dec 3, 2008, at 2:34 AM, Or Gerlitz wrote:

[ofa-general] [PATCH 6/6] IB/ipath - Add locking for interrupt useof ipath_pd contexts vs free.

Ralph Campbell — 2008-12-03T18:37:18

From: Dave Olson qlogic.com> Fixes timing race resulting in panic. Not a performance sensitive path. Signed-off-by: Dave Olson qlogic.com> --- drivers/infiniband/hw/ipath/ipath_driver.c | 49 +++++++++++++++---------- drivers/infiniband/hw/ipath/ipath_file_ops.c | 21 +++++------ drivers/infiniband/hw/ipath/ipath_init_chip.c | 1 + drivers/infiniband/hw/ipath/ipath_kernel.h | 2 + drivers/infiniband/hw/ipath/ipath_keys.c | 2 + drivers/infiniband/hw/ipath/ipath_mad.c | 2 + drivers/infiniband/hw/ipath/ipath_verbs.c | 3 +- 7 files changed, 49 insertions(+), 31 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_driver.c b/drivers/infiniband/hw/ipath/ipath_driver.c index ad0aab6..69c0ce3 100644 --- a/drivers/infiniband/hw/ipath/ipath_driver.c +++ b/drivers/infiniband/hw/ipath/ipath_driver.c < at >< at > -661,6 +661,8 < at >< at > bail: static void __devexit cleanup_device(struct ipath_devdata *dd) { int port; +struct ipath_portdata **tmp; +unsig

[ofa-general] [PATCH 5/6] IB/ipath - fix spi_pioindex value.

Ralph Campbell — 2008-12-03T18:37:12

From: Dave Olson qlogic.com> ipath_piobufbase was a single value offset, but is multiple values on newer chips, so use only the 32 bits for the 2K buffers (4K buffers are currently used only by the driver). Signed-off-by: Dave Olson qlogic.com> --- drivers/infiniband/hw/ipath/ipath_file_ops.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_file_ops.c b/drivers/infiniband/hw/ipath/ipath_file_ops.c index 1af1f3a..ceab52c 100644 --- a/drivers/infiniband/hw/ipath/ipath_file_ops.c +++ b/drivers/infiniband/hw/ipath/ipath_file_ops.c < at >< at > -223,8 +223,13 < at >< at > static int ipath_get_base_info(struct file *fp, (unsigned long long) kinfo->spi_subport_rcvhdr_base); } -kinfo->spi_pioindex = (kinfo->spi_piobufbase - dd->ipath_piobufbase) / -dd->ipath_palign; +/* + * All user buffers are 2KB buffers. If we ever support + * giving 4KB buffers to user processes, this will need some + * work. + */ +kinfo->spi_pioindex = (

[ofa-general] [PATCH 4/6] IB/ipath - Only do 1X workaround on rev1chips.

Ralph Campbell — 2008-12-03T18:37:07

From: Dave Olson qlogic.com> Only do 1X workaround on rev1 chips. Signed-off-by: Dave Olson qlogic.com> --- drivers/infiniband/hw/ipath/ipath_iba7220.c | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba7220.c b/drivers/infiniband/hw/ipath/ipath_iba7220.c index 3b38bc9..b2a9d4c 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba7220.c +++ b/drivers/infiniband/hw/ipath/ipath_iba7220.c < at >< at > -2452,13 +2452,14 < at >< at > static int ipath_7220_ib_updown(struct ipath_devdata *dd, int ibup, u64 ibcs) } } /* - * if we are in 1X, and are in autoneg width, it - * could be due to an xgxs problem, so if we haven't + * if we are in 1X on rev1 only, and are in autoneg width, + * it could be due to an xgxs problem, so if we haven't * already tried, try twice to get to 4X; if we * tried, and couldn't, report it, since it will * probably not be what is desired. */ -if ((dd->ipath_link_width_enabled & (IB_WIDT

[ofa-general] [PATCH 3/6] IB/ipath - don't count IB symbol and linkerrors unless link is UP

Ralph Campbell — 2008-12-03T18:37:01

From: Dave Olson qlogic.com> Implements the ignoring of ibsymbol errors and linkrecover errors while the link is at less than INIT (long needed), to get accurate counts. Particularly an issue when doing non-IBTA DDR negotiation with chips from vendors that do not support IBTA mode negotiation. If the driver is unloaded, and there is a delta, the adjusted counters are written back to the chip, so they stay adjusted across driver reload. Signed-off-by: Dave Olson qlogic.com> --- drivers/infiniband/hw/ipath/ipath_iba6120.c | 61 ++++++++++++++++++++++ drivers/infiniband/hw/ipath/ipath_iba7220.c | 76 ++++++++++++++++++++++++++- drivers/infiniband/hw/ipath/ipath_kernel.h | 13 +++++ drivers/infiniband/hw/ipath/ipath_stats.c | 8 +++ 4 files changed, 155 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_iba6120.c b/drivers/infiniband/hw/ipath/ipath_iba6120.c index 421cc2a..fbf8c53 100644 --- a/drivers/infiniband/hw/ipath/ipath_iba6120.c +++ b/d

[ofa-general] [PATCH 2/6] IB/ipath - Check the return value ofdma_map_single for errors

Ralph Campbell — 2008-12-03T18:36:56

This fixes an obvious oversight where the return value is not checked for error. Signed-off-by: Ralph Campbell qlogic.com> --- drivers/infiniband/hw/ipath/ipath_sdma.c | 21 ++++++++++++++++----- 1 files changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_sdma.c b/drivers/infiniband/hw/ipath/ipath_sdma.c index 284c9bc..8e255ad 100644 --- a/drivers/infiniband/hw/ipath/ipath_sdma.c +++ b/drivers/infiniband/hw/ipath/ipath_sdma.c < at >< at > -698,10 +698,8 < at >< at > retry: addr = dma_map_single(&dd->pcidev->dev, tx->txreq.map_addr, tx->map_len, DMA_TO_DEVICE); -if (dma_mapping_error(&dd->pcidev->dev, addr)) { -ret = -EIO; -goto unlock; -} +if (dma_mapping_error(&dd->pcidev->dev, addr)) +goto ioerr; dwoffset = tx->map_len >> 2; make_sdma_desc(dd, sdmadesc, (u64) addr, dwoffset, 0); < at >< at > -741,6 +739,8 < at >< at > retry: dw = (len + 3) >> 2; addr = dma_map_single(&dd->pcidev->dev, sge->vaddr, dw << 2, DMA_TO_DEVICE); +if (dma_mapping_er

[ofa-general] [PATCH 1/6] IB/ipath - Fix PSN of subsequent send WQEsafter an RDMA read resend

Ralph Campbell — 2008-12-03T18:36:51

The PSN of the first packet after an RDMA read is based on the size of the RDMA read request. This is calculated correctly for the WQE sent after the first request message but not on subsequent requests if the RDMA read is resent. Signed-off-by: Ralph Campbell qlogic.com> --- drivers/infiniband/hw/ipath/ipath_rc.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/ipath/ipath_rc.c b/drivers/infiniband/hw/ipath/ipath_rc.c index 7b93cda..9170710 100644 --- a/drivers/infiniband/hw/ipath/ipath_rc.c +++ b/drivers/infiniband/hw/ipath/ipath_rc.c < at >< at > -573,9 +573,8 < at >< at > int ipath_make_rc_req(struct ipath_qp *qp) ohdr->u.rc.reth.length = cpu_to_be32(qp->s_len); qp->s_state = OP(RDMA_READ_REQUEST); hwords += sizeof(ohdr->u.rc.reth) / sizeof(u32); -bth2 = qp->s_psn++ & IPATH_PSN_MASK; -if (ipath_cmp24(qp->s_psn, qp->s_next_psn) > 0) -qp->s_next_psn = qp->s_psn; +bth2 = qp->s_psn & IPATH_PSN_MASK; +qp->s_psn = wqe->lpsn + 1; ss = NULL;

[ofa-general] [PATCH 0/6] IB/ipath -- fixes for 2.6.29

Ralph Campbell — 2008-12-03T18:36:46

The following patches fix some problems recently found: IB/ipath - Fix PSN of subsequent send WQEs after an RDMA read resend IB/ipath - Check the return value of dma_map_single for errors IB/ipath - Add locking for interrupt use of ipath_pd contexts vs free. IB/ipath - don't count IB symbol and link errors unless link is UP IB/ipath - Only do 1X workaround on rev1 chips. IB/ipath - fix spi_pioindex value.

[ofa-general] [PATCH V2] mlx4: check for FW version which properlysupports resize_cq

Jack Morgenstein — 2008-12-03T18:20:04

mlx4: check for FW version which properly supports resize_cq. If the ConnectX cards have earlier FW installed, the resize_cq command will return -ENOSYS. Fixes Bugzilla 1415. Signed-off-by: Jack Morgenstein dev.mellanox.co.il> --- Roland, I decided to solve the resize_cq problem "locally", and not deal with the can of worms that my last patch opens. We already do something similar for qp's (NoErrorCompletion bit) I did this properly this time (#define in cq.h) diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c index 1830849..a637f89 100644 --- a/drivers/infiniband/hw/mlx4/cq.c +++ b/drivers/infiniband/hw/mlx4/cq.c < at >< at > -347,6 +347,9 < at >< at > int mlx4_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata) int outst_cqe; int err; +if (dev->dev->caps.fw_ver < MLX4_FW_VER_RESIZE_CQ) +return -ENOSYS; + mutex_lock(&cq->resize_mutex); if (entries < 1 || entries > dev->dev->caps.max_cqes) { diff --git a/include/linux/mlx4/cq.h b/include/linux/mlx4/cq.h

[ofa-general] CentOS non-OFED opensm package needs /dev/infiniband

Cameron Harr — 2008-12-03T18:01:32

This is more of an FYI that can maybe help others, but perhaps it is a bug that needs to be filed somewhere. I've been trying to get the SM running without using OFED, rather "yum install opensm" and using the distro's kernel-provided IB drivers. I've banged my head around for a while trying to figure out why I was getting the following messages: --- Error from osm_opensm_bind (0x2A) Perhaps another instance of OpenSM is already running --- It turns out the SM is looking for devices in /dev/infiniband (umad0, uverbs0), but the kernel-provided ib_umad and ib_uverbs modules place the devices in /dev. By creating a link "infiniband" in the /dev/ directory to the /dev/ directory, things magically started to work. "cd /dev; ln -s . infiniband"

[ofa-general] error building ofa_kernel RPMs

David Rioja — 2008-12-03T17:58:06

Dear all, I've tried to build OFED 1.2.5 in my server with fedora core 9. First I had a problem while building ofa_user RPMs but this got solved by installing zlib-static package. I think I have all the rest of dependencies installed, both x68_64 and i386 versions. Now the error comes in the next step: building ofa_kernel RPMs. Here I paste the echoes: *** Building InfiniBand Software RPMs. Please wait... Building ofa_user RPMs. Please wait... Running rpmbuild --rebuild --define '_topdir /var/tmp/OFEDRPM' --define '_prefix /usr' --define 'build_root /var/tmp/OFED' --define 'configure_options --with-libcxgb3 --with-libibcm --with-libibverbs --with-libmlx4 --with-libmthca --with-librdmacm --with-mstflint --with-perftest --sysconfdir=/etc --mandir=/usr/share/man' --define 'configure_options32 --with-libcxgb3 --with-libibcm --with-libibverbs --with-libmlx4 --with-libmthca --with-librdmacm --sysconfdir=/etc --mandir=/usr/share/man' --define 'build_32bit 1' --define '_mandir /usr/share/man' /ro

Re: [ofa-general] ibv_post_send fails when using malloc in a specialway

Roland Dreier — 2008-12-03T17:44:59

The most obvious explanation is that the physical pages underlying your allocation are different after the free/re-valloc. This could happen without a system call I guess if a page is faulted in. - R.

[ofa-general] Multiple job failures at same time

Glanfield, Wayne — 2008-12-03T17:37:33

We have just experienced a problem where 5 jobs failed at the same time ~15:50 GMT with similar messages in their respective output files. Does anybody have any idea what could have cause this and what the messages mean. One of the nodes "cfd-cnsl-0364" was found to have shutdown but could this take out other jobs? They were not running on this node, This is a commercial CFD code which is using hp-mpi 2.2.5, we are running ofed 1.3.1 and using verbs api with Mellanox ConnectX HCA Thanks Wayne JOB #1 starccm+: Rank 0:52: MPI_Test: ibv_poll_cq(): bad status 12 starccm+: Rank 0:50: MPI_Test: ibv_poll_cq(): bad status 12 starccm+: Rank 0:55: MPI_Test: ibv_poll_cq(): bad status 12 starccm+: Rank 0:55: MPI_Test: self cfd-cnsl-0355 peer cfd-cnsl-0365 (rank: 126) starccm+: Rank 0:55: MPI_Test: error message: transport retry exceeded error starccm+: Rank 0:52: MPI_Test: self cfd-cnsl-0355 peer cfd-cnsl-0365 (rank: 120) starccm+: Rank 0:52: MPI_Test: error message: transport retry exceeded error starccm+: Rank 0:5

RE: [ofa-general] CX4 Optical Cables

Gilad Shainer — 2008-12-03T17:25:06

The Topspin cards might not have the power circuitry required for the fiber cables. We might be able to determine this with the PSID info from those cards. Feel free to send me that info directly. Gilad. -----Original Message----- From: general-bounces< at >lists.openfabrics.org [mailto:general-bounces< at >lists.openfabrics.org] On Behalf Of Charles Taylor Sent: Wednesday, December 03, 2008 8:15 AM To: OpenFabrics General Cc: Craig Prescott Subject: [ofa-general] CX4 Optical Cables Operational Question.... We have some storage nodes across the room from our IB switch and reach them with 10M copper cables. These 10M cables have been an ongoing issue for us as they fail (start showing symbol errors) fairly frequently. We decided to start replacing them with fiber (optics imbedded in CX4 connectors) cables. We got a couple of "intel- connect" 20m cables (note that I think Intel sold the business to EMCore). When we installed the first cable yesterday we were surprised to see that we could not g

Re: [ofa-general] RE: [BUG REPORT] mlx4: Incorrect event is generatedwhen SM is changing

Moni Shoua — 2008-12-03T17:17:03

Tried rc7. It works OK now. Thanks.

[ofa-general] RE: [BUG REPORT] mlx4: Incorrect event is generatedwhen SM is changing

Jack Morgenstein — 2008-12-03T16:42:58

It is related, since one of the capability mask bits that was wiped out before the fix mentioned below was: 25: IsClientReregistrationSupported (IB Spec 1.2.1, table 146, page 831, Capability Mask bits) - Jack

Re: [ofa-general] git http url for ofed_1_3/management.git

Sasha Khapyorsky — 2008-12-03T16:31:32

It looks http:// basically works when you are using http://www.openfabrics.org/... (http://git.openfabrics.org redirects to gitweb http://www.openfabrics.org/git/ ). The problem with http://www.openfabrics.org/ofed_1_3/* repositories is a lack of server-info needed for http/ftp protocols. You can ask Vlad to run 'git update-server-info' there. Alternatively you can use my repository http://www.openfabrics.org/~sashak/management.git , there is ofed_1_3 branch. Sasha

[ofa-general] Re: [BUG REPORT] mlx4: Incorrect event is generatedwhen SM is changing

Roland Dreier — 2008-12-03T16:31:54

Can you try 2.6.28-rc7? I think the fix 9a5aa622 ("mlx4_core: Save/restore default port IB capability mask") sounds related, and that went in after -rc6. - R.

Re: [ofa-general] CX4 Optical Cables

Roland Dreier — 2008-12-03T16:28:14

> We are just wondering if anyone knows what changed between these two > card/chip versions such that the optical connectors work transparently > with the the mellanox cards but not at all with the TopSpin/Cisco > cards. optical cables require power from the HCA port (to run the optics, obviously). Not all HCAs have power available at the port. I don't know of a very good way to tell which do. The Topspin/Cisco HCAs are basically OEM versions of Mellanox HCAs so maybe Mellanox can tell you which versions support power. - R.

[ofa-general] CX4 Optical Cables

Charles Taylor — 2008-12-03T16:14:41

Operational Question.... We have some storage nodes across the room from our IB switch and reach them with 10M copper cables. These 10M cables have been an ongoing issue for us as they fail (start showing symbol errors) fairly frequently. We decided to start replacing them with fiber (optics imbedded in CX4 connectors) cables. We got a couple of "intel- connect" 20m cables (note that I think Intel sold the business to EMCore). When we installed the first cable yesterday we were surprised to see that we could not get link status on the connection. The HCA in the host is a Cisco/TopSpin "LionCub" (board id=MT_00A0000001). We have some newer (by one year) Mellanox branded HCAs on some other nodes (MHEA28-1TC, Lion cub, board id=MT_02F0110001). When we ran the optical cable from the same switch port to the mellanox HCAs, we got link status and a LID was issued - no problem. We are just wondering if anyone knows what changed between these two card/chip versions such that

Re: [ofa-general] RE: [BUG REPORT] mlx4: Incorrect event is generatedwhen SM is changing

Moni Shoua — 2008-12-03T16:14:00

Since OFED-1.4 baseline is linux-2.6.27, the patches work fine there. It will be a problem in **next** OFED (which will be probably based at least on linux-2.6.28) if the bug that is described here isn't solved. In the bottom line I'd like to keep the patches in OFED-1.4 but return to this issue in OFED-1.5 thanks