http://permalink.gmane.org/gmane.network.openfabrics.enterprise hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5900 <pre> Indeed. However, libcxgb3 seems to return -1 if num_sge &gt;1: static inline int iwch_build_rdma_read(union t3_wr *wqe, struct ibv_send_wr *wr, uint8_t *flit_cnt) { if (wr-&gt;num_sge &gt; 1) return -1; ... and t3b_post_send() returns the error returned by iwch_build_rdma_read()... What version of libcxgb3 are you using? </pre> Steve Wise 2013-06-13T16:10:22 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5899 <pre> We get the following when enabling all debug levels: # export DAPL_DBG_TYPE=0xffff # dapltest -T P -s 10.0.0.1 -D ofa-v2-iwarp -d -i 10 RR 4096 2 ... Test[b0df]: Begin... 3650M3:10f8:5d4ea700: 42132 us(177 us): dapl_ep_post_rdma_read (0x63c0d0, 2, 0x63f8e0, (nil), 0x7fff51d7b200, 0) 3650M3:10f8:5d4ea700: 42137 us(5 us): post_snd: ep 0x7f905c548a80 op 6537424 ck 0x4 sgs3650M3:10f8:5d4ea700: 42138 us(1 us): post_snd: lkey 0x5ff va 0x63d700 len 4096 3650M3:10f8:5d4ea700: 42142 us(4 us): post_snd: lkey 0x5ff va 0x63e700 len 4096 3650M3:10f8:5d4ea700: 42145 us(3 us): post_snd_rdma: rkey 0xbff va 0x00000000645900 3650M3:10f8:5d4ea700: 42149 us(4 us): post_snd: op 0x4 flags 0x2 sglist 0x63f8e0, 2 3650M3:10f8:5d4ea700: 42153 us(4 us): dapl_ep_post_rdma_read () returns 0x0 3650M3:10f8:5d4ea700: 42159 us(6 us): dapl_ep_post_rdma_read (0x63c0d0, 2, 0x63f8e0, (nil), 0x7fff51d7b200, 0) 3650M3:10f8:5d4ea700: 42164 us(5 us): post_snd: ep 0x7f905c548a80 op 6537424 ck 0x4 sgs3650M3:10f8:5d4ea700: 42165 us(1 us): p</pre> John Jolly 2013-06-13T15:51:47 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5898 <pre> It definitely isn't supported. We should see some synchronous error, though, when posting the read work requests. IE dat_ep_post_rdma_read() should be failing. It would be nice if you could verify this. There were some fixes in the flush logic of the t4 sw recently. They aren't upstream yet, but you can most likely get them from the latest T4 kit from chelsio's web site. You should definitely send email to support-ut6Up61K2wZBDgjK7y7TUQ&lt; at &gt;public.gmane.org and get a problem report opened for this. Steve. </pre> Steve Wise 2013-06-13T14:03:41 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5897 <pre>Steve, On Tue, Jun 11, 2013 at 02:26:01PM -0500, Steve Wise wrote: 22:00.0 Ethernet controller: Chelsio Communications Inc T320 10GbE Dual Port Adapter Subsystem: Chelsio Communications Inc Device 0001 Flags: bus master, fast devsel, latency 0, IRQ 11 Memory at e7880000 (64-bit, non-prefetchable) [size=4K] Memory at e7000000 (64-bit, non-prefetchable) [size=8M] Memory at e7881000 (64-bit, non-prefetchable) [size=4K] [virtual] Expansion ROM at e6000000 [disabled] [size=512K] Capabilities: [40] Power Management version 3 Capabilities: [48] Message Signalled Interrupts: Mask- 64bit+ Count=1/32 Enable- Capabilities: [58] Express Endpoint, MSI 00 Capabilities: [94] Vital Product Data &lt;?&gt; Capabilities: [9c] MSI-X: Enable+ Mask- TabSize=32 Capabilities: [100] Device Serial Number 01-00-00-00-01-00-00-00 Capabilities: [300] Advanced Error Reporting Kernel driver in use: cxgb3 Kernel modules: cxgb3 So is this a matter of telling our customer to stop doin</pre> John Jolly 2013-06-12T22:45:12 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5896 <pre>Is this T3 or T4? If dapl calls ibv_post_send() with wr-&gt;num_sge &gt; 1 for a read request, the chelsio provider will return -EINVAL in T4 and -1 in T3. We shouldn't see a hang though. :) On 6/11/2013 2:18 PM, Steve Wise wrote: </pre> Steve Wise 2013-06-11T19:26:01 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5895 <pre>Note: The iWARP wire protocol only supports 1 sink (and source) scatter-gather-entry. On 6/11/2013 9:01 AM, John Jolly wrote: </pre> Steve Wise 2013-06-11T19:18:24 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5894 <pre>All, We're trying to track down a problem where dapltest hangs using more than one segment on the RR (RDMA read) performance test. Tried uDAPL RDMA_READ test with "number of segments" parameter set to 1, test worked fine. RDMA_WRITE and SEND/RECV tests were executed without any issue. ---Steps to Reproduce--- 1. Configure iWARP on server and client machines connected via Chelsio adapter. Server configuration: x3755b:/ # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:07:43:07:1B:B8 inet addr:10.0.0.1 Bcast:10.255.255.255 Mask:255.0.0.0 inet6 addr: fe80::207:43ff:fe07:1bb8/64 Scope:Link UP BROADCAST MULTICAST MTU:1500 Metric:1 RX packets:10011 errors:0 dropped:0 overruns:0 frame:0 TX packets:9969 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:12629636 (12.0 Mb) TX bytes:2510976 (2.3 Mb) Interrupt:11 Memory:e7880000-e7880fff x3755b:/ # cat /etc/dat.conf ofa-v2-iwarp u2.0 nonth</pre> John Jolly 2013-06-11T14:01:26 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5893 <pre>Here is a quick update on today's discussions: Thanks, Arlin Davis (for Woody) </pre> Davis, Arlin R 2013-06-10T16:29:08 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5892 <pre>Agenda: 1. OFED 3.5 dot releases status: Rupert a. OFED 3.5.1 - with RHEL 6.4 b. OFED 3.5.2 - with small fixes and libraries update 2. Opens Tziporet </pre> Tziporet Koren 2013-06-10T14:34:28 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5890 <pre> the device name was not being updated during a query. Copy the hca name into ia_attr-&gt;adapter_name for consumers. Signed-off-by: Arlin Davis &lt;arlin.r.davis-ral2JQCrhuEAvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; --- dapl/openib_common/util.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/dapl/openib_common/util.c b/dapl/openib_common/util.c index 33629b8..8b97263 100644 --- a/dapl/openib_common/util.c +++ b/dapl/openib_common/util.c &lt; at &gt;&lt; at &gt; -310,6 +310,9 &lt; at &gt;&lt; at &gt; DAT_RETURN dapls_ib_query_hca(IN DAPL_HCA * hca_ptr, if (ia_attr != NULL) { (void)dapl_os_memzero(ia_attr, sizeof(*ia_attr)); +strncpy(ia_attr-&gt;adapter_name, + ibv_get_device_name(hca_ptr-&gt;ib_trans.ib_dev), + DAT_NAME_MAX_LENGTH - 1); ia_attr-&gt;adapter_name[DAT_NAME_MAX_LENGTH - 1] = '\0'; ia_attr-&gt;vendor_name[DAT_NAME_MAX_LENGTH - 1] = '\0'; ia_attr-&gt;ia_address_ptr = &lt; at &gt;&lt; at &gt; -317,7 +320,7 &lt; at &gt;&lt; at &gt; DAT_RETURN dapls_ib_query_hca(IN DAPL_HCA * hca_ptr, dapl_dbg_log(DAPL_DBG_TYPE_UTIL, " query_hca: %s %s \n</pre> Davis, Arlin R 2013-06-05T23:32:18 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5889 <pre> Signed-off-by: Aravinda Venkatramana &lt;Aravinda.Venkatramana-laKkSmNT4hbQT0dZR+AlfA&lt; at &gt;public.gmane.org&gt; Signed-off-by: Pradeep Satyanarayana &lt;pradeep-r/Jw6+rmf7HQT0dZR+AlfA&lt; at &gt;public.gmane.org&gt; Acked-by: Arlin Davis &lt;arlin.r.davis-ral2JQCrhuEAvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; --- dapl/udapl/linux/dapl_osd.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/dapl/udapl/linux/dapl_osd.h b/dapl/udapl/linux/dapl_osd.h index 7198439..0412461 100644 --- a/dapl/udapl/linux/dapl_osd.h +++ b/dapl/udapl/linux/dapl_osd.h &lt; at &gt;&lt; at &gt; -188,7 +188,7 &lt; at &gt;&lt; at &gt; dapl_os_atomic_inc ( stwcx.%0,0,%2\n\ bne-1b" : "=&amp;r" (tmp), "+m" (v) -: "r" (&amp;v) +: "b" (v) : "cc"); #else /* !__ia64__ */ __asm__ __volatile__ ( &lt; at &gt;&lt; at &gt; -227,7 +227,7 &lt; at &gt;&lt; at &gt; dapl_os_atomic_dec ( stwcx.%0,0,%2\n\ bne-1b" : "=&amp;r" (tmp), "+m" (v) -: "r" (&amp;v) +: "b" (v) : "cc"); #else /* !__ia64__ */ __asm__ __volatile__ ( </pre> Davis, Arlin R 2013-06-05T23:31:52 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5888 <pre>cc'ing ewg Signed-off-by: Pradeep Satyanarayana &lt;pradeep-r/Jw6+rmf7HQT0dZR+AlfA&lt; at &gt;public.gmane.org&gt; Acked-by: Arlin Davis &lt;arlin.r.davis-ral2JQCrhuEAvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; --- test/dapltest/common/dapl_endian.c | 4 ---- 1 files changed, 0 insertions(+), 4 deletions(-) diff --git a/test/dapltest/common/dapl_endian.c b/test/dapltest/common/dapl_endian.c index d93fbb9..c77f2f2 100644 --- a/test/dapltest/common/dapl_endian.c +++ b/test/dapltest/common/dapl_endian.c &lt; at &gt;&lt; at &gt; -77,8 +77,6 &lt; at &gt;&lt; at &gt; DAT_UINT64 DT_Endian64(DAT_UINT64 val) DAT_UINT32 DT_EndianMemHandle(DAT_UINT32 val) { -if (DT_local_is_little_endian) -return val; val = ((val &amp; c1a32) &lt;&lt; 8) | ((val &amp; c1b32) &gt;&gt; 8); val = ((val &amp; c2a32) &lt;&lt; 16) | ((val &amp; c2b32) &gt;&gt; 16); return (val); &lt; at &gt;&lt; at &gt; -88,8 +86,6 &lt; at &gt;&lt; at &gt; DAT_UINT64 DT_EndianMemAddress(DAT_UINT64 val) { DAT_UINT64 val64; -if (DT_local_is_little_endian) -return val; val64 = val; val64 = ((val64 &amp; c1a64) &lt;&lt; 8) | ((val64 &amp; c1b64) &gt;&gt; 8); val64 = ((val64 &amp; c2a64) &lt;&lt; 16) | ((val64 &amp; c2b64) </pre> Davis, Arlin R 2013-06-05T23:31:48 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5887 <pre>Answered to Pavan with the relevant info since its related to the Mellanox release and not OFA Tziporet From: ewg-bounces-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org [mailto:ewg-bounces-ZwoEplunGu2DiMhRP7xlEA&lt; at &gt;public.gmane.orgcs.org] On Behalf Of pavan tc Sent: Monday, May 27, 2013 1:01 PM To: ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org Subject: [ewg] mlx5 and ib_modify_cq Hi, Please respond to my mail ID directly since I am not part of the ewg mailing list. I am using the Mellanox OFED 2.0 release. ib_modify_cq() fails with -ENOSYS here. I see that the mlx5_ib_modify_cq() routine is not implemented in the new mlx5 driver. Can somebody kindly clarify if this will be supported in later releases? Also, what is the recommendation for existing consumers of this API? Thanks, Pavan _______________________________________________ ewg mailing list ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg</pre> Tziporet Koren 2013-05-28T08:16:16 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5886 <pre>Thanks, Vladimir! This makes sense to me. So, today we (will) have CONFIG_COMPAT_RHEL_6_{3,4} and CONFIG_COMPAT_SLES_11_2 macros in OFED-3.5.x? I'm also looking forward to see your dynamic construction changes! Thanks again.. Ihara On May 27, 2013, at 10:36 PM, Vladimir Sokolovsky &lt;vlad-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb&lt; at &gt;public.gmane.org&gt; wrote: </pre> Shuichi Ihara 2013-05-27T16:36:48 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5885 <pre> ----- Original Message ----- Currently, in OFED-3.5*, this is the only way to compile kernel module with OFED. I plan to add dynamic construction of the compat.mk by parsing kernel headers to avoid using "CONFIG_COMPAT_RHEL_6_4" like macros. It depends if I will accomplish dynamic construction of the compat.mk by then. For other distributions need to define the corresponding macro or create a set of CFLAGS that will fit the kernel API. Currently, there is no relevant documentation available. I hope it will be ready by the next OFED release. Regards, Vladimir </pre> Vladimir Sokolovsky 2013-05-27T13:36:39 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5884 <pre> Hi, I'm testing lustre with OFED-3.5.1 against RHEL6.4 kernel. There are a lot of duplicated functions and macro name in the compat headers that there are already defined in RHEL6.4's kernel as their own backport. However, this is fine. In order to ignore them, I see there is special macro CONFIG_COMPAT_RHEL_6_4. Thanks for this superset for RHEL6.4 kernel! I have succeeded Lustre build with "-DCONFIG_COMPAT_RHEL_6_4". However, in general (not only lustre, but also building kernel module with OFED-3.x against RHEL kernel) I'm wondering what is proper process if we want to build with OFED and RHEL kernel. And, here is my additional questions... 1. OFED-3.x is NO auto detection kernel version and set superset macro (like CONFIG_COMPAT_RHEL_6_4) in OFED itself? Does user have to make sure kernel version and set -DCONFIG_COMPAT_RHEL_6_4 to build? 2. What happen if new new distribution update release out? We will see CONFIG_COMPAT_RHEL_6_{5,6}?? 3. How about other distribution case? Any development docume</pre> Shuichi Ihara 2013-05-27T11:47:49 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5883 <pre>Hi, Please respond to my mail ID directly since I am not part of the ewg mailing list. I am using the Mellanox OFED 2.0 release. ib_modify_cq() fails with -ENOSYS here. I see that the mlx5_ib_modify_cq() routine is not implemented in the new mlx5 driver. Can somebody kindly clarify if this will be supported in later releases? Also, what is the recommendation for existing consumers of this API? Thanks, Pavan _______________________________________________ ewg mailing list ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg</pre> pavan tc 2013-05-27T09:53:26 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5882 <pre>Monday, January 21, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 2647251 Speed dialer: inteldialer://4,2647251 | Learn more Monday, February 04, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 2987236 Speed dialer: inteldialer://4,2987236 | Learn more Monday, February 18, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 2829772 Speed dialer: inteldialer://4,2829772 | Learn more Monday, March 04, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 2127060 Speed dialer: inteldialer://4,2127060 | Learn more Monday, March 18, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 2606814 Speed dialer: inteldialer://4,2606814 | Learn more Monday, April 01, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Bridge: 4, Passcode: 6176190 Speed dialer: inteldialer://4,6176190 | Learn more Monday, April 15, 2013, 09:00 AM US Pacific Time 916-356-2663, 8-356-2663, Brid</pre> Tziporet Koren 2013-05-27T08:17:57 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5881 <pre>Page 202 - get_alt_port_details() http://www.mellanox.com/related-docs/prod_software/RDMA_Aware_Programming_user_manual.pdf Thanks, Pavan _______________________________________________ ewg mailing list ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg</pre> pavan tc 2013-05-20T04:46:19 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5880 <pre> Thanks much for confirming. I was misled by the following comment in the mellanox programming guide (apm.c): /* This example assumes the alternate port we want to use is on the same Thanks, Pavan _______________________________________________ ewg mailing list ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5&lt; at &gt;public.gmane.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg</pre> pavan tc 2013-05-20T04:41:56 http://permalink.gmane.org/gmane.network.openfabrics.enterprise/5879 <pre>APM is supported only between two ports of the same HCA Tziporet </pre> Tziporet Koren 2013-05-19T14:35:06 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.network.openfabrics.enterprise