gmane.linux.nfs

Re: [PATCH 4/4] SUNRPC: Use gssproxy upcall for nfsd's RPCGSS authentication.

J. Bruce Fields — 2012-05-25T14:05:01

Whatever results in the easier-to-understand code at the end.


Since this would be a bug in our code (kernel or userspace), let's do
whatever makes it most obvious it's a bug.  At a minimum, a /* userspace
is buggy */ comment there.

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] NFSD: TEST_STATEID should not return NFS4ERR_STALE_STATEID

J. Bruce Fields — 2012-05-25T13:46:36

After the client's lease expires, the SEQUENCE operation will fail.

(Which I believe to be a valid, if unforgiving, server implementation.
If we were to implement "courtesy locks" in this case, I believe we'd
remember the clientid for longer, permit the SEQUENCE, and fail
individual stateid's with EXPIRED as appropriate?)


Thanks for the explanation!


Sure; nits:


Or inval?  This is just a buggy client.


Again, this is just a buggy client, since we shouldn't have gotten past
the SEQUENCE in this case unless the client's sending a stateid that's
actually someone else's.

If you think it's worth checking for those buggy client cases, we could
instaed check that stateid->si_opaque.so_clid and cl->clientid agree.
That'd cover the special-stateid checks too.


So this must be the case you actually hit.  Agreed with this change.


--b.

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

BUG: At key_instantiate_and_link+0x2e/0xa0

Boaz Harrosh — 2012-05-25T10:12:47

I got the below on a 3.4-rc7 based Benny's pnfs tree.

During normal nfs4 only operations. (Just my development trees mounted on client machine)

If it will happen again I will investigate. I'll leave it off for now (busy with other stuff)

Just wanted to know if it might have already been fix or seen by someone else

[ 3830.441139] BUG: unable to handle kernel NULL pointer dereference at 0000000000000078
[ 3830.442024] IP: [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024] PGD 0 
[ 3830.442024] Oops: 0000 [#1] SMP 
[ 3830.442024] CPU 1 
[ 3830.442024] Modules linked in: osd libosd nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT be2iscsi nf_conntrack_ipv6 nf_defrag_ipv6 iscsi_boot_sysfs xt_state bnx2i nf_conntrack cnic uio cxgb3i libcxgbi cxgb3 mdio ip6table_filter ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ip6_tables iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi joydev virtio_balloon i2c_piix4 i2c_core pcspkr virtio_net microcode floppy virtio_blk [last unloaded: libosd]
[ 3830.442024] 
[ 3830.442024] Pid: 1288, comm: rpc.idmapd Not tainted 3.4.0-rc7-pnfs+ #50 Bochs Bochs
[ 3830.442024] RIP: 0010:[<ffffffff812894ce>]  [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024] RSP: 0018:ffff88003a327d98  EFLAGS: 00010286
[ 3830.442024] RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff8800378579c0
[ 3830.442024] RDX: 0000000000000003 RSI: ffff88003a327e75 RDI: 0000000000000000
[ 3830.442024] RBP: ffff88003a327dd8 R08: ffff88003a26a9c0 R09: 000000000000fffd
[ 3830.442024] R10: 000000000000000a R11: 0000000000000246 R12: ffff8800378579c0
[ 3830.442024] R13: 0000000000000003 R14: ffff88003a327e75 R15: ffff88003a26a9c0
[ 3830.442024] FS:  00007f8c539d1720(0000) GS:ffff88003e500000(0000) knlGS:0000000000000000
[ 3830.442024] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3830.442024] CR2: 0000000000000078 CR3: 00000000379fe000 CR4: 00000000000006e0
[ 3830.442024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3830.442024] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3830.442024] Process rpc.idmapd (pid: 1288, threadinfo ffff88003a326000, task ffff88003a329710)
[ 3830.442024] Stack:
[ 3830.442024]  ffff88003a327de8 ffff88003a327da8 ffff88003a327dd8 ffffffffa01bfe8d
[ 3830.442024]  ffff8800399ce930 ffff880039ff1340 ffff88003a327dea 0000000000000000
[ 3830.442024]  ffff88003a327eb8 ffffffffa026360b 4073726573750101 2e736173616e6170
[ 3830.442024] Call Trace:
[ 3830.442024]  [<ffffffffa01bfe8d>] ? rpc_pipe_poll+0x6d/0xa0 [sunrpc]
[ 3830.442024]  [<ffffffffa026360b>] idmap_pipe_downcall+0x1cb/0x1e0 [nfs]
[ 3830.442024]  [<ffffffffa01bff27>] rpc_pipe_write+0x67/0x90 [sunrpc]
[ 3830.442024]  [<ffffffff8117e8e3>] vfs_write+0xb3/0x180
[ 3830.442024]  [<ffffffff8117ec0a>] sys_write+0x4a/0x90
[ 3830.442024]  [<ffffffff81077811>] ? posix_ktime_get_ts+0x11/0x20
[ 3830.442024]  [<ffffffff816215e9>] system_call_fastpath+0x16/0x1b
[ 3830.442024] Code: e5 48 83 ec 40 48 89 5d e0 4c 89 65 e8 4c 89 6d f0 4c 89 75 f8 66 66 66 66 90 48 85 c9 48 89 fb 49 89 f6 49 89 d5 49 89 cc 74 62 <48> 8b 57 78 48 8b 77 20 48 8d 4d d8 4c 89 e7 4c 89 45 c8 e8 0a 
[ 3830.442024] RIP  [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024]  RSP <ffff88003a327d98>
[ 3830.442024] CR2: 0000000000000078
[ 3830.492152] ---[ end trace a0dbf47ab1df84a5 ]---

Thanks
Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 3.4. sunrpc oops during shutdown

Stanislav Kinsbursky — 2012-05-25T08:12:34

Trond, looks like you are mistaken here.
Any process holds references to all namespaces it belong to (copy_net_ns() 
increase usage counter). And network namespace is released after mount namespace 
in free_nsproxy.


I've sent patches, which moves svc_shutdown_net() from svc_destroy() ("SUNRPC: 
separate per-net data creation from service").
with this patch set it's assumed, that per-net resources will be created or 
released prior to service creation and destruction.

3.0+ NFS issues

Michael Tokarev — 2012-05-25T06:53:11

Hello.

I updated my nfs server machine to kernel 3.0, and
noticed that its main usage become, well, problematic.

While trying to dig deeper, I also found a few other
interesting issues, which are mentioned below.

But first thing first: nfs.

i686pae kernel, lots of RAM, Atom-based (cedar trail)
machine with usual rtl8169 NIC.  3.0 or 3.2 kernel
(I will try current 3.4 but I don't have much hopes
there).  NFSv4.

When a client machine (also 3.0 kernel) does some reading,
the process often stalls somewhere in the read syscall,
or, rarer, during close, for up to two MINUTES.  During
this time, the client (kernel) reports "NFS server <foo>
does not respond" several times, and finally "NFS server
<foo> ok", client process "unstucks" from the read(2),
and is able to perform a few more reads till the whole
thing repeats.

While this happens, ie, while the client process is stuck,
any access to the mounted filesystem results in an
uninterruptible sleep - all processes including the
initial client are unkillable.  This is very annoying,
but I was never able to fix this, including the real
case when the server becomes unreachable - "intr" mount
option which is supposed to help here makes no difference.
This is first and longstanding issue, but it is not really
related to the issue at hand.

On the server side things are more interesting.  During
the client stall, all NFSD threads are busy running,
using all available CPU.  The machine has 2 cores, and
if I reduce number of NFSD threads to 1, one core will
be used by the single NFSD to 100%.  With default 8
NFSDs, both cores are used, by 4 NFSD threads each.
This is according to top(1), which show proportional
and equal CPU activity for each of NFSD thread.

But.  And this is where things becomes more interesting.
perf top shows almost no activity with any number of NFSD
threads but one.  For example, this is a typical perf top
display with 2 NFSD threads:

     samples  pcnt function                  DSO
     1558.00  6.4% svc_recv                  [sunrpc]
     1084.00  5.2% __schedule                [kernel]
     1065.00  5.1% select_task_rq_fair       [kernel]
      692.00  3.3% try_to_wake_up            [kernel]


Yes, there's some 6.4% time spent in svc_recv, but it is
FAR from the 200% CPU usage reported by top(1).  Now, when
I reduce number of NFSD threds to 1, perf top output becomes
dramatically different:

     samples  pcnt function                    DSO
     5996.00 59.2% svc_recv                    [sunrpc]
      750.00  7.4% cache_check                 [sunrpc]
      519.00  5.1% svc_tcp_has_wspace          [sunrpc]
      465.00  4.6% local_bh_enable_ip          [kernel.kallsyms]
      301.00  3.0% svc_defer                   [sunrpc]
      294.00  2.9% page_address                [kernel.kallsyms]

Ie, scv_recv time usage jumps from 6 to 60%, ie, 10 times
the difference.

So, apparently, `perf top' tool does not show the truth.

Trying to catch this high CPU usage with Sysrq+T does not
show anything interesting, the NFSD thread is running (and
is the only thread(s) shown in the "Runnable tasks" output):


May 25 10:48:37 gnome vmunix: [ 1709.483771] nfsd            R running      0  5853      2 0x00000000
May 25 10:48:37 gnome vmunix: [ 1709.483796]  f4477030 00000046 00000000 5c8e8929 0000018e f4c700f0 f4043a70 f545b140
May 25 10:48:37 gnome vmunix: [ 1709.483841]  050a8544 0000018e c1469140 c1469140 f4043a70 00000086 00000000 c1048575
May 25 10:48:37 gnome vmunix: [ 1709.483887]  c101ba11 c12fac18 f31bf800 f8f7bd30 0000003a c12fb156 0000003a 00000000
May 25 10:48:37 gnome vmunix: [ 1709.483931] Call Trace:
May 25 10:48:37 gnome vmunix: [ 1709.483950]  [<c1048575>] ? irq_exit+0x35/0x90
May 25 10:48:37 gnome vmunix: [ 1709.483973]  [<c101ba11>] ? smp_apic_timer_interrupt+0x51/0x90
May 25 10:48:37 gnome vmunix: [ 1709.483997]  [<c12fac18>] ? _raw_spin_lock_bh+0x8/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484029]  [<f8f7bd30>] ? svc_xprt_enqueue+0x40/0x1e0 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484053]  [<c12fb156>] ? apic_timer_interrupt+0x2a/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484084]  [<f8f7007b>] ? svc_destroy+0xdb/0x110 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484107]  [<c103d3e3>] ? __cond_resched+0x13/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484129]  [<c12f92b5>] ? _cond_resched+0x25/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484159]  [<f8f7cb5d>] ? svc_recv+0x1cd/0x7e0 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484182]  [<c101ba11>] ? smp_apic_timer_interrupt+0x51/0x90
May 25 10:48:37 gnome vmunix: [ 1709.484206]  [<c103a3d0>] ? try_to_wake_up+0x200/0x200
May 25 10:48:37 gnome vmunix: [ 1709.484233]  [<f90ea08c>] ? nfsd+0x8c/0x120 [nfsd]
May 25 10:48:37 gnome vmunix: [ 1709.484255]  [<c102d240>] ? complete+0x40/0x60
May 25 10:48:37 gnome vmunix: [ 1709.484279]  [<f90ea000>] ? 0xf90e9fff
May 25 10:48:37 gnome vmunix: [ 1709.484299]  [<c105ea99>] ? kthread+0x69/0x70
May 25 10:48:37 gnome vmunix: [ 1709.484321]  [<c105ea30>] ? kthread_worker_fn+0x150/0x150
May 25 10:48:37 gnome vmunix: [ 1709.484345]  [<c1301876>] ? kernel_thread_helper+0x6/0xd

May 25 10:49:37 gnome vmunix: [ 1769.982791] nfsd            R running      0  5853      2 0x00000000
May 25 10:49:37 gnome vmunix: [ 1769.982816]  f5449f80 00000000 00000000 7636b581 0000019c 00000003 7603dcb8 0000019c
May 25 10:49:37 gnome vmunix: [ 1769.982859]  7603dcb8 0000019c 00000046 00000000 f4043a70 00000086 00000000 c1048575
May 25 10:49:37 gnome vmunix: [ 1769.982903]  f31bf800 c12fac18 f31bf800 f8f7bd30 00000047 f31bf800 f418c2c0 f8f7c03c
May 25 10:49:37 gnome vmunix: [ 1769.982948] Call Trace:
May 25 10:49:37 gnome vmunix: [ 1769.982966]  [<c1048575>] ? irq_exit+0x35/0x90
May 25 10:49:37 gnome vmunix: [ 1769.982987]  [<c12fac18>] ? _raw_spin_lock_bh+0x8/0x30
May 25 10:49:37 gnome vmunix: [ 1769.983019]  [<f8f7bd30>] ? svc_xprt_enqueue+0x40/0x1e0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983050]  [<f8f7c03c>] ? svc_xprt_release+0x1c/0xd0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983081]  [<f8f7c0ad>] ? svc_xprt_release+0x8d/0xd0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983112]  [<f8f7cc03>] ? svc_recv+0x273/0x7e0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983143]  [<f8f7c300>] ? svc_age_temp_xprts+0x110/0x190 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983166]  [<c103a3d0>] ? try_to_wake_up+0x200/0x200
May 25 10:49:37 gnome vmunix: [ 1769.983193]  [<f90ea08c>] ? nfsd+0x8c/0x120 [nfsd]
May 25 10:49:37 gnome vmunix: [ 1769.983215]  [<c102d240>] ? complete+0x40/0x60
May 25 10:49:37 gnome vmunix: [ 1769.983238]  [<f90ea000>] ? 0xf90e9fff
May 25 10:49:37 gnome vmunix: [ 1769.983258]  [<c105ea99>] ? kthread+0x69/0x70
May 25 10:49:37 gnome vmunix: [ 1769.983278]  [<c105ea30>] ? kthread_worker_fn+0x150/0x150
May 25 10:49:37 gnome vmunix: [ 1769.983301]  [<c1301876>] ? kernel_thread_helper+0x6/0xd

This is 3.2, on 3.0 it was in different place.

This problem is persistent, always reproducible, happens
on reads from the server (I haven't seen it with writes),
and completely goes away when reverting to the kernel I
previously used on this machine - 2.6.32.  But unfortunately
2.6.32 does not quite work on the new hardware (I replaced
the motherboard), due to lack of drivers for several pieces.
(This is a different story, I think NFSD should work in 3.0
regardless of 2.6.32 issues :)

Anything I can do to make NFS work again?

Thanks,

/mjt
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] nfs41: Use BIND_CONN_TO_SESSION for CB_PATH_DOWN*

Weston Andros Adamson — 2012-05-24T21:19:34

The state manager can handle SEQ4_STATUS_CB_PATH_DOWN* flags with a
BIND_CONN_TO_SESSION instead of destroying the session and creating a new one.

Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Updated with comments from Trond.

I got to remember to test with different CONFIG options *after* I clean up the
patch :)

 fs/nfs/nfs4_fs.h   |    1 +
 fs/nfs/nfs4state.c |   31 +++++++++++++++++++++++++++----
 2 files changed, 28 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index ad90bc7..9feff0f 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -24,6 +24,7 < at >< at > enum nfs4_client_state {
 NFS4CLNT_RECALL_SLOT,
 NFS4CLNT_LEASE_CONFIRM,
 NFS4CLNT_SERVER_SCOPE_MISMATCH,
+NFS4CLNT_BIND_CONN_TO_SESSION,
 };
 
 enum nfs4_session_state {
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 7f0fcfc..4f41144 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
< at >< at > -1639,13 +1639,20 < at >< at > static void nfs41_handle_recallable_state_revoked(struct nfs_client *clp)
 nfs_expire_all_delegations(clp);
 }
 
-static void nfs41_handle_cb_path_down(struct nfs_client *clp)
+static void nfs41_handle_backchannel_fault(struct nfs_client *clp)
 {
 nfs_expire_all_delegations(clp);
 if (test_and_set_bit(NFS4CLNT_SESSION_RESET, &clp->cl_state) == 0)
 nfs4_schedule_state_manager(clp);
 }
 
+static void nfs41_handle_cb_path_down(struct nfs_client *clp)
+{
+if (test_and_set_bit(NFS4CLNT_BIND_CONN_TO_SESSION,
+&clp->cl_state) == 0)
+nfs4_schedule_state_manager(clp);
+}
+
 void nfs41_handle_sequence_flag_errors(struct nfs_client *clp, u32 flags)
 {
 if (!flags)
< at >< at > -1659,9 +1666,10 < at >< at > void nfs41_handle_sequence_flag_errors(struct nfs_client *clp, u32 flags)
 nfs41_handle_state_revoked(clp);
 if (flags & SEQ4_STATUS_RECALLABLE_STATE_REVOKED)
 nfs41_handle_recallable_state_revoked(clp);
-if (flags & (SEQ4_STATUS_CB_PATH_DOWN |
-    SEQ4_STATUS_BACKCHANNEL_FAULT |
-    SEQ4_STATUS_CB_PATH_DOWN_SESSION))
+if (flags & SEQ4_STATUS_BACKCHANNEL_FAULT)
+nfs41_handle_backchannel_fault(clp);
+else if (flags & (SEQ4_STATUS_CB_PATH_DOWN |
+SEQ4_STATUS_CB_PATH_DOWN_SESSION))
 nfs41_handle_cb_path_down(clp);
 }
 
< at >< at > -1686,6 +1694,7 < at >< at > static int nfs4_reset_session(struct nfs_client *clp)
 clear_bit(NFS4CLNT_SESSION_RESET, &clp->cl_state);
 /* create_session negotiated new slot table */
 clear_bit(NFS4CLNT_RECALL_SLOT, &clp->cl_state);
+clear_bit(NFS4CLNT_BIND_CONN_TO_SESSION, &clp->cl_state);
 
  /* Let the state manager reestablish state */
 if (!test_bit(NFS4CLNT_LEASE_EXPIRED, &clp->cl_state))
< at >< at > -1722,10 +1731,15 < at >< at > static int nfs4_recall_slot(struct nfs_client *clp)
 return 0;
 }
 
+static int nfs4_do_bind_conn_to_session(struct nfs_client *clp)
+{
+return nfs4_proc_bind_conn_to_session(clp);
+}
 #else /* CONFIG_NFS_V4_1 */
 static int nfs4_reset_session(struct nfs_client *clp) { return 0; }
 static int nfs4_end_drain_session(struct nfs_client *clp) { return 0; }
 static int nfs4_recall_slot(struct nfs_client *clp) { return 0; }
+static int nfs4_do_bind_conn_to_session(struct nfs_client *clp) { return 0; }
 #endif /* CONFIG_NFS_V4_1 */
 
 /* Set NFS4CLNT_LEASE_EXPIRED for all v4.0 errors and for recoverable errors
< at >< at > -1803,6 +1817,15 < at >< at > static void nfs4_state_manager(struct nfs_client *clp)
 goto out_error;
 }
 
+/* Send BIND_CONN_TO_SESSION */
+if (clp->cl_mvops->minor_version == 1 &&
+    test_and_clear_bit(NFS4CLNT_BIND_CONN_TO_SESSION,
+&clp->cl_state) && nfs4_has_session(clp)) {
+status = nfs4_do_bind_conn_to_session(clp);
+if (status < 0)
+goto out_error;
+}
+
 /* First recover reboot state... */
 if (test_bit(NFS4CLNT_RECLAIM_REBOOT, &clp->cl_state)) {
 status = nfs4_do_reclaim(clp,

[PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation

Weston Andros Adamson — 2012-05-24T21:19:33

This patch adds the BIND_CONN_TO_SESSION operation which is needed for
upcoming SP4_MACH_CRED work and useful for recovering from broken connections
without destroying the session.

Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Updated with comment from Trond

 fs/nfs/nfs4_fs.h        |    1 +
 fs/nfs/nfs4proc.c       |   54 ++++++++++++++++++++++++++
 fs/nfs/nfs4xdr.c        |   95 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfs4.h    |    5 ++
 include/linux/nfs_xdr.h |    6 +++
 5 files changed, 161 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index edeef71..ad90bc7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -215,6 +215,7 < at >< at > struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
 extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
 extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
 extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
+extern int nfs4_proc_bind_conn_to_session(struct nfs_client *);
 extern int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred);
 extern int nfs4_init_clientid(struct nfs_client *, struct rpc_cred *);
 extern int nfs41_init_clientid(struct nfs_client *, struct rpc_cred *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78784e5..1d1bef6 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5062,6 +5062,60 < at >< at > nfs41_same_server_scope(struct server_scope *a, struct server_scope *b)
 }
 
 /*
+ * nfs4_proc_bind_conn_to_session()
+ *
+ * The 4.1 client currently uses the same TCP connection for the
+ * fore and backchannel.
+ */
+int nfs4_proc_bind_conn_to_session(struct nfs_client *clp)
+{
+int status;
+struct nfs41_bind_conn_to_session_res res;
+struct rpc_message msg = {
+.rpc_proc =
+&nfs4_procedures[NFSPROC4_CLNT_BIND_CONN_TO_SESSION],
+.rpc_argp = clp,
+.rpc_resp = &res,
+};
+
+dprintk("--> %s\n", __func__);
+BUG_ON(clp == NULL);
+
+res.session = kzalloc(sizeof(struct nfs4_session), GFP_NOFS);
+if (unlikely(res.session == NULL)) {
+status = -ENOMEM;
+goto out;
+}
+
+status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+if (status == 0) {
+if (memcmp(res.session->sess_id.data,
+    clp->cl_session->sess_id.data, NFS4_MAX_SESSIONID_LEN)) {
+dprintk("NFS: %s: Session ID mismatch\n", __func__);
+status = -EIO;
+goto out_session;
+}
+if (res.dir != NFS4_CDFS4_BOTH) {
+dprintk("NFS: %s: Unexpected direction from server\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+if (res.use_conn_in_rdma_mode) {
+dprintk("NFS: %s: Server returned RDMA mode = true\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+}
+out_session:
+kfree(res.session);
+out:
+dprintk("<-- %s status= %d\n", __func__, status);
+return status;
+}
+
+/*
  * nfs4_proc_exchange_id()
  *
  * Since the clientid has expired, all compounds using sessions
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..e9be74f 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -321,6 +321,16 < at >< at > static int nfs4_stat_to_errno(int);
      1 /* csr_flags */ + \
      decode_channel_attrs_maxsz + \
      decode_channel_attrs_maxsz)
+#define encode_bind_conn_to_session_maxsz  (op_encode_hdr_maxsz + \
+     /* bctsa_sessid */ \
+     XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+     1 /* bctsa_dir */ + \
+     1 /* bctsa_use_conn_in_rdma_mode */)
+#define decode_bind_conn_to_session_maxsz  (op_decode_hdr_maxsz +\
+     /* bctsr_sessid */ \
+     XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+     1 /* bctsr_dir */ + \
+     1 /* bctsr_use_conn_in_rdma_mode */)
 #define encode_destroy_session_maxsz    (op_encode_hdr_maxsz + 4)
 #define decode_destroy_session_maxsz    (op_decode_hdr_maxsz)
 #define encode_sequence_maxsz(op_encode_hdr_maxsz + \
< at >< at > -714,6 +724,12 < at >< at > static int nfs4_stat_to_errno(int);
 decode_putfh_maxsz + \
 decode_secinfo_maxsz)
 #if defined(CONFIG_NFS_V4_1)
+#define NFS4_enc_bind_conn_to_session_sz \
+(compound_encode_hdr_maxsz + \
+ encode_bind_conn_to_session_maxsz)
+#define NFS4_dec_bind_conn_to_session_sz \
+(compound_decode_hdr_maxsz + \
+ decode_bind_conn_to_session_maxsz)
 #define NFS4_enc_exchange_id_sz \
 (compound_encode_hdr_maxsz + \
  encode_exchange_id_maxsz)
< at >< at > -1654,6 +1670,20 < at >< at > static void encode_secinfo(struct xdr_stream *xdr, const struct qstr *name, stru
 
 #if defined(CONFIG_NFS_V4_1)
 /* NFSv4.1 operations */
+static void encode_bind_conn_to_session(struct xdr_stream *xdr,
+   struct nfs4_session *session,
+   struct compound_hdr *hdr)
+{
+__be32 *p;
+
+encode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION,
+decode_bind_conn_to_session_maxsz, hdr);
+encode_opaque_fixed(xdr, session->sess_id.data, NFS4_MAX_SESSIONID_LEN);
+p = xdr_reserve_space(xdr, 8);
+*p++ = cpu_to_be32(NFS4_CDFC4_BACK_OR_BOTH);
+*p = 0;/* use_conn_in_rdma_mode = False */
+}
+
 static void encode_exchange_id(struct xdr_stream *xdr,
        struct nfs41_exchange_id_args *args,
        struct compound_hdr *hdr)
< at >< at > -2614,6 +2644,22 < at >< at > static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
 
 #if defined(CONFIG_NFS_V4_1)
 /*
+ * BIND_CONN_TO_SESSION request
+ */
+static void nfs4_xdr_enc_bind_conn_to_session(struct rpc_rqst *req,
+struct xdr_stream *xdr,
+struct nfs_client *clp)
+{
+struct compound_hdr hdr = {
+.minorversion = clp->cl_mvops->minor_version,
+};
+
+encode_compound_hdr(xdr, req, &hdr);
+encode_bind_conn_to_session(xdr, clp->cl_session, &hdr);
+encode_nops(&hdr);
+}
+
+/*
  * EXCHANGE_ID request
  */
 static void nfs4_xdr_enc_exchange_id(struct rpc_rqst *req,
< at >< at > -5239,6 +5285,37 < at >< at > static int decode_sessionid(struct xdr_stream *xdr, struct nfs4_sessionid *sid)
 return decode_opaque_fixed(xdr, sid->data, NFS4_MAX_SESSIONID_LEN);
 }
 
+static int decode_bind_conn_to_session(struct xdr_stream *xdr,
+struct nfs41_bind_conn_to_session_res *res)
+{
+__be32 *p;
+int status;
+
+status = decode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION);
+if (!status)
+status = decode_sessionid(xdr, &res->session->sess_id);
+if (unlikely(status))
+return status;
+
+/* dir flags, rdma mode bool */
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+
+res->dir = be32_to_cpup(p++);
+if (res->dir == 0 || res->dir > NFS4_CDFS4_BOTH)
+return -EIO;
+if (be32_to_cpup(p) == 0)
+res->use_conn_in_rdma_mode = false;
+else
+res->use_conn_in_rdma_mode = true;
+
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
 static int decode_create_session(struct xdr_stream *xdr,
  struct nfs41_create_session_res *res)
 {
< at >< at > -6521,6 +6598,22 < at >< at > out:
 
 #if defined(CONFIG_NFS_V4_1)
 /*
+ * Decode BIND_CONN_TO_SESSION response
+ */
+static int nfs4_xdr_dec_bind_conn_to_session(struct rpc_rqst *rqstp,
+struct xdr_stream *xdr,
+void *res)
+{
+struct compound_hdr hdr;
+int status;
+
+status = decode_compound_hdr(xdr, &hdr);
+if (!status)
+status = decode_bind_conn_to_session(xdr, res);
+return status;
+}
+
+/*
  * Decode EXCHANGE_ID response
  */
 static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp,
< at >< at > -7001,6 +7094,8 < at >< at > struct rpc_procinfonfs4_procedures[] = {
 PROC(RELEASE_LOCKOWNER,enc_release_lockowner,dec_release_lockowner),
 PROC(SECINFO,enc_secinfo,dec_secinfo),
 #if defined(CONFIG_NFS_V4_1)
+PROC(BIND_CONN_TO_SESSION,
+enc_bind_conn_to_session, dec_bind_conn_to_session),
 PROC(EXCHANGE_ID,enc_exchange_id,dec_exchange_id),
 PROC(CREATE_SESSION,enc_create_session,dec_create_session),
 PROC(DESTROY_SESSION,enc_destroy_session,dec_destroy_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..5b1a504 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -69,6 +69,10 < at >< at >
 #define NFS4_CDFC4_FORE_OR_BOTH 0x3
 #define NFS4_CDFC4_BACK_OR_BOTH 0x7
 
+#define NFS4_CDFS4_FORE 0x1
+#define NFS4_CDFS4_BACK 0x2
+#define NFS4_CDFS4_BOTH 0x3
+
 #define NFS4_SET_TO_SERVER_TIME0
 #define NFS4_SET_TO_CLIENT_TIME1
 
< at >< at > -582,6 +586,7 < at >< at > enum {
 NFSPROC4_CLNT_SECINFO,
 
 /* nfs41 */
+NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
 NFSPROC4_CLNT_EXCHANGE_ID,
 NFSPROC4_CLNT_CREATE_SESSION,
 NFSPROC4_CLNT_DESTROY_SESSION,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..aa2a3bb 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -1115,6 +1115,12 < at >< at > struct nfs41_impl_id {
 struct nfstime4date;
 };
 
+struct nfs41_bind_conn_to_session_res {
+struct nfs4_session*session;
+u32dir;
+booluse_conn_in_rdma_mode;
+};
+
 struct nfs41_exchange_id_res {
 struct nfs_client*client;
 u32flags;

Re: [PATCH 2/2] nfs41: Use BIND_CONN_TO_SESSION for CB_PATH_DOWN*

Adamson, Dros — 2012-05-24T21:07:14

On May 24, 2012, at 5:00 PM, Adamson, Dros wrote:


Oops, actually that was an older patch with a wrapper around nfs4_proc_bind_conn_to_session - I'll add it back.

-dros

Re: [PATCH 2/2] nfs41: Use BIND_CONN_TO_SESSION for CB_PATH_DOWN*

Adamson, Dros — 2012-05-24T21:00:40

On May 24, 2012, at 4:39 PM, Myklebust, Trond wrote:


Ah, I actually did test with CONFIG_NFS_V4_1 undefined - it compiles, but you're right, this should be #ifdef'd


Good catch.

Thanks!
-dros

Re: [PATCH 2/2] nfs41: Use BIND_CONN_TO_SESSION for CB_PATH_DOWN*

Myklebust, Trond — 2012-05-24T20:39:06

On Thu, 2012-05-24 at 12:26 -0400, Weston Andros Adamson wrote:

Committed after fixing ifndef CONFIG_NFS_V4_1 issues and adding a
clear_bit() for NFS4CLNT_BIND_CONN_TO_SESSION in nfs4_reset_session.

Re: [PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation

Myklebust, Trond — 2012-05-24T20:35:55

On Thu, 2012-05-24 at 13:22 -0400, Weston Andros Adamson wrote:

Committed after changing the allocation mode to GFP_NOFS. GFP_KERNEL
allocations may end up deadlocking in a recovery situation if they
result in an attempt to write back dirty NFS pages.

Re: 3.4. sunrpc oops during shutdown

2012-05-24T20:27:20

Thanks, makes sense.

Uh, I meant to cc: Stanislav on that last reply but didn't somehow.

--b.

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] nfsd: probe the back channel on new connections

Weston Andros Adamson — 2012-05-24T19:42:17

Initiate a CB probe when a new connection with the correct direction is added
to a session (IFF backchannel is marked as down).  Without this a
BIND_CONN_TO_SESSION has no effect on the internal backchannel state, which
causes the server to reply to every SEQUENCE op with the
SEQ4_STATUS_CB_PATH_DOWN flag set until DESTROY_SESSION.

Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Hey Bruce -
 In an earlier thread I mentioned that I was setting the backchannel state to
CB_UNKNOWN to clear the SEQ4_STATUS flag, but this was wrong - I don't believe
it would ever change (to CB_UP/CB_DOWN).  This patch seems like the right way
to handle the internal backchannel state.

Also, I'm a bit confused as how we're supposed to maintain atomicity around
cl_cb_state:

from nfs4callback.c:nfsd4_probe_callback():
  /* XXX: atomicity?  Also, should we be using cl_flags? */
  clp->cl_cb_state = NFSD4_CB_UNKNOWN;

and there are several places that check the value of cl_cb_state without holding
a lock (as far as I can tell).

 fs/nfsd/nfs4state.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9afc902..99f092e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
< at >< at > -862,6 +862,11 < at >< at > static __be32 nfsd4_new_conn(struct svc_rqst *rqstp, struct nfsd4_session *ses,
 if (ret)
 /* oops; xprt is already down: */
 nfsd4_conn_lost(&conn->cn_xpt_user);
+if (ses->se_client->cl_cb_state == NFSD4_CB_DOWN &&
+dir & NFS4_CDFC4_BACK) {
+/* callback channel may be back up */
+nfsd4_probe_callback(ses->se_client);
+}
 return nfs_ok;
 }

[PATCH 1/2] nfsd: add fault injection for CB_PATH_DOWN

Weston Andros Adamson — 2012-05-24T19:42:16

This fault injection hook causes all SEQUENCE operation replies to have the
SEQ4_STATUS_CB_PATH_DOWN flag set until the client calls BIND_CONN_TO_SESSION
or creates a new session.

Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
 fs/nfsd/fault_inject.c |    4 ++++
 fs/nfsd/fault_inject.h |    2 ++
 fs/nfsd/nfs4state.c    |   20 ++++++++++++++++++++
 3 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/fs/nfsd/fault_inject.c b/fs/nfsd/fault_inject.c
index e6c3815..ab81144 100644
--- a/fs/nfsd/fault_inject.c
+++ b/fs/nfsd/fault_inject.c
< at >< at > -38,6 +38,10 < at >< at > static struct nfsd_fault_inject_op inject_ops[] = {
 .file   = "recall_delegations",
 .func   = nfsd_recall_delegations,
 },
+{
+.file   = "kill_backchannel",
+.func   = nfsd_kill_backchannel,
+},
 };
 
 static long int NUM_INJECT_OPS = sizeof(inject_ops) / sizeof(struct nfsd_fault_inject_op);
diff --git a/fs/nfsd/fault_inject.h b/fs/nfsd/fault_inject.h
index 90bd057..b058e03 100644
--- a/fs/nfsd/fault_inject.h
+++ b/fs/nfsd/fault_inject.h
< at >< at > -15,6 +15,7 < at >< at > void nfsd_forget_locks(u64);
 void nfsd_forget_openowners(u64);
 void nfsd_forget_delegations(u64);
 void nfsd_recall_delegations(u64);
+void nfsd_kill_backchannel(u64);
 #else /* CONFIG_NFSD_FAULT_INJECTION */
 static inline int nfsd_fault_inject_init(void) { return 0; }
 static inline void nfsd_fault_inject_cleanup(void) {}
< at >< at > -23,6 +24,7 < at >< at > static inline void nfsd_forget_locks(u64 num) {}
 static inline void nfsd_forget_openowners(u64 num) {}
 static inline void nfsd_forget_delegations(u64 num) {}
 static inline void nfsd_recall_delegations(u64 num) {}
+static inline void nfsd_kill_backchannel(u64 num) {}
 #endif /* CONFIG_NFSD_FAULT_INJECTION */
 
 #endif /* LINUX_NFSD_FAULT_INJECT_H */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 21266c7..9afc902 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
< at >< at > -4737,6 +4737,26 < at >< at > void nfsd_recall_delegations(u64 num)
 printk(KERN_INFO "NFSD: Recalled %d delegations", count);
 }
 
+/*
+ * Force SEQUENCE operations to have SEQ4_STATUS_CB_PATH_DOWN flag set
+ * until backchannel is reestablished by BIND_CONN_TO_SESSION or
+ * DESTROY_SESSION/CREATE_SESSION with SP4_NONE.
+ *
+ * The argument 'num' is ignored, any value will trigger the fault on
+ * all clients.
+ */
+void nfsd_kill_backchannel(u64 num)
+{
+struct nfs4_client *clp, *next;
+
+nfs4_lock_state();
+list_for_each_entry_safe(clp, next, &client_lru, cl_lru) {
+clp->cl_cb_state = NFSD4_CB_DOWN;
+}
+nfs4_unlock_state();
+
+printk(KERN_INFO "NFSD: killed backchannel");
+}
 #endif /* CONFIG_NFSD_FAULT_INJECTION */
 
 /* initialization to perform at module load time: */

[PATCH] NFSD: TEST_STATEID should not return NFS4ERR_STALE_STATEID

Chuck Lever — 2012-05-24T19:29:51

The error values that TEST_STATEID is allowed to return does not
include NFS4ERR_STALE_STATEID.  In addition, RFC 5661 says:

15.1.16.5.  NFS4ERR_STALE_STATEID (Error Code 10023)

   A stateid generated by an earlier server instance was used.  This
   error is moot in NFSv4.1 because all operations that take a stateid
   MUST be preceded by the SEQUENCE operation, and the earlier server
   instance is detected by the session infrastructure that supports
   SEQUENCE.

I triggered the NFS4ERR_STALE_STATEID during nograce recovery testing.
My client had updated its boot verifier, so the server instance hadn't
changed, but the client instance had.  Thus the server allowed the
SEQUENCE operation, but returned NFS4ERR_STALE_STATEID on the
TEST_STATEID operation.

After a client's lease expires, TEST_STATEID should report
NFS4ERR_EXPIRED for state IDs that the client tries to recover.  I
don't see a way to make that happen, though.

Finally, RFC 5661, section 18.48.3 has this:

 o  Special stateids are always considered invalid (they result in the
    error code NFS4ERR_BAD_STATEID).

Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA< at >public.gmane.org>
---

Bruce, would you consider taking something like this?

 fs/nfsd/nfs4state.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9235cfa..ae1fab3 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
< at >< at > -3365,12 +3365,13 < at >< at > __be32 nfs4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
 struct nfs4_ol_stateid *ols;
 __be32 status;
 
+if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
+return nfserr_bad_stateid;
 if (STALE_STATEID(stateid))
-return nfserr_stale_stateid;
-
+return nfserr_bad_stateid;
 s = find_stateid(cl, stateid);
 if (!s)
- return nfserr_stale_stateid;
+ return nfserr_bad_stateid;
 status = check_stateid_generation(stateid, &s->sc_stateid, 1);
 if (status)
 return status;

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 3.4. sunrpc oops during shutdown

Myklebust, Trond — 2012-05-24T19:20:41

On Thu, 2012-05-24 at 11:55 -0400, bfields< at >fieldses.org wrote:

For NFSv4, I need to create sockets for the same net namespace as the
struct nfs_client is running in. When all the struct nfs_clients on that
net namespace are destroyed, I would ideally get rid of those sockets.

For NFSv4.1, all I want to do is create a back channel using the same
socket as the struct nfs_client.

[PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation

Weston Andros Adamson — 2012-05-24T17:22:50

This patch adds the BIND_CONN_TO_SESSION operation which is needed for
upcoming SP4_MACH_CRED work and useful for recovering from broken connections
without destroying the session.

Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Reposted with changes addressing Chuck's comments.

 fs/nfs/nfs4_fs.h        |    1 +
 fs/nfs/nfs4proc.c       |   54 ++++++++++++++++++++++++++
 fs/nfs/nfs4xdr.c        |   95 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfs4.h    |    5 ++
 include/linux/nfs_xdr.h |    6 +++
 5 files changed, 161 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index edeef71..ad90bc7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -215,6 +215,7 < at >< at > struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
 extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
 extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
 extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
+extern int nfs4_proc_bind_conn_to_session(struct nfs_client *);
 extern int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred);
 extern int nfs4_init_clientid(struct nfs_client *, struct rpc_cred *);
 extern int nfs41_init_clientid(struct nfs_client *, struct rpc_cred *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78784e5..95aca62 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5062,6 +5062,60 < at >< at > nfs41_same_server_scope(struct server_scope *a, struct server_scope *b)
 }
 
 /*
+ * nfs4_proc_bind_conn_to_session()
+ *
+ * The 4.1 client currently uses the same TCP connection for the
+ * fore and backchannel.
+ */
+int nfs4_proc_bind_conn_to_session(struct nfs_client *clp)
+{
+int status;
+struct nfs41_bind_conn_to_session_res res;
+struct rpc_message msg = {
+.rpc_proc =
+&nfs4_procedures[NFSPROC4_CLNT_BIND_CONN_TO_SESSION],
+.rpc_argp = clp,
+.rpc_resp = &res,
+};
+
+dprintk("--> %s\n", __func__);
+BUG_ON(clp == NULL);
+
+res.session = kzalloc(sizeof(struct nfs4_session), GFP_KERNEL);
+if (unlikely(res.session == NULL)) {
+status = -ENOMEM;
+goto out;
+}
+
+status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+if (status == 0) {
+if (memcmp(res.session->sess_id.data,
+    clp->cl_session->sess_id.data, NFS4_MAX_SESSIONID_LEN)) {
+dprintk("NFS: %s: Session ID mismatch\n", __func__);
+status = -EIO;
+goto out_session;
+}
+if (res.dir != NFS4_CDFS4_BOTH) {
+dprintk("NFS: %s: Unexpected direction from server\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+if (res.use_conn_in_rdma_mode) {
+dprintk("NFS: %s: Server returned RDMA mode = true\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+}
+out_session:
+kfree(res.session);
+out:
+dprintk("<-- %s status= %d\n", __func__, status);
+return status;
+}
+
+/*
  * nfs4_proc_exchange_id()
  *
  * Since the clientid has expired, all compounds using sessions
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..e9be74f 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -321,6 +321,16 < at >< at > static int nfs4_stat_to_errno(int);
      1 /* csr_flags */ + \
      decode_channel_attrs_maxsz + \
      decode_channel_attrs_maxsz)
+#define encode_bind_conn_to_session_maxsz  (op_encode_hdr_maxsz + \
+     /* bctsa_sessid */ \
+     XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+     1 /* bctsa_dir */ + \
+     1 /* bctsa_use_conn_in_rdma_mode */)
+#define decode_bind_conn_to_session_maxsz  (op_decode_hdr_maxsz +\
+     /* bctsr_sessid */ \
+     XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+     1 /* bctsr_dir */ + \
+     1 /* bctsr_use_conn_in_rdma_mode */)
 #define encode_destroy_session_maxsz    (op_encode_hdr_maxsz + 4)
 #define decode_destroy_session_maxsz    (op_decode_hdr_maxsz)
 #define encode_sequence_maxsz(op_encode_hdr_maxsz + \
< at >< at > -714,6 +724,12 < at >< at > static int nfs4_stat_to_errno(int);
 decode_putfh_maxsz + \
 decode_secinfo_maxsz)
 #if defined(CONFIG_NFS_V4_1)
+#define NFS4_enc_bind_conn_to_session_sz \
+(compound_encode_hdr_maxsz + \
+ encode_bind_conn_to_session_maxsz)
+#define NFS4_dec_bind_conn_to_session_sz \
+(compound_decode_hdr_maxsz + \
+ decode_bind_conn_to_session_maxsz)
 #define NFS4_enc_exchange_id_sz \
 (compound_encode_hdr_maxsz + \
  encode_exchange_id_maxsz)
< at >< at > -1654,6 +1670,20 < at >< at > static void encode_secinfo(struct xdr_stream *xdr, const struct qstr *name, stru
 
 #if defined(CONFIG_NFS_V4_1)
 /* NFSv4.1 operations */
+static void encode_bind_conn_to_session(struct xdr_stream *xdr,
+   struct nfs4_session *session,
+   struct compound_hdr *hdr)
+{
+__be32 *p;
+
+encode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION,
+decode_bind_conn_to_session_maxsz, hdr);
+encode_opaque_fixed(xdr, session->sess_id.data, NFS4_MAX_SESSIONID_LEN);
+p = xdr_reserve_space(xdr, 8);
+*p++ = cpu_to_be32(NFS4_CDFC4_BACK_OR_BOTH);
+*p = 0;/* use_conn_in_rdma_mode = False */
+}
+
 static void encode_exchange_id(struct xdr_stream *xdr,
        struct nfs41_exchange_id_args *args,
        struct compound_hdr *hdr)
< at >< at > -2614,6 +2644,22 < at >< at > static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
 
 #if defined(CONFIG_NFS_V4_1)
 /*
+ * BIND_CONN_TO_SESSION request
+ */
+static void nfs4_xdr_enc_bind_conn_to_session(struct rpc_rqst *req,
+struct xdr_stream *xdr,
+struct nfs_client *clp)
+{
+struct compound_hdr hdr = {
+.minorversion = clp->cl_mvops->minor_version,
+};
+
+encode_compound_hdr(xdr, req, &hdr);
+encode_bind_conn_to_session(xdr, clp->cl_session, &hdr);
+encode_nops(&hdr);
+}
+
+/*
  * EXCHANGE_ID request
  */
 static void nfs4_xdr_enc_exchange_id(struct rpc_rqst *req,
< at >< at > -5239,6 +5285,37 < at >< at > static int decode_sessionid(struct xdr_stream *xdr, struct nfs4_sessionid *sid)
 return decode_opaque_fixed(xdr, sid->data, NFS4_MAX_SESSIONID_LEN);
 }
 
+static int decode_bind_conn_to_session(struct xdr_stream *xdr,
+struct nfs41_bind_conn_to_session_res *res)
+{
+__be32 *p;
+int status;
+
+status = decode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION);
+if (!status)
+status = decode_sessionid(xdr, &res->session->sess_id);
+if (unlikely(status))
+return status;
+
+/* dir flags, rdma mode bool */
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+
+res->dir = be32_to_cpup(p++);
+if (res->dir == 0 || res->dir > NFS4_CDFS4_BOTH)
+return -EIO;
+if (be32_to_cpup(p) == 0)
+res->use_conn_in_rdma_mode = false;
+else
+res->use_conn_in_rdma_mode = true;
+
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
 static int decode_create_session(struct xdr_stream *xdr,
  struct nfs41_create_session_res *res)
 {
< at >< at > -6521,6 +6598,22 < at >< at > out:
 
 #if defined(CONFIG_NFS_V4_1)
 /*
+ * Decode BIND_CONN_TO_SESSION response
+ */
+static int nfs4_xdr_dec_bind_conn_to_session(struct rpc_rqst *rqstp,
+struct xdr_stream *xdr,
+void *res)
+{
+struct compound_hdr hdr;
+int status;
+
+status = decode_compound_hdr(xdr, &hdr);
+if (!status)
+status = decode_bind_conn_to_session(xdr, res);
+return status;
+}
+
+/*
  * Decode EXCHANGE_ID response
  */
 static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp,
< at >< at > -7001,6 +7094,8 < at >< at > struct rpc_procinfonfs4_procedures[] = {
 PROC(RELEASE_LOCKOWNER,enc_release_lockowner,dec_release_lockowner),
 PROC(SECINFO,enc_secinfo,dec_secinfo),
 #if defined(CONFIG_NFS_V4_1)
+PROC(BIND_CONN_TO_SESSION,
+enc_bind_conn_to_session, dec_bind_conn_to_session),
 PROC(EXCHANGE_ID,enc_exchange_id,dec_exchange_id),
 PROC(CREATE_SESSION,enc_create_session,dec_create_session),
 PROC(DESTROY_SESSION,enc_destroy_session,dec_destroy_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..5b1a504 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -69,6 +69,10 < at >< at >
 #define NFS4_CDFC4_FORE_OR_BOTH 0x3
 #define NFS4_CDFC4_BACK_OR_BOTH 0x7
 
+#define NFS4_CDFS4_FORE 0x1
+#define NFS4_CDFS4_BACK 0x2
+#define NFS4_CDFS4_BOTH 0x3
+
 #define NFS4_SET_TO_SERVER_TIME0
 #define NFS4_SET_TO_CLIENT_TIME1
 
< at >< at > -582,6 +586,7 < at >< at > enum {
 NFSPROC4_CLNT_SECINFO,
 
 /* nfs41 */
+NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
 NFSPROC4_CLNT_EXCHANGE_ID,
 NFSPROC4_CLNT_CREATE_SESSION,
 NFSPROC4_CLNT_DESTROY_SESSION,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..aa2a3bb 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -1115,6 +1115,12 < at >< at > struct nfs41_impl_id {
 struct nfstime4date;
 };
 
+struct nfs41_bind_conn_to_session_res {
+struct nfs4_session*session;
+u32dir;
+booluse_conn_in_rdma_mode;
+};
+
 struct nfs41_exchange_id_res {
 struct nfs_client*client;
 u32flags;

[PATCH 3/4] NFSv4.1 add nfs_inode book keeping for mdsthreshold

2012-05-24T17:13:24

From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>

Keep track of the number of bytes read or written via buffered, direct, and
mem-mapped i/o for use by mdsthreshold size_io hints.

Signed-off-by: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
 fs/nfs/direct.c        |    2 ++
 fs/nfs/file.c          |    1 +
 fs/nfs/inode.c         |    2 ++
 fs/nfs/pnfs.c          |    3 +++
 fs/nfs/read.c          |    2 ++
 include/linux/nfs_fs.h |    3 +++
 6 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c
index c47a46e..23d170b 100644
--- a/fs/nfs/direct.c
+++ b/fs/nfs/direct.c
< at >< at > -447,6 +447,7 < at >< at > static ssize_t nfs_direct_read(struct kiocb *iocb, const struct iovec *iov,
 result = nfs_direct_read_schedule_iovec(dreq, iov, nr_segs, pos);
 if (!result)
 result = nfs_direct_wait(dreq);
+NFS_I(inode)->read_io += result;
 out_release:
 nfs_direct_req_release(dreq);
 out:
< at >< at > -785,6 +786,7 < at >< at > static ssize_t nfs_direct_write_schedule_iovec(struct nfs_direct_req *dreq,
 pos += vec->iov_len;
 }
 nfs_pageio_complete(&desc);
+NFS_I(dreq->inode)->write_io += desc.pg_bytes_written;
 
 /*
  * If no bytes were started, return the error, and let the
diff --git a/fs/nfs/file.c b/fs/nfs/file.c
index 8eda8a6..56311ca 100644
--- a/fs/nfs/file.c
+++ b/fs/nfs/file.c
< at >< at > -424,6 +424,7 < at >< at > static int nfs_write_end(struct file *file, struct address_space *mapping,
 
 if (status < 0)
 return status;
+NFS_I(mapping->host)->write_io += copied;
 return copied;
 }
 
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 889f7e5..a6f5fbb 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
< at >< at > -323,6 +323,8 < at >< at > nfs_fhget(struct super_block *sb, struct nfs_fh *fh, struct nfs_fattr *fattr)
 inode->i_gid = -2;
 inode->i_blocks = 0;
 memset(nfsi->cookieverf, 0, sizeof(nfsi->cookieverf));
+nfsi->write_io = 0;
+nfsi->read_io = 0;
 
 nfsi->read_cache_jiffies = fattr->time_start;
 nfsi->attr_gencount = fattr->gencount;
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index cbcb6ae..6620606 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
< at >< at > -395,6 +395,9 < at >< at > mark_matching_lsegs_invalid(struct pnfs_layout_hdr *lo,
 dprintk("%s:Begin lo %p\n", __func__, lo);
 
 if (list_empty(&lo->plh_segs)) {
+/* Reset MDS Threshold I/O counters */
+NFS_I(lo->plh_inode)->write_io = 0;
+NFS_I(lo->plh_inode)->read_io = 0;
 if (!test_and_set_bit(NFS_LAYOUT_DESTROYED, &lo->plh_flags))
 put_layout_hdr_locked(lo);
 return 0;
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index 2cfdd77..86ced78 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
< at >< at > -152,6 +152,7 < at >< at > int nfs_readpage_async(struct nfs_open_context *ctx, struct inode *inode,
 nfs_pageio_init_read(&pgio, inode, &nfs_async_read_completion_ops);
 nfs_pageio_add_request(&pgio, new);
 nfs_pageio_complete(&pgio);
+NFS_I(inode)->read_io += pgio.pg_bytes_written;
 return 0;
 }
 
< at >< at > -656,6 +657,7 < at >< at > int nfs_readpages(struct file *filp, struct address_space *mapping,
 ret = read_cache_pages(mapping, pages, readpage_async_filler, &desc);
 
 nfs_pageio_complete(&pgio);
+NFS_I(inode)->read_io += pgio.pg_bytes_written;
 npages = (pgio.pg_bytes_written + PAGE_CACHE_SIZE - 1) >> PAGE_CACHE_SHIFT;
 nfs_add_stats(inode, NFSIOS_READPAGES, npages);
 read_complete:
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index ca4a707..9d44860 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
< at >< at > -202,6 +202,9 < at >< at > struct nfs_inode {
 /* pNFS layout information */
 struct pnfs_layout_hdr *layout;
 #endif /* CONFIG_NFS_V4*/
+/* how many bytes have been written/read and how many bytes queued up */
+__u64 write_io;
+__u64 read_io;
 #ifdef CONFIG_NFS_FSCACHE
 struct fscache_cookie*fscache;
 #endif

[PATCH 0/4] NFSv4.1 add nfs_inode book keeping for mdsthreshold

2012-05-24T17:13:23

From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>

Only resending NFSv4.1 add nfs_inode book keeping for mdsthreshold. 
Responded to Trond's comments.
I have not tested the directio nor the memmap io cases. The regular file
i/o works as intended.

Andy Adamson (4):
  NFSv4.1 mdsthreshold attribute xdr
  NFSv4.1 cache mdsthreshold values on OPEN
  NFSv4.1 add nfs_inode book keeping for mdsthreshold
  NFSv4.1 test the mdsthreshold hint parameters

 fs/nfs/direct.c         |    2 +
 fs/nfs/file.c           |    1 +
 fs/nfs/inode.c          |    4 ++
 fs/nfs/nfs4proc.c       |   38 ++++++++++++--
 fs/nfs/nfs4xdr.c        |  125 ++++++++++++++++++++++++++++++++++++++++++++++-
 fs/nfs/pnfs.c           |   94 +++++++++++++++++++++++++++++++++++
 fs/nfs/pnfs.h           |   21 ++++++++
 fs/nfs/read.c           |    2 +
 include/linux/nfs4.h    |    7 +++
 include/linux/nfs_fs.h  |    4 ++
 include/linux/nfs_xdr.h |   10 ++++
 11 files changed, 301 insertions(+), 7 deletions(-)

Re: [PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation

Adamson, Dros — 2012-05-24T17:07:52

On May 24, 2012, at 1:06 PM, Chuck Lever wrote:


I'm right there with you :)

-dros

Re: [PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation

Chuck Lever — 2012-05-24T17:06:42

On May 24, 2012, at 12:49 PM, Adamson, Dros wrote:


Right, that's an implementation-imposed limit.  proc is the right place for that.  The XDR check is to sanity check servers, and should allow all possible values of that field, and no more.  FWIW.