gmane.linux.nfs
http://blog.gmane.org/gmane.linux.nfs
hourly11901-01-01T00:00+00:00Gmanehttp://gmane.org/img/gmane-25t.png
http://gmane.org
[PATCH v2 1/9] NFSv4.1: Exchange ID must use GFP_NOFS allocation mode
http://comments.gmane.org/gmane.linux.nfs/50078
<pre>Exchange ID can be called in a lease reclaim situation, so it
will deadlock if it then tries to write out dirty NFS pages.
Signed-off-by: Trond Myklebust <Trond.Myklebust-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfs/nfs4proc.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index e8988c0..f8817e8 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5192,20 +5192,20 < at >< at > int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
clp->cl_rpcclient->cl_auth->au_flavor);
res.server_owner = kzalloc(sizeof(struct nfs41_server_owner),
-GFP_KERNEL);
+GFP_NOFS);
if (unlikely(res.server_owner == NULL)) {
status = -ENOMEM;
goto out;
}
res.server_scope = kzalloc(sizeof(struct nfs41_server_scope),
-GFP_KERNEL);
+GFP_NOFS);
if (unlikely(res.server_scope == NULL)) {
status = -ENOMEM;
goto out_server_owner;
}
-res.impl_id = kzalloc(sizeof(struct nfs41_impl_id), GFP_KERNEL);
+res.impl_id = kzalloc(sizeof(struct nfs41_impl_id), GFP_NOFS);
if (unlikely(res.impl_id == NULL)) {
status = -ENOMEM;
goto out_server_scope;
</pre>Trond Myklebust2012-05-26T18:39:52nfs-utils 1.2.6: error: #error It appears you have already included the Heimdal gssapi.h file
http://comments.gmane.org/gmane.linux.nfs/50076
<pre>Hello,
I am trying to compile nfs-utils 1.2.6 with NFSv4 and gss support, but
am running into the following error:
./configure --prefix=/usr --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --docdir=/usr/share/doc/nfs-utils-1.2.6 --sysconfdir=/etc --localstatedir=/var/lib --disable-dependency-tracking --disable-silent-rules --enable-fast-install --libdir=/usr/lib64 --mandir=/usr/share/man --enable-largefile --enable-libmount-mount --enable-mount --enable-mountconfig --enable-tirpc --with-statduser=nobody --with-statedir=/var/lib/nfs --disable-caps --enable-gss --enable-ipv6 --enable-nfsv4 --enable-nfsv41 --enable-nfsdcld --with-tcp-wrappers
[...]
x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../../support/include -I/usr/include/tirpc -D_GNU_SOURCE -Wall -Wextra -Wstrict-prototypes -pipe -march=native -O2 -pipe -I/usr/include/gssglue -I/usr/include -march=native -O2 -pipe -c -o gssd-context_lucid.o `test -f 'context_lucid.c' || echo './'`context_lucid.c
In file included from /usr/include/gssapi.h:39:0,
from /usr/include/gssapi/gssapi_krb5.h:39,
from context_lucid.c:47:
/usr/include/gssglue/gssapi/gssapi.h:42:2: error: #error It appears you have already included the Heimdal gssapi.h file
In file included from context_lucid.c:47:0:
/usr/include/gssapi/gssapi_krb5.h:56:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘extern’
/usr/include/gssapi/gssapi_krb5.h:63:41: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘__gss_krb5_mechanism_oid_desc’
/usr/include/gssapi/gssapi_krb5.h:79:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:79:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:79:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:84:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:84:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:84:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:87:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:87:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:87:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:90:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:90:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:90:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:95:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:95:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:95:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:102:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:102:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:102:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:107:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:107:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:107:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:114:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:114:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:114:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:122:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:122:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:122:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:126:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:126:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:126:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:129:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:129:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:129:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:134:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:134:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:134:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:138:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:138:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:138:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:142:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:142:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:142:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:147:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:147:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:147:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:150:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:150:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:150:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:159:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:159:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:159:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:190:2: error: unknown type name ‘OM_uint64’
/usr/include/gssapi/gssapi_krb5.h:191:2: error: unknown type name ‘OM_uint64’
/usr/include/gssapi/gssapi_krb5.h:205:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:205:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:205:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:212:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:212:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:212:31: error: unknown type name ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:217:1: error: unknown type name ‘GSSAPI_LIB_FUNCTION’
/usr/include/gssapi/gssapi_krb5.h:217:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘GSSAPI_LIB_CALL’
/usr/include/gssapi/gssapi_krb5.h:217:31: error: unknown type name ‘GSSAPI_LIB_CALL’
In file included from /usr/include/stdlib.h:33:0,
from gss_util.h:34,
from context_lucid.c:49:
/usr/lib64/gcc/x86_64-pc-linux-gnu/4.6.3/include/stddef.h:324:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘typedef’
In file included from gss_util.h:34:0,
from context_lucid.c:49:
/usr/include/stdlib.h:863:20: error: unknown type name ‘wchar_t’
/usr/include/stdlib.h:867:31: error: unknown type name ‘wchar_t’
/usr/include/stdlib.h:871:25: error: unknown type name ‘wchar_t’
/usr/include/stdlib.h:875:4: error: unknown type name ‘wchar_t’
make[2]: *** [gssd-context_lucid.o] Error 1
make[2]: Leaving directory `/var/tmp/paludis/build/net-fs-nfs-utils-1.2.6/work/nfs-utils-1.2.6/utils/gssd'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/tmp/paludis/build/net-fs-nfs-utils-1.2.6/work/nfs-utils-1.2.6/utils'
make: *** [all-recursive] Error 1
I have the following packages installed:
libgssglue 0.4
heimdal 1.5.2
This is an error with nfs-utils or libgssglue/heimdal?
Regards,
Markus
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Markus Rothe2012-05-26T12:37:52FROM MRS SUSAN SHABANGU(OPEN THE ATTACH FILES)
http://comments.gmane.org/gmane.linux.nfs/50075
<pre></pre>FROM MRS SUSAN SHABANGU2012-05-26T10:56:23[PATCH 0/4] Add support for new RPCSEC_GSS upcall mechanism for nfsd
http://comments.gmane.org/gmane.linux.nfs/50069
<pre>This patchset implements a new upcall mechanism that uses the sunrpc
client to talk to gssproxy[1], a new userspace daemon that handles gssapi
operations on behalf of other processes on the system.
The main driver for this new mechanism is to overcome limitations with
the current daemon and upcall. The current code cannot handle tickets
larger than approximatively 2k and cannot handle sending back large user
credential sets to the kernel.
These patches have been tested against the development version of gssproxy
tagged as kernel_v0.1 in the master repo[2].
I have tested walking into mountpoints using tickets artificially pumped
up to 64k and the user is properly authorized, after the accept_se_context
call is performed through the new upcall mechanism and gssproxy.
The gssproxy has the potential of handling also init_sec_context calls,
but at the moment the only targeted system is nfsd.
Simo.
[1] https://fedorahosted.org/gss-proxy/
[2] http://git.fedorahosted.org/git/?p=gss-proxy.git;a=shortlog;h=refs/tags/kernel_v0.1
NOTE: Included are all changes request and agreed on the list recently,
kmalloc changes and containers compatibility changes.
Pacthes rebased on top of Bruce Field's for-3.5 tree.
Simo Sorce (4):
SUNRPC: conditionally return endtime from import_sec_context
SUNRPC: Document a bit RPCGSS handling in the NFS Server
SUNRPC: Add RPC based upcall mechanism for RPCGSS auth
SUNRPC: Use gssproxy upcall for nfsd's RPCGSS authentication.
Documentation/filesystems/nfs/00-INDEX | 2 +
Documentation/filesystems/nfs/knfsd-rpcgss.txt | 65 ++
include/linux/sunrpc/auth_gss.h | 3 +
include/linux/sunrpc/gss_api.h | 2 +
include/linux/sunrpc/svcauth_gss.h | 2 +-
net/sunrpc/auth_gss/Makefile | 4 +-
net/sunrpc/auth_gss/auth_gss.c | 11 +-
net/sunrpc/auth_gss/gss_krb5_mech.c | 3 +
net/sunrpc/auth_gss/gss_mech_switch.c | 5 +-
net/sunrpc/auth_gss/gss_rpc_upcall.c | 353 +++++++++
net/sunrpc/auth_gss/gss_rpc_upcall.h | 43 ++
net/sunrpc/auth_gss/gss_rpc_xdr.c | 907 ++++++++++++++++++++++++
net/sunrpc/auth_gss/gss_rpc_xdr.h | 269 +++++++
net/sunrpc/auth_gss/svcauth_gss.c | 226 ++++++-
14 files changed, 1875 insertions(+), 20 deletions(-)
create mode 100644 Documentation/filesystems/nfs/knfsd-rpcgss.txt
create mode 100644 net/sunrpc/auth_gss/gss_rpc_upcall.c
create mode 100644 net/sunrpc/auth_gss/gss_rpc_upcall.h
create mode 100644 net/sunrpc/auth_gss/gss_rpc_xdr.c
create mode 100644 net/sunrpc/auth_gss/gss_rpc_xdr.h
</pre>Simo Sorce2012-05-25T22:09:52[PATCH 1/8] NFSv4.1: Exchange ID must use GFP_NOFS allocation mode
http://comments.gmane.org/gmane.linux.nfs/50061
<pre>Exchange ID can be called in a lease reclaim situation, so it
will deadlock if it then tries to write out dirty NFS pages.
Signed-off-by: Trond Myklebust <Trond.Myklebust-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfs/nfs4proc.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index e8988c0..f8817e8 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5192,20 +5192,20 < at >< at > int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
clp->cl_rpcclient->cl_auth->au_flavor);
res.server_owner = kzalloc(sizeof(struct nfs41_server_owner),
-GFP_KERNEL);
+GFP_NOFS);
if (unlikely(res.server_owner == NULL)) {
status = -ENOMEM;
goto out;
}
res.server_scope = kzalloc(sizeof(struct nfs41_server_scope),
-GFP_KERNEL);
+GFP_NOFS);
if (unlikely(res.server_scope == NULL)) {
status = -ENOMEM;
goto out_server_owner;
}
-res.impl_id = kzalloc(sizeof(struct nfs41_impl_id), GFP_KERNEL);
+res.impl_id = kzalloc(sizeof(struct nfs41_impl_id), GFP_NOFS);
if (unlikely(res.impl_id == NULL)) {
status = -ENOMEM;
goto out_server_scope;
</pre>Trond Myklebust2012-05-25T22:03:15BUG: At key_instantiate_and_link+0x2e/0xa0
http://comments.gmane.org/gmane.linux.nfs/50054
<pre>I got the below on a 3.4-rc7 based Benny's pnfs tree.
During normal nfs4 only operations. (Just my development trees mounted on client machine)
If it will happen again I will investigate. I'll leave it off for now (busy with other stuff)
Just wanted to know if it might have already been fix or seen by someone else
[ 3830.441139] BUG: unable to handle kernel NULL pointer dereference at 0000000000000078
[ 3830.442024] IP: [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024] PGD 0
[ 3830.442024] Oops: 0000 [#1] SMP
[ 3830.442024] CPU 1
[ 3830.442024] Modules linked in: osd libosd nfsd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ip6t_REJECT be2iscsi nf_conntrack_ipv6 nf_defrag_ipv6 iscsi_boot_sysfs xt_state bnx2i nf_conntrack cnic uio cxgb3i libcxgbi cxgb3 mdio ip6table_filter ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ip6_tables iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi joydev virtio_balloon i2c_piix4 i2c_core pcspkr virtio_net microcode floppy virtio_blk [last unloaded: libosd]
[ 3830.442024]
[ 3830.442024] Pid: 1288, comm: rpc.idmapd Not tainted 3.4.0-rc7-pnfs+ #50 Bochs Bochs
[ 3830.442024] RIP: 0010:[<ffffffff812894ce>] [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024] RSP: 0018:ffff88003a327d98 EFLAGS: 00010286
[ 3830.442024] RAX: 0000000000000002 RBX: 0000000000000000 RCX: ffff8800378579c0
[ 3830.442024] RDX: 0000000000000003 RSI: ffff88003a327e75 RDI: 0000000000000000
[ 3830.442024] RBP: ffff88003a327dd8 R08: ffff88003a26a9c0 R09: 000000000000fffd
[ 3830.442024] R10: 000000000000000a R11: 0000000000000246 R12: ffff8800378579c0
[ 3830.442024] R13: 0000000000000003 R14: ffff88003a327e75 R15: ffff88003a26a9c0
[ 3830.442024] FS: 00007f8c539d1720(0000) GS:ffff88003e500000(0000) knlGS:0000000000000000
[ 3830.442024] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3830.442024] CR2: 0000000000000078 CR3: 00000000379fe000 CR4: 00000000000006e0
[ 3830.442024] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3830.442024] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3830.442024] Process rpc.idmapd (pid: 1288, threadinfo ffff88003a326000, task ffff88003a329710)
[ 3830.442024] Stack:
[ 3830.442024] ffff88003a327de8 ffff88003a327da8 ffff88003a327dd8 ffffffffa01bfe8d
[ 3830.442024] ffff8800399ce930 ffff880039ff1340 ffff88003a327dea 0000000000000000
[ 3830.442024] ffff88003a327eb8 ffffffffa026360b 4073726573750101 2e736173616e6170
[ 3830.442024] Call Trace:
[ 3830.442024] [<ffffffffa01bfe8d>] ? rpc_pipe_poll+0x6d/0xa0 [sunrpc]
[ 3830.442024] [<ffffffffa026360b>] idmap_pipe_downcall+0x1cb/0x1e0 [nfs]
[ 3830.442024] [<ffffffffa01bff27>] rpc_pipe_write+0x67/0x90 [sunrpc]
[ 3830.442024] [<ffffffff8117e8e3>] vfs_write+0xb3/0x180
[ 3830.442024] [<ffffffff8117ec0a>] sys_write+0x4a/0x90
[ 3830.442024] [<ffffffff81077811>] ? posix_ktime_get_ts+0x11/0x20
[ 3830.442024] [<ffffffff816215e9>] system_call_fastpath+0x16/0x1b
[ 3830.442024] Code: e5 48 83 ec 40 48 89 5d e0 4c 89 65 e8 4c 89 6d f0 4c 89 75 f8 66 66 66 66 90 48 85 c9 48 89 fb 49 89 f6 49 89 d5 49 89 cc 74 62 <48> 8b 57 78 48 8b 77 20 48 8d 4d d8 4c 89 e7 4c 89 45 c8 e8 0a
[ 3830.442024] RIP [<ffffffff812894ce>] key_instantiate_and_link+0x2e/0xa0
[ 3830.442024] RSP <ffff88003a327d98>
[ 3830.442024] CR2: 0000000000000078
[ 3830.492152] ---[ end trace a0dbf47ab1df84a5 ]---
Thanks
Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Boaz Harrosh2012-05-25T10:12:473.0+ NFS issues
http://comments.gmane.org/gmane.linux.nfs/50052
<pre>Hello.
I updated my nfs server machine to kernel 3.0, and
noticed that its main usage become, well, problematic.
While trying to dig deeper, I also found a few other
interesting issues, which are mentioned below.
But first thing first: nfs.
i686pae kernel, lots of RAM, Atom-based (cedar trail)
machine with usual rtl8169 NIC. 3.0 or 3.2 kernel
(I will try current 3.4 but I don't have much hopes
there). NFSv4.
When a client machine (also 3.0 kernel) does some reading,
the process often stalls somewhere in the read syscall,
or, rarer, during close, for up to two MINUTES. During
this time, the client (kernel) reports "NFS server <foo>
does not respond" several times, and finally "NFS server
<foo> ok", client process "unstucks" from the read(2),
and is able to perform a few more reads till the whole
thing repeats.
While this happens, ie, while the client process is stuck,
any access to the mounted filesystem results in an
uninterruptible sleep - all processes including the
initial client are unkillable. This is very annoying,
but I was never able to fix this, including the real
case when the server becomes unreachable - "intr" mount
option which is supposed to help here makes no difference.
This is first and longstanding issue, but it is not really
related to the issue at hand.
On the server side things are more interesting. During
the client stall, all NFSD threads are busy running,
using all available CPU. The machine has 2 cores, and
if I reduce number of NFSD threads to 1, one core will
be used by the single NFSD to 100%. With default 8
NFSDs, both cores are used, by 4 NFSD threads each.
This is according to top(1), which show proportional
and equal CPU activity for each of NFSD thread.
But. And this is where things becomes more interesting.
perf top shows almost no activity with any number of NFSD
threads but one. For example, this is a typical perf top
display with 2 NFSD threads:
samples pcnt function DSO
1558.00 6.4% svc_recv [sunrpc]
1084.00 5.2% __schedule [kernel]
1065.00 5.1% select_task_rq_fair [kernel]
692.00 3.3% try_to_wake_up [kernel]
Yes, there's some 6.4% time spent in svc_recv, but it is
FAR from the 200% CPU usage reported by top(1). Now, when
I reduce number of NFSD threds to 1, perf top output becomes
dramatically different:
samples pcnt function DSO
5996.00 59.2% svc_recv [sunrpc]
750.00 7.4% cache_check [sunrpc]
519.00 5.1% svc_tcp_has_wspace [sunrpc]
465.00 4.6% local_bh_enable_ip [kernel.kallsyms]
301.00 3.0% svc_defer [sunrpc]
294.00 2.9% page_address [kernel.kallsyms]
Ie, scv_recv time usage jumps from 6 to 60%, ie, 10 times
the difference.
So, apparently, `perf top' tool does not show the truth.
Trying to catch this high CPU usage with Sysrq+T does not
show anything interesting, the NFSD thread is running (and
is the only thread(s) shown in the "Runnable tasks" output):
May 25 10:48:37 gnome vmunix: [ 1709.483771] nfsd R running 0 5853 2 0x00000000
May 25 10:48:37 gnome vmunix: [ 1709.483796] f4477030 00000046 00000000 5c8e8929 0000018e f4c700f0 f4043a70 f545b140
May 25 10:48:37 gnome vmunix: [ 1709.483841] 050a8544 0000018e c1469140 c1469140 f4043a70 00000086 00000000 c1048575
May 25 10:48:37 gnome vmunix: [ 1709.483887] c101ba11 c12fac18 f31bf800 f8f7bd30 0000003a c12fb156 0000003a 00000000
May 25 10:48:37 gnome vmunix: [ 1709.483931] Call Trace:
May 25 10:48:37 gnome vmunix: [ 1709.483950] [<c1048575>] ? irq_exit+0x35/0x90
May 25 10:48:37 gnome vmunix: [ 1709.483973] [<c101ba11>] ? smp_apic_timer_interrupt+0x51/0x90
May 25 10:48:37 gnome vmunix: [ 1709.483997] [<c12fac18>] ? _raw_spin_lock_bh+0x8/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484029] [<f8f7bd30>] ? svc_xprt_enqueue+0x40/0x1e0 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484053] [<c12fb156>] ? apic_timer_interrupt+0x2a/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484084] [<f8f7007b>] ? svc_destroy+0xdb/0x110 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484107] [<c103d3e3>] ? __cond_resched+0x13/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484129] [<c12f92b5>] ? _cond_resched+0x25/0x30
May 25 10:48:37 gnome vmunix: [ 1709.484159] [<f8f7cb5d>] ? svc_recv+0x1cd/0x7e0 [sunrpc]
May 25 10:48:37 gnome vmunix: [ 1709.484182] [<c101ba11>] ? smp_apic_timer_interrupt+0x51/0x90
May 25 10:48:37 gnome vmunix: [ 1709.484206] [<c103a3d0>] ? try_to_wake_up+0x200/0x200
May 25 10:48:37 gnome vmunix: [ 1709.484233] [<f90ea08c>] ? nfsd+0x8c/0x120 [nfsd]
May 25 10:48:37 gnome vmunix: [ 1709.484255] [<c102d240>] ? complete+0x40/0x60
May 25 10:48:37 gnome vmunix: [ 1709.484279] [<f90ea000>] ? 0xf90e9fff
May 25 10:48:37 gnome vmunix: [ 1709.484299] [<c105ea99>] ? kthread+0x69/0x70
May 25 10:48:37 gnome vmunix: [ 1709.484321] [<c105ea30>] ? kthread_worker_fn+0x150/0x150
May 25 10:48:37 gnome vmunix: [ 1709.484345] [<c1301876>] ? kernel_thread_helper+0x6/0xd
May 25 10:49:37 gnome vmunix: [ 1769.982791] nfsd R running 0 5853 2 0x00000000
May 25 10:49:37 gnome vmunix: [ 1769.982816] f5449f80 00000000 00000000 7636b581 0000019c 00000003 7603dcb8 0000019c
May 25 10:49:37 gnome vmunix: [ 1769.982859] 7603dcb8 0000019c 00000046 00000000 f4043a70 00000086 00000000 c1048575
May 25 10:49:37 gnome vmunix: [ 1769.982903] f31bf800 c12fac18 f31bf800 f8f7bd30 00000047 f31bf800 f418c2c0 f8f7c03c
May 25 10:49:37 gnome vmunix: [ 1769.982948] Call Trace:
May 25 10:49:37 gnome vmunix: [ 1769.982966] [<c1048575>] ? irq_exit+0x35/0x90
May 25 10:49:37 gnome vmunix: [ 1769.982987] [<c12fac18>] ? _raw_spin_lock_bh+0x8/0x30
May 25 10:49:37 gnome vmunix: [ 1769.983019] [<f8f7bd30>] ? svc_xprt_enqueue+0x40/0x1e0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983050] [<f8f7c03c>] ? svc_xprt_release+0x1c/0xd0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983081] [<f8f7c0ad>] ? svc_xprt_release+0x8d/0xd0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983112] [<f8f7cc03>] ? svc_recv+0x273/0x7e0 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983143] [<f8f7c300>] ? svc_age_temp_xprts+0x110/0x190 [sunrpc]
May 25 10:49:37 gnome vmunix: [ 1769.983166] [<c103a3d0>] ? try_to_wake_up+0x200/0x200
May 25 10:49:37 gnome vmunix: [ 1769.983193] [<f90ea08c>] ? nfsd+0x8c/0x120 [nfsd]
May 25 10:49:37 gnome vmunix: [ 1769.983215] [<c102d240>] ? complete+0x40/0x60
May 25 10:49:37 gnome vmunix: [ 1769.983238] [<f90ea000>] ? 0xf90e9fff
May 25 10:49:37 gnome vmunix: [ 1769.983258] [<c105ea99>] ? kthread+0x69/0x70
May 25 10:49:37 gnome vmunix: [ 1769.983278] [<c105ea30>] ? kthread_worker_fn+0x150/0x150
May 25 10:49:37 gnome vmunix: [ 1769.983301] [<c1301876>] ? kernel_thread_helper+0x6/0xd
This is 3.2, on 3.0 it was in different place.
This problem is persistent, always reproducible, happens
on reads from the server (I haven't seen it with writes),
and completely goes away when reverting to the kernel I
previously used on this machine - 2.6.32. But unfortunately
2.6.32 does not quite work on the new hardware (I replaced
the motherboard), due to lack of drivers for several pieces.
(This is a different story, I think NFSD should work in 3.0
regardless of 2.6.32 issues :)
Anything I can do to make NFS work again?
Thanks,
/mjt
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Michael Tokarev2012-05-25T06:53:11[PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation
http://comments.gmane.org/gmane.linux.nfs/50050
<pre>This patch adds the BIND_CONN_TO_SESSION operation which is needed for
upcoming SP4_MACH_CRED work and useful for recovering from broken connections
without destroying the session.
Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Updated with comment from Trond
fs/nfs/nfs4_fs.h | 1 +
fs/nfs/nfs4proc.c | 54 ++++++++++++++++++++++++++
fs/nfs/nfs4xdr.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++
include/linux/nfs4.h | 5 ++
include/linux/nfs_xdr.h | 6 +++
5 files changed, 161 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index edeef71..ad90bc7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -215,6 +215,7 < at >< at > struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
+extern int nfs4_proc_bind_conn_to_session(struct nfs_client *);
extern int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred);
extern int nfs4_init_clientid(struct nfs_client *, struct rpc_cred *);
extern int nfs41_init_clientid(struct nfs_client *, struct rpc_cred *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78784e5..1d1bef6 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5062,6 +5062,60 < at >< at > nfs41_same_server_scope(struct server_scope *a, struct server_scope *b)
}
/*
+ * nfs4_proc_bind_conn_to_session()
+ *
+ * The 4.1 client currently uses the same TCP connection for the
+ * fore and backchannel.
+ */
+int nfs4_proc_bind_conn_to_session(struct nfs_client *clp)
+{
+int status;
+struct nfs41_bind_conn_to_session_res res;
+struct rpc_message msg = {
+.rpc_proc =
+&nfs4_procedures[NFSPROC4_CLNT_BIND_CONN_TO_SESSION],
+.rpc_argp = clp,
+.rpc_resp = &res,
+};
+
+dprintk("--> %s\n", __func__);
+BUG_ON(clp == NULL);
+
+res.session = kzalloc(sizeof(struct nfs4_session), GFP_NOFS);
+if (unlikely(res.session == NULL)) {
+status = -ENOMEM;
+goto out;
+}
+
+status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+if (status == 0) {
+if (memcmp(res.session->sess_id.data,
+ clp->cl_session->sess_id.data, NFS4_MAX_SESSIONID_LEN)) {
+dprintk("NFS: %s: Session ID mismatch\n", __func__);
+status = -EIO;
+goto out_session;
+}
+if (res.dir != NFS4_CDFS4_BOTH) {
+dprintk("NFS: %s: Unexpected direction from server\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+if (res.use_conn_in_rdma_mode) {
+dprintk("NFS: %s: Server returned RDMA mode = true\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+}
+out_session:
+kfree(res.session);
+out:
+dprintk("<-- %s status= %d\n", __func__, status);
+return status;
+}
+
+/*
* nfs4_proc_exchange_id()
*
* Since the clientid has expired, all compounds using sessions
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..e9be74f 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -321,6 +321,16 < at >< at > static int nfs4_stat_to_errno(int);
1 /* csr_flags */ + \
decode_channel_attrs_maxsz + \
decode_channel_attrs_maxsz)
+#define encode_bind_conn_to_session_maxsz (op_encode_hdr_maxsz + \
+ /* bctsa_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsa_dir */ + \
+ 1 /* bctsa_use_conn_in_rdma_mode */)
+#define decode_bind_conn_to_session_maxsz (op_decode_hdr_maxsz +\
+ /* bctsr_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsr_dir */ + \
+ 1 /* bctsr_use_conn_in_rdma_mode */)
#define encode_destroy_session_maxsz (op_encode_hdr_maxsz + 4)
#define decode_destroy_session_maxsz (op_decode_hdr_maxsz)
#define encode_sequence_maxsz(op_encode_hdr_maxsz + \
< at >< at > -714,6 +724,12 < at >< at > static int nfs4_stat_to_errno(int);
decode_putfh_maxsz + \
decode_secinfo_maxsz)
#if defined(CONFIG_NFS_V4_1)
+#define NFS4_enc_bind_conn_to_session_sz \
+(compound_encode_hdr_maxsz + \
+ encode_bind_conn_to_session_maxsz)
+#define NFS4_dec_bind_conn_to_session_sz \
+(compound_decode_hdr_maxsz + \
+ decode_bind_conn_to_session_maxsz)
#define NFS4_enc_exchange_id_sz \
(compound_encode_hdr_maxsz + \
encode_exchange_id_maxsz)
< at >< at > -1654,6 +1670,20 < at >< at > static void encode_secinfo(struct xdr_stream *xdr, const struct qstr *name, stru
#if defined(CONFIG_NFS_V4_1)
/* NFSv4.1 operations */
+static void encode_bind_conn_to_session(struct xdr_stream *xdr,
+ struct nfs4_session *session,
+ struct compound_hdr *hdr)
+{
+__be32 *p;
+
+encode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION,
+decode_bind_conn_to_session_maxsz, hdr);
+encode_opaque_fixed(xdr, session->sess_id.data, NFS4_MAX_SESSIONID_LEN);
+p = xdr_reserve_space(xdr, 8);
+*p++ = cpu_to_be32(NFS4_CDFC4_BACK_OR_BOTH);
+*p = 0;/* use_conn_in_rdma_mode = False */
+}
+
static void encode_exchange_id(struct xdr_stream *xdr,
struct nfs41_exchange_id_args *args,
struct compound_hdr *hdr)
< at >< at > -2614,6 +2644,22 < at >< at > static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
#if defined(CONFIG_NFS_V4_1)
/*
+ * BIND_CONN_TO_SESSION request
+ */
+static void nfs4_xdr_enc_bind_conn_to_session(struct rpc_rqst *req,
+struct xdr_stream *xdr,
+struct nfs_client *clp)
+{
+struct compound_hdr hdr = {
+.minorversion = clp->cl_mvops->minor_version,
+};
+
+encode_compound_hdr(xdr, req, &hdr);
+encode_bind_conn_to_session(xdr, clp->cl_session, &hdr);
+encode_nops(&hdr);
+}
+
+/*
* EXCHANGE_ID request
*/
static void nfs4_xdr_enc_exchange_id(struct rpc_rqst *req,
< at >< at > -5239,6 +5285,37 < at >< at > static int decode_sessionid(struct xdr_stream *xdr, struct nfs4_sessionid *sid)
return decode_opaque_fixed(xdr, sid->data, NFS4_MAX_SESSIONID_LEN);
}
+static int decode_bind_conn_to_session(struct xdr_stream *xdr,
+struct nfs41_bind_conn_to_session_res *res)
+{
+__be32 *p;
+int status;
+
+status = decode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION);
+if (!status)
+status = decode_sessionid(xdr, &res->session->sess_id);
+if (unlikely(status))
+return status;
+
+/* dir flags, rdma mode bool */
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+
+res->dir = be32_to_cpup(p++);
+if (res->dir == 0 || res->dir > NFS4_CDFS4_BOTH)
+return -EIO;
+if (be32_to_cpup(p) == 0)
+res->use_conn_in_rdma_mode = false;
+else
+res->use_conn_in_rdma_mode = true;
+
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
static int decode_create_session(struct xdr_stream *xdr,
struct nfs41_create_session_res *res)
{
< at >< at > -6521,6 +6598,22 < at >< at > out:
#if defined(CONFIG_NFS_V4_1)
/*
+ * Decode BIND_CONN_TO_SESSION response
+ */
+static int nfs4_xdr_dec_bind_conn_to_session(struct rpc_rqst *rqstp,
+struct xdr_stream *xdr,
+void *res)
+{
+struct compound_hdr hdr;
+int status;
+
+status = decode_compound_hdr(xdr, &hdr);
+if (!status)
+status = decode_bind_conn_to_session(xdr, res);
+return status;
+}
+
+/*
* Decode EXCHANGE_ID response
*/
static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp,
< at >< at > -7001,6 +7094,8 < at >< at > struct rpc_procinfonfs4_procedures[] = {
PROC(RELEASE_LOCKOWNER,enc_release_lockowner,dec_release_lockowner),
PROC(SECINFO,enc_secinfo,dec_secinfo),
#if defined(CONFIG_NFS_V4_1)
+PROC(BIND_CONN_TO_SESSION,
+enc_bind_conn_to_session, dec_bind_conn_to_session),
PROC(EXCHANGE_ID,enc_exchange_id,dec_exchange_id),
PROC(CREATE_SESSION,enc_create_session,dec_create_session),
PROC(DESTROY_SESSION,enc_destroy_session,dec_destroy_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..5b1a504 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -69,6 +69,10 < at >< at >
#define NFS4_CDFC4_FORE_OR_BOTH 0x3
#define NFS4_CDFC4_BACK_OR_BOTH 0x7
+#define NFS4_CDFS4_FORE 0x1
+#define NFS4_CDFS4_BACK 0x2
+#define NFS4_CDFS4_BOTH 0x3
+
#define NFS4_SET_TO_SERVER_TIME0
#define NFS4_SET_TO_CLIENT_TIME1
< at >< at > -582,6 +586,7 < at >< at > enum {
NFSPROC4_CLNT_SECINFO,
/* nfs41 */
+NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
NFSPROC4_CLNT_EXCHANGE_ID,
NFSPROC4_CLNT_CREATE_SESSION,
NFSPROC4_CLNT_DESTROY_SESSION,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..aa2a3bb 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -1115,6 +1115,12 < at >< at > struct nfs41_impl_id {
struct nfstime4date;
};
+struct nfs41_bind_conn_to_session_res {
+struct nfs4_session*session;
+u32dir;
+booluse_conn_in_rdma_mode;
+};
+
struct nfs41_exchange_id_res {
struct nfs_client*client;
u32flags;
</pre>Weston Andros Adamson2012-05-24T21:19:33[PATCH 1/2] nfsd: add fault injection for CB_PATH_DOWN
http://comments.gmane.org/gmane.linux.nfs/50042
<pre>This fault injection hook causes all SEQUENCE operation replies to have the
SEQ4_STATUS_CB_PATH_DOWN flag set until the client calls BIND_CONN_TO_SESSION
or creates a new session.
Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfsd/fault_inject.c | 4 ++++
fs/nfsd/fault_inject.h | 2 ++
fs/nfsd/nfs4state.c | 20 ++++++++++++++++++++
3 files changed, 26 insertions(+), 0 deletions(-)
diff --git a/fs/nfsd/fault_inject.c b/fs/nfsd/fault_inject.c
index e6c3815..ab81144 100644
--- a/fs/nfsd/fault_inject.c
+++ b/fs/nfsd/fault_inject.c
< at >< at > -38,6 +38,10 < at >< at > static struct nfsd_fault_inject_op inject_ops[] = {
.file = "recall_delegations",
.func = nfsd_recall_delegations,
},
+{
+.file = "kill_backchannel",
+.func = nfsd_kill_backchannel,
+},
};
static long int NUM_INJECT_OPS = sizeof(inject_ops) / sizeof(struct nfsd_fault_inject_op);
diff --git a/fs/nfsd/fault_inject.h b/fs/nfsd/fault_inject.h
index 90bd057..b058e03 100644
--- a/fs/nfsd/fault_inject.h
+++ b/fs/nfsd/fault_inject.h
< at >< at > -15,6 +15,7 < at >< at > void nfsd_forget_locks(u64);
void nfsd_forget_openowners(u64);
void nfsd_forget_delegations(u64);
void nfsd_recall_delegations(u64);
+void nfsd_kill_backchannel(u64);
#else /* CONFIG_NFSD_FAULT_INJECTION */
static inline int nfsd_fault_inject_init(void) { return 0; }
static inline void nfsd_fault_inject_cleanup(void) {}
< at >< at > -23,6 +24,7 < at >< at > static inline void nfsd_forget_locks(u64 num) {}
static inline void nfsd_forget_openowners(u64 num) {}
static inline void nfsd_forget_delegations(u64 num) {}
static inline void nfsd_recall_delegations(u64 num) {}
+static inline void nfsd_kill_backchannel(u64 num) {}
#endif /* CONFIG_NFSD_FAULT_INJECTION */
#endif /* LINUX_NFSD_FAULT_INJECT_H */
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 21266c7..9afc902 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
< at >< at > -4737,6 +4737,26 < at >< at > void nfsd_recall_delegations(u64 num)
printk(KERN_INFO "NFSD: Recalled %d delegations", count);
}
+/*
+ * Force SEQUENCE operations to have SEQ4_STATUS_CB_PATH_DOWN flag set
+ * until backchannel is reestablished by BIND_CONN_TO_SESSION or
+ * DESTROY_SESSION/CREATE_SESSION with SP4_NONE.
+ *
+ * The argument 'num' is ignored, any value will trigger the fault on
+ * all clients.
+ */
+void nfsd_kill_backchannel(u64 num)
+{
+struct nfs4_client *clp, *next;
+
+nfs4_lock_state();
+list_for_each_entry_safe(clp, next, &client_lru, cl_lru) {
+clp->cl_cb_state = NFSD4_CB_DOWN;
+}
+nfs4_unlock_state();
+
+printk(KERN_INFO "NFSD: killed backchannel");
+}
#endif /* CONFIG_NFSD_FAULT_INJECTION */
/* initialization to perform at module load time: */
</pre>Weston Andros Adamson2012-05-24T19:42:16[PATCH] NFSD: TEST_STATEID should not return NFS4ERR_STALE_STATEID
http://comments.gmane.org/gmane.linux.nfs/50041
<pre>The error values that TEST_STATEID is allowed to return does not
include NFS4ERR_STALE_STATEID. In addition, RFC 5661 says:
15.1.16.5. NFS4ERR_STALE_STATEID (Error Code 10023)
A stateid generated by an earlier server instance was used. This
error is moot in NFSv4.1 because all operations that take a stateid
MUST be preceded by the SEQUENCE operation, and the earlier server
instance is detected by the session infrastructure that supports
SEQUENCE.
I triggered the NFS4ERR_STALE_STATEID during nograce recovery testing.
My client had updated its boot verifier, so the server instance hadn't
changed, but the client instance had. Thus the server allowed the
SEQUENCE operation, but returned NFS4ERR_STALE_STATEID on the
TEST_STATEID operation.
After a client's lease expires, TEST_STATEID should report
NFS4ERR_EXPIRED for state IDs that the client tries to recover. I
don't see a way to make that happen, though.
Finally, RFC 5661, section 18.48.3 has this:
o Special stateids are always considered invalid (they result in the
error code NFS4ERR_BAD_STATEID).
Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA< at >public.gmane.org>
---
Bruce, would you consider taking something like this?
fs/nfsd/nfs4state.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 9235cfa..ae1fab3 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
< at >< at > -3365,12 +3365,13 < at >< at > __be32 nfs4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
struct nfs4_ol_stateid *ols;
__be32 status;
+if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
+return nfserr_bad_stateid;
if (STALE_STATEID(stateid))
-return nfserr_stale_stateid;
-
+return nfserr_bad_stateid;
s = find_stateid(cl, stateid);
if (!s)
- return nfserr_stale_stateid;
+ return nfserr_bad_stateid;
status = check_stateid_generation(stateid, &s->sc_stateid, 1);
if (status)
return status;
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Chuck Lever2012-05-24T19:29:51[PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation
http://comments.gmane.org/gmane.linux.nfs/50039
<pre>This patch adds the BIND_CONN_TO_SESSION operation which is needed for
upcoming SP4_MACH_CRED work and useful for recovering from broken connections
without destroying the session.
Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
Reposted with changes addressing Chuck's comments.
fs/nfs/nfs4_fs.h | 1 +
fs/nfs/nfs4proc.c | 54 ++++++++++++++++++++++++++
fs/nfs/nfs4xdr.c | 95 +++++++++++++++++++++++++++++++++++++++++++++++
include/linux/nfs4.h | 5 ++
include/linux/nfs_xdr.h | 6 +++
5 files changed, 161 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index edeef71..ad90bc7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -215,6 +215,7 < at >< at > struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
+extern int nfs4_proc_bind_conn_to_session(struct nfs_client *);
extern int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred);
extern int nfs4_init_clientid(struct nfs_client *, struct rpc_cred *);
extern int nfs41_init_clientid(struct nfs_client *, struct rpc_cred *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78784e5..95aca62 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5062,6 +5062,60 < at >< at > nfs41_same_server_scope(struct server_scope *a, struct server_scope *b)
}
/*
+ * nfs4_proc_bind_conn_to_session()
+ *
+ * The 4.1 client currently uses the same TCP connection for the
+ * fore and backchannel.
+ */
+int nfs4_proc_bind_conn_to_session(struct nfs_client *clp)
+{
+int status;
+struct nfs41_bind_conn_to_session_res res;
+struct rpc_message msg = {
+.rpc_proc =
+&nfs4_procedures[NFSPROC4_CLNT_BIND_CONN_TO_SESSION],
+.rpc_argp = clp,
+.rpc_resp = &res,
+};
+
+dprintk("--> %s\n", __func__);
+BUG_ON(clp == NULL);
+
+res.session = kzalloc(sizeof(struct nfs4_session), GFP_KERNEL);
+if (unlikely(res.session == NULL)) {
+status = -ENOMEM;
+goto out;
+}
+
+status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+if (status == 0) {
+if (memcmp(res.session->sess_id.data,
+ clp->cl_session->sess_id.data, NFS4_MAX_SESSIONID_LEN)) {
+dprintk("NFS: %s: Session ID mismatch\n", __func__);
+status = -EIO;
+goto out_session;
+}
+if (res.dir != NFS4_CDFS4_BOTH) {
+dprintk("NFS: %s: Unexpected direction from server\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+if (res.use_conn_in_rdma_mode) {
+dprintk("NFS: %s: Server returned RDMA mode = true\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+}
+out_session:
+kfree(res.session);
+out:
+dprintk("<-- %s status= %d\n", __func__, status);
+return status;
+}
+
+/*
* nfs4_proc_exchange_id()
*
* Since the clientid has expired, all compounds using sessions
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..e9be74f 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -321,6 +321,16 < at >< at > static int nfs4_stat_to_errno(int);
1 /* csr_flags */ + \
decode_channel_attrs_maxsz + \
decode_channel_attrs_maxsz)
+#define encode_bind_conn_to_session_maxsz (op_encode_hdr_maxsz + \
+ /* bctsa_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsa_dir */ + \
+ 1 /* bctsa_use_conn_in_rdma_mode */)
+#define decode_bind_conn_to_session_maxsz (op_decode_hdr_maxsz +\
+ /* bctsr_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsr_dir */ + \
+ 1 /* bctsr_use_conn_in_rdma_mode */)
#define encode_destroy_session_maxsz (op_encode_hdr_maxsz + 4)
#define decode_destroy_session_maxsz (op_decode_hdr_maxsz)
#define encode_sequence_maxsz(op_encode_hdr_maxsz + \
< at >< at > -714,6 +724,12 < at >< at > static int nfs4_stat_to_errno(int);
decode_putfh_maxsz + \
decode_secinfo_maxsz)
#if defined(CONFIG_NFS_V4_1)
+#define NFS4_enc_bind_conn_to_session_sz \
+(compound_encode_hdr_maxsz + \
+ encode_bind_conn_to_session_maxsz)
+#define NFS4_dec_bind_conn_to_session_sz \
+(compound_decode_hdr_maxsz + \
+ decode_bind_conn_to_session_maxsz)
#define NFS4_enc_exchange_id_sz \
(compound_encode_hdr_maxsz + \
encode_exchange_id_maxsz)
< at >< at > -1654,6 +1670,20 < at >< at > static void encode_secinfo(struct xdr_stream *xdr, const struct qstr *name, stru
#if defined(CONFIG_NFS_V4_1)
/* NFSv4.1 operations */
+static void encode_bind_conn_to_session(struct xdr_stream *xdr,
+ struct nfs4_session *session,
+ struct compound_hdr *hdr)
+{
+__be32 *p;
+
+encode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION,
+decode_bind_conn_to_session_maxsz, hdr);
+encode_opaque_fixed(xdr, session->sess_id.data, NFS4_MAX_SESSIONID_LEN);
+p = xdr_reserve_space(xdr, 8);
+*p++ = cpu_to_be32(NFS4_CDFC4_BACK_OR_BOTH);
+*p = 0;/* use_conn_in_rdma_mode = False */
+}
+
static void encode_exchange_id(struct xdr_stream *xdr,
struct nfs41_exchange_id_args *args,
struct compound_hdr *hdr)
< at >< at > -2614,6 +2644,22 < at >< at > static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
#if defined(CONFIG_NFS_V4_1)
/*
+ * BIND_CONN_TO_SESSION request
+ */
+static void nfs4_xdr_enc_bind_conn_to_session(struct rpc_rqst *req,
+struct xdr_stream *xdr,
+struct nfs_client *clp)
+{
+struct compound_hdr hdr = {
+.minorversion = clp->cl_mvops->minor_version,
+};
+
+encode_compound_hdr(xdr, req, &hdr);
+encode_bind_conn_to_session(xdr, clp->cl_session, &hdr);
+encode_nops(&hdr);
+}
+
+/*
* EXCHANGE_ID request
*/
static void nfs4_xdr_enc_exchange_id(struct rpc_rqst *req,
< at >< at > -5239,6 +5285,37 < at >< at > static int decode_sessionid(struct xdr_stream *xdr, struct nfs4_sessionid *sid)
return decode_opaque_fixed(xdr, sid->data, NFS4_MAX_SESSIONID_LEN);
}
+static int decode_bind_conn_to_session(struct xdr_stream *xdr,
+struct nfs41_bind_conn_to_session_res *res)
+{
+__be32 *p;
+int status;
+
+status = decode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION);
+if (!status)
+status = decode_sessionid(xdr, &res->session->sess_id);
+if (unlikely(status))
+return status;
+
+/* dir flags, rdma mode bool */
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+
+res->dir = be32_to_cpup(p++);
+if (res->dir == 0 || res->dir > NFS4_CDFS4_BOTH)
+return -EIO;
+if (be32_to_cpup(p) == 0)
+res->use_conn_in_rdma_mode = false;
+else
+res->use_conn_in_rdma_mode = true;
+
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
static int decode_create_session(struct xdr_stream *xdr,
struct nfs41_create_session_res *res)
{
< at >< at > -6521,6 +6598,22 < at >< at > out:
#if defined(CONFIG_NFS_V4_1)
/*
+ * Decode BIND_CONN_TO_SESSION response
+ */
+static int nfs4_xdr_dec_bind_conn_to_session(struct rpc_rqst *rqstp,
+struct xdr_stream *xdr,
+void *res)
+{
+struct compound_hdr hdr;
+int status;
+
+status = decode_compound_hdr(xdr, &hdr);
+if (!status)
+status = decode_bind_conn_to_session(xdr, res);
+return status;
+}
+
+/*
* Decode EXCHANGE_ID response
*/
static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp,
< at >< at > -7001,6 +7094,8 < at >< at > struct rpc_procinfonfs4_procedures[] = {
PROC(RELEASE_LOCKOWNER,enc_release_lockowner,dec_release_lockowner),
PROC(SECINFO,enc_secinfo,dec_secinfo),
#if defined(CONFIG_NFS_V4_1)
+PROC(BIND_CONN_TO_SESSION,
+enc_bind_conn_to_session, dec_bind_conn_to_session),
PROC(EXCHANGE_ID,enc_exchange_id,dec_exchange_id),
PROC(CREATE_SESSION,enc_create_session,dec_create_session),
PROC(DESTROY_SESSION,enc_destroy_session,dec_destroy_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..5b1a504 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -69,6 +69,10 < at >< at >
#define NFS4_CDFC4_FORE_OR_BOTH 0x3
#define NFS4_CDFC4_BACK_OR_BOTH 0x7
+#define NFS4_CDFS4_FORE 0x1
+#define NFS4_CDFS4_BACK 0x2
+#define NFS4_CDFS4_BOTH 0x3
+
#define NFS4_SET_TO_SERVER_TIME0
#define NFS4_SET_TO_CLIENT_TIME1
< at >< at > -582,6 +586,7 < at >< at > enum {
NFSPROC4_CLNT_SECINFO,
/* nfs41 */
+NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
NFSPROC4_CLNT_EXCHANGE_ID,
NFSPROC4_CLNT_CREATE_SESSION,
NFSPROC4_CLNT_DESTROY_SESSION,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..aa2a3bb 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -1115,6 +1115,12 < at >< at > struct nfs41_impl_id {
struct nfstime4date;
};
+struct nfs41_bind_conn_to_session_res {
+struct nfs4_session*session;
+u32dir;
+booluse_conn_in_rdma_mode;
+};
+
struct nfs41_exchange_id_res {
struct nfs_client*client;
u32flags;
</pre>Weston Andros Adamson2012-05-24T17:22:50[PATCH 0/4] NFSv4.1 add nfs_inode book keeping for mdsthreshold
http://comments.gmane.org/gmane.linux.nfs/50037
<pre>From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
Only resending NFSv4.1 add nfs_inode book keeping for mdsthreshold.
Responded to Trond's comments.
I have not tested the directio nor the memmap io cases. The regular file
i/o works as intended.
Andy Adamson (4):
NFSv4.1 mdsthreshold attribute xdr
NFSv4.1 cache mdsthreshold values on OPEN
NFSv4.1 add nfs_inode book keeping for mdsthreshold
NFSv4.1 test the mdsthreshold hint parameters
fs/nfs/direct.c | 2 +
fs/nfs/file.c | 1 +
fs/nfs/inode.c | 4 ++
fs/nfs/nfs4proc.c | 38 ++++++++++++--
fs/nfs/nfs4xdr.c | 125 ++++++++++++++++++++++++++++++++++++++++++++++-
fs/nfs/pnfs.c | 94 +++++++++++++++++++++++++++++++++++
fs/nfs/pnfs.h | 21 ++++++++
fs/nfs/read.c | 2 +
include/linux/nfs4.h | 7 +++
include/linux/nfs_fs.h | 4 ++
include/linux/nfs_xdr.h | 10 ++++
11 files changed, 301 insertions(+), 7 deletions(-)
</pre>andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org2012-05-24T17:13:23[PATCH 1/2] nfs4.1: add BIND_CONN_TO_SESSION operation
http://comments.gmane.org/gmane.linux.nfs/50030
<pre>This patch adds the BIND_CONN_TO_SESSION operation which is needed for
upcoming SP4_MACH_CRED work and useful for recovering from broken connections
without destroying the session.
Signed-off-by: Weston Andros Adamson <dros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfs/nfs4_fs.h | 1 +
fs/nfs/nfs4proc.c | 54 ++++++++++++++++++++++++++++
fs/nfs/nfs4xdr.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++
include/linux/nfs4.h | 5 +++
include/linux/nfs_xdr.h | 6 +++
5 files changed, 156 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index edeef71..ad90bc7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
< at >< at > -215,6 +215,7 < at >< at > struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
extern int nfs4_proc_setclientid_confirm(struct nfs_client *, struct nfs4_setclientid_res *arg, struct rpc_cred *);
extern int nfs4_proc_get_rootfh(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
+extern int nfs4_proc_bind_conn_to_session(struct nfs_client *);
extern int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred);
extern int nfs4_init_clientid(struct nfs_client *, struct rpc_cred *);
extern int nfs41_init_clientid(struct nfs_client *, struct rpc_cred *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78784e5..7a4b320 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5062,6 +5062,60 < at >< at > nfs41_same_server_scope(struct server_scope *a, struct server_scope *b)
}
/*
+ * nfs4_proc_bind_conn_to_session()
+ *
+ * The 4.1 client currently uses the same TCP connection for the
+ * fore and backchannel.
+ */
+int nfs4_proc_bind_conn_to_session(struct nfs_client *clp)
+{
+int status;
+struct nfs41_bind_conn_to_session_res res;
+struct rpc_message msg = {
+.rpc_proc =
+&nfs4_procedures[NFSPROC4_CLNT_BIND_CONN_TO_SESSION],
+.rpc_argp = clp,
+.rpc_resp = &res,
+};
+
+dprintk("--> %s\n", __func__);
+BUG_ON(clp == NULL);
+
+res.session = kzalloc(sizeof(struct nfs4_session), GFP_KERNEL);
+if (unlikely(res.session == NULL)) {
+status = -ENOMEM;
+goto out;
+}
+
+status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+if (status == 0) {
+if (memcmp(res.session->sess_id.data,
+ clp->cl_session->sess_id.data, NFS4_MAX_SESSIONID_LEN)) {
+dprintk("NFS: %s: Session ID mismatch\n", __func__);
+status = -EIO;
+goto out_session;
+}
+if (res.dir != CDFS4_BOTH) {
+dprintk("NFS: %s: Unexpected direction from server\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+if (res.use_conn_in_rdma_mode != 0) {
+dprintk("NFS: %s: Server returned RDMA mode = true\n",
+__func__);
+status = -EIO;
+goto out_session;
+}
+}
+out_session:
+kfree(res.session);
+out:
+dprintk("<-- %s status= %d\n", __func__, status);
+return status;
+}
+
+/*
* nfs4_proc_exchange_id()
*
* Since the clientid has expired, all compounds using sessions
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..37e09c8 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -321,6 +321,16 < at >< at > static int nfs4_stat_to_errno(int);
1 /* csr_flags */ + \
decode_channel_attrs_maxsz + \
decode_channel_attrs_maxsz)
+#define encode_bind_conn_to_session_maxsz (op_encode_hdr_maxsz + \
+ /* bctsa_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsa_dir */ + \
+ 1 /* bctsa_use_conn_in_rdma_mode */)
+#define decode_bind_conn_to_session_maxsz (op_decode_hdr_maxsz +\
+ /* bctsr_sessid */ \
+ XDR_QUADLEN(NFS4_MAX_SESSIONID_LEN) + \
+ 1 /* bctsr_dir */ + \
+ 1 /* bctsr_use_conn_in_rdma_mode */)
#define encode_destroy_session_maxsz (op_encode_hdr_maxsz + 4)
#define decode_destroy_session_maxsz (op_decode_hdr_maxsz)
#define encode_sequence_maxsz(op_encode_hdr_maxsz + \
< at >< at > -714,6 +724,12 < at >< at > static int nfs4_stat_to_errno(int);
decode_putfh_maxsz + \
decode_secinfo_maxsz)
#if defined(CONFIG_NFS_V4_1)
+#define NFS4_enc_bind_conn_to_session_sz \
+(compound_encode_hdr_maxsz + \
+ encode_bind_conn_to_session_maxsz)
+#define NFS4_dec_bind_conn_to_session_sz \
+(compound_decode_hdr_maxsz + \
+ decode_bind_conn_to_session_maxsz)
#define NFS4_enc_exchange_id_sz \
(compound_encode_hdr_maxsz + \
encode_exchange_id_maxsz)
< at >< at > -1654,6 +1670,20 < at >< at > static void encode_secinfo(struct xdr_stream *xdr, const struct qstr *name, stru
#if defined(CONFIG_NFS_V4_1)
/* NFSv4.1 operations */
+static void encode_bind_conn_to_session(struct xdr_stream *xdr,
+ struct nfs4_session *session,
+ struct compound_hdr *hdr)
+{
+__be32 *p;
+
+encode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION,
+decode_bind_conn_to_session_maxsz, hdr);
+encode_opaque_fixed(xdr, session->sess_id.data, NFS4_MAX_SESSIONID_LEN);
+p = xdr_reserve_space(xdr, 8);
+*p++ = cpu_to_be32(NFS4_CDFC4_BACK_OR_BOTH);
+*p = 0;/* use_conn_in_rdma_mode = False */
+}
+
static void encode_exchange_id(struct xdr_stream *xdr,
struct nfs41_exchange_id_args *args,
struct compound_hdr *hdr)
< at >< at > -2614,6 +2644,22 < at >< at > static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
#if defined(CONFIG_NFS_V4_1)
/*
+ * BIND_CONN_TO_SESSION request
+ */
+static void nfs4_xdr_enc_bind_conn_to_session(struct rpc_rqst *req,
+struct xdr_stream *xdr,
+struct nfs_client *clp)
+{
+struct compound_hdr hdr = {
+.minorversion = clp->cl_mvops->minor_version,
+};
+
+encode_compound_hdr(xdr, req, &hdr);
+encode_bind_conn_to_session(xdr, clp->cl_session, &hdr);
+encode_nops(&hdr);
+}
+
+/*
* EXCHANGE_ID request
*/
static void nfs4_xdr_enc_exchange_id(struct rpc_rqst *req,
< at >< at > -5239,6 +5285,32 < at >< at > static int decode_sessionid(struct xdr_stream *xdr, struct nfs4_sessionid *sid)
return decode_opaque_fixed(xdr, sid->data, NFS4_MAX_SESSIONID_LEN);
}
+static int decode_bind_conn_to_session(struct xdr_stream *xdr,
+struct nfs41_bind_conn_to_session_res *res)
+{
+__be32 *p;
+int status;
+
+status = decode_op_hdr(xdr, OP_BIND_CONN_TO_SESSION);
+if (!status)
+status = decode_sessionid(xdr, &res->session->sess_id);
+if (unlikely(status))
+return status;
+
+/* dir flags, rdma mode bool */
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+
+res->dir = be32_to_cpup(p++);
+res->use_conn_in_rdma_mode = be32_to_cpup(p);
+
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
static int decode_create_session(struct xdr_stream *xdr,
struct nfs41_create_session_res *res)
{
< at >< at > -6521,6 +6593,22 < at >< at > out:
#if defined(CONFIG_NFS_V4_1)
/*
+ * Decode BIND_CONN_TO_SESSION response
+ */
+static int nfs4_xdr_dec_bind_conn_to_session(struct rpc_rqst *rqstp,
+struct xdr_stream *xdr,
+void *res)
+{
+struct compound_hdr hdr;
+int status;
+
+status = decode_compound_hdr(xdr, &hdr);
+if (!status)
+status = decode_bind_conn_to_session(xdr, res);
+return status;
+}
+
+/*
* Decode EXCHANGE_ID response
*/
static int nfs4_xdr_dec_exchange_id(struct rpc_rqst *rqstp,
< at >< at > -7001,6 +7089,8 < at >< at > struct rpc_procinfonfs4_procedures[] = {
PROC(RELEASE_LOCKOWNER,enc_release_lockowner,dec_release_lockowner),
PROC(SECINFO,enc_secinfo,dec_secinfo),
#if defined(CONFIG_NFS_V4_1)
+PROC(BIND_CONN_TO_SESSION,
+enc_bind_conn_to_session, dec_bind_conn_to_session),
PROC(EXCHANGE_ID,enc_exchange_id,dec_exchange_id),
PROC(CREATE_SESSION,enc_create_session,dec_create_session),
PROC(DESTROY_SESSION,enc_destroy_session,dec_destroy_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..80fbbfc6 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -69,6 +69,10 < at >< at >
#define NFS4_CDFC4_FORE_OR_BOTH 0x3
#define NFS4_CDFC4_BACK_OR_BOTH 0x7
+#define CDFS4_FORE 0x1
+#define CDFS4_BACK 0x2
+#define CDFS4_BOTH 0x3
+
#define NFS4_SET_TO_SERVER_TIME0
#define NFS4_SET_TO_CLIENT_TIME1
< at >< at > -582,6 +586,7 < at >< at > enum {
NFSPROC4_CLNT_SECINFO,
/* nfs41 */
+NFSPROC4_CLNT_BIND_CONN_TO_SESSION,
NFSPROC4_CLNT_EXCHANGE_ID,
NFSPROC4_CLNT_CREATE_SESSION,
NFSPROC4_CLNT_DESTROY_SESSION,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..dd3bc2f 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -1115,6 +1115,12 < at >< at > struct nfs41_impl_id {
struct nfstime4date;
};
+struct nfs41_bind_conn_to_session_res {
+struct nfs4_session*session;
+u32dir;
+u32use_conn_in_rdma_mode;
+};
+
struct nfs41_exchange_id_res {
struct nfs_client*client;
u32flags;
</pre>Weston Andros Adamson2012-05-24T16:26:36Linux pNFS status meeting 05/24/12
http://comments.gmane.org/gmane.linux.nfs/50017
<pre>Time: 9:30 AM PST (12:30 PM EST)
The call in numbers are:
U.S. and Canada: (800) 451-8679
International dial-in number: (212) 729-5016
Israel Toll-Free: 1-809-462-557
Conference code: 564.272.1552
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Steve Dickson2012-05-24T10:56:34[PATCH v2] NFSd: set nfsd_serv to NULL after service destruction
http://comments.gmane.org/gmane.linux.nfs/50015
<pre>v2:
1) Set global nfsd_serv pointer to NULL only if no running threads left.
Otherwise we will get NULL pointer dereference.
Signed-off-by: Stanislav Kinsbursky <skinsbursky-bzQdu9zFT3WakBO8gow8eQ< at >public.gmane.org>
---
fs/nfsd/nfsctl.c | 8 ++------
fs/nfsd/nfsd.h | 11 +++++++++++
fs/nfsd/nfssvc.c | 24 ++++++++----------------
3 files changed, 21 insertions(+), 22 deletions(-)
diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
index c55298e..fa49cff 100644
--- a/fs/nfsd/nfsctl.c
+++ b/fs/nfsd/nfsctl.c
< at >< at > -673,9 +673,7 < at >< at > static ssize_t __write_ports_addfd(char *buf)
err = svc_addsock(nfsd_serv, fd, buf, SIMPLE_TRANSACTION_LIMIT);
if (err < 0) {
-if (nfsd_serv->sv_nrthreads == 1)
-svc_shutdown_net(nfsd_serv, net);
-svc_destroy(nfsd_serv);
+nfsd_destroy(net);
return err;
}
< at >< at > -744,9 +742,7 < at >< at > out_close:
svc_xprt_put(xprt);
}
out_err:
-if (nfsd_serv->sv_nrthreads == 1)
-svc_shutdown_net(nfsd_serv, net);
-svc_destroy(nfsd_serv);
+nfsd_destroy(net);
return err;
}
diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
index 1671429..1336a65 100644
--- a/fs/nfsd/nfsd.h
+++ b/fs/nfsd/nfsd.h
< at >< at > -73,6 +73,17 < at >< at > intnfsd_nrpools(void);
intnfsd_get_nrthreads(int n, int *);
intnfsd_set_nrthreads(int n, int *);
+static inline void nfsd_destroy(struct net *net)
+{
+int destroy = (nfsd_serv->sv_nrthreads == 1);
+
+if (destroy)
+svc_shutdown_net(nfsd_serv, net);
+svc_destroy(nfsd_serv);
+if (destroy)
+nfsd_serv = NULL;
+}
+
#if defined(CONFIG_NFSD_V2_ACL) || defined(CONFIG_NFSD_V3_ACL)
#ifdef CONFIG_NFSD_V2_ACL
extern struct svc_version nfsd_acl_version2;
diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
index ee709fc..240473c 100644
--- a/fs/nfsd/nfssvc.c
+++ b/fs/nfsd/nfssvc.c
< at >< at > -254,8 +254,6 < at >< at > static void nfsd_shutdown(void)
static void nfsd_last_thread(struct svc_serv *serv, struct net *net)
{
-/* When last nfsd thread exits we need to do some clean-up */
-nfsd_serv = NULL;
nfsd_shutdown();
svc_rpcb_cleanup(serv, net);
< at >< at > -332,6 +330,7 < at >< at > static int nfsd_get_default_max_blksize(void)
int nfsd_create_serv(void)
{
int error;
+struct net *net = current->nsproxy->net_ns;
WARN_ON(!mutex_is_locked(&nfsd_mutex));
if (nfsd_serv) {
< at >< at > -346,7 +345,7 < at >< at > int nfsd_create_serv(void)
if (nfsd_serv == NULL)
return -ENOMEM;
-error = svc_bind(nfsd_serv, current->nsproxy->net_ns);
+error = svc_bind(nfsd_serv, net);
if (error < 0) {
svc_destroy(nfsd_serv);
return error;
< at >< at > -427,11 +426,7 < at >< at > int nfsd_set_nrthreads(int n, int *nthreads)
if (err)
break;
}
-
-if (nfsd_serv->sv_nrthreads == 1)
-svc_shutdown_net(nfsd_serv, net);
-svc_destroy(nfsd_serv);
-
+nfsd_destroy(net);
return err;
}
< at >< at > -478,9 +473,7 < at >< at > out_shutdown:
if (error < 0 && !nfsd_up_before)
nfsd_shutdown();
out_destroy:
-if (nfsd_serv->sv_nrthreads == 1)
-svc_shutdown_net(nfsd_serv, net);
-svc_destroy(nfsd_serv);/* Release server */
+nfsd_destroy(net);/* Release server */
out:
mutex_unlock(&nfsd_mutex);
return error;
< at >< at > -563,12 +556,13 < at >< at > nfsd(void *vrqstp)
nfsdstats.th_cnt --;
out:
-if (rqstp->rq_server->sv_nrthreads == 1)
-svc_shutdown_net(rqstp->rq_server, &init_net);
+rqstp->rq_server = NULL;
/* Release the thread */
svc_exit_thread(rqstp);
+nfsd_destroy(&init_net);
+
/* Release module */
mutex_unlock(&nfsd_mutex);
module_put_and_exit(0);
< at >< at > -682,9 +676,7 < at >< at > int nfsd_pool_stats_release(struct inode *inode, struct file *file)
mutex_lock(&nfsd_mutex);
/* this function really, really should have been called svc_put() */
-if (nfsd_serv->sv_nrthreads == 1)
-svc_shutdown_net(nfsd_serv, net);
-svc_destroy(nfsd_serv);
+nfsd_destroy(net);
mutex_unlock(&nfsd_mutex);
return ret;
}
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Stanislav Kinsbursky2012-05-24T10:15:39Does our Kernel PNFSD-Server supports recurring layout_get with open_state_id
http://comments.gmane.org/gmane.linux.nfs/50011
<pre>
Benny hi
If I remember/understand correctly, there is a mode in the RFC errata
about the layout forgetful-model and a client sending a layout_get with
an open_state_id after he already had previous state (layouts) on the file.
As I understood this is an indication to the server that client has
"forgotten" all it's layouts on a file, and Server can assume their
return.
Is my understanding correct?
If Yes:
Did we implement the internal return of all layouts, if above
open_state_id is encountered?
I thought we did but I can't find this code.
Currently, I always set ROC so there is no leak. But theoretically
ROC does not have to be set. I'm doing some heavy lifting of
layout_return, and I want to make sure I have not missed a spot.
If I'm correct that it is needed, and it's missing:
My suggestion for now is that we always set ROC, disregarding FS so not to
leak layouts and therefor inode-refs, until such time that we implement it.
Thanks
Boaz
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Boaz Harrosh2012-05-24T00:16:40mainly setclientid/confirm cleanup
http://comments.gmane.org/gmane.linux.nfs/49996
<pre>While working on some locking fixes I ran into trouble understanding the
setclientid and setclientid_confirm code. So, here's some cleanup and
minor bugfixes.
The result is a lot simpler.
--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>J. Bruce Fields2012-05-23T21:40:30[PATCH 1/3] NFSv4.1: Fix session initialisation races
http://comments.gmane.org/gmane.linux.nfs/49985
<pre>Session initialisation is not complete until the lease manager
has run. We need to ensure that both nfs4_init_session and
nfs4_init_ds_session do so, and that they check for any resulting
errors in clp->cl_cons_state.
Only after this is done, can nfs4_ds_connect check the contents
of clp->cl_exchange_flags.
Signed-off-by: Trond Myklebust <Trond.Myklebust-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
Cc: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfs/client.c | 16 ---------
fs/nfs/internal.h | 3 +-
fs/nfs/nfs4filelayoutdev.c | 23 +------------
fs/nfs/nfs4proc.c | 77 ++++++++++++++++++++++++++++---------------
4 files changed, 52 insertions(+), 67 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 60f7e4e..78970a1 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
< at >< at > -593,22 +593,6 < at >< at > void nfs_mark_client_ready(struct nfs_client *clp, int state)
}
/*
- * With sessions, the client is not marked ready until after a
- * successful EXCHANGE_ID and CREATE_SESSION.
- *
- * Map errors cl_cons_state errors to EPROTONOSUPPORT to indicate
- * other versions of NFS can be tried.
- */
-int nfs4_check_client_ready(struct nfs_client *clp)
-{
-if (!nfs4_has_session(clp))
-return 0;
-if (clp->cl_cons_state < NFS_CS_READY)
-return -EPROTONOSUPPORT;
-return 0;
-}
-
-/*
* Initialise the timeout values for a connection
*/
static void nfs_init_timeout_values(struct rpc_timeout *to, int proto,
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index b777bda..00b66de 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
< at >< at > -168,7 +168,6 < at >< at > extern struct nfs_server *nfs_clone_server(struct nfs_server *,
struct nfs_fattr *,
rpc_authflavor_t);
extern void nfs_mark_client_ready(struct nfs_client *clp, int state);
-extern int nfs4_check_client_ready(struct nfs_client *clp);
extern struct nfs_client *nfs4_set_ds_client(struct nfs_client* mds_clp,
const struct sockaddr *ds_addr,
int ds_addrlen, int ds_proto);
< at >< at > -237,7 +236,7 < at >< at > extern const u32 nfs41_maxwrite_overhead;
extern struct rpc_procinfo nfs4_procedures[];
#endif
-extern int nfs4_init_ds_session(struct nfs_client *clp);
+extern int nfs4_init_ds_session(struct nfs_client *, unsigned long);
/* proc.c */
void nfs_close_context(struct nfs_open_context *ctx, int is_sync);
diff --git a/fs/nfs/nfs4filelayoutdev.c b/fs/nfs/nfs4filelayoutdev.c
index c9cff9a..0764c98 100644
--- a/fs/nfs/nfs4filelayoutdev.c
+++ b/fs/nfs/nfs4filelayoutdev.c
< at >< at > -176,28 +176,7 < at >< at > nfs4_ds_connect(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
goto out;
}
-if ((clp->cl_exchange_flags & EXCHGID4_FLAG_MASK_PNFS) != 0) {
-if (!is_ds_client(clp)) {
-status = -ENODEV;
-goto out_put;
-}
-ds->ds_clp = clp;
-dprintk("%s [existing] server=%s\n", __func__,
-ds->ds_remotestr);
-goto out;
-}
-
-/*
- * Do not set NFS_CS_CHECK_LEASE_TIME instead set the DS lease to
- * be equal to the MDS lease. Renewal is scheduled in create_session.
- */
-spin_lock(&mds_srv->nfs_client->cl_lock);
-clp->cl_lease_time = mds_srv->nfs_client->cl_lease_time;
-spin_unlock(&mds_srv->nfs_client->cl_lock);
-clp->cl_last_renewal = jiffies;
-
-/* New nfs_client */
-status = nfs4_init_ds_session(clp);
+status = nfs4_init_ds_session(clp, mds_srv->nfs_client->cl_lease_time);
if (status)
goto out_put;
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 99650aa..272c4ad 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
< at >< at > -5605,53 +5605,76 < at >< at > int nfs4_proc_destroy_session(struct nfs4_session *session)
return status;
}
+/*
+ * With sessions, the client is not marked ready until after a
+ * successful EXCHANGE_ID and CREATE_SESSION.
+ *
+ * Map errors cl_cons_state errors to EPROTONOSUPPORT to indicate
+ * other versions of NFS can be tried.
+ */
+static int nfs41_check_session_ready(struct nfs_client *clp)
+{
+int ret;
+
+ret = nfs4_client_recover_expired_lease(clp);
+if (ret)
+return ret;
+if (clp->cl_cons_state < NFS_CS_READY)
+return -EPROTONOSUPPORT;
+return 0;
+}
+
int nfs4_init_session(struct nfs_server *server)
{
struct nfs_client *clp = server->nfs_client;
struct nfs4_session *session;
unsigned int rsize, wsize;
-int ret;
if (!nfs4_has_session(clp))
return 0;
session = clp->cl_session;
-if (!test_and_clear_bit(NFS4_SESSION_INITING, &session->session_state))
-return 0;
+spin_lock(&clp->cl_lock);
+if (test_and_clear_bit(NFS4_SESSION_INITING, &session->session_state)) {
-rsize = server->rsize;
-if (rsize == 0)
-rsize = NFS_MAX_FILE_IO_SIZE;
-wsize = server->wsize;
-if (wsize == 0)
-wsize = NFS_MAX_FILE_IO_SIZE;
+rsize = server->rsize;
+if (rsize == 0)
+rsize = NFS_MAX_FILE_IO_SIZE;
+wsize = server->wsize;
+if (wsize == 0)
+wsize = NFS_MAX_FILE_IO_SIZE;
-session->fc_attrs.max_rqst_sz = wsize + nfs41_maxwrite_overhead;
-session->fc_attrs.max_resp_sz = rsize + nfs41_maxread_overhead;
+session->fc_attrs.max_rqst_sz = wsize + nfs41_maxwrite_overhead;
+session->fc_attrs.max_resp_sz = rsize + nfs41_maxread_overhead;
+}
+spin_unlock(&clp->cl_lock);
-ret = nfs4_recover_expired_lease(server);
-if (!ret)
-ret = nfs4_check_client_ready(clp);
-return ret;
+return nfs41_check_session_ready(clp);
}
-int nfs4_init_ds_session(struct nfs_client *clp)
+int nfs4_init_ds_session(struct nfs_client *clp, unsigned long lease_time)
{
struct nfs4_session *session = clp->cl_session;
int ret;
-if (!test_and_clear_bit(NFS4_SESSION_INITING, &session->session_state))
-return 0;
-
-ret = nfs4_client_recover_expired_lease(clp);
-if (!ret)
-/* Test for the DS role */
-if (!is_ds_client(clp))
-ret = -ENODEV;
-if (!ret)
-ret = nfs4_check_client_ready(clp);
-return ret;
+spin_lock(&clp->cl_lock);
+if (test_and_clear_bit(NFS4_SESSION_INITING, &session->session_state)) {
+/*
+ * Do not set NFS_CS_CHECK_LEASE_TIME instead set the
+ * DS lease to be equal to the MDS lease.
+ */
+clp->cl_lease_time = lease_time;
+clp->cl_last_renewal = jiffies;
+}
+spin_unlock(&clp->cl_lock);
+ret = nfs41_check_session_ready(clp);
+if (ret)
+return ret;
+/* Test for the DS role */
+if (!is_ds_client(clp))
+return -ENODEV;
+return 0;
}
EXPORT_SYMBOL_GPL(nfs4_init_ds_session);
</pre>Trond Myklebust2012-05-23T17:32:41[PATCH 1/4] NFSv4.1 mdsthreshold attribute xdr
http://comments.gmane.org/gmane.linux.nfs/49981
<pre>From: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
We only support one layout type per file system, so one threshold_item4 per
mdsthreshold4.
Signed-off-by: Andy Adamson <andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org>
---
fs/nfs/nfs4xdr.c | 125 ++++++++++++++++++++++++++++++++++++++++++++++-
include/linux/nfs4.h | 7 +++
include/linux/nfs_xdr.h | 10 ++++
3 files changed, 140 insertions(+), 2 deletions(-)
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index db040e9..db199f8 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
< at >< at > -99,9 +99,12 < at >< at > static int nfs4_stat_to_errno(int);
#define nfs4_path_maxsz(1 + ((3 + NFS4_MAXPATHLEN) >> 2))
#define nfs4_owner_maxsz(1 + XDR_QUADLEN(IDMAP_NAMESZ))
#define nfs4_group_maxsz(1 + XDR_QUADLEN(IDMAP_NAMESZ))
+/* We support only one layout type per file system */
+#define decode_mdsthreshold_maxsz (1 + 1 + nfs4_fattr_bitmap_maxsz + 1 + 8)
/* This is based on getfattr, which uses the most attributes: */
#define nfs4_fattr_value_maxsz(1 + (1 + 2 + 2 + 4 + 2 + 1 + 1 + 2 + 2 + \
-3 + 3 + 3 + nfs4_owner_maxsz + nfs4_group_maxsz))
+3 + 3 + 3 + nfs4_owner_maxsz + \
+nfs4_group_maxsz + decode_mdsthreshold_maxsz))
#define nfs4_fattr_maxsz(nfs4_fattr_bitmap_maxsz + \
nfs4_fattr_value_maxsz)
#define decode_getattr_maxsz (op_decode_hdr_maxsz + nfs4_fattr_maxsz)
< at >< at > -1170,6 +1173,16 < at >< at > static void encode_getfattr(struct xdr_stream *xdr, const u32* bitmask, struct c
bitmask[1] & nfs4_fattr_bitmap[1], hdr);
}
+static void encode_getfattr_open(struct xdr_stream *xdr, const u32 *bitmask,
+ struct compound_hdr *hdr)
+{
+encode_getattr_three(xdr,
+ bitmask[0] & nfs4_fattr_bitmap[0],
+ bitmask[1] & nfs4_fattr_bitmap[1],
+ bitmask[2] & FATTR4_WORD2_MDSTHRESHOLD,
+ hdr);
+}
+
static void encode_fsinfo(struct xdr_stream *xdr, const u32* bitmask, struct compound_hdr *hdr)
{
encode_getattr_three(xdr,
< at >< at > -2161,7 +2174,7 < at >< at > static void nfs4_xdr_enc_open(struct rpc_rqst *req, struct xdr_stream *xdr,
encode_putfh(xdr, args->fh, &hdr);
encode_open(xdr, args, &hdr);
encode_getfh(xdr, &hdr);
-encode_getfattr(xdr, args->bitmask, &hdr);
+encode_getfattr_open(xdr, args->bitmask, &hdr);
encode_nops(&hdr);
}
< at >< at > -4183,6 +4196,110 < at >< at > xdr_error:
return status;
}
+static int decode_threshold_hint(struct xdr_stream *xdr,
+ uint32_t *bitmap,
+ uint64_t *res,
+ uint32_t hint_bit)
+{
+__be32 *p;
+
+*res = 0;
+if (likely(bitmap[0] & hint_bit)) {
+p = xdr_inline_decode(xdr, 8);
+if (unlikely(!p))
+goto out_overflow;
+xdr_decode_hyper(p, res);
+}
+return 0;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
+static int decode_first_threshold_item4(struct xdr_stream *xdr,
+struct nfs4_threshold *res)
+{
+__be32 *p, *savep;
+uint32_t bitmap[3] = {0,}, attrlen;
+int status;
+
+/* layout type */
+p = xdr_inline_decode(xdr, 4);
+if (unlikely(!p)) {
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+res->l_type = be32_to_cpup(p);
+
+/* thi_hintset bitmap */
+status = decode_attr_bitmap(xdr, bitmap);
+if (status < 0)
+goto xdr_error;
+
+/* thi_hintlist length */
+status = decode_attr_length(xdr, &attrlen, &savep);
+if (status < 0)
+goto xdr_error;
+/* thi_hintlist */
+status = decode_threshold_hint(xdr, bitmap, &res->rd_sz, THRESHOLD_RD);
+if (status < 0)
+goto xdr_error;
+status = decode_threshold_hint(xdr, bitmap, &res->wr_sz, THRESHOLD_WR);
+if (status < 0)
+goto xdr_error;
+status = decode_threshold_hint(xdr, bitmap, &res->rd_io_sz,
+ THRESHOLD_RD_IO);
+if (status < 0)
+goto xdr_error;
+status = decode_threshold_hint(xdr, bitmap, &res->wr_io_sz,
+ THRESHOLD_WR_IO);
+if (status < 0)
+goto xdr_error;
+
+status = verify_attr_len(xdr, savep, attrlen);
+res->bm = bitmap[0];
+
+dprintk("%s bm=0x%x rd_sz=%llu wr_sz=%llu rd_io=%llu wr_io=%llu\n",
+ __func__, res->bm, res->rd_sz, res->wr_sz, res->rd_io_sz,
+res->wr_io_sz);
+xdr_error:
+dprintk("%s ret=%d!\n", __func__, status);
+return status;
+}
+
+/*
+ * Thresholds on pNFS direct I/O vrs MDS I/O
+ */
+static int decode_attr_mdsthreshold(struct xdr_stream *xdr,
+ uint32_t *bitmap,
+ struct nfs4_threshold *res)
+{
+__be32 *p;
+int status = 0;
+uint32_t num;
+
+if (unlikely(bitmap[2] & (FATTR4_WORD2_MDSTHRESHOLD - 1U)))
+return -EIO;
+if (likely(bitmap[2] & FATTR4_WORD2_MDSTHRESHOLD)) {
+p = xdr_inline_decode(xdr, 4);
+if (unlikely(!p))
+goto out_overflow;
+num = be32_to_cpup(p);
+if (num == 0)
+return 0;
+if (num > 1)
+printk(KERN_INFO "%s: Warning: Multiple pNFS layout "
+"drivers per filesystem not supported\n",
+__func__);
+
+status = decode_first_threshold_item4(xdr, res);
+}
+return status;
+out_overflow:
+print_overflow_msg(__func__, xdr);
+return -EIO;
+}
+
static int decode_getfattr_attrs(struct xdr_stream *xdr, uint32_t *bitmap,
struct nfs_fattr *fattr, struct nfs_fh *fh,
struct nfs4_fs_locations *fs_loc,
< at >< at > -4289,6 +4406,10 < at >< at > static int decode_getfattr_attrs(struct xdr_stream *xdr, uint32_t *bitmap,
goto xdr_error;
fattr->valid |= status;
+status = decode_attr_mdsthreshold(xdr, bitmap, fattr->mdsthreshold);
+if (status < 0)
+goto xdr_error;
+
xdr_error:
dprintk("%s: xdr returned %d\n", __func__, -status);
return status;
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 0987146..72b6bad 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
< at >< at > -526,6 +526,13 < at >< at > enum lock_type4 {
#define FATTR4_WORD1_MOUNTED_ON_FILEID (1UL << 23)
#define FATTR4_WORD1_FS_LAYOUT_TYPES (1UL << 30)
#define FATTR4_WORD2_LAYOUT_BLKSIZE (1UL << 1)
+#define FATTR4_WORD2_MDSTHRESHOLD (1UL << 4)
+
+/* MDS threshold bitmap bits */
+#define THRESHOLD_RD (1UL << 0)
+#define THRESHOLD_WR (1UL << 1)
+#define THRESHOLD_RD_IO (1UL << 2)
+#define THRESHOLD_WR_IO (1UL << 3)
#define NFSPROC4_NULL 0
#define NFSPROC4_COMPOUND 1
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 2e53a3f..5b8e42e 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
< at >< at > -35,6 +35,15 < at >< at > static inline int nfs_fsid_equal(const struct nfs_fsid *a, const struct nfs_fsid
return a->major == b->major && a->minor == b->minor;
}
+struct nfs4_threshold {
+__u32bm;
+__u32l_type;
+__u64rd_sz;
+__u64wr_sz;
+__u64rd_io_sz;
+__u64wr_io_sz;
+};
+
struct nfs_fattr {
unsigned intvalid;/* which fields are valid */
umode_tmode;
< at >< at > -67,6 +76,7 < at >< at > struct nfs_fattr {
unsigned longgencount;
struct nfs4_string*owner_name;
struct nfs4_string*group_name;
+struct nfs4_threshold*mdsthreshold;/* pNFS threshold hints */
};
#define NFS_ATTR_FATTR_TYPE(1U << 0)
</pre>andros-HgOvQuBEEgTQT0dZR+AlfA< at >public.gmane.org2012-05-23T09:02:34[PATCH] nfsidmap: Allow verbosity level to be set in the config file.
http://comments.gmane.org/gmane.linux.nfs/49970
<pre>To make it easier to enable ID mapping debugging, nfsidmap
should read /etc/idmap.conf to see if the verbosity level
is set, similar to what rpc.idmapd does
Signed-off-by: Steve Dickson <steved-H+wXaHxf7aLQT0dZR+AlfA< at >public.gmane.org>
---
utils/nfsidmap/nfsidmap.c | 19 ++++++++++++++++++-
1 files changed, 18 insertions(+), 1 deletions(-)
diff --git a/utils/nfsidmap/nfsidmap.c b/utils/nfsidmap/nfsidmap.c
index cf11551..86cddd1 100644
--- a/utils/nfsidmap/nfsidmap.c
+++ b/utils/nfsidmap/nfsidmap.c
< at >< at > -12,6 +12,7 < at >< at >
#include <unistd.h>
#include "xlog.h"
+#include "conffile.h"
int verbose = 0;
char *usage="Usage: %s [-v] [-c || [-u|-g|-r key] || [-t timeout] key desc]";
< at >< at > -26,12 +27,26 < at >< at > char *usage="Usage: %s [-v] [-c || [-u|-g|-r key] || [-t timeout] key desc]";
#define DEFAULT_KEYRING "id_resolver"
#endif
+#ifndef PATH_IDMAPDCONF
+#define PATH_IDMAPDCONF "/etc/idmapd.conf"
+#endif
+
static int keyring_clear(char *keyring);
#define UIDKEYS 0x1
#define GIDKEYS 0x2
/*
+ * Check to the config file for the verbosity level
+ */
+get_config_verbose(char *path)
+{
+conf_path = path;
+conf_init();
+return conf_get_num("General", "Verbosity", 0);
+}
+
+/*
* Find either a user or group id based on the name< at >domain string
*/
int id_lookup(char *name_at_domain, key_serial_t key, int type)
< at >< at > -266,7 +281,9 < at >< at > int main(int argc, char **argv)
break;
}
}
-
+if (!verbose) {
+verbose = get_config_verbose(PATH_IDMAPDCONF);
+}
if (keystr) {
rc = key_revoke(keystr, keymask);
return rc;
</pre>Steve Dickson2012-05-22T19:38:17Help with NFS over 10GbE performance - possible NFS client to TCP bottleneck
http://comments.gmane.org/gmane.linux.nfs/49960
<pre>Team,
I am working on a team implementing a configuration with an OEL kernel
(2.6.32-300.3.1.el6uek.x86_64) and kernel NFS accessing an NFS server
over 10GbE a Solaris 10. We are trying to resolve what appears to be a
bottleneck between the Linux kernel NFS client and the TCP stack.
Specifically, the TCP send queue on the Linux client is empty (save a
couple of bursts) when we are running write I/O from the file system,
the TCP receive queue on the Solaris 10 NFS server is empty, and the RPC
pending request queue on the Solaris 10 NFS server is zero. If we dial
the network to 1GbE we get a nice deep TCP send queue on the client,
which is the bottleneck I was hoping to get to with 10GbE. At this
point, we am pretty sure the S10 NFS server can run to at least 1000 MBPS.
So far, we have implemented the following Linux kernel tunes:
sunrpc.tcp_slot_table_entries = 128
net.core.rmem_default = 4194304
net.core.wmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_max = 4194304
net.ipv4.tcp_rmem = 4096 1048576 4194304
net.ipv4.tcp_wmem = 4096 1048576 4194304
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_syncookies = 1
net.core.netdev_max_backlog = 300000
In addition, we am running jumbo frames on the 10GbE NIC and we have
cpuspeed and irqbalance disabled (no noticeable changes when we did
this). The mount options on the client side are as follows:
192.168.44.51:/export/share on /export/share type nfs
(rw,nointr,bg,hard,rsize=1048576,wsize=1048576,proto=tcp,vers=3,addr=192.168.44.51)
In this configuration we get about 330 MBPS of write throughput with 16
pending stable (open with O_DIRECT) synchronous (no kernel aio in the
I/O application) writes. If we scale beyond 16 pending I/O response
time increases but throughput remains fixed. It feels like there is a
problem with getting more than 16 pending I/O out to TCP, but we can't
tell for sure based on our observations so far. We did notice that
tuning the wsize down to 32kB increased throughput to 400 MBPS, but we
could not identify the root cause of this change.
Please let us know if you have any suggestions for either diagnosing the
bottleneck more accurately or relieving the bottleneck. Thank you in
advance.
Sincerely,
Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA< at >public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
</pre>Jeff Wright2012-05-22T16:21:58Search EngineSearch the mailing list at Gmanequery
http://search.gmane.org/?group=$group=gmane.linux.nfs