21 Sep
2015
21 Sep
'15
7:54 p.m.
Hi,
I am looking for a "slimmer" NFS when I have a file server with a lkot
of space - while clients with not much.
I would like to export parts of the server file systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
The file server needs to provide read access to files and be able to
write what's given to it from the client.
UIDs etc. could be send through, the permission check would happen at
the client side.
I guess that's all needed and would be faster and less complex than NFS.
Does this kind of network file system exist?
Any other ideas for very fast access on file system level over the net welcome.
Regards
Peter
21 Sep
21 Sep
9:06 p.m.
On Mon, 21 Sep 2015 07:54:55 PM Peter Ross wrote:
I would like to export parts of the server file
systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Isn't this basically NFSv4 delegations? If you're exporting it to just a
single client then I'd expect no other client can interfere with them.
https://www.ietf.org/rfc/rfc3530.txt
1.4.6. Client Caching and Delegation
[...]
The major addition to NFS version 4 in the area of caching is the
ability of the server to delegate certain responsibilities to the
client. When the server grants a delegation for a file to a client,
the client is guaranteed certain semantics with respect to the
sharing of that file with other clients. At OPEN, the server may
provide the client either a read or write delegation for the file.
If the client is granted a read delegation, it is assured that no
other client has the ability to write to the file for the duration of
the delegation. If the client is granted a write delegation, the
client is assured that no other client has read or write access to
the file.
Delegations can be recalled by the server. If another client
requests access to the file in such a way that the access conflicts
with the granted delegation, the server is able to notify the initial
client and recall the delegation. This requires that a callback path
exist between the server and client. If this callback path does not
exist, then delegations can not be granted. The essence of a
delegation is that it allows the client to locally service operations
such as OPEN, CLOSE, LOCK, LOCKU, READ, WRITE without immediate
interaction with the server.
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
22 Sep
22 Sep
9:41 a.m.
Chris Samuel wrote:
On Mon, 21 Sep 2015 07:54:55 PM Peter Ross wrote:
Yes, that sounds like it.
I have to investigate how to configure this behaviour. I could not find it
in the nfs(5) manpage of CentOS6 servers.
Yes, Russell, I thought of iSCSI but in this case I want to make use of the
server filesystem.
Regards
Peter
On Mon, Sep 21, 2015 at 9:06 PM, Chris Samuel <chris(a)csamuel.org> wrote:
I would like to export parts of the server file
systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Isn't this basically NFSv4 delegations? If you're exporting it to just a
single client then I'd expect no other client can interfere with them.
https://www.ietf.org/rfc/rfc3530.txt
1.4.6. Client Caching and Delegation
[...]
The major addition to NFS version 4 in the area of caching is the
ability of the server to delegate certain responsibilities to the
client. When the server grants a delegation for a file to a client,
the client is guaranteed certain semantics with respect to the
sharing of that file with other clients. At OPEN, the server may
provide the client either a read or write delegation for the file.
If the client is granted a read delegation, it is assured that no
other client has the ability to write to the file for the duration of
the delegation. If the client is granted a write delegation, the
client is assured that no other client has read or write access to
the file.
Delegations can be recalled by the server. If another client
requests access to the file in such a way that the access conflicts
with the granted delegation, the server is able to notify the initial
client and recall the delegation. This requires that a callback path
exist between the server and client. If this callback path does not
exist, then delegations can not be granted. The essence of a
delegation is that it allows the client to locally service operations
such as OPEN, CLOSE, LOCK, LOCKU, READ, WRITE without immediate
interaction with the server. On Mon, 21 Sep 2015 07:54:55 PM Peter Ross wrote:
>
> --
> Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
>
> _______________________________________________
> luv-main mailing list
> luv-main(a)luv.asn.au
> http://lists.luv.asn.au/listinfo/luv-main
>
I would like to export parts of the server file
systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Isn't this basically NFSv4 delegations? If you're exporting it to just a
single client then I'd expect no other client can interfere with them.
https://www.ietf.org/rfc/rfc3530.txt
1.4.6. Client Caching and Delegation
[...]
The major addition to NFS version 4 in the area of caching is the
ability of the server to delegate certain responsibilities to the
client. When the server grants a delegation for a file to a client,
the client is guaranteed certain semantics with respect to the
sharing of that file with other clients. At OPEN, the server may
provide the client either a read or write delegation for the file.
If the client is granted a read delegation, it is assured that no
other client has the ability to write to the file for the duration of
the delegation. If the client is granted a write delegation, the
client is assured that no other client has read or write access to
the file.
Delegations can be recalled by the server. If another client
requests access to the file in such a way that the access conflicts
with the granted delegation, the server is able to notify the initial
client and recall the delegation. This requires that a callback path
exist between the server and client. If this callback path does not
exist, then delegations can not be granted. The essence of a
delegation is that it allows the client to locally service operations
such as OPEN, CLOSE, LOCK, LOCKU, READ, WRITE without immediate
interaction with the server.
10:09 a.m.
I have to investigate how to configure this [NFSv4 delegation]
behaviour.
For FreeBSD I could find this in the nfsv4(4) manpage:
If the file systems you are exporting are only being accessed via NFSv4
there are a couple of sysctl(8) variables that you can change, which
might improve performance.
vfs.nfsd.issue_delegations
when set non-zero, allows the server to issue Open Delegations
to
clients. These delegations permit the client to manipulate the
file locally on the client. Unfortunately, at this time,
client
use of delegations is limited, so performance gains may not be
observed. This can only be enabled when the file systems being
exported to NFSv4 clients are not being accessed locally on the
server and, if being accessed via NFS Version 2 or 3 clients,
these clients cannot be using the NLM.
vfs.nfsd.enable_locallocks
can be set to 0 to disable acquisition of local byte range
locks.
Disabling local locking can only be done if neither local
accesses to the exported file systems nor the NLM is operating
on
them.
CLIENT MOUNTS
If the NFSv4 server that is being mounted on supports delegations, you
can start the nfscbd(8) daemon to handle client side callbacks. This
will occur if
nfsuserd_enable="YES"
nfscbd_enable="YES"
are set in rc.conf(5).
I looked around on CentOS 7 - I could not find similar functionality.
That is where I would like to have it at the moment..
Regards
Peter
On Tue, Sep 22, 2015 at 9:41 AM, Peter Ross <petrosssit(a)gmail.com> wrote:
Chris Samuel wrote:
On Mon, 21 Sep 2015 07:54:55 PM Peter Ross
wrote:
Yes, that sounds like it.
I have to investigate how to configure this behaviour. I could not find it
in the nfs(5) manpage of CentOS6 servers.
Yes, Russell, I thought of iSCSI but in this case I want to make use of
the server filesystem.
Regards
Peter
On Mon, Sep 21, 2015 at 9:06 PM, Chris Samuel <chris(a)csamuel.org> wrote:
I would like to export parts of the server file
systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Isn't this basically NFSv4 delegations? If you're exporting it to just a
single client then I'd expect no other client can interfere with them.
https://www.ietf.org/rfc/rfc3530.txt
1.4.6. Client Caching and Delegation
[...]
The major addition to NFS version 4 in the area of caching is the
ability of the server to delegate certain responsibilities to the
client. When the server grants a delegation for a file to a client,
the client is guaranteed certain semantics with respect to the
sharing of that file with other clients. At OPEN, the server may
provide the client either a read or write delegation for the file.
If the client is granted a read delegation, it is assured that no
other client has the ability to write to the file for the duration of
the delegation. If the client is granted a write delegation, the
client is assured that no other client has read or write access to
the file.
Delegations can be recalled by the server. If another client
requests access to the file in such a way that the access conflicts
with the granted delegation, the server is able to notify the initial
client and recall the delegation. This requires that a callback path
exist between the server and client. If this callback path does not
exist, then delegations can not be granted. The essence of a
delegation is that it allows the client to locally service operations
such as OPEN, CLOSE, LOCK, LOCKU, READ, WRITE without immediate
interaction with the server. On Mon, 21 Sep 2015 07:54:55 PM Peter Ross
wrote:
>
> --
> Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
>
> _______________________________________________
> luv-main mailing list
> luv-main(a)luv.asn.au
> http://lists.luv.asn.au/listinfo/luv-main
>
I would like to export parts of the server file
systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Isn't this basically NFSv4 delegations? If you're exporting it to just a
single client then I'd expect no other client can interfere with them.
https://www.ietf.org/rfc/rfc3530.txt
1.4.6. Client Caching and Delegation
[...]
The major addition to NFS version 4 in the area of caching is the
ability of the server to delegate certain responsibilities to the
client. When the server grants a delegation for a file to a client,
the client is guaranteed certain semantics with respect to the
sharing of that file with other clients. At OPEN, the server may
provide the client either a read or write delegation for the file.
If the client is granted a read delegation, it is assured that no
other client has the ability to write to the file for the duration of
the delegation. If the client is granted a write delegation, the
client is assured that no other client has read or write access to
the file.
Delegations can be recalled by the server. If another client
requests access to the file in such a way that the access conflicts
with the granted delegation, the server is able to notify the initial
client and recall the delegation. This requires that a callback path
exist between the server and client. If this callback path does not
exist, then delegations can not be granted. The essence of a
delegation is that it allows the client to locally service operations
such as OPEN, CLOSE, LOCK, LOCKU, READ, WRITE without immediate
interaction with the server.
10:38 a.m.
Hi Peter,
On 22/09/15 10:09, Peter Ross wrote:
I looked around on CentOS 7 - I could not find similar
functionality.
According to:
http://www.citi.umich.edu/projects/nfsv4/linux/features.html
delegation support was added to the NFSv4 client in 2.6.9-rc1 and to the
NFS server in 2.6.10, so my suspicion is that it's enabled by default.
The sysctl you are looking for (on RHEL6 at least) is:
[root@snowy-m ~]# sysctl fs.leases-enable
fs.leases-enable = 1
So yes, it should be enabled by default in RHEL6 (haven't got any RHEL7
boxes to check).
All the best,
Chris
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
8:34 p.m.
Hi Chris,
On Tue, Sep 22, 2015 at 10:38 AM, Chris Samuel <chris(a)csamuel.org> wrote:
Hi Peter,
On 22/09/15 10:09, Peter Ross wrote:
I am afraid the leases are not the same.
I came across leases first when looking for a NFSv3 extention integrated in
FreeBSD, NQNFS (not quite NFS).
http://docs.freebsd.org/44doc/papers/nqnfs.html
These leases expire after a short period so have to be renewed regularly to
"keep" the file.
In NFSv4 a callback server receives a revocation of the exclusive file
access instead.The callback server, nfs4cbd, is needed according to the
FreeBSD implementation, and Oracle's (
http://docs.oracle.com/cd/E19253-01/816-4555/rfsrefer-140/index.html)
I cannot find any reference on Linux for it, though.
Trent, NFS is called Nightmare File System for a reason. My experience
started with crashing Nextstep machines with block sizes of 8192 (when
trying to connect them with Solaris machines), and it went downhill from
there.
NFSv4 is a step forward, I hope.. I did not touch NFS for ca. 10 years, I
think. I did not miss it much;-)
Regards
Peter
I looked around on CentOS 7 - I could not find
similar functionality.
According to:
http://www.citi.umich.edu/projects/nfsv4/linux/features.html
delegation support was added to the NFSv4 client in 2.6.9-rc1 and to the
NFS server in 2.6.10, so my suspicion is that it's enabled by default.
The sysctl you are looking for (on RHEL6 at least) is:
[root@snowy-m ~]# sysctl fs.leases-enable
fs.leases-enable = 1
So yes, it should be enabled by default in RHEL6 (haven't got any RHEL7
boxes to check).
9:08 p.m.
On Tue, Sep 22, 2015 at 8:34 PM, Peter Ross <petrosssit(a)gmail.com> wrote:
Hi Chris,
On Tue, Sep 22, 2015 at 10:38 AM, Chris Samuel <chris(a)csamuel.org> wrote:
Hi Peter,
I have to correct myself. You are right, it is part of Linux NFSv4
delegation implementation.
http://wiki.linux-nfs.org/wiki/index.php/Cluster_Coherent_NFSv4_and_Delegat…
How the recall gets transmitted is still a mystery to me. If there is no
daemon, it must be "inbound" using the TCP port 2049? No, that would cause
real trouble when interacting with other NFS servers. Or??
The issues on the page below scare me a bit, especially
http://wiki.linux-nfs.org/wiki/index.php/Server_4.0_and_4.1_issues#Callback…
May all not really important for "my case" but in general it looks a bit
like a shortcut.
I still think about an alternative way for me. It's all too complicated.
Imagine an asynchronic connection to the server. The server does not have
any leases, callbacks etc. It does not lock on its end because I don't want
to have it changed by anything else.
All what it has to do is supplying data, as from a local disk, and return
"data written".
So, a journal written for another filesystem, e.g. ext4, could be used to
provide the local cache of changes.
Is there a "generic" journal which could be used for this?
Would it be feasible to implement such a network filesystem? Why doesn't it
exist? Or does it?
Regards
Peter
On 22/09/15 10:09, Peter Ross wrote:
I looked around on CentOS 7 - I could not find
similar functionality.
According to:
http://www.citi.umich.edu/projects/nfsv4/linux/features.html
delegation support was added to the NFSv4 client in 2.6.9-rc1 and to the
NFS server in 2.6.10, so my suspicion is that it's enabled by default.
The sysctl you are looking for (on RHEL6 at least) is:
[root@snowy-m ~]# sysctl fs.leases-enable
fs.leases-enable = 1
10:42 p.m.
On Tue, 22 Sep 2015 09:08:01 PM Peter Ross wrote:
I have to correct myself. You are right, it is part of
Linux NFSv4
delegation implementation.
http://wiki.linux-nfs.org/wiki/index.php/Cluster_Coherent_NFSv4_and_Delegati
ons
How the recall gets transmitted is still a mystery to me. If there is no
daemon, it must be "inbound" using the TCP port 2049? No, that would cause
real trouble when interacting with other NFS servers. Or??
It's set via a kernel module parameter I believe - see:
http://comments.gmane.org/gmane.linux.nfsv4/11560
Sorry for brevity, I've developed a nasty cough and am heading for some rest
now.. :-/
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
11:41 p.m.
On Tue, Sep 22, 2015 at 10:42 PM, Chris Samuel <chris(a)csamuel.org> wrote:
On Tue, 22 Sep 2015 09:08:01 PM Peter Ross wrote:
Thanks for the enlightment, and all the best
Peter
How the recall gets transmitted is still a
mystery to me. If there is no
daemon, it must be "inbound" using the TCP port 2049? No, that would
cause
real trouble when interacting with other NFS
servers. Or??
It's set via a kernel module parameter I believe - see:
http://comments.gmane.org/gmane.linux.nfsv4/11560
Sorry for brevity, I've developed a nasty cough and am heading for some
rest
now.. :-/
11:51 p.m.
On Tue, Sep 22, 2015 at 9:08 PM, Peter Ross <petrosssit(a)gmail.com> wrote:
Is there a "generic" journal which could be
used for this?
BTW: From FreeBSD I know gjournal, and thanks to the GEOM framework, it is
easy to stack:
https://www.freebsd.org/doc/en/books/handbook/geom-gjournal.html
https://www.freebsd.org/cgi/man.cgi?query=gjournal&sektion=8&apropo…
In Linuxland seems to be JBD but is it that easy to apply or only through
extensive programming?
Regards
Peter
P.S. Sorry about my enlightment spelling. I got enlightened;-)
21 Sep
21 Sep
10:12 p.m.
On Mon, 21 Sep 2015 07:54:55 PM Peter Ross wrote:
I am looking for a "slimmer" NFS when I have
a file server with a lkot
of space - while clients with not much.
I would like to export parts of the server file systems but do not
need to share them between the clients. So I don't need all the
overhead of leases that expire, server-side locking etc.
Instead it would be good enough to do locking on the client side and
write a journal which can be send to the file server.
Have you considered iSCSI or one of the other NBD options?
--
My Main Blog http://etbe.coker.com.au/
My Documents Blog http://doc.coker.com.au/
22 Sep
22 Sep
12:20 p.m.
Peter Ross <petrosssit(a)gmail.com> writes:
Instead it would be good enough to do locking on the
client side and
write a journal which can be send to the file server.
Does locking *ever* work properly on a network filesystem?
On sshfs (SFTP) it's not implemented.
On AFS it's "advisory", i.e. clients can ignore it.
On Samba3/4 I usually have to set it somewhere between "off" and
"more off" to appease MS Office (& LO).
What I've seen of rpc.statd (NFSv3) looks pretty hinkey, too.
10:39 p.m.
On Tue, 22 Sep 2015 12:20:16 PM Trent W. Buck wrote:
Does locking *ever* work properly on a network
filesystem?
I believe both GPFS and Lustre work well (GPFS certainly does it as we use it
in production at work).
https://en.wikipedia.org/wiki/IBM_General_Parallel_File_System#Architecture
https://en.wikipedia.org/wiki/Lustre_(file_system)#Locking
GPFS is proprietary of course, and Lustre is open source (kind-of-sort-of
premiumware with a bunch of people selling appliances and consultancy).
Neither could be characterised as simple..
--
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC
2741
days inactive
2742
days old
12 comments
4 participants
participants (4)
-
Chris Samuel -
Peter Ross -
Russell Coker -
trentbuck＠gmail.com