Skip to Content
Last modified by Drunk Monkey on 2024-09-01 11:46
Hide last authors
Drunk Monkey 2.1 1 I spent the previous week celebrating the Christmas holiday with family and friends, and as a result, took a break from blogging. However, other than the New Year, I'm finished with holidays for a while, and eager to get back to blogging, and finishing off this series. Only handful of posts left to go. So, let's continue our discussion with ZFS administration on GNU/Linux, by discussing sharing datasets.
Drunk Monkey 1.2 2
Drunk Monkey 2.1 3 == Disclaimer ==
4
5 I have been trying to keep these ZFS posts as operating system agnostic as much as possible. Even though they have had a slant towards Linux kernels, you should be able to take much of this to BSD or any of the Solaris derivatives, such as OpenIndiana or Nexenta. With this post, however, it's going to be Linux-kernel specific, and even Ubuntu and Debian specific at that. The reason being is iSCSI support is not compiled into ZFS on Linux as of the writing of this post, but sharing via NFS and SMB is. Further, the implementation details for sharing via NFS and SMB will be specific to Debian and Ubuntu in this post. So, you may need to make adjustments if using Fedora, openSUSE, Gentoo, Arch, et cetera.
6
7 == Motivation ==
8
9 You are probably asking why you would want to use ZFS specific sharing of datasets rather than using the "tried and true" methods with standard software. The reason is simple. When the system boots up, and goes through its service initialization process (typically by executing the shell scripts found in /etc/init.d/), it has a method to the madness on which service starts first. Loosely speaking, filesystems are mounted first, networking is enabled, then services are started at last. Some of these are tied together, such as NFS exports, which requires the filesystem to be mounted, a firewall in place, networking started, and the NFS daemon running. But, what happens when the filesystem is not mounted? If the directory is still accessible, it will be exported via NFS, and applications could begin dumping data into the export. This could lead to all sorts of issues, such as data inconsistencies. As such, administrators have put checks into place, such as exporting only nested directories in the mount point, which would not be available if the filesystem fails to mount. These are clever hacks, but certainly not elegant.
10
11 When tying the export directly into the filesystem, you can solve this beautifully, which ZFS does. In the case of ZFS, you can share a specific dataset via NFS, for example. However, if the dataset does not mount, then the export will not be available to the application, and the NFS client will block. Because the network share is inherent to the filesystem, there is no concern for data inconsistencies, and no need for silly check hacks or scripts. As a result, ZFS from Oracle has the ability to share a dataset via NFS, SMB (CIFS or Samba) and iSCSI. ZFS on Linux only supports NFS and SMB currently, with iSCSI support on the way.
12
13 In each case, you still must install the necessary daemon software to make the share available. For example, if you wish to share a dataset via NFS, then you need to install the NFS server software, and it must be running. Then, all you need to do is flip the sharing NFS switch on the dataset, and it will be immediately available.
14
15 == Sharing via NFS ==
16
17 To share a dataset via NFS, you first need to make sure the NFS daemon is running. On Debian and Ubuntu, this is the "nfs-kernel-server" package. Further, with Debian and Ubuntu, the NFS daemon will not start unless there is an export in the /etc/exports file. So, you have two options: you can create a dummy export, only available to localhost, or you can edit the init script to start without checking for a current export. I prefer the former. Let's get that setup:
18
19 {{code language="bash session"}}$ sudo aptitude install -R nfs-kernel-server
20 $ echo '/mnt localhost(ro)' >> /etc/exports
21 $ sudo /etc/init.d/nfs-kernel-server start
22 $ showmount -e hostname.example.com
23 Export list for hostname.example.com:
24 /mnt localhost{{/code}}
25
26 With our NFS daemon running, we can now start sharing ZFS datasets. I'll assume already that you have created your dataset, it's mounted, and you're ready to start committing data to it. You'll notice in the zfs(8) manpage, that for the "sharenfs" property, it can be "on", "off" or "opts", where "opts" are valid NFS export options. So, if I wanted to share my "pool/srv" dataset, which is mounted to "/srv" to the 10.80.86.0/24 network, I could do something like:
27
28 {{code language="bash session"}}# zfs set sharenfs="rw=@10.80.86.0/24" pool/srv
29 # zfs share pool/srv
30 # showmount -e hostname.example.com
31 Export list for hostname.example.com:
32 /srv 10.80.86.0/24
33 /mnt localhost{{/code}}
34
35 If you want your ZFS datasets to be shared on boot, then you need to install the /etc/default/zfs config file. If using the Ubuntu PPA, this will be installed by default for you. If compiling from source, this will not be provided. Here are the contents of that file. I've added emphasis to the two lines that should be modified for persistence across boots, if you want to enable sharing via NFS. Default is 'no':
36
37 {{code language="bash session"}}$ cat /etc/default/zfs
38 # /etc/default/zfs
39 #
40 # Instead of changing these default ZFS options, Debian systems should install
41 # the zfs-mount package, and Ubuntu systems should install the zfs-mountall
42 # package. The debian-zfs and ubuntu-zfs metapackages ensure a correct system
43 # configuration.
44 #
45 # If the system runs parallel init jobs, like upstart or systemd, then the
46 # `zfs mount -a` command races in a way that causes sporadic mount failures.
47
48 # Automatically run `zfs mount -a` at system start. Disabled by default.
49 ZFS_MOUNT='yes'
50 ZFS_UNMOUNT='no'
51
52 # Automatically run `zfs share -a` at system start. Disabled by default.
53 # Requires nfsd and/or smbd. Incompletely implemented for Linux.
54 ZFS_SHARE='yes'
55 ZFS_UNSHARE='no'{{/code}}
56
57 As mentioned in the comments, running a parallel init system creates problems for ZFS. This is something I recently banged my head against, as my /var/log/ and /var/cache/ datasets were not mounting on boot. To fix the problem, and run a serialized boot, thus ensuring that everything gets executed in the proper order, you need to touch a file:
58
59 {{code language="bash session"}}# touch /etc/init.d/.legacy-bootordering{{/code}}
60
61 This will add time to your bootup, but given the fact that my system is up months at a time, I'm not worried about the extra 5 seconds this puts on my boot. This is documented in the /etc/init.d/rc script, setting the "CONCURRENCY=none" variable.
62
63 You should now be able to mount the NFS export from an NFS client:
64
65 {{code language="bash session"}}(client)# mount -t nfs hostname.example.com:/srv /mnt{{/code}}
66
67 == Sharing via SMB ==
68
69 Currently, SMB integration is not working 100%. See bug [[#1170 I reported on Github>>url:https://web.archive.org/web/20210430213417/https://github.com/zfsonlinux/zfs/issues/1170]]. However, when things get working, this will likely be the way.
70
71 As with NFS, to share a ZFS dataset via SMB/CIFS, you need to have the daemon installed and running. Recently, the [[Samba development team released Samba version 4>>url:https://web.archive.org/web/20210430213417/https://www.samba.org/samba/news/releases/4.0.0.html]]. This release gives the Free Software world a Free Software implementation of Active Directory running on GNU/Linux systems, SMB 2.1 file sharing support, clustered file servers, and much more. Currently, Debian testing has the beta 2 packages. Debian experimental has the stable release, and it may make its way up the chain for the next stable release. One can hope. Samba v4 is not needed to share ZFS datasets via SMB/CIFS, but it's worth mentioning. We'll stick with version 3 of the Samba packages, until version 4 stabilizes.
72
73 {{code language="bash session"}}# aptitude install -R aptitude install samba samba-client samba-doc samba-tools samba-doc-pdf
74 # ps -ef | grep smb
75 root 22413 1 0 09:05 ? 00:00:00 /usr/sbin/smbd -D
76 root 22423 22413 0 09:05 ? 00:00:00 /usr/sbin/smbd -D
77 root 22451 21308 0 09:06 pts/1 00:00:00 grep smb{{/code}}
78
79 At this point, all we need to do is share the dataset, and verify that it's been shared. It is also worth noting that Microsoft Windows machines are not case sensitive, as things are in Unix. As such, if you are in a heterogenious environment, it may be worth disabling case sensitivity on the ZFS dataset. Setting this value can only be done on creation time. So, you may wish to issue the following when creating that dataset:
80
81 {{code language="bash session"}}# zfs create -o casesensitivity=mixed pool/srv{{/code}}
82
83 Now you can continue with configuring the rest of the dataset:
84
85 {{code language="bash session"}}# zfs set sharesmb=on pool/srv
86 # zfs share pool/srv
87 # smbclient -U guest -N -L localhost
88 Domain=[WORKGROUP] OS=[Unix] Server=[Samba 3.6.6]
89
90 Sharename Type Comment
91 --------- ---- -------
92 print$ Disk Printer Drivers
93 sysvol Disk
94 netlogon Disk
95 IPC$ IPC IPC Service (eightyeight server)
96 Canon-imageRunner-3300 Printer Canon imageRunner 3300
97 HP-Color-LaserJet-3600 Printer HP Color LaserJet 3600
98 salesprinter Printer Canon ImageClass MF7460
99 pool_srv Disk Comment: /srv
100 Domain=[WORKGROUP] OS=[Unix] Server=[Samba 3.6.6]
101
102 Server Comment
103 --------- -------
104 EIGHTYEIGHT eightyeight server
105
106 Workgroup Master
107 --------- -------
108 WORKGROUP EIGHTYEIGHT{{/code}}
109
110 You can see that in this environment (my workstation hostname is 'eightyeight'), there are some printers being shared, and a couple disks. I've emphasized the disk that we are sharing in my output, to verify that it is working correctly. So, we should be able to mount that share as a CIFS mount, and access the data:
111
112 {{code language="bash session"}}# aptitude install -R cifs-utils
113 # mount -t cifs -o username=USERNAME //localhost/srv /mnt
114 # ls /mnt
115 foo{{/code}}
116
117 == Sharing via iSCSI ==
118
119 Unfortunately, sharing ZFS datasets via iSCSI is not yet supported with ZFS on Linux. However, it is available in the Illumos source code upstream, and work is being done to get it working in GNU/Linux. As with SMB and NFS, you will need the iSCSI daemon installed and running. When support is enabled, I'll finish writing up this post on demonstrating how you can access iSCSI targets that are ZFS datasets. In the meantime, you would do something like the following:
120
121 {{code language="bash session"}}# aptitude install -R openiscsi
122 # zfs set shareiscsi=on pool/srv{{/code}}
123
124 At which point, from the iSCSI client, you would access the target, format it, mount it, and start working with the data..
125
126
Drunk Monkey 1.3 127 ----
Drunk Monkey 1.2 128
129 (% style="text-align: center;" %)
Drunk Monkey 1.3 130 Posted by Aaron Toponce on Monday, December 31, 2012, at 6:00 am.
131 Filed under [[Debian>>url:https://web.archive.org/web/20210430213417/https://pthree.org/category/debian/]], [[Linux>>url:https://web.archive.org/web/20210430213417/https://pthree.org/category/linux/]], [[Ubuntu>>url:https://web.archive.org/web/20210430213417/https://pthree.org/category/ubuntu/]], [[ZFS>>url:https://web.archive.org/web/20210430213417/https://pthree.org/category/zfs/]].
132 Follow any responses to this post with its [[comments RSS>>url:https://web.archive.org/web/20210430213417/https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/feed/]] feed.
133 You can [[post a comment>>url:https://web.archive.org/web/20210430213417/https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/#respond]] or [[trackback>>url:https://web.archive.org/web/20210430213417/https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/trackback/]] from your blog.
134 For IM, Email or Microblogs, here is the [[Shortlink>>url:https://web.archive.org/web/20210430213417/https://pthree.org/?p=2943]].
135
Drunk Monkey 1.2 136 ----
Drunk Monkey 1.3 137
Drunk Monkey 1.2 138 {{box title="**Archived From:**"}}
Drunk Monkey 1.3 139 [[https:~~/~~/web.archive.org/web/20210430213417/https:~~/~~/pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/>>https://web.archive.org/web/20210430213417/https://pthree.org/2012/12/31/zfs-administration-part-xv-iscsi-nfs-and-samba/]]
Drunk Monkey 1.2 140 {{/box}}