Minimal but useful Linux userland, COW filesystems footprint
Hi, I want to try Linux Containers (LVC 1.0 is out) and BTRFS (in a way I use FreeBSD and Jails and ZFS). I want to use filesystem cloning (at the moment under FreeBSD "cd /usr/src; make buildworld; make distribution; zfs snapshot; zfs clone") I also want to have a small but useful userland.. I am thinking of a "minimalist" Debian installation but other ideas are welcome too. Do you have good advice to achieve this? (TW: I am not a huge Busybox fan, the tools always seem to miss the one option I want to use.. so, no scaled down tools, just a good set of "standard tools") I do not know about the memory footstep and minimal requirements of BTRFS and ZFS under Linux, comments and advice is welcome as well. Thanks Peter
Petros <Petros.Listig@fdrive.com.au> wrote:
I also want to have a small but useful userland.. I am thinking of a "minimalist" Debian installation but other ideas are welcome too.
I would suggest finding out what users of lxc are running. If you want a minimal Debian distribution, have a look at the installation guide (there's an appendix that covers installation using Debootstrap). Debootstrap will give you a very minimal installation to which you can then add a kernel and make all necessary configuration changes, e.g., to /etc/fstab. You'll need to enter a chroot environment and bind mount some virtual file systems such as /proc. I've done all of this successfully but I can't remember all of the steps.
On Tue, 18 Feb 2014, Petros <Petros.Listig@fdrive.com.au> wrote:
I want to try Linux Containers (LVC 1.0 is out) and BTRFS (in a way I use FreeBSD and Jails and ZFS).
I want to use filesystem cloning (at the moment under FreeBSD "cd /usr/src; make buildworld; make distribution; zfs snapshot; zfs clone")
I also want to have a small but useful userland.. I am thinking of a "minimalist" Debian installation but other ideas are welcome too.
Do you have good advice to achieve this? (TW: I am not a huge Busybox fan, the tools always seem to miss the one option I want to use.. so, no scaled down tools, just a good set of "standard tools")
I do not know about the memory footstep and minimal requirements of BTRFS and ZFS under Linux, comments and advice is welcome as well.
ZFS manages kernel memory differently to every filesystem driver that was written for Linux. In my tests a lightly loaded Samba server with 4G of RAM regularly had problems with kernel OOM, I upgraded it to 12G of RAM because more RAM was cheaper for my client than paying me to figure out ZFS problems. BTRFS doesn't seem to have any real RAM issues on somewhat modern systems. I have a system with 512M of RAM that's running BTRFS with no problems at all. BTRFS support is included in kernels from most distributions, Debian/Unstable has kernels that work well and Oracle supported BTRFS last time I checked. ZFS isn't supported by anyone. BTRFS doesn't have any sort of RAID-5/6 support that's remotely usable. RAID- Z/Z2/Z3 works really well on ZFS. If you use a BTRFS root filesystem with systemd then you can't balance or scrub the filesystem because systemd journal file use triggers a BTRFS data corruption bug. But then root on ZFS is pretty much unusable on Linux anyway. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On 18 February 2014 14:30, Russell Coker <russell@coker.com.au> wrote:
On Tue, 18 Feb 2014, Petros <Petros.Listig@fdrive.com.au> wrote: BTRFS doesn't have any sort of RAID-5/6 support that's remotely usable. RAID- Z/Z2/Z3 works really well on ZFS.
If you use a BTRFS root filesystem with systemd then you can't balance or scrub the filesystem because systemd journal file use triggers a BTRFS data corruption bug. But then root on ZFS is pretty much unusable on Linux anyway.
The version of btrfs that ships with Ubuntu Saucy supports RAID levels: 0, 1, 5, 6, 10 You can mix and match, so metadata gets mirrored while data gets raid5, for instance, if you really want. I've been running btrfs as root on a few systems, and they've all been fine. Note that they're Ubuntu though, so they'll be Upstart, not systemd, and with almost certainly later versions (and more heavily patched versions) of the linux kernel. But presumably available on Debian via some backports. -T
On Tue, 18 Feb 2014, Toby Corkindale <toby@dryft.net> wrote:
On 18 February 2014 14:30, Russell Coker <russell@coker.com.au> wrote:
On Tue, 18 Feb 2014, Petros <Petros.Listig@fdrive.com.au> wrote: BTRFS doesn't have any sort of RAID-5/6 support that's remotely usable. RAID- Z/Z2/Z3 works really well on ZFS.
If you use a BTRFS root filesystem with systemd then you can't balance or scrub the filesystem because systemd journal file use triggers a BTRFS data corruption bug. But then root on ZFS is pretty much unusable on Linux anyway.
The version of btrfs that ships with Ubuntu Saucy supports RAID levels: 0, 1, 5, 6, 10 You can mix and match, so metadata gets mirrored while data gets raid5, for instance, if you really want.
There are bugs in the code for balancing and regenerating data from parity. So while BTRFS will technically support RAID 5/6 if you have a hardware failure and need to recover then you may need to wait for new kernel code that hasn't been written yet. BTRFS does support RAID-10, the way it does RAID-1 makes it RAID-10 if you add more disks.
I've been running btrfs as root on a few systems, and they've all been fine. Note that they're Ubuntu though, so they'll be Upstart, not systemd,
Yes, upstart doesn't do the logging stuff that systemd does so doesn't trigger that bug. sysvinit is also OK.
and with almost certainly later versions (and more heavily patched versions) of the linux kernel. But presumably available on Debian via some backports.
No. The latest kernel in Debian/Unstable doesn't have a fix. 3.13 fixes it and I think there's a new kernel.org kernel in the 3.12 series that does it. But it's not in Debian/Unstable yet. I haven't checked Debian/Experimental. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/
On Tue, 18 Feb 2014 03:12:28 PM Toby Corkindale wrote:
The version of btrfs that ships with Ubuntu Saucy supports RAID levels: 0, 1, 5, 6, 10
There is highly experimental RAID 5 and higher, but I would not suggest committing any sort of data that you are in any way attached to to it. https://btrfs.wiki.kernel.org/index.php/FAQ#Can_I_use_RAID.5B56.5D_on_my_Btr... The last relevant change to that code was a memory leak fix that was released in 3.12. cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
On 18 Feb 2014, at 2:11 pm, Petros <Petros.Listig@fdrive.com.au> wrote:
Do you have good advice to achieve this?
I reckon you should give Docker[1] a try. It’ll do everything you’re looking for, as far as I can tell, including integration with btrfs snapshots[2] in the latest 0.8 version. Cheers, Avi [1] https://www.docker.io [2] http://blog.docker.io/2014/02/docker-0-8-quality-new-builder-features-btrfs-...
On 18 February 2014 14:42, Avi Miller <avi.miller@gmail.com> wrote:
On 18 Feb 2014, at 2:11 pm, Petros <Petros.Listig@fdrive.com.au> wrote:
Do you have good advice to achieve this?
I reckon you should give Docker[1] a try. It’ll do everything you’re looking for, as far as I can tell, including integration with btrfs snapshots[2] in the latest 0.8 version.
+1 for Docker.
On 18 February 2014 14:42, Avi Miller <avi.miller@gmail.com> wrote:
I reckon you should give Docker a try. It’ll do everything you’re looking for, as far as I can tell, including integration with btrfs snapshots in the latest 0.8 version.
Whilst I can recommend Docker for container management, the btrfs support is experimental; so I'd be interested to hear of anyone's experience with this. ~ Joel
participants (7)
-
Avi Miller -
Chris Samuel -
Jason White -
Joel W Shea -
Petros -
Russell Coker -
Toby Corkindale