November 2014 - luv-main

ZFS compression - looking for space savings
by Peter Ross 23 Nov '14

23 Nov '14

Hi all, I want to reorganize some backups and use ZFS snapshots copied to a "backup NAS" (I am having). Traditionally some zfs were sent as a .tgz to the NAS. I want to use ZFS compression on the box instead and look for the best method to compress it. Space savings are important, speed isn't. Below the man page describing the compression methods available. Do you have experience to recommend a method working well? I do not want to "muck around" and change too much because it is Terabytes worth of data and I do not want to reconfigure too often. I do not use compression on my production systems a lot so I do not have too much experience with it. I have a few "not frequently used/non-critical" filesystems with lz4 - but that's it. Speed-wise it did not seem to matter much compared to uncompressed filesystems (but I did not benchmark properly) Thanks for any advice Peter compression=on | off | lzjb | gzip | gzip-N | zle | lz4 Controls the compression algorithm used for this dataset. The lzjb compression algorithm is optimized for performance while providing decent data compression. Setting compression to on uses the lzjb com- pression algorithm. The gzip compression algorithm uses the same com- pression as the gzip(1) command. You can specify the gzip level by using the value gzip-N where N is an integer from 1 (fastest) to 9 (best compression ratio). Currently, gzip is equivalent to gzip-6 (which is also the default for gzip(1)). The zle compression algo- rithm compresses runs of zeros. The lz4 compression algorithm is a high-performance replacement for the lzjb algorithm. It features significantly faster compression and decompression, as well as a moderately higher compression ratio than lzjb, but can only be used on pools with the lz4_compress feature set to enabled. See zpool-features(7) for details on ZFS feature flags and the lz4_compress feature.

4 4

Varnish :80 New Zealand Apache Problems
by Piers Rowan 23 Nov '14

23 Nov '14

Hi All, This is a strange one. Ages ago we suddenly had NZ visitors complaining about time outs and white screens for our web apps. Noting else was broken except the people across the ditch (and a few other overseas customers). After testing and trying a vast number of things we put up varnish and told them to connect on port :6xxx and hey presto it all went back to normal. We did this for a few months and then decided to move everything on :80 through varnish. The speed improvement is great. Love it. Amazing. And now we have the NZ clients complaining that things are dropping out and whitescreening again. Is there a great firewall of Australia on port 80? How can one service on port :6xxx work flawlessly and then when moved to :80 kill overseas users? Really this is doing my head in. I am faced with changing everything back (and removing varnish and the speed it gives) and getting the NZ folks to use a weird port again. Clues appreciated. Cheers Piers

10 16

LUV server etc
by Russell Coker 21 Nov '14

21 Nov '14

The old LUV server has now been replaced. Due to multiple hardware problems (drive errors and CPU/RAM problems) I had to do a rush job. If you see this then the mailing lists are now working. The web site isn't working yet, I'm still trying to get it going. Any suggestions appreciated. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 1

ASUS EEE 701 GRUB problem
by Peter Ross 21 Nov '14

21 Nov '14

Hi all, I found an older ASUS EEE 701. The Ubuntu login on it was unknown and the person who used it on company travel left a while ago. I decided to put Debian 7.6.0 on it (BTW: FreeBSD 10 does not recognize the WLAN). I installed "standard" (text) and all seemed fine. Just grub refused to install. So I installed the grub packages manually (using the "live" USB stick) and run "grub install /dev/sda" in a chroot environment. All worked but it still does not boot. "grub-setup /dev/sda" seems to be working too.. I also used "grub-mkconfig" successfully. The Asus EEE 701 has a 8GB SSD drive. Is there something "special" about the Asus I should do? Workarounds? Thanks for ideas Peter

3 2

Re: ZFS compression - looking for space savings
by Peter Ross 20 Nov '14

20 Nov '14

Hi Toby and all, Before you ask;-) > I can make a test on a 32 GB RAM, 2 Xeon processor machine but this is a > bit of limited value, I think. I can actually test it now and will see the space savings. And then I hope I do not "kill" the 1GB machine. Regards Peter

1 0

Re: ZFS compression - looking for space savings
by Peter Ross 20 Nov '14

20 Nov '14

Hi Toby, thanks for the answer. > So.... what's stopping you doing a quick test to find out what is best > for you? I have the appliance not available yet but I will get there next week. Actually I forgot one condition.. the 1 GB RAM. I installed PC-BSD on a laptop with 1GB of RAM before (with ZFS only) and realized that it works in general (with Flukebox, no Gnome/KDE..) So I hope this works for this appliance too (no GUI needed here) I can make a test on a 32 GB RAM, 2 Xeon processor machine but this is a bit of limited value, I think. [For historical reasons - this problem is solved] The other issue was a "start". ZFS is only compressing new data. So I do not have an issue to send data from incremental snapshots from an uncompressed filesystem to a compressed one. But if I have a few hundred GB already? The "zfs receive" will create the ZFS.. I figured out a way, you can create the parent filesystem with compression on and the new one inherits this value:-) I may need some smarts in a script to get the path names right. > I ran a quick test using (non-zfs) equivalents of various compression tools, over a 2.0G filesystem image. (ie. hoping that represents a fair variety of binary+text files) > lz4 1.7s 221M > lz4 -6 10s 189M > gzip 25s 151M > bzip2 54s 135M > 7z 147s 102M > xz 253s 103M Thanks for it. I will see whether it matches the ZFS compression. > So if space is really at a premium, you're better off using an archive tool to compress everything, rather than zfs' built-in compression. The fun comes if you can keep a bunch of older copies thanks to snapshots. At the moment the backup is keeping six weekly backups online (compressed full backups) plus a year worth of quarterly backups. I hope to push it further back with snapshots. > But otherwise, gzip is better than lz4, but at significantly slower performance. Agreed and thanks Peter

1 0

mythtv settings
by Brian May 19 Nov '14

19 Nov '14

2 2

Computer issues with Linux
by Brian May 19 Nov '14

19 Nov '14

Hello, Am having two strange problems that are starting to eerk me with this (relatively new) computer, running Debian wheezy: 1. If I boot anything later then 3.12 kernel, I don't get any display. As in the monitors display black. No cursors of any sort. Changing to a virtual console doesn't help. Booting in rescue mode doesn't help (I think this rules out X-Windows being a problem). If I go back to the 3.12 kernel, everything works perfectly. I also tried plugging monitors into alternative ports just in case it is going to the wrong place, but get nothing - in any case, under 3.12 the computer seems pretty good at automatically working out what ports are active under X-Windows. 01:00.0 VGA compatible controller: NVIDIA Corporation G96 [GeForce 9500 GT] (rev a1) Am currently using the non-free nvidia drivers. Had exactly the same symptoms when I installed the latest kernel without the non-free nvidia kernel modules. I think the problem is occurring before X starts. Computer seems to be up and running, and responsive to crl+alt+del despite not having a display. 2. There seems to be some weird performance problem. e.g. save a 2 kilobyte file in vim, and the computer can completely freeze (all other windows, including xterms, stop responding to user input) for, say 30 seconds, while it is writing that file. Chromium takes ages to load with several tabs, and pages can fail to start properly while it is doing so. Computer has 16GB RAM and is not using any swap. It has 11GB of buffer/cache space: total used free shared buffers cached Mem: 15G 13G 2.1G 0B 1.1G 7.9G -/+ buffers/cache: 4.6G 11G Swap: 3.8G 0B 3.8G Problems occurred before starting chromium, previously I wondered if it was chromium's fault. This is moving disk + RAID1 + LVM + ext4. Bonnie++ results seem to be pretty good, better in fact then my work computer, which doesn't suffer from similar problems. Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP falidae 31904M 1228 94 107313 4 47471 2 +++++ +++ 137886 3 441.8 2 Latency 19141us 12435ms 251ms 19913us 88073us 283ms Version 1.96 ------Sequential Create------ --------Random Create-------- falidae -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 36us 223us 226us 36us 10us 24us 1.96,1.96,falidae,1,1404290654,31904M,,1228,94,107313,4,47471,2,+++++,+++,137886,3,441.8,2,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,19141us,12435ms,251ms,19913us,88073us,283ms,36us,223us,226us,36us,10us,24us I am currently working on a new theory that the performance problems only occur when the computer is cold and first turned on. I think I have seen evidence to disprove this, but guess I should run bonnie++ as soon as I turn the computer on, just to be sure. Any other ideas? Thanks -- Brian May <brian(a)microcomaustralia.com.au>

9 25

New openSUSE 13.2 Install failure
by Andrew Greig 12 Nov '14

12 Nov '14

Hi All, Toshiba Satellite 32 bit, has been running under OpenSUSE very nicely since 11. Tried an Upgrade to 13.2 from 13.1 KDE splash screen indicated that it failed to load the Display. In 13.1 it was a Gnome desktop. So I selected CTRL +ALT +F2 and logged in there to try again, this time with a Gnome desktop. Worse outcome. Clean shut down from the Alt F2 screen and decided to run the install with LVM btrfs and encryption to force a format of /home for the next attempt. Now I know that openSUSE has moved to systemd for this release, but is that the problem, or do I look elsewhere? I still have the isos for 13.1 so regression is an option. I am seriously apprehensive about upgrading my production machine. Thanks Andrew Greig

2 2

Oracle and Linux
by Russell Coker 11 Nov '14

11 Nov '14

I'm writing this message as a Linux user and a member of LUV - not as a member of the LUV committee. Avi, you have a history of making good technical contributions to LUV and of representing your employer well. However I think that Oracle has been under- represented given their technical work and their importance in the industry. I think that it would be good to have some Oracle employees give talks at LUV meetings as they are doing a lot that LUV members could learn about. Avi you are obviously well qualified to give such talks, but I think it would be good to get other Oracle employees with different skills involved too. A lecture about running the Oracle Database on Linux would be of interest. Also Red Hat has been running free public events teaching people about their products and services which I have found to be good value (and I have recommended some on this list). Does Oracle run such events? If so please inform the LUV committee and we can forward to the list if appropriate (I don't think it's appropriate for Linux companies to advertise on this list, but it's OK for LUV members who aren't associated with companies in question to recommend them). Some years ago there was an Oracle database install-fest, that was a useful event and many people learned a lot there. I think it would be good if Oracle ran another of those, if that happens I think it would be appropriate for LUV to promote that. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

6 19