October 2012 - luv-main

getting video resolution
by Russell Coker 21 Oct '12

21 Oct '12

Is there any good way to get the resolution of a video file? For jpegs ImageMagick has the "identify" command which works really well, I'd like to have something similar for movies, particularly flv files. Currently I use the below shell script which is ugly but works, unfortunately it flashes up an X window and probably doesn't work if you aren't using X. #!/bin/sh while [ "$1" != "" ]; do echo q| mplayer $1 2>&1|grep ^VO.*x11| sed -e s/\ =.*$// -e s/^.*\ // shift done -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

6 6

log compression
by Russell Coker 21 Oct '12

21 Oct '12

Does it make sense to compress log files nowadays? BTRFS and ZFS have compression built in, that compression probably won't be as good as gzip on a large file because the filesystems will use smaller block sizes. But it will still provide some benefit. On modern systems writes are more of an issue. Rewriting every log file every night contributes to the wear of SSDs for no real benefit (any system which has log files any fraction of the size of a 60G minimal SSD will probably use ZFS or BTRFS on some big disks for main storage). It seems to me that the default log compression has more problems than benefits with today's technology. It made a lot of sense when 1G was a big hard drive, but nowadays a 60G SSD is about as small as you can buy. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

3 4

[luv-main] flashing a Samsung Galaxy S
by Russell Coker 18 Oct '12

18 Oct '12

http://wiki.cyanogenmod.com/wiki/Samsung_Galaxy_S:_Full_Update_Guide I've got a Galaxy S that I want to use for Cyanogenmod. I went through the part of the above that starts with "assumes the Samsung Galaxy S already has Android 2.3" while my phone had Android 2.2. The result is that it just flashes on the kernel boot screen. I can still get into download mode so all is not lost (yet). http://www.samfirmware.com/WEBPROTECT-i9000.htm I tried getting other zImage files from the above, I tried the 2011 March copy of Android 2.2 from the "America Australia Samsung I9000 Firmwares" section of the above. http://www.samfirmware.com/apps/blog/show/8512833-android-2-3-5-for-galaxy-… I also tried the 2.3.5 from the above. Neither worked. Any suggestions on what I should try next? The phone came from Optus and was running Android 2.2 when I got it. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

8 8

cache, Xen, and zfsonlinux
by Russell Coker 18 Oct '12

18 Oct '12

I'm looking at converting some Xen servers to ZFS. This includes a couple of servers for a reasonable size mail store (8,000,000 files and 600G of Maildir storage). For much of the Xen on ZFS stuff I'll just use zvols for block devices and then use regular Linux filesystems such as Ext3 inside them. This isn't particularly efficient but for most DomUs it doesn't matter at all. Most of the DomUs have little disk access as they don't do much writing and have enough cache to cover most reads. For the mail spool a zvol would be a bad idea, fsck on a 400G Ext3/4 filesystem is a bad thing and having the double filesystem overhead of Ext3/4 on top of a zvol is going to suck for the most disk intensive filesystem. So it seems that the correct solution is to do one of the following: 1) Run the mail store in the Dom0 which will be good for performance at the cost of management. A server which has direct user access in any form (including POP and IMAP) needs to be running all the latest security patches while a Dom0 can have patches delayed if they don't seem relevant to network issues or virtualisation. 2) Use NFS to mount a ZFS filesystem from the Dom0. This will be good for management but there's the problem of caching. I don't think that NFS caches that aggressively so I'd need to give more RAM to the Dom0 for ZFS caching and I'd probably still lose some read performance. 3) Run ZFS in the DomU separately from the Dom0. This will work well for the DomU as long as there is enough RAM. But having a ZFS filesystem in the Dom0 as well as a separate one in the DomU (which would use different partitions of the same disks) would be difficult (ZFS in the Dom0 will probably want to grab all zpools). Also write performance will take a hit if there are two separate zpools on the same disks as there will be seeks between writes - this will be particularly bad for mail delivery where the message and the log entry will be written to different parts of the disk. 4) Run ZFS for mail storage in the DomU and use something other than ZFS for the Dom0. This has the same performance problems as 3) but without the issue of different ZFS instances fighting about it. Also I would lose the support for hashes on the zvol data, I could use files on BTRFS for similar data integrity (I'm using RAID-1 so RAID-Z isn't an option and therefore the benefits of ZFS over BTRFS are fewer) but that would still give performance issues. Any suggestions? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

5 16

Dovecot/Perdition IMAP session timeout
by Russell Coker 17 Oct '12

17 Oct '12

I have a Dovecot server that gets some old sessions. One back-end server has 5/224 sessions open since September and another 14 from this month that are more than 2 days old. A total of 8.5% of sessions are more than 2 days old. This isn't necessarily a problem on it's own, none of the servers have enough load that 8.5% more or less will matter. But it becomes a problem when a user accumulates enough open and possibly dead sessions over a month that the limit of open sessions is reached and they open a support ticket. It's documented that some MUAs have problems with sessions getting killed, the problems in question can't be that great because no Internet connection is 100% reliable and a significant portion of the user-base doesn't have persistent IP addresses. In my own testing the most I've been able to do is generate a warning message about a session being closed. Is there any reason for not killing the sessions? If there is no good reason not to kill them then is there a way of making Dovecot or Perdition kill a session after it's been IDLE for a couple of days? Otherwise I guess I'll have to use a cron job that kills them. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 2

Re: cache, Xen, and zfsonlinux
by Russell Coker 17 Oct '12

17 Oct '12

On Mon, 15 Oct 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > My fallback would be NFS shared ZFS in the domU - much cheaper because > you only have one ARC, set of checksums, etc, to manage, but with the > added bonus of NFS between the domU and dom0. Fun times. http://wiki.dovecot.org/MainConfig The above page says "Dovecot can also use dnotify, inotify and kqueue to find out immediately when changes occur" in regard to the IMAP "IDLE" command (waiting for notification of new mail). So it seems that using NFS is going to incur extra overhead for polling (which probably isn't that great) as well as delaying the notification of new mail. This probably doesn't matter much as it seems unlikely that there is a reasonable demand by a user for how soon they should be notified which would also involve frequent enough polling to cause significant load. But it is another reason for not using NFS. It's really a pity that the Xen DomU kernel doesn't have it's own filesystem driver which proxies access to the Dom0. Then things like Inotify would just work. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

Re: cache, Xen, and zfsonlinux
by Russell Coker 17 Oct '12

17 Oct '12

On Mon, 15 Oct 2012, Daniel Pittman <daniel(a)rimspace.net> wrote: > > For the mail spool a zvol would be a bad idea, fsck on a 400G Ext3/4 > > filesystem is a bad thing and having the double filesystem overhead of > > Ext3/4 on top of a zvol is going to suck for the most disk intensive > > filesystem. > > zvol is more like an LVM logical volume than a filesystem, so the > overhead isn't nearly as much as this comment suggests. That said, > running ext3 (especially) or ext4 on top of it is going to be slower, > and means you can't use the RAID style features of ZFS, and you give > up object level checksums. I had the impression that a zvol gets all the consistency benefits of ZFS but for a block device. ZFS on top of a zvol may not be a good idea as ZFS is fairly heavy, BTRFS on top of a zvol might be an option though. > > Any suggestions? > > I would aim to run ZFS in the mail domU, and treat the zvol as a > "logical volume" block device. You will have some overhead from the > double checksums, but robust performance. It treats the underlying > dom0 ZFS as a fancy LVM, essentially. You probably also need to > allocate substantially more memory to the domU than you would > otherwise. Yes, lots more RAM in both Dom0 and DomU. The problem is that I have servers with 16G of RAM that I'd prefer not to upgrade. > My fallback would be NFS shared ZFS in the domU - much cheaper because > you only have one ARC, set of checksums, etc, to manage, but with the > added bonus of NFS between the domU and dom0. Fun times. > > Are you sure that LXC or OpenVZ wouldn't better fit your needs than > Xen? You trade off marginally less isolation between containers for > the simplicity of having a single kernel image - so native ZFS > performance. That's a possibility. What is the support for them like in Debian/Wheezy? On Mon, 15 Oct 2012, Jason White <jason(a)jasonjgw.net> wrote: > XFS doesn't have the fsck problem, but it isn't optimized for large numbers > of small files, as I recall. I can't comment on reliability/performance. I > don't know much about JFS either - XFS seems to be receiving more > development attention from Red Hat and elsewhere at the moment. I don't > think Reiser 3 is seeing much work anymore either. XFS has had FSCK problems in the past. I think that JFS is as good as dead. Even before Hans was arrested there were some issues with ReiserFS such as the fact that hostile data written to a file by a user could cause a corruption at FSCK time. If you use ReiserFS for any server that allows binary data to be stored by a user then you risk security problems. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

4 4

Gentoo Linux on a Macbook Pro.
by brettawiggins＠tpg.com.au 16 Oct '12

16 Oct '12

Hi Everyone, I'm new to the list and this is my first post. I've been given a Macbook 4,1 (2008) from a friend and I would like to install Linux on it. I have never installed Linux on a Macbook before so I'm in unfamiliar waters. I know that it has no bios and I think uses something called EFI for booting. I want to install Gentoo Linux on the Macbook. I'm using the SystemRecueCD usb installer because it's gentoo based and the DVD drive on this Macbook is shot. The main gaps in my knowledge with doing this install surround Grub and partitioning. Is installing and configuring grub different? Do I have to change the way I setup partitions? I don't want to dual boot, I just want a pure Gentoo MacBook. Has anyone on the list done a Macbook before? As I said I've never done this type of install before and know nothing about Macbooks or EFI. thanks, Brett.

5 5

ocfs2 vs gfs
by James Harper 15 Oct '12

15 Oct '12

I am presently using ocfs2 managed by a pacemaker cluster on Debian Wheezy, and while it works great while everything is up, rebooting a node has mixed results, things really start to go south as on reboot another node will get upset and crash which has a bit of a cascade effect sometimes requiring manual intervention to resolve. Can anyone comment on their experience with gfs and/or ocfs2 and reliability? I'm using ocfs2 to store xen configs, and iso images for xen vm's, so performance really isn't a consideration while reliability is. Thanks James

2 3

Problems getting stack traces from Python program under Kubuntu 12.10
by Chris Samuel 15 Oct '12

15 Oct '12

Hi folks, I'm trying to get to the bottom of this bug: https://bugs.launchpad.net/ubuntu/+source/synaptiks/+bug/1039261 The tl;dr version is that the Python interpreter is somehow emitting two calls to the Xorg libXi function XIQueryVersion(), the first call sends a client XInput version number of 2.1 and then the second one sends 2.0 (seen using xtrace). The second call causes a BadValue error, because you're not meant to send a lower value on any later calls. This causes the comical error: # The version of the XInput extension installed on your system is # too old. Version 2.0 was found, but at least version 2.0 is required The problem is that the Python code only has the second call sending the 2.0 version number, there is no other call in the package that will send anything else, let alone the 2.1 value. So I want to generate a call trace every time the XIQueryVersion() function is called, but I'm struggling to get it to work. The killer at the moment is that both ltrace and gdb (when told to trace children) hang when python runs dash to run ldconfig which calls ldconfig.real and that blocks - so I never get to the point where the function gets called the first time. Any ideas? cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC

1 1