January 2016 - luv-main

Castrated netbook (Windows 10 only)
by Peter Ross 23 Jan '16

23 Jan '16

Hi all, I bought a Lenovo Ideapad 100S before Christmas. My old netbook is not reliable (loose LVDS connector inside the glued screen frame, it seems) and I need a lightweight and trustworthy laptop when on call. So the idea was installing a proper OS on it but the UEFI is castrated to a point that I cannot even boot from USB (after SecureBoot is off). "System does not have any USB boot option" I get if I jump through all hoops and restart with USB device chosen. With a 32GB eMMC I do not have an option to replace that either it seems..(or do I have another option?) So I went to the Lenovo support website. A form with e-mail support - but the submit button does not work. Firefox is specifically mentioned as compatible with this website. Okay, chat. The support apologized for the system limitations and recommended to return it to the shop. Well, I will try my luck with this in my hands. I claimed that the device violates UEFI specification. Well, I actually do not know,and it may a bit bogus: http://www.uefi.org/faq --- Can all systems disable UEFI Secure Boot? While it is designed to protect the system by only allowing authenticated binaries in the boot process, UEFI Secure Boot is an optional feature for most general-purpose systems. By default, UEFI Secure Boot can be disabled on the majority of general-purpose machines. It is up to the system vendors to decide which system policies are implemented on a given machine. However, there are a few cases—such as with kiosks, ATM or subsidized device deployments—in which, for security reasons, the owner of that system doesn’t want the system changed. --- It does not mention the range of devices to boot from at all. My wife has another Lenovo netbook bought half a year earlier where I am able to boot from USB (well, the FreeBSD 10.2 UEFI memstick kernel panics shortly after that but that may have another reason). In general, I see the lockdown as a serious threat for open source. I wonder whether there are ways to alert the ACCC or other venues. At least locked down devices should be clearly marked to be aware of this. Any ideas in that regard? Should we do something on organisational level (via Linux Australia)? I think this is serious. We loose the ability to run open source on modern hardware completely, if we do not act, I think. Regards Peter

6 7

Linux firewall distribution
by zlinw＠mcmedia.com.au 22 Jan '16

22 Jan '16

I have been asked by a number of windows users for a firewall set up for use on an old PC. The reason for asking is they say most of the items currently commercialy availble have serious problems, like impossible to upgraded, upgrading trashs any alterations one has done to the config or the fire wall has hidden openings allowing certain what they regard is spy traffic through. windows 10 firewall getting many mentions here. I use a PC running debian as a firewall, this as I have set it up would probably not be regarded as easy to use. I use a PC with Debian installed with a custom kernel with all required modules compiled in. This uses a simply fire wall script that allows outgoing connection but blocks all incoming connections except those related to an existing outgoing connections. While most would regard it as to restrictve I am happy with it. I assume something one could look at via a web broswer from inside the secure area would what is required. sugestions very welcome. Lindsay

1 0

ata errors in dmesg/syslog - any pointers from the more ATA/AHCI literate?
by Anthony Hogan 21 Jan '16

21 Jan '16

Hey folks, I have my system (Gigabyte P55A-UD4 r1 F15 firmware) configured in AHCI mode with a 1+3 TB HDDs, and a DVD drive. ata5 = 1TB SATA (msdos partition scheme) ata9 = DVD SATA ata10 = 3TB SATA (GPT scheme to use 3TB as one, but not a boot drive) ata16 = Appears to be virtual "Marvell" device (I am not using hardware or fake RAID, and have disabled eSATA port) I have two 6Gbps SATA ports, and the rest are 3 I think. One of the 6GB ports goes to the 3TB drive, the other goes to a docking bay in the case that I use to do occasional offline backups. The kernel seems to be bitching about the DVD drive all the time (and sometimes my 1TB drive), and I'm not quite sure why. Buggy AHCI? Crappy SATA cables? Something else? Used AHCI mode for ages, but only with upgrade to Ubuntu version that I've noticed more bitching. The other day, my 1TB drive got pushed into read only mode on boot - I did several SMART tests and an fsck from a live environment and nothing out of the ordinary came up (I have recently installed Windows 10 in dual boot, but prior to that had Windows 7 - Windows didn't touch grub at all). Using search: zegrep 'ata([59]|10|16)|status: \{' /var/log/syslog.3.gz I've pulled out two examples of boot time fun: [ 1.062473] ata5: SATA max UDMA/133 abar m2048@0xfbffd000 port 0xfbffd200 irq 37 [ 1.078468] ata9: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff100 irq 39 [ 1.078470] ata10: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff180 irq 39 [ 1.078480] ata16: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff480 irq 39 [ 1.400286] ata16: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 1.400383] ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 1.400411] ata16.00: ATAPI: MARVELL VIRTUALL, 1.09, max UDMA/66 [ 1.400689] ata16.00: configured for UDMA/66 [ 1.401396] ata10.00: ATA-8: WDC WD3000FYYZ-01UL1B0, 01.01K01, max UDMA/133 [ 1.401401] ata10.00: 5860533168 sectors, multi 8: LBA48 NCQ (depth 31/32), AA [ 1.403407] ata10.00: configured for UDMA/133 [ 1.404179] ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 1.405880] ata9.00: ATAPI: ATAPI iHAS124 Y, BL0V, max UDMA/100 [ 1.408124] ata9.00: configured for UDMA/100 [ 2.192223] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [ 2.207881] ata5.00: ATA-8: WDC WD1002FAEX-00Z3A0, 05.01D05, max UDMA/133 [ 2.207887] ata5.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA [ 2.209995] ata5.00: configured for UDMA/133 [ 3.184967] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.185027] ata9.00: irq_stat 0x40000001 [ 3.200944] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.201002] ata9.00: irq_stat 0x40000001 [ 3.216173] ata16.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.216223] ata16.00: irq_stat 0x40000001 [ 3.216271] ata16.00: cmd a0/01:00:00:00:01/00:00:00:00:00/a0 tag 1 dma 16640 in [ 3.216336] ata16.00: status: { DRDY } [ 3.268939] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.268988] ata9.00: irq_stat 0x40000001 [ 3.284496] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.284545] ata9.00: irq_stat 0x40000001 [ 3.300450] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.300499] ata9.00: irq_stat 0x40000001 [ 3.324919] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.324967] ata9.00: irq_stat 0x40000001 [ 3.340548] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.340596] ata9.00: irq_stat 0x40000001 [ 3.356589] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.356636] ata9.00: irq_stat 0x40000001 [ 3.380945] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.380993] ata9.00: irq_stat 0x40000001 [ 10.885181] ata5.00: exception Emask 0x0 SAct 0x40000007 SErr 0x0 action 0x0 [ 10.885257] ata5.00: irq_stat 0x40000008 [ 10.885322] ata5.00: failed command: READ FPDMA QUEUED [ 10.885382] ata5.00: cmd 60/00:f0:c0:3d:83/01:00:1d:00:00/40 tag 30 ncq 131072 in [ 10.885490] ata5.00: status: { DRDY ERR } [ 10.885545] ata5.00: error: { UNC } [ 10.889441] ata5.00: configured for UDMA/133 [ 10.889571] ata5: EH complete [ 13.185375] ata5.00: exception Emask 0x0 SAct 0x3000000 SErr 0x0 action 0x0 [ 13.185463] ata5.00: irq_stat 0x40000008 [ 13.185526] ata5.00: failed command: READ FPDMA QUEUED [ 13.185585] ata5.00: cmd 60/08:c0:e0:3d:83/00:00:1d:00:00/40 tag 24 ncq 4096 in [ 13.185693] ata5.00: status: { DRDY ERR } [ 13.185749] ata5.00: error: { UNC } [ 13.189954] ata5.00: configured for UDMA/133 [ 13.190146] ata5: EH complete [ 15.037279] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.037342] ata9.00: irq_stat 0x40000001 [ 15.053304] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.053368] ata9.00: irq_stat 0x40000001 [ 15.073351] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.073423] ata9.00: irq_stat 0x40000001 [ 15.097661] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.097734] ata9.00: irq_stat 0x40000001 [ 15.117315] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.117378] ata9.00: irq_stat 0x40000001 [ 15.137616] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 15.137689] ata9.00: irq_stat 0x40000001 [ 139.870144] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 139.870148] ata9.00: irq_stat 0x40000001 [ 139.890320] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 139.890324] ata9.00: irq_stat 0x40000001 [ 139.897804] ata9: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 t4 [ 139.897807] ata9: irq_stat 0x40000001 [ 1.063019] ata5: SATA max UDMA/133 abar m2048@0xfbffd000 port 0xfbffd200 irq 37 [ 1.078151] ata9: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff100 irq 39 [ 1.078153] ata10: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff180 irq 39 [ 1.078163] ata16: SATA max UDMA/133 abar m2048@0xfbdff000 port 0xfbdff480 irq 39 [ 1.396028] ata16: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 1.396046] ata10: SATA link up 6.0 Gbps (SStatus 133 SControl 300) [ 1.396200] ata16.00: ATAPI: MARVELL VIRTUALL, 1.09, max UDMA/66 [ 1.396470] ata16.00: configured for UDMA/66 [ 1.397147] ata10.00: ATA-8: WDC WD3000FYYZ-01UL1B0, 01.01K01, max UDMA/133 [ 1.397153] ata10.00: 5860533168 sectors, multi 8: LBA48 NCQ (depth 31/32), AA [ 1.399072] ata10.00: configured for UDMA/133 [ 1.403953] ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 300) [ 1.405630] ata9.00: ATAPI: ATAPI iHAS124 Y, BL0V, max UDMA/100 [ 1.407746] ata9.00: configured for UDMA/100 [ 2.191935] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300) [ 2.204992] ata5.00: ATA-8: WDC WD1002FAEX-00Z3A0, 05.01D05, max UDMA/133 [ 2.204998] ata5.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA [ 2.207106] ata5.00: configured for UDMA/133 [ 3.184727] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.184787] ata9.00: irq_stat 0x40000001 [ 3.200768] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.200826] ata9.00: irq_stat 0x40000001 [ 3.215932] ata16.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.215991] ata16.00: irq_stat 0x40000001 [ 3.216044] ata16.00: cmd a0/01:00:00:00:01/00:00:00:00:00/a0 tag 1 dma 16640 in [ 3.216111] ata16.00: status: { DRDY } [ 3.276729] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.276788] ata9.00: irq_stat 0x40000001 [ 3.292414] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.292472] ata9.00: irq_stat 0x40000001 [ 3.308295] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.308343] ata9.00: irq_stat 0x40000001 [ 3.332740] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.332799] ata9.00: irq_stat 0x40000001 [ 3.352472] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.352530] ata9.00: irq_stat 0x40000001 [ 3.372280] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.372328] ata9.00: irq_stat 0x40000001 [ 3.396795] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 3.396854] ata9.00: irq_stat 0x40000001 [ 7.936707] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 7.936770] ata9.00: irq_stat 0x40000001 [ 7.984588] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 7.984652] ata9.00: irq_stat 0x40000001 [ 8.000647] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 8.000714] ata9.00: irq_stat 0x40000001 [ 8.024891] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 8.024958] ata9.00: irq_stat 0x40000001 [ 8.040638] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 8.040700] ata9.00: irq_stat 0x40000001 [ 8.060998] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 8.061060] ata9.00: irq_stat 0x40000001 [ 649.042550] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 649.042557] ata9.00: irq_stat 0x40000001 [ 649.062840] ata9.00: exception Emask 0x1 SAct 0x0 SErr 0x0 action 0x0 [ 649.062846] ata9.00: irq_stat 0x40000001

3 6

LUV server offerings
by Russell Coker 21 Jan '16

21 Jan '16

There is currently discussion on the LCA mailing list about alternative IM software etc. I'm considering the possibility of running things for LUV members. Do people consider that it would be a benefit to have something like an XMPP server for LUV members? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

debugging disk performance problems
by Russell Coker 21 Jan '16

21 Jan '16

http://etbe.coker.com.au/2016/01/21/storage-performance-problems/ The above blog post might be useful for some people. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

RE: Mail Server Really Slow
by James Harper 20 Jan '16

20 Jan '16

(it seems that "reply-all" no longer includes luv-main (from ms outlook at least), so I have to include it manually... what's with that?) > > Of course a RAID-1 of SSDs will massively outperform the RAID-5 you have. > > Given the size I guess it's one of the older HP servers that only takes ~70G disks. If you buy a cheap > Dell PowerEdge server and put a couple of Intel SSDs (not > bought from Dell because Dell charges > heaps for storage) in a RAID-1 configuration it will massively outperform the old HP server. If you use SSDs for any sort of intensive storage, do keep an eye on the SMART "media wearout" values, and replace them before the counter hits 0 (or 1). For the disks we were using (Intel DataCentre SSD's), the docs say that while the disk may well keep running for a long time after the counter hits 1, it is considered worn out and is no longer covered by warranty. SMART does not consider this a failure/old age (the threshold is 0 but the counter never goes below 1), so you have to actually monitor the counter. The RAID controller probably won't tell you this either (that the disk has worn out), and in our case the performance went to crap sometime after the counter hit 1, causing considerable frustration to all involved. Different models and manufacturers obviously differ in this respect too. I'm seeing time-to-replacement of about 12 months on high load system where the SSD's are used for a RAID cache (ZFS, Intel RAID controllers, etc). At home, my little router that is just a laptop running Squid, has used up 2 of 100 SMART units in the ~12 months it has been running. Not particularly relevant to the discussion at hand, but with suggestions of "put in SSD's and all your trouble will go away", it is something you need to consider. James

4 6

Mail Server Really Slow
by Piers Rowan 20 Jan '16

20 Jan '16

Hi there, The server runs: - dovecot - apache (roundcube webmail) - sendmail - clamav-milter - amavisd-milter RAM: 4.10 GB CPU: QEMU Virtual CPU version (cpu64-rhel6), 5 cores The server is a VM on a host server that also provides http / mysql services. The host server runs cron jobs to poll the email server (importing data from mail boxes into the CRM) so - to clutch at straws - I am not sure if the host and guest are competing for the disk IO at the same time with these calls. Contrary to that is that the host server does not experience any slow downs. Before the holidays we added another 30 users to the servers. Below is some of the output of some commands. At the moment I am thinking of moving the VM services into a cloud based host but this it not ideal in the short term. Any ideas? Thanks P CPU load averages 8.74 (1 min) 8.05 (5 mins) 5.63 (15 mins) [root]# vmstat 5 10 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 5 211056 141268 42948 2472992 1 1 246 47 2 2 2 1 93 5 0 0 3 211056 127876 43208 2486312 0 0 2710 46 1212 895 2 1 66 31 0 0 3 211104 141252 41136 2470440 0 10 3164 45 1109 804 2 1 74 23 0 0 3 211104 128228 41412 2484500 0 0 2859 18 566 640 0 0 60 39 0 0 3 211180 143320 40464 2464324 0 15 3122 70 1254 1030 2 1 57 40 0 Tasks: 502 total, 1 running, 501 sleeping, 0 stopped, 0 zombie Cpu(s): 7.7%us, 2.1%sy, 0.0%ni, 48.3%id, 41.6%wa, 0.0%hi, 0.0%si, 0.2%st Mem: 4296356k total, 4169448k used, 126908k free, 41144k buffers Swap: 2621436k total, 211980k used, 2409456k free, 2484540k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7067 clam 20 0 1771m 361m 1584 S 29.5 8.6 265:23.91 clamd 6450 kylier 20 0 19300 2828 1404 D 1.0 0.1 0:00.03 imap 6544 root 20 0 15396 1552 888 R 1.0 0.0 0:00.29 top 3848 root 20 0 78752 3024 2244 S 0.7 0.1 0:00.23 auth 6398 vickij 20 0 18528 2072 1400 D 0.7 0.0 0:00.48 imap 6419 uuu.s 20 0 18524 2024 1388 D 0.7 0.0 0:00.42 imap 6424 uuuu 20 0 19300 2812 1404 D 0.7 0.1 0:00.23 imap 23252 jasmin.s 20 0 19004 2652 1572 S 0.7 0.1 0:00.42 imap 23345 jasmin.s 20 0 26136 2472 1688 S 0.7 0.1 0:02.72 imap 29215 abigail. 20 0 18840 2068 1536 S 0.7 0.0 0:00.58 imap 1 root 20 0 19356 872 648 S 0.3 0.0 6:07.28 init 1336 root 20 0 244m 5128 508 S 0.3 0.1 19:20.96 rsyslogd 6293 glenn.ge 20 0 18388 1888 1432 S 0.3 0.0 0:00.01 imap 6567 nic.will 20 0 18408 1996 1512 D 0.3 0.0 0:00.08 imap 7091 root 20 0 19268 568 404 S 0.3 0.0 12:40.65 dovecot 7175 root 20 0 98.6m 364 320 S 0.3 0.0 3:43.93 tail 23388 uuuu 20 0 19000 2568 1544 S 0.3 0.1 0:14.31 imap 23460 jasmin.s 20 0 23248 2756 1756 D 0.3 0.1 0:01.54 imap ---8<---------8<---- # like this from here down 2 root 20 0 0 0 0 S 0.0 0.0 0:28.72 kthreadd

6 16

Re: Mail Server Really Slow
by Russell Coker 20 Jan '16

20 Jan '16

On Tue, 19 Jan 2016 11:24:44 PM James Harper wrote: > If you use SSDs for any sort of intensive storage, do keep an eye on the > SMART "media wearout" values, and replace them before the counter hits 0 > (or 1). For the disks we were using (Intel DataCentre SSD's), the docs say Interesting, I didn't realise that was the case. I've seen various claims of SSD wearout times being 5+ years which means they would be obsolete before they wear out. > disk has worn out), and in our case the performance went to crap sometime > after the counter hit 1, causing considerable frustration to all involved. Do you know of any good software to measure performance changes in disks? Performance often changes before drives develop faults and it would be good to measure that. EG if the number of iops the disk delivers when it's neat 100% according to the iostat algorithm suddenly changes by an order of magnitude then it's probably due for replacement. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 1

Re: Mail Server Really Slow
by Piers Rowan 20 Jan '16

20 Jan '16

On 20/01/16 08:13, Craig Sanders via luv-main wrote: > if you're wondering why Russell and I are asking all these questions > it's because you haven't provided anywhere near enough details to > answer your performance question. I just wanted to chime in that I really appreciate the help and have benefited from Russell, yours and the others' input immensely. I have a few things on so please don't take my lack of response and disinterest. I need to delve into some of the points and check out the server more before I can respond with anything worth while. Cheers Piers

1 0

BTRFS
by Russell Coker 19 Jan '16

19 Jan '16

The LUV VM is now running BTRFS. I've setup a cron job to make read-only snapshots of the root filesystem. Such snapshots are atomic which means that the people who are making offsite backups of the server can rsync the MySQL data files without integrity problems. Also the filesystem has duplicate metadata to cover the case of corruption. The Dom0 is running Linux Software RAID-1 which saves us from a total disk failure but doesn't handle corruption at all. If data corruption hits the data blocks then we have to rely on offsite backups. BTRFS doesn't support duplicate data blocks (in any normal or desirable configuration) unlike ZFS which has the copies= configuration. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

2 2