- luv-main - luv.asn.au

ups <-> psu comms
by James Harper 17 Apr '13

17 Apr '13

Is there such a thing as a ups to power supply communications channel over the 240V lines, such that the UPS can talk to each power supply in a server and figure out what is plugged in where? It seems such an obviously useful idea but I can't see that such a thing exists anywhere. Comms would only need to be very basic (hundreds of bps would be sufficient) - PSU would tell the UPS what its maximum rating was and maybe an identifier (eg for grouping redundant PSUs), and the UPS would tell the PSU when it was on battery etc. I'm shopping for UPS's and looking at features etc. I'm opting for network based comms on my UPS's these days but even seems like a lot more mucking around than it could be. James

4 5

file sharing over the Internet
by Russell Coker 15 Apr '13

15 Apr '13

A client has a Samba server running ZFS that's working quite well for what it does. The problem is that a bunch of staff want to work from home and have been using Dropbox and Gmail to share the files - not what we want. Among other things we'd like to have the files stay within Australian jurisdiction. What's a good way for sharing files to Macs and Windows systems over the Internet that's easy to use and relatively secure? -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

13 17

Re: ZFS vs RAID (was gpt and grub)
by Petros 15 Apr '13

15 Apr '13

Quoting "Craig Sanders" <cas(a)taz.net.au> > On Fri, Apr 12, 2013 at 03:31:20PM +1000, Kevin wrote: >> On Fri, Apr 12, 2013 at 3:17 PM, James Harper >> <james.harper(a)bendigoit.com.au> wrote: >> >> > My last remaining reservation on going ahead with some testing is is >> > there an equivalent of clvm for zfs? Or is that the right >> approach for zfs? >> > My main server cluster is: >> > >> > 2 machines each running 2 x 2TB disks with DRBD with the primary >> exporting the whole disk as an iSCSI volume >> > 2 machines each importing the iSCSI volume running lvm (clvm) on >> top, and using the lv's as backing stores for xen VM's. > >> > How would this best be done using zfs? > > short answer: zfs doesn't do that. > > in theory you could export each disk individually with iscsi and build > ZFS pools (two mirrored pools). if that actually worked, you'd have to > do a lot of manual stuffing around to make sure that the pools were only > in use on one machine at a time, and more drudgery to handle fail-over > events. seems like a fragile PITA and not worth the bother, even if it > could be made to work. James, you want ZFS mirrored over the net as the backend for VM storage (virtual disks?) In case you really want.. "Poor man's version" similar as Russell described (zfs send/receive) is used by me: I am looking for running Jails and VirtualBoxes (well, the later is just one here..) and run zfs snapshot/send/receive to send all filesystems of the active Jails/VMs to the other side. In case one machine is broken I bring them up on the "other side". To mirror them without delay, you can create, one both sides, a volume on top of ZFS (zfs create -V 5gb MyPool/MyVM), export them over iscsi and mirror them. The fail-over between Virtual Machines is a problem independent of the sharing on the file system layer. A semaphore on the shared volume can be used for a cluster solution to manage it. BTW: You can put any file system, including a clustered one, inside these ZFS volumes, and combine the benefits of ZFS volume management with the benefits of the filesystem "inside". Regards Peter

1 0

RE: CEPH (was ZFS vs RAID (was gpt and grub))
by James Harper 15 Apr '13

15 Apr '13

> > James Harper <james.harper(a)bendigoit.com.au> wrote: > > > CEPH does sound exciting. Is anyone here doing it with Xen? > > It was still under heavy development when last I read about it. The kernel > module entered the mainline and, I assume, has undergone further work > since > then. > > File systems take a long time to mature. Testing and bug finding efforts > always help, however. > Full of enthusiasm, I've just created 3 xen VM's and created a 3 node CEPH cluster inside. It's amazingly easy to set up, but from what I can see some of the init tools automate a lot of the steps that I assume need to be done manually for ongoing maintenance, so I may go back and set it up the long way around to get a feel for the steps involved, or maybe just add a few more nodes. For a production CEPH installation, I think I would still use RAID1 underneath and then deploy CEPH on top, as it lowers the risk of a node-down situation, at the expense of requiring additional disks. Debian appears to have some OCF RA's which I haven't investigated yet, as I'd still want to use pacemaker for managing the actual VM's. For a 5 node cluster with 2 copies of all xen virtual disks I assume it becomes important to run VM's on a node that has a local replica of the data... or is the data striped across all nodes transparently in a "you don't need to know" way? I'd better do some more reading I guess :) James

2 3

Re: file sharing over the internet
by Greg Bromage 15 Apr '13

15 Apr '13

Brian May wrote: > > Dropbox and Gmail to share the files - not what we want. Among other > > things > > we'd like to have the files stay within Australian jurisdiction. > > >If the files were securely encrypted, does it matter that they stay in >Australia? IANAL, but my understanding is a definite "Yes" to this. National Privacy Principle 9 applies even to encrypted data, because there's no hard and fast definition of what "encryption" means. NPP #4 requires us to take "reasonable steps" to secure information, and encryption is certainly one of those steps, for what the IT profession generally considers reasonable. For example, if I simply password protected an MS Word 97 document, there's no way in this day and age I could defend that as being "reasonable" encryption.

1 0

Extracting a distributable file tree from git and svn
by Allan Duncan 14 Apr '13

14 Apr '13

I would like to get a file tree representing a specific (or latest) tag/branch/label from a repository. Trawling the help files shows me how to manipulate the repository itself, but not to extract the subset that represents just the code. I have done the brute force recursive removal of the .git (or .svn) dirs to get the latest set of files, but there must be a better way.

4 5

Re: ZFS vs RAID (was gpt and grub)
by Kevin 14 Apr '13

14 Apr '13

I originally sent this direct to James resending it to the list > On Fri, Apr 12, 2013 at 3:17 PM, James Harper < james.harper(a)bendigoit.com.au> wrote: > > > > . Online resize/reconfigure > > > > both btrfs and zfs offer this. > > > > Can it seamlessly continue over reboot? Obviously it can't progress while the system is rebooting like a hardware raid but I'd hope it could pick up where it left of automatically. Yes it does. > This is where a lot of people get this wrong. Once the BIOS has succeeded in reading the bootsector from a boot disk it's committed. If the bootsector reads okay (even after a long time on a failing disk) but anything between the bootsector and the OS fails, your boot has failed. This 'anything between' includes the grub bootstrap, xen hypervisor, linux kernel, and initramfs, so it's a substantial amount of data to read from a disk that may be on its last legs. A good hardware RAID will have long since failed the disk by this point and booting will succeed. > > My last remaining reservation on going ahead with some testing is is there an equivalent of clvm for zfs? Or is that the right approach for zfs? My main server cluster is: > > 2 machines each running 2 x 2TB disks with DRBD with the primary exporting the whole disk as an iSCSI volume > 2 machines each importing the iSCSI volume running lvm (clvm) on top, and using the lv's as backing stores for xen VM's. > > How would this best be done using zfs? If i was building new infrastructure today with 2 or more machines hosting VMs i would probably look at using CEPH as the storage layer for the Virtual machines. this would provide distributed mirrored storage that is accessible from all machines. all machines could then be storage and VM hosts. ref: http://www.slideshare.net/xen_com_mgr/block-storage-for-vms-with-ceph http://ceph.com/

5 8

Re: gpt and grub
by Greg Bromage 13 Apr '13

13 Apr '13

On 2013-04-09 02:40, James Harper wrote: > I have a server that had 4 x 1.5TB disks installed in a RAID5 > configuration (except /boot is a 'RAID1' across all 4 disks). One of > the disks failed recently and so was replaced with a 3TB disk, I'd be very wary of running RAID5 on disks >2TB Remember that, when you have a disk failure, in order to rebuild the array, it needs to scan every sector of every remaining disk, then write to every sector of the replacement disk. Compare those number of read & write operations to the Mean Time Between Failures of each disk, and you're (statistically) starting to get close to the point there's a significant risk of a second drive failing prior to the rebuild finishing. (For a given definition of "significant" risk)

10 29

distributed storage
by Russell Coker 12 Apr '13

12 Apr '13

http://www.hetzner.de/en/hosting/produkte_rootserver/poweredge-r720 The above server from Hetzner is an interesting offer. A system with 128G of RAM for only E299 per month. I'm not aware of any company making a similar offer at this time (if you know of one then let me know, some of my clients will be interested). http://www.hetzner.de/en/hosting/produkte_rootserver/ex4 The problem is storage, E25 per month per disk is a lot of money. They offer "ex4" servers for E49 per month that have 2*3TB disks which saves E1 per month and also gives 16G of cache per node. What would be a good distributed filesystem to run across 4*EX4 nodes with one R720 as the main server? Would ZFS using RAID-Z2 over iSCSI be a viable option? I don't have a real use for such things at this time. But sometimes when you tell clients about options for cheaply doing things that used to be expensive then they go for it. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

1 0

ZFS vs RAID (was gpt and grub)
by James Harper 12 Apr '13

12 Apr '13

> > as you suggest bcache and flashcache seem to offer a way around this for > mdadm but i've never used either of them - i was already using zfs by > the time they became available. i don't think the SATA interface speed > is a deal-breaker for them because the only way around that is spending > huge amounts of money. > There were some not-too-expensive battery backed PCI ramdisks available a while ago. Not anymore though. > > > . Battery backed write cache. Bcache/flashcache offer this but they > > have their shortcomings, in particular that most available cache > > modules are still on top of the SATA channel. > > this is the only real advantage of hardware raid over mdadm. IMO, ZFS's > ability to use an SSD or other fast block device as cache completely > eliminates this last remaining superiority of hardware raid over > software raid. Yes I've now been enlightened on this subject :) > > . Online resize/reconfigure > > both btrfs and zfs offer this. > Can it seamlessly continue over reboot? Obviously it can't progress while the system is rebooting like a hardware raid but I'd hope it could pick up where it left of automatically. > > . BIOS boot support (see recent thread "RAID, again" by me) > > this is a misfeature of a crappy BIOS rather than a fault with software > raid. > > any decent BIOS not only has the ability to choose which disk to boot > from (rather than hard-code it to only boot from whichever disk is > plugged into the first disk port) but will also let you specify a boot > order so that it will try disk 1 followed by disk 2 and then disk 3 or > whatever. they'll also typically let you press F2 or F12 or whatever at > boot time to pop up a boot device selection menu. > > even server motherboards like supermicro let you choose the boot device > and have a boot menu option accessible over IPMI. > This is where a lot of people get this wrong. Once the BIOS has succeeded in reading the bootsector from a boot disk it's committed. If the bootsector reads okay (even after a long time on a failing disk) but anything between the bootsector and the OS fails, your boot has failed. This 'anything between' includes the grub bootstrap, xen hypervisor, linux kernel, and initramfs, so it's a substantial amount of data to read from a disk that may be on its last legs. A good hardware RAID will have long since failed the disk by this point and booting will succeed. My last remaining reservation on going ahead with some testing is is there an equivalent of clvm for zfs? Or is that the right approach for zfs? My main server cluster is: 2 machines each running 2 x 2TB disks with DRBD with the primary exporting the whole disk as an iSCSI volume 2 machines each importing the iSCSI volume running lvm (clvm) on top, and using the lv's as backing stores for xen VM's. How would this best be done using zfs? Thanks James

2 1