- luv-main - luv.asn.au

Re: reproducibility of results
by Craig Sanders 27 Dec '16

27 Dec '16

On Sun, Dec 25, 2016 at 04:56:05PM +1100, Paul van den Bergen wrote: > Funny, I was asked about exactly the same problem when I started > @WEHI... only there was no attempt made to even start tackling the > problem... yeah, we were constantly getting individual academics and research groups asking us about storage, and then trying to do the best we could with minimal resources. the unfortunate fact is that disks/storage arrays and file-servers and tape libraries etc are expensive. You can replace a very large percentage of your up-front capital expense with skilled techs, which are an on-going cost (you're going to need them to look after expensive equipment anyway, and it has to be maintained & upgraded for 7+ years), but it's still going to cost a lot for huge data storage anyway, even if you avoid over-priced name-brand gear. > Virtualisation of workload makes the problem a lot easier to tackle, > but even so... 7 years is a long time in IT... cheap big disks helps a lot too. but you need a lot of them, plus backup - on-site and off-site. CPU & RAM are more than adequate for pretty nearly any file-storage needs these days...could always use more of both for computational stuff. craig -- craig sanders <cas(a)taz.net.au>

2 1

A link for when we are in conversations on what's Linux & FOSS ...
by Wen Lin 27 Dec '16

27 Dec '16

Hi, Just found the following interesting piece from the Internet I thought I would like to share with you, as a gift for your bed time reading ... A sort of all-in-one piece on what's Linux and Free & Open Source Software, and ... "How I Learned to Stop Worrying and Love the FOSS!" 8-) http://www.makeuseof.com/tag/help-make-year-linux-desktop/ So, when we happened to be in some end-of-year/new year dinner parties or picnics with relatives & friends, and the conversations come up on topics like "What's Linux? What is this FOSS thing? I'm retiring and looking for a computer to write my memoir... or, I hate this Windows 10 thing, what I gonna do now ..." - I thought this piece can be a place we can go look for some inspirations ... The reason I bring this up is because for the last few days of the festive season, I'd got similar experience when during conversations, such questions were thrown at me, and I was like, Ur ... Where do I start? Enjoy! And may all have a Happy & Healthy New Year! Cheers, Wen

1 0

Re: /usr/bin/env
by Andrew Mather 23 Dec '16

23 Dec '16

> > Message: 5 > Date: Fri, 23 Dec 2016 15:12:05 +1100 > From: Craig Sanders <cas(a)taz.net.au> > To: Luv Main <luv-main(a)luv.asn.au> > Subject: Re: /usr/bin/env > Message-ID: <20161223041205.qbkvfszdiy4tpakh(a)taz.net.au> > Content-Type: text/plain; charset=us-ascii > > On Fri, Dec 23, 2016 at 02:44:28PM +1100, Andrew Mather wrote: > > Module files are generally set up by the admins, so they don't require > > anything more from the user than including the appropriate loading > > statements in their scripts. It's not unlike a wrapper script really. > > it sounds similar to (but quite a bit more advanced than) what i've done in > the past with wrapper scripts and collections of environment setting files > sourced (#included) as needed. > Yep. Pretty much. It's not uncommon in scientific computing to need multiple versions of compilers and various bits of software compiled against a range of different libraries and the like. You have to retain old versions of software, often long past its use-by date in case someone queries a scientific paper based on using that particular version. By using a chain of module load commands the user can easily set up an environment very different from the current OS state (apart from the kernel itself), repeatably, across an entire cluster if needs be. They can even swap environments around between various steps in a script if that is needed. Obviously it's overkill for some requirements and won't suit everyone's Modus Operandi, but well worth knowing about if that's the sort of thing you need to do. -- - https://picasaweb.google.com/107747436224613508618 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- "Voting is a lot like going to Bunnings really: You walk in confused, you stand in line, you have a sausage on the way out and at the end, you wind up with a bunch of useless tools" Joe Rios -

3 3

Every 2 Minutes cronjob
by David Zuccaro 23 Dec '16

23 Dec '16

This is my crontab file: # m h dom mon dow command */2 * * * * /sbin/scap 30 00 * * * /sbin/dzbu 30 06 * * * /sbin/dzbu 30 12 * * * /sbin/dzbu 30 18 * * * /sbin/dzbu 37 * * * * /sbin/cleanpng The first cron job (scap) does not seem to be being started. However dzbu does get started. Running scap from the command line works fine. Any ideas as to what I am doing wrong? Thanks David

6 18

Re: /usr/bin/env
by Andrew Mather 23 Dec '16

23 Dec '16

> > Message: 5 > Date: Fri, 23 Dec 2016 14:04:39 +1100 > From: Craig Sanders <cas(a)taz.net.au> > To: Luv Main <luv-main(a)luv.asn.au> > Subject: Re: luv-main Digest, Vol 64, Issue 15 > Message-ID: <20161223030439.alqgnoiollf6vuos(a)taz.net.au> > Content-Type: text/plain; charset=us-ascii > > On Fri, Dec 23, 2016 at 01:06:47PM +1100, Andrew Mather wrote: > > We use the "modules" environment (TACC's lmod implementation > specifically) > > for this type of thing. > > > > https://www.tacc.utexas.edu/research-development/tacc-projects/lmod > > > > It allows multiple versions of packages to exist without library > collisions > > and so on. Loading the appropriate modules allows the user to set up the > > execution environment and even swap between versions if necessary. > > this looks interesting. i'll have to read more about it but at first sight > it > seems like a specific language and system for setting up the environment > for a > particular program/script. > > one of the main reasons i prefer wrapper scripts (or symlinks) is that they > don't rely on undocumented and unknown settings (PATH, LDPATH, etc) in some > random individual's environment. scripts document those settings > explicitly. > symlinks just use the standard system environment. > > this has the huge benefit of NOT relying on fallible human memory, > resulting > in reproducible, auditable, and easily debugged software usage. also avoids > seemingly random breakage from changes to the environment - "why doesn't > this > work? it ran perfectly 3 weeks ago when i last ran it." > > > > i'm kind of surprised that a language with the slogan "explicit is better > than > implicit" is one of the main perpetrators of the #!/usr/bin/env > abomination. > > As Rodney mentioned, modules, or some variation thereof is quite common in HPC environments. Module files are generally set up by the admins, so they don't require anything more from the user than including the appropriate loading statements in their scripts. It's not unlike a wrapper script really. Through some of the other commands available, it also allows for querying of what modules and versions are available and what particular packages actually do. Our users are slowly getting used to this and beginning to check before asking for packages to be installed. One other advantage is that if configured, modules allows for logging, which can help in software management. -- - https://picasaweb.google.com/107747436224613508618 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- "Voting is a lot like going to Bunnings really: You walk in confused, you stand in line, you have a sausage on the way out and at the end, you wind up with a bunch of useless tools" Joe Rios -

2 1

Re: luv-main Digest, Vol 64, Issue 15
by Andrew Mather 23 Dec '16

23 Dec '16

> > Message: 1 > Date: Fri, 23 Dec 2016 08:11:15 +1100 > From: Sean Crosby <richardnixonshead(a)gmail.com> > To: Craig Sanders <cas(a)taz.net.au>, luv-main <luv-main(a)luv.asn.au> > Subject: Re: /usr/bin/env > Message-ID: > <CADg9O+Ncb-3DWJBdBc8mvBkYJYqitVcs+Wht5NxnYrXpg4K17g(a)mail.gmail. > com> > Content-Type: text/plain; charset="utf-8" > > On 23 December 2016 at 01:37, Craig Sanders via luv-main < > luv-main(a)luv.asn.au> wrote: > > > > > the one argument in favour of doing this (that the script will be run by > > the > > first matching interpreter found in the PATH) is both a blessing and a > > curse. > > at best it's a minor convenience. at worst, it's a potential security > risk > > - > > it's not an accident or an oversight that every unix system since the #! > > line > > was invented DOESN'T search $PATH for the interpreter. > > > I've taken to using /usr/bin/env a bit more because of the max length limit > in shebang lines. We store newer versions of Ruby, Python etc on a separate > filesystem, where there are many versions of these directories, and they > are hidden down quite far in the dirtree. So we regularly hit the max > shebang length limit of 128 characters. > > We use the "modules" environment (TACC's lmod implementation specifically) for this type of thing. https://www.tacc.utexas.edu/research-development/tacc-projects/lmod It allows multiple versions of packages to exist without library collisions and so on. Loading the appropriate modules allows the user to set up the execution environment and even swap between versions if necessary. I was a late conversion, but wouldn't be without now. -- - https://picasaweb.google.com/107747436224613508618 -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- "Voting is a lot like going to Bunnings really: You walk in confused, you stand in line, you have a sausage on the way out and at the end, you wind up with a bunch of useless tools" Joe Rios -

2 1

modules Re: luv-main Digest, Vol 64, Issue 15
by Rodney Brown 23 Dec '16

23 Dec '16

On Fri, Dec 23, 2016 at 1:06 PM, Andrew Mather via luv-main < luv-main(a)luv.asn.au> wrote: > > > We use the "modules" environment (TACC's lmod implementation specifically) > for this type of thing. > > https://www.tacc.utexas.edu/research-development/tacc-projects/lmod > > It allows multiple versions of packages to exist without library > collisions and so on. Loading the appropriate modules allows the user to > set up the execution environment and even swap between versions if > necessary. > > I was a late conversion, but wouldn't be without now. > > From Lev's talks, modules are used in most all the HPC environments so that versions of compilers and other tools can remain stable. So introducing lmod modules to students early may be worthwhile.

1 0

phone support
by Russell Coker 22 Dec '16

22 Dec '16

You have probably seen the news about GSM AKA 2G mobile phone support going away, today is the end day for Telstra GSM. You probably think (like I did) that any 3G phone will do, but that's not the case. The Samsung Galaxy S and the Sony Ericsson Xperia X10i don't support 3G in the 850MHz speed and Telstra specifically states that they don't provide full coverage on phones that only support 2100MHz. I wanted to give one of my old phones to a relative who only makes phone calls (no games or Internet use) so one of those 2 phones would have worked nicely except that he lives in one of those areas where Telstra doesn't support those speeds. As you apparently need 850MHz to get Telstra 3G (including Aldi and other Telstra resellers) in Melbourne suburbs a lot of older 3G phones will suddenly stop being usable. I don't know if Optus and Vodafone will have similar issues. It's possible that they could as Vodafone supports 3 bands including 850MHz and could possibly have some areas that are only well covered at 850MHz-3G and GSM speeds. Optus supports 900MHz and 2100MHz for 3G (which I think are more common frequencies) but there are some 3G phones which don't support 900MHz such as the Xperia X10a (which was sold by Telstra). If you bought a 3G phone from Optus/Virgin a few years ago (as I did with the Galaxy S and XperiaX10i) it might not be usable now on the Telstra network. If you bought a 3G phone from Telstra a few years ago if might not be usable on Optus depending on how they do things (which might not be well known for a few months). I've just given away my Galaxy S to someone who wants to use it as a security camera. I encourage anyone with Android phones in such situations to give them to the LUV hardware library. Even 5yo Android phones are nice little embedded Linux systems that can be used for running your own programs. -- My Main Blog http://etbe.coker.com.au/ My Documents Blog http://doc.coker.com.au/

4 4

pop3 email service provider
by zlinew9＠virginbroadband.com.au 20 Dec '16

20 Dec '16

Hi all, I am looking for an email service provider which offers pop3 to download email to a local machine. Mcmedia no longer providing this since they were bought by Bendigo telco I would also like this to be a constant email address that does not change every time I need to change ISP's. Privacy is also something of an issue and there seems to be some question on this issue with Gmail which I believe does allow the use of pop3. Lindsay BTW best wishs's to all.

8 9

OpenDKIM and Postfix configuration question
by Jason White 17 Dec '16

17 Dec '16

Postfix is configured on my server to use SASL authentication for messages submitted via port 587. OpenDKIM has been set up for both signing and verification. In /etc/opendkim.conf, InternalHosts refers to a file that lists various addresses, including the IPv6 range of my home network, as well as my domain name (".jasonjgw.net"). Consequently, messages submitted by hosts on my home network are signed by OpenDKIM. However, if, for example, I take a laptop to a different network and then attempt to send mail from it via port 587 on the server, it isn't signed, even though it has been authenticated with SASL in the Postfix session. Obviously, it doesn't match any of the addresses given in the InternalHosts file. Is there a way to configure OpenDKIM to sign messages from hosts that have SASL credentials, regardless of which network they're on?

2 2