Virtualisation can help a lot. The ability to snapshot a server version along with the data it analysed is probably the right approach (and a relatively small footprint, since if you are doing it right, you are taking snapshots anyway) 

But you still have to set it up... 


Research data is interesting. Massive volumes, relatively short "live" time frame before it can be migrated to slower storage. Tiered storage solution help a lot. Archiving to tape is viable but really need a way for it to be automatically accessed (by users without sysadmin involvement - less work, more likely to be used, and users test the archiving validity constantly... ) 

On 25 Dec 2016 11:44 pm, "Craig Sanders via luv-main" <luv-main@luv.asn.au> wrote:
On Sun, Dec 25, 2016 at 04:56:05PM +1100, Paul van den Bergen wrote:
> Funny, I was asked about exactly the same problem when I started
> @WEHI... only there was no attempt made to even start tackling the
> problem...

yeah, we were constantly getting individual academics and research
groups asking us about storage, and then trying to do the best we could
with minimal resources.

the unfortunate fact is that disks/storage arrays and file-servers
and tape libraries etc are expensive. You can replace a very large
percentage of your up-front capital expense with skilled techs, which
are an on-going cost (you're going to need them to look after expensive
equipment anyway, and it has to be maintained & upgraded for 7+ years),
but it's still going to cost a lot for huge data storage anyway, even if
you avoid over-priced name-brand gear.

> Virtualisation of workload makes the problem a lot easier to tackle,
> but even so... 7 years is a long time in IT...

cheap big disks helps a lot too. but you need a lot of them, plus
backup - on-site and off-site.

CPU & RAM are more than adequate for pretty nearly any file-storage
needs these days...could always use more of both for computational
stuff.

craig

--
craig sanders <cas@taz.net.au>
_______________________________________________
luv-main mailing list
luv-main@luv.asn.au
https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main