have a look at OMERO then. that's what it does - it's a metadata manager. (fundamentally though, any DB is a front end to a filesystem anyway...- even if it is it's own "special" FS) On Tue, Jun 14, 2016 at 4:12 PM, Craig Sanders via luv-main < luv-main@luv.asn.au> wrote:
On Mon, Jun 13, 2016 at 06:30:46PM +1000, Andrew McN wrote:
On 13/06/16 17:21, James Harper wrote:
I find that article unconvincing, but my concerns are mostly around performance. If performance is of no concern at all, then the atomicity/maintenance argument might be significant (though not the stuff about storing as a text data type).
I'm not convinced either. Same as I'm never convinced by the recurring idea of using an SQL database as a mail-storage backend.
IMO the best option is to use a database to store information about the image (path and filename or URI, size, width, height, geo-location, description, and whatever other details are required. Maybe even a small thumbnail image), whilst storing the actual image in either:
- a filesystem, with a hashed directory structure to avoid having too many files in one directory. e.g.
.../images/{00-ff}/{00-ff}/{00-ff}/imagefile.png
If you're worried about the image pathname getting out of sync with the database, you could write your own FUSE fs layered on top of the actual fs to automatically update the database if a file is renamed or moved.
BTW, there are FUSE modules for perl and python to make this easier if you don't want to use C.
- An object store like Amazon S3, Openstack's swift, or ceph.
BTW, there are existing FUSE modules for these that could be modified to update a database if an object is changed.
For a very large number of images, an object store is the best option...you get hugely scalable performance, redundancy, and storage capacity.
craig
-- craig sanders <cas@taz.net.au>
BOFH excuse #302:
microelectronic Riemannian curved-space fault in write-only file system _______________________________________________ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
-- Dr Paul van den Bergen