[lug] Cluster File Systems

Fri Aug 3 14:55:40 MDT 2018

... and swift:
https://www.swiftstack.com/product/open-source/openstack-swift

On Fri, Aug 3, 2018 at 2:54 PM Steve Webb <bigwebb at gmail.com> wrote:

> I think that I would need to know a few things before I could answer what
> a good cluster filesystem would be for the task.  You already covered the
> standard answers: NFS, GlusterFS & Lustre.  Do you need support for dozens,
> hundreds or thousands of clients?  Is there a NAS available for shared
> storage or is it truly distributed storage (spanning multiple hosts)?  What
> is the app that's running that needs access to > 10G files?  video?  Large
> databases?  Weather Model?
>
> NFS would be my go-to if it handles the workload.  Pretty stable, runs on
> tons of platforms.  Could be a SPOF though in production environments.
> GlusterFS would be a good solution if the storage is truely distributed
> and you don't want to do much cluster management.
> Lustre is the go-to for many clients - good for large parallel computing
> platforms.
> GFS is awesome, supported by RedHat/CentOS, but requires a SAN or central
> storage.
> Ceph is awesome, but requires a bunch of machines for infrastructure (is
> mainly an object storage system but has a filesystem driver for it)
>
> If this is in AWS, you could look into EFS (AWS's version of NFSv4) or an
> S3-based FUSE wrapper.
>
> - Steve Webb
>
> On Fri, Aug 3, 2018 at 10:29 AM Bear Giles <bgiles at coyotesong.com> wrote:
>
>> 1. Do you need full filesystem support or is it enough to be able to read
>> and write files programmatically? This could be a relative minor change or
>> a huge headache, of course.
>>
>> 2. Do you only need to read and write files sequentially, or do you need
>> to be able to move within the file, update the file in-place, etc.?
>>
>> If both conditions are met then hadoop (HDFS) could be well-supported
>> solution. However it's definitely not a solution if you need to be able to
>> treat it like a regular filesystem or can't tweak code.
>>
>> On Fri, Aug 3, 2018 at 6:46 AM, Davide Del Vento <
>> davide.del.vento at gmail.com> wrote:
>>
>>> You are very welcome.
>>> I suspect you can make it work with CentOS and perhaps even with Fedora
>>> or other distros, but if you have easier routes....
>>> The only other option I know is IBM's spectrum scale, which is
>>> proprietary and so expensive you do not even want to know the price....
>>> Keep us posted!
>>> Davide
>>>
>>> On Fri, Aug 3, 2018 at 1:14 AM, Lee Woodworth <blug-mail at duboulder.com>
>>> wrote:
>>>
>>>> On 08/02/2018 05:33 PM, Davide Del Vento wrote:
>>>>
>>>>> I'd suggest you take a look at http://lustre.org/
>>>>> I don't know how it is from an administrative perspective, but it
>>>>> certainly
>>>>> can do what you ask. It might be overkill, but that's your call.
>>>>>
>>>>
>>>> Thanks for the suggestion. It appears to require kernel patches and
>>>> using
>>>> RHEL for the server (Lustre Support Matrix only has RHEL for the
>>>> server).
>>>>
>>>>   http://lustre.org/getting-started-with-lustre/:
>>>>     Metadata and Object Storage Server require the Lustre patched Linux
>>>> kernel,
>>>>     Lustre modules, Lustre utilities and e2fsprogs installed. The
>>>> clients
>>>>     require the Lustre client modules, client utilities and, optionally,
>>>>     the Lustre patched kernel.
>>>>
>>>>
>>>>
>>>>> On Thu, Aug 2, 2018 at 5:01 PM, Lee Woodworth <blug-mail at duboulder.com
>>>>> >
>>>>> wrote:
>>>>>
>>>>> Can anyone recommend an open-source cluster file system
>>>>>> that can handle doing lots of reads/writes to a remote
>>>>>> file, especially >10GB files? Would like to have a
>>>>>> mix of host architectures and not have to setup a full
>>>>>> cluster + management.
>>>>>>
>>>>>> A few years ago I used glusterfs with the native fs driver
>>>>>> in the kernel. I would get hard kernel-level lockups
>>>>>> in the above scenario (client reboot was the only way
>>>>>> to recover).
>>>>>>
>>>>>> Even nfsv4 servers locked up, though I haven't tried doing
>>>>>> that in awhile.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Web Page:  http://lug.boulder.co.us
>>>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>>>> Join us on IRC: irc.hackingsociety.org port=6667
>>>>>> channel=#hackingsociety
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Web Page:  http://lug.boulder.co.us
>>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>>> Join us on IRC: irc.hackingsociety.org port=6667
>>>>> channel=#hackingsociety
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Web Page:  http://lug.boulder.co.us
>>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>>> Join us on IRC: irc.hackingsociety.org port=6667
>>>> channel=#hackingsociety
>>>>
>>>
>>>
>>> _______________________________________________
>>> Web Page:  http://lug.boulder.co.us
>>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>>>
>>
>> _______________________________________________
>> Web Page:  http://lug.boulder.co.us
>> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
>> Join us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20180803/76e3ba67/attachment.html>