[lug] Clustering and GFS

[Alan Robertson]

Jeff Schroeder wrote:
> D. wrote:
> 
> 
>>But what is GFS? Is it the base filesystem, or is it a layer over the
>>real filesystem? For example, NFS is called a filesystem, but it
>>always has something else *under* it, e.g., an exported ext2
>>partition. Or is GFS natively a base filesystem that also does
>>something more?
>>
> 
> GFS is a layer over, say, ext3.  When installed on each client machine, 
> it provides a "seamless" filesystem interface so all clients can access 
> the same global disk (which may, in fact, be a single ext3 partition, a 
> RAID device, or even something like a solid-state storage device).

GFS is a *native filesystem*, not a layer over another filesystem.  It has 
64-bit block numbers, and 64-bit inodes, and 64-bit locks.  It sits right on 
top of the raw disks (or RAID controllers).  It has a very nice design, and 
handles true parallel updates from all the machines in the cluster at once. 
  No layer on top of another filesystem could do that.  There are lots of 
cache coherency issues that have to be dealt with.  It's quite an 
interesting problem from a technical perspective.

Each machine DMAs from the fiber channel disks for I/O.  There is NO central 
server, no single point of failure (if you design the FC setup correctly).

> Unfortunately, the hardware storage for this system is beyond my 
> control, and I'm at the mercy of the Powers That Be, who have decided 
> to use GFS... while I could propose alternative technologies (and even 
> a test period, as you suggested) it would probably appear that I was 
> only trying to endanger a solution they've already accepted.  I've been 
> asked to "integrate", not "redesign". ;)

GFS is unique in many ways.  It solves some very interesting problems that 
few packages attempt to solve.  GPFS from IBM, or CXFS from SGI and some 
other filesystems do this, but there aren't that many (CXFS != XFS).

> My original post on this topic was simply to find someone with 
> clustering (and GFS) experience... although we've reached quite a 
> tangent at this point.  Nonetheless, this is good information and very 
> useful.  Many thanks.

I've never used it, but I know a number of their people reasonably well.  I 
first heard about GFS when they were still at the University of Minnesota - 
probably 4-5 years ago.  They avoided the problem of lock servers, etc. by 
having locks in the disk controllers (dlocks).  It sounds like your client 
has already bought this hardware.

	-- Alan Robertson
	   alanr at unix.sh