[lug] Max file size

[John Karns]

I would like to use dd to create a compressed image file of a vfat
partition which exceeds 2GB in size, and I bump into the limit which
aborts the dd process.  What I'm trying to do is write to a reiserfs
exported via nfs on another box where there is sufficient space.  Ideally
I would like to fragment the file to burn it to several CD's.  For a
minute, it looked like the dump utility was the answer, but it will only
read ext2 partitions.  I'm running SuSE 7.1 with a 2.2.20 kernel.  I have
already done a back up to tape, but the Adaptec pcmcia SCSI card I have
maxes out at 1MB/s, making it a bit painful.  I would also like something
a little more permanent and reliable than tape.

Seems like I remember reading sometime during the past year or so the max
file size limit in the kernel had increased substantially.  I waded
through some of the kernel source docs and headers and saw some reference
to a maximum size of something like 4TB.  A quick search of Deja brought
up the following:

--------------------------

The max file size is determined by the offset
used in fseek. Even though fseek has an offset of
a 64-bit number, the offset is only 32-bits for
compatibility purposes with older processors.
Also the offset is signed because an offset can
be used to go backwards from the current position:

        fseek(fd, -32L, SEEK_CUR);

...

You can change this to 63-bits by either using fopen64
and it's corresponding 64-bit functions or by
compiling source code with the -D_FILE_OFFSET_BITS=64

Also, the file system itself has limits on the maximum
file size. Minix is limited to 64 Megabytes while Ext2
is limited to 2 Gigabytes and xfs has a theoreticat
limit of 8388608TB.

--------------------------

Can anybody fill me in?  Do I need to make the target fs something like
ext3 in order to accomodate a file size greater than 2GB, or will ext2 /
reiserfs suffice?  Will simply recompiling dd with the
-D_FILE_OFFSET_BITS=64 parameter mentioned above do the trick?

Thanx.

----------------------------------------------------------------
John Karns                                        jkarns at csd.net