[lug] Convertion of ext4 Sparse Files

[stimits at comcast.net]

Where is the "--sparse=3Dnever" from? My Fedoras all use just "--sparse=never", no "3D"...the same goes for an Ubuntu 16.04 system I have. I've never seen the "3Dnever" version.
 
FYI, I can see the allocated block size ("ls -ls") versus written size ("ls -l") and verify my partition image has sparse ignored and no matter what I do cp and dd always copy this verbatim without any expansion. If I start with a file created from truncate I do get the same results, but the tools seem to understand an empty file.
 
PS: I really should get rid of my comcast web email, it makes it hard to reply on lists. :(
 
----- Original Message -----From: Aaron D. Johnson <adj at fnord.greeley.co.us>To: Boulder (Colorado) Linux Users Group -- General Mailing List <lug at lug.boulder.co.us>Sent: Fri, 05 Jan 2018 23:16:56 -0000 (UTC)Subject: Re: [lug] Convertion of ext4 Sparse Files

stimits at comcast.net writes:> Unfortunately this does not work (currently testing on Fedora 27),> at least not for this file which was originally a dd of a GPT> partition and then converted to sparse at a later time. Output and> input files are both the sparse file size.>

> It may be a matter of dd or other tools not detecting this as sparse> for whatever reason, but I keep running into options for creating> sparse files and not t he reverse, e.g., dd has "conv=sparse", but> not the reverse, so I can't force it to do a sparse to non-sparse> conversion.

GNU coreutils is being entirely too smart for you. I didn't know itdid this, but here's what happens on one of my machines:

$ truncate -s 1G sparse-file # make 1Gibyte sparse file $ cp sparse-file this-should-be-a-sparse-file $ cp --sparse=3Dnever sparse-file not-sparse-file # copy to non-spa=rse file =

$ ls -ls *sparse-file 1048580 -rw-r--r-- 1 adj adj 1073741824 Jan 5 15:57 not-sparse-file 0 -rw-r--r-- 1 adj adj 1073741824 Jan 5 15:56 sparse-file 0 -rw-r--r-- 1 adj adj 1073741824 Jan 5 16:07 this-should-be-a-sparse-=file $

A sparse file isn't a special type of file (at least on UNIX-yfilesystems -- I think it might be special on NTFS.) Being sparsejust means that not all of the data blocks have been written to andthe filesystem nicely doesn't allocate on-disk space for thoseunwritten blocks. Reads from offsets that haven't been written towill return all zeroes. Writing zeros on your own will allocateblocks. (At least on ext[234] filesystems; ZFS knows better.) The"--sparse=3Dnever" option on the cp above tells it "write all the zeroesout, don't just seek to the next non-zero block"

Hope this helps.

- Aaron_______________________________________________Web Page: http://lug.boulder.co.usMailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lugJoin us on IRC: irc.hackingsociety.org port=6667 channel=#hackingsociety
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20180105/30122d56/attachment.html>