[lug] ftp problem

[George Sexton]

There are known problems with using compression options in tar. It's safer
for compression to use an external program that compresses the tarred data.
As I recall, it was some sort of padding issue that happens. The tar archive
doesn't get padded to the correct multiple length when using compression.

It could be this is a bzip2 only problem, or it could be any compression
program. I tried finding my references again, and I can't. But, this is a
real problem.



George Sexton
MH Software, Inc.
http://www.mhsoftware.com/
Voice: 303 438 9585
  

> -----Original Message-----
> From: lug-bounces at lug.boulder.co.us 
> [mailto:lug-bounces at lug.boulder.co.us] On Behalf Of Gary Hodges
> Sent: Tuesday, March 08, 2005 10:36 AM
> To: Boulder (Colorado) Linux Users Group -- General Mailing List
> Subject: [lug] ftp problem
> 
> Thought I would run this by the experts here...
> 
> I have a Perl script running on two remote machines that tars and 
> compresses a directory of data and uses the Net::FTP module 
> to send the 
> file to a machine here in our lab.  On one machine (both are Debian 
> Sarge) it has been running fine since the first of the year.  
> I just set 
> up the same script on the second machine and the uploaded 
> files often, 
> but not always, have problems.  When a problem file is 
> uncompressed and 
> untarred it gives the following error:
> 
> gzip: stdin: unexpected end of file
> tar: Read 9089 bytes from 20050307.tar.gz
> tar: Unexpected EOF in archive
> tar: Unexpected EOF in archive
> tar: Error is not recoverable: exiting now
> 
> It does extract files, but many files from the day of data 
> are randomly 
> missing.  That is, we'll have something like
> 
> 20050307182100.jpg
> 20050307182200.jpg
> 20050307182600.jpg
> 20050307182700.jpg
> 
> Notice that minutes 23 through 25 are missing.  The missing 
> files exist 
> on the remote machine and are part of the created tar file, 
> but during 
> the transfer they are lost.  The machine in question is 
> located behind a 
> rather strick DOE firewall.  Getting permission to set this 
> machine up 
> took forever.  The hard part was getting permission to 
> remotely access 
> it, but I was told that sending data out from this machine 
> shouldn't be 
> a problem.
> 
> Is it possible the firewall is stripping files out of the tar 
> archive or 
> should we look somewhere else?
> 
> Thanks,
> Gary
> 
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: lug.boulder.co.us port=6667 channel=#colug
> 
>