[lug] Important backup lesson

[Sean Reifschneider]

On Tue, Sep 19, 2000 at 06:48:12PM -0600, Hugh Brown wrote:
>"Atkinson, Chip" wrote:
>> Before restoring from a backup tape, set it to be write protected!
>Sounds like the lesson was learned the hard way.  Ouch!

On that note, we've found it's relatively inexpensive to have a data recovery
house stream the data past the EOD off to another tape.  Though in our
case the place we chose was <ahem> competancy challenged.

They ended up sending the tape back UPS ground, then telling us they
couldn't recover it becase the tape was compressed.  It was, but on a
file-by-file basis, just as my letter included with the tape indicated
(and asked them not to try to recover the files, just the data after
the EOD mark).  When I finally got through to a recovery tech instead
of just the phone drones they have on the front-end, and they agreed
they could recover the data, I found the tape had already been shipped
out.  So we had to wait a week for UPS to get the tape back to us before
we could ship it back out to have the recovery done.

Part of the problem was that the tape was written with the "afbackup"
software, which the recovery house didn't have a set of scripts to
recover.  So they just threw up their hands...  I ended up writing my
own set of tools to recover the damaged archive we received in return.
Mostly I wrote a verification program so I could understand the format
well enough to speak intelligently with the data recovery techs, but
by that time it was a short leap and a hop to write the program to actually
do a recovery.

You see, afbackup is a fairly complicated system and can be kind of a
pain in the butt to set up, it's not like doing a "tar tf" on a file,
you have to set up the server and the client and have them interact.
Ours broke because of some subtle host-name difference.  So, I ended
up with a tool that WAS able to act like "tar tf".  ;-)

On DDS media, from what I was able to find, it's impossible to read
past the end of data marker on commercial units.  These tapes have a
header which includes how much data is on the tape.  The tape drive
reads this information on insert, and writes it on eject (and at
various other times I'm sure).  There's no way to "skip past" EOD
because the tape acts essentially like a file on your file-system.
You can seek and read, but when you get to the end of the file you
can't really just keep reading and get extra data.  The OS says
"Whoa!".

There was a rumor that you could begin writing to a tape, when you were
sure it was past the EOD you wanted to read past, yank the power.  I set
up a test of this, and when I tried to read it the tape drive just
freaked out when it got past EOD.  It wouldn't even rewind, all I
could do was eject it.

Everything you wanted to know about DDS backups and were afraid to
ask.

Oh, the other funny thing...  I of course (having been burned by it
in the past) always write-protect a tape before I try a recovery on
it.  The tape we needed to recover was, of course, write-protected
before sending it off to the data recovery house.  I received back
TWO tapes that were write-enabled.  The mind boggles.  I guess they
were hoping that I'd accidentally write to the recovered tape requiring
us to pay them to do the job again.  ;-)

Sean
-- 
 "I was on IRC once and got mistaken for Dan Bernstein. I still have
 nightmares."  -- Donnie Barnes
Sean Reifschneider, Inimitably Superfluous <jafo at tummy.com>
tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python