[Coldstuff] DB backup issues when compressed...

[Brad Roberts]

On Mon, 18 Feb 2002, Brandon Gillespie wrote:
> On Mon, Feb 18, 2002 at 08:20:09PM -0800, Jonathan Robertson wrote:
> > Ok.. I've gotten myself into somewhat of a problem.
> > Recently I created my own backup script as part of a
> > shell script and I scheduled the backup to run every
> > hour and to reschedule itself every day... On to the
> > problem...
> >
> > Last thursday Genesis stopped running on my server for
> > some reason or another, and as you probably well know
> > if it doesn't close properly the DB will be corrupted
> > which it was.  I got on the server and attempted to
> > restore the most recent backup (only 20 minutes old)
> > and it appeared to restore properly from the console.
> > Fired up Genesis and got the "Corrupt Binary Database"
> > error message.
>
> Most likely the backup itself wasn't marked as clean... there is a way
> to cheat genesis (but it isn't necessarily stable) by copying the
> .clean file from a good binary db into the one genesis is claiming is
> 'corrupted'... Basically, it isn't really doing any checksums or
> anything, just looking for the .clean file.
>
> If you do that, I'd recommend copying in the .clean file, running a
> decompile and then a recompile.

As always when experimenting with untried stuff, make a copy of the binary
directory before you play around.  What Brandon suggests is safe, but when
you've only got one copy of the file like you do right now, its a good
idea to have a fallback.

> > My script compresses the
> > /cold/binary.bak folder.  The command I used was
> >
> > tar -czf /cold/backups/BU.$(date...) /cold/binary.bak
>
> Perhaps you did it before genesis was done backing it up?

Is the backup script being executed from $sys.backup_done or from some
external event like cron?  Its hard to tell from the original email.

> > 4. Is there a way to look at a "binary" DB and edit it,
> > or get parts of it to decompile to a textdump file?
> yes, you can mangle the snot out of a binarydb using the 'coldcc -p'
> (patch) argument.  It uses the exact same syntax as a textdump but
> rather than create a new db, it opens an existing one.  Just remember
> to not use 'new' because that'll blast the existing object, if there
> is one (i.e. new object $root bad, just do 'object $root' to load the
> root object).
>
> >From there use 'eval' which is like a non-method:
>
> eval {
>     ColdC code
> };
>
> Checkout the test suite to see some extensive uses of eval.
>
> As for decompiling only parts of it, no luck there... it is all or
> nothing.  I had considered back in the day having an exclude/include
> argument to coldcc, where you could give it object numbers to skip
> (for the cases where the db is corrupted) or only decompile (for the
> cases where you just want to dump one object).  Feel free to dig into
> textdb.c and add it as an option, I'm sure Bruce wouldn't mind the
> patch :)

Much like my comment for the copied .clean file.. before performing this
sort of task on your binary file, since its a rather low level and
operator beware sort of task, do make a copy of the binary dir, just to be
safe.

As to coldcc patches, sure.. the more crash recovery features the better.
This past weekend I added the ability to dump a single object to
facilitate recovery of a corrupted binary file.  I'm still working on a
more useful version of this patch though as its just /way/ to slow to call
coldcc for each object.

The bottom line is that backups are the right answer, but at times, heroic
recovery efforts are called for.  If dropping the .clean file in there
doesn't do the trick, ask for more help.

Later,
Brad