*nix data wipe tools

[Thomas C. Greene]

I've posted the final versions of a few simple, free shell scripts that i've
been working on to make data hygiene more convenient on *nix systems. Thanks
to list members who helped test them and contributed improvements.

Download them at http://basicsec.org/tools.html The file is called
LinuxWipeTools.tar.gz

The purpose here is to simplify regular maintenance. These tools are not
intended as substitutes for the wipe and shred utilities, which should always
be used on sensitive individual files. What i have here are backup tools that
will easily and securely wipe large areas of the disk that might contain data
traces you've neglected, or failed to eliminate properly.

The scripts are meant to clean large disk areas safely and conveniently while
you work with your system. They are intended for basic, regular maintenance:
i.e., to eliminate duplicate data traces in obscure areas of the disk, and
the remnants of files that have merely been deleted. There is nothing here
that you couldn't do from the command line: the idea is to make it convenient
so that you *will* do it. Often.

The WipeSwap script will automatically detect your swap device, stop it, wipe
it securely, and re-create it. This usually takes only 20-30 minutes. The
swap partition is a great accumulator of unforseen and/or forgotten data, and
should be wiped regularly. This makes it easy and safe.

The WipeFree scripts will securely wipe un-allocated disk space, where the
remnants of deleted files may remain. Again, this merely simplifies the
process.

Please see the README file for caveats and a more detailed explanation.

Thanks to the courage of numerous volunteers, i can say that the scripts
appear to work safely and effectively on a variety of Unix, BSD and Linux
systems.

Many thanks to Conrad Wood and David C. Niemi for improvements they
contributed, and to Jim Knopf for an important fix and several excellent
suggestions.

chrz,
t.

==============
Thomas C. Greene
Associate Editor
The Register
http://theregister.co.uk
http://basicsec.org

[David Cannings]

Thomas C. Greene wrote:
> I've posted the final versions of a few simple, free shell scripts that i've
> been working on to make data hygiene more convenient on *nix systems. Thanks
> to list members who helped test them and contributed improvements.

Is there any specific advantage of these scripts over bcwipe?

http://www.jetico.com/index.htm#/bcwipe_unix.htm

David

[Wietse Venema]

David Cannings:
> Thomas C. Greene wrote:
> > I've posted the final versions of a few simple, free shell scripts that i've
> > been working on to make data hygiene more convenient on *nix systems. Thanks
> > to list members who helped test them and contributed improvements.
>
> Is there any specific advantage of these scripts over bcwipe?
>
> http://www.jetico.com/index.htm#/bcwipe_unix.htm

There's a general problem with applications that go through the
file system to destroy the contents of a file. Unless one uses
very simplistic disk harware and file systems, there is no guarantee
that overwrite requests will actually overwrite the intended bits.
For a example, Solaris 10 ZFS uses copy-on-write, to avoid corruption
when the system crashes in the middle of an update; many disk drives
have write caches built-in so only the last overwrite request takes
effect; and non-volatile memory "disks" have a limited number of
write cycles and try to avoid hot spots.

Wietse

[Thomas C. Greene]

BCWipe for *nix is a bit complicated for novices. AFAIK, it does not
automatically do swapoff, mkswap & swapon, so some users will either not wipe
their swap device (most likely), or will wipe it and fail to get it working
again. My humble WipeSwap script -- all eleven lines of it -- will
automatically detect the swap device, turn it off, wipe it, re-create it, and
turn it back on for you. Because it's easy to use and safe, more users will
be encouraged to clean out that rat's nest of data traces now and then.

Also, BCWipe can be dangerous in some hands: a simple typo, and there goes a
working disk volume. My scripts are separate and clearly labeled. It's
impossible to destroy data with the WipeSwap or WipeFree scripts. Only the
WipeAll script will do that, and it's disabled by default. Thus, unless you
are at least an experienced user, you will have to read the README to get
that one working.

Furthermore, my scripts (http://basicsec.org/tools.html) are incredibly simple
and totally transparent. Almost anyone can understand what they do. You
don't have to take my word for *anything*.

OTOH, BCWipe uses a more secure wiping routine, they say. But once you've
gone from merely deleting to filling with several passes of random data,
there is little more to get out of a software solution, except a false sense
of security. The next logical level, really, is physical destruction.

Still, there is always room for improvement, and several list members have
made excellent suggestions for improving the scripts. I'll be posting
updated versions in a few weeks' time, as soon as i get them all tested on
enough different systems.

chrz,
t.

On Thursday 16 December 2004 15:06, David Cannings wrote:
> Is there any specific advantage of these scripts over bcwipe?
>
> http://www.jetico.com/index.htm#/bcwipe_unix.htm
>
> David

[Casper.Dik@Sun.COM]

>David Cannings:
>> Thomas C. Greene wrote:
>> > I've posted the final versions of a few simple, free shell scripts that i've
>> > been working on to make data hygiene more convenient on *nix systems. Thanks
>> > to list members who helped test them and contributed improvements.
>>
>> Is there any specific advantage of these scripts over bcwipe?
>>
>> http://www.jetico.com/index.htm#/bcwipe_unix.htm
>
>There's a general problem with applications that go through the
>file system to destroy the contents of a file. Unless one uses
>very simplistic disk harware and file systems, there is no guarantee
>that overwrite requests will actually overwrite the intended bits.
>For a example, Solaris 10 ZFS uses copy-on-write, to avoid corruption
>when the system crashes in the middle of an update; many disk drives
>have write caches built-in so only the last overwrite request takes
>effect; and non-volatile memory "disks" have a limited number of
>write cycles and try to avoid hot spots.

I seem to remember that the first version of "PGP wipefile" which
was written for DOS also made the assumption that file writes
were immediate. No fsync/sync or what not appeared in the Unix
port which made it pretty much without effect. (Overwrite it 10
times in the case, then unlink; don't think the kernel will bother
to flush the data even once).

ZFS is just one of several "special circumstances" that you may
encounter; flash memory devices, e.g., only pretend to be an array of bytes;
in actual fact overwriting a single block multiple times will likely
cause several different blocks to be overwritten; the original
block may not even be overwritten once as the flash memory controller
tries to extend the life of the memory by spreading writes evenly around.

We are in fact looking at ways to do "secure delete" as integral part
of the ZFS filesystems for those that require it (it obviously will
come at a performance price as you will need to do at least 5 I/O
ops per block to have some chance of actually erasing the data so
it's not likely going to be an "always on feature")

Personally I prefer a "lose key, lose data" approach to secure erase;
it's much easier to lose or securely erase a key than it is to
do so with a lot of data.

Casper

[George Georgalis]

On Thu, Dec 16, 2004 at 08:06:07PM +0000, David Cannings wrote:
>Thomas C. Greene wrote:
>> I've posted the final versions of a few simple, free shell scripts that i've
>> been working on to make data hygiene more convenient on *nix systems. Thanks
>> to list members who helped test them and contributed improvements.
>
>Is there any specific advantage of these scripts over bcwipe?

The introduction indicates it does swap and unused disk space.

http://basicsec.org/tools.html
Linux Wipe Tools: Four shell scripts for securely wiping all data from
the swap partition, wiping unused disk space on the root partition, or
wiping an entire disk, by Thomas C. Greene.

http://www.jetico.com/index.htm#/bcwipe_unix.htm
The BCWipe software is intended to give you a confidence that your
deleted files cannot be recovered by an intruder. BCWipe repeatedly
overwrites special patterns to the files to be destroyed.


// George

--
George Georgalis, systems architect, administrator Linux BSD IXOYE
http://galis.org/george/ cell:646-331-2027 mailto:george@galis.org