PDA

Bekijk Volledige Versie : Re: [BugTraq] Peter Gutmann data deletion theaory?



Robin Whittle
23/07/05, 02:05
Peter Gutmann's 1996 paper is at:

Secure Deletion of Data from Magnetic and Solid-State Memory
http://www.treachery.net/~jdyson/infosec/secure_del.html

I will discuss four types of memory: Magnetic media (tape, hard-drive,
floppy etc.), Static RAM, Dynamic RAM and FLASH (or EPROM or EEPROM).

Magnetic media typically involves a flat recording surface, with a read
head, write head and sometimes an erase head before the write head. The
width of the read head may be less than that of the write head, so that
slight misalignments in reading or writing still cause the read signal
to be picked up from within the wider written band.

Magnetic media probably has some kind of depth effect too - it is a
three-dimensional object which is magnetised in one step and read in
another. Unless all trace of previous writes can be removed, some
remnant of the original signal will remain. To what extent this can be
read - either by the standard read system or by fancy forensic
techniques - would vary enormously.

Any recovered original signal will be only a small fraction of the
actual read signal. The rest will be noise, the last recorded
signal, and remnants of other recordings in the past. There needs to be
a certain signal-to-noise ratio before data can be recovered in any
meaningful manner.

I can't imagine how these forensic techniques work with modern hard
drives, where data is packed so densely and recorded with highly complex
encoding systems.

However, to ensure the destruction of previously written data, I think
its easier to physically destroy the media than do enough research to
prove beyond doubt that there is no possible recovery technique.

Probably what I have written above also applies in principle to optical
media as well.

Semiconductor memory (and core memory) is completely different. There
is no three-dimensional recording medium - so there is no misalignment
between conventional read and write operations. There is a single
quantity which is used to encode a bit of information - the voltage of a
capacitor (Dynamic and Flash RAM) or of one side of a flip-flop (Static
RAM).

A Static RAM cell consists of a four transistor flip-flop with two
additional components, either transistors or resistors, as loads for
each side. Each side is an inverting amplifier, usually made with an N-
and a P-channel Field Effect Transistor. Reading involves switching one
or both of the sides of the flip-flop (using additional transistors) to
bit lines, and then reading these lines. Writing involves a similar
connection to the bit lines, but then forcing the state of the flip-flop
to one state or the other.

At the end of a write operation, the flip-flop is constantly powered and
is free from external intervention. Each inverting amplifier constantly
inverts its input and forms the input to the other. This constant
amplification would quickly (fractions of nanoseconds) move the voltage
of one side towards ground and the other towards the power supply
voltage. I can't imagine any detectable difference in the physical
state of the flip-flop due to its previous state surviving more than
tiny fractions of a nanosecond. Since all these flip-flops are
operational as soon as power is applied, any charge states after the
chip was turned off would soon be over-ridden by the amplifying nature
of the flip-flops. The question is whether the chip could be powered up
in a way which would amplify slight charge differences remaining in the
chip since it was last operating. Such differences would need to be
significant compared to the natural bias in each cell towards powering
up one way or another, due to atom-scale differences in dimensions of
channels, gates and doping. This seems unlikely to me, but I am not a
semiconductor engineer.

I recall reading, via some third-hand account, of some very old memory
chips from the 1970s which could have their internal structure changed
by the sub-micron voltage gradients over long periods of time spent with
a particular data pattern, and this could be perceived in the power-up
state of each bit, as the usual randomness was skewed by these
particular changes.

Peter's paper discusses changes to the device due to repeated writes of
1 and 0. Only a semiconductor engineer would be able to advise how
valid this theory is, and whether any such changes could possibly be
detected by powering-up the chip in a special way, or by some other
external test mode or invasive microscopical technique.

Dynamic RAM involves a single transistor connecting a single capacitor
to a single bit line, to which many other capacitors could also be
connected - but only one at a time. Reading involves measuring the
voltage of the bit line to discern whether it is high or low. In
practice, this involves a non-inverting amplifier connected to the bit
line, with its output connected to its input, and therefore to the bit
line. There is a pre-charge system to bias the amplifier in the middle
of the range of voltages before connecting the bit line and turning it
on, so that, for instance, in a 5 volt DRAM cell (now they are all 3.3
volts or less) any bit-line voltage above 2.5 volts will be amplified up
to 5 volts (in a nanosecond or so) and any voltage below this will
likewise be amplified to 0 volts.

Thus a read involves a "refresh" - forcing memory cells which are deemed
to be "low" to (or strongly towards and usually very close to) 0 volts
and those which are deemed to be "high" to 5 volts.

Since capacitors leak, DRAM systems repeatedly read and refresh all the
capacitors, such as every 16 milliseconds. (Dipping at random into a
1990 Hitachi memory data book. Nonetheless, I recall interrupting
refresh in a Z80 CPM system and having the data, or at least enough of
it to run the system, survive for 10 seconds or so.) Each such refresh
or read effectively erases any detail of the prior voltage of the
capacitor. There might be some extremely small difference in the final
voltage of the capacitor in the following situations:

1 - Initially 5 volts, then written to "low", so almost 0 volts, with
just a little of the residual charge, due to limited resistance of
the bit line and the limited time of the write cycle.

2 - Initially 0 volts then written or refreshed to "low". This would be
effectively 0 volts - there is no other charge for there to be a
residual.

3 - Initially 2.4 volts, so deemed to be low, but refreshed or written
to low. (There's no reason in ordinary operation for a capacitor to
every be at this voltage, unless it was left to leak due to there
being no refreshes for a long time).

So I can't see how any trace of previous states could possibly be
measured, including by directly probing the capacitor by some means -
rather than by using the sense amplifiers, which are only making a
decision about whether the voltage is above or below 2.5 volts - after
one or more read, refresh or write cycles.

Perhaps keeping a DRAM chip for weeks, months or years with a particular
state of data, as may well happen, could result in long-term changes in
the exact physical nature of the semiconductor material, insulators etc.
of the capacitor and its immediate surrounds. However, unlike Static
RAM, I can't see how this could be measured (except by some direct
probing of the chip, which seems highly impractical, since such probes
would be noisy compared to the slight changes) because the base state of
the capacitors at power up is 0 volts or close to it, and the sense
amplifiers can only discern fine differences in voltage around the 2.5
volt range.

So I think that short term storage of data in Static or Dynamic RAM
leaves absolutely no detectable trace after it has been over-written. I
can't imagine how long-term storage patterns could be detected in DRAM
and I think it is probably impossible, by even the most heroic means, to
discern any such long-term patterns in SRAM.

There are two classes of data recovery here:

1 - Turning the chip on in a special way to discern its state when it
was turned off.

2 - Trying to sense long-term changes in the device to find out what
data it has stored for long periods of time.

The first approach might involve removing the chip from its system by
desoldering it (or maybe unplugging a SIMM) or by cutting the PCB and
making low-temperature bond connections to the chip to it in a forensic
test rig. Alternatively, the device may be left in the system and
powered up with different signals being forced onto the memory chip to
stop the usual system operations, which usually include erasing DRAM or
SRAM (except for battery backed up SRAM, in which case the previous data
is easily readable).

So maybe someone could build a special system to plug a DRAM SIMM into
in order to detect tiny charges remaining in the capacitors from the
state the chip was in when turned off. There could be many reasons why
this is impossible, not least the highly complex nature of modern DRAM
chips, with their fancy protocols, compared to the much more direct
nature of chips from the 70s and 80s.

Flash, EEPROM and EPROM memories also use a single capacitor, but this
is not connected or sensed directly by using a transistor to connect it
to a bit line. The aim of data recovery in these cases is to find what
data was stored in the chip prior to the data which was most recently
written. Flash memory has block erase modes, so it is possible that an
area has just been erased, without new data being written.

The capacitor of each memory cell is a small island, typically of
silicon, entirely surrounded by quartz or some other insulator. Its
voltage (Flash and EEPROM) is raised and lowered by high voltage
temporary breakdown (quantum mechanical tunnelling of electrons) through
the insulator. The capacitor's voltage is sensed by it part of a gate
of a MOSFET - so it can be read without any current entering or leaving.
EPROM capacitors are charged and read as described above, but they can
only be discharged by illumination with short wavelength UV light, which
gives individual electrons enough energy to tunnel through the
capacitors insulator.

I expect that in all these cases programming and erasure is not an
exactly complete process - that some trace would remain of the previous
state. To what extent this could be used it hard to say, but perhaps
there is a way in some cases, for instance with a FLASH or EEPROM
memory. Lets say the chip was programmed with some data we wish to
recover. Then lets say it was all erased to "1". Now we power up the
chip and try to discern individual voltages of each cell. Maybe there's
a way of increasing the power supply voltage so that the sense
amplifiers are discerning fine voltage difference in the range of the
normal "1" voltage. (Flash, EEPROM and EPROM typically use internal or
external power supplies to create voltages much higher than the standard
power supply rails used for reading, so the exact voltage of the
capacitor, may be rather high. What counts is the much higher power
supply voltage of the cell or read amplifier which is needed to read a
"0" when the cell is actually programmed as a "1". Then by changing the
power supply voltage slightly, multiple reads can be used to finely
measure the supply voltage at which this cells reads as a 0 instead of a
1. Maybe we could discern some variations which would have something to
do with the original data. I can't see how this could work if the
capacitors were written to 0 volts.

Unlike DRAM and SRAM, Flash etc. has no continual or repetitive
amplification function. Traces of previous charges may remain, but I
doubt they would be recoverable, except perhaps by the most drastic
forensic techniques - and even then, each write or erase operation would
reduce the remnants further still.


- Robin http://www.firstpr.com.au

Thor
23/07/05, 02:25
I *love* nuking CD's.... But apparently, the practice is no longer
"officially" supported due to the toxic emmissions of the Phthalocyanine
contained in the sublimation layer (and other chems)

Great fun, though.

t


----- Original Message -----
From: "Jay D. Dyson" <jdyson@treachery.net>
To: "Bugtraq" <bugtraq@securityfocus.com>
Sent: Thursday, July 21, 2005 11:46 AM
Subject: Re: Peter Gutmann data deletion theaory?


> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Wed, 20 Jul 2005, Jared Johnson wrote:
>
>> It seems that the perhaps the only real way to rid your Hard Drives of
>> data is to burn them.
>
> The sophistication of data recovery in LEAs and TLAs is such that drives
> that once held classified data are routinely shredded and slagged.
>
> But for those of us who handle sensitive and personal data (and,
> unfortunately, don't have access to a smelter), having a kiln, a drill
> press, and a belt sander is almost as good. :)
>
> As for retiring CD-ROMs, I just do what I learned from the DoD:
> http://www.treachery.net/~jdyson/infosec/dod_cd_declassification.html .
>
> - -Jay
>
> ( ( _______
> )) )) .-"There's always time for a good cup of coffee"-. >====<--.
> C|~~|C|~~| \----- Jay D. Dyson -- jdyson@treachery.net -----/ | = |-'
> `--' `--' `---- You don't want to make me be myself. ----' `------'
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (TreacherOS)
> Comment: See http://www.treachery.net/~jdyson/ for current keys.
>
> iD8DBQFC3+2XxzN3WIW0edsRAieqAJ4xWpeJGJ4qxNtgLAhfXQ J5qp7pAACgqeSu
> 6EuufQazjcA+3z2hSn5XZY4=
> =gD4T
> -----END PGP SIGNATURE-----
>
>

Robert Thompson Jr.
23/07/05, 03:05
"Do you all agree with Peter Gutman's conclusion on his theory that data
can never really be erased, as noted in his quote below:"

Absolutely...

If you have ever done any form of data recovery, you will see how much
information is recoverable, with just basic tools off of the internet.
If you haven't, just google "data recovery", find almost any program
with a free demo and take a hard drive, catalog it, format it (after
backing up what you need of course) then recover it. Watch how much
information you retrieve. Should be all of it, and then some.

I recall the first time I ever did a recovery from a hard drive that had
something off happen to it. I pulled up information on that drive from
back when it was first used. YEARS before...

That is just with a basic program off of the internet.

With wiping/sanitizing of your hard drives, you have elimiated having to
worry about any mediocre programs doing any data recovery, but "good"
programs or hardware recovery is still an option. The software recovery
will eventually fail if you are careful enough...

Now imagine what a hardware based recovery could pull off?

I would recommend using the sanitizing products as they will help keep
the people that don't have the time or money from locating anything on
your box, but for those out there that have the money or have the time,
they will be able to get just about anything off of your disk.

To keep your drives completely secure, you have two choices: either
don't use them, ever... OR physically destroy them when you are
finished.

Rob.

-----Original Message-----
From: Jared Johnson [mailto:jaredsjazz@Yahoo.com]=20
Sent: Wednesday, July 20, 2005 4:49 PM
To: focus-ms@securityfocus.com
Cc: bugtraq@securityfocus.com
Subject: Peter Gutmann data deletion theaory?

All,

Do you all agree with Peter Gutman's conclusion on his theory that data
can never really be erased, as noted in his quote below:

"Data overwritten once or twice may be recovered by subtracting what is
expected to be read from a storage location from what is actually read.
Data which is overwritten an arbitrarily large number of times can still
be recovered provided that the new data isn't written to the same
location as the original data (for magnetic media), or that the recovery
attempt is carried out fairly soon after the new data was written (for
RAM). For this reason it is effectively impossible to sanitise storage
locations by simple overwriting them, no matter how many overwrite
passes are made or what data patterns are written. However by using the
relatively simple methods presented in this paper the task of an
attacker can be made significantly more difficult, if not prohibitively
expensive."

It seems that the perhaps the only real way to rid your Hard Drives of
data is to burn them.=20

I'd love to hear some thoughts on this from security and data experts
out there.

Ron van Daal
27/07/05, 17:55
> We were not allowed to do a seven pass government wipe to dispose of the drives as our security people deemed it inadequate, we turned them over to our classified waste people who stored them until there were enough to justify having the platters remove
d and mechanicaly beaten into little lumps of metal.

Aren't you being too paranoid? I think a simple zeroing out of your entire
drive using dd(1) starting with the first sector is enough to cover your
privacy. I don't know about other ""secret"" government agencies in NL or
other counties who actually do microscopic magnetic recovery efforts, but
dd(1) does the trick to defeat disk analysis by our national digital crime
unit. From what I've read in one of their internal memo's is that they just
use a hexdump(1) alike utility to find any non-zero bytes on the drive to
conclude "the drive has been wiped entirely".

As far as I know will our National Forensics Institute not go any further.
To be more precise: most disks analyses are being done automaticly rather
than by hand (which is even more the case with the digital crime unit).

For this they use registry-catalogs, browser cache/cookie/history inventory
programs, raw disk searching on strings, and the like. Which is pretty
logical as disk sizes are rapidly increasing, making the analysers' job
pretty difficult because of the ever increasing haystack. While data hiding
techniques continue to develop - making the needle even harder to find.

Grt,

Ron van Daal

Andreas Beck
27/07/05, 19:25
"Robert Thompson Jr." <rthompson@columbiabank.com> wrote:
> If you have ever done any form of data recovery, you will see how much
> information is recoverable, with just basic tools off of the internet.

It's just that way, if you don't take any care deleting your data.


> with a free demo and take a hard drive, catalog it, format it (after
> backing up what you need of course) then recover it. Watch how much
> information you retrieve. Should be all of it, and then some.

This is not the case, if you follow a proper procedure. The effect of
"formatting" a harddisk is grossly overestimated by the average user -
probably due to its historic effect on floppy disks.
The same is true for "deleting".

Both operations usually only change a very small part of the harddisk.
For efficiency reasons. Formatting usually only deletes tables of free
blocks, root directory and some management information.
Deleting usually only removes the directory linkage and evetually frees
up the disk space, if no hardlinks are present, but doesn't touch the
data itself.


However, while it is pretty hard to securely delete data on modern
filesystems, if the filesystems were not designed to do this themselves,
it is relatively easy to destroy almost any data when wiping entire
drives.

Try your above experiment after you have not merely "formatted" the
disk, but rather wiped it with even a single pass of
dd if=/dev/zero of=/dev/[harddiskdevice]

This will render almost any attempt of software recovery useless. The
only data that should be recoverable by software tools is old weak data
from mapped out sectors and the like. This requires specialized software
that talks to the drives on a pretty low level, but is doable. Of
course, only very small amounts of data should be recoverable.
Just look at the mapped out sector counts from the SMART data of old
harddisks. You'd be lucky, if you find a few hundred sectors.


> I recall the first time I ever did a recovery from a hard drive that had
> something off happen to it. I pulled up information on that drive from
> back when it was first used. YEARS before...

Sure. But that data was never deleted in a secure manner.

> With wiping/sanitizing of your hard drives, you have elimiated having to
> worry about any mediocre programs doing any data recovery, but "good"
> programs or hardware recovery is still an option.

Any software recovery of a properly wiped drive will only have very
limited success.


> Now imagine what a hardware based recovery could pull off?

IMHO: Not so much more. Modern harddisks have such a high density, that
those "off track reading" and "remanent magnetism" arguments don't quite
hold. If the signal from there were useable with a reasonable amount
of hardware cost, it would be used to put more data on the media.

Are there any public studies about what commercial data recovery providers
can achieve after a harddisk was overwritten with a single sweep of
zeroes?


> I would recommend using the sanitizing products as they will help keep
> the people that don't have the time or money from locating anything on
> your box, but for those out there that have the money or have the time,
> they will be able to get just about anything off of your disk.

I doubt that, but if you think your data is valueable enough to make
such an attack feasible, I'd rather not recommend your choices:


> To keep your drives completely secure, you have two choices: either
> don't use them, ever... OR physically destroy them when you are
> finished.

but recommend to encrypt your sensitive data.

Reason: If you data is valueable enough to spend a few thousand dollars
to pull it off a discarded harddrive, it is almost certain, that
you need to spend less and gain more by getting the drive right from
your office while it is still in use and no deletion has been attempted.


Kind regards,

Andreas Beck

--
Andreas Beck
http://www.bedatec.de/

Jake Appelbaum
27/07/05, 19:35
--=-rJEQC2L0xWFd/oZLLHyD
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Fri, 2005-07-22 at 15:01 +0200, "Vincent DUVERNET (Nolm=EB
Informatique)" wrote:
> every body speaks about hardware & the best way for datas to be=20
> unrecoverable.
> Where states can use eletron microcope or other great machines, data=20
> encryption like EFS is another way to reinforce security no ?

If you don't want someone to be able to read your data, the best choice
is to encrypt it before it reaches the storage medium. Then destroy the
platters or wipe them or whatever feel good plan you have. Something
like loop-aes in linux or gbde in freebsd does that on the software side
(loop-aes even does in memory encryption key scrubbing [1]).

If you want something kept secret, loop-aes and gbde are your best bets
for offline security even before you've tried to destroy the data.
Sometimes you don't get a chance to destroy your cache of drives in the
closet before someone gets them.

It's obviously a choice about the threat model. With laptops it seems
like an obvious choice. Sometimes the performance hit isn't worth it but
for mobile devices and other high theft devices, it seems like unless
your data is worthless, you should encrypt it.

[1]: "Loop encryption key scrubbing moves and inverts key bits in
kernel RAM so that the thin oxide which forms the storage capacitor
dielectric of DRAM cells is not permitted to develop detectable property."

--=20
Jake Appelbaum <jacob@appelbaum.net>

--=-rJEQC2L0xWFd/oZLLHyD
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

-----BEGIN PGP SIGNATURE-----

iD8DBQBC4pgfmCiURc9yJggRAgOtAKClycLtKjrGi1fMERMdOP BhDrffQwCeIOa3
cRUKyL8dN/j/s4spuODeTD8=
=t0A+
-----END PGP SIGNATURE-----

--=-rJEQC2L0xWFd/oZLLHyD--

Casper.Dik@Sun.COM
27/07/05, 21:05
>is a dangerously naïve approach. With point-and-click easy to use =
>freeware tools under windows, I can do almost 100% retrieval of files=
> after a full reformat, and even after reloading the OS and using it =
>for a while, the simple point-and-click freeware tools can retieve an=
> awful lot of stuff. And if I have the skills to use more powerful, =
>complex tools, I can do even better, without needing a lot of money, =
>time, or even strong motivation.

That's not after a "reformat"; that's after making a new filesystem
on the disk.

(In this context, a "full reformat" doubtlessly refers to what
is called a "low level format" on systems with DOS heritage where
"newfs" is called "format")

Casper

Bret Morey
27/07/05, 21:25
A simple format is nothing like a low level format or a 3* overwrite. It is
theoretically possible to recover sporadic portions of data on a drive that
has been 3* overwritten, by utilizing special equipment to pick up that
areas that remain between tracks, but it requires equipment that is
typically prohibitively expensive and the chances of reconstructing a
significant amount of data are very slim. Consider this methods reliance on
write head wobble, and the fact that if the write heads are wobbling during
the original write, they will also wobble during the 3 recursive 1 and 0
overwrite passes, meaning that you not only have to read the remaining data
between the tracks, but you also have to distinguish between the original
data and the overwrite pass data, but you also have to manage to recover
enough contiguous data to reconstruct files and also figure out how to
reassemble it correctly. I have yet to hear anyone claiming to have
actually been able to recover and reconstruct the data from a system that
has been 3* overwritten per dss standards. And if you are that concerned
about data being recovered from decommissioned drives you can do like we
used to, and disassemble them and sandblast the platters. Or you can simply
send them to NSA, who to the best of my knowledge still degausses drives
free of charge with degaussers that are in some case actually capable of
physically destroying the platters.

If I am mistaken and someone has actually developed a method for reliably
reconstructing data from a drive that has been overwritten 3 times I would
be very interested in hearing about the details of the equipment and methods
involved.

regards
-Bret

-----Original Message-----
From: Robert Thompson Jr. [mailto:rthompson@columbiabank.com]
Sent: Thursday, July 21, 2005 3:03 PM
To: Jared Johnson; focus-ms@securityfocus.com
Cc: bugtraq@securityfocus.com
Subject: RE: Peter Gutmann data deletion theaory?

"Do you all agree with Peter Gutman's conclusion on his theory that data can
never really be erased, as noted in his quote below:"

Absolutely...

If you have ever done any form of data recovery, you will see how much
information is recoverable, with just basic tools off of the internet.
If you haven't, just google "data recovery", find almost any program with a
free demo and take a hard drive, catalog it, format it (after backing up
what you need of course) then recover it. Watch how much information you
retrieve. Should be all of it, and then some.

I recall the first time I ever did a recovery from a hard drive that had
something off happen to it. I pulled up information on that drive from back
when it was first used. YEARS before...

That is just with a basic program off of the internet.

With wiping/sanitizing of your hard drives, you have elimiated having to
worry about any mediocre programs doing any data recovery, but "good"
programs or hardware recovery is still an option. The software recovery
will eventually fail if you are careful enough...

Now imagine what a hardware based recovery could pull off?

I would recommend using the sanitizing products as they will help keep the
people that don't have the time or money from locating anything on your box,
but for those out there that have the money or have the time, they will be
able to get just about anything off of your disk.

To keep your drives completely secure, you have two choices: either don't
use them, ever... OR physically destroy them when you are finished.

Rob.

-----Original Message-----
From: Jared Johnson [mailto:jaredsjazz@Yahoo.com]
Sent: Wednesday, July 20, 2005 4:49 PM
To: focus-ms@securityfocus.com
Cc: bugtraq@securityfocus.com
Subject: Peter Gutmann data deletion theaory?

All,

Do you all agree with Peter Gutman's conclusion on his theory that data can
never really be erased, as noted in his quote below:

"Data overwritten once or twice may be recovered by subtracting what is
expected to be read from a storage location from what is actually read.
Data which is overwritten an arbitrarily large number of times can still be
recovered provided that the new data isn't written to the same location as
the original data (for magnetic media), or that the recovery attempt is
carried out fairly soon after the new data was written (for RAM). For this
reason it is effectively impossible to sanitise storage locations by simple
overwriting them, no matter how many overwrite passes are made or what data
patterns are written. However by using the relatively simple methods
presented in this paper the task of an attacker can be made significantly
more difficult, if not prohibitively expensive."

It seems that the perhaps the only real way to rid your Hard Drives of data
is to burn them.

I'd love to hear some thoughts on this from security and data experts out
there.

Alexander L. Ivanchev
27/07/05, 23:15
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--------------enig2935E932594CEE847ECED8B9
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

[Re-sending in plain-text]

Hello Volker,

> 1. writing zeroes all over
> 2. low level format

I cannot believe the concept of "low-level" format as a last resort
measure is still considered to be anything else, but "zeroing" out a
drive. Modern IDE drives are only, truly LLF'ed at factory-level. A long
time ago, in the world of RLL/MFM disks it was possible to define
interleave levels, etc., and actually re-create the physical tracks on
the platter. Not any more.

Personally, in the modern world, I've been hearing more and more of the
application HDD degaussers to the end of data wiping. I'd be more
interested in actual research on the extent of data recovery after a
thorough demagnetization via specialized hardware... Has anyone done
this kind of research?

Thanks,
Alexander

Volker Tanger wrote:

>Greetings!
>
>On Thu, 21 Jul 2005 14:07:12 -0500
>Simple Nomad <thegnome@nmrc.org> wrote:
>
>
>>On Wednesday 20 July 2005 18:48, Jared Johnson wrote:
>>
>>
>>>Data overwritten once or twice
>>>
>>>
>[...]
>
>
>>The quote is from 1996. I spoke with Guttman about this at AusCERT a
>>few years ago and even *he* doesn't believe it anymore. Drive
>>technology has changed substantially since then.
>>
>>
>
>His theory no longer does hold true. His 1996 paper is available at
>http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html, targeting
>MFM and RLL disk technology, where a typical 5.25" disk held 20-80MB
>(yes, MEGAbyte, not GB). His recommendations were based on old magnetic
>disc technology where each bit was represented by the magnetical
>orientation on the platter (north=1, south=0). After that came other
>technologies, where bits are defined by changes of the magnetic field
>even down to probabilistic field measurements - which allowes tighter
>data packing but rendered the base of his recommendations useless.
>
>Of course - if you write often enough with different data over "the
>same" spot, the original data will become more and more unreadable.
>
>OTOH I have seen one company with a *really* thorough disk & tape
>cleaning technique:
>
> 1. writing zeroes all over
> 2. low level format
> 3. shredding the disc drive into small pieces
> 4. magnet treatment of the scrap metal
> 5. burning in their own waste incinerating plant
>
>For "home use" a grinder/raw polish/sandblast treatment of both platter
>sides should be fine, too... ;-)
>
>Bye
>
>Volker
>
>
>
>

--
Sincerely,
Mr. Alexander L. Ivanchev

Phone/Fax: +359 2 929-3984
Cell Phone: +359 898 557-980
X.509 / PGP keys at:
http://www.ivanchev.org/

--------------enig2935E932594CEE847ECED8B9
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQuIpTj7pawcy/IzhAQquFA/9Fp9jmcA4bEm6kSk5uGy7ZxgXbTM1VV0A
1vRcXKpw5AB14U1Nz7icjNiw0HTmtNsbQS4UPacxmhf8NpOJU8 HMu/Ym9yCwv0FQ
keY7QkWLsiqExPgLcz77FnUJn58LjhF2hUKKMindxQeCrtG3iB Kako7UPj2hhYI+
Gh9RZFAxo8/cFJ8+6U12Sm8hiAJKbg7jyarTCXn6pF3QW81+lX7luESIk+Dvz lI9
Dp7PAIV7Nc1IO/t3SW+i3dWuVO/HNmhtkKcctwGgVwfvXA+Ntj86pHVbn6CoAGo6
+zFWe7B9D1xsVxoAFebmotELQhd157sjWhvBYa8l7loEdiYOMn eY1wnZynkTD/mh
DXNdXpGjSzA7HvIsB7hygcNwjNYVzVeAPiAH52ycr48X7D0VNQ VMDCCWMv1nVKed
zZ9E+ln/lEgn7a4D2D6T6NgVVxLw5xjipAmUMwN+jF+GRcNpU4+gjHWk5S 16GBKX
8KlJiTjmPEyKxs0Ki5ocQ40PNaYAUush2JcnaGkfrDKGCyknJ4 gOHjBfnnkUmxRu
vctyfooFv4nqLfjHMRJUU1BMhXQlCnEh+HgZ/gYbJtGBFHIRPLXTsg+2NuZLJEgN
F/8zOqEdh5FtJgQHq7ISHZrYWX/MEFGg2Srqw9btcSR8gOHhSIcFC3wzNezkbpp0
Rum8GBB26hU=
=RJVL
-----END PGP SIGNATURE-----

--------------enig2935E932594CEE847ECED8B9--

devnull@Rodents.Montreal.QC.CA
28/07/05, 02:25
[The From: is a bit-bucket, thanks for the hordes of broken
autoresponders. Use the address in my signature to reach me.]

> With point-and-click easy to use freeware tools under windows, I can
> do almost 100% retrieval of files after a full reformat,

I don't believe this for a moment. What you probably can do is recover
data after running DOS/Windows "format". Except on floppies, this does
not actually do a reformat; what it does do is more properly called
making a filesystem. (The misleading name is probably due to DOS's
floppy-based origins and Windows's DOS origins.)

A reformat - a *real* reformat, as in the SCSI FORMAT UNIT command or
whatever the analog is under other interfaces - will, with the possible
exception of blocks spared out while holding sensitive data[%], erase
beyond hope of retrieval by any means short of opening up the drive -
that is, any means that uses the usual data interface to the drive. If
you don't care about anyone with the resources and interest to open up
the drive and examine the magnetic patterns with tools more sensitive
than the disk's own read/write heads, this is almost certainly[%] all
you need to do.

But if you do, then you probably are paranoid enough that thermite (or
some equivalent that melts the drive into a puddle of liquid metal) is
the best option for you. Multiple overwrites are *probably* enough
with modern drives - but if your data are sensitive enough for your
adversary to be willing to open the drive up in a cleanroom, melting it
down is cheap, and about as sure as you're going to get.

[%] Some drives may have commands to best-effort read spared-out
blocks, which may leave data recoverable if the reformat believes
the current bad-block list instead of (re)constructing it from the
surface scan inherent in the reformat.

/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse@rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B

Casper.Dik@Sun.COM
28/07/05, 02:35
>His theory no longer does hold true. His 1996 paper is available at
>http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html, targeting
>MFM and RLL disk technology, where a typical 5.25" disk held 20-80MB
>(yes, MEGAbyte, not GB). His recommendations were based on old magnetic
>disc technology where each bit was represented by the magnetical
>orientation on the platter (north=1, south=0). After that came other
>technologies, where bits are defined by changes of the magnetic field
>even down to probabilistic field measurements - which allowes tighter
>data packing but rendered the base of his recommendations useless.

Overwriting the data a couple of times seems to be relatively effective
for modern disks.

In my opninion, the best way to destroy your data is to keep it
encrypted and then destroy the keys.

Recovery will succeed in only recovering a fraction of the data;
but with an unknown key and a typical encryption algorithm where
a single bit changed in the input changes all bits in the output,
recovery will need to find quite a bit more than the 1 bit per byte
required to recover typical text.

>OTOH I have seen one company with a *really* thorough disk & tape
>cleaning technique:
>
> 1. writing zeroes all over
> 2. low level format
> 3. shredding the disc drive into small pieces
> 4. magnet treatment of the scrap metal
> 5. burning in their own waste incinerating plant


Note that many of the people using such technologies do this
to prevent against future breakthrouhgs in recovery technology.
So it's required only for data with a long half-life.

Casper

dave kleiman
28/07/05, 02:55
Here is a quote directly from Peter I received Saturday, he asked to have it
passed on to the list.

--------------Snip-------------------------
>I'd love to hear some thoughts on this from security and data experts
>out there.

People should note the epilogue to the paper:

Epilogue

In the time since this paper was published, some people have treated the
35-
pass overwrite technique described in it more as a kind of voodoo
incantation to banish evil spirits than the result of a technical analysis
of drive encoding techniques. As a result, they advocate applying the
voodoo to PRML and EPRML drives even though it will have no more effect
than
a simple scrubbing with random data. In fact performing the full 35-pass
overwrite is pointless for any drive since it targets a blend of scenarios
involving all types of (normally-used) encoding technology, which covers
everything back to 30+-year-old MFM methods (if you don't understand that
statement, re-read the paper). If you're using a drive which uses
encoding
technology X, you only need to perform the passes specific to X, and you
never need to perform all 35 passes. For any modern PRML/EPRML drive, a
few
passes of random scrubbing is the best you can do. As the paper says, "A
good scrubbing with random data will do about as well as can be expected".
This was true in 1996, and is still true now.

Looking at this from the other point of view, with the ever-increasing
data
density on disk platters and a corresponding reduction in feature size and
use of exotic techniques to record data on the medium, it's unlikely that
anything can be recovered from any recent drive except perhaps one or two
levels via basic error-cancelling techniques. In particular the the
drives
in use at the time that this paper was originally written have mostly
fallen
out of use, so the methods that applied specifically to the older, lower-
density technology don't apply any more. Conversely, with modern high-
density drives, even if you've got 10KB of sensitive data on a drive and
can't erase it with 100% certainty, the chances of an adversary being able
to find the erased traces of that 10KB in 80GB of other erased traces are
close to zero.

Peter.

--------------Snip-------------------------


Dave Kleiman

Volker Kuhlmann
28/07/05, 03:15
> Unlike DRAM and SRAM, Flash etc. has no continual or repetitive
> amplification function. Traces of previous charges may remain, but I
> doubt they would be recoverable, except perhaps by the most drastic
> forensic techniques

I would expect flash memory content to be easily recoverable when
opening the chip.

> - and even then, each write or erase operation would
> reduce the remnants further still.

True, but don't put yourself to rest on it. Ever noticed that flash
memory is always smaller than the power of two which it is advertised
as? Memory is manufctured in matrices, so where is the missing memory? I
suggest these two possibilities: memory blocks coming out of manufacture
damaged are "turned off" (prevented from being used), a percentage of
memory blocks is reserved for defect management. As everything flash
uses a simple sort of FAT filesystem on these memories, the same areas
get repeatedly used and worn out, rendering the thing useless very
early. As a result, I expect flash memory to have defect management
similar to hard disks - when you think you're overwriting it, you're in
reality writing elsewhere. For either flash or hard disk you'll need to
get past internal defect management to sanitise the data. Or physically
destroy the memory media - always safest.

Volker

--
Volker Kuhlmann is possibly list0570 with the domain in header
http://volker.dnsalias.net/ Please do not CC list postings to me.

Simple Nomad
28/07/05, 19:45
On Sat, 23 Jul 2005, Ron van Daal wrote:

>> We were not allowed to do a seven pass government wipe to dispose of the
>> drives as our security people deemed it inadequate, we turned them over to
>> our classified waste people who stored them until there were enough to
>> justify having the platters removed and mechanicaly beaten into little
>> lumps of metal.

There is no 7 pass government wipe. It is a 3 pass wipe. It is referred to
as a 7 pass wipe because an app that did a 7 pass wipe passed govt muster
and was purchased. Odds are that if it had done it in 3 wipes it would
have still passed. If a vendor is saying "we do a 7 pass govt wipe" ask
them if one of those passes involves *verifiying* the writing of random
data, and if one of the passes in the inversion of another wipe (i.e. a
wipe with 0x0f and a wipe with 0xf0) to the drive. If not, it won't pass
that "government standard" I referred to in another post a few days ago.

> Aren't you being too paranoid? I think a simple zeroing out of your entire
> drive using dd(1) starting with the first sector is enough to cover your
> privacy. I don't know about other ""secret"" government agencies in NL or
> other counties who actually do microscopic magnetic recovery efforts, but
> dd(1) does the trick to defeat disk analysis by our national digital crime
> unit. From what I've read in one of their internal memo's is that they just
> use a hexdump(1) alike utility to find any non-zero bytes on the drive to
> conclude "the drive has been wiped entirely".

I basically agree with this. If any government can recover data via some
ninja electron microscope fu, odds are it is a state secret and they
wouldn't reveal they got your data nor reveal it in court (then it
wouldn't be a state secret anymore...) so it truly is a moot point, unless
the recovered data makes you an enemy combatant or something. Again, we
really have covered this topic several times here.

-SN, fairly drunk in Vegas so hopefully this made sense....

Michael Sierchio
01/08/05, 21:05
Jared Johnson wrote:

> Do you all agree with Peter Gutman's conclusion on his theory that data can
> never really be erased, as noted in his quote below:

Yes. Patterns of overwriting can offer a probabilistic assurance of
deletion, but not a promise. There is also the problem that disks
will silently remap sectors if they're error-prone, so it's not possible
in all cases to overwrite them.

If you're short on fishwrap, puppy training supplies, or birdcage liner,
you can print out an old whitepaper I wrote when I was trying to promote
a company based on a secure deletion scheme. My proposal is to transparently
encrypt every file with a different key, and assure deletion by losing the
key.

http://www.tenebras.com/papers/wp-draft.pdf

Richard Clayton
01/08/05, 22:45
In article <20050724001536.GC32500@paradise.net.nz>, Volker Kuhlmann
<list0570@paradise.net.nz> writes

>> Unlike DRAM and SRAM, Flash etc. has no continual or repetitive
>> amplification function. Traces of previous charges may remain, but I
>> doubt they would be recoverable, except perhaps by the most drastic
>> forensic techniques
>
>I would expect flash memory content to be easily recoverable when
>opening the chip.

yes indeed ! there's remnants of charge left on the floating gates of
transistors, even after explicit attempts to erase; and they are
relatively straightforward to read out

have a look at Sergei Skorobogatov's research on this topic (and on data
remanence on semiconductors generally)

http://www.cl.cam.ac.uk/users/sps32/

--
richard Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety. Benjamin Franklin