Re: [HACKERS] Postgres: pg_hba.conf, md5, pg_shadow, encrypted passwords

[Tom Lane]

Stephen Frost <sfrost@snowman.net> writes:
> The md5 hash which is generated for and stored in pg_shadow does not
> use a random salt but instead uses the username which can generally be
> determined ahead of time (especially for the 'postgres' superuser
> account).

So?

The fact that we encrypt the contents of pg_shadow at all is not to
provide security against breakins by people who have managed to
obtain the contents of pg_shadow. Any such attacker knows as much as
the postmaster does, and so there isn't anything much the postmaster can
do to prevent a breakin. The reason we do it is to prevent such a
person (or a dishonest DBA) from obtaining the user's actual original
password. This doesn't improve the security of the database at all,
of course, but it does improve security globally if the user used the
same password for other systems.

> This would allow for the pregeneration of the entire md5
> keyspace using that 'salt' and then quick breakage of the hash once
> it's retrieved by the attacker.

Considering the size of the possible keyspace, this is pretty silly.

> Were a decent random salt of some
> size used it would be difficult to guess and pregenerate the keyspace
> for. Thus, keyspace generation would have to happen after pg_shadow
> was compramised, giving the admin time to detect the compramise and
> take corrective action.

Another large assumption: that the admin knows about the compromise
before the results are used.

>. It is also not made clear that if you are
> already handling transport-level security via SSL and/or IPSEC that
> using md5 actually reduces security by not adding anything to the
> transport-level security and defeating the on-disk security
> effectivness of using md5 for pg_shadow.

That's simply false. The contents of pg_shadow are never sent over the
wire.

You're going to have to work a lot harder to convince us there's any
significant issue here.

regards, tom lane

[Bruce Momjian]

Tom Lane wrote:
> "Jim C. Nasby" <decibel@decibel.org> writes:
> > Simply put, MD5 is no longer strong enough for protecting secrets. It's
> > just too easy to brute-force. SHA1 is ok for now, but it's days are
> > numbered as well. I think it would be good to alter SHA1 (or something
> > stronger) as an alternative to MD5, and I see no reason not to use a
> > random salt instead of username.
>
> Well, I have no particular problem with offering SHA1 as an alternative
> hash method for those who find MD5 too weak ... but I still question the
> value of putting any random salt in the table. AFAICS you would have to
> send that salt as part of the initial password challenge, which means
> any potential attacker could find it out even before trying to
> compromise pg_shadow; so Stephen's argument that there is a useful
> improvement in protection against precomputation of password hashes
> still falls down.
>
> BTW, one could also ask exactly what threat model Stephen is concerned
> about. ISTM anyone who can obtain the contents of pg_shadow has
> *already* broken your database security.

That's what I told him. I think his concern about pre-computed hashes
is the only real issue, and give 'postgres' is usually the super-user, I
can see someone pre-computing md5 postgres hashes and doing quick
comparisons, perhaps as a root kit so you don't have to do the hashing
yourself. I personally don't find that very compelling either.

--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

[Jim C. Nasby]

On Wed, Apr 20, 2005 at 06:03:18PM -0400, Tom Lane wrote:
> Well, I have no particular problem with offering SHA1 as an alternative
> hash method for those who find MD5 too weak ... but I still question the
> value of putting any random salt in the table. AFAICS you would have to
> send that salt as part of the initial password challenge, which means
> any potential attacker could find it out even before trying to
> compromise pg_shadow; so Stephen's argument that there is a useful
> improvement in protection against precomputation of password hashes
> still falls down.
>
> BTW, one could also ask exactly what threat model Stephen is concerned
> about. ISTM anyone who can obtain the contents of pg_shadow has
> *already* broken your database security.

FWIW, I do think there's some benefit to not being able to pre-compute
an entire hash table for accounts such as 'postgres' and 'www'. But I
agree it would be useful to know the actual threat model.
--
Jim C. Nasby, Database Consultant decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"

[Tom Lane]

"Jim C. Nasby" <decibel@decibel.org> writes:
> Simply put, MD5 is no longer strong enough for protecting secrets. It's
> just too easy to brute-force. SHA1 is ok for now, but it's days are
> numbered as well. I think it would be good to alter SHA1 (or something
> stronger) as an alternative to MD5, and I see no reason not to use a
> random salt instead of username.

Well, I have no particular problem with offering SHA1 as an alternative
hash method for those who find MD5 too weak ... but I still question the
value of putting any random salt in the table. AFAICS you would have to
send that salt as part of the initial password challenge, which means
any potential attacker could find it out even before trying to
compromise pg_shadow; so Stephen's argument that there is a useful
improvement in protection against precomputation of password hashes
still falls down.

BTW, one could also ask exactly what threat model Stephen is concerned
about. ISTM anyone who can obtain the contents of pg_shadow has
*already* broken your database security.

regards, tom lane

[Tom Lane]

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> That's what I told him. I think his concern about pre-computed hashes
> is the only real issue, and give 'postgres' is usually the super-user, I
> can see someone pre-computing md5 postgres hashes and doing quick
> comparisons, perhaps as a root kit so you don't have to do the hashing
> yourself. I personally don't find that very compelling either.

Lessee ... we'll include a complete password hash table in a root kit,
which will be used at a point where we've already managed to read
pg_shadow but are somehow still lacking the ability to do anything else
we could want to the database ... nope, not very compelling.

regards, tom lane

[Josh Berkus]

David, Stephen,

> I noted that this was a problem back in August, 2002:
>
> http://archives.postgresql.org/pgsql-admin/2002-08/msg00253.php
>
> Then, as now, the developers weren't very concerned.

Well, from our perspective, a random salt only protects against a very narrow
range of attack types -- ones in which the attacker already has access to the
physical database and wants to reverse-engineer user's passwords. We'd be
much more interested in the implementation of more/better authentication
mechanisms. See follow-up dicussion on pgsql-hackers.

Of course, if either of you *wrote* a random-salt patch for PostgreSQL, psql
and libpq, then that would be a different story. I don't know that anyone
has anything *against* a random salt. It's just not nearly as useful as,
for example, implementing SHA1.

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

[Joshua D. Drake]

> Simply put, MD5 is no longer strong enough for protecting secrets. It's
> just too easy to brute-force. SHA1 is ok for now, but it's days are
> numbered as well. I think it would be good to alter SHA1 (or something
> stronger) as an alternative to MD5, and I see no reason not to use a
> random salt instead of username.

If you aren't paying close enough attention to your database server to
see that someone is trying to brute force your MD5 password you have
bigger problems.

Sincerely,

Joshua D. Drake





--
Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240
PostgreSQL Replication, Consulting, Custom Programming, 24x7 support
Managed Services, Shared and Dedication Hosting
Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/

[Stephen Frost]

--McQJJYUzjTUqtsGv
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Joshua D. Drake (jd@commandprompt.com) wrote:
> >Simply put, MD5 is no longer strong enough for protecting secrets. It's
> >just too easy to brute-force. SHA1 is ok for now, but it's days are
> >numbered as well. I think it would be good to alter SHA1 (or something
> >stronger) as an alternative to MD5, and I see no reason not to use a
> >random salt instead of username.
>=20
> If you aren't paying close enough attention to your database server to
> see that someone is trying to brute force your MD5 password you have=20
> bigger problems.

If the attacker knows the salt then she can do almost all of the
brute-force work up-front before even attempting to attack the system
directly. The sys admin would have no way of knowing this was
happening. Once this is done the attacker just needs the hash from
pg_shadow to quickly find the user's password (or something that hashes
to the same thing). =20

In this situation I'm assuming the system is using the 'password'=20
method in pg_hba.conf. If the system used the 'md5' method in=20
pg_hba.conf the attacker would need only the hash from pg_shadow to=20
authenticate and wouldn't need the users original password at all.

If a random salt were used in this situation, and 'password' were used
in pg_hba.conf then the attacker would be unable to determine the salt
ahead of time and would be forced to generate the keyspace after
obtaining pg_shadow to brute-force the password, a time consuming
process hopefully given the admin an opportunity to discover the
compromise.

Thanks,

Stephen

--McQJJYUzjTUqtsGv
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCZ+uBrzgMPqB3kigRAkS0AJ9aippVr5ZB2lT1SmqAjz g2axhwpQCcDnFB
NevigRViQC+VeY/nlnHEEWs=
=jhr9
-----END PGP SIGNATURE-----

--McQJJYUzjTUqtsGv--

[David F. Skoll]

Tom Lane wrote:

> Lessee ... we'll include a complete password hash table in a root kit,
> which will be used at a point where we've already managed to read
> pg_shadow but are somehow still lacking the ability to do anything else
> we could want to the database ... nope, not very compelling.

You are correct that the threat against the PostgreSQL installation itself
is not very compelling. However, take a look at the bigger picture:
People crack into systems and then try to use those systems to crack
into other systems. If you can make it harder to recover passwords
in the PostgreSQL system, then you've made it harder for attackers
to use those recovered passwords to attack other systems.

Think of the complete security environment, not just the security
of a particular PostgreSQL installation. Having random salts makes
it much harder for attackers to answer questions like "Does user X
have the same password in PostgreSQL installation 1 as he does in PostgreSQL
installation 2".

Regards,

David.

[Stephen Frost]

--EguBBKnZWdUQS9Kz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> "Jim C. Nasby" <decibel@decibel.org> writes:
> > Simply put, MD5 is no longer strong enough for protecting secrets. It's
> > just too easy to brute-force. SHA1 is ok for now, but it's days are
> > numbered as well. I think it would be good to alter SHA1 (or something
> > stronger) as an alternative to MD5, and I see no reason not to use a
> > random salt instead of username.
>=20
> Well, I have no particular problem with offering SHA1 as an alternative
> hash method for those who find MD5 too weak ... but I still question the

SHA2 would also be nice.

> value of putting any random salt in the table. AFAICS you would have to
> send that salt as part of the initial password challenge, which means
> any potential attacker could find it out even before trying to
> compromise pg_shadow; so Stephen's argument that there is a useful
> improvement in protection against precomputation of password hashes
> still falls down.

Only if you're using 'md5' in pg_hba.conf. In that case, yes, you would
have to send the salt as part of the password challenge. Personally, I
would discourage use of 'md5' in pg_hba.conf because of this and because
it changes the authentication token from the password to the hash which
is what is directly stored in the database.

> BTW, one could also ask exactly what threat model Stephen is concerned
> about. ISTM anyone who can obtain the contents of pg_shadow has
> *already* broken your database security.

Just because they have access to pg_shadow does not necessairly mean
they have access to the database files directly or are able to write to
anything (such as to destroy data). It's possible they pulled
pg_shadow off a backup tape and are bent on destroying all the data in
your live database, and not in just reading it, or if your data is very
time sensitive then they want a more recent version for its value, etc.

There are other possibilities but my concern centers around a partial
system compromise where pg_shadow is obtained by the attacker, yes.

Thanks,

Stephen

--EguBBKnZWdUQS9Kz
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCZw91rzgMPqB3kigRApltAJ0RxyI3KcY7xv8s0fGRhS ceAP9YiwCfQ9mX
vE9zA0sYY/m/DXLBRBA8TgE=
=LBOK
-----END PGP SIGNATURE-----

--EguBBKnZWdUQS9Kz--

[Stephen Frost]

--v4PrGfol6wGsg/Ra
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Mike Fratto (mfratto@nwc.com) wrote:
> > I thought the idea of the salt was to aid in expanding the=20
> > keyspace. Even though the salt is known (in traditional Unix=20
> > passwd/shadow/master.passwd databases,=20
>=20
> I am pretty sure the intent the salt is to make pre-computation of a
> dictionaries infeasable due to storage requirements. It doesn't really add
> to the keyspace because the salt is known and doesn't have to be guessed.

The salt isn't always known... I don't know how an unprivledged user on
a system w/ /etc/shadow could get at it anyway. I'm sure alot of people
would be very anxious to know if you know of a way to do that...

Stephen

--v4PrGfol6wGsg/Ra
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCaBInrzgMPqB3kigRAq8nAKCU4ufcOco+mop/t0Lic6S5SiUPgACeND9m
tklMFotHeW81u8+NeI6P7UE=
=QFt6
-----END PGP SIGNATURE-----

--v4PrGfol6wGsg/Ra--

[Stephen Frost]

--fwblGvOBo7NCOYks
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Bruno Wolff III (bruno@wolff.to) wrote:
> On Wed, Apr 20, 2005 at 22:27:01 -0400,
> Stephen Frost <sfrost@snowman.net> wrote:
> > SHA2 would also be nice.
>=20
> I think the new hash functions are called SHA256 and SHA512.
> For Postgres' purposes the recent weaknesses found in SHA1 and MD5
> aren't a big deal.

Well, alright, SHA256 and SHA512. It would be nice to have Postgres
support them.

Stephen

--fwblGvOBo7NCOYks
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCaD/JrzgMPqB3kigRAndWAJwPLkk014FEUHGkfSWrR7oKNTfe4ACbB SCB
MHM4g85UkBflh9jGAV4/BxY=
=uxkI
-----END PGP SIGNATURE-----

--fwblGvOBo7NCOYks--

[Lance James]

Joshua D. Drake wrote:
>> Simply put, MD5 is no longer strong enough for protecting secrets. It's
>> just too easy to brute-force. SHA1 is ok for now, but it's days are
>> numbered as well. I think it would be good to alter SHA1 (or something
>> stronger) as an alternative to MD5, and I see no reason not to use a
>> random salt instead of username.
>
>
> If you aren't paying close enough attention to your database server to
> see that someone is trying to brute force your MD5 password you have
> bigger problems.

The comments on md5 and sha1 are both inaccurate if you're comparing
them. Encrypted passwords are as strong as the design of the password.
In some cases, SHA-1 is a faster brute force because SHA-1 is a faster
hash. There are two issues here. Using SHA-1 to hash a password, and the
strength of a password. If the implementation of SHA-1 is not effective,
there could be weaknesses that enable reducing the time required to
perform exhaustive/dictionary attacks against sha-1 protected password.

I'm out of context, but I had to make some corrections.


--
Best Regards,
Lance James
Secure Science Corporation
www.securescience.com
Author of 'Phishing Exposed'
http://www.securescience.net/amazon/
Have Phishers stolen your customers' logins? Find out with DIA
https://slam.securescience.com/signup.cgi - it's free!

[Mark Senior]

It also slows down cracking numerous passwords in parallel using a
dictionary/heuristic approach a la john the ripper - without a salt, you
can calculate the hash of each password guess once, and then scan
through an entire shadow file for the hash. With salts, you have to
hash each guess once per user in the list, or at least per user you're
interested in attacking.

In the case of the postgres passwords, the user name seems to act as a
sort of "public" salt. Knowing some user IDs, you might precompute a
set of hashes for each targetted user, in anticipation of getting your
hands on the password hashes later.

Mark

> -----Original Message-----
> From: Mike Fratto =20

> I am pretty sure the intent the salt is to make=20
> pre-computation of a dictionaries infeasable due to storage=20
> requirements. It doesn't really add to the keyspace because=20
> the salt is known and doesn't have to be guessed.

--- end of sensible content ---

This email and any files transmitted with it are confidential and intende=
d solely for the use of the individual or entity to whom they are address=
ed. If you have received this email in error please notify the system man=
ager. This message contains confidential information and is intended only=
for the individual named. If you are not the named addressee you should =
not disseminate, distribute or copy this e-mail.

[Mike Fratto]

> That's the whole point of the discussion- the way Postgres's=20
> pg_shadow stuff works the salt is known and *because* of that=20
> it might as well not exist since it means that you can=20
> pre-compute the keyspace. =20

I see your point. I don't know anything about postgres. I don't use it. =
But
if someone can get to the pg_hba.conf file (I assume (hope) it is =
read/write
by the process owner or root only?) then your screwed anyway. So while =
there
may be better ways to store and use passwords, perhaps in light of the =
root
of the problem (getting to the file) the fore-knowledge of a salt isn't =
that
important. If an admin created a "strong" password (whatever that =
means),
then pre-computation won't help an attacker get it. At worst for the =
admon,
pre-computation will shorten the attackers time to know if the password =
can
be broken or not. At best it might slow them down a bit (but not =
really).=20

I dunno.=20

[Mike Fratto]

> The salt isn't always known... I don't know how an=20
> unprivledged user on a system w/ /etc/shadow could get at it=20
> anyway. I'm sure alot of people would be very anxious to=20
> know if you know of a way to do that...

The salt is known, just not by an unpriveledged user. But if you can =
access
/etc/shadow, then you know the salt and the hashed passwords.=20

The salt is added (dunno where in the original string) to the password =
prior
to hashing. Then the salt is prepended to the hased value and stored
somewhere, in your case, /etc/shadow.=20

For a salt to be usable, you have to know what it is. Secure hashes are =
"one
way" which means that you shouldn't be able to derive the original text
solely from the hash value. To create password: create 2 byte salt from
ASCII -> prepend salt to password -> hash salt+password with MD5 -> =
prepend
salt to hash value -> store it. To check password: get username and =
password
-> lookup hash string using username from /etc/shadow -> get salt from =
hash
value -> hash salt+password with MD5 -> compare value computed hash =
value
from stored hash value -> If match, user authenticated, if not, user not
authenticated.

So if you hash a password with a salt, and then lose the salt, you can =
never
check the password again. You have to know salt.=20

Since the salt is known, it has no effect on the "keyspace" because you
don't have to guess it. If there was no salt, then pre-computing a
dictionary is a much smaller task.=20

mike

[Bruno Wolff III]

On Wed, Apr 20, 2005 at 22:27:01 -0400,
Stephen Frost <sfrost@snowman.net> wrote:
>
> SHA2 would also be nice.

I think the new hash functions are called SHA256 and SHA512.
For Postgres' purposes the recent weaknesses found in SHA1 and MD5
aren't a big deal.

[Jim Knoble]

Circa 2005-04-21 dixit Mike Fratto:

:
: > I thought the idea of the salt was to aid in expanding the
: > keyspace. Even though the salt is known (in traditional Unix
: > passwd/shadow/master.passwd databases,
:
: I am pretty sure the intent the salt is to make pre-computation of a
: dictionaries infeasable due to storage requirements. It doesn't really add
: to the keyspace because the salt is known and doesn't have to be guessed.

.... which is exactly what i was speaking of. The salt increases the
keyspace for the precomputed table of password hashes. The conversation
was not about brute force attacks....

--
jim knoble | jmknoble@pobox.com | http://www.pobox.com/~jmknoble/
(GnuPG fingerprint: 809F:09B9:9686:D035:4AB0::9455:124B:0A62:DD6A:76D6 )
.................................................. ...................
:"The methods now being used to merchandise the political candidate :
: as though he were a deodorant positively guarantee the electorate :
: against ever hearing the truth about anything." --Aldous Huxley :
:................................................. ..................:

[Stephen Frost]

--QBJnsLb0QyYEYRYK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

* Mike Fratto (mfratto@nwc.com) wrote:
> Since the salt is known, it has no effect on the "keyspace" because you
> don't have to guess it. If there was no salt, then pre-computing a
> dictionary is a much smaller task.=20

That's the whole point of the discussion- the way Postgres's pg_shadow
stuff works the salt is known and *because* of that it might as well not
exist since it means that you can pre-compute the keyspace. Knowing the
salt means you can pre-compute the keyspace ahead of time. If you don't
know the salt until you've gained access then you'll have to wait till
then to begin computing the keyspace.

I suppose technically you could start pre-computing the keyspace before
then, but then it's a much larger keyspace which makes it much more
difficult.

Stephen

--QBJnsLb0QyYEYRYK
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCaBxCrzgMPqB3kigRAlR6AJ4jUIxdf3kw3gaGQZtcFI/51Z1gmgCfcLOZ
j6vHFtu3RIihYAkBKro3j5k=
=x7r/
-----END PGP SIGNATURE-----

--QBJnsLb0QyYEYRYK--