Salt in encrypted password in pg_shadow

[David Garamond]

I read that the password hash in pg_shadow is salted with username. Is
this still the case? If so, since probably 99% of all PostgreSQL has
"postgres" as the superuser name, wouldn't it be better to use standard
Unix/Apache MD5 hash instead?

--
dave
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

[Tom Lane]

David Garamond <li***@zara.6.i sreserved.com> writes:

Hm, I thought the purpose of salt is generally well understood?

Apparently not.

The purpose of salting the encrypted passwords in pg_shadow is *not* to
protect them against attackers who have somehow managed to
illegitimately read pg_shadow. (As I explained before, such attackers
are effectively superuser already, and so protecting the superuser
password from them is not nearly as interesting as all that.) The
purpose is to prevent unscrupulous DBAs from deducing the cleartext
passwords being used by their users. Since the users presumably are not
all named "postgres", the argument you are advancing is not relevant.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[Greg Stark]

Tom Lane <tg*@sss.pgh.pa .us> writes:

Steve Atkins <st***@blighty. com> writes:

A random salt stored with the hashed password increases the storage

.... [ raised eyebrow... ] It is not immediately obvious that a factor of
2^16 makes the difference between feasible and infeasible. As
counterexamples , if it would otherwise take you one microsecond to break
the password, 64 milliseconds isn't going to scare you; if it would
otherwise take you a century to break the password, raising it to
64k centuries isn't going to make for a very meaningful improvement in
security either.

*Storage* requirements. There's a pretty big range in the middle where the
extra 2^16 does make the difference.

The entire premise of cryptographic hashes after all depends on the assumption
that you cannot simply store an index of every possible hash value with the
plain-text that generated it.

That's only true if the number of plain-texts of concern is large enough to
make storing the hash value of each impractical. That's not true for human
chosen guessable passwords.

Now it's true that if you only want to try the top 1,000 guessable passwords
and you store a dictionary of all those with all 2^16 possible salts then it
requires only a gigabyte of storage. Perhaps a four character random salt
would make more sense.

However with a known salt you only have to store the 1,000 hashes with the
known salt. You could instead store a dictionary of 64 million password
guesses in the same gigabyte.

There's no question using a predictable salt is bad idea, postgres may as well
have not bothered salting at all. But realistically it seems kind of
irrelevant. It seems pretty unlikely that someone will gain access to
pg_shadow on many postgres installs which is the only way a dictionary attack
becomes a worry.

pg_shadow is not a public /etc/passwd like on a traditional unix machine where
I first saw salted crypt strings and it doesn't have hundreds or thousands of
entries like a public unix machine (at least not with predictable salts). The
threat model just doesn't apply.

--
greg
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Steve Atkins]

On Tue, Sep 07, 2004 at 10:27:40PM -0400, Tom Lane wrote:

Steve Atkins <st***@blighty. com> writes:

A random salt stored with the hashed password increases the storage
and precomputation time required by the size of the salt (so a 16 bit
salt would increase the storage and precomputation time needed by
a factor of 65536). That increase makes the pre-computed dictionary
attack pretty much infeasible.

[ raised eyebrow... ] It is not immediately obvious that a factor of
2^16 makes the difference between feasible and infeasible. As
counterexamples , if it would otherwise take you one microsecond to break
the password, 64 milliseconds isn't going to scare you; if it would
otherwise take you a century to break the password, raising it to
64k centuries isn't going to make for a very meaningful improvement in
security either.

Show me a scheme where the random salt isn't stored right beside the
password, and I might start to get interested.

A password salt is a pretty well understood benefit. It's very, very
standard technology, and has been in use for decades. The primary use
for it is to increase the cost of a pre-computed dictionary attack.
While the value of that has decreased as the ratio of CPU speed to
mass storage size has changed it's still a significant
value. Particularly as it still applies to parallel attacks.

We're talking about one-way hashes here, again a fairly standard
technology. Given the hash of a password the only (currently known)
way to compute a password that will validate against it is to guess
a password, compute the hash, see if it's the same, repeat.

You're never going to 'decode' a password from the hash - it's about
guessing the right password. Some passwords will be well-chosen, some
will be poorly chosen, some will be in between. The ideal situation
is where I have a number of hashes and just need to find a password
to match one of them.

So, assume I have a password generator that will generate me an
endless stream of passwords, starting from the obvious ones and
getting increasingly complex - this is the usual approach, as used
by crack and other unix password file crackers.

As one example, say I have 1,000 hashes, that it takes me ten
milliseconds to hash a password and the easiest to guess password will
be the hundred millionth produced by my password generator.

I can simply calculate the hash of each password in turn and compare
that hash against each of the thousand hashes. That'll take me about
11.5 days to crack the simplest to guess account of that list of
a thouasand.

Now, lets say that instead I have a thousand hashes, and associated
with each hash is a salt, all different. That means that to test
each generated password I'll need to calculate not a single hash,
but a thousand hashes - one against the combination of my guessed
password and the first salt, then the combination of the passsword
and the second salt and so on.

To crack the simplest to guess account of the thousand accounts I
have access to in this case will take me a thousand times as long -
about 30 years.

That's an example of why a salt is still extremely valuable, despite
the change in CPU speed:storage speed/size ration

It actually takes around 300us to compute an MD5 hash on modern
hardware (which is fast enough that use of MD5 for password validation
isn't clearly a good idea anyway, but that's another issue[1]) which
changes the math somewhat, but the principle is the same.

There are other benefits to a random salt too. Lets assume that
I have (by doing something dubious with a combination of google
and an insecure application server) ten thousand password files.
If no salt is used, or the same ('postgres') salt is used for
all of them then any two accounts with the same password will
have the same hashed value. That means I can identify two
people using the same password - if I can social engineer it
out of one of them I can use it to access the others account.

Cheers,
Steve

[1] The other issue being: When the concept was originally deployed it
took around half a second to calculate the hash. As hardware has got
faster, the calculation has got faster and brute force attacks against
a password file have become easier. MD5 is way too fast to be used as
part of a password based system that has attack trees including
compromise of the crypted password file. That's one reason that
unixen use shadow password files rather than making /etc/passwd
readable to all local users, to reduce the chance of the known
hash attack occuring.
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Steve Atkins]

On Tue, Sep 07, 2004 at 08:48:13PM -0700, Steve Atkins wrote:

That's an example of why a salt is still extremely valuable, despite
the change in CPU speed:storage speed/size ration

But, to clarify, I don't see any practical problem in the current
PostgreSQL implementation. It's not particularly secure, but not much
worse than the underlying OS authentication. Most of the feasible
attack trees are going to start with compromising the OS platform, by
which point weaknesses in the postgresql authentication are fairly
meaningless.

If we need to tweak the authentication protocol _anyway_ at some
point it'd be great to improve things. But until then... not worth
the pain.

Cheers,
Steve

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

[Tom Lane]

Greg Stark <gs*****@mit.ed u> writes:

However with a known salt you only have to store the 1,000 hashes with the
known salt. You could instead store a dictionary of 64 million password
guesses in the same gigabyte.
This is still not responding to my original point though: if you know
the salt that was used, you can try brute-force scan of a few thousand
probable passwords in less CPU time than it will take to read a gigabyte
of precomputed hashes. The fact that common passwords are much shorter
than the fixed-size MD5 hashes works against you in a big way.

I think the only way for the defender to get any real traction is to not
store the random salt right next to the encrypted password, so that the
attacker who hypothetically has read pg_shadow still has to guess about
the salt that was used. If someone shows me a plausible way to do that,
I'm all ears.
The threat model just doesn't apply.

This we agree on ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

[Tom Lane]

Steve Atkins <st***@blighty. com> writes:

If we need to tweak the authentication protocol _anyway_ at some
point it'd be great to improve things. But until then... not worth
the pain.

I've been hearing rumblings that MD5 and all other known crypto
protocols are known vulnerable since the latest crypto symposiums.
(Not that we didn't all suspect the NSA et al could break 'em, but
now they've told us exactly how they do it.)

So as soon as someone wheels up a new crypto hash method that looks
trustworthy, we can invent a new auth protocol and maybe throw in
another level of random salting while we're at it. But right now
I doubt it's worth the effort :-(

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly

[Mike Nolan]

> So as soon as someone wheels up a new crypto hash method that looks

trustworthy, we can invent a new auth protocol and maybe throw in
another level of random salting while we're at it. But right now
I doubt it's worth the effort :-(

A relatively simple enhancement would be to use some or all of the user
name as the salt. That makes reverse engineering the passwords a bit
harder because there are multiple salts being used.

I suspect that with the speed of modern microprocessors that nearly any
crypto scheme is breakable using a few thousand dollars worth of hardware
and a few hours of time. If the NSA has built in shortcuts to their
sanctioned algorithms, that just makes the cracking process faster.

I know of an ecryption technique developed by a friend of mine, a retired
mathematician, that would probably be sufficiently robust. It uses group
theory to permutate the bit field and has both reversible and
non-reversible forms.

It would probably be subject to export restrictions. As I recall, he
couldn't even send a copy of the program to his son in Greece without
State Department approval.

But as long as people use vulnerable passwords, there is no password
encryption scheme that isn't vulnerable to attack, with or without
salt.

Challenge/response and one-time password schemes are more secure but
a lot more hassle for users.
--
Mike Nolan

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

[David Garamond]

Tom Lane wrote:

Hm, I thought the purpose of salt is generally well understood?

Apparently not.

The purpose of salting the encrypted passwords in pg_shadow is *not* to
protect them against attackers who have somehow managed to
illegitimately read pg_shadow. (As I explained before, such attackers
are effectively superuser already, and so protecting the superuser
password from them is not nearly as interesting as all that.) The
purpose is to prevent unscrupulous DBAs from deducing the cleartext
passwords being used by their users. Since the users presumably are not
all named "postgres", the argument you are advancing is not relevant.

Then I'd say the purpose is wrong. There is not much hope in protecting
unscrupulous DBAs from getting their users' password anyway (they can
most probably sniff traffic, trap/log queries, or shut down postmaster
and replace it with a trojan binary).

The purpose of a salt, by most definition, should be to discourage
dictionary attack.

Anyway, I think we've agreed that the current protocol need not be
changed, on the basis of too much pain caused. But there's no reason not
to use random salt in future protocol. It offers the benefit you
mentioned plus protects against dictionary attack.

--
dave

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

[Greg Stark]

Tom Lane <tg*@sss.pgh.pa .us> writes:

Greg Stark <gs*****@mit.ed u> writes:

However with a known salt you only have to store the 1,000 hashes with the
known salt. You could instead store a dictionary of 64 million password
guesses in the same gigabyte.

This is still not responding to my original point though: if you know
the salt that was used, you can try brute-force scan of a few thousand
probable passwords in less CPU time than it will take to read a gigabyte
of precomputed hashes. The fact that common passwords are much shorter
than the fixed-size MD5 hashes works against you in a big way.

We must be talking past each other. The threat model salts are meant to defend
against is someone who has access to the data in pg_shadow for many users.
Without the salts or with salts you can predict beforehand you look up the
hash value in your precomputed dictionary using an index and instantaneously
get a working password.

Postgres's current method is in fact doing it wrong. Because the salt is
predictable for the "postgres" users it doesn't protect against the attack
above. If I knew I could get access to lots of pg_shadow's, say I work at an
off-site backup storage company, I could check each of them instantaneously
against a precomputed index of hundreds of thousands of guessable passwords.
The same attack against a well salted hash would require running the entire
battery of hashes against each client's password individually.

The reason I say the threat model doesn't apply is only because it's unlikely
that someone would have access to many postgres installs's pg_shadow. That's
the only situation where that attack would arise. (Or if a given install had
hundreds of users with the same initial letters I guess, but that also seems
rare)

But if you're not going to worry about that threat then salting is buying you
nothing anyways. If you're going to use a salt you may as well use one that
accomplishes what it's there for.

--
greg
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

[Bruno Wolff III]

On Wed, Sep 08, 2004 at 00:33:39 -0400,
Tom Lane <tg*@sss.pgh.pa .us> wrote:

I've been hearing rumblings that MD5 and all other known crypto
protocols are known vulnerable since the latest crypto symposiums.
(Not that we didn't all suspect the NSA et al could break 'em, but
now they've told us exactly how they do it.)

Things aren't currently that bad. So far people have found a way to find
two strings that give the same hash using MD5. They haven't yet found a way
to find a string which hashes to a given hash. SHA-0 was also shown to
have some weakness. From comments I have read, I don't think SHA-1 was
shown to have any weaknesses. One comment specifically mentioned that
the change made between SHA-0 and SHA-1 seems to have been made to address
the weakness found in SHA-0. I haven't read the source papers, so take this
all with a grain of salt.

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend