Onderwerp: DJB's students release 44 *nix software vulnerability advisories

Steven M. Christey wrote:

>In addition to modeling the level of authentication needed, I've been
>thinking that it might also be important to note how much user/victim
>participation is required for activation of the exploit, i.e. whether
>the issue can be automatically exploited by normal user activity
>(e.g. by simply reading an email message) or whether there's some
>social engineering involved. However, I haven't put much thought into
>terminology for this besides:
>
> - automatic: exploit is automatically activated as a result of
> normal usage of the product
>
>
I call this class "worms", or more grammatically a class of remote
vulnerabilities subject to worm attack. where the malware can propagate
unassisted.

> - complicit: requires some victim participation or inaction
>
>
I call this class "viruses, same grammar hack as above. These require
the victim to click on something, or such like, before the malware can
propagate.

> - opportunistic: can not really control when, or if, the victim
> activates the exploit
>
>
I'm having a hard time seeing the difference between "complicit" and
"opportunistic".

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

robert@dyadsecurity.com wrote:

>What you have to ask yourself here is what do you fear more?:
>A) Do you fear wide spread worm based attacks where everyone knows about
>the problems at about the same time, and is more annoying than
>devistating?
>
>B) Do you fear directed malice attacks using information that that the
>defense does not know about?
>
>For my customers I fear B far more than A.
>
>
Lets do the math. The vulnerable code has been around for a year. A
vulnerability is discovered. The software provider wants 2 weeks. Lets
say (generously) that there are 10 times as many SOHO users defending
against A as there are critical infrastructure users defending against B.

* If we do A (responsible disclosure) then we expose (say) X users
to 2/50 additional weeks of time in which an el33t hax0r has a
private 0day against them that might be deployed, so you get
X*0.04 of time/risk exposure.
* If we do B (full disclosure), then we save the critical
infrastructure people that X*0.04 risk days, and instead we get
the 10X of SOHO users exposed to 2 weeks in which there is no
patch available. Assume that the software had another year of
lifespan before it was obsoleted, so again there is a factor of
2/50 or 0.04 on the risk days, but there are 10X users, so the
total risk factor is X*0.4.

Netted out, the risk/benefit is roughly the same but multiplied by the
size of the respective communities, and the SOHO community is much, much
larger.

I further submit that not all critical infrastructure people agree with
you, and some would rather see responsible disclosure of
vulnerabilities. You can tell by the number of enterprises subscribing
to early warning services for responsible disclosure for big $$$.


>The stick your head in the sand approach to Vulnerability Disclosure is
>not the direction I want to see the industry go.
>
>
Neither is a colloquial ad homenim approach to describing your opponents
pie in the sky position

>> Furthermore, if you force a fire drill in releasing the security
>>patch, you compromise the quality of the patch. See my work on patch
>>quality "Timing the Application of Security Patches for Optimal
>>Uptime", Beattie et al Postscript
>><http://immunix.com/%7Ecrispin/time-t...x-lisa02.ps.gz>. or
>>ugly PDF
>><http://immunix.com/%7Ecrispin/time-t...nix-lisa02.pdf>.
>>
>>
>Perhaps a Patch isn't the only option here. Perhaps changing vendors,
>or removing the vulnerable service is an alternative to being
>compromised, or installing a buggy patch. No matter what, without the
>vulnerability information, you are making an uninformed decision.
>
>
Indeed. Another approach is to deploy intrusion prevention technologies
such as Immunix <http://immunix.com/technology/> that can block 0-day
attacks without needing any specific knowledge of whatever DJB's class
will come up with next time.

>>So while I am sympathetic to DJB's passion for correct software and to
>>hell with the tender feelings of developers who ship buggy code, in
>>practice this kind of 0-day notice of vulnerabilities *mostly* just
>>harms end-users.
>>
>>
>I think this depends on the segment you're talking about. If you're
>talking about enterprise or mission critical systems, full disclosure
>should help the end user more than hurt, as they are more likely to have
>the resources available to take appropriate action. If you're talking
>about soho/home PC users, then I would agreee with your point above.
>
>
More precisely, if you are talking about a service that can be *taken
down* while waiting for a fix or a work-around, then full disclosure is
an advantage. If the service cannot (or will not) be taken down and will
be just left to run the risk of compromise, then responsible disclosure
helps. I submit that most enterprise systems are of the "damn the
torpedoes" school of thought.

>The delima for me only comes when the enterprise/mission critical
>infrastructure is the same as the home/soho infrastructure.
>
>
Microsoft: shared between SOHO and Enterprise
Linux: shared between SOHO and Enterprise
Solaris: Enterprise only, but declining share, rapidly being replaced
with Microsoft and Linux

This does not make me feel better about full disclosure

Big fat caveat: All of the above assumes that the investigator is
actually well-intentioned. Actual black hats will do whatever they want,
and what we all advocate is of no matter, so defenders have to be able
to withstand 0-days anyway.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

Jonathan,

You touch on a couple of topics which are worthy of exploring further.

First, lets talk about the nature of risk. It is not black and white. It
is a continuum, very difficult to measure exactly but can usually be
estimated. In the security profession you quickly learn that not only
can you not eliminate risk, but in many cases it doesn't even make sense
to try. Your job is to manage and reduce risk as cost-effectively as
possible.

I agree that full disclosure serves a purpose. However, I disagree that
_immediate_ disclosure is the best solution because it imposes too much
risk on the end-users (the victims). In your example, you had the option
of disabling SSH until a patch was available. Others probably could not
pursue that course of action as the opportunity cost would have been too
high. That is, there are many enterprises in this world that absolutely
depend on SSH to function efficiently. Many probably didn't disable SSH
even when they knew there was an unpatched vulnerability. Were they
lazy? Were they just stupid? I doubt it. They most likely performed a
risk assessment even if it was made unconsciously. That is, they
estimated the potential cost to their organizations of leaving SSH
active until a patch was available against the lost revenue and/or
increased cost of doing business they would incur if they pulled SSH
until a patch was available. You did the same thing. However, in your
case the risk of leaving SSH active was probably much higher than the
cost of not having it.

What immediate disclosure does is immediately increases the total risk
incurred by the users of the vulnerable software until a patch becomes
available. Contrast this against what is currently generally accepted
practice: notify the vendor; give them a reasonable amount of time to
fix the immediate problem; give them a reasonable amount of time to
audit for any similar problems; give them a reasonable amount of time to
verify that they haven't introduced any other problems in the course of
fixing the current one; post your advisory after either a patch is
available or a reasonable time has expired. This practice greatly
reduces the risk to the end users (remember they ARE the victims here).
The vendor is still "punished", maybe not as harshly as you would
prefer, but these announcements do still take their toll. The key here
is that no unnecessary collateral damage is imposed upon the victims in
the interest of making sure that the "vendors get what is coming to
them"...

Now lets talk about the LAZY humans and software vendors. They aren't
any more lazy than you are. They are arguably quite efficient (as a
group) with their limited resources however. Perfectly secure software
has a development cost. In fact, it is a VERY high development cost.
This cost would have to be passed on to the customers and now we get
back into our discussion of risk and opportunity cost. If a vendor
provided a commercial version of SSH that they have made flawless for
$10,000-100,000 a copy, would you buy it? (Why so much? The cost of
development increases and must be passed on to the customer base. As the
cost goes up the potential customer base shrinks and therefore, the cost
burden per remaining customer also must increase. A very non-linear
relationship! Granted, this greatly simplifies a very complex subject in
its own right, but it is accurate enough for this discussion) I doubt
that you would. It wouldn't be very cost effective of you. Your personal
risk in dealing with the occasional SSH bug that occurs has an amortized
cost far less than $10,000 (the minimum opportunity cost of being more
secure in this scenario). You still use the publicly available SSH even
after its history of bugs don't you. Even when clearly presented with
evidence that the software has a history flaws (and likely has other as
yet undiscovered ones), you still do not choose a very significantly
more secure alternative of removing your system from the Internet. Does
that make you lazy, stupid, or smart/efficient/practical?

Why don't the commercial vendors of SSH produce a guaranteed flawless
product? Is it because they are full of lazy programmers or greedy
executives? I do not think so. I would argue that the marketplace
doesn't currently support it. Customers do want more secure products,
but they just are not willing to pay enough extra for it to support the
development costs.
=20
Paul

-----Original Message-----
From: Jonathan Rockway [mailto:jrockw2@uic.edu]=20
Sent: Wednesday, December 22, 2004 1:06 AM
To: bugtraq@securityfocus.com
Subject: Re: DJB's students release 44 *nix software vulnerability
advisories


On 21 Dec 2004, at 3:22 PM, laffer1 wrote:

> As for the other comments in this thread about telling the vendor
> early, I personally feel it helps users if the vendor has a few days=20
> to look at the hole and devise a patch BEFORE everyone on the planet=20
> knows about it. You punish users of software in addition to vendors.

> All software has a security problem of one kind or another, and its=20
> silly to think that a perfect application will every be written.

Why are users using insecure software? Or rather, why do users accept=20
the fact that their software may be insecure?

Besides, full disclosure helps the users too.

I remember a few years ago when the major SSH remote hole was found. I=20
read about it on slashdot between classes. Since there was no patch=20
yet, but there was an exploit, I ssh'd into my home machine and turned=20
off ssh. Even if there wasn't an exploit, I wasn't going to leave a=20
vulnerable service up and running. By the time I got home, sshd was=20
patched, and I installed that patch. If nobody had disclosed the=20
threat to me, I could have been compromised. But, because I was=20
notified, I was able to take preventative measures.

The sooner someone told me about the problem, the sooner I was able to=20
protect myself from the threat.

Full disclosure is important because vendors will drag their feet if=20
they're the only ones who know about it. Imagine you are a student and=20
have a paper due "whenever". When are you going to write the paper? =20
Today? No, you'll do it later. After all, nothing bad happens if you=20
don't do it, and not doing it is much easier than doing it.

Humans and software vendors are LAZY. If there's no reason to do=20
something, they won't do it. Full disclosure forces the issue and puts=20
everything out in the open. No "it'll be ready in 90 days" stalling. =20
It will be ready NOW or users will look to more secure alternatives. =20
They will make the first move and choose a program that doesn't have=20
security problems to begin with. This is better for everyone (well,=20
except people with financial interest in selling crappy software. that=20
doesn't particularly upset me, though.)

I do have more sympathy for open source developers. They are not=20
trying to profit from the security of their software, so I think they=20
deserve a little leeway. BUT, fixes are usually contributed by outside=20
experts. The experts can't just guess "Oh, I bet Person A of the NASM=20
project needs help with security problems. I'll send him an email and=20
ask if he needs help." They need to know about the vulnerability=20
before they can attempt to fix it. If they're reading a full=20
disclosure mailing list, they'll know about the problem. Then they can=20
code up a fix, email it to the author, and bang, it's fixed. It all=20
starts with full disclosure, though. (Without full disclosure, the=20
author of the software would be on his own to fix it. With full=20
disclosure, someone more experienced can help him out. That's a Good=20
Thing and is what makes the Open Source movement work.)

Full disclosure makes the Internet more secure. It forces vendors to=20
fix their broken software, and it forces users to update their broken=20
software. Less broken-ness is good for everybody.

If you disagree, you are probably writing broken software and are=20
afraid of what your users will do to you when they find out about it. =20
Good luck with that, and remember: don't shoot the messenger. If you=20
wrote the buggy code, you have only yourself to blame.

Regards,
--=20
Jonathan Rockway <jrockw2@uic.edu>

D. J. Bernstein wrote:

>Crispin starts from these three examples of intrusions occurring _after_
>full disclosure, and---applying the principle ``post hoc, ergo propter
>hoc''---leaps to the astounding conclusion that the intrusions were
>_caused_ by full disclosure, i.e., that avoiding disclosure would have
>prevented the intrusions.
>
>
You are right, it is circumstantial evidence. But it is mighty strong
circumstantial evidence. Brown et al only study three vulnerabilities,
but it is not an uncommon result. Reportedly a primary reason that
Microsoft switched to monthly disclosures is that they were seeing a
bloom of malware attacks following every single disclosure, and they
wanted to batch them up to get control of those blooms.

>Crispin's conclusion is obviously incorrect. We've all seen reports of
>extensive damage caused by attackers exploiting security holes that
>_weren't_ publicly known before the attacks. Clearly the attackers are
>capable of reading software and finding security holes for themselves.
>This isn't rocket science.
>
>
That you can find instances of something other than full disclosure
causing a bloom of attacks does not invalidate the inference that full
and *abrupt* disclosure can cause a bloom of attacks. It just means that
full disclosure is not the *only* cause of a bloom. The above logic is
obviously faulty, even if it does include Latin words

However, if I may offer a different criticism of my own claim, there is
a question of the evidence of positive benefits of responsible
disclosure. Rescola presents data
http://www.usenix.org/events/sec03/tech/rescorla.html that even with a
well-timed responsible disclosure of a major vulnerability (OpenSSL
chunking bug) many many people just never bothered to patch. However,
this study concerns only a single vulnerability, and it is possible that
OpenSSL was not widely patched because it is reputed to be a difficult
upgrade to perform correctly.

>There is, by the way, a more subtle problem with the argument against
>full disclosure: the argument focuses entirely on short-term effects and
>ignores long-term effects.
>
Forcible disclosure with a time like as specified in the RFPolicy
http://www.wiretrip.net/rfp/policy.html would seem to have nearly
identical long-term effects with much less damaging short-term effects.
Is it your contention that a "patch up or else" disclosure policy like
the RFPolicy does *not* cause programmers to clean up their act? Can you
justify how abrupt disclosure encourages any better behavior than timed
disclosure at the discretion of the bug-finder?

> But the basic problem with the argument is
>that it's out of whack with reality. If you think that hiding security
>information keeps us safe, you're deluding yourself.
>
>
But I *do not* think that hiding security information keeps us safe.
Rather, I think that disclosing vulnerability information has impact on
attackers and defenders, and the *timing* of that disclosure, especially
with respect to the availability of a patch, has critical impact on who
it impacts and how much.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

Steven M. Christey wrote:
>Besides which packages were found to be vulnerable, it seems like it
>would be equally or more informative to know which other packages were
>audited and not found to have bugs. The bulk of the "7500 man-hours"
>were probably spent *confirming* the security of some of the software,
>and some students may have accidentally selected well-written
>software.

Sadly, that is intractably hard. As Dijkstra would be quick to remind
us, testing can prove the existence of bugs, but never their absence.
The problem is that confirming the security of any real software package
is far too hard to be done in any class project; it is a task that is
way beyond what is feasible in the time available.

Crispin Cowan tried to put together an effort (Sardonix) to get people
to audit software packages and report on which ones were audited and
confirmed to be secure. The problem is that this is ridiculously
hard, far harder than trolling for a security hole. There is also the
sociological problem that we reward people more for finding security
holes than for finding none.

To give you some idea of the cost of finding security holes in modern
software packages, let me relate some experience from the graduate
security class I teach. In Fall 2002, I gave the students a homework
where I asked them to pick some program of interest to them and spend 3-6
hours auditing its security. About half of the students found at least
one security hole in the package they had picked. Because the exercise
was so well received by students, I gave the same homework in the class
this year, and saw about the same ratio. This is not a very scientific
experiment, but it suggests that if we pick a software package at random,
point a smart person with some security training (not necessarily a
security expert) at it, and let them have a few hours, then about 50% of
the time the software package will turn out to have some security hole.
Obviously, this is only a lower bound on how many of our applications
are insecure. I will leave you to speculate how many software packages
might harbor security flaws if we gave the auditor more time, training,
and resources.

If you're curious, you can see the homework assignments here:
http://www.cs.berkeley.edu/~daw/teac...works/hw2.html
http://www.cs.berkeley.edu/~daw/teac...works/hw2.html
Also, you can see some remarks I wrote back in Fall 2002 about the audits
people did that semester:
http://www.cs.berkeley.edu/~daw/teac...solns/hw2.html
You can see their audits from Fall 2002 on the Sardonix web site:
https://www.sardonix.org/Audited.html

Anyway, the point is that with 3-6 hours, you can often find a security
flaw in a real software package -- but confirming the security of
a security package in that amount of time is hopelessly impractical.
Coming to any confidence whatsoever that the package is free of bugs would
take orders of magnitude more time than available in the aforementioned
homework exercises. It is unfortunate that it is not easier to verify
the security of the software we rely upon, but these are the facts of
life today.

You asked about the relative proportion of different kinds of security
flaws. Of the audits I saw in my class, most were buffer overruns, and
there were a number of access control flaws (either missing or incorrect
access control measures). However, I wouldn't read too much into this --
the students only saw in class a few examples of implementation flaws,
so they might not have been looking for other kinds of flaws. Also, I
asked them to use RATS to help them with part of their audit, and that
tends to direct your attention more towards low-level implementation
defects that RATS can find with simple lexical analysis rather than
higher-level application security issues.