DJB's students release 44 *nix software vulnerability advisories

[Thor Larholm]

Widely deployed open source software is commonly believed to contain
fewer security vulnerabilities than similar closed source software due
to the possibility of unrestricted third party source code auditing.
Predictably, most users of open source software do not invest a
significant amount of time to audit the applications they use and now a
class of 25 students has discovered 44 vulnerabilities during a CS
course.

This small group of students highlights how individuals outside the
security industry without special security prerequisites can still
manage to outperform the average Bugtraq poster in sheer quantity of
discoveries. This adequately validates the typical estimate of between 5
and 15 errors in every thousand lines of code.

D.J. Bernstein (http://cr.yp.to/djb.html) is lecturing a course this
fall at the University of Illinois at Chicago called "MCS 494: Unix
Security Holes" (http://cr.yp.to/2004-494.html). One of the requirements
to pass the course was to find and exploit 10 previously undiscovered
security holes in currently deployed Unix software.=20

With a class of 25 students discovering 44 vulnerabilities most students
now expect to fail the course
(http://it.slashdot.org/article.pl?sid=3D04/12/15/2113202).

The 44 security advisories have been published at=20

http://tigger.uic.edu/~jlongs2/holes/


Ariel Berkman has discovered a remotely exploitable security hole in
2fax, a text-to-TIFF converter.
[remote] [control] 2fax 3.04 expandtabs overflows s buffer
http://tigger.uic.edu/~jlongs2/holes/2fax.txt

Limin Wang has discovered two remotely exploitable security holes in
abc2midi.=20
[remote] [control] abc2midi 2004.12.04 event_text overflows msg buffer;
event_specific overflows msg buffer
http://tigger.uic.edu/~jlongs2/holes/abc2midi.txt

Limin Wang has discovered a remotely exploitable security hole in
abc2mtex.
[remote] [control] abc2mtex 1.6.1 process_abc overflows key buffer
http://tigger.uic.edu/~jlongs2/holes/abc2mtex.txt

Limin Wang has discovered a remotely exploitable security hole in
abcm2ps.
[remote] [control] abcm2ps 3.7.20 put_words overflows str buffer
http://tigger.uic.edu/~jlongs2/holes/abcm2ps.txt

Yosef Klein has discovered a remotely exploitable security hole in
abcpp.
[remote] [control] abcpp 1.3.0 process_directive overflows token buffer
http://tigger.uic.edu/~jlongs2/holes/abcpp.txt

Limin Wang has discovered two remotely exploitable security holes in
abctab2ps.
[remote] [control] abctab2ps 1.6.3 write_heading overflows t; trim_title
overflows rest
http://tigger.uic.edu/~jlongs2/holes/abctab2ps.txt

Qiao Zhang has discovered two remotely exploitable security holes in
asp2php.
[remote] [control] asp2php 0.76.23 preparse() overflows token buffer;
preparse() overflows temp buffer
http://tigger.uic.edu/~jlongs2/holes/asp2php.txt

James Longstreet and Tom Indelli have discovered a remotely exploitable
security hole in bsb2ppm, a program to convert BSB image files to PPM
image
files.
[remote] [control] bsb2ppm 0.0.6 overflows line buffer
http://tigger.uic.edu/~jlongs2/holes/bsb2ppm.txt

Ariel Berkman has discovered a locally exploitable security hole in
ChangePassword, a YP/Samba/Squid password-changing tool.
[local] [control] ChangePassword 0.8 runs setuid shell
http://tigger.uic.edu/~jlongs2/holes/changepassword.txt

Danny Lungstrom has discovered a remotely exploitable security hole in
ChBg, a tool to change background pictures.
[remote] [control] chbg 1.5 simplify_path overflows res buffer
http://tigger.uic.edu/~jlongs2/holes/chbg.txt

Ariel Berkman has discovered a remotely exploitable security hole in
Convex 3D.
[remote] [control] Convex 3D 0.8pre1 readObjectChunk overflows
objectname buffer
http://tigger.uic.edu/~jlongs2/holes/convex3d.txt

Limin Wang has discovered a remotely exploitable security hole in
csv2xml.
[remote] [control] csv2xml 0.5.1 get_field_headers overflows token
http://tigger.uic.edu/~jlongs2/holes/csv2xml.txt

Ariel Berkman has discovered a remotely exploitable security hole in
CUPS.
[remote] [control] CUPS 1.1.22 hpgltops ParseCommand overflows buf
http://tigger.uic.edu/~jlongs2/holes/cups.txt

Bartlomiej Sieka has discovered several security problems in how
lppasswd, version 1.1.22 (current), edits /usr/local/etc/cups/passwd.
[local] [kill] CUPS 1.1.22 lppasswd ignores write errors, etc.
http://tigger.uic.edu/~jlongs2/holes/cups2.txt

Ariel Berkman has discovered a remotely exploitable security hole in
dxfscope, a viewer for DXF drawings.
[remote] [control] dxfscope 0.2 overflows ent_name buffer
http://tigger.uic.edu/~jlongs2/holes/dxfscope.txt

Ariel Berkman has discovered a remotely exploitable security hole in the
elm/bolthole filter program.
[remote] [control] elm/bolthole filter 2.6.1 save_embedded_address
overflows address buffer
http://tigger.uic.edu/~jlongs2/holes/elm-bolthole-filter.txt

Manigandan Radhakrishnan has discovered two remotely exploitable
security holes in ``Get and Resume Elite Edition'' (greed), an FTP/HTTP
downloading tool.
[remote] [control] greed 0.81p DownloadLoop overflows COMMAND;
DownloadLoop does not check for nasty characters
http://tigger.uic.edu/~jlongs2/holes/greed.txt

Wiktor Kopec and Matthew Dabrowski have discovered a remotely
exploitable security hole in html2hdml.
[remote] [control] html2hdml 1.0.3 remove_quote overflows print_buf
buffer
http://tigger.uic.edu/~jlongs2/holes/html2hdml.txt

Manigandan Radhakrishnan has discovered a locally exploitable security
hole in IglooFTP, at least version 0.6.1 (the current version in FreeBSD
ports).
[local] [control] IglooFTP 0.6.1 uses fopen in /tmp
http://tigger.uic.edu/~jlongs2/holes/iglooftp.txt

Yosef Klein has discovered a remotely exploitable security hole in
IglooFTP, at least version 0.6.1 (the current version in FreeBSD ports).
[remote] [control] IglooFTP 0.6.1 does not check for directory escapes
http://tigger.uic.edu/~jlongs2/holes/iglooftp2.txt

Tom Palarz and Limin Wang have discovered a remotely exploitable
security hole in jcabc2ps.
[remote] [control] jcabc2ps switch_voice() overflows t1 buffer
http://tigger.uic.edu/~jlongs2/holes/jcabc2ps.txt

James Longstreet has discovered a remotely exploitable security hole in
jpegtoavi.
[remote] [control] jpegtoavi 1.5 get_file_list_stdin overflows fn buffer
http://tigger.uic.edu/~jlongs2/holes/jpegtoavi.txt

Yosef Klein has discovered two remotely exploitable security holes in
junkie, an FTP client, version 0.3.1 (current).
[remote] [control] junkie 0.3.1 gui_popup_view_fly does not check for
nasty characters; ftp_retr does not check for directory escapes
http://tigger.uic.edu/~jlongs2/holes/junkie.txt

Stephen Dranger has discovered a remotely exploitable security hole in
LinPopUp, an instant-messaging tool
[remote] [control] LinPopUp 1.2.0 overflows sub_string buffer
http://tigger.uic.edu/~jlongs2/holes/linpopup.txt

Mohammed Khan and Danny Lungstrom have discovered a remotely exploitable
security hole in Mesh Viewer.
[remote] [control] Mesh Viewer 0.2.2 Mesh::type overflows s1 buffer
http://tigger.uic.edu/~jlongs2/holes/meshviewer.txt

Bartlomiej Sieka has discovered a remotely exploitable security hole in
mpg123.
[remote] [control] mpg123 0.59r find_next_file overflows linetmp buffer
http://tigger.uic.edu/~jlongs2/holes/mpg123.txt

Ariel Berkman has discovered a remotely exploitable security hole in
MPlayer.
[remote] [control] MPlayer 1.0pre5 get_header overflows data buffer
http://tigger.uic.edu/~jlongs2/holes/mplayer.txt

Bartlomiej Sieka has discovered a remotely exploitable security hole in
NapShare, at
least version 1.2 (the current version in FreeBSD ports).
[remote] [control] NapShare 1.2 auto_filter_extern overflows filename
buffer
http://tigger.uic.edu/~jlongs2/holes/napshare.txt

Jonathan Rockway has discovered a remotely exploitable security hole in
NASM.
[remote] [control] NASM 0.98.38 error() overflows buff[]
http://tigger.uic.edu/~jlongs2/holes/nasm.txt

Wiktor Kopec has discovered a remotely exploitable security hole in
o3read, a converter for SXW files.
[remote] [control] o3read 0.0.3 parse_html overflows t buffer
http://tigger.uic.edu/~jlongs2/holes/o3read.txt

Danny Lungstrom has discovered two remotely exploitable security holes
in pcal.
[remote] [control] pcal 4.7.1 getline overflows tmpbuf; get_holiday
overflows tmp
http://tigger.uic.edu/~jlongs2/holes/pcal.txt

Tom Palarz and Kris Kubicki have discovered a remotely exploitable
security hole in pgn2web, a converter from PGN-format chess games to web
pages.
[remote] [control] pgn2web 0.3 process_moves overflows token buffer
http://tigger.uic.edu/~jlongs2/holes/pgn2web.txt

Jonathan Rockway has discovered that qwik-smtpd, version 0.3, allows
spammers to freely relay mail.
[remote] [exhaust] qwik-smtpd overflows clientHelo buffer
http://tigger.uic.edu/~jlongs2/holes/qwik-smtpd.txt

Qiao Zhang has discovered a remotely exploitable security hole in
ringtonetools.
[remote] [control] ringtonetools 2.22 parse_emelody overflows song
buffer
http://tigger.uic.edu/~jlongs2/holes/ringtonetools.txt

Limin Wang has discovered a remotely exploitable security hole in
rtf2latex2e.
[remote] [control] rtf2latex2e 1.0fc2 ReadFontTbl overflows buffer
http://tigger.uic.edu/~jlongs2/holes/rtf2latex2e.txt

Yosef Klein has discovered a remotely exploitable security hole in
tnftp, an FTP client, version 20030825 (current at least in FreeBSD
ports).
[remote] [control] tnftp 20030825 does not check for directory escapes
http://tigger.uic.edu/~jlongs2/holes/tnftp.txt

Danny Lungstrom has discovered that uml_net allows local users to take
down the computer's Ethernet connection.
[local] [kill] uml-utilities 20030903 uml_net slip_down() fails to check
permissions
http://tigger.uic.edu/~jlongs2/holes/uml-utilites.txt

Yosef Klein and Limin Wang have discovered a remotely exploitable
security hole in unrtf.
[remote] [control] unrtf 0.19.3 process_font_table overflows name buffer
http://tigger.uic.edu/~jlongs2/holes/unrtf.txt

Qiao Zhang has discovered a remotely exploitable security hole in vb2c.
[remote] [control] vb2c 0.02 parse_sub overflows token buffer
http://tigger.uic.edu/~jlongs2/holes/vb2c.txt

Ariel Berkman has discovered a remotely exploitable security hole in
vilistextum, an HTML-to-text converter.
[remote] [control] vilistextum 2.6.6 get_attr overflows temp buffer
http://tigger.uic.edu/~jlongs2/holes/vilistextum.txt

Ariel Berkman has discovered a remotely exploitable security hole in
xine-lib.
[remote] [control] xine-lib open_aiff_file overflows buffer
http://tigger.uic.edu/~jlongs2/holes/xine-lib.txt

Tom Palarz and Kris Kubicki have discovered a remotely exploitable
security hole in xlreader, a program to read Excel files.
[remote] [control] xlreader 0.9.0 overflows insert_start buffer
http://tigger.uic.edu/~jlongs2/holes/xlreader.txt

Manigandan Radhakrishnan has discovered a remotely exploitable security
hole in YAMT, an MP3-organization tool.
[remote] [control] YAMT 0.5 id3tag_sort does not check for nasty
characters
http://tigger.uic.edu/~jlongs2/holes/yamt.txt

Ariel Berkman has discovered a remotely exploitable security hole in
Yanf.
[remote] [control] Yanf 0.4 get() overflows buf
http://tigger.uic.edu/~jlongs2/holes/yanf.txt


Regards

Thor Larholm
Senior Security Researcher
PivX Solutions
23 Corporate Plaza #280
Newport Beach, CA 92660
http://www.pivx.com
thor@pivx.com
Stock symbol: (PIVX.OB)
Phone: +1 (949) 231-8496
PGP: 0x4207AEE9
B5AB D1A4 D4FD 5731 89D6 20CD 5BDB 3D99 4207 AEE9

PivX defines a new genre in Desktop Security: Proactive Threat
Mitigation.=20
<http://www.pivx.com/qwikfix> =20

[Crispin Cowan]

Thor Larholm wrote:

>This small group of students highlights how individuals outside the
>security industry without special security prerequisites can still
>manage to outperform the average Bugtraq poster in sheer quantity of
>discoveries.
>
That might be just a tad overstated.

The slashdot article
http://it.slashdot.org/article.pl?sid=04/12/15/2113202 was submitted by
one of these students. The student said that he spent 300 hours on the
project. The class had 25 students, so if we assume that is typical,
that is 7500 man-hours to find 44 vulnerabilities, or 170 hours per bug.

I don't believe that this "outperforms" the typical bugtraq poster. More
likely, it shows that when you are a professor, you can mandate a lot of
work if you want to :)

> This adequately validates the typical estimate of between 5
>and 15 errors in every thousand lines of code.
>
>
How so? The assignment was to find bugs in "UNIX" code, which arguably
is at least 10,000,000 lines of code for a typical UNIX desktop, which
should have over 50,000 bugs. That the class could find approx. 50 of
them does not come close to validating a rate that predicts 50,000.

None of which is to denigrate the fine work that DJB and his class have
done. I just don't think it validates the claims that Thor says it does.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

[cees-bart]

Thor Larholm wrote:

>This small group of students highlights how individuals outside the
>security industry without special security prerequisites can still
>manage to outperform the average Bugtraq poster in sheer quantity of
>discoveries. This adequately validates the typical estimate of between 5
>and 15 errors in every thousand lines of code.
>
>
Most of the 44 posted "security" advisories are about software bugs with
a very low security risk. See for example the posted bug on NASM
(http://tigger.uic.edu/~jlongs2/holes/nasm.txt): what's the chance of an
evil asm file being sent to an ignorant user that calls nasm to compile
this file? And this nasm bug is then called a "remotely exploitable
security hole". If I mail out a shell script that does "rm -rf $HOME/*",
this can also be considered a remotely exploitable security hole.

A proper (wide-scale) remotely exploitable security hole is one than can
be exploited without any ignorant user on the other side: for example,
the bug Windows Messenger service which was enabled by default, making
every user vulnerable, regardless of their stupidity.

>With a class of 25 students discovering 44 vulnerabilities most students
>now expect to fail the course
>(http://it.slashdot.org/article.pl?sid=04/12/15/2113202).
>
>

I think punishing students that have actually found security holes does
not make the world a better place ;)



--
cees-bart.

[security curmudgeon]

: Widely deployed open source software is commonly believed to contain
: fewer security vulnerabilities than similar closed source software due
: to the possibility of unrestricted third party source code auditing.
: Predictably, most users of open source software do not invest a
: significant amount of time to audit the applications they use and now a
: class of 25 students has discovered 44 vulnerabilities during a CS
: course.

: D.J. Bernstein (http://cr.yp.to/djb.html) is lecturing a course this
: fall at the University of Illinois at Chicago called "MCS 494: Unix
: Security Holes" (http://cr.yp.to/2004-494.html). One of the requirements
: to pass the course was to find and exploit 10 previously undiscovered
: security holes in currently deployed Unix software.
:
: With a class of 25 students discovering 44 vulnerabilities most students
: now expect to fail the course
: (http://it.slashdot.org/article.pl?sid=04/12/15/2113202).
:
: The 44 security advisories have been published at
:
: http://tigger.uic.edu/~jlongs2/holes/

In each case, Professor Bernstein notified the author of the vulnerable
package on Dec 15 via e-mail. This mail hit Bugtraq on the 16th, giving
one day for vendors to provide fixes.

Is the class on responsible disclosure next semester perhaps?

[D. J. Bernstein]

Shu T. Messenger writes:
> In each case, Professor Bernstein notified the author of the
> vulnerable package on Dec 15 via e-mail. This mail hit Bugtraq on the
> 16th, giving one day for vendors to provide fixes.

Actually, I sent all of these notifications to the public securesoftware
mailing list (http://securesoftware.list.cr.yp.to) at the same time that
I sent them to the authors. It certainly wasn't my intention to give the
authors an extra day of self-delusion.

> Is the class on responsible disclosure next semester perhaps?

If you had bothered to look at the slides on the course web page, you
would have seen a half day dedicated to the topic, plus some examples on
subsequent days of how people react to full disclosure when they're
trying to protect their shoddy security practices.

The reason that the 16 students sent their 91 reports to me privately is
so that they wouldn't have to deal with people like you. It was entirely
my decision to send out these 44 public notices.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

[Julian T J Midgley]

On Fri, 17 Dec 2004, security curmudgeon wrote:
>
> In each case, Professor Bernstein notified the author of the vulnerable
> package on Dec 15 via e-mail. This mail hit Bugtraq on the 16th, giving
> one day for vendors to provide fixes.
>
> Is the class on responsible disclosure next semester perhaps?

To be honest, I was pleasantly surprised that DJB had bothered to contact
the authors at all. He once said, in a discussion about the
securesoftware mailing list, which he touted at the time as a competitor
to bugtraq:

"Immediate full disclosure, with a working exploit, punishes the
programmer for his bad code. He panics; he has to rush to fix the problem;
he loses users.

You're whining that punishment is painful. You're ignoring the effect
that punishment has on future behavior. It encourages programmers to
invest the time and effort necessary to eliminate security problems."

http://groups-beta.google.com/group/comp.security.unix/msg/e576548f53195b01

By which standard, 24 hours is the height of responsibility. Admittedly,
the vulnerabilities were notified to the securesoftware list
(http://securesoftware.list.cr.yp.to/archives.html) concurrently with
author notification, but since there have been only 57 messages sent to
that list in the last three years, 44 of which were the student discovered
vulnerabilities themselves, I doubt it has a large readership ;-)

The actual notifications to the world (via slashdot and later bugtraq)
don't seem necessarily to have occurred at DJB's instigation, so he may
have been intending to give the authors a chance after all.

Notwithstanding all that, the course itself seems like an excellent idea
to me, and it will be interesting to see if useful statistics on the rates
incidence of security holes in software, and techniques for detecting them
and, ideally, preventing their inclusion in the first place, come out of
it.

Julian

--
Julian T. J. Midgley http://www.xenoclast.org/
Cambridge, England.
PGP: BCC7863F FP: 52D9 1750 5721 7E58 C9E1 A7D5 3027 2F2E BCC7 863F

[Marcin Owsiany]

On Fri, Dec 17, 2004 at 01:16:44PM +0100, cees-bart wrote:
> Most of the 44 posted "security" advisories are about software bugs with
> a very low security risk. See for example the posted bug on NASM
> (http://tigger.uic.edu/~jlongs2/holes/nasm.txt): what's the chance of an
> evil asm file being sent to an ignorant user that calls nasm to compile
> this file?

You are right, that this is very low risk.

> And this nasm bug is then called a "remotely exploitable
> security hole".

Obviously it is not. I don't think it is even locally exploitable.

> If I mail out a shell script that does "rm -rf $HOME/*",
> this can also be considered a remotely exploitable security hole.

The difference between nasm executing arbitrary code and a shell script
causing a shell to execute "rm -rf $HOME/*" is that the first IS NOT
meant to do that, and the second IS meant to do that. So let's not
compare those two.

Actually while the bug in nasm is very low risk, it is such bugs that
make it difficult to build new software using other software as building
blocks - you would need to audit and fix nasm before you could use it in
an web-to-assembler-gateway :-)

regards,

Marcin
--
Marcin Owsiany <marcin@owsiany.pl> http://marcin.owsiany.pl/
GnuPG: 1024D/60F41216 FE67 DA2D 0ACA FC5E 3F75 D6F6 3A0D 8AA0 60F4 1216

"Every program in development at MIT expands until it can read mail."
-- Unknown

[Jonathan T Rockway]

Two points.

Regarding local versus remote, look at it this way: You have a 100%
secure system. Then you install NASM. Now a user FROM THE NETWORK can
send you some tainted assembly code for you to assemble and he can
compromise your account. That is why it is considered remote. Local
would mean that I, the attacker, need an account on the target machine to
compromise the target account. In this nasm case, I do not need an
account. That is why the wording "remote" was chosen.

Now in regards to full disclosure, I think you should all be happy that we
bothered to tell you all about these exploits. We could have selfishly
used them to compromise machines, but instead we wrote them up and mailed
them off to the users and the authors! That is very nice of us.

If you would like notification sooner than the "public", find the exploit
yourself. If I can find them, then surely anyone can.

Regards,
--
Jonathan Rockway <jrockw2@uic.edu>

[milw0rm Inc.]

/*
Two points.
Regarding local versus remote, look at it this way: You have a 100%
secure system. Then you install NASM. Now a user FROM THE NETWORK can
send you some tainted assembly code for you to assemble and he can
compromise your account.
*/

quote "for you to assemble"

Its a user error. Your not remotely exploiting anything but the trust
from the user.

//str0ke

[Dave Holland]

On Mon, Dec 20, 2004 at 05:14:22PM -0600, Jonathan T Rockway wrote:
> Two points.

Me too, two.

> Regarding local versus remote, look at it this way: You have a 100%
> secure system.

There's no such thing.

> Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account.

As has already been pointed out, a user from the network could send me a
shell script containing "rm -rf ~". I'd be mad not to inspect a random
unsolicited shell script before I ran it. I'd be mad not to inspect a
similarly unsolicited NASM file before I assembled it; ditto input to
csv2xml, abcm2ps, or many of the other "vulnerable" programs.

This is not to belittle the effort that went into finding these bugs.
Just to say that labelling them all as "remotely exploitable" is rather
overstating the case.

Dave
--
** Dave Holland ** Systems Support -- Special Projects Team **
** 01223 494965 ** Sanger Institute, Hinxton, Cambridge, UK **

[Thor]

> Two points.
>
> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account. That is why it is considered remote. Local
> would mean that I, the attacker, need an account on the target machine to
> compromise the target account. In this nasm case, I do not need an
> account. That is why the wording "remote" was chosen.

You still need a local account: the one used by the user coaxed into
assembling the code. The fact that the source of the code in your example
was from a network user does not make it a remote exploit; one could just as
easily write the code on a bubble gum wrapper and have the user type it in,
then assemble it. Your example simply bypasses this logic by providing
circumstances where the attacker need not possess the local account
credentials, though a local account is most certainly needed to exploit the
vulnerability.

> Now in regards to full disclosure, I think you should all be happy that we
> bothered to tell you all about these exploits. We could have selfishly
> used them to compromise machines, but instead we wrote them up and mailed
> them off to the users and the authors! That is very nice of us.
>
> If you would like notification sooner than the "public", find the exploit
> yourself. If I can find them, then surely anyone can.

I doubt seriously that many will read this statement and interpret it
magnanimously. The "you're lucky we didn't use it for criminal purposes"
position doesn't exactly leave one all warm and fuzzy. To me, there are
many other aspects of the "full-disclosure" debate that should be considered
within the context of how the community is affected before the "well, I
could have done this as an individual" statements are made.

T





> Regards,
> --
> Jonathan Rockway <jrockw2@uic.edu>
>
>

[Stephen Samuel]

Side effects is my issue here.

It'a a fine thing to encourage programers to design their software well,
and fix bugs quickly. I've got no problems with punnishing programmers
for bad code, but there are different issues to deal with in the classroom
and out in the wild.

If you wanted to run a course where students are failed for having one bug
in a 10,000 line project, they were aware of the standards going in, and
given the training and tools such that the standard wss achievable, not
only wouldn't I have a problem with it -- I'd encourage everybody I knew
to hire anybody who passed the course. (at least, anybody I knew who
gave a rat's ass about producing quality code).

When dealing with code out in the wild, however, you have two communities
to consider -- the programming community and the user community. Hmm. OK,
three -- there's also the cracking community.
We're now living in a world where the time from description of a bug
and creation of a commercially viable exploit can approach zero. The
'commercially viable' exploits are viruses and worms used to generate
zombie armies of tens of thousands of remote-controllable 'bot machines
that can spread spam, launch DDOS exploits and do whatever else their
controllers wish.


Out in the wild, there is a wide gamut of care about producing quality
code and fixing bugs. At one extreme is Microsoft taking about 6 months
to *NOT* fix a security bug in WMP 9. (The fix is only available in WMP
10 released this week). That the programmer who reported the bug to MS
would wait the 6 months for _any_ real response from MS, I would agree
with your probble determination that it is a disservice to the user
community and does little more than reward MS for "trying to protect
their shoddy security practices."

Near the other extreme are BSD coders working feverishly to fix a bug
(and find any related errors) with _theoretical_ secrity issues in
their code in mere hours because the code was security related and they
felt a very real responsibility to ensure that their community had secure
code.

I would think that an optimal responsible disclosure policy should be
designed to (as much as possible) punnish the former while enabling the
latter. Not all coders hav achieved your standard of never, ever
releasing buggy code at any time of their life. As much as we may wish
otherwise, I doubt that that day will arrive soon either. A realistic
disclosure policy needs to take into accunt the interests of the three
communities I mentioned above.

I would suggest that a realistic disclosure policy would:

1) Discourage and punnish shoddy security practices.
2) Enable responsible software maintainers to realisticly respond
to bugs.
3) Give the user community timely and usable information about bugs in
the code that they're using.
4) Minimize the ability of the cracker community to exploit these bugs
before 1 and 2 have

Your zero Time To Respond approach would address 1, but do nothing
to astisfy 2-4. It doesn't so much punnish bad programmers as it
does encourage crackers to read your missives. Even the best of
programmers have to sleep, and if you release your exploit just
exploit just as the team/person responsible for the maintenance of
a piece of software is going to sleep, crackers will have up to
8 hours to code and use your bug before the responsible programmer
even wakes up to read the email about exploited code.

Even if rd party coders have managed to generate a fix, there won't
be an official release of the fix until the responsibe programmers
have at least had an opportunity to vette the proposed fixes.

A reasonable and responsible disclosure policy should be to (unless
there are signs that an exploit is already in use) allow responsible
a realistic (if short) opportunity to fix the instant bug and audit
their code for similar errors -- thus at least allowing the
*possibility* that a fix could be released at the same time as the
announcement of the bug.

If a fix can be announced at the same time as the bug, users have an
opportunity to install the fix while crackers are still crafting a
commercial exploit.

I don't think that 24-48 hours (or even longer) is an unreasonable
arning time for unexploited bugs. I'm not asking for time to 'be
delusional', I'm asking for a reasonable ammount of time for a
responsible programmer to ensure that his/her user community is
properly served and protected from the effects of the bugs.

6 months with no fix, on the other hand is obscene. I honestly
doubt if companies like Microsoft will ever treat Security as
a genuine responsibility aas opposed to a PR issue. On the other
hand, it is far less likely that they'll start responding
reasonably if they don't even have a hope of being able to craft
a fix before crackers have created a 'live' exploit.

D. J. Bernstein wrote:
> Shu T. Messenger writes:
>
>>In each case, Professor Bernstein notified the author of the
>>vulnerable package on Dec 15 via e-mail. This mail hit Bugtraq on the
>>16th, giving one day for vendors to provide fixes.
>
>
> Actually, I sent all of these notifications to the public securesoftware
> mailing list (http://securesoftware.list.cr.yp.to) at the same time that
> I sent them to the authors. It certainly wasn't my intention to give the
> authors an extra day of self-delusion.
>
>
>>Is the class on responsible disclosure next semester perhaps?

Also posted: http://www.bcgreen.com/comments/2004/bug-release.html
--
Stephen Samuel +1(604)876-0426 samuel@bcgreen.com
http://www.bcgreen.com/~samuel/
Powerful committed communication. Transformation touching
the jewel within each person and bringing it to light.

[Devin Ganger]

Jonathan T Rockway writes:

> Regarding local versus remote, look at it this way: You have=20
> a 100% secure system. Then you install NASM. Now a user=20
> FROM THE NETWORK can send you some tainted assembly code for=20
> you to assemble and he can compromise your account. That is=20
> why it is considered remote. Local would mean that I, the=20
> attacker, need an account on the target machine to compromise=20
> the target account. In this nasm case, I do not need an=20
> account. That is why the wording "remote" was chosen.

By that logic, *no* exploit can ever be considered local. A truly remote
exploit doesn't rely on manual interaction from a local user on the
system.

> Now in regards to full disclosure, I think you should all be=20
> happy that we bothered to tell you all about these exploits. =20
> We could have selfishly used them to compromise machines,

When you're trying to argue that your actions were ethical and moral,
pointing out that you could have committed crimes isn't the way to do
it.

> but instead we wrote them up and mailed them off to the users and=20
> the authors! That is very nice of us.

No, it's the absolute minimum level of ethical behavior one expects from
serious security researchers. Nothing nice about it.

> If you would like notification sooner than the "public", find=20
> the exploit yourself. If I can find them, then surely anyone can.

One day, when you are writing software for distribution, someone will
find an error in your code. When that day comes, remember this attitude.
While those who discover those bugs and exploits in your software have
no obligation to give you advanced warning of your bugs, I'm certain
you'd be grateful to have that warning so you have time to fix the bug
before word goes public.

--=20
Devin L. Ganger Email: deving@3sharp.com
3Sharp LLC Phone: 425.882.1032 x 109
15311 NE 90th Street Cell: 425.239.2575
Redmond, WA 98052 Fax: 425.702.8455

[David F. Skoll]

On Mon, 20 Dec 2004, Jonathan T Rockway wrote:

> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account.

That's nonsense. If you have /bin/sh installed, I can send you a shell
script FROM THE NETWORK that will give me root access if you run it.
Therefore, every UNIX system on Earth has a remote hole, according to
your definition.

> Now in regards to full disclosure, I think you should all be happy
> that we bothered to tell you all about these exploits. We could
> have selfishly used them to compromise machines, but instead we
> wrote them up and mailed them off to the users and the authors!

Could you have? How, pray tell, would you compromise a machine with
the NASM exploit? Even if you have a local account, the NASM exploit
lets you run arbitrary code as... yourself. Big deal.

--
David.

[Stephen Harris]

On Mon, Dec 20, 2004 at 05:14:22PM -0600, Jonathan T Rockway wrote:
> Two points.
>
> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account. That is why it is considered remote. Local

And so we have a distinction without meaning.

I could take a machine with no external connectivity beyond a single
keyboard and monitor and by typing in a listing from a piece of paper
use a command to reformat the system and lose all data. Remote exploit?
By your usage of the term it is.

In your example, a local user MUST take action in order to perform
the exploit, therefore the exploit is local. Your "FROM THE NETWORK"
is meaningless; it could be sent on floppy or printout or via psychic
rays and get the same result.

> Now in regards to full disclosure, I think you should all be happy that we
> bothered to tell you all about these exploits. We could have selfishly

And thus years of discussion on this, and other, lists about responsibility
and concern goes out of the window. But that's a matter between you and
your conscience. Be happy!

--

rgds
Stephen

[laffer1]

I'm not so sure about this. In the nasm case, a local use must run nasm
therefore requireing a local account. In my opinion, remote exploits are
holes that I can attack from the network without waiting for a local user
to execute something, i.e. services that are running or exposed protocols.

As for the other comments in this thread about telling the vendor early, I
personally feel it helps users if the vendor has a few days to look at the
hole and devise a patch BEFORE everyone on the planet knows about it. You
punish users of software in addition to vendors. All software has a
security problem of one kind or another, and its silly to think that a
perfect application will every be written.

On Mon, 20 Dec 2004, Jonathan T Rockway wrote:

> Two points.
>
> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account. That is why it is considered remote. Local
> would mean that I, the attacker, need an account on the target machine to
> compromise the target account. In this nasm case, I do not need an
> account. That is why the wording "remote" was chosen.
>
> Now in regards to full disclosure, I think you should all be happy that we
> bothered to tell you all about these exploits. We could have selfishly
> used them to compromise machines, but instead we wrote them up and mailed
> them off to the users and the authors! That is very nice of us.
>
> If you would like notification sooner than the "public", find the exploit
> yourself. If I can find them, then surely anyone can.
>
> Regards,
> --
> Jonathan Rockway <jrockw2@uic.edu>
>

[Raymond M. Reskusich]

--/9DWx/yDrRhgMJTb
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Dec 20, 2004 at 05:14:22PM -0600, Jonathan T Rockway wrote:
> Two points.
>=20
> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account. That is why it is considered remote. Local
> would mean that I, the attacker, need an account on the target machine to
> compromise the target account. In this nasm case, I do not need an
> account. That is why the wording "remote" was chosen.

What you really have is a local exploit that you can sometimes trick a
person into executing for you. Not the same thing as a remote
exploit. You as a remote user have no way of compomising the system
without first penetrating the system in some other way. Just because
there is a social engineering method to achieve this penetration does
not make this a remote exploit.

I might add that if a person is assembling an untrusted piece of
assembly code, there is a decent chance that he is planning on
executing it.

>=20
> Now in regards to full disclosure, I think you should all be happy that we
> bothered to tell you all about these exploits. We could have selfishly
> used them to compromise machines, but instead we wrote them up and mailed
> them off to the users and the authors! That is very nice of us.

Yeah, you know, if the choice is between criminal activity and this
kind of announcement, you did ok, but I think we all assume that people
in the security community want to actually protect users from security
problems. Assuming that this is your goal, I think you can do better.

--=20
Raymond M. Reskusich
System Programmer/Administrator
CITES/Engineering Workstation Group
University of Illinois at Urbana-Champaign
reskusic@uiuc.edu
pgp key id: D9D38030


--/9DWx/yDrRhgMJTb
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFByIRCWZWv9dnTgDARAtp0AKCBx2Bl4qQCV/XLvwU3uSgqeFMF+QCfXMbh
qRduSGy5NG06vkn4KKkWIZk=
=kW8D
-----END PGP SIGNATURE-----

--/9DWx/yDrRhgMJTb--

[D. J. Bernstein]

Stephen Samuel writes:
> crackers will have up to 8 hours to code and use your bug

It's not _my_ bug. It's also not my student's bug. It's the _program's_
bug. Sorry to have to break the news to you, but the attacker has had a
_year_ to exploit the bug if the program was released a year ago.

You're under the delusion that the bug's existence and exploitability
were created by the messenger---the bug-report publisher---rather than
by the original programmer. I realize that this is a common delusion,
one of the big excuses for inadequate security efforts; one of the
virtues of full disclosure is that it forcibly overrides the delusion.

> I'm asking for a reasonable ammount of time for a responsible
> programmer to ensure that his/her user community is properly served
> and protected from the effects of the bugs.

Same delusion: you think that users are protected from security holes if
the security holes are patched before they're announced. Sorry, but
that's not nearly fast enough. Protecting the users means making the
programs secure before they're deployed in the first place.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

[Jonathan Rockway]

Hello everyone.

On 21 Dec 2004, at 1:59 PM, David F. Skoll wrote:

> If you have /bin/sh installed, I can send you a shell
> script FROM THE NETWORK that will give me root access if you run it.
> Therefore, every UNIX system on Earth has a remote hole, according to
> your definition.

/bin/sh exists to run shell commands. That is the purpose of the
shell. NASM, on the other hand, is designed to create object files
from assembly files. If NASM starts running arbitrary code on your
machine, it's doing something unauthorized. That is a security hole.
By typing "nasm file.S" you are not intending to authorize the author
of file.S to take over your account, right?

Also, could you please show me this shell script you speak of? All the
shell scripts I know of that give me root access require me to type the
root password. If you have found a way around this, then you are
correct, "every UNIX system on Earth has a remote hole". :)

You (not you personally, but many people from whom I received a reply)
seem to think that "local" and "remote" are indicators of severity, but
that's just not the case. The NASM hole I discovered, for example, is
not very severe. I estimate that a total of 0 users will have their
accounts compromised via this hole. But the possibility exists for a
remote user to compromise an account, so it is called remote. Local
holes require a local user to do something special, like write a
program that closes fd 2 and execs chsh (the intent being for chsh to
open /etc/passwd under fd 2, and then write something
attacker-controlled to "stderr" which is actually /etc/passwd.)

Summary: Local exploits are bad. Remote exploits are bad.

milw0rm' writes:

> > "for you to assemble"
> Its a user error. Your not remotely exploiting anything but the trust
> from the user.

The user trusts NASM to not wipe their homedir. NASM is betraying the
user's trust.

Many University classes utilize an assignment submission mechanism that
automatically compiles and runs submitted programs. The execution of
the compiled code is done in a jail, but the compiling is not (because
it's hard to get all the compiler executables and resource files into
the jail). The NASM bug allows someone to compromise the system in the
compiling stage.

Even if the compile step were done in a jail, the attacker could remove
the NASM binary, causing the system to not work for other users. It,
therefore, IMPOSSIBLE to build a secure auto-grading system that uses
NASM.

Wesley Shields writes:

> This is far from a remote vulnerability. That's like posting some
> tainted shellcode in an exploit and waiting for people to blindly
> compile and run it. Stupid users blindly running untrusted code is
not
> a remote vulnerability.

Yes, but stupid users compiling said code and being compromised IS a
vulnerability. If you run the executable that NASM produces, you get
what you deserve. Merely compiling it, though, should not allow
arbitrary code to be executed.

(This raises the question, "why would you compile it and not run it?"
to which I don't have a good answer. This detail limits the impact of
this particular bug.)

Roger A. Grimes writes:

> Do you want us to be thankful because you did not commit an illegal
> crime?

No, I want you to be aware that some people aren't interested in the
academic value of security research. They want to 0wn b0x3n, and they
are doing that without our help. Were this NASM exploit useful in
spreading viruses, it probably would have been discovered by a
malicious cracker and used to create botnets that relay spam.

Also, I am of the belief that laws do not equal safety. You can tell
me all you want that I can't hack your network, but until you deploy
secure software, you are vulnerable. Hacking your network is only
illegal if I get caught.

If you use secure software, however, it is, by definition, physically
impossible for you to be hacked. I would rather protect my data by
making it impossible to get at than by imprisoning anyone that tries to
get it. (For every one that you catch, how many do you miss?)

And finally, in regards to "responsible disclosure", here is the patch
for NASM:

old: vsprintf(buff, fmt, arg);
new: vsnprintf(buff, 1024, fmt, arg);

Setting buff[1023] to '\0' is a good idea, since vsnprintf won't do
that if vsprintf(buff, fmt, args) generates 1024 bytes.

Be careful to not let a user supply fmt, however, since that would lead
to a format string vulnerability which would allow an attacker to write
to any byte in memory. :-)

Regards,
--
Jonathan Rockway <jrockw2@uic.edu>

[Jonathan Rockway]

On 21 Dec 2004, at 3:22 PM, laffer1 wrote:

> As for the other comments in this thread about telling the vendor
> early, I personally feel it helps users if the vendor has a few days
> to look at the hole and devise a patch BEFORE everyone on the planet
> knows about it. You punish users of software in addition to vendors.
> All software has a security problem of one kind or another, and its
> silly to think that a perfect application will every be written.

Why are users using insecure software? Or rather, why do users accept
the fact that their software may be insecure?

Besides, full disclosure helps the users too.

I remember a few years ago when the major SSH remote hole was found. I
read about it on slashdot between classes. Since there was no patch
yet, but there was an exploit, I ssh'd into my home machine and turned
off ssh. Even if there wasn't an exploit, I wasn't going to leave a
vulnerable service up and running. By the time I got home, sshd was
patched, and I installed that patch. If nobody had disclosed the
threat to me, I could have been compromised. But, because I was
notified, I was able to take preventative measures.

The sooner someone told me about the problem, the sooner I was able to
protect myself from the threat.

Full disclosure is important because vendors will drag their feet if
they're the only ones who know about it. Imagine you are a student and
have a paper due "whenever". When are you going to write the paper?
Today? No, you'll do it later. After all, nothing bad happens if you
don't do it, and not doing it is much easier than doing it.

Humans and software vendors are LAZY. If there's no reason to do
something, they won't do it. Full disclosure forces the issue and puts
everything out in the open. No "it'll be ready in 90 days" stalling.
It will be ready NOW or users will look to more secure alternatives.
They will make the first move and choose a program that doesn't have
security problems to begin with. This is better for everyone (well,
except people with financial interest in selling crappy software. that
doesn't particularly upset me, though.)

I do have more sympathy for open source developers. They are not
trying to profit from the security of their software, so I think they
deserve a little leeway. BUT, fixes are usually contributed by outside
experts. The experts can't just guess "Oh, I bet Person A of the NASM
project needs help with security problems. I'll send him an email and
ask if he needs help." They need to know about the vulnerability
before they can attempt to fix it. If they're reading a full
disclosure mailing list, they'll know about the problem. Then they can
code up a fix, email it to the author, and bang, it's fixed. It all
starts with full disclosure, though. (Without full disclosure, the
author of the software would be on his own to fix it. With full
disclosure, someone more experienced can help him out. That's a Good
Thing and is what makes the Open Source movement work.)

Full disclosure makes the Internet more secure. It forces vendors to
fix their broken software, and it forces users to update their broken
software. Less broken-ness is good for everybody.

If you disagree, you are probably writing broken software and are
afraid of what your users will do to you when they find out about it.
Good luck with that, and remember: don't shoot the messenger. If you
wrote the buggy code, you have only yourself to blame.

Regards,
--
Jonathan Rockway <jrockw2@uic.edu>

[Valdis.Kletnieks@vt.edu]

--==_Exmh_-1328937180P
Content-Type: text/plain; charset=us-ascii

On Tue, 21 Dec 2004 14:59:15 EST, "David F. Skoll" said:
> Could you have? How, pray tell, would you compromise a machine with
> the NASM exploit? Even if you have a local account, the NASM exploit
> lets you run arbitrary code as... yourself. Big deal.

Do you audit every line of code you receive from the network? Even for a
package the size of Apache or the X11 distribution? And you miss the point -
if *I* can hand you a trojaned program that will run arbitrary code as
"yourself" *when I don't have a userid on your system*, I have a toehold on
your system.

Remember that "I get you to run arbitrary code as yourself" is the *primary*
way that spyware and zombie software get onto people's systems. So it's not
an academic moot point.

Having said that, running 'more' on the foo.S file will almost certainly show
up the exploit as a oddly formatted line. What is *much* more likely to
actually work is.. Hmm.. thinking for a moment..

Yeah.. ship software with "optional MMX for speed" support, and have the package's
Makefile invoke gcc. gcc will invoke the C preprocessor on the assembler source,
allowing for all sorts of #ifdef and #define magic to make the code look like
one thing but do another.

Probably take a *lot* longer for people to twig onto what was going on than the
Trojan that showed up in the Sendmail distrib and a number of other things a while
back - the ./configure script would compile-and-run a backdoor-shell program.

All the same, getting *any* program to execute arbitrary code other than what
the programmer intended is a *vulnerability*. The fact that some social engineering
is required to actually *exploit* the hole doesn't change the fact that there's
still a hole.

If I dig a deep hole, with lots of pointy poisoned sticks at the bottom, and
cleverly concealed with netting, there's *still* a hole there even if I fail
to convince you to take a stroll with me down this trail, and oh would you
mind going first, there's a narrow spot here.....

--==_Exmh_-1328937180P
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Exmh version 2.5 07/13/2001

iD8DBQFByP7+cC3lWbTT17ARAg5JAJ9EtdsZLt51z2sTMRT7ks QOld99UACeLT3B
d+G2Jmg2d9RbmNS5P7tMKFo=
=ex8O
-----END PGP SIGNATURE-----

--==_Exmh_-1328937180P--

[Steven M. Christey]

Besides which packages were found to be vulnerable, it seems like it
would be equally or more informative to know which other packages were
audited and not found to have bugs. The bulk of the "7500 man-hours"
were probably spent *confirming* the security of some of the software,
and some students may have accidentally selected well-written
software.

It might also be useful to know which individuals looked for which
kinds of vulnerabilities. For example, one report dealt with buffer
overflows in a long SQL query that was apparently controllable by the
user, but there was no mention of whether SQL injection
vulnerabilities were investigated. Another mentioned directory
traversal via ".." sequences but didn't mention "/absolute/pathname"
vulns (which could be thought of as the same general issue, except a
lot of software is vulnerable to one but not the other.)

The bulk of the reported issues seem to be classic buffer overflows, a
rate of about 70%, which is rather high compared to the industry-wide
average of about 20% in recent years. This could suggest an
unintended bias somewhere in the audits, e.g. in the tools or
techniques used by the students, or the types of packages that were
audited. However, there were insufficient details to know whether
these overflows resulted from newer-style attacks, e.g. by modifying
length fields to be inconsistent with the real length of the input.
And to be fair, the industry-wide figures for publicly reported issues
reflect their own biases.

About 20% of the issues weren't classifiable into well-known
categories, while the remainder fell into 5 categories, a small
percentage of the 20 or so well-known categories.

All that said, I agree that this is an excellent effort, and it would
be great to see even more focused activities like this in the academic
world.

- Steve

[David Eisner]

In the following, I assume that we want to minimize the probability that
a user falls victim to an attack based on a preexisting vulnerability in
a piece of software, one that you, the security researcher, have just
discovered. I'll call this probability P.


D. J. Bernstein wrote:

>It's not _my_ bug. It's also not my student's bug. It's the _program's_
>bug. Sorry to have to break the news to you, but the attacker has had a
>_year_ to exploit the bug if the program was released a year ago.
>
>

Only attackers who know about the bug, though. P is proportional to the
number of people, N, who know about the bug before it is fixed.

Observation 1: When you disclose the existence of the bug, you
increase N, which tends to increase P.

But P is also proportional to the time, T, that elapses before the bug
is fixed. By disclosing the existence of the bug, you greatly increase
the likelihood that the bug is fixed in the near term.

Observation 2: When you disclose the existence of the bug, you
decrease T, which tends to decrease P.

There is a trade-off.

>Same delusion: you think that users are protected from security holes if
>the security holes are patched before they're announced. Sorry, but
>that's not nearly fast enough. Protecting the users means making the
>programs secure before they're deployed in the first place.
>
>

But the question of vendor notification is one of software that *has*
been deployed with security bugs. In that case, notifying the vendor
before disclosure (with the understanding that public disclosure will
follow after a reasonable period of time) provides the benefits of
Observation 2, without the costs of Observation 1.

You would argue, I think, that by foregoing vendor notification, you put
the fear of God into programmers, and this will ultimately lead to the
eradication of bugs in the first place. I'm skeptical.

-David Eisner

[Steven M. Christey]

>> And this nasm bug is then called a "remotely exploitable
>> security hole".
>
>Obviously it is not. I don't think it is even locally exploitable.

Many of the DJB-reported issues deal with exploitation of errors in
parsers or converters for specific file formats. Many files are
typically "shared," such as images, text documents, data exchange
formats, and so on. So, there can be vectors in which the attacker
can send a crafted file across network-based channels - web, email,
whatever - that then could conceivably be processed by the user,
either manually or automatically.

Granted, this is a different scenario than might be encountered in
what's typically labeled a "remotely exploitable" vulnerability, but I
haven't seen any emerging terminology that's been able to make this
reasonable distinction. However, with the growing researcher interest
in finding vulnerabilities based on irregularities in file formats,
the need for better terminology is growing.

In addition to modeling the level of authentication needed, I've been
thinking that it might also be important to note how much user/victim
participation is required for activation of the exploit, i.e. whether
the issue can be automatically exploited by normal user activity
(e.g. by simply reading an email message) or whether there's some
social engineering involved. However, I haven't put much thought into
terminology for this besides:

- automatic: exploit is automatically activated as a result of
normal usage of the product

- complicit: requires some victim participation or inaction

- opportunistic: can not really control when, or if, the victim
activates the exploit

Exploitation of web or email client vulnerabilities that happen as
soon as someone reads a message might be called "automatic," e.g. if
there's a buffer overflow when preparing a preview of the message. If
the user needs to click on a button or two, e.g. to extract something
whose icon shows as a JPG when in fact it's an executable, that might
be "complicit." If you insert terminal escape sequences into some log
file in the hopes that an admin accidentally runs "more" or "grep" on
that log file from the proper terminal, then that might be
"opportunistic." Phishing might be regarded as either automatic or
complicit, depending on whether or not you think web client users
should check their browser's status bar every time they click on a
link. Again, though, these are just rough ideas.

- Steve

[Casper.Dik@Sun.COM]

>/bin/sh exists to run shell commands. That is the purpose of the
>shell. NASM, on the other hand, is designed to create object files
>from assembly files. If NASM starts running arbitrary code on your
>machine, it's doing something unauthorized. That is a security hole.
>By typing "nasm file.S" you are not intending to authorize the author
>of file.S to take over your account, right?

What other purpose does NASM have other than to compile code
and then, implicitely, run it?

I could buy the argument for a webbrowser or a wordprocessor;
but a assembler or compiler?

>Also, could you please show me this shell script you speak of? All the
>shell scripts I know of that give me root access require me to type the
>root password. If you have found a way around this, then you are
>correct, "every UNIX system on Earth has a remote hole". :)

Any script which exploits a local security hole would do.

>Setting buff[1023] to '\0' is a good idea, since vsnprintf won't do
>that if vsprintf(buff, fmt, args) generates 1024 bytes.

You should have paid better attention in class.

Casper

[Manning, Robert]

Actually, I think this would make for a fascinating course. If I were =
teaching it, I don't know that I would have insisted on each student =
finding 10 holes, that does seem too optimistic. Rather, I would think =
requiring students to document their efforts at attempting to find =
holes, and assign extra credit for those who actually discover them. =
That would allow all students an equal chance at a grade, without =
penalizing those who aren't fortunate enough (as it were) to run across =
a vulnerability.

This sort of course is well outside what those in academia think of as =
the normal CS curriculum (see =
http://www.computer.org/cspress/CATALOG/cs01499.htm). Instead of =
criticism, I would think that we in the software industry (and =
especially those of us who also teach CS) should champion inventive (and =
currently quite relevant) courses such as this. It is hard to convince =
those who make such decisions to allow such courses, especially in light =
of insufficient precedence (the bureaucratic inertia of academia rivals =
that of other government offices).

Arguing over how to classify an exploit seems to be fair game for this =
list, but you folks often forget just how 'l33t you are. You forget how =
long it took you to know what you know. College students are probably =
more over-worked than you are. If I were the instructor of this class, I =
would probably caution my students on submitting their work to lists =
like Bugtraq, and if they chose to, to be well aware of the flames that =
may result. Arguing over full disclosure or whether something is =
remotely exploitable or not misses the point of the class, near as I can =
tell.

Happy Holidays to all.
Robert Manning


> -----Original Message-----
> From: D. J. Bernstein [mailto:djb@cr.yp.to]
> Sent: Friday, December 17, 2004 8:25 PM
> To: bugtraq@securityfocus.com
> Subject: Re: DJB's students release 44 *nix software vulnerability
> advisories
>=20
>=20
> Shu T. Messenger writes:
> > In each case, Professor Bernstein notified the author of the
> > vulnerable package on Dec 15 via e-mail. This mail hit=20
> Bugtraq on the
> > 16th, giving one day for vendors to provide fixes.
>=20
> Actually, I sent all of these notifications to the public=20
> securesoftware
> mailing list (http://securesoftware.list.cr.yp.to) at the=20
> same time that
> I sent them to the authors. It certainly wasn't my intention=20
> to give the
> authors an extra day of self-delusion.
>=20
> > Is the class on responsible disclosure next semester perhaps?
>=20
> If you had bothered to look at the slides on the course web page, you
> would have seen a half day dedicated to the topic, plus some=20
> examples on
> subsequent days of how people react to full disclosure when they're
> trying to protect their shoddy security practices.
>=20
> The reason that the 16 students sent their 91 reports to me=20
> privately is
> so that they wouldn't have to deal with people like you. It=20
> was entirely
> my decision to send out these 44 public notices.
>=20
> ---D. J. Bernstein, Associate Professor, Department of Mathematics,
> Statistics, and Computer Science, University of Illinois at Chicago
>=20

[Crispin Cowan]

D. J. Bernstein wrote:

>Stephen Samuel writes:
>
>
>>I'm asking for a reasonable ammount of time for a responsible
>>programmer to ensure that his/her user community is properly served
>>and protected from the effects of the bugs.
>>
>>
>Same delusion: you think that users are protected from security holes if
>the security holes are patched before they're announced. Sorry, but
>that's not nearly fast enough. Protecting the users means making the
>programs secure before they're deployed in the first place.
>
>
In theory, theory is just like practice. But in practice, it isn't.

In theory, I agree with DJB: an infinitely-funded adversary can hire a
gang of code analysts to scour a code base looking for vulnerabilities
and *not* publish them, effectively manufacturing private exploits in
whatever quantity they are willing to pay for. It is conjectured that
certain foreign governments are actually doing this.

But in practice, there is a *substantial* amount of epidemiological data
that shows that wide-spread attacks against software follow shortly
after the disclosure (responsible or otherwise) of a vulnerability. See
Brown, Arbaugh et al A Trend Analysis of Exploitations
<http://www.cs.umd.edu/%7Ewaa/pubs/CS-TR-4200.pdf> for great data on
when attacks happen with respect to disclosure. Furthermore, if you
force a fire drill in releasing the security patch, you compromise the
quality of the patch. See my work on patch quality "Timing the
Application of Security Patches for Optimal Uptime", Beattie et al
Postscript
<http://immunix.com/%7Ecrispin/time-to-patch-usenix-lisa02.ps.gz>. or
ugly PDF <http://immunix.com/%7Ecrispin/time-to-patch-usenix-lisa02.pdf>.

So while I am sympathetic to DJB's passion for correct software and to
hell with the tender feelings of developers who ship buggy code, in
practice this kind of 0-day notice of vulnerabilities *mostly* just
harms end-users.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

[Jack Lloyd]

On Tue, Dec 21, 2004 at 02:34:20PM -0600, milw0rm Inc. wrote:
> /*
> Two points.
> Regarding local versus remote, look at it this way: You have a 100%
> secure system. Then you install NASM. Now a user FROM THE NETWORK can
> send you some tainted assembly code for you to assemble and he can
> compromise your account.
> */
>
> quote "for you to assemble"
>
> Its a user error. Your not remotely exploiting anything but the trust
> from the user.
>
> //str0ke

Would you also argue that if someone views a web page and is compromised
because of a bug in the HTML parser of their web browser, it is their fault?
Certainly, assembling and then executing code of unknown provenence is a dumb
idea. But at the point you assemble it, it is just ASCII text. There is a big
difference between running code that you don't know, and running code you do
know on potentially malicious data. The first might be asking for it, but you
do the second every time you connect to a network.

Jack

[Michal Zalewski]

On Tue, 21 Dec 2004, Jonathan Rockway wrote:

> /bin/sh exists to run shell commands. That is the purpose of the
> shell. NASM, on the other hand, is designed to create object files
> from assembly files. If NASM starts running arbitrary code on your
> machine, it's doing something unauthorized. That is a security hole.

Consider this example: I am sending you an e-mail telling you to type
"/usr/local/bin/game_of_pong AAAAAAAAAAAAAAAA(...)$ASCII_SHELLCODE". The
game_of_pong utility happens to be have a command-line parameter buffer
overflow problem. If you follow my instructions, you will get 0wned.

The aforementioned application has a flaw, and as a result, did something
it was not designed for. This does not constitute a remotely exploitable
vulnerability by our standards, however. If it did, *all* bugs would
belong to that category.

What happened is that the user had became a wetware REMOTE->LOCAL gateway
for all kinds of attacks, if he can be tricked into feeding untrusted and
unchecked input received over the network into programs that were never
designed to handle such information.

We may and should blame the author for writing shoddy code, but this is
not a remotely exploitable flaw in his software.

I am not saying such a two-factor attack fits the "local" label; I am
simply saying it does not belong to the "remote" bunch.

/mz
http://lcamtuf.coredump.cx/

[D. J. Bernstein]

http://www.cs.umd.edu/~waa/pubs/Windows_of_Vulnerability.pdf summarizes
reports to CERT of intrusions through three particular security holes.
Most intrusions occurred months or years after the holes were disclosed
to the public. (Let's assume that reports to CERT are noticeably
correlated with actual damage.)

Crispin starts from these three examples of intrusions occurring _after_
full disclosure, and---applying the principle ``post hoc, ergo propter
hoc''---leaps to the astounding conclusion that the intrusions were
_caused_ by full disclosure, i.e., that avoiding disclosure would have
prevented the intrusions.

Crispin's conclusion is obviously incorrect. We've all seen reports of
extensive damage caused by attackers exploiting security holes that
_weren't_ publicly known before the attacks. Clearly the attackers are
capable of reading software and finding security holes for themselves.
This isn't rocket science.

There is, by the way, a more subtle problem with the argument against
full disclosure: the argument focuses entirely on short-term effects and
ignores long-term effects. But the basic problem with the argument is
that it's out of whack with reality. If you think that hiding security
information keeps us safe, you're deluding yourself.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

[Crispin Cowan]

Steven M. Christey wrote:

>In addition to modeling the level of authentication needed, I've been
>thinking that it might also be important to note how much user/victim
>participation is required for activation of the exploit, i.e. whether
>the issue can be automatically exploited by normal user activity
>(e.g. by simply reading an email message) or whether there's some
>social engineering involved. However, I haven't put much thought into
>terminology for this besides:
>
> - automatic: exploit is automatically activated as a result of
> normal usage of the product
>
>
I call this class "worms", or more grammatically a class of remote
vulnerabilities subject to worm attack. where the malware can propagate
unassisted.

> - complicit: requires some victim participation or inaction
>
>
I call this class "viruses, same grammar hack as above. These require
the victim to click on something, or such like, before the malware can
propagate.

> - opportunistic: can not really control when, or if, the victim
> activates the exploit
>
>
I'm having a hard time seeing the difference between "complicit" and
"opportunistic".

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

[Crispin Cowan]

robert@dyadsecurity.com wrote:

>What you have to ask yourself here is what do you fear more?:
>A) Do you fear wide spread worm based attacks where everyone knows about
>the problems at about the same time, and is more annoying than
>devistating?
>
>B) Do you fear directed malice attacks using information that that the
>defense does not know about?
>
>For my customers I fear B far more than A.
>
>
Lets do the math. The vulnerable code has been around for a year. A
vulnerability is discovered. The software provider wants 2 weeks. Lets
say (generously) that there are 10 times as many SOHO users defending
against A as there are critical infrastructure users defending against B.

* If we do A (responsible disclosure) then we expose (say) X users
to 2/50 additional weeks of time in which an el33t hax0r has a
private 0day against them that might be deployed, so you get
X*0.04 of time/risk exposure.
* If we do B (full disclosure), then we save the critical
infrastructure people that X*0.04 risk days, and instead we get
the 10X of SOHO users exposed to 2 weeks in which there is no
patch available. Assume that the software had another year of
lifespan before it was obsoleted, so again there is a factor of
2/50 or 0.04 on the risk days, but there are 10X users, so the
total risk factor is X*0.4.

Netted out, the risk/benefit is roughly the same but multiplied by the
size of the respective communities, and the SOHO community is much, much
larger.

I further submit that not all critical infrastructure people agree with
you, and some would rather see responsible disclosure of
vulnerabilities. You can tell by the number of enterprises subscribing
to early warning services for responsible disclosure for big $$$.


>The stick your head in the sand approach to Vulnerability Disclosure is
>not the direction I want to see the industry go.
>
>
Neither is a colloquial ad homenim approach to describing your opponents
pie in the sky position :)

>> Furthermore, if you force a fire drill in releasing the security
>>patch, you compromise the quality of the patch. See my work on patch
>>quality "Timing the Application of Security Patches for Optimal
>>Uptime", Beattie et al Postscript
>><http://immunix.com/%7Ecrispin/time-to-patch-usenix-lisa02.ps.gz>. or
>>ugly PDF
>><http://immunix.com/%7Ecrispin/time-to-patch-usenix-lisa02.pdf>.
>>
>>
>Perhaps a Patch isn't the only option here. Perhaps changing vendors,
>or removing the vulnerable service is an alternative to being
>compromised, or installing a buggy patch. No matter what, without the
>vulnerability information, you are making an uninformed decision.
>
>
Indeed. Another approach is to deploy intrusion prevention technologies
such as Immunix <http://immunix.com/technology/> that can block 0-day
attacks without needing any specific knowledge of whatever DJB's class
will come up with next time.

>>So while I am sympathetic to DJB's passion for correct software and to
>>hell with the tender feelings of developers who ship buggy code, in
>>practice this kind of 0-day notice of vulnerabilities *mostly* just
>>harms end-users.
>>
>>
>I think this depends on the segment you're talking about. If you're
>talking about enterprise or mission critical systems, full disclosure
>should help the end user more than hurt, as they are more likely to have
>the resources available to take appropriate action. If you're talking
>about soho/home PC users, then I would agreee with your point above.
>
>
More precisely, if you are talking about a service that can be *taken
down* while waiting for a fix or a work-around, then full disclosure is
an advantage. If the service cannot (or will not) be taken down and will
be just left to run the risk of compromise, then responsible disclosure
helps. I submit that most enterprise systems are of the "damn the
torpedoes" school of thought.

>The delima for me only comes when the enterprise/mission critical
>infrastructure is the same as the home/soho infrastructure.
>
>
Microsoft: shared between SOHO and Enterprise
Linux: shared between SOHO and Enterprise
Solaris: Enterprise only, but declining share, rapidly being replaced
with Microsoft and Linux

This does not make me feel better about full disclosure :)

Big fat caveat: All of the above assumes that the investigator is
actually well-intentioned. Actual black hats will do whatever they want,
and what we all advocate is of no matter, so defenders have to be able
to withstand 0-days anyway.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

[Palmer, Paul]

Jonathan,

You touch on a couple of topics which are worthy of exploring further.

First, lets talk about the nature of risk. It is not black and white. It
is a continuum, very difficult to measure exactly but can usually be
estimated. In the security profession you quickly learn that not only
can you not eliminate risk, but in many cases it doesn't even make sense
to try. Your job is to manage and reduce risk as cost-effectively as
possible.

I agree that full disclosure serves a purpose. However, I disagree that
_immediate_ disclosure is the best solution because it imposes too much
risk on the end-users (the victims). In your example, you had the option
of disabling SSH until a patch was available. Others probably could not
pursue that course of action as the opportunity cost would have been too
high. That is, there are many enterprises in this world that absolutely
depend on SSH to function efficiently. Many probably didn't disable SSH
even when they knew there was an unpatched vulnerability. Were they
lazy? Were they just stupid? I doubt it. They most likely performed a
risk assessment even if it was made unconsciously. That is, they
estimated the potential cost to their organizations of leaving SSH
active until a patch was available against the lost revenue and/or
increased cost of doing business they would incur if they pulled SSH
until a patch was available. You did the same thing. However, in your
case the risk of leaving SSH active was probably much higher than the
cost of not having it.

What immediate disclosure does is immediately increases the total risk
incurred by the users of the vulnerable software until a patch becomes
available. Contrast this against what is currently generally accepted
practice: notify the vendor; give them a reasonable amount of time to
fix the immediate problem; give them a reasonable amount of time to
audit for any similar problems; give them a reasonable amount of time to
verify that they haven't introduced any other problems in the course of
fixing the current one; post your advisory after either a patch is
available or a reasonable time has expired. This practice greatly
reduces the risk to the end users (remember they ARE the victims here).
The vendor is still "punished", maybe not as harshly as you would
prefer, but these announcements do still take their toll. The key here
is that no unnecessary collateral damage is imposed upon the victims in
the interest of making sure that the "vendors get what is coming to
them"...

Now lets talk about the LAZY humans and software vendors. They aren't
any more lazy than you are. They are arguably quite efficient (as a
group) with their limited resources however. Perfectly secure software
has a development cost. In fact, it is a VERY high development cost.
This cost would have to be passed on to the customers and now we get
back into our discussion of risk and opportunity cost. If a vendor
provided a commercial version of SSH that they have made flawless for
$10,000-100,000 a copy, would you buy it? (Why so much? The cost of
development increases and must be passed on to the customer base. As the
cost goes up the potential customer base shrinks and therefore, the cost
burden per remaining customer also must increase. A very non-linear
relationship! Granted, this greatly simplifies a very complex subject in
its own right, but it is accurate enough for this discussion) I doubt
that you would. It wouldn't be very cost effective of you. Your personal
risk in dealing with the occasional SSH bug that occurs has an amortized
cost far less than $10,000 (the minimum opportunity cost of being more
secure in this scenario). You still use the publicly available SSH even
after its history of bugs don't you. Even when clearly presented with
evidence that the software has a history flaws (and likely has other as
yet undiscovered ones), you still do not choose a very significantly
more secure alternative of removing your system from the Internet. Does
that make you lazy, stupid, or smart/efficient/practical?

Why don't the commercial vendors of SSH produce a guaranteed flawless
product? Is it because they are full of lazy programmers or greedy
executives? I do not think so. I would argue that the marketplace
doesn't currently support it. Customers do want more secure products,
but they just are not willing to pay enough extra for it to support the
development costs.
=20
Paul

-----Original Message-----
From: Jonathan Rockway [mailto:jrockw2@uic.edu]=20
Sent: Wednesday, December 22, 2004 1:06 AM
To: bugtraq@securityfocus.com
Subject: Re: DJB's students release 44 *nix software vulnerability
advisories


On 21 Dec 2004, at 3:22 PM, laffer1 wrote:

> As for the other comments in this thread about telling the vendor
> early, I personally feel it helps users if the vendor has a few days=20
> to look at the hole and devise a patch BEFORE everyone on the planet=20
> knows about it. You punish users of software in addition to vendors.

> All software has a security problem of one kind or another, and its=20
> silly to think that a perfect application will every be written.

Why are users using insecure software? Or rather, why do users accept=20
the fact that their software may be insecure?

Besides, full disclosure helps the users too.

I remember a few years ago when the major SSH remote hole was found. I=20
read about it on slashdot between classes. Since there was no patch=20
yet, but there was an exploit, I ssh'd into my home machine and turned=20
off ssh. Even if there wasn't an exploit, I wasn't going to leave a=20
vulnerable service up and running. By the time I got home, sshd was=20
patched, and I installed that patch. If nobody had disclosed the=20
threat to me, I could have been compromised. But, because I was=20
notified, I was able to take preventative measures.

The sooner someone told me about the problem, the sooner I was able to=20
protect myself from the threat.

Full disclosure is important because vendors will drag their feet if=20
they're the only ones who know about it. Imagine you are a student and=20
have a paper due "whenever". When are you going to write the paper? =20
Today? No, you'll do it later. After all, nothing bad happens if you=20
don't do it, and not doing it is much easier than doing it.

Humans and software vendors are LAZY. If there's no reason to do=20
something, they won't do it. Full disclosure forces the issue and puts=20
everything out in the open. No "it'll be ready in 90 days" stalling. =20
It will be ready NOW or users will look to more secure alternatives. =20
They will make the first move and choose a program that doesn't have=20
security problems to begin with. This is better for everyone (well,=20
except people with financial interest in selling crappy software. that=20
doesn't particularly upset me, though.)

I do have more sympathy for open source developers. They are not=20
trying to profit from the security of their software, so I think they=20
deserve a little leeway. BUT, fixes are usually contributed by outside=20
experts. The experts can't just guess "Oh, I bet Person A of the NASM=20
project needs help with security problems. I'll send him an email and=20
ask if he needs help." They need to know about the vulnerability=20
before they can attempt to fix it. If they're reading a full=20
disclosure mailing list, they'll know about the problem. Then they can=20
code up a fix, email it to the author, and bang, it's fixed. It all=20
starts with full disclosure, though. (Without full disclosure, the=20
author of the software would be on his own to fix it. With full=20
disclosure, someone more experienced can help him out. That's a Good=20
Thing and is what makes the Open Source movement work.)

Full disclosure makes the Internet more secure. It forces vendors to=20
fix their broken software, and it forces users to update their broken=20
software. Less broken-ness is good for everybody.

If you disagree, you are probably writing broken software and are=20
afraid of what your users will do to you when they find out about it. =20
Good luck with that, and remember: don't shoot the messenger. If you=20
wrote the buggy code, you have only yourself to blame.

Regards,
--=20
Jonathan Rockway <jrockw2@uic.edu>

[Crispin Cowan]

D. J. Bernstein wrote:

>Crispin starts from these three examples of intrusions occurring _after_
>full disclosure, and---applying the principle ``post hoc, ergo propter
>hoc''---leaps to the astounding conclusion that the intrusions were
>_caused_ by full disclosure, i.e., that avoiding disclosure would have
>prevented the intrusions.
>
>
You are right, it is circumstantial evidence. But it is mighty strong
circumstantial evidence. Brown et al only study three vulnerabilities,
but it is not an uncommon result. Reportedly a primary reason that
Microsoft switched to monthly disclosures is that they were seeing a
bloom of malware attacks following every single disclosure, and they
wanted to batch them up to get control of those blooms.

>Crispin's conclusion is obviously incorrect. We've all seen reports of
>extensive damage caused by attackers exploiting security holes that
>_weren't_ publicly known before the attacks. Clearly the attackers are
>capable of reading software and finding security holes for themselves.
>This isn't rocket science.
>
>
That you can find instances of something other than full disclosure
causing a bloom of attacks does not invalidate the inference that full
and *abrupt* disclosure can cause a bloom of attacks. It just means that
full disclosure is not the *only* cause of a bloom. The above logic is
obviously faulty, even if it does include Latin words :)

However, if I may offer a different criticism of my own claim, there is
a question of the evidence of positive benefits of responsible
disclosure. Rescola presents data
http://www.usenix.org/events/sec03/tech/rescorla.html that even with a
well-timed responsible disclosure of a major vulnerability (OpenSSL
chunking bug) many many people just never bothered to patch. However,
this study concerns only a single vulnerability, and it is possible that
OpenSSL was not widely patched because it is reputed to be a difficult
upgrade to perform correctly.

>There is, by the way, a more subtle problem with the argument against
>full disclosure: the argument focuses entirely on short-term effects and
>ignores long-term effects.
>
Forcible disclosure with a time like as specified in the RFPolicy
http://www.wiretrip.net/rfp/policy.html would seem to have nearly
identical long-term effects with much less damaging short-term effects.
Is it your contention that a "patch up or else" disclosure policy like
the RFPolicy does *not* cause programmers to clean up their act? Can you
justify how abrupt disclosure encourages any better behavior than timed
disclosure at the discretion of the bug-finder?

> But the basic problem with the argument is
>that it's out of whack with reality. If you think that hiding security
>information keeps us safe, you're deluding yourself.
>
>
But I *do not* think that hiding security information keeps us safe.
Rather, I think that disclosing vulnerability information has impact on
attackers and defenders, and the *timing* of that disclosure, especially
with respect to the availability of a patch, has critical impact on who
it impacts and how much.

Crispin

--
Crispin Cowan, Ph.D. http://immunix.com/~crispin/
CTO, Immunix http://immunix.com

[David Wagner]

Steven M. Christey wrote:
>Besides which packages were found to be vulnerable, it seems like it
>would be equally or more informative to know which other packages were
>audited and not found to have bugs. The bulk of the "7500 man-hours"
>were probably spent *confirming* the security of some of the software,
>and some students may have accidentally selected well-written
>software.

Sadly, that is intractably hard. As Dijkstra would be quick to remind
us, testing can prove the existence of bugs, but never their absence.
The problem is that confirming the security of any real software package
is far too hard to be done in any class project; it is a task that is
way beyond what is feasible in the time available.

Crispin Cowan tried to put together an effort (Sardonix) to get people
to audit software packages and report on which ones were audited and
confirmed to be secure. The problem is that this is ridiculously
hard, far harder than trolling for a security hole. There is also the
sociological problem that we reward people more for finding security
holes than for finding none.

To give you some idea of the cost of finding security holes in modern
software packages, let me relate some experience from the graduate
security class I teach. In Fall 2002, I gave the students a homework
where I asked them to pick some program of interest to them and spend 3-6
hours auditing its security. About half of the students found at least
one security hole in the package they had picked. Because the exercise
was so well received by students, I gave the same homework in the class
this year, and saw about the same ratio. This is not a very scientific
experiment, but it suggests that if we pick a software package at random,
point a smart person with some security training (not necessarily a
security expert) at it, and let them have a few hours, then about 50% of
the time the software package will turn out to have some security hole.
Obviously, this is only a lower bound on how many of our applications
are insecure. I will leave you to speculate how many software packages
might harbor security flaws if we gave the auditor more time, training,
and resources.

If you're curious, you can see the homework assignments here:
http://www.cs.berkeley.edu/~daw/teaching/cs261-f02/homeworks/hw2.html
http://www.cs.berkeley.edu/~daw/teaching/cs261-f04/homeworks/hw2.html
Also, you can see some remarks I wrote back in Fall 2002 about the audits
people did that semester:
http://www.cs.berkeley.edu/~daw/teaching/cs261-f02/solns/hw2.html
You can see their audits from Fall 2002 on the Sardonix web site:
https://www.sardonix.org/Audited.html

Anyway, the point is that with 3-6 hours, you can often find a security
flaw in a real software package -- but confirming the security of
a security package in that amount of time is hopelessly impractical.
Coming to any confidence whatsoever that the package is free of bugs would
take orders of magnitude more time than available in the aforementioned
homework exercises. It is unfortunate that it is not easier to verify
the security of the software we rely upon, but these are the facts of
life today.

You asked about the relative proportion of different kinds of security
flaws. Of the audits I saw in my class, most were buffer overruns, and
there were a number of access control flaws (either missing or incorrect
access control measures). However, I wouldn't read too much into this --
the students only saw in class a few examples of implementation flaws,
so they might not have been looking for other kinds of flaws. Also, I
asked them to use RATS to help them with part of their audit, and that
tends to direct your attention more towards low-level implementation
defects that RATS can find with simple lexical analysis rather than
higher-level application security issues.