gmane.comp.security.openwall.announce

[openwall-announce] new Owl on CD; JtR 1.7.6-jumbo-7

Solar Designer — 2010-08-22T18:46:06

Hi,

This is to announce two things at once:

1. The July 29 snapshot of Owl-current is now available for purchase on CD
(both 32- and 64-bit):

http://www.openwall.com/Owl/order

(in fact, it's been available on CD since August 7).

2. The jumbo patch for John the Ripper is now up to revision
1.7.6-jumbo-7 adding MSCHAPv2, several external modes, bugfixes, and
license updates:

http://www.openwall.com/john/#contrib

More detail on the changes:

http://www.openwall.com/lists/john-users/2010/08/22/1

Alexander

[openwall-announce] new Owl ISOs; JtR news; books; another phpass article

Solar Designer — 2010-08-03T01:06:17

Hi,

As usual, this is a cumulative announcement for several things at once.
I'll start with the newest and most important (for us at least):

1. New ISO images and pre-created OpenVZ container templates of
Owl-current for i686 and x86-64 are available on our FTP mirrors.  The
ISOs are also available via direct download links on the Owl homepage:

http://www.openwall.com/Owl/

We have once again updated Owl to use OpenVZ's latest kernel from their
"rhel5" branch, and we've switched to using RPM-packaged kernels, but in
a way allowing for easy non-packaged builds as well.  At the same time,
we've introduced support for the ext4 filesystem (in fact, it is now
offered by default for new installs), and we've improved CD bootup and
the installer ("settle") in numerous ways.  The packages of passwdqc,
strace, lftp, tcb, JtR, and Postfix have been updated to new versions,
and changes have been made to several other packages.  Please refer to
the more detailed announcement on owl-users:

http://www.openwall.com/lists/owl-users/2010/08/03/1

and to the change log:

http://www.openwall.com/Owl/CHANGES-current.shtml

This new snapshot will also be available for purchase on CD (both i686
and x86-64) in a couple of days from now:

http://www.openwall.com/Owl/order

2. Speaking of John the Ripper, examples of how to crack SMTP's AUTH
CRAM-MD5 and LM and NTLM challenge/response exchanges have been posted
to the john-users mailing list:

http://www.openwall.com/lists/john-users/2010/07/27/1 - SMTP AUTH CRAM-MD5
http://www.openwall.com/lists/john-users/2010/07/09/1 - LM and NTLM C/R

The jumbo patch is now up to revision 1.7.6-jumbo-6 (adding some
bugfixes), and it will be updated even further soon:

http://www.openwall.com/john/#contrib

Petur Ingi Egilsson wrote a step-by-step guide entitled "John the Ripper
on a Ubuntu 10.04 MPI Cluster":

http://www.openwall.com/lists/john-users/2010/07/10/1

3. We've setup a web page with recommended computer security books (and more):

http://www.openwall.com/books/

4. A short and very easy to follow article on introducing phpass
password hashing into an existing PHP application has been published on
the UE Developer website:

http://dev.myunv.com/articles/secure-passwords-with-phpass/

Alexander

[openwall-announce] JtR 1.7.6 is stable; Solaris packages; GI John; 1.7.6-jumbo-4

Solar Designer — 2010-07-05T05:45:01

Hi,

John the Ripper 1.7.6, originally released as a development version
because of the extent of the changes made, has just been re-labeled the
new stable version.  There hasn't been a single bug report against this
version since it was released over two weeks ago, yet people
successfully built, ran, and some even packaged it on a variety of
operating systems.  Specifically:

Steven M. Christensen of Sunfreeware has produced packages of JtR 1.7.6
for many versions of Solaris, both SPARC and x86, including both 32-bit
and 64-bit builds.  I've mirrored them here:

http://download.openwall.net/pub/projects/john/contrib/solaris/

GI John - Grid Implemented John the Ripper, a curious non-Openwall
project - has been updated to build upon JtR 1.7.6-jumbo-3:

http://gijohn.info

Meanwhile, I've updated the jumbo patch with some fixes to KRB5_*, which
were previously discussed on john-users.  The new revision is
1.7.6-jumbo-4, but that's a very minor update (compared to -jumbo-3):

http://www.openwall.com/john/#contrib

Alexander

[openwall-announce] Owl-current on CD; JtR DES crypt(3) and LM hash speedup

Solar Designer — 2010-07-04T23:11:24

Hi,

As usual, this is a cumulative announcement for several things at once.
These were previously tweeted about - http://twitter.com/openwall - and
posted on the news page - http://www.openwall.com/news

For this announcement, I'll group them into two categories:

1. It is now possible to get Openwall GNU/*/Linux -current snapshots on
CD (with delivery worldwide) - 32-bit and/or 64-bit (your choice).  The
pricing starts at $9.35 (which just covers our costs), but you're
encouraged to pick a more expensive option (which supports our project):

http://www.openwall.com/Owl/order

The intent is to keep recent -current snapshots available for purchase
on CD along with releases, although that will depend on demand or lack
thereof.  Previously, only the last release was available for purchase
on CD.

2. John the Ripper's bitslice DES code is being re-worked much further,
resulting in greater ease of use on multi-core systems, as well as in
major per-core speedups at LM hashes.

This includes optional OpenMP parallelization, which allows a single
"john" process, invoked in the usual manner, to take advantage of
multiple CPU cores when auditing DES-based Unix crypt(3) hashes.
(JtR 1.7.6 release only supported this kind of parallelization for
certain other/slower hash types.)

This also includes a new vectorization- and parallelization-friendly
key setup algorithm, which makes LM hash computations more than twice
faster per-core (as tested on x86-64) and allows for parallelization of
DES-based crypt(3) hash computations even for the single-salt case
(including parallelization of most of the key setup "overhead").

The current in-development yet publicly released patches for JtR 1.7.6
achieve the following performance numbers on a single Core i7 920 2.67 GHz
CPU (quad-core capable of running 2 threads per core):

LM hashes, single process, single thread (no OpenMP), "--test" - 39M c/s
... ditto, actual "incremental" mode run (more "overhead") - 30M+ c/s
... 8 simultaneous processes, combined "--test" speeds - 173M c/s
LM hashes, single process, 8 threads (OpenMP), "--test" - 65M c/s
... ditto, actual "incremental" mode run (more "overhead") - 45M+ c/s

DES crypt(3), 1 process, 8 threads (OpenMP), "--test", multi-salt - 10.2M c/s
... ditto, actual "incremental" mode run (more "overhead") - 10.0M+ c/s
DES crypt(3), 1 process, 8 threads (OpenMP), "--test", single salt - 8.6M c/s
... ditto, actual "incremental" mode run (more "overhead") - 8.1M+ c/s

These numbers for DES crypt(3) correspond to an OpenMP parallelization
efficiency of 80% to 90% (vs. multiple separate processes running the
non-OpenMP build with separate candidate password streams) - e.g., the
same system would do 11.5M c/s combined for multi-salt with separate
processes.  This slight efficiency loss may be compensated for by the
greater ease of use (just one JtR invocation to manage instead of 8) and
by likely more optimal order in which candidate passwords are tried when
there's just one stream of those.

Finally, here are some more exciting performance numbers for a dual Xeon
X5460 3.16 GHz server (8 CPU cores total) under light unrelated load:

LM hashes, single process, single thread (no OpenMP), "--test" - 45M c/s
... 8 simultaneous processes, combined "--test" speeds - 356M c/s
LM hashes, single process, 8 threads (OpenMP), "--test" - 64M c/s

The 356M c/s figure is pretty exciting.  Previously, one would expect
this kind of performance from a GPU, but here it is achieved with two
CPUs found in a single system, and even under light unrelated load.

The 64M c/s figure for the OpenMP build is pretty good, but not exciting -
we've already seen better speed for a single Core i7.  Unrelated system
load truly kills OpenMP performance in many cases, and the efficiency of
LM hash parallelization with OpenMP is not great anyway.

Now to DES-based crypt(3) on the dual Xeon:

DES crypt(3), 1 process, 8 threads (OpenMP), "--test", multi-salt - 21M c/s
DES crypt(3), 1 process, 8 threads (OpenMP), "--test", single salt - 15.5M c/s

The OpenMP parallelization efficiency is 67% to 86% - that is, even
better speeds may be achieved with 8 simultaneous processes - such as
24M c/s for multi-salt and 23M c/s for single salt.

The patches may be found at:

http://openwall.info/wiki/john/patches

Here are some john-users postings with even more detail and substantiation
for the performance numbers given above:

http://www.openwall.com/lists/john-users/2010/07/03/1
http://www.openwall.com/lists/john-users/2010/06/30/2

As usual, feedback is welcome - on the john-users list, please.

Alexander

[openwall-announce] passwdqc updates; JtR SHA-crypt/OpenMP tutorials; JtR MPI patch

Solar Designer — 2010-06-23T02:46:12

Hi,

This is to announce several things at once:

0. Besides staying on this mailing list, it is now possible to receive
more timely updates by following us on Twitter:

http://twitter.com/openwall

I've already made a few initial tweets to set the expectations.

1. A new version of our password/passphrase strength checker,
passwdqc 1.2.2, has been released.  This version makes minor Makefile
updates to make the "install" and "uninstall" targets with their default
settings friendlier to Solaris systems:

http://www.openwall.com/passwdqc/

At the same time, a wiki page with detailed Solaris-specific
instructions on setting up passwdqc has been created:

http://openwall.info/wiki/passwdqc/solaris

2. A Python package re-implementing some algorithms from passwdqc has been
created by Alastair Houghton.  It is found on the passwdqc contributed
resources list:

http://www.openwall.com/passwdqc/#contrib
http://alastairs-place.net/pwtools/

3. Detailed tutorials on cracking/auditing SHA-crypt hashed user
passwords on recent Ubuntu, Fedora, and Solaris 10 systems have been
posted to the john-users mailing list, separately for Linux (using
Fedora 12 as the specific example):

http://www.openwall.com/lists/john-users/2010/06/20/3

and for Solaris 10:

http://www.openwall.com/lists/john-users/2010/06/20/2

These include optional OpenMP parallelization instructions and examples
(to use multiple CPUs and/or CPU cores).  (These same instructions will
work for bcrypt and SunMD5 hashes as well.)

4. The jumbo patch for John the Ripper has been updated further to
revision 1.7.6-jumbo-3, and the MPI parallelization patch has been
updated to apply on top of this revision:

http://www.openwall.com/john/#contrib
http://www.openwall.com/lists/john-users/2010/06/23/1

Alexander

[openwall-announce] JtR 1.7.6 (and -jumbo-2); tcb 1.0.6; bitslice DES; articles

Solar Designer — 2010-06-15T06:24:06

Hi,

This is to announce several items at once:

1. John the Ripper 1.7.6 is out:

http://www.openwall.com/john/

The additions and changes since 1.7.5 are as follows:

* Generic crypt(3) support (enabled with "--format=crypt") has been
added for auditing password hash types supported by the system but not
yet supported by John's own optimized cryptographic routines (such as
"SHA-crypt" and SunMD5).

* Optional parallelization of the above has been implemented by means of
OpenMP along with glibc's crypt_r(3) or Solaris' MT-safe crypt(3C).

* Optional parallelization of John's own optimized code for the
OpenBSD-style Blowfish-based crypt(3) (bcrypt) hashes with OpenMP has
been added.

* A more suitable version of 32-bit x86 assembly code for Blowfish is
now chosen on Core i7 and similar CPUs (when they happen to run a 32-bit
build of John).

* More optimal DES S-box expressions for PowerPC with AltiVec (making
use of the conditional select operation) contributed by Dumplinger Boy
(Dango-Chu) have been integrated.

* The bitslice DES C source code has been reworked to allow for the use
of arbitrary SIMD intrinsics, which was previously only implemented for
AltiVec as a special case.

* Support for SSE2 and MMX intrinsics with bitslice DES (as an
alternative to the supplied assembly code) has been added (currently
only enabled for SSE2 on x86-64 when compiling with GCC 4.4+).

* Support for mixed-type longer virtual vectors (such as SSE2+MMX,
SSE2+ALU, AltiVec+ALU, and other combinations) with bitslice DES has
been added (not enabled by default yet, primarily intended for easy
benchmarks on future CPUs, with future compiler versions, with even more
SIMD instruction sets, and with different DES S-box expressions that
might be available in the future).

* The obsolete 32-bit SPARC assembly implementation of DES has been
dropped.

* The loader will now detect password hashes specified on a line on
their own, not only as part of an /etc/passwd or PWDUMP format file.

* When run in "--stdin" mode and reading candidate passwords from a
terminal (to be typed by the user), John will no longer mess with the
terminal settings.

* John will now restore terminal settings not only on normal termination
or interrupt, but also when forcibly interrupted with two Ctrl-C
keypresses.

And one last change that was supposed to be in 1.7.5, but it was not:

* A new numeric variable has been added to the word mangling rules
engine: "p" for position of the character last found with the "/" or "%"
commands.

I'd like to thank Dumplinger Boy (Dango-Chu) for the S-box expressions,
and Erik Winkler for testing the AltiVec+ALU vectors (which didn't
provide a speedup, unfortunately, at least not on a G5 with a specific
version of GCC).  So right now the default on PowerPC/AltiVec is to use
Dango-Chu's S-boxes as-is (although the source code has been reworked),
which obviously does provide a speedup over the previous version.

The weird vector types (on x86, x86-64, and PowerPC) may now be easily
enabled by editing the #if's in the corresponding .h files.  Please
experiment with this and post your results to the john-users mailing
list.  I am especially interested in SSE2+MMX and SSE2+ALU benchmarks on
AMD processors when building with GCC 4.5 since my testing was mostly
limited to Intel CPUs.  Also, I am interested in AltiVec+ALU and double
AltiVec benchmarks (vs. single AltiVec) on IBM's POWER6 and newer CPUs.

2. The jumbo patch has been updated to John the Ripper 1.7.6, and
additionally updated with fixes for previously-integrated contributions
and with additional modules.  The current revision is 1.7.6-jumbo-2:

http://www.openwall.com/john/#contrib

More detail on the changes:

http://www.openwall.com/lists/john-users/2010/06/15/1

3. The tcb suite, implementing our alternative password shadowing scheme
on Owl (and reused by a number of other systems), has been updated to
version 1.0.6.  The only change since version 1.0.5 is removal of a
faulty check for sparse files.  This change was needed for compatibility
with modern filesystems such as btrfs.  I'd like to thank Jim Darby for
identifying and reporting the problem to us.

http://www.openwall.com/tcb/

More detail on the change:

http://www.openwall.com/tcb/ChangeLog

Of course, Owl-current already includes tcb 1.0.6 and JtR 1.7.6.

4. We've setup a collection of papers, source code, etc. related to
bitslice implementations of DES (focusing on the S-boxes):

http://download.openwall.net/pub/projects/john/contrib/bitslice-des/

This was prompted by websites hosting important/unique content in this
area going down and then back up (for now).  With copies in our files
archive, this content has propagated to multiple FTP/HTTP mirrors by now.

More content is likely to be added.  Submissions are welcome.

5. My article entitled "How to manage a PHP application's users and
passwords", originally published on the Month of PHP Security website,
has been republished on the Openwall website (with slightly different
formatting):

http://www.openwall.com/articles/PHP-Users-Passwords

Also, it was declared the winning entry in the MOPS contest:

http://php-security.org/2010/06/10/winners-of-the-month-of-php-security/

You could want to check out other articles published under the MOPS,
including on static PHP source code analysis, fuzzing the PHP
interpreter, the many ways in which PHP can happen to process user
input, and more:

http://php-security.org/category/articlesandtools/

At the same time, three old Openwall security advisories have been
updated to focus on currently relevant aspects and turned into articles:

http://www.openwall.com/articles/

That's all for now. :-)

Alexander

[openwall-announce] How to manage a PHP application's users and passwords; JtR & OpenMP

Solar Designer — 2010-05-27T23:45:41

Hi,

This is to announce two items at once:

1. Last month, I wrote and submitted a lengthy article for the Month of
PHP Security (MOPS). This article, entitled "How to manage a PHP
application's users and passwords", is now published on the MOPS website:

http://php-security.org/2010/05/26/mops-submission-10-how-to-manage-a-php-applications-users-and-passwords/index.html

In this article/tutorial, I will guide you through the steps needed to
introduce proper (in my opinion at least) user/password management into
a new PHP application. I will start by briefly explaining
password/passphrase hashing and how to access the database safely. Then
we will proceed through several revisions of the sample program. We'll
start with a very simple PHP program capable of creating new users only
and having some subtle issues. We will gradually improve this program
adding functionality (logging in to existing user accounts, changing
user passwords, and enforcing a password policy) and "discovering" and
dealing with the issues. We will also briefly touch many related topics.

This article also serves as documentation on introducing phpass, our PHP
password hashing framework, into a PHP application. A tarball and ZIP
archive with the article (HTML) along with sample programs is available
for download from the phpass homepage:

http://www.openwall.com/phpass/

The table of contents is:

Introduction
Password/passphrase hashing
Salting
Stretching
Choice of the underlying cryptographic primitive
phpass - the password/passphrase hashing framework for PHP applications
The database (and how to access it safely)
SQL injections
What SQL injections are
How to deal with SQL injections
Prepared statements with PHP and MySQL
Employ the principle of least privilege
Schema
The sample program is born
How to create new users
What if the user already exists?
Avoid leaking server setup details
How to differentiate MySQL errors
The "Magic Quotes" issue
Input filtering
How to authenticate existing users
How to change user passwords
How to enforce a password policy
Future work
Timing attacks
Other related concerns
Randomly-generated passwords/passphrases
Randomness
Resetting forgotten passwords/passphrases
Online password guessing
Denial of Service (DoS) attacks
Password policy enforcement and usability concerns
Challenge/response authentication
Sessions
Licensing

That's it. You may want to check out other material posted on the MOPS
website as well.

2. John the Ripper's implementation of OpenBSD-style Blowfish-based
crypt(3) hashes is being parallelized with OpenMP (which is readily
available with recent C compiler versions, including with gcc). This is
expected to be made official with the next development release.
Meanwhile, there's a patch on the wiki:

http://openwall.info/wiki/john/patches

and here are benchmarks on 8-way x86-64 systems (Core i7 and Dual
quad-core Xeon):

http://www.openwall.com/lists/john-users/2010/05/08/1

and 32-way UltraSPARC T2 (quad-core, 8 threads per core):

http://www.openwall.com/lists/john-users/2010/05/16/2

The efficiency is very close to 100% (vs. multiple separate processes).
The quad-cores with SMT (Core i7 and UltraSPARC T2) show a 5.5x speedup
(over a single-process build/run of unpatched JtR 1.7.5). The "true"
8-core system shows a 7.9x speedup.

An advantage of this approach is in its transparency and reliability -
JtR parallelized in this way works as usual, including the
interrupt/restore functionality (only one .rec file is created).
A drawback is that this has to be implemented per hash type (and it's
been implemented for just one hash type so far).

Implementation for specific other hash types may be considered,
especially in response to commercial demand (the resulting code will be
available to everyone, as usual). Please let me know if interested.

Thank you for reading this far!

Alexander

[openwall-announce] JtR 1.7.5-jumbo-3; phpass 0.3; password recovery updates

Solar Designer — 2010-04-25T09:07:33

Hi,

This is to announce three news items at once.  I'll start with the latest:

1. The jumbo patch for John the Ripper 1.7.5 has been updated to
revision 3.  Most notably, this adds documentation on LM/NTLM
challenge/response authentication cracking (doc/NETNTLM_README),
improves the netntlm.pl script, and adds the "--config" option to
"john".  These changes have been contributed by JoMo-Kun:

http://www.openwall.com/john/#contrib
http://www.openwall.com/lists/john-users/2010/04/14/4

2. There's a new revision of our PHP password hashing framework -
phpass 0.3:

http://www.openwall.com/phpass/

This revision no longer requires the getmypid() PHP function (which a
few shared hosting providers disable) and it recognizes the "$H$" hash
encoding prefix (as used by phpBB3).

Also, the size of an array in the C reimplementation, which is unused by
the framework itself and is meant for testing the correctness of the PHP
implementation only, has been corrected.  (Obviously, I was careless
about that code; I should have reviewed it when I decided to include it
along with phpass.)

I'd like to thank Christian von Schultz for reporting the bug.

3. Rixler Software's password recovery products for Word, Excel, and
some other Microsoft Office documents have been added to Openwall's
collection of password recovery resources.  These new products use an
innovative approach where the 40-bit encryption key gets cracked within
seconds on Rixler's server, yet the actual "document" is not "revealed"
to the server (only some technical information is sent to the server).

http://www.openwall.com/passwords/microsoft-word
http://www.openwall.com/passwords/microsoft-excel
http://www.openwall.com/passwords/microsoft-office

At the same time, ElcomSoft's Microsoft SQL Server password change/reset
and WiFi WPA/WPA2-PSK password security auditing products (using the
processing power of GPUs of up to four modern NVidia or ATI graphics
cards when available!) have also been added to the collection:

http://www.openwall.com/passwords/microsoft-sql-server
http://www.openwall.com/passwords/wifi-wep-wpa-wpa2-psk

Enjoy!

Alexander

[openwall-announce] passwdqc 1.2.1; C/R algorithms

Solar Designer — 2010-03-29T12:52:26

Hi,

This is to announce two minor items at once:

1. passwdqc 1.2.1 is out:

http://www.openwall.com/passwdqc/

In this version, a password strength check has been adjusted to no
longer subject certain passwords that start with a digit and/or end with
a capital letter to an unintentionally stricter policy.

Those interested in more detail about this change may refer to the
verbose commit message and maybe the code changes here:

http://cvsweb.openwall.com/cgi/cvsweb.cgi/Owl/packages/passwdqc/passwdqc/passwdqc_check.c?only_with_tag=PASSWDQC_1_2_1

2. I've published a couple of enhanced challenge/response authentication
algorithms that I came up with while working on popa3d 10+ years ago:

http://openwall.info/wiki/people/solar/algorithms/challenge-response-authentication

The goal was to address the major drawback of existing simple C/R
schemes such as APOP and CRAM-MD5 (where these would require storage of
plaintext passwords or of plaintext-equivalents on the server, thereby
possibly making the setup less secure than it would be with simple
password authentication not involving C/R), yet not go all the way for
public-key crypto (stay simple).  This goal was achieved, although the
algorithms do have certain limitations.  They didn't fit in the existing
C/R exchanges supported in POP3 and in its existing extensions, hence
they never made it into popa3d.

Please feel free to reuse these.

Alexander

[openwall-announce] new OpenVZ kernel, new Owl ISOs and OpenVZ container templates

Solar Designer — 2010-03-23T03:51:14

Hi,

Today's ISO images and pre-created OpenVZ container templates of
Owl-current for x86 and x86-64 are currently propagating to our FTP
mirrors.  The ISOs are also available via direct download links right
off the Owl homepage:

http://www.openwall.com/Owl/
http://www.openwall.com/Owl/DOWNLOAD.shtml

We have updated Owl to use OpenVZ's latest kernel from their "rhel5"
branch (released on 03/18), with RHEL5 patches further updated from
Red Hat's latest stable kernel (released on 03/16) and with some minor
changes of our own.  Thus, we're ahead of OpenVZ official kernels in
terms of security fixes right now, and there have been quite a few of
those lately...

The packages of gzip, VIM, tcb, JtR, tcsh, quota, passwdqc, libnids,
pciutils, hdparm, and tar have been updated to new versions or
patchlevels, and changes have been made to several other packages (cpio,
glibc, bash to name a few).

Please refer to the Owl-current change log for more information on some
of the changes:

http://www.openwall.com/Owl/CHANGES-current.shtml

As usual, minor changes are only documented in those packages' change
logs.  Additionally, this time all packages have been rebuilt to ensure
that we avoid any surprises as we get closer to making a new release.

Alexander

[openwall-announce] passwdqc 1.2.0, screenshots, policy considerations, passwdqc-users mailing list

Solar Designer — 2010-03-16T15:24:48

Hi,

passwdqc, our proactive password/passphrase strength checking and policy
enforcement toolset, has been enhanced in many ways, bringing it up to
version 1.2.0:

http://www.openwall.com/passwdqc/

The pwqcheck program is now directly usable as the passwordcheck program
on OpenBSD - that is, to check users' passwords as they're set with the
"passwd" program, much like it is done on systems with PAM.  The man page
for pwqcheck and the PLATFORMS file have been updated to provide brief
instructions on setting this up:

http://www.openwall.com/passwdqc/PLATFORMS.shtml

pwqcheck is now also able to check multiple passwords/passphrases at
once - e.g., for policy testing on large password/passphrase lists.
Simply running "pwqcheck -1 --multi" reads passwords/passphrases to
check from standard input (until EOF) and prints the check status for
each.  This functionality was in fact used on large publicly-available
lists of cracked passwords to see/verify the effect of other changes
made in this version of passwdqc (described below).

The random passphrases offered by pam_passwdqc, pwqgen, as well as by
the passwdqc_random() function in libpasswdqc, will now encode more
entropy per separator character and per word, increasing their default
size from 42 to 47 bits.  The size of 42 bits was adequate to withstand
not-too-powerful attacks against bcrypt hashes that we use on Owl, but
it was inadequate with weaker hashes that many other systems use.  (In
fact, for the weakest hash types, such as those used by some "web apps",
47 bits is inadequate too, but we can't reasonably increase the default
much further.  Instead, those systems/programs should be fixed to use
hashes that implement "password stretching".)

Substring matching will now partially discount rather than fully remove
weak substrings, support leetspeak, and detect some common sequences of
characters (sequential digits, letters in alphabetical order, adjacent
keys on a keyboard).  The combined effect of these changes is that it
should become slightly easier to come up with a password that would pass
the requirements (there will be fewer spurious "based on a dictionary
word" rejections), yet the percentage of likely-crackable passwords
passing the checks should decrease.

The passphrase strength checking code will now detect and allow
passphrases with non-ASCII (8-bit) characters in the words.  This should
make it easier to use non-English passphrases.

A number of optimizations have been implemented resulting in significant
speedup of passwdqc_check() on real-world passwords.  This matters for
"pwqcheck --multi".

RPM packages can now be built out of the distribution tarballs.  This is
briefly described in the INSTALL file:

http://www.openwall.com/passwdqc/INSTALL.shtml

We've setup a web page with screenshots demonstrating the uses and setup
of passwdqc:

http://www.openwall.com/passwdqc/screenshots

and a wiki page with password strength policy considerations aimed at
systems administrators deploying and configuring passwdqc:

http://openwall.info/wiki/passwdqc/policy

We have also setup the passwdqc-users mailing list.  Please use it to
share your experience with passwdqc and ask questions.  The subscription
instructions are found right on the passwdqc homepage:

http://www.openwall.com/passwdqc/

Alexander

P.S. Social bookmarking buttons have been added to most pages on the
Openwall website, as well as on the Wiki.  Please use these to add your
favorite Openwall web pages to your favorite social websites.

[openwall-announce] file archive; wiki pages; JtR MPI patch

Solar Designer — 2010-03-11T22:53:05

Hi,

This is to announce three items at once, mostly related to John the Ripper
password cracker.

1. We've setup the Openwall file archive - a locally-hosted web-based
archive with current and old revisions of Openwall software releases,
user contributions, and other related files.  Previously, this content
was only available via FTP locally and from the mirrors.

The file archive is available at:

http://download.openwall.net

Of specific interest are user contributions and other files related to
John the Ripper (269 files as of this writing):

http://download.openwall.net/pub/projects/john/contrib/

Many directories contain README.txt files, which are automatically
displayed below the file lists.

2. New community wiki pages have been created on topics related to John
the Ripper: "How to retrieve and audit password hashes from remote Linux
servers" and "Sample password hash encoding strings":

http://openwall.info/wiki/john/tutorials/remote-linux
http://openwall.info/wiki/john/sample-hashes

Further contributions to the sample hashes page are welcome.

3. magnum has contributed a new MPI patch for John the Ripper, which
supports parallelization of cracking modes other than "incremental":

http://www.openwall.com/lists/john-users/2010/03/09/2
http://download.openwall.net/pub/projects/john/contrib/mpi/2010-magnum/
http://openwall.info/wiki/john/parallelization#Extended-efforts
http://www.openwall.com/john/doc/MODES.shtml

Older MPI patches were limited to just the "incremental" mode.

Alexander

[openwall-announce] JtR 1.7.5-jumbo-1 builds; jumbo patch update; passwdqc wiki page

Solar Designer — 2010-03-02T20:32:01

Hi,

This is to announce three items at once:

1. Erik Winkler has contributed Win32 and Mac OS X builds of John the
Ripper 1.7.5 with revision 1 of the jumbo patch.  I've placed these into
the contrib/ directory and updated the links at:

http://www.openwall.com/john/#contrib

The Mac OS X build is usable on a wide range of Mac hardware and
versions of Mac OS X - it is a universal binary with four architectures
and it is statically-linked against Erik's special build of OpenSSL.
Additionally, it uses faster bitslice DES S-box code for PowerPC with
AltiVec by Dumplinger Boy (Dango-Chu).

I posted some further detail on the Mac OS X build to john-users:

http://www.openwall.com/lists/john-users/2010/03/02/3

2. The jumbo patch for JtR 1.7.5 has been updated to revision 2.  It
turned out that I had inadvertently omitted a file from a pre-1.7.5 CVS
commit introducing the "p" numeric variable into the word mangling rules
engine.  I've included this functionality into 1.7.5-jumbo-2 now, and
indeed it will be in the next official version of JtR.  Additionally,
this update of the jumbo patch adds support for external filters to the
Markov mode (patch by Simon Marechal).

http://www.openwall.com/john/#contrib
http://www.openwall.com/lists/john-users/2010/03/02/2

3. A wiki page on passwdqc (our password/passphrase strength checking
and policy enforcement toolset) has been setup with pointers to
user-created OS-specific instructions and packages of passwdqc:

http://openwall.info/wiki/passwdqc
http://www.openwall.com/passwdqc/

You are encouraged to register for a wiki account and add more content
to the passwdqc page, as well as to create sub-pages (perhaps with
instructions specific to your OS).

Alexander

[openwall-announce] JtR 1.7.5; tcb 1.0.5

Solar Designer — 2010-02-26T03:25:29

Hi,

This is to announce two updates at once:

1. John the Ripper version 1.7.5 is out, along with its corresponding
jumbo patch update.  This is yet another development version.  There was
no specific focus for this update, so a variety of minor enhancements
were implemented (mostly in response to requests made, questions asked,
and issues raised on the john-users mailing list lately).

http://www.openwall.com/john/
http://www.openwall.com/john/#contrib

The changes since 1.7.4.2 are as follows:

* Support for the use of "--format" along with "--show" or "--make-charset"
has been added.

* The choice of .rec and .log filenames for custom session names has been
made more intuitive.

* A new numeric variable has been added to the word mangling rules engine:
"p" for position of the character last found with the "/" or "%" commands.

* Support for "\r" (character lists with repeats) and "\p0" (reference
to the immediately preceding character list/range) has been added to the
word mangling rules preprocessor.

* The undefined and undocumented behavior of some subtle word mangling
rules preprocessor constructs has been changed to arguably be more sensible.

* Some bugs were fixed, most notably JtR crashing on no password hashes
loaded (bug introduced in 1.7.4.2).

2. The tcb suite has been updated further to version 1.0.5.  The primary
change since version 1.0.4 is the reduction of the .data section size
and thus of on-disk size of some components by 256 KB when tcb is
compiled against Linux 2.6 kernel headers.

http://www.openwall.com/tcb/
http://www.openwall.com/tcb/ChangeLog

Alexander

[openwall-announce] Linux 2.4.37.9-ow1; tcb 1.0.4; crypt_blowfish 1.0.4; JtR 1.7.4.2-jumbo-3

Solar Designer — 2010-02-24T17:38:25

Hi,

This is to announce four minor updates at once:

1. The Linux 2.4 kernel patch has been updated to Linux 2.4.37.9. One
of the changes made between 2.4.37.7 and 2.4.37.9 is a security fix for
the e1000 Ethernet driver issue that could have allowed remote attackers
to bypass packet filters (CVE-2009-4536). The Linux 2.4.37.9-ow1 patch
additionally includes a post-2.4.37.9 fix for FAT filesystems:

http://www.openwall.com/linux/

http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.37.8
http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.37.9
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2009-4536
http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.4.37.y.git;a=commitdiff;h=940716e5206ebda003fca89b4ac1076b1fff5c99

2. We've released version 1.0.4 of our tcb suite (which implements the
alternative password shadowing scheme on Owl). In this version, a
non-security buffer overflow bug with more than NGROUPS_MAX groups per
user has been fixed. We do not treat the bug as a security issue
because there's no untrusted user input involved. Also, the bug is not
even triggerable with typical uses of tcb, where the groups array in
question will be a root user's (perhaps just one group).

http://www.openwall.com/tcb/
http://www.openwall.com/tcb/ChangeLog

3. There's a minor update of crypt_blowfish (version 1.0.4), our public
domain password hashing framework for C/C++. In this version, the check
for unsupported iteration counts has been corrected to reject certain
iteration counts that would previously be misinterpreted. Also, section
.note.GNU-stack has been added to the x86 assembly file to avoid the
stack area unnecessarily being made executable on Linux systems that use
this convention.

http://www.openwall.com/crypt/

On a related note, a Python interface to crypt_blowfish by Daniel Holth
has been added to the contributed resources list on the crypt_blowfish
homepage:

http://www.openwall.com/crypt/#contrib

4. Revision 3 of the jumbo patch for JtR 1.7.4.2 has been released,
adding support for cracking NTLMv2 challenge/response exchanges
(contributed by JoMo-Kun), as well as support for Oracle 11g SHA-1 based
hashes (contributed by Alexandre Hamelin):

http://www.openwall.com/john/#contrib
http://www.openwall.com/lists/john-users/2010/02/14/1
http://www.openwall.com/lists/john-users/2010/02/12/2

Alexander

[openwall-announce] new Owl ISOs, OpenVZ container templates; Debian integrates new passwdqc

Solar Designer — 2010-01-28T22:41:00

Hi,

This is to announce two unrelated items at once: Owl updates and
Debian's integration of new versions of passwdqc.  Let's start with Owl:

Fresh ISO images and pre-created OpenVZ container templates of
Owl-current for x86 and x86-64 (generated a few hours ago) are available
on our FTP mirrors (maybe not on all yet, but should be by tomorrow).
There are also direct download links for the ISOs on the Owl homepage:

http://www.openwall.com/Owl/

Yes, we're now generating not only ISOs, but also OpenVZ container
templates of the Owl userland.  These may be used on Owl and/or on other
Linux systems with OpenVZ.  The templates are found under
/pub/Owl/current/vztemplate on our FTP mirrors.  The size of an Owl
template file is around 120 MB, and a container instantiated from it
occupies around 400 MB of disk space.  This compares favorably with
pre-created templates of other Linux distros found on openvz.org
considering that our template is actually an almost-complete install of
the Owl userland, including all development tools and libraries, so it
is capable of (re)building our and third-party packages from source.

The "make vztemplate" target has been added to the Owl build
environment (which is, and has always been, publicly available,
including as a part of these ISOs), making it easy for us and for Owl
users to generate new OpenVZ container templates of the Owl userland.

The 32-bit x86 userland is now being built for "i686" (Pentium Pro and
above) by default.  With the previous ISO snapshot for 32-bit x86, we
made this move for the kernel.  Now we also made it for the userland.

The packages of JtR, Nmap, and pciutils have been updated to new
versions (yes, this means Nmap 5.21, which was released yesterday),
libtool and gzip had minor security vulnerabilities fixed, and changes
have been made to several other packages.  Please refer to the
Owl-current change log for more information on some of the changes:

http://www.openwall.com/Owl/CHANGES-current.shtml

Finally, there's now a wiki page on getting started with Owl's OpenVZ
support:

http://openwall.info/wiki/Owl/usage-examples/OpenVZ/getting-started

Now to Debian and passwdqc:

Martin F. Krafft adopted the passwdqc Debian package and brought it up
to date.  Our password/passphrase strength checking and policy
enforcement toolset now integrates nicely with PAM on Debian systems,
and command-line utilities as well as the shared library providing the
functionality will soon be available in separate packages.

Martin also wrote a blog post on the topic:

http://madduck.net/blog/2010.01.28:adopted-passwdqc/

Enjoy!

Alexander

[openwall-announce] JtR 1.7.4.2 and jumbo patch update

Solar Designer — 2010-01-19T11:07:39

Hi,

John the Ripper version 1.7.4.2 is out, along with its corresponding
jumbo patch update.  This is another development version, and this time
the focus was on performance improvements with very large password files
or sets of files.

http://www.openwall.com/john/
http://www.openwall.com/john/#contrib

The changes since 1.7.4 are as follows:

* Major performance improvements for processing of very large password
files or sets of files, especially with salt-less or same-salt hashes,
achieved primarily through introduction of two additional hash table
sizes (64K and 1M entries), changes to the loader, and smarter
processing of successful guesses (to accommodate getting thousands of
hashes successfully cracked per second).

* Many default buffer and hash table sizes have been increased and
thresholds for the use of hash tables lowered, meaning that John will
now tend to use more memory to achieve better speed (unless it is told
not to with the "--save-memory" option).

* Some previously missed common website passwords found on public lists
of "top N passwords" have been added to the bundled common passwords list.

* Some bugs introduced in 1.7.4 and affecting wordlist mode's elimination
of consecutive duplicate candidate passwords have been fixed.

The jumbo patches had many of the same performance bottlenecks addressed
or worked around for a while.  With this update, the performance
improvements have been (re)implemented in an "official" manner and
included into JtR itself, and a couple of additional bottlenecks have
been addressed.

Those interested in more detail may check the CVS commit messages at:

http://cvsweb.openwall.com/john

I was committing the changes one by one and with very descriptive commit
messages.  I added quite some source code comments, too.

On a related note, support for "generic" MD5-based hashes (optionally
salted or/and iterated) has been added to the jumbo patch (starting with
1.7.4-jumbo-2), due to code contributed by JimF.  Previously, this
existed as a separate "JimF patch", which one could apply on top of the
jumbo patch.

Enjoy, and please provide your feedback on the john-users list.

Alexander

[openwall-announce] JtR 1.7.4 and jumbo patch update

Solar Designer — 2009-12-26T13:42:28

Hi,

John the Ripper 1.7.4 is out, along with its corresponding jumbo patch
update.  This is a development version focusing on many improvements to
the word mangling rules engine.

http://www.openwall.com/john/
http://www.openwall.com/john/#contrib

The changes since 1.7.3.4 are as follows:

* Support for back-references and "parallel" ranges has been added to
the word mangling rules preprocessor.

* The notion of numeric variables (to be used for character positions
and substring lengths along with numeric constants supported previously)
has been introduced into the rules engine.  Two pre-defined variables
("l" for initial or updated word's length and "m" for initial or
memorized word's last character position) and 11 user-defined variables
("a" through "k") have been added.  Additionally, there's a new numeric
constant: "z" for "infinite" position or length.

* New rule commands have been added: "A" (append, insert, or prefix with
a string), "X" (extract a substring from memory and insert), "v"
(subtract and assign to a numeric variable).

* New rule reject flags have been added: ":" (no-op, for use along with
the "parallel" ranges feature of the preprocessor) and "p" (reject
unless word pair commands are allowed, for sharing of the same ruleset
between "single crack" and wordlist modes).

* Processing of word mangling rules has been made significantly faster
in multiple ways (caching of the current length, less copying of data,
code and data placement changes for better branch prediction and L1
cache usage, compiler-friendly use of local variables, code
micro-optimizations, removal of no-op rule commands in an initial pass).

* The default rulesets for "single crack" and wordlist modes have been
revised to make use of the new features, for speed, to produce fewer
duplicates, and to attempt additional kinds of candidate passwords (such
as for years 2010 through 2019 with "year-based" rules).

* The idle priority emulation code has been optimized for lower overhead
when there appears to be no other demand for CPU time.

* The default for the Idle setting has been changed from N to Y.

Speaking of the jumbo patch, besides having been updated to the new
version of JtR it includes only one change: a bug fix patch for
mysql_fmt.c and mysql05_fmt.c by JimF.

Please test and provide your feedback on the john-users mailing list.
Specifically, you may want to benchmark 1.7.3.4 vs. 1.7.4 with their
different john.conf files, as well as with 1.7.3.4's old john.conf file
(which should be valid for 1.7.4 as well).  That way, you'll separate
the effect of code optimizations within JtR vs. that of changes to the
rulesets and to the Idle setting.

You may also benchmark JtR 1.7.4 with Idle=Y (the new default) vs.
Idle=N.  I am specifically interested in such benchmarks on systems
other than Linux.  I might need to reconsider the change of default for
some or for all systems if it causes a significant slowdown on some
systems without much other load.

By "benchmarking", I refer to actual cracking runs (with the same
initial john.pot contents), not just "john --test".  The latter should
be unaffected by the changes between 1.7.3.4 and 1.7.4.

Enjoy, and please don't forget to provide your feedback!

Alexander

[openwall-announce] Owl moves to 2.6 kernels, integrates OpenVZ; new ISOs

Solar Designer — 2009-11-23T22:57:19

Hi,

Fresh ISO images of Owl-current for x86 and x86-64 (generated today) are
available on our FTP mirrors (well, maybe not on all yet, but should be
by tomorrow).  There are also direct download links on the Owl homepage
(pointing to a specific already-updated and fast mirror):

http://www.openwall.com/Owl/

These ISOs represent a major development milestone.  We have replaced
the default kernel with a 2.6 OpenVZ one (featuring optional
container-based virtualization), we've integrated OpenVZ tools (vzctl
and vzquota packages needed to create, control, examine, and/or destroy
OpenVZ containers), and we've dropped support for Linux 2.4 kernels
(although they're still supported in the maintained Owl 2.0-stable
branch - until our next release).

Besides various changes related to the new kernel and OpenVZ
integration, we happened to update vsftpd and diffstat to new upstream
versions.

Please refer to the Owl-current change log for more detailed information
on the changes:

http://www.openwall.com/Owl/CHANGES-current.shtml

The new ISOs for 32-bit x86 will now make use of and require a "686" CPU
and PAE (Pentium Pro and above), supporting more than 4 GB of RAM (up to
64 GB) and NX bit when present.  Of course, the 64-bit ISOs are (and
always were) even better in that respect.

On a related note, included below is some technical detail for the
curious.  Any feedback is welcome on the owl-users mailing list.

These are "development" ISOs (after all, this is Owl-current), yet we've
tested them quite a bit.  One of the curious tests was to rebuild the
entire Owl userland while CD-booted.  Yes, this is possible and it is
actually very easy to do, by typing:

mount /ram -oremount,size=750M # about the minimum, use more if you can
su - build
rm RPMS # remove the symlink to pre-built packages
make

For a stress test, I actually did the above on an old Dual-P3 system
with a mere 768 MB of RAM (no hard drives used, no swap), and in under
4 hours I had all of the userland packages rebuilt (on tmpfs, in RAM).
I could then use the "settle" installer to actually use the newly built
packages for an install if I wanted to.  There's normally no point in
going for a rebuild like that as we're providing all the packages on the
same ISOs pre-built, but it is a nice test and it shows the capabilities
of our live system.  This would make more sense, for example, if you did:

su - build
rm RPMS
rm native
cp -a /rom/world/native .
vi native/Owl/packages/dhcp/dhcp.spec
# Read the disclaimer, set BUILD_DHCP_CLIENT to 1, save changes and exit
make PACKAGE=dhcp

(Just tested the above on Owl-current-20091123-i686.iso booted up in
QEMU, it works.)

One of the next steps is for us to provide pre-created OpenVZ templates
of Owl itself (such that you can easily create containers with Owl in
them).  Meanwhile, you can experiment with pre-created templates of many
other Linux distributions available here:

http://download.openvz.org/template/precreated/

Indeed, we should also work on proper documentation for the new OpenVZ
related features of Owl.

Alexander

[openwall-announce] Linux 2.4.37.7-ow1; passwdqc 1.1.4; new Owl ISO; public domain source code snippets

Solar Designer — 2009-11-19T01:55:15

Hi,

This is to announce several things at once:

1. Linux 2.4.37.7-ow1 is out:

http://www.openwall.com/linux/

This is merely an update of the patch to the new 2.4.37.7 kernel
release, which fixes a number of security-related bugs:

http://www.kernel.org/pub/linux/kernel/v2.4/ChangeLog-2.4.37.7

One of these is documented as "fs: pipe.c null pointer dereference".
Let me use this opportunity to remind you that having vm.mmap_min_addr
set to a non-zero value is a must (e.g., it is set to 98304 on the
system I'm typing this on).  There are way too many NULL pointer
dereference bugs and they are and will be getting discovered too often
for reasonably keeping systems up-to-date with the fixes.  A better
strategy may be to treat possible vm.mmap_min_addr bypass bugs as higher
severity ones, simply because there's an expectation that there are a
lot fewer of these (if any are still left).  This is the strategy we're
going to use for Owl.  vm.mmap_min_addr has defaulted to non-zero
(specifically, 32768) in -ow patches and thus on Owl systems for a while.
Thus, we're not treating NULL pointer dereference bugs as "local root"
ones; instead, we'd treat possible vm.mmap_min_addr bypasses as such.

2. There's a new Owl-current ISO image for 32-bit x86 (generated on
November 17) available on our FTP mirrors:

http://www.openwall.com/Owl/DOWNLOAD.shtml

There's also a direct download link (using one of the mirrors) right on
the Owl homepage:

http://www.openwall.com/Owl/

This is a very minor update.  It uses Linux 2.4.37.7-ow1 as the kernel.

Quite possibly, this is the last Owl ISO snapshot to use a 2.4 kernel,
as we're working on fully switching Owl to 2.6 kernels.

3. We've released version 1.1.4 of passwdqc, our password/passphrase
strength checking and policy enforcement toolset:

http://www.openwall.com/passwdqc/

We declare version 1.1.4 the new "stable" release.  The changes since
1.1.3 are mostly limited to minor code and manual pages markup cleanups
(such as for proper formatting on OpenBSD, thanks to Kevin Steves and
Jason McIntyre).

We've learned that passwdqc releases are now being packaged for NetBSD:

http://pkgsrc.se/security/pam-passwdqc

(Many other OS distributions have been doing it for years.)

4. I have published some assorted source code snippets and frameworks
(mostly in C), which I placed in the public domain:

http://openwall.info/wiki/people/solar/software/public-domain-source-code

Some of these were available under Openwall before, some not.  Please
feel free to reuse these in your programs.

Alexander

[openwall-announce] fresh Owl ISOs; Linux 2.4.37.6-ow1; Packetfactory mirror

Solar Designer — 2009-10-26T08:29:29

Hi,

This is to announce three items at once (yes, I will be trying to make
postings to this list less frequent):

1. Fresh ISO images of Owl-current for x86 and x86-64 (generated on
October 25) are available on our FTP mirrors.  There are also direct
download links on the Owl homepage:

http://www.openwall.com/Owl/

These ISOs use Linux 2.4.37.6-ow1 as the kernel, and, compared to last
month's ISO snapshots, they contain updated versions of many packages
(vsftpd, iptables, passwdqc, cpio, e2fsprogs, strace, VIM, and xinetd),
as well as minor changes to some other packages.  As usual, the major
changes are documented:

http://www.openwall.com/Owl/CHANGES-current.shtml

Like last month, these updates are due to work by Dmitry V. Levin,
Michail Litvak, and me.

2. Speaking of the kernel, Linux 2.4.37.6 fixes a number of information
leak vulnerabilities.  One of these was already fixed in 2.4.37.5-ow1
(as used in last month's Owl-current ISOs), and the remaining ones may
or may not affect specific systems depending on both kernel and
userspace configuration.  The 2.4.37.6-ow1 patch additionally fixes a
typo in one of the information leak fixes (CVE-2009-3612).  The updated
kernel patch is available at the usual location:

http://www.openwall.com/linux/

3. We've setup an unofficial mirror of www.packetfactory.net.  We did
this because the main Packetfactory site appeared to have gone down
"permanently" (staying down for about a year), whereas much of its
content was still valuable.  The Packetfactory was hosting a number of
networking and network security projects (with a focus on raw IP
networking) and related publications.  All of this content is now
available on the mirror, although some of the projects (the actively
maintained ones) have since moved elsewhere.  The mirror is here:

http://packetfactory.openwall.net

Alexander