http://blog.gmane.org/gmane.comp.security.openwall.john.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.comp.security.openwall.john.user/3060 <pre>Bruce Schneier, and probably other SHA-3 competitors, but more specifically Bruce, is looking for people with FPGA and ASIC skillz. I figure someone on the list may know someone who knows someone with such a skill set and or interest. You can read more about it here: http://www.schneier.com/blog/archives/2010/09/wanted_skein_ha.html I'm just putting this out there (cleared with Solar first) to see if anyone can help. If so contact Mr Schneier or other entrants you may favor, and offer your help to them directly, I've posted this on my own volition. Pass along the link if you know anyone, thanks! -rich </pre> Rich Rumble 2010-09-02T02:52:59 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3052 <pre>Hi. I have a small time and convert the assembler multiply into sum as Solar do. The patch is apply on top the jumbo distribution. I make a rapid test in linux and mac and works well. saludos, alain </pre> Alain Espinosa 2010-09-01T21:45:01 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3050 <pre>Thanks for the reply. It does need to be 10 alnum as the passphrase won't just be digits (and I'd be using Mac OSX). So its not possible to just do 10-10 alphanum chars on JtR as far as anyone knows? Given that I don't want 9, 8, 7 or less alnum chars nor 11+ alnum chars *just 10*, seems quite a reasonable, not too overly time-consuming crack to want to feed through from John, I'd be surprised if its not possible. I take what you're saying in terms of generating a custom .chr file and have re-read http://www.openwall.com/john/doc/EXAMPLES.shtml but isn't that method merely a way of filtering, i.e. it won't actually make John do a min 10 / max 10, or am I just not understanding it right? -EX ----- Forwarded Message ---- From: Rich Rumble &lt;richrumble-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt; To: john-users-ZwoEplunGu1jrUoiu81ncdBPR1lH4CV8&lt; at &gt;public.gmane.org Sent: Wed, 1 September, 2010 15:35:49 Subject: Re: [john-users] Noob question: how to feed 10 alphanum char min&amp;max incremental to aircrack when "MaxLen = 10 exceeds the compile-time limit of 8" http://openwall.info/wiki/john/mailing-list-excerpts http://www.openwall.com/lists/john-users/2007/07/04/6 Specifically... You have to create your own CHR files and recompile. I have not had success with compiling anything other than digits over 8, you can DL my digit only build (up to 16 digits) here: hxxp://xinn.org/jtr__digits/john-digits-16.7z (windows only) If you don't have 7zip you can find it easily. This build will only crack digit only hahses with lengths from 1-16 digit characters long. The build is against JtR 1.7.6 jumbo rev 5. I'm not sure how to run it against a pcap, but if it's supported by JtR or the jumbo patch it should work. I should try again to build an AlphaNum JtR but I haven't had the time. You do have to modify the john.conf parameters to suite your needs: # Incremental modes [Incremental:Digits16] File = $JOHN/digits16.chr MinLen = 1 MaxLen = 16 CharCount = 10 Command line: john-digits-16.exe pass.pwd -i=digits16 -format=nt I hope that helps some... -rich On Wed, Sep 1, 2010 at 3:28 AM, Mr Ex &lt;ex_says-/E1597aS9LT10XsdtD+oqA&lt; at &gt;public.gmane.org&gt; wrote: modified </pre> Mr Ex 2010-09-01T21:18:07 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3046 <pre>Hi I'm a total noob who is trying to run a crack specified to effectively brute force 10 characters only and alphanumerical only (against a pcap)... I modified the config file; the alpha numerical bit, so that it was minimum 10, maximum 10, as this just seemed logical from reading the help file, and then john --stdout --incremental:alnum | aircrack-ng (etc etc...) but then got... "MaxLen = 10 exceeds the compile-time limit of 8 There are several good reasons why you probably don't need to raise it: - many hash types don't support passwords (or password halves) longer than 7 or 8 characters; - you probably don't have sufficient statistical information to generate a charset file for lengths beyond 8; - the limitation applies to incremental mode only." But for what I want to do, feeding, nothing to do with hashes, 10-only and alphanum only seems reasonable..? Like I say, I have read the documentation but as a novice don't wish to get too hopelessly lost so I wondered can anyone help... do I need to run some sort of patch or mod john in some way to alter the limit or perhaps better still is there another effective John command line for defining '10 only, alphanum only' that someone knows of so I can feed it through. Many many thanks for your time and patience EX </pre> Mr Ex 2010-09-01T07:28:22 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3045 <pre>John-Users, I have compiled JtR 1.7.6 with the jumbo 7 and the netscreen script v2.01 patches for: Windows Linux 32-bit Linux 64-bit These builds are now uploaded to http://openwall.info/wiki/john/custom-builds Enjoy, -Robert B. harris </pre> Robert Harris 2010-08-31T00:10:19 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3044 <pre></pre> Robert Harris 2010-08-30T22:00:03 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3041 <pre>Hello, I recently analyzed the firmware of a certain discontinued router that I have and managed to extract a cramfs filesystem. There, I found /etc/shadow with a strange hash: user:$1juTFYmn618Y:12086::99999:::: To my understanding, a DES hash should have 13 characters in base64 digits (including the salt). Any ideas why does it start with a '$'? It appears that the firmware utilized tinylogin 0.80 and uClibc 0.9.26. I had a look at the source but found no clues. Regards, JM </pre> fijam 7 2010-08-27T08:18:07 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3037 <pre>With the disscussion about "real-world" passwords going on, I thought I'd ask a question: Is there a script or method anyone knows of that we all might use to both collaborate and report on our real-world passwords? Is there something I can run that looks at the pot file, the hash file and perhaps the session log file(s), and might pull out useful information; like what rules seem to crack passwords first, and what those passwords are made of, perhaps even their CV patterns... CVVCDPS (d= digit p= punctuation s= special (like tilde)). Perhaps we could get stats on password lengths, how many were alpha only, digits only ...etc. I know someone who could write something like that in PHP, but I'm sure perl or python may be preferred. I'm just an "idea guy" and a script kiddie, so I know next to nothing on what it would take to do something like that, but I think if we could all work together and ascertain what real-world passes we see and what hash types we see them in we might improve JtR that much further! I know at my day job I see plenty of date passwords, it's when the users were forced to reset, of the helpdesk reset it for them, then they just incremented it by one or rearranged it. We dump the users histories and the patterns become clear when you look at them, you can see what their next password will be 99% certain! I also noted that our users do the l337 substitutions, but not every o=0 or every e=3, and Solars suggested rules for my question definitely helped me find passes much faster than incremental mode did/would. Even after reading and re-reading the RULES file, I have a hard time knowing how it's doing what it's doing. I know I've been hesitant to ask for assistance on the list because I was blindly running JtR and it's rules as is and not fully understanding how to take advantage of it's powerful modes and mangling abilities. Solar also scared/scares me, your program and it's abilities are awe inspiring to say the least, ahead of it's time for sure, and I think my first response or question to the list I top or bottom posted, then my email margins were over 80 characters, and another person had a "confidentiality" disclaimer... might have been another list, but somehow I felt awkward/out of place at first. Nonetheless I don't care about that now, I'm here to stay. I am going to ask my friend to help write up something in php in the hopes that it will be useful perhaps to others. Maybe Markov stat files do something like this already and it could be adapted/expanded, I've not used that mode yet, but I am going to sometime soon. Anyway thought I'd share my idea and my initial apprehension (it was probably mostly in my head and again could of been from another list altogether) about posting here. Whatever my issue was before has long since past, and I'm here to contribute to the betterment of JtR. Long live John. -rich Xinn.org </pre> Rich Rumble 2010-08-25T01:28:52 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3030 <pre> John Users, I found an error in my current netscreen.py. I mistakenly have a tab in my current version of the netscreen.py, so I need to fix that. However, I am wondering if I should update it to run in python 3.1.x. The fixed script, won't run in python 3.1.x, since python 2.x and 3.x are not fully compatible. So, I'm not sure what to do, we may need to have two different version of the script, one for python 3.x and one for python 2.x, since they are not fully compatible. So, should I keep the script in python 2.x format (but it won't run in python 3.x) , or upgrade to python 3.x format (but it won't run in python 2.x), or have two scripts, one in each format? -Robert Harris </pre> rs904c-VsqqI1RANlHk1uMJSBkQmQ< at >public.gmane.org 2010-08-24T20:48:46 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3027 <pre>Hey all, I've been playing around with the plaintext answers KoreLogic released after the "Crack Me If You Can" competition at this year's Defcon, and I got it into my head to use the list of words to try and create a complete JtR .pot file from all of the hashes. There were a couple of reason for this. First of all, I wanted to start doing comparisons of the different teams' cracking techniques; More specifically the techniques they used to train on the cracked NTLM passwords to attack the other hash types. For that I needed training and test sets. Also, I REALLY wanted to see if JtR correctly handled those #$&lt; at &gt;!# Oracle10 hashes, and if so, what the plaintexts were for them. Since I figure other people might be interested in this as well, I'm making the .pot file available at: https://sites.google.com/site/reusablesec/Home/random/KoreLogic_Defcon2010.pot As for the highlights, yes JtR does handle the Oracle passwords, though it's no wonder no one managed to crack them in the 48 hours we had. For example, here is a typical Oracle password from the set: !'O2?CHDOWN Yes, it's based off of 'TOUCHDOWN', but nobody had those rules in their mangling set... What was much more interesting though was the time it took to run the plaintext passwords through each hash type. To give you some background, the plaintext list of all the passwords contained 54,932 unique words. I ran these cracking sessions on my Mac 2.2GHz Intel Core Duo laptop, (only using one instance of JtR), with no mangling rules, (since I had the perfect dictionary). Also note, some of the salted hashes were already cracked from the competition so I did not attempt to re-crack them, (though they are all in the downloadable .pot file). I didn't time it, but if I am generous it took approximately 1 second to run the attack against all of the NTLM password hashes. Against 10157 Netscape Salted Sha (ssha) hashes, it took me 1 minute and 30 seconds, with me making 3095K c/s Against 1000 Oracle10 password hashes it took 1 minute and 28 seconds, with me making 260645 c/s Against 80 Blowfish hashes, it took me 3 hours and 36 minutes to crack them all, with me making 200 c/s Against 4077 Crypt-MD5 hashes it took 10 hours and 10 minutes to crack them all, with me making 3228 c/s What this really brings home is how important hash type is to the cracking session. There's been a lot of talk in the news lately how GPU password crackers will soon force everyone to choose 12 character passwords: http://www.bbc.co.uk/news/technology-10963967 While team HashCat showed GPU password crackers are extremely effective, (and I'm still in awe of their work), even a 10x speedup against Crypt-MD5 hashes would only allows me to make ~30k guesses a second. That's compared to the 328296K, yes that's 328 MILLION guesses a second I'm able to make against NTLM password hashes on my laptop. And that completely ignores the effect that the password salt has when auditing large lists. So once again, thanks to KoreLogic for running this competition, and I can't wait for the next one at Defcon 19! Matt Weir </pre> Charles Weir 2010-08-24T04:30:56 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3021 <pre>Hi, I've just released a jumbo patch update, revision 1.7.6-jumbo-7. The changes since -jumbo-6 are as follows (in the order interdiff reminds me of them): Added external modes DateTime, Repeats, and Subsets to the default john.conf. These have been previously posted in here: http://www.openwall.com/lists/john-users/2010/08/01/2 http://www.openwall.com/lists/john-users/2010/08/01/1 http://www.openwall.com/lists/john-users/2010/08/06/1 Applied Robert's updates to netscreen.py (john-1.7.6-jumbo-6-netscreen-script-2.diff): http://www.openwall.com/lists/john-users/2010/08/13/1 Merged in jmk's MSCHAPv2 "format" addition (john-1.7.6-jumbo-5-jmk-mschapv2.diff): http://www.openwall.com/lists/john-users/2010/07/20/1 Applied Alain's public domain statement updates (john-1.7.6-jumbo-6-alain-copyright.diff): http://www.openwall.com/lists/john-users/2010/08/11/2 Applied bartavelle's copyright statement and licensing updates from john-1.7.6-jumbo6-intrinsics-2.diff.gz as it relates to code he had contributed before: http://www.openwall.com/lists/john-users/2010/08/13/2 Corrected the worst one of the bugs in the implementation of the "--salt-list" option, documented that the implementation is broken. Now this code, when not explicitly enabled with the command-line option, should not be breaking things anymore (e.g., matching salt detection for Oracle hashes works correctly again): http://www.openwall.com/lists/john-users/2010/08/02/11 Applied jmk's "--config" option bugfix: http://www.openwall.com/lists/john-users/2010/07/28/2 I'd like to thank everyone who contributed! Alexander </pre> Solar Designer 2010-08-22T17:41:16 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3017 <pre>Is there a way to access the user name from within an external function? I have a db that stores the password as sha1(username+password) and would rather not make a world list of all user name and word list combinations. </pre> Waylon Grange 2010-08-18T23:41:05 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3013 <pre>Hi I try new way to crack fastly passes. This my wordlist : CCVC CVCV CVCC VCVC VCCV CCVV CVVC VCCC VVCC VVCV VCVV CVVV VVVC CCCV How to change all occurences "C" with consonant [bcdfghjklmnpqrstvwxz] and "V" with [aeiouy]. The wordlist is only here to show where consonant and vowels are located in the word. for example : CCVC become bric, croc, broc, choc, etc... I don't have enough knowledge with rules to do this myself. Thanks, W.A. </pre> websiteaccess-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org 2010-08-13T12:46:55 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3012 <pre>I've updated my patch on the Wiki. I hope it is working, as it has been done hastily. It is to be applied over john-1.7.6-jumbo6. It includes the following changes : * Some copyright clarifications, which could be included in the jumbo patch * SSE intrinsics code for the raw MD5 function, raw SHA1 and MD5 crypt * format updates to use these functions in raw-md5, md5, raw-sha1 * new salted-sha format, to be used with LDAP SSHA hashes. It is probably not working as expected because I just wrote the set_key function * new targets for 64 bit icc and clang It is recommanded to use this patch with icc, or clang if icc is not an option. Using it with gcc should result in decreased performance over "vanilla" jumbo. To be downloaded here : http://openwall.info/wiki/_media/john/john-1.7.6-jumbo6-intrinsics-2.diff.gz </pre> bartavelle-JGabbFpyS5Pk1uMJSBkQmQ< at >public.gmane.org 2010-08-13T09:14:22 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3011 <pre></pre> Robert Harris 2010-08-12T23:12:02 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3010 <pre>Hi, Here's our team's "Crack Me If You Can" DEFCON 2010 contest writeup. I wrote it with input from others on our team. Unfortunately, this resulted in the writeup being somewhat focused on what I did vs. what other did during the contest. This is partially compensated for by Simon (team bartavelle) and Matt having posted their own writeups, which I refer to. Warning: this writeup is lengthy! Sorry about that. ;-) --- Thanks (brief). We'd like to thank: KoreLogic, and Minga and Hank in particular - for the contest; Team bartavelle and Frank Dittrich - for their contributions; Team CrackHeads - for several things (see end of this writeup); Fyodor - for volunteering to (co-)represent us at DEFCON; Alain Espinosa - for the NTLM hashing code (in JtR jumbo patch). We would also like to thank and apologize to team smelly_weigand for failing to use their offered contribution. Please refer to the full "Thanks" section at the end of this writeup for more detail. (It was too long to start the writeup with it.) The team and contributors. Active members (those who uploaded cracked passwords, listed in order they joined the team): Solar Designer (Russia) Rich Rumble (US) Matt Weir (US) jmk (US) Dhiru Kholia (Canada) elijah (Russia) bartavelle (person) (France) websiteaccess (France) Guth (France) Other contributors (who did not participate on the team, so there was no coordination with them, yet they sent in cracked passwords to us): bartavelle (team) (France) Frank Dittrich (Germany) With few exceptions, we're unable to reliably determine the effect of individual contributions. We focused on getting a higher score as a team rather than on keeping track of everyone's individual contributions, and after about 10 hours into the contest we started reusing "team-wide" cracked passwords as wordlists and as material for .chr files and the like, and for manual analysis. So each contribution also improved other people's further contributions. With almost all cracked password uploads, except for a few early ones, there was huge overlap with passwords cracked by others on our team - typically on the order of 90% or more (that is, only around 10% or less of independently-cracked passwords tended to be new/unique rather than already cracked by others on our team). Yet we're grateful to all who have contributed! The small numbers of non-overlapping passwords simply reflect the nature of password cracking and password security, confirming that it makes sense to detect and eliminate weak passwords (then only relatively few of the remaining ones are crackable by another attacker). Computing resources. Direct use by the team (not counting contributions by team bartavelle), averages for contest duration (48 hours): approx. 30 CPU cores, approx. 1 GPU. Solar: otherwise-idle cycles (approx. 90%) of up to 12 CPU cores (over 3 quad-core CPUs in servers), roughly 8 CPU cores in use on average. (Could use several quad-core machines more, but had no time to reasonably put them to use. Spent time on launching more focused attacks instead.) Rich: 8 CPU cores (total for 3 computers), mostly in use. Matt: "2.2 GHz Intel Core 2 Duo Mac Laptop", "Asus EEE Netbook". Matt's own detailed writeup is at: http://www.openwall.com/lists/john-users/2010/08/11/1 jmk, own use ("averaging about 3 cores for 35-40 hours"): Dual Intel Xeon E5410 (2.33 GHz) quad core processors - 8 cores total jmk, contributed to bartavelle's cluster ("connected to the JtR server for about 35-40 hours"): Intel M520 (2.53 Ghz) dual core w/ HT AMD X2 5600+ dual core Dhiru: Core i5, AMD X3 720, Intel Atom 1.6 GHz (ran JtR and rcracki-mt), ATI 4870 and 5970 (ran ighashgpu), but less than one GPU used on average during contest time. "Most of the cracking was done on Intel i5 CPU, some minor work was done on AMD X3 720 and Intel Atom 1.6Ghz." elijah: AMD Athlon x2 4200 (JtR with jumbo patch, "lots of rules and a pinch of luck") bartavelle: see "bartavelle" team writeup: http://www.openwall.com/lists/john-users/2010/08/10/1 websiteaccess: Core i7 (Mac) in use during part of the contest time Guth: 3-4 CPU cores during part of the contest time Frank: a few CPU cores for quick JtR wordlist runs and a one-time contribution of the results (approx. one hour total wall clock running time, so must be no more than a few CPU-hours) Tools. John the Ripper, jumbo patch, other patches - used by all on the team. Matt's probabilistic cracker (uses JtR) - used by Matt only. rcracki-mt (on LM hashes, mostly overlapping with those cracked by JtR) - used by Dhiru (LM) and Guth (Oracle SYS and SYSTEM usernames). rcrack - used by jmk to crack the remaining two LM hashes. ighashgpu (on NTLM hashes, cracking a total of 1732 by far most of which overlapped with hashes cracked by JtR) - used by Dhiru only. Custom code written or modified during the contest. Custom Perl scripts, such as revisions of mix.pl to generate double-word lists: http://www.openwall.com/lists/john-users/2010/02/14/5 JtR wordlist rules and external modes. Custom shell scripts to automate merging of uploaded files, to generate cracked/uncracked password/hash lists and .chr files for the team to possibly reuse, and to make contest submissions (on cron). Wordlists. We used the following wordlists: JtR password.lst. Previously-cracked passwords (both from contest hashes and others). Manually-created contest-specific tiny wordlists based on analysis of cracked contest passwords. RockYou list (and "Top N" sub-lists from it to use with lots of rules). Combinations of the above (e.g., contest-specific words concatenated with RockYou Top 1000). JtR Pro revision of all.lst (in addition to words in many languages, includes a little bit of incremental mode and --external=Keyboard output, which thus got subjected to wordlist rules). Openwall collection revision of all.lst, insidepro_big (used by Dhiru). "Various InsidePro "From Queue" dictionaries" (used by Matt) wikipedia-wordlist-sraveau-20090325 (late in the contest and against NTLM hashes only, by Solar and Matt). A large second-level domain name list, with rules adding com/net/org TLDs after that pattern was identified in contest passwords. (Other TLDs were also probed, including all two-letter ones - no luck - so the focus was made on just com/net/org, which proved to be correct.) Perhaps many others. Team building and management. We (team john-users) did not prepare for the contest at all. Solar decided to participate and invited others to join roughly 12 hours before contest start. Thus, people were joining during the first day of the contest (when some of us were already cracking the hashes). Solar was completing setup of the file exchange server, creating accounts, adding SSH keys, troubleshooting some team members' login issues, etc. during the contest. We used an OpenVZ container with and on an Owl system (Openwall's Linux distro), with per-person accounts and some shared directories, with file permissions set/adjusted during the contest as needed - and eventually with some scripts on cron. Solar also setup a private mailing list for discussions internal to the team; by the end of the contest, we had 11 subscribers (most of them active team members, some not - we gave everyone a chance). The coordination was very loose, largely because of the lack of advance planning, because Solar was also one of the primary people to actually attack the hashes (leaving a lot less time for coordination), because it is hard and often unreasonable to avoid overlap (avoiding it costs people's time, so we could as well throw more CPUs at the task instead), and frankly because many of us appeared to be unwilling to coordinate (most went with whatever they liked to do, which is quite natural, yet it resulted in higher overlap in attacks attempted and hashes cracked). How the different hash types were approached. One of the first tasks was to determine what hash types we were given, although this proceeded in parallel with some non-focused initial JtR runs on whatever hash types were already identified. Soon we reliably identified all hash types but the Oracle hashes, which we were not 100% sure of. As suggested by some team members, the hashes list that was provided to us was eventually split into separate files by hash type, and then we also created separate files with admin accounts only (which we assumed were those with a GID of 0 or/and with the "admin" substring in the username). This was not essential for use with JtR (which is why it was not done right away), yet apparently it was convenient to some of us. Closer to the end of the first day, we also had a cron job producing per-hash-type lists of uncracked hashes - for faster further attacks on salted hashes. For all hash types, we ran JtR with default settings against them at least briefly to catch the weakest passwords first. The per hash type sections below mostly describe other attacks. NTLM hashes. These are what the contest was about. Although it was apparent from the number of these hashes and the speed at which they could be computed, we only truly focused on them closer to the end of the first day (and not everyone on our team did). We were concerned that we could be behind other teams due to them getting more hashes of other types if we focused almost solely on NTLM. Clearly, for a chance to win, we did need to attack all other saltless hashes as well (LM, SHA) and attack the salted ones at least lightly (although we ended up doing a lot more than that - maybe too much given that NTLMs could use more of our attention). Also, most admin accounts, which give extra points, corresponded to hashes of other types. Anyhow, early attacks against NTLM hashes, besides JtR's default settings, included runs of the RockYou list (with duplicates purged) with small rulesets, and runs of smaller wordlists (password.lst, RockYou Top N) with larger rulesets. Specifically, the "single crack" mode ruleset was used with wordlist mode, and its "crazy" section was uncommented and even expanded with 4-digit append/prepend rules. A further attack was to run rockyou.chr released by Kore against NTLM hashes on a Q6600 machine that Solar temporarily dedicated for the purpose. The workload was distributed across 4 CPU cores by password length: 0-5, 6, 7, and 8. The 0-5 run completed shortly, and was replaced by other quick attacks (sequentially, sometimes leaving this core mostly-idle for quite a while). These included --external=Keyboard for lengths 1 to 10 (and a bit of 11), which only cracked a password of length 7 and lots of length 8 (Kore, this lengths distribution is not realistic). It also included four modified-KnownForce runs to exhaustively search the abcd19nn, abcd20nn, 19nnabcd, and 20nnabcd patterns (identified from previously cracked passwords), where abcd was an arbitrary string of four lowercase letters ("aaaa" through "zzzz") and nn was an arbitrary two-digit string ("00" through "99"). Closer to the second day of the contest, a list of contest-specific words was compiled. Although early revisions of the list varied a bit, ultimately (for our team) it was: korelogic defcon Defcon blackhat facebook lasvegas LasVegas vegas whitehat hello 1234 jan feb mar apr may jun jul aug sep oct nov dec january february march april may june july august september october november december janu febr marc apri augu sept octo nove dece monday tuesday wednesday thursday friday saturday sunday one two three four five six seven eight nine ten eleven twelve twenty thirty fourty fifty sixty seventy eighty ninety hundred thousand million billion winter spring summer autumn wintertime springtime summertime autumntime (although some people on the team had their own lists). Yes, "fourty" was misspelled, and this was not noticed, unfortunately (should have run a spellchecker over the wordlist). Also, this did not include "fall" as it did not appear to be common enough, but looking at Kore's published rulesets it is there along with the common words above. This list was run (on a free core of yet another quad-core machine) against NTLM hashes with very large numbers of rules (a few million after preprocessor expansion). Specifically, a very effective ruleset line was: o[0-9][ -~] o[0-9][ -~] (overstrike any one or two characters with any other printable ASCII characters). Due to the speed at which NTLM hashes could be computed, we did not really have to identify specific substitutions (or at least not yet), as long as no more than two characters in a (pass)word were substituted at once. Ditto for inserts of up to two characters. Going to three would be a bit too slow (would take 1000x longer); instead, we caught some of those passwords by applying the same approach to previously-cracked passwords (and eventually by identifying and encoding some specific substitutions, although we did not do much in this area). We also combined this with case toggling. This was done by outputting the tiny wordlist mangled with one set of rules (such as the above) to "unique" and saving to a file, then applying a case-toggling ruleset (the default "NT" ruleset or the like) to the resulting file. Besides using the tiny contest-focused wordlist on its own, we were combining it with itself (to form strings such as "sixtysix") and with other common password lists using a variation of mix.pl (mentioned above). And we also ran the above rules (and the dual-application of rules approach) against common password lists on their own. We also used "regular" rulesets (like the "single crack" mode one) against these combined wordlists. All of this was very effective and quite quick. Another very effective approach was to use all substrings of cracked passwords (rather than cracked passwords in their entirety only) for "wordlists". Substrings were being extracted with: [List.Rules:substr] x[0-9][4-9] invoked with "... --stdout | ./unique cracked-substr". Then cracked-substr was subjected to the kinds of processing described above (mix.pl and/or dual-applied rulesets). Although the above text uses "we", these things were pretty much done by Solar alone (others were invited to apply similar processes and extend or revise them to reduce overlap, but did not report on having done so). Overall, this cracked thousands of NTLM hashes, but it is impossible to tell exactly how many because of the reuse of previously-cracked passwords for wordlists (including those cracked by others on the team who were using other methods). At about the same time, Dhiru was running some NTLM hashes through ighashgpu, eventually uploading 1732 of cracked passwords (a small number compared to what the approaches described above achieved), most of which overlapped with those cracked with JtR (but there were some unique/new ones as well - mostly those containing more than two uncommon characters). On the second day of the contest, Solar made a build of JtR capable of applying incremental mode to lengths up to 10: http://www.openwall.com/lists/john-users/2007/07/04/6 The corresponding .chr file was generated based on contest passwords cracked by that time. It was then run against NTLM hashes separately for length 9 and length 10 on two CPU cores (on a third quad-core machine that was finally put to use). It quickly cracked some new passwords, but then its progress slowed down a lot (as expected). Overall, by contest end this cracked only about 150 of new passwords. Finally, closer to the end of the contest large wordlists such as wikipedia-wordlist-sraveau-20090325 were run against NTLM hashes with various rulesets created by that point. Some of those runs completed quickly (resulting in hundreds of new cracks), whereas a couple of others were still running by contest end time and slowly cracking more passwords (on one of Solar's machines). The limiting factor appeared to be the time of people on our team. With more effort (not requiring any more computing power), we could have reverse-engineered more of Kore's rules, patterns, and wordlists, which would enable cracking more of the passwords before contest end. Without those sufficiently specific rules and patterns, we had to be using more generic but less efficient rules and wordlists, and we did not run specific attacks on certain patterns that were seen. For example, we never derived the exact list of Kore's character substitutions, although we had the material to do so. We simply dropped not-so-common "Kore words" such as "stdio" or sports teams because no one on our team would compile a complete list of them anyway (so we were combining very common "Kore words" listed above with pre-existing common (pass)word lists instead). This was largely compensated for by the "all substrings" approach, though, which obviously caught all common and not-so-common words seen in cracked passwords (but it also caught some "noise"). One approach we considered on the second day of the contest was auto-generating rulesets based on cracked passwords. JimF had posted something along these lines to the john-users list in 2009: http://www.openwall.com/lists/john-users/2009/07/27/3 There was a brief attempt (by Solar) to get someone interested in trying this out - but almost no one was interested/available (it was just 12 hours to go). One person volunteered, but then never reported anything back. It would still be curious to explore this area on contest hashes even after the contest has ended. LM hashes. For LM hashes, Solar initially ran a build of JtR with faster DES key setup (john-1.7.6-fast-des-key-setup-3.diff.gz) on a single core of a Core i7 CPU. This completed lengths 0 through 6 promptly and it also cracked many length 7 password "halves". The intent was to go for an exhaustive search over the printable ASCII space for length 7 by dedicating the Core i7 machine to it the next day (we had several extra quad-core machines available for use anyway). This should have completed in time. However, Dhiru was quicker to go with rainbow tables, and then jmk cracked the two remaining hashes with a different set of rainbow tables. So the "JtR plan" against LM hashes was canceled, thereby saving Solar some time on (not) setting that attack up. Although we cracked all LM hashes, there was an issue with submitting the corresponding passwords properly (see below). Netscape LDAP SHA hashes (saltless). These were similar to NTLM in terms of attacks to run against them, however their number was substantially smaller, so we did not focus on them - essentially only running simple attacks and variations of previously-cracked passwords (for all hash types) against them. As an exception, bartavelle actually spent some time on them, re-encoding them from base64 to hex such that he could use his unreleased faster raw SHA-1 code instead of the Netscape LDAP specific code. Then he had to have them re-encoded back to base64 for submission, because our submission script would filter out non-contest hashes, which was in turn caused by one of the team members not cleaning up their pot file for the contest. ;-) Netscape LDAP SSHA hashes (salted). These were almost exclusively left for team members other than Solar to attack, as a way to reduce overlap. Probably the usual sets of attacks were run against them (this was not documented by team members). Only very brief attacks were run by Solar (JtR defaults for a little while, trivial variations of cracked contest passwords from all hash types, and not so trivial variations against admin accounts with this hash type). Clearly, we could have done better, although this would not provide a lot of additional points (not enough to make a difference overall). MD5-based crypt hashes. For the MD5-based crypt hashes, Solar initially ran JtR with default settings (just to make use of a CPU core while focusing on other things), which was then permitted to run for a long while. The plan was to make a build of JtR with bartavelle's patch for much faster SSE2-enabled support for these hashes and use that. However, when bartavelle himself joined our team closer to the end of the first day, this plan was canceled, and bartavelle was the person to focus on these hashes. Ditto for Markov mode runs. DES-based crypt hashes. For the DES-based crypt hashes, some attacks were split over 2 CPU cores with the "--salts=3" and "--salts=-3" options. That is, salts shared by more hashes were attacked quicker or harder than those with fewer hashes. This was done at least for some incremental mode runs and then for an exhaustive search over abcdYYYY and YYYYabcd patterns, where YYYY was a year number in the range 1959 to 2019 and abcd was an arbitrary string of four lowercase letters ("aaaa" through "zzzz"). The years range was previously determined from incremental mode runs against NTLM hashes and then from the modified-KnownForce run over a much wider years range against the NTLM hashes, which was described above. Somehow this years range is slightly inconsistent with rules published by Kore; we did not investigate whether it was due to an error on our part or due to Kore using only a subset of the hashes that their rulesets could generate. Anyhow, for NTLM hashes restricting the years range did not matter, but for DES-based crypt ones it did (we'd need to dedicate more than two CPU cores to the task to complete the exhaustive search in time if the range were wider). The above was done by Solar, informing others on the team in case anyone would want to do it for other hash types (besides NTLM and DES crypt), but apparently that was never done (which was OK - we did not have all that many other hash types suitable for this attack given the resources and time available... although SHA and SSHA were suitable). elijah also did incremental mode runs against these hashes using contest-specific .chr files (based on previously cracked passwords), and he tried some custom substitution rules, as well as this ruleset previously posted by Minga: http://www.openwall.com/lists/john-users/2009/03/28/2 Given its origin, we probably should have played with it more (e.g., updated for year 2010 and ran against other hash types as well), but we did not. On the other hand, other attacks we performed against NTLM hashes should have covered these patterns. Some others tried cracking the DES-based crypt hashes by various means as well, but overall we did not focus on them as much as we did on NTLM. Blowfish-based crypt hashes. Solar (and likely others) initially ran JtR with default settings against all 80 hashes on a single CPU core (which would be otherwise idle anyway). After a couple of hours (mostly spent on tasks unrelated to this specific hash type), it was determined that none of the 80 had very weak passwords, and considering the contest scoring and the slowness of these hashes, it made sense to continue attacking only the 20 admin hashes (out of 80). So that's what was done (by interrupting, editing the .rec file to increment the options count and add "-g=0", and continuing the session). Others on the team were informed of this decision. Solar's attack on the 20 admin Blowfish hashes remained running for many hours more, eventually being replaced with NTLM attacks when it became convenient to run more of those on that machine. Perhaps others on the team attempted certain attacks as well. None of these hashes were cracked. Running an OpenMP-enabled build against these hashes (the 20 admin ones only) on a dedicated quad-core machine (and we had some spare ones) was briefly considered (by Solar), but was not attempted - running more focused attacks against NTLM hashes appeared to be a better use of time. Looking at the plaintext passwords published by Kore after contest end, some of these could probably be cracked by running heavier rules on cracked passwords from other hash types - but even that kind of attack would be very time-consuming (albeit feasible) and hardly worth it against hashes of this type (it would make more sense to focus on uncracked admin hashes of other types), unless the scoring were different. Oracle 10 hashes. For a lot of detail on these "mystery" hashes, why no team cracked a single one of them, what we tried against them, and the effect this had on our overall performance, see: http://www.openwall.com/lists/john-users/2010/08/02/11 Issues with submission of cracked passwords. No one on our team was verifying whether the cracked passwords we were submitting to Kore were actually correct. Additionally, cracked hashes were being excluded from the "uncracked" lists (that some of us used for further attacks) without verification that the cracked passwords were actually correct. Lacking this kind of verification was obviously wrong, but we did not appear to have the human resources to set it up (it'd be a distraction from cracking more hashes). Perhaps we should have created this sort of scripts prior to contest start, although for that we should have made the determination to participate in the contest much sooner. Anyhow, we were looking at the contest stats web page, and our score displayed there was only a little bit lower than our own estimate for what it should have been. If it were substantially lower, we would indeed be forced to verify our stuff. As it turned out (was noticed by us during the contest), Kore's passwords contained an abnormally high number of colons. Of the passwords we cracked, about 1000 contained colons. For submissions in john.pot format, this would make no difference (although it could have affected scripts we'd use during password cracking). However, when Kore clarified that they wanted only the plaintexts, and complete ones for LM hashes, Solar quickly hacked together a script that used "cut -d: -f2" on a john.pot format file for non-LM hashes and "john --show ... | cut -d: -f2" for LM hashes, planning to get back to correcting this at a later time. Unfortunately, this was only recalled 1 hour before contest end, at which time Solar, needing to focus on lots of contest-related tasks at once, only had sufficient time to make the trivial change of "-f2" (field two only) to "-f2-" (fields starting with the second) for non-LM hashes. The same trivial change would not work right for LM hashes due to "john --show" output containing more than two colon-delimited fields. A slightly more complicated fix for LM hashes was introduced and tested only 10 minutes _after_ contest end (so if Kore ever publishes scores for post-content submissions, this should be seen). This should have affected hundreds of LM hash passwords. Another issue was with DES-based crypt hashes, which process only 7 bits of each character (ignoring the 8th bit). This means that for a given valid passwords, many variations of it are possible (with the 8th bit of every character possibly flipped), most of which will not match those on Kore's list of correct passwords, yet all of them are correct. We ended up submitting some of these passwords with the 8th bit on some characters set (just because this is what was tested first in certain attacks run by some members of our team). In many cases (but not in all), we also had the pure 7-bit versions of the same passwords cracked, and these were also being submitted (due to the non-use of "john --show" for this hash type, all variations that we had cracked were being submitted). This was noticed during the contest, and ideally we'd convert all of these passwords to pure 7-bit, but we didn't have time and arguably this was not worth the bother (it affected maybe around 100 passwords). We do not know whether Kore's scripts were smart enough to count these valid passwords with 8-bit characters towards our team's score or not. Finally, it appears that some yet unidentified software in use by two members of the team did not handle backslashes in passwords correctly. Some of their uploads contained missing, double, or quadruple backslashes in place of single backslashes in passwords. Luckily, most of those passwords were also cracked by others of us, and all variations were being submitted for all hash types but LM. For LM hashes, this might have affected our score somewhat. Overall, we deserve some penalty on our contest score (which we must in fact have incurred) for not being very careful with our submissions. Making use of otherwise-idle CPU cycles only. For Solar's JtR runs during the contest, this was achieved in two ways: For machines running OpenVZ kernels (two of the three quad-core machines that were put to use), a dedicated OpenVZ container with an x86-64 build of Owl was created (from a recent pre-created template released by Openwall). The container was set to have a relatively low number of cpuunits - specifically, it was set to 100 cpuunits, whereas all others on the system were set to at least 1000 cpuunits each. Then JtR's "Idle" setting was disabled (set to "N"), because OpenVZ uses a two-level CPU scheduler anyway and we only needed to be "nice" to other containers running on the system. Some quick tests proved that this worked as expected. For a machine running a "regular" (non-OpenVZ) Linux kernel, the "Idle" setting was made use of (kept at its current default of "Y"), which had been tested to work well before (during John development). Summary. The contest was fun indeed, but besides being fun it also required a lot of concentration over the 48-hour period (and for a bit longer than that since there were some preparations to make shortly before contest start). Although we did not incur a direct monetary cost, the cost in people's time was substantial. The passwords were not real, and the distribution of different kinds of passwords was somewhat non-realistic... but so what. This meant that part of the challenge was for us (and for other teams) to quickly adapt to attacking these somewhat unusual passwords. This also meant that certain techniques that didn't work very well in this contest would have in fact worked much better on real-world passwords, and vice versa. For example, a few of the passwords hashed with Blowfish-based crypt could be a lot weaker (and would actually get cracked): there exist many systems that use hashes of this type without any password policy enforcement. Most of the Oracle passwords would be weak and would get cracked, and the corresponding usernames would be known reliably. There would be lot more of digits-only passwords. The keyboard-based patterns would not be magically restricted to length 8. Passwords would sometimes be username-based. On the other hand, extensive case toggling would be a bit less effective. On the bright side, many of us learned new things, and we've identified shortcomings of our approaches and software that are also relevant for real-world password security audits. Certain improvements to John the Ripper will likely be made as a result of this contest. The files released by KoreLogic will play an important role in testing and tuning of current and future password security software and techniques. It is now possible to derive lists of cracked and uncracked passwords. These passwords, through their hashes, have been tested by many people from many teams using significant cumulative computing resources, as well as many different tools, techniques, and wordlists. This makes them very valuable. Thanks. Minga and Hank of KoreLogic did a great job at making this contest possible - thank you! We'd like to thank team bartavelle and Frank Dittrich for their contributions to our team's cracked passwords pool. We're also grateful to team smelly_weigand for offering their cracked passwords to us, and we're sorry that we never merged those due to a coordination error on our part. Matt and Fyodor volunteered to represent our team at DEFCON, making our participation official - thank you from the rest of us! We would also like to thank all other teams that participated and made this contest a real challenge for every team involved. Our special thanks are to team CrackHeads who, while remaining completely separate from us during the contest, have also made use primarily of John the Ripper and provided useful feedback in their writeup: http://www.openwall.com/lists/john-users/2010/08/03/1 Additionally, we'd like to thank KoreLogic for funding the contest, including the cash prizes (and our "3rd eligible place" $100 prize specifically), and CrackHeads for kindly donating $100 out of their $300 cash prize towards JtR development (with the remaining $200 covering their Amazon EC2 costs). Finally, we'd like to thank Alain Espinosa for contributing his efficient NTLM hashing code (currently in JtR jumbo patch), which was instrumental to efficient use of John the Ripper during the contest. With approval from CrackHeads, we're going to direct the $200 ($100 from team john-users and $100 from CrackHeads) to Alain as a way to thank him in a more tangible way. His contribution is clearly worth it (and more), and not only for the contest! --- Alexander </pre> Solar Designer 2010-08-12T19:50:48 http://comments.gmane.org/gmane.comp.security.openwall.john.user/3001 <pre>I wanted to ask if others had experimented with consonant vowel patterns in password cracking? Perhaps others know this approach by a different name? I believe the proper term is phonology (I may be wrong on that). Here is an example pattern: CVCCVC That pattern is common and found in many words. Here are some words based on it: batman badguy bigboy (catfig)ht (barfig)ht (Falcon)s bullet defcon ... That is just one common pattern, there are many others. One thing I like about these patterns is that they are cheap to compute and once computed, you have all passwords that fit that pattern. The C list is only consonants and the V list is only vowels. N is only numbers while S is only special chars. C = 30s-40s (drop chars that are seldom used z,x,j if you like) V = aeiouAEIOU N = 1234567890 S = 30s (less if you only want the commonly used chars) So any variation of batmanNN would be cracked by CVCCVCNN and it only takes 40 * 10 * 40 * 40 * 10 * 40 * 10 * 10 = 25,600,000,000: BATMAN78 Batman01 bAtMaN00 ... This is the approach I took with 16Crack in the KoreLogic contest. And again, I lost by *a large margin* as I was only averaging about 80 cracks per hour and in order to be competitive I would need to average about 800 cracks per hour. Nonetheless, it was interesting to me to see how this approach would fair against the more conventional approaches. There may be a mode somewhere in JTR that does this or something similar. I only wanted to share it with the list just in case others find it useful for some sort of attack. Obviously, it does not stand on its own and needs other approaches combined with it to be competitive and may be a bad idea altogether compared to traditional approaches. Any advice on the approach or thoughts of its general worthiness is appreciated! Brad </pre> Brad Tilley 2010-08-12T13:17:40 http://comments.gmane.org/gmane.comp.security.openwall.john.user/2994 <pre>I was provided a WEP Key and asked to see if I can crack it. From what I have been reading, it seems the only way to crack a WEP key is to have traffic from that keys network to capture with a util that is trying to crack the key. Once the util has seen enough data, it is then able to crack the wep key. Am I wrong? Is it possible to just crack a WEP key with nothing but the key, and something like JTR, and no supporting traffic, or do I need to run a tool like aircrack that can sniff the networks traffic for a while in order to perform the crack? </pre> Meyer, Bruce 2010-08-11T17:54:06 http://comments.gmane.org/gmane.comp.security.openwall.john.user/2993 <pre>Hi. Attached a diff applied on top of the last jumbo patch modifying the copyright of NTLM and MSCASH formats following the new guidelines. Best Regards, alain </pre> Alain Espinosa 2010-08-11T14:14:49 http://comments.gmane.org/gmane.comp.security.openwall.john.user/2992 <pre>I had the pleasure recently in competing alongside the john_user's group in KoreLogic's Crack Me If You Can Challenge. I'd like to start off by first thanking KoreLogic who not only devoted their time to set up the contest, but also staffed the contest booth throughout the challenge, (and put up with our crazy submissions). I'd also like to thank Solar for organizing all of us, and from what I can tell from the timestamps on the e-mails he sent out, devoting a sleepless weekend to this contest. Overview: Since I was attending Defcon at the time, I along with Fyord represented the john_users group. The downside of this was that it was very hard for me to keep up with what the rest of the team was doing since I was hesitant to check my webmail on the secure wireless setup, and my AT&amp;T data coverage was spotty at best, (since several thousand other people were checking their I-Phones at the same time). That being said, I had a great time hanging around the Kore booth and finding out what techniques other people were using. I really valued everyones' inputs since, as was pointed out several times to me, (rightfully so), all of my past work on password cracking has been conducted on web-passwords. Getting a chance to talk to people who have audited corporate passwords was an amazing opportunity for me. Hardware: Primary Computer: 2.2 GHz Intel Core 2 Duo Mac Laptop with 1 GB of Ram. This is the computer I did most of my cracking on, and I left it running in my hotel room throughout most of the contest. Unfortunately I was not staying at the Riviera so with a few exceptions I was only able to check it when I came back at night and in the morning before I left. This meant I was limited in my ability to tweak my attacks to to take into account the patterns that appeared in the cracked passwords. Secondary Computer: Asus EEE Netbook, 1 GB of Ram. This was the computer I carried around with me at the conference. I ran short cracking sessions on it when I had some free time, but power (electrical) was an issue. I also frequently stopped my cracking session when I needed to do something else, (like demo tools with other people, or try to help some friends out in the OpenCTF competition). BTW, if you are bored at Defcon, you are doing something wrong ;) Password Cracking Overview: I managed to download the password hashes sometime around 1:00am Thursday night. I started by focusing on standard pw cracking attacks using common dictionaries, such as password.lst. I also mostly focused on the NTLM password hashes. In short, I was pretty much duplicating everyone else's work... Sometime around 2:30am Friday according to my notes I switched to the Crypt-MD5 hashes and ran JtR's password.lst using the single mode ruleset, (the reason for this is I desperately wanted to get some sleep and figured it was a worthwhile attack to run through the night) . I stopped that attack at 9:20am after completing only 1.15% of the cracking session. At that point I had cracked 603 of 4716 (12.78%) of the Crypt-MD5 hashes. On a side note, I was only making around 6000 guesses a second against the Crypt-MD5 hashes. I'm not sure if that was 6000 g/s against all of the hashes, or if I was only making about unique 2 g/s. Needless to say, I wasn't making much progress... Before I left to attend the conference Friday morning, I launched another attack using the single mode ruleset and the various InsidePro "From Queue" dictionaries. I also created a wordlist containing all of the extracted lower-alpha characters from the passwords the john_user's team had cracked, and uploaded it to the team's server. Aka I created an input dictionary by extracting the words, (in a fairly naive manner), from the cracked passwords.I also created an input dictionary of all of the usernames, but this turned out to be not very useful. The Kore folks admitted later that they didn't think about matching passwords up with usernames until after they had created all of the hashes. I arrived back at my room that evening around 7:00PM ish, and while getting ready to head out to the EFF charity benefit party, (did I mention there's a few things to do at Defcon), I finally got around to using my probabilistic password cracker against the NTLM password hashes. For the training list, I used the MySpace passwords, and for input dictionaries I selected dic-0294, and a list of 500 common passwords. I let this attack run for the next couple of hours while I attended a couple of the nightly Defcon events. After I got back around 1:30am Saturday morning I decided to retrain my probabilistic cracker on the passwords our team had already cracked. Normally I don't like doing this, (training on a set I'm attempting to crack), since it's like drinking your own bathwater. The cracker gets better at cracking passwords you've already cracked, but worse at cracking passwords you haven't seen. That being said, I almost immediately started cracking a lot more NTLM password hashes. It was actually pretty cool to witness. At around 3:00 am I received an invitation to join the HashCat group on their IRC server and spent a while talking to them. They were very classy, and even offered me some of their rulesets. I declined to use them though during the contest since I didn't want to take unfair advantage of their generosity... I also didn't have a Windows computer to run Hashcat off of, but I prefer the first explanation ;) I got off to a slow start Saturday morning, but I ended up running my probabilistic cracker against NTLM passwords using the re-trained ruleset, and using the input dictionary I had previously created from cracked passwords. Throughout the day I also ran attacks on my EEE PC using custom JtR rules such as adding 2010 inside words, (aka pa2010ssword). I created these rules based on patterns I saw in the cracked passwords. It was also at this point when I realized I should have used SRaveau's Wikipedia dictionary sooner. It cracked quite a few NTLM passwords I hadn't managed to crack before. I should also mention that I ran shorter cracking sessions against some of the other password hashes, such as DES, using my EEE PC. Saturday afternoon I went back to my hotel room and switched to Crypt-Sha passwords, since I realized I was pretty much duplicating the work everyone else had done with the NTLM password hashes.Once again, I was using my probabilistic cracker. I returned shortly before 11:00 pm but I pretty much just let my attacks keep running. I was still cracking around 1 Crypt-Sha hash a minute, and I was uploading my results until the the contest ended. And that was that. Lessons Learned and Mistakes Made: 1) I believe the key to this contest was analyzing the cracked passwords and creating custom rulesets based on them. The HashCat and CrackHeads groups did this amazingly well, (I have no idea what InsidePro's strategy was). 2) While JtR has the most powerful built-in rule creation system, based on the listserv posts, it looks like our team had a hard time taking advantage of it. 3) From my conversations with other people at the conference, JtR's config files/rules are pretty much universally hated/feared/misunderstood. It's almost comical since many of the people I talked to are fluent in several programming/scripting languages. 4) I discovered numerous improvements I need to make to my probabilistic cracker, (which is one of the many reasons why I competed in this challenge). Right now my probabilistic cracker is designed to be trained on a list of plaintext passwords that resemble the target. This limits its effectiveness when retraining it on a set of passwords it currently is attacking. I need to modify the training program to include an option where it will be more effective when trained on a partially cracked set of passwords. One example of this is to ignore password length, (aka we were much more likely to crack shorter passwords earlier on which meant the probabilistic cracker focused on shorter passwords when it was retrained). 5) There's a lot of work left to be done weaponizing my probabilistic cracker, such as reducing the memory usage, and adding support for insertion rules, (aka pa2010ssword). 6) I'm really envious of team CrackHeads's use of Amazon EC2 clusters. I wish I had thought of that! 7) Rather than starting my probabilistic cracker from the most probable password first, I should have started from a less probable starting point to avoid duplicating everyone else's work. That's one serous downside with using it since it doesn't play well with other types of password cracking attacks, (aka avoid duplicating work). I'm almost certainly going to expand on this writeup later, but I wanted to get something out in a somewhat reasonable time. Thanks once again to everyone, and hopefully we'll get a chance to do better next year! Matt Weir http://reusablesec.blogspot.com </pre> Charles Weir 2010-08-11T11:32:42 http://comments.gmane.org/gmane.comp.security.openwall.john.user/2991 <pre>Here is the writeup for team "bartavelle", which contributed its results to the john-users team. [MEMBERS] The "active" members were me and a couple friends. Another one handled the LM hashes part. Quite a few people helped us by running our software on computers they owned. Thanks to them ! [PRE-CONTEST] Me and one of the team members have been working for a few weeks on a automated password cracking system. This system would also provide statistics, graphics that could be copied and pasted into reports, automatic analysis and stuff like that. It could also be used to perform other computing tasks. While the "interface" part of the system is quite mature by now, the actual cracking part was not truly tested. Early tests, a week before the contest, highlighted design errors that absolutely needed to be addressed for it to scale for this contest. We spent the days before the contest rewriting part of the system, and doing not much else. Some early advertising was also done so that people would join our "cluster" system, but only one answered. [CONTEST TIME] When the contest started, we only had a limited set of resources available for various reasons. The first hours were spent recruiting. After that, I was moving out 700km away and was just packing my stuff, leaving only two members on the team to accomplish things. The computing resources were targeted at all hashes, with about 3 hours spend for each. After that was done, it was decided that we would go for the MD5 crypt hashes, as my implementation should be faster than the other contributors. I spent most of my spare time fixing the software and creating the infrastructure (such as the automatic submission of the passwords) required for the contest. We then experienced a huge client crash (not yet resolved). I had the brilliant idea to reset the environment by trashing the database, without dumping it first. I lost most of my stats during that process. The last 20 hours were spent by the cluster crunching the md5 admin hashes, with no success. At the same time, the LM job finished, and the two team members left were busy trying to figure out the patterns. [RESOURCES] John the Ripper was used exclusively, except for the LM hashes cracking. For the CPU throughput during the last 20 hours (multiply by 18 for the actual hashing operations): http://bigbox.banquise.net/jtr/cpus.txt http://bigbox.banquise.net/jtr/tested.png (The last graph shows cracking done after the end of the contest). We had more resources available before the crash : +------+-------------------------------------------------+ | core | name | +------+-------------------------------------------------+ | 8 | AMD Athlon(tm) 64 X2 Dual Core Processor 5600+ | | 4 | AMD Athlon(tm) II X2 B22 Processor | | 2 | Genuine Intel(R) CPU T2500 &lt; at &gt; 2.00GHz | | 4 | Intel(R) Atom(TM) CPU 330 &lt; at &gt; 1.60GHz | | 2 | Intel(R) Celeron(R) CPU 220 &lt; at &gt; 1.20GHz | | 2 | Intel(R) Celeron(R) CPU E1400 &lt; at &gt; 2.00GHz | | 1 | Intel(R) Celeron(R) CPU 2.60GHz | | 1 | Intel(R) Celeron(R) CPU 2.66GHz | | 4 | Intel(R) Core(TM) i5 CPU M 540 &lt; at &gt; 2.53GHz | | 8 | Intel(R) Core(TM) i7 CPU 860 &lt; at &gt; 2.80GHz | | 8 | Intel(R) Core(TM) i7 CPU 950 &lt; at &gt; 3.07GHz | | 2 | Intel(R) Core(TM)2 CPU T7400 &lt; at &gt; 2.16GHz | | 2 | Intel(R) Core(TM)2 Duo CPU E6750 &lt; at &gt; 2.66GHz | | 2 | Intel(R) Core(TM)2 Duo CPU E8400 &lt; at &gt; 3.00GHz | | 2 | Intel(R) Core(TM)2 Duo CPU E8500 &lt; at &gt; 3.16GHz | | 4 | Intel(R) Core(TM)2 Quad CPU Q9550 &lt; at &gt; 2.83GHz | | 4 | Intel(R) Core(TM)2 Quad CPU Q6600 &lt; at &gt; 2.40GHz | | 4 | Intel(R) Core(TM)2 Quad CPU Q8300 &lt; at &gt; 2.50GHz | | 4 | Intel(R) Core(TM)2 Quad CPU Q9400 &lt; at &gt; 2.66GHz | | 4 | Intel(R) Pentium(R) 4 CPU 2.80GHz | | 2 | Intel(R) Xeon(R) CPU 5160 &lt; at &gt; 3.00GHz | | 16 | Intel(R) Xeon(R) CPU E5405 &lt; at &gt; 2.00GHz | | 4 | Intel(R) Xeon(R) CPU E5420 &lt; at &gt; 2.50GHz | | 8 | Intel(R) Xeon(R) CPU E5504 &lt; at &gt; 2.00GHz | | 24 | Intel(R) Xeon(R) CPU E5506 &lt; at &gt; 2.13GHz | | 2 | Intel(R) Xeon(R) CPU E5540 &lt; at &gt; 2.53GHz | | 32 | Intel(R) Xeon(R) CPU E7440 &lt; at &gt; 2.40GHz | | 8 | Intel(R) Xeon(R) CPU X5472 &lt; at &gt; 3.00GHz | | 24 | Intel(R) Xeon(TM) CPU 3.60GHz | | 2 | Pentium(R) Dual-Core CPU E6500 &lt; at &gt; 2.93GHz | | 8 | Quad-Core Intel Xeon | | 1 | VIA Esther processor 2000MHz | | 1 | VIA Nano processor U2250 (1.6GHz Capable) | +------+-------------------------------------------------+ [WHAT WENT WRONG] First of all, the preparation was not sufficient. The software should have been finished beforehand (wordlist support was not in yet), debugged (there was this huge crash in the middle of the contest where we lost many clients, with minor issues). One of the biggest problem was the work distribution. For the sake of simplicity, a design decision was that cracking would be distributed using the markov system in jtR. The work space would then be cut in equal slices and these slices would be distributed to the clients. The problem was that some clients were *really* slower than the fastest, which was used as a basis for the slice size determination. It turned out that what would last 10 minutes on the fastest core could last 2 hours on another. Some slices were never completed in time by the really slow ones. Too much time was spent cracking the MD5-crypt whereas with a lot of computer it could have been possible to crack more MD4s or SHAs and gain a better insight on the patterns used. The MD5 job that was last run only completed 20% of its assigned search space by the end of the contest. The parameters were really badly selected. This large computing force accounted for only 30% of the score of this team. Finally, this platform was designed to crack "normal" (weak) passwords and to require minimal interaction. It was not suited at all for this contest. [WHAT WENT RIGHT] The remaining 70% of the score was gained by two members of the team, using JtR and some ad-hoc script that generated candidate passwords. The server didn't scale too horribly, even across slow connexions, and I managed to only break a couple glasses during my 700km move. [EPILOGUE] I'll probably write some stats about what could have been done if the work had been targeted any other way with the cluster. But here is an interesting fact. The weaker password of the MD5 admin list had strength 234 with my training file (it was "John3"). If I knew that there was a password with strength 234, it would have required about 17 hours of computation, using all the previously mentioned resources to be sure it was cracked. Getting a second would have taken 143 hours. I retrained the stats file during the contest using already cracked passwords. If I had reset everything to use the new stat file (a feature not available right now), it would have taken less than an hour to crack two passwords ("John3" and "M20107y"). The next one would have consumed 77 hours. [WHAT SHOULD HAVE BEEN DONE] It was not obvious when the challenge started, but the preferred resolution method for people with large clusters looks like : - crack hashes using whatever "not too smart" method (JtR incremental mode, markov mode, etc.) - find patterns that could be translated to mangling rules and the corresponding wordlist - use a system to queue wordlists / mangling rules job units and to schedule them on your cores - rince an repeat </pre> bartavelle-JGabbFpyS5Pk1uMJSBkQmQ< at >public.gmane.org 2010-08-10T12:03:19 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.security.openwall.john.user