In the Beginning There was Full Disclosure

Two of the largest companies in the world are bickering with each other about how best to protect users. I won’t get into just how historically hypocritical this is for both Microsoft and Google or how childish it makes them both look but it brings up a debate that has been raging in security circles for over a hundred years starting way back in the 1890s with the release of locksmithing information. An organization I was involved with, L0pht Heavy Industries, raised the debate again in the 1990’s as security researchers started finding vulnerabilities in products.

In the beginning there was full disclosure, and there was only full disclosure, and we liked it. In the beginning the goal was to get stuff fixed, it wasn’t about glory, it wasn’t about bug bounties, it wasn’t about embarrassing your competition. No, in the beginning it was about getting bugs fixed. It was about getting the software that you used, the software you deployed to your users, it was about getting it fixed, getting it to be safe. However, in the beginning vendors didn’t see it that way, many of them still don’t. Vendors would ignore you, or purposely delay you. There is no money in fixing bugs that no one else is complaining about so most vendors wouldn’t fix them, at least not until it became public and all of their customers started to complain about them. That was the power of full disclosure.

Vendors of course hated full disclosure because they had no control over the process, in fact there was no process at all and so they complained, vociferously. Vendors talked about ethics and morality and how full disclosure helped the bad guys. So a guy named Rain Forest Puppy published the first Full Disclosure Policy promising to release vulnerabilities to vendors privately first but only so long as the vendors promised to fix things in a timely manner. If the vendor didn’t get stuff fixed the researcher could still pull out their most effective tool, full disclosure, to get the job done.

But vendors didn’t like this one bit and so Microsoft developed a policy on their own and called it Coordinated Disclosure. Coordinated Disclosure calls on researchers to work with the vendor until a fix can be released regardless of how long that takes. Under Coordinated Disclosure there is no option for Full Disclosure at all. Of course Coordinated Disclosure assumes that the vendor is even interested in fixing the bug in the first place.

The problem that many companies who have vulnerability disclosure policies don’t realize, such as Microsoft, is they have forgotten that they are not the ones in control. Vendor disclosure policies are not binding on the researcher. It is the researchers choice whether or not to follow a company’s disclosure policy. Vendor policies work great for the vendor, it gives them all the time in the world to fix a bug but for researchers who want to get stuff fixed such policies can be a major pain to work within.

Disclosing vulnerabilities isn’t an easy thing. In the mid nineties at L0pht Heavy Industries we quickly found that vendors had absolutely no interest in fixing bugs at all and instead would prefer that we just kept our mouths shut. A lifetime later it was part of my job to help coordinate vulnerability disclosure with various vendors that were found by our pentesters. If you’re a lone researcher and only have one vulnerability its not such a big deal, you send a few emails, wait a little while and if the vendor is cooperative a fix is pushed out in a few days or months time. If you happen to have several dozen vulnerabilities that you are attempting to get fixed, all at the same time, and all by different vendors, the process can be a little more involved. In fact simply coordinating these disclosures can be a full time job for multiple people within an organization. There is no ROI here either, the ‘simple’ process of attempting to disclose vulnerabilities eats up revenue in the time your employees take trying to coordinate vulnerabilities and get stuff fixed.

In 2009 several researchers found the disclosure process so onerous that they started the “No More Free Bugs” campaign and promised not to release any vulnerabilities for free. In response vendors started bug bounty programs where they rightly paid researchers for their hard work. However, even that process comes at a cost for both the vendor and the researcher. So much so that there are now third party companies that will help vendors run bug bounty programs and help researchers disclose vulnerabilities.

Of course there are still vendors who refuse to fix stuff or who wait forever to do so. According to Tipping Point’s Zero Day Initiative there are currently 212 known security vulnerabilities without fixes, several of which are over a year old. It is likely that the only way any of these ancient bugs will get fixed is by pulling out the old standby of Full Disclosure. In fact Tipping Point has threaten to do just that, giving vendors just six months to get stuff fixed before they publish limited details on the bugs.

This has all lead us to the point where Google has a disclosure policy that basically says they’re going full disclosure in 90 days if the bug is fixed or not. And the point where Microsoft is asking for just a few more days so they can include the fix with their regular Patch Tuesday. Two big kids who should be setting the example are instead acting like a couple of teenagers on the playground. How does any of this get stuff fixed and protect users?

This is why you see many companies and individual researchers not disclosing anything at all, and this should not happen. And I haven’t even gotten into the issue of vendors filing lawsuits against researchers as a means to keep them quiet.

The entire process has gotten out of hand. The number one goal here should be getting stuff fixed because getting stuff fixed helps protect the user, it helps defeat the bad guys and it helps make the world a better place.

Microsoft says that full disclosure “forces customers to defend themselves” which is the wrong way to look at it. Full disclosure allows companies to defend themselves if they so choose. The opposite is non-disclosure, which helps no one. Just because a bug hasn’t been disclosed doesn’t mean it is not there. It doesn’t magically pop into existence only when someone publishes something about it. The bug is there, waiting to be found. Maybe the bad guys already found it. Maybe they are already using that bug against you. And yet you are blissfully unaware that the bug even exists. Full disclosure gives you knowledge that you can use to protect yourself even if a patch is not available. You can choose to turn off the affected device, or add additional protections to your environment to help you mitigate the risk. Once disclosure happens this is now your choice, you can evaluate the risk this particular bug presents to your environment and make an educated decision of what steps to take depending on your own risk tolerance. While most users will continue on blissfully unaware or choose to ignore the information that too is their choice, not Microsoft’s, and not Google’s.

Google’s goal of getting everything they find fixed within three months is laudable but unrealistic. Some bugs just take a little bit longer to verify, develop patches for, and test. It is not unreasonable to be a little flexible if you feel the vendor is working in good faith to develop a patch. To arbitrarily go full disclosure when you know the vendor has a patch just days away is immoral. It puts users at risk and makes you look like a stubborn child.

In this particular case both the vendor and the researcher are wrong. Microsoft obviously communicated the status of the fix to Google and told Google when to expect the patch. It is not unreasonable for Microsoft to ask for a few extra days and it should not be unreasonable for Google to grant such a request. On the other hand I am sure Google informed Microsoft that they would only wait 90 days before going full disclosure, Microsoft was informed of the risk of full disclosure and should have pushed harder to meet the deadline.

And so the disclosure debate continues unabated for over a hundred years. With two of the giants in our industry acting like spoiled children we as security professionals must take the reigns from our supposed leaders and set a better example. It needs to be about protecting the user. It should not be about grandstanding or whining or even making a buck, in the end it should be about getting stuff fixed.

UPDATE 2015.02.13
Google has made an update to its 90-day disclosure deadline. They have decided to make allowances for deadlines that fall on weekends and holidays and more importantly have granted a grace period for vendors who communicate their intent to release a patch with 14-days of the 90-day deadline. It is nice to see vendors and researchers working together. The goal here should be to protect the users and not embarrass vendors. This grace period shows an understanding of the issues surrounding disclosure that impact vendors while at the same time continuing to hold them to a high standard.

Interested in reading more?

Microsoft’s latest plea for VCD is as much propaganda as sincere – OSVDB

Microsoft blasts Google for vulnerability disclosure policy – CSO Online

A Call for Better Vulnerability Response – ErrataSec

Four Unnamed Sources

Or: If a pipeline explodes in the desert and there is no one there to hear it was it really a cyberwar attack?

No one questions the importance of keeping abreast of current trends and developments with regards to information security. Whether it is new malware techniques, attack vectors or just the motivation of some attackers. That means looking into the details of the Target and Sony breaches, checking out the specifics of Heartbleed and Poodle, and keeping abreast of the latest patches from Microsoft and other vendors. It also means trying to separate the facts from the fear, uncertainty, and doubt used to generate page views.

One recent story has me questioning if a pipeline explosion in Turkey was actually in fact an early example of cyberwar. The article claims that a large explosion along the Baku-Tbilisi-Ceyhan (BTC) pipeline, near the Eastern Turkish city of Erzincan on Aug. 7, 2008 was in fact a cyber attack. The article attempts to downplay claims of the Turkish government who said the explosion was caused by a malfunction, as well as discounting the claims of the Kurdistan Worker’s Party who claimed credit for the explosion despite the groups history of blowing up pipelines. Of course there was also a statement by the Botas International Ltd. company which operates the pipeline which said that the pipeline’s computers systems had not been tampered with.

The explosion occurred two years before Stuxnet and while I doubt Stuxnet was the first operation of its kind the evidence to support a similar type of attack on this pipeline is mostly circumstantial at best. Even if this was a cyber attack it would not “rewrite the history of cyberwar,” as one expert quoted in the article claimed. It would just add one more data point to an already interesting history. Unfortunately the article does not give any proof that this was in fact a cyber attack.

Certainly the article lists plenty of circumstantial evidence to support the theory of a cyber attack to blow up the pipeline but the actual proof comes down to “four people familiar with the incident who asked not to be identified.” Obviously in some cases journalists must rely on unidentifiable sources however usually when they must do so the information provided is corroborated by other authoritative and named sources. That is not the case here. All of the named quotes in the article are speaking in general terms, adding background if you will, and are not speaking directly to this event.

Pipeline and cyber attacks have a long history in and of themselves that goes back at least as far as 1982 when the CIA convinced a Canadian company to deliberately put flaws into pipeline control software that was then sold to the Soviet Union. This reportedly led to a massive explosion along the pipeline in June of that year. This story also has its detractors, some saying the explosion was caused by poor construction and others saying it was flawed turbines and not flawed software that caused the Siberian explosion.

There was also a confidential report released by DHS in early 2013 claiming that key personnel in 23 different gas pipeline companies had been targeted by Chinese hackers with spear phishing attacks. And lets not forget the plot of the movie DieHard 4 where the evil hacker bad guy is able to redirect all the natural gas in the pipelines to converge on a power station causing a massive Die Hardesque explosion.

One really has to ask themselves why would anyone go to such great lengths to disrupt a pipeline when a simple misplaced cigarette butt can cause a massive explosion like what happened in Kenya in 2011 killing over 100 people. Stuxnet is thought to have required numerous teams of coders working for several months to create the software to disable the centrifuges at Natanz, a task that arguably could be accomplished in no other way. There are a lot more efficient ways to blow up a pipeline than to expend months of effort and untold dollars to accomplish what a small team and some explosives could do just as well if not even more efficiently.

So was the explosion along the Baku-Tbilisi-Ceyhan (BTC) pipeline an early act of cyberwar potentially setting back the clock on the earliest known cyber operation of this size? Sure, its possible, but without additional facts from someone other than an ‘unnamed source familiar with the incident who asked not to be identified” I will have my doubts. Until those facts are presented I’ll go back to reading my Microsoft Patch Tuesday reports.

UPDATE 2015.02.16
I was just sent this link
https://cablegatesearch.wikileaks.org/cable.php?id=08BAKU790
which indicates that physical security of the pipeline would be difficult if not impossible and it further supports that PKK was the primary suspect for the explosion via conventional means. The cable makes no mention of a cyber attack of any kind.

Additional Reading
Looks like I wasn’t the only one with a problem with this article.
Cyberwar revisionism: 2008 BTC pipeline explosion

Another BIG hack that wasn’t

No time to do a full analysis but the basics are a story out of Israel of a tunnel that was hit by a sophisticated cyber attack that caused a… traffic jam. The story went out on the Associated Press newswire on a Sunday afternoon so by Monday morning it was pretty much everywhere you looked.

The “attack” was supposedly a “classified matter” involving “a Trojan horse attack” that targeted the security camera system in the Carmel Tunnels toll road on Sept. 8. The attack caused an immediate 20-minute lockdown of the roadway and then an eight hour shutdown the next day causing a pretty big traffic jam. Supposedly the attack was the work of “unknown, sophisticated hackers” which were then compared to Anonymous but not sophisticated enough to be nation state funded attackers from Iran.

Even just by reading this it sounds like a run of the mill malware infestation and not some targeted sophisticated state sponsored cyber attack. I mean why would anyone specifically target a tunnel? There is no money there, no intellectual property to be stolen, so unless your goal is to create an isolated traffic jam, whats the point? But there is more. The tunnel operators, CarmelTun, issued a statement saying Nope, no cyber attack here. And blamed the traffic jam on a “an internal component malfunction” and went on to say “this was not a hacker attack.”

@snd_wagenseil @4Dgifts @WeldPond more than one source confirmed.

— Daniel Estrin (@DanielEstrin) October 28, 2013

According to @DanielEstrin whose name is on the byline of the story, more than one source confirmed this Trojan Horse attack story and yet he did not bother to confirm with the people most likely to know, the actual operators of the tunnel.

So we can either believe the unnamed “cybersecurity experts” who warned of a sophisticated “Trojan horse attack” that was compared to Anonymous and was conducted for no monetary gain or intelectual property theft or we can believe the operators of the actual tunnel system itself. Who has more to gain here?

Late Update:
Looks like I am not the only one to think this might not have been a cyber attack.
“Cyberattack Against Israeli Highway System? Maybe Not”

Anatomy of Hype, Take 2

I almost wasn’t going to write about the supposed cyber attack at the New York Times last week as reported by Fox Business because I just haven’t had the time but after the NASDAQ went down today and everyone and their brother started to speculate as to the nature of the ‘technical glitch’ I figured I should throw something together.

In my talk ‘Hackers and Media Hype or Big Hacks That Never Really Happened’ I mention that I see this sort of thing every day. That it is rampant throughout the tech press and often leaches over into traditional media outlets as well. I’ve detailed this sort of thing before as in this blog post ‘Anatomy of Hype’ however this time reporters Matt Egan and Jennifer Booton published their unconfirmed ‘cyber attack’ on the FOX Business website and while FOX takes a lot of shit for their style of nearly tabloid journalism they have a much greater reach than tech news outlets like ZDNet.

So lets see if we can piece together what happened here. At approximately 11:30 on August 14th 2013 the New York Times website went down. And by down I mean down hard, nytimes.com and nytco.com were both throwing up 503 site unavailable errors. Hey, shit happens, sites go down, they get fixed they come back up. As anyone who has ever worked on-call for an IT department will tell you despite backups, failovers and triple redundancies this happens ALL THE TIME.

tweet

By 11:53am, about half hour into the outage the official verified New York Times twitter account cited technical difficulties as the reason for the outage.

At 11:55am Matt Egan Matt Egan (@MattEgan5) and Jennifer Booton (@jbooton) pushed the first version (screenshot) of their story “Source: New York Times Website Hit by Cyber Attack”. Their entire basis for the story was ‘a source close to the matter’. A source they fail to identify. A source as it turns out wasn’t all that close to the matter after all.

By 12:31am, internal New York Times employees start referencing an internal email that cites a malfunctioning system patch as the cause for the outage. While Microsoft’s Patch Tuesday was the day before, which may or may not have been the cause of the outage, it made much more sense than a cyber attack.

At 12:47pm, a little over an hour into the outage the New York Times Official twitter account finally offers up an explanation citing a ‘server issue’.

In the face of all this new evidence did FOX Business pull the erroneous story about a cyber attack? Did Matt Egan and Jennifer Booton update their story to reflect the new information?

Well, they did update their story (screenshot), put they updated it with quotes that make it sound like there was still some sort of cyber attack, quotes that are obviously of a hypothetical nature. Quotes that appear to be taken completely out of context but which support the original erroneous hypothesis of a cyber attack.

One of the people who was quoted in the article said afterwards that the reporters came to him saying that they had already confirmed the cyber attack which was the only reason he agreed to speak with them. I have to ask, where was the confirmation? I have never been to journalism school but I suspect that Matt Egan and Jennifer Booton must have slept through the class on confirmation. I always thought you needed two independent sources to confirm a story. A lone ‘source close to the matter’ does not count as confirmation. Where were the FOX Business editors that reviewed this tripe before it was posted to the FOX Business website?

As I did with ZDNet I call on FOX Business to either completely rewrite the story on their site to reflect the currently known facts or to remove it altogether. Leaving a story such as this to fester on their website reflects poorly not just on FOX Business Matt Egan and Jennifer Booton but on the InfoSec industry as a whole, not to mention the damage that it is doing to the New York Times.

The excuse that it fast breaking new story does not fly; a news website has a responsibility to the public to publish accurate and timely information. There is no excuse in this modern age not to update stories with new information as it becomes available. FOX Business has updated this story, several times, but the information is entirely skewed to support the original erroneous hypothesis.

So how about FOX, Matt, and Jennifer, can you take the high road and report the facts or do you prefer to wallow in the muck of fear, uncertainty, and doubt?

Update: Dave Lewis at CSO Magazine has also blogged about this story.

Fitness and Discipline for Cyber Warriors

“More PT Drill Sergeant, more PT! We like it, We love it, We want more of it!”

There is a basic tenant in most of the worlds military forces that regardless of what your actual job or rank is, whether you are a private or a General, whether you are a cook, clerk, or mechanic, below everything, at the very core of your existence you are nothing but a gravel crunching, ground pounding infantry soldier (11B). Or as an old Colonel once told me, the poor slob in the kill zone. (Thank you, Sir!)

As part of the basic core existence in your nations military all soldiers, airmen, and sailors are required to be able to perform a basic set of tasks. Things like knowledge of how to wear your countries uniform, the ability to maintain and operate a firearm, how to use protective equipment such as a gas mask, and above all the ability to give and follow orders. But these items are more than just basic knowledge and rout tasks, it comes down to discipline, self-discipline mostly, that quality of doing what needs to be done without needing to be told or even wanting to do it.

This is what basic training is for, an intense six or maybe ten week training regimen that not only teaches all soldiers basic tasks like how to operate their firearm or shine their boots but also self discipline, the ability to continue doing your job under stressful and adverse conditions. This being the military, lives literally depend on that basic skill. It is discipline alone that is more important than any other trait or skill taught during that introductory basic training course of the worlds militaries.

The only way to teach discipline is to place an individual under stress and at the same time ensure that they can complete required tasks. The easiest way to place an individual under stress without placing them in a potentially hazardous situation is through physical activity. This is one of the reasons why most of the world’s militaries have minimum requirements of physical fitness. Things like a set time and distance for running, a minimum number of pushups or sit-ups. This ensures a minimum level of fitness for all soldiers and helps to ensure basic levels of self-discipline. These basic requirements apply to all soldiers, private or General, cook or mechanic.

There are a few military job specialties that are harder to recruit for than others. Explosive Ordnance Disposal (89D) comes to mind, and there often incentives offered for new recruits to choose one job over another, often these incentives are monetary in the form of signing bonuses or hazardous duty pay. By and large however serving in the military is its own reward for most people for whatever personal reason they have, whether it is monetary compensation, future educational opportunities, patriotism, or in some cases they just like guns.

Recently a new military occupation has evidently become exceeding difficult to recruit for, that of the mythical ‘cyber warrior’ (25B, 35N, 35Q). Militaries around the world are complaining that they just can’t get enough people to fill the jobs they have available for any ‘cyber’ type position. As a way to incentivize new recruits there has been consistent talk that reoccurs every few months of dropping the physical fitness requirements for soldiers, airmen and sailors involved in ‘cyber’ activities. This is a colossally bad idea. Such an action would greatly impact morale of the entire military, will do nothing to increase recruitment numbers for these specialties and draws on an unfounded stereotype of those people who have traditionally been called ‘hackers’.

To create a special class of soldiers that are exempt from minimum fitness requirements will create resentment among other non-exempt units. It will also cause those who are exempt to suffer from issues of elitism and they will feel that they are no longer part of the basic military or required to abide by its rules. With the lack of discipline that will come with the removal of a physical fitness requirement this increase in elitism and individuality in a military setting could prove deadly.

The physical requirements and training aspects of military service are seldom a reason why someone who is interested in joining the military finally decides not to join. On the contrary, there are many examples of people who join the military specifically for the physical aspect that service requires. In fact in my own experience there were two people in my basic training unit who said the primary reason they joined the service was to lose weight, they said that nothing else worked for them and that they hoped the discipline they would learn and the physical exercise would finally accomplish what they could not do on their own.

Claiming that the only people who are qualified or want to do ‘cyber’ jobs in the military are only people who are not interested in physical activity plays on the age-old stereotype of ‘hackers’ who live in their parents basement eating nothing but pizza. Obviously the politicians and Generals who are advocating this no physical fitness requirement for ‘cyber’ operatives have no idea who it is they are trying to recruit anyway. Take a look around at any security industry or hacker conference, sure there are some obviously overweight and out of shape people in attendance but I would be willing to wager that the percentage of people who are somewhat physically fit would be far greater than the regular population.

If the militaries of the world are having problems in recruiting for ‘cyber’ specialties finding the proper incentives to increase recruitment in those areas is critical. As the world ramps up its electronic warfare capabilities being short handed at a precarious time would obviously be ill advised. However, dropping the physical fitness requirement for these soldiers, airmen and sailors is not going to increase their recruitment and retention levels and could potentially damage the effectiveness of the entire military through resentment and lowered morale. The politicians, military analysts and officers who advocate such a major change in military policies are obviously ignorant of not only who it is they are trying to recruit but the basic core of how todays modern military actually works.

MarineTimes_cover2013.03

Book Review: This Machine Kills Secrets

Book Review: This Machine Kills Secrets
By: Andy Greenberg
Penguin Group 2012
ISBN 978-1-101-59358-5

*Page references have been taken from the electronic iPad version

I’ll admit I haven’t finished the whole book yet but the way the book portrays some events I was involved in differs from my own memory. I wanted to highlight those sections, especially since I am quoted in the book more than once. In general Greenberg has done an excellent job in describing the L0pht and some of the events that took place around it but I take issue with some of the descriptions of places and things, while not inaccurate, Greenberg’s choice of adjectives describes settings in entirely different lights than how I remember them.

“exploring the dark corners of the Internet and charting the back doors in labyrinth alleys” (pg. 203)

I have never understood this type of definition of the early Internet. The mid nineties Internet was small, it was unbelievably tiny compared with today. There were no “labyrinth alleys”, it was not a dark and foreboding place at all, at least not to me. To me it was just the opposite, the Internet helped to shine bright lights on subjects I knew little or nothing about at the time and not just technological topics. In the mid nineties the net was a wealth of information with easy access to experts on any subject. It was free from advertisements or sites just looking for page views. There was nothing really dark or labyrinth about it at all. Describing it as such two decades later makes for great reading though.

“where Mudge was often regarded as the most visible and brilliant member.” (pg 203)

This sentence implies that I, and the rest of the L0pht, thought Mudge was the most brilliant of all of us. Was he the most visible? Absolutely, and that was mostly by design. But was he the most brilliant? No, none of us were. All of us had our own strengths, our own areas of brilliance, including Mudge. The L0pht is the only organization I have ever been involved in that came as close as you can to a true egalitarian structure, a meritocracy, where no one was any more brilliant than any one else. We all had individual strengths, each strength complimented each others weaknesses, a lot of those strengths over lapped, but to imply, as Greenberg has, that Mudge was considered the most brilliant by the other members of the L0pht is woefully inaccurate.

“It was a young male scene drawn from an online bulletin board called the Works, where Zatko had made a name for himself under the pseudonym “Mudge.” (pg. 232)

First the board was known as The Works, a minor nitpick for sure, and it wasn’t 100% male but women were definitely outside the norm. By the time Works Gatherings were occurring everyone pretty much new Mudge anyway. Other boards such as ATDT East and Black Crawling Systems where considered much more ‘elite’ than The Works. The Works was more of a social hangout and info repository while other boards took the technological lead. That is why it fell onto The Works to have these in the flesh get-togethers known as Works Gatherings. This was long before 2600 meetings started happening in Boston, which the Works Gatherings eventually morphed into. But to say that Mudge or anyone made a name for themselves on The Works shows a lack of understanding of the dynamics of the early 90s BBS scene in the 617 area code. Such an understanding would probably take a lot longer to explain than the one sentence Greenberg gives it or the one paragraph I am giving it here.

“In later incarnations, the L0pht would add a PC with web access rigged to the toilet for convenient web browsing.” (pg. 232)

Yes, we had an old terminal in the bathroom. No, it was not rigged to browse the Internet or anything else. If I remember correctly it was either an early POS terminal or something used at an airline, I don’t remember, either way as far as I remember it did not work and you could not surf or do anything else on it. Even if it did the screen was about five inches diagonal and monochrome so who would want to?

“Space Rogue, a former army soldier with close cropped hair, hosted the Mac Whacked Archive, an FTP download site with the worlds largest collection of Apple hacking tools.” (pg 233)

It was the Whacked Mac Archives! I am going to blame this on Greenberg’s editors because I gave him an interview for this book and I know I didn’t give him the wrong name. Come on Andy, a simple Google search by your fact checker should have found this one. And another minor nitpick, it hosted Macintosh tools, not Apple. These days Mac and Apple pretty much mean the same thing but even as late as the mid nineties Macintosh software and Apple software were two completely different things.

“The first night Mudge entered the L0pht, the elite group of hackers were struck by his technical genius…” (pg 233)

Oh please, we were not, or at least I wasn’t. Greenberg is making it sound like some deity had descended from the heavens to walk among us mere mortals. Greenberg paints a very radiant picture here that would make a great movie scene but the reality is much more mundane. Very very few people were ever invited into the L0pht that we didn’t know, either in person or online, beforehand. So when Mudge first entered the L0pht we already knew him, who he was, and what he knew and he already knew, or knew of, us. The first meeting in the L0pht was mostly to discus L0pht logistics, like how much each person payed in rent, were he would sit, when we had meetings, etc… It was not an introduction. Were we impressed by his technical genius? Only so much as it matched our own. Mudge definitely has his own reality distortion field; his own cult of personality and that was definitely something that the L0pht needed at the time.

“But Count Zero was going through a messy divorce that kept him away from the L0pht for months at a time, long enough for Mudge to stake his claim.” (pg 233)

This reads like Mudge engineered some kind of coup to oust Count Zero and take control and that is absolutely NOT what happened. I will admit this episode was messy and handled about as well as a bunch of socially inept computer geeks could handle it but to imply that Mudge came in, kicked out Count Zero and took over is just flat out plain wrong.

“They sold T-shirts, attracted groupies…” (pg 234)

OK, how come no one told me about the groupies? Are there any left?

“At the next Black Hat security conference in Las Vegas, the software megalith’s executives took the L0pht out for an expensive dinner…” (pg 235)

This meeting did actually take place, I don’t remember if it was in conjunction with Black Hat or not, I seem to remember that it was not. Greenberg implies that the whole L0pht was present, we were not. Mudge was there, of course, and I think someone else might have attended but it definitely was not the whole L0pht as Greenberg implies.

“Eventually, several of the L0pht’s members would be hired to work for Microsoft as security consultants.” (pg 235)

As far as I know this is false, none of us were hired by Microsoft directly. I’ll admit I haven’t kept up with everyone’s employment history over the years so it is possible that maybe one of us did a few days or weeks of consulting but as far as I know that was not the case. What did happen sometime in the early 2000s is that Microsoft went on a massive security hiring binge, scooping up all the laid off talent from the security industry implosion after the dot com bubble burst. Many people who worked at @Stake, Guardent, Foundstone, etc ended up at Microsoft, some of them are still there but as far as I know no one from L0pht worked there in any capacity.

“…high level cabinet official travelled alone to clandestine meetings with digital miscreants.” (pg 241)

This sentence annoys me, especially the use of the words clandestine and miscreants. The meeting described here was not clandestine, I am sure it was on Clarke’s official travel schedule, and its not like we met in a dark alley or anything. In fact I’m not entirely sure this meeting happened exactly as it is described. I distinctly remember meeting Clarke with other L0pht members for the first time at John Harvard’s, we both had the chicken pot pie. Now maybe Mudge had an earlier meeting with Clarke as Greenberg described that I wasn’t aware of, I don’t know. Greenberg’s description of this cloak and dagger meeting seems more like a setup for a movie deal than something that actually happened. And what’s with the use of the word miscreant, the definition of which is depraved or villainous, come on.

“For a moment, Clarke huddled with his NSC colleagues in private conversation.” (pg 242)

The meeting Greenberg describes includes the L0pht, Clarke and four NSC guys but that is not how I remember it. At most there were two other guys with Clarke but I am pretty sure there was only one other guy with Clarke. I don’t remember most of the rest of this paragraph either. What I do remember took place in the parking lot outside the L0pht. Clarke was huddling with the other one or two NSC guys who were there, when Mudge standing of to the side with the rest of the L0pht guys yelled over to them, “Hey, we opened the Kimono and showed you ours, what are you guys talking about?” To which Clarke responded that he was very surprised by what he had witnessed at the L0pht and that up until that point he had always assumed that to do what we had been doing would take the support of a nation-state or other large organization, and not seven guys in a rented space in some warehouse. So Greenberg’s version has the same gist to it, just not exactly as how I remember.

“On the way they stopped at the NSA’s Cryptologic Museum and accidently drove past the guards into the agencies secure facility, before timidly backing out.” (pg 242)

If you have ever been to the Cryptological Museum you know that as described this isn’t really possible. The museum is public and open to anyone, however on the drive down we missed the exit off the highway for the museum, so we took the next exit. We found a place to turn around but before we realized it we were passing the NSA guard shack. Imagine a large Ford Econline van with out of state plates, at least four antennas on top and heavily tinted windows. We didn’t know if we should stop or keep going, the guard saluted us, we saluted back and the guard waved us through so we kept on driving. There really wasn’t anything timid about it. Once inside we quickly turned around, left and went back to the Museum. In fact if you ever go to the Cryptological Museum and look in the guest book back to 1998 you will see an entire page that we signed as “L0pht World Tour”

“and ended their trip hanging out with Secret Service agents at Archibald’s, a nearby strip club.” (pg 243)

Umm, no. We did not hang out with Secret Service agents at a strip club or any other type of club. I have no idea where Greenberg got this. It would definitely play well if Greenberg sells the movie rights to this book but it didn’t happen. I remember hanging out in the hotels Irish bar, having one glass of Guinness and then going to bed.

 

None of the items I have listed here are really all that egregious or detrimental to the story. However, since I was there, and I remember things slightly differently than how they have been portrayed by Greenberg I thought it important to illustrate those differences here. I think the biggest thing I have issue with is the tone Greenberg uses in certain sections, he accurately describes the physical L0pht as a technological clubhouse but then describes clandestine meetings and labels us as miscreants. The description of the L0pht and the events surrounding it only make up a few pages of the over all book but considering the inaccuracies and or liberties Greenberg has taken to describe this one small section I have to wonder what other parts have been slightly embellished or possibly misremembered from his other sources throughout the rest of the book.

On the other hand I am impressed by just how much Greenberg has gotten right. There have been numerous attempts over the years to accurately describe the L0pht and some of the events that surrounded it, despite the inaccuracies I have listed, this is as close as anyone has come. It is obvious that Greenberg put a lot of work into this book, or at least this section, and gathered information from a lot of sources.

Given the topical subject matter I would not be surprised at all to see this book optioned to a movie. Unfortunately a movie will only be two hours long and I don’t see how you would be able to fit this one chapter, let alone the entire book, into two hours without cutting out large chunks and glossing over the many details that took Greenberg so long to gather.

Emails From Michael In Iran

If publishing unsourced emails claiming to be from Iran is a newsworthy event then I guess we should all copy Mikko and do the same thing.

A few years ago I received a chain of emails from ‘Michael’ that started out as the normal ‘teach me to hack’ emails I receive on an almost daily basis but this email chain went on longer than usual and took several turns I don’t usually see in such emails. I thought they might be good for a laugh or a tear depending on your viewpoint.

TL;DR

The emails start in May of 2009 and go through to December, I have not included them all and have edited some for brevity.

Things start out simple enough saying how he is a 20yr old Iranian and is a fan of the L0pht. Pretty straight forward. I responded as I usually do to emails that are at least half way intelligent. I admit I don’t always get emails from Iran with a verifiable Iranian IP address.

Then comes the first turn, ‘Micheal’ asks me to teach him to ‘hack’ specifically so he can change his grades at University. For me thats a big no no right there. If you ask me to do, or teach you to do, anything even remotely illegal in email thats where I stop. I will no longer respond. I don’t want to be considered an accessory or an accomplice or be put in an un-winnable Adrian Lamo type situation. Not to mention the whole assisting a foreign power angle. So I just stopped responding.

But Michael wouldn’t give up, he sent me an email every day for weeks, then slowed down to a few times per week. Eventually he reached out other old L0pht members, those whose email address he could find, asking them if I was OK, saying he feared for my safety since I was not responding to emails. I will admit I felt a little bad at this because who knows maybe people just disappearing like that in his country is a sign of something sinister happening. I don’t know. My remorsefulness did not last long however.

Next came the names and the threats. ‘Michael’ called me a raciest and threatened to ‘destroy my life’ and that despite my lack of assistance he was going to become the worlds greatest hacker anyway and he was going direct his efforts at me. Then he was going to hack his University, graduate and travel to America to prove to me in person that he was a great hacker and that he did it all without my help.

I had a good laugh and a tear at the time, 2009, but as I read over these emails again and place them into the context of the ongoing ‘cyber’ cold war they really take on a different meaning. How many other people in Iran have similar motivations? I wonder if Michael ever made it through University, or maybe he got caught and ‘disappeared’? I will probably never know.

Email exchange with Michael from Iran

OMG the SCADA is Falling!!!

Let me say first that SCADA (supervisory control and data acquisition) attacks are real, they do happen and should be a real concern. But if we look at the recent press surrounding such attacks we see little in the way of any hard evidence that such an attack actually occurred. Instead we see rumor and accusation and a hell of of a lot of theory.

The most recent example is the report, first reported on by The Register that someone broke into a local water utility and caused a pump to fail by turning it off and on repeatedly. This is a completely plausible scenario but when we look a little closer at the report some holes start to develop.

The media gabbed a hold of this story and quickly spread it around, over sixty different articles that I can find so far, yet none of them cite ANY primary sources for the incident. That’s Journalism 101 folks, and I didn’t even take journalism class. The Register article quotes Joe Weiss, a managing partner for Applied Control Solutions talking about the attack. This would seem to lend provenance to the story and that the attack actually happened, but Weiss was not a primary source. Most of his quotes are hypothetical and refer to an ‘official government report’ that he refused to name. Weiss refused to state which water district was targeted other than to say the report was released on November 10th. According to Weiss a software vendor lost control of its customer username and password database which allowed attackers, who had been traced back to Russia, access to the systems.

The Register at least got a comment from the US Department of Homeland Security indicating the utility in question was located in Springfield, Illinois. I’m not sure why the Register did not pick up the phone and call Springfield but Kim Zetter from Wired did call. The Springfield water department denied it was them and said the attack took place in the Curran-Gardner water district. When she called Curran-Gardner they hung up on her.

By the time the story made it to C|Net they actually had a quote from DHS.


“DHS and the FBI are gathering facts surrounding the report of a water pump failure in Springfield Ill.,”
DHS spokesman Peter Boogaard said in a statement. “At this time there is no credible corroborated data that indicates a risk to critical infrastructure entities or a threat to public safety.”

The key words that I see are ‘no credible corroborated data’ – Bingo! Now, it is possible that DHS is downplaying this so as to not cause widespread panic but lets face it, this is DHS, their whole reason for existing is wide spread panic. So if they say there is ‘no credible corroborated data’ I’m going to go with that.

So what facts do we have that can be confirmed? I think it is pretty safe to say that a water pump somewhere in Illinois failed. I also think it is pretty safe to say that some secret government report blamed that failure on Russian hackers. Thats it. Everything else is pure speculation.

Now lets read between the lines shall we? Lets assume that a pump somewhere in Illinois, over the course of several weeks or even months turned itself off and on and failed. Pumps fail all the time, it happens, doesn’t mean they were hacked. Unfortunately we don’t know what kind of pump, who manufactured it or how long it had been turning off and on before someone noticed. Now what if the code controlling this system was flawed in such a way that the control loop code wasn’t working properly? Control loops are tricky things and it is easy to screw them up, especially if your a pump manufacturer and don’t really pay attention to closely to the software that controls them. Now I have no more evidence to say that this was a software glitch than I do to prove it was an external intrusion. But doesn’t a control software glitch sound a hell of a lot more possible than a russian breaking into a small Illinois township water district?

I think @Jack_daniel said it best “No one sentient doubts the vulnerability of SCADA systems, but for the love of $DEITY SHARE REAL DETAILS or crank up the skeptic settings.”

Late Update:

“Whether the burnout of that pump was related to this what might or might not have been a hacking, we don’t know,” said Don Craven, a water district trustee.”

That pretty much settles that in my book.

Although I have to share one last quote from the Curran-Gardner Water District trustee “I drank the water this morning.”

– SR

2011.11.25 – Update
One last update, looks like those strange Russian IP addresses actually came from Russia! Via a contractor who had authorized remote access. Imagine that. Yup, blame the contractor.

– SR

Hackers Need Not Apply

Back in the nineties, the glory days of Hacking, just after the golden age of the late eighties, many companies were starting to get into the whole Internet Security thing. Everyone and their brother had an Internet Security company and VC were just crawling over each other to give them money. One thing most of the early companies had in common was a staunch refusal to hire ‘hackers’. They would give speeches at conferences and say ‘We hire only the best security experts, but no hackers’ They would issue press releases that said the same thing. I remember reading these and laughing because all the hackers I knew worked at these very same companies. (The ISS XForce said this all the time, and everyone who worked there was a hacker.)

At the time this was a brand new industry that basically took shape over night. There were so many security startups you literally couldn’t through a rock without hitting one. Foundstone, Guardent, @Stake, and those are just the big names that I remember off the top of my head, there were dozens of other smaller firms all vying for a piece of the pie and for the ever decreasing pool of talent. Basically if you knew what a war dialer was, could run a file of hashes through L0phtCrack and knew how to clear your browser cache you were hired as a Security expert at a 100K a year. It was that easy.

So what did all us hackers do? Well, we got jobs naturally. We got jobs at the very same companies who said “We don’t hire hackers”. Very very few of us actually had criminal records and those who did usually had them sealed due to a juvenile status at the time. So when it came time to fill out the employment history on the job application you filled it out truthfully, Landscaper, Burger King, Tech Support, and now Security Expert. Nowhere did you write down ‘Hacker’. When we went into the job interview we did not wear a big sign around our necks that said ‘Hacker’.

At some point after @Stake acquired the hacker think tank L0pht Heavy Industries this whole ‘we don’t hire hackers’ thing started to change. A lot of companies saw that it added to their credibility to say that they had a hacker or two on staff or if they didn’t actually publicize it they definately didn’t make assinine statements like “We don’t hire Hackers”.

Well, I guess things have come around full circle. Because Enrique Salem over at Symantec has stated that “You always worry about [grey hats]. Symantec has a standing policy that we don’t hire anyone to be a part of our company who has done any kind of known hacking,” he said. “We will not employ hackers.”

Enrique has been at Symantec for 16 years now but maybe he was to busy doing whatever is was he was doing before he got the CEO job in April to realize that his company does hire hackers. Or at least they did ten years ago when they bought @Stake and its old L0pht (and CDC) members. (OK, so I guess technically they bought them and didn’t actually hire them but semantics.)(Hey, always wanted to make that pun, hehe) At least one of the old L0pht folks was still working there up until a few years ago.

But even now there are more people than I can count on one hand who I know personally that work at Symantec who are publicly well known hackers. They speak at Hacker cons, are known by their handles and call themselves hackers. They don’t go around advertising where they work but its not a big secret to those of us in the community. I don’t think they have criminal records and I doubt they go around breaking into other peoples computers but then hacker does not equal criminal.

If you want to go around and say “We don’t hire hackers” that’s fine, just realize that there aren’t going to be very many people left to hire and you sound like an idiot when you say it. (Hey, DHS, are you listening?)

L0phtCrack 6 to Be Released at Source Boston

L0phtCrack, the original and still the best password auditing tool for MS windows based systems, will be re-released at Source Boston by the original authors! That’s right Mudge, Dildog and Weld Pond have required the rights to the original L0phtCrack and plan to release a new version at the upcoming conference. The new L0phtCrack will have support for 64-bit windows and upgraded rainbow tables. Woohoo! Details on potential additional new features, and pricing have not yet been released but you can bet that it will be better than Symantec’s.

Source Boston 2009
L0phtCrack.com