Follow by Email

Saturday, September 28, 2013

Senator Feinstein Admits the NSA Taps the Internet Backbone

We know from the Snowden documents (and other sources) that the NSA taps Internet backbone through secret-agreements with major U.S. telcos., but the U.S. government still hasn't admitted it.
In late August, the Obama administration declassified a ruling from the Foreign Intelligence Surveillance Court. Footnote 3 reads:
The term 'upstream collection' refers to NSA's interception of Internet communications as they transit [LONG REDACTED CLAUSE], [REDACTED], rather than to acquisitions directly from Internet service providers such as [LIST OF REDACTED THINGS, PRESUMABLY THE PRISM DOWNSTREAM COMPANIES].
Here's one analysis of the document.
On Thursday, Senator Diane Feinstein filled in some of the details:
Upstream collection…occurs when NSA obtains internet communications, such as e-mails, from certain US companies that operate the Internet background [sic, she means "backbone"], i.e., the companies that own and operate the domestic telecommunications lines over which internet traffic flows.
Note that we knew this in 2006:
One thing the NSA wanted was access to the growing fraction of global telecommunications that passed through junctions on U.S. territory. According to former senator Bob Graham (D-Fla.), who chaired the Intelligence Committee at the time, briefers told him in Cheney's office in October 2002 that Bush had authorized the agency to tap into those junctions. That decision, Graham said in an interview first reported in The Washington Post on Dec. 18, allowed the NSA to intercept "conversations that . . . went through a transit facility inside the United States."
And this in 2007:
[The Program] requires the NSA, as noted by Rep. Peter Hoekstra, "to steal light off of different cables" in order to acquire the "information that’s most important to us" Interview with Rep. Peter Hoekstra by Paul Gigot, Lack of Intelligence: Congress Dawdles on Terrorist Wiretapping, JOURNAL EDITORIAL REPORT, FOX NEWS CHANNEL (Aug. 6, 2007) at 2.
So we knew it already, but now we know it even more. So why won't President Obama admit it?

Friday, September 27, 2013

Google Recently Made A Silent Shift To A New Search Algorithm, “Hummingbird”

Have you noticed recently that Google has gotten a bit better at offering up direct answers to questions? If so, there’s a reason for it: they recently flipped the switch on a new search algorithm they call “Hummingbird”, which focuses on parsing searches as complex questions.
Google mentioned the new algorithm for the first time today, at an event that was (in a confusing surprise to everyone who arrived at Google HQ and was put on a bus) hosted in the garage that Larry and Sergey rented as Google started to prove successful. Other things announced include a tweak to Google’s Knowledge Graph to allow it to handle comparison questions (“Which is better for me — olive oil or butter?”), and Push Notifications for Google Now on iOS.
Despite a good amount of questioning from the audience on just how Hummingbird worked, Google avoiding getting too technical. While they did say that this was the biggest overhaul to their engine since the 2009 “Caffeine” overhaul (which focused on speed and integrating social network results into search) and that it affects “around 90% of searches”, there wasn’t much offered in terms of technical details.
The main focus, and something that went repeated many a time, was that the new algorithm allows Google to more quickly parse full questions (as opposed to parsing searches word-by-word), and to identify and rank answers to those questions from the content they’ve indexed.
As for how it’ll affect results, moving forward (the ears of a zillion SEO dudes/dudettes just perked): the engine overhaul was silently put in place weeks ago, right under all of our noses. If you haven’t noticed any huge jumps or drops in your search engine placement, you probably won’t any time soon — at least, not as a result of the new algorithm.

Python + Hadoop: Real Python in Pig trunk:



For a long time, data scientists and engineers had to choose between leveraging the power of Hadoop and using Python’s amazing data science libraries (like NLTK, NumPy, and SciPy). It’s a painful decision, and one we thought should be eliminated.
So about a year ago, we solved this problem by extending Pig to work with CPython, allowing our users to take advantage of Hadoop with real Python (see our presentation here). To say Mortar users have loved that combination would be an understatement.
However, only Mortar users could use Pig and real Python together…until now.
As of this week, our work with Pig and CPython has now been committed into Apache Pig trunk. We’ve always been deeply dedicated to open source and have contributed as much as possible back to the community, so this is just one more example of that commitment.
Why is CPython support so exciting? To fully understand, you need to know a little bit about the previous options for writing Python code with Hadoop.
One common option is for people to use a Python-specific Hadoop framework like mrjob, Pydoop, or Dumbo. While these frameworks make it easy to write Python, you’re stuck writing low-level MapReduce jobs and thus miss out on most of Pig’s benefits as compared to MapReduce.
So what about Python in Pig? Before CPython support, you had two options: Jython User Defined Functions (UDFs) or Pig streaming.
Jython UDFs are really easy to write in Pig and work well for a lot of common use cases. Unfortunately, they also have a couple of limitations. For serious data science work in Python, you often want to turn to libraries like NLTK, NumPy, and SciPy. However, using Jython means that all of these libraries that rely on C implementations are out of reach and unusable. Jython also lags behind CPython in support for new Python features, so porting any of your existing Python code to Jython isn’t always a pleasant experience.
Streaming is a powerful and flexible tool that is Pig’s way of working with any external process. Unfortunately, streaming is difficult to use for all but the most trivial of Python scripts. It’s up to the user to write Python code to manage reading from the input stream and writing to the output stream, and that user needs to understand Pig’s serialization formats and write their own deserialization/serialization code in Python. Moreover, the serialization code lacks support for many common cases like data containing newline characters, parentheses, commas, etc. Errors in the Python code are hard to capture and send back to Pig, and even harder to diagnose and debug. It’s not a process for the faint of heart.
Looking at these alternatives, people who want to use Python and CPython libraries in Pig are stuck. But with CPython UDFs, users can leverage Pig to get the power and flexibility of streaming directly to a CPython process without the headaches associated with Pig streaming.
Here’s a quick example: Let’s say you want to use NLTK to find the 5 most common bigrams by place name in some Twitter data. Here’s how you can do that (using data from the Twitter gardenhose we provide as a public convenience):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
REGISTER ‘<python_file>’ USING streaming_python AS nltk_udfs;
tweets = LOAD 's3n://twitter-gardenhose-mortar/tweets'
'text: chararray, place:tuple(name:chararray)');
-- Group the tweets by place name and use a CPython UDF to find the top 5 bigrams
-- for each of these places.
bigrams_by_place = FOREACH (GROUP tweets BY GENERATE
group AS place:chararray,
COUNT(tweets) AS sample_size;
top_100_places = LIMIT (ORDER bigrams_by_place BY sample_size DESC) 100;
STORE top_100_places INTO '<your_output_path>';
view raw nltk.pig hosted with ❤ by GitHub
1 2 3 4 5 6 7 8 9 10 11 12
from pig_util import outputSchema
import nltk
def top_5_bigrams(tweets):
tokenized_tweets = [ nltk.tokenize.WhitespaceTokenizer().tokenize(t[0]) for t in tweets ]
bgm = nltk.collocations.BigramAssocMeasures()
finder = nltk.collocations.BigramCollocationFinder.from_documents(tokenized_tweets)
top_5 = finder.nbest(bgm.likelihood_ratio, 5)
return [ ("%s %s" % (s[0], s[1]),) for s in top_5 ]
view raw hosted with ❤ by GitHub
And that’s it.  You get to focus just on the logic you need, and streaming Python takes care of all the plumbing.
To run this yourself, you’ll need a Pig 0.12 build and a Hadoop cluster with Python and NLTK installed on it. If that’s too much hassle, you can run it locally with the Mortar framework or at scale on the Mortar platform for free.

Wednesday, September 25, 2013

How a Crypto ‘Backdoor’ Pitted the Tech World Against the NSA

Illustration: alengo/Getty Images

In August 2007, a young programmer in Microsoft’s Windows security group stood up to give a five-minute turbo talk at the annual Crypto conference in Santa Barbara. It was a Tuesday evening, part of the conference’s traditional rump session, when a hodge-podge of short talks are presented outside of the conference’s main lineup. To draw attendees away from the wine and beer that competed for their attention at that hour, presenters sometimes tried to sex up their talks with provocative titles like “Does Bob Go to Prison?” or “How to Steal Cars – A Practical Attack on KeeLoq” or “The Only Rump Session Talk With Pamela Anderson.”
Dan Shumow and his Microsoft colleague Niels Ferguson titled theirs, provocatively, “On the Possibility of a Back Door in the NIST SP800-90 Dual Ec Prng.” It was a title only a crypto geek would love or get.
The talk was only nine slides long (.pdf). But those nine slides were potentially dynamite. They laid out a case showing that a new encryption standard, given a stamp of approval by the U.S. government, possessed a glaring weakness that made an algorithm in it susceptible to cracking. But the weakness they described wasn’t just an average vulnerability, it had the kind of properties one would want if one were intentionally inserting a backdoor to make the algorithm susceptible to cracking by design.
For such a dramatic presentation — by mathematicians’ standards — the reaction to it was surprisingly muted. “I think folks thought, ‘Well that’s interesting,’ and, ‘Wow, it looks like maybe there was a flaw in the design,’” says a senior Microsoft manager who was at the talk. “But there wasn’t a huge reaction.”
Six years later, that’s all changed.
Early this month the New York Times drew a connection between their talk and memos leaked by Edward Snowden, classified Top Secret, that apparently confirms that the weakness in the standard and so-called Dual_EC_DRBG algorithm was indeed a backdoor. The Times story implies that the backdoor was intentionally put there by the NSA as part of a $250-million, decade-long covert operation by the agency to weaken and undermine the integrity of a number of encryption systems used by millions of people around the world.
The Times story has kindled a firestorm over the integrity of the byzantine process that produces security standards. The National Institute of Standards and Technology, which approved Dual_EC_DRBG and the standard, is now facing a crisis of confidence, having been forced to re-open the standard for public discussion, while security and crypto firms scramble to unravel how deeply the suspect algorithm infiltrated their code, if at all. On Thursday, corporate giant RSA Security publicly renounced Dual_EC_DRBG, while also conceding that its commercial suite of cryptographic libraries had been using the bad algorithm as its default algorithm for years.
But beneath the flames, a surprising uncertainty is still smoldering over whether Dual_EC_DRBG really is backdoored. The Times, crypto experts note, hasn’t released the memos that purport to prove the existence of a backdoor, and the paper’s direct quotes from the classified documents don’t mention any backdoor in the algorithm or efforts by the NSA to weaken it or the standard. They only discuss efforts to push the standard through committees for approval.
Jon Callas, the CTO of Silent Circle, whose company offers encrypted phone communication, delivered a different rump session talk at the Crypto conference in 2007 and saw the presentation by Shumow. He says he wasn’t alarmed by it at the time and still has doubts that what was exposed was actually a backdoor, in part because the algorithm is so badly done.
“If [NSA] spent $250 million weakening the standard and this is the best that they could do, then we have nothing to fear from them,” he says. “Because this was really ham-fisted. When you put on your conspiratorial hat about what the NSA would be doing, you would expect something more devious, Machiavellian … and this thing is just laughably bad. This is Boris and Natasha sort of stuff.”
Indeed, the Microsoft presenters themselves — who declined to comment for this article — didn’t press the backdoor theory in their talk. They didn’t mention NSA at all, and went out of their way to avoid accusing NIST of anything. “WE ARE NOT SAYING: NIST intentionally put a back door in this PRNG,” read the last slide of their deck.
The Microsoft manager who spoke with WIRED on condition of anonymity thinks the provocative title of the 2007 presentation overstates the issue with the algorithm and is being misinterpreted — that perhaps reporters at the Times read something in a classified document showing that the NSA worked on the algorithm and pushed it through the standards process, and quickly took it as proof that the title of the 2007 talk had been right to call the weakness in the standard and algorithm a backdoor.
But Paul Kocher, president and chief scientist of Cryptography Research, says that regardless of the lack of evidence in the Times story, he discounts the “bad cryptography” explanation for the weakness, in favor of the backdoor one.
“Bad cryptography happens through laziness and ignorance,” he says. “But in this case, a great deal of effort went into creating this and choosing a structure that happens to be amenable to attack.
“What’s mathematically creative [with this algorithm] is that when you look at it, you can’t even prove whether there is a backdoor or not, which is very bizarre in cryptography,” he says. “Usually the presence of a backdoor is something you can prove is there, because you can see it and exploit it…. In my entire career in cryptography, I’ve never seen a vulnerability like this.”

National Security Agency headquarters, Fort Meade, Maryland. Photo: Wikipedia
It’s not the first time the NSA has been accused of installing backdoors. Crypto trapdoors, real and imagined, have been part of NSA lore for decades. In some ways the current controversy echoes the long-ago debate over the first U.S. Data Encryption Standard in the 1970s. The NSA was widely suspected of weakening DES to make it more crackable by the agency by tinkering with a table of numeric constants called an S-Box and shortening the algorithm’s key length. In 1994, though, the NSA was exonerated when it turned out that the agency had actually changed the S-Box numbers to harden DES against a code-breaking technique that had been known only within NSA at the time.
In 1995, another case came up that seemed to confirm suspicions about the NSA. The Baltimore Sun reported that year that the NSA had inserted a backdoor into cryptographic machines made by the respected Swiss company Crypto AG, apparently substantiating longstanding rumors to that effect.
Then in 1999, Microsoft inadvertently kicked off another controversy when it leaked its internal name for a cryptographic signing key built into Windows NT. The key was called _NSAKEY, spawning speculation that Microsoft had secretly given the agency the power to write and sign its own updates to Windows NT’s crypto engine. Microsoft said this was incorrect, that the key was an internal Microsoft key only and that it was called “_NSAKEY” because the NSA was the technical reviewing authority for U.S. export controls. The key was part of Microsoft’s compliance with U.S. export laws.
Suspicions about the NSA and backdoors were lingering in 2006 when Shumow and Ferguson began looking at Dual_EC_DRBG after NIST approved it for inclusion in a standard (.pdf). The standard discussed four federally sanctioned random number generators approved for use in encrypting government classified and unclassified-but-sensitive communication.
Each of the four algorithms was based on a different cryptographic design family. One was based on hash functions, one on so-called HMAC (hash-based message authentication code), one on block ciphers and the fourth one was based on elliptic curves. The NSA had been pushing elliptic curve cryptography for a number of years, and it publicly championed the last one — Dual_EC_DRBG — to be included in the standard.
Elliptic curve algorithms are based on slightly different mathematics than the more common RSA algorithm, and the NSA believes they’re the future of cryptography, asserting that elliptic curve algorithms are smaller, faster and offer better security.
But as Shumow and Ferguson examined the properties of the elliptic curve random number generator in the standard, to determine how to incorporate it into the Windows operating system, a couple of strange things stood out. First, the random number generator was very slow – two to three orders of magnitude slower than another algorithm in the standard.
Second, it didn’t seem to be very secure.
“There was a property [in it] that seemed to make the prediction-resistance of the algorithm not what you would necessarily want it to be,” the Microsoft manager says. In non-geek speak, there was a weakness that made the random number generator not so random.
Good random number generation is at the core of encryption, and a weak RNG can undo the entire encryption system. Random number generators play a role in creating cryptographic keys, in opening secure communications between users and web sites and in resetting passwords for email accounts. Without assured randomness, an attacker can predict what the system will generate and undermine the algorithm.
Shumow and Ferguson found that the obstacles to predicting what the random number generator would generate were low. It wasn’t a catastrophic problem, but it seemed strange for a security system being promulgated by the government.
Then they noticed something else.
The standard, which contained guidelines for implementing the algorithm, included a list of constants – static numbers – that were used in the elliptic curve on which the random number generator was based. Whoever generated the constants, which served as a kind of public key for the algorithm, could have generated a second set of numbers at the same time – a private key.
Anyone possessing that second set of numbers would have what’s known in the cryptography community as “trapdoor information” – that is, they would be able to essentially unlock the encryption algorithm by predicting what the random number generator generated. And, Shumow and Ferguson realized, they could predict this after seeing as few as 32 bytes of output from the generator. With a very small sample, they could crack the entire encryption system used to secure the output.
“Even if no one knows the secret numbers, the fact that the backdoor is present makes Dual_EC_DRBG very fragile,” cryptographer wrote at the time, in a piece for WIRED. “If someone were to solve just one instance of the algorithm’s elliptic-curve problem, he would effectively have the keys to the kingdom. He could then use it for whatever nefarious purpose he wanted. Or he could publish his result, and render every implementation of the random-number generator completely insecure.”
No one knew who had produced the constants, but it was assumed that because the NSA had pushed the algorithm into the standard, the agency had generated the numbers. The spy agency might also, then, have generated a secret key.
Schneier called it “scary stuff indeed,” but he also said at the time that it made no sense as a backdoor, since it was so obvious to anyone who looked at the algorithm and standard that there was this flaw in it. As a result, developers of web sites and software applications wouldn’t use it to help secure their products and systems, he said.
But in fact, many developers did use it.
The U.S. government has enormous purchasing power, and vendors soon were forced to implement the suspect standard as a condition of selling their products to federal agencies under so-called FIPS certification requirements. Microsoft added support for the standard, including the elliptic curve random-number generator, in a Vista update in February 2008, though it did not make the problematic generator the default algorithm.
Asked why Microsoft supported the algorithm when two of its own employees had shown it to be weakened, a second Microsoft senior manager who spoke with WIRED said that while the weakness in the algorithm and standard was “weird” it “wasn’t a smoking gun.” It was more of an “odd property.”
Microsoft decided to include the algorithm in its operating system because a major customer was asking for it, because it had been sanctioned by NIST, and because it wasn’t going to be enabled as the default algorithm in the system, thus having no impact on other customers.
“In fact it is nearly impossible for any user to implement or to get this particular random number generator instantiating on their machines without going into the guts of the machine and reconfiguring it,” he says.
Other major companies, like Cisco and RSA, added it as well. NIST in fact provides a lengthy list of companies that have included it in their libraries, though the list doesn’t say which companies made it the default algorithm in their library or which products have been developed that invoke the algorithm.

A Cisco spokesman told WIRED that the algorithm was implemented in its standard crypto library around mid-2012, a library that is used in more than 120 product lines, but the algorithm is not the default, and the default algorithm cannot be changed by users. The company is currently completing an internal audit of all of its products that leverage the NIST standard.
RSA, however, made the algorithm the default in its BShare toolkit for Java and C developers until this week when it told WIRED that it was changing the default following the renewed controversy over it. The company sent an advisory to developer customers “strongly” urging them to change the default to one of a number of other random number generator algorithms RSA supports. RSA also changed the default on its own end in BSafe and in an RSA key management system. The company is currently doing an internal review of all of its products to see where the algorithm gets invoked in order to change those.
RSA actually added the algorithm to its libraries in 2004 or 2005, before NIST approved it for the standard in 2006 and before the government made it a requirement for FIPS certification, says Sam Curry, the company’s chief technology officer. The company then made it the default algorithm in BSafe and in its key management system after the algorithm was added to the standard. Curry said that elliptic curve algorithms were all the rage at the time and RSA chose it as the default because it provided certain advantages over the other random number generators, including what he says was better security.
“Cryptography is a changing field. Some algorithms go up and some come down and we make the best decisions we can in any point in time,” he says.”A lot of the hash-based algorithms were getting struck down by some weaknesses in how they chose numbers and in fact what kind of sample set they chose for initial seeding. From our perspective it looked like elliptic curve would be immune to those things.”
Curry says the fact that the algorithm is slower actually provides it with better security in at least one respect.
“The length of time that you have to gather samples will determine the strength of your random number generation. So the fact that it’s slower sometimes gives it a wider sample set to do initial seeding,” he says. “Precisely because it takes a little longer, it actually winds up giving you more randomness in your initial seeding, and that can be an advantage.”
Despite the renewed controversy over the algorithm and standard, Microsoft managers say they still don’t think the weaknesses constitute an intentional backdoor.
Callas agrees. He thinks it is simply bad cryptography that was included in the standard to round-out the selection so that there would be at least one elliptic curve algorithm in the standard.
But one advantage to having the algorithm supported in products like Vista — and which may be the reason the NSA pushed it into the standard — is that even if it’s not the default algorithm for encryption on a system, as long as it’s an option on the system, an intruder, like the NSA, can get into the system and change the registry to make it the default algorithm used for encryption, thereby theoretically making it easy for the NSA to undermine the encryption and spy on users of the machine.
Schneier says this is a much more efficient and stealth way of undermining the encryption than simply installing a keystroke logger or other Trojan malware that could be detected.
“A Trojan is really, really big. You can’t say that was a mistake. It’s a massive piece of code collecting keystrokes,” he said. “But changing a bit-one to a bit-two [in the registry to change the default random number generator on the machine] is probably going to be undetected. It is a low conspiracy, highly deniable way of getting a backdoor. So there’s a benefit to getting it into the library and into the product.”
To date, the only confirmation that the algorithm has a backdoor comes in the Times story, based on NSA documents leaked by Edward Snowden, which the Times and two other media outlets saw.
“[I]nternal memos leaked by a former NSA contractor, Edward Snowden, suggest that the NSA generated one of the random number generators used in a 2006 NIST standard — called the Dual EC DRBG standard — which contains a back door for the NSA,” the Times wrote.
An editorial published by the Times this weekend re-asserted the claim: “Unbeknown to the many users of the system, a different government arm, the National Security Agency, secretly inserted a ‘back door’ into the system that allowed federal spies to crack open any data that was encoded using its technology.”
But all of the quotes that the Times published from the memos refer to the NSA getting the standard passed by an international standards body; they do not say the NSA intentionally weakened the algorithm and standard, though the Times implies that this is what the memos mean by tying them to the 2007 presentation by Shumow and Ferguson.
NIST has denied any knowledge of a backdoor and has also denied that the NSA authored its standard. The institute has, however, re-opened the standard for public comment as a result of the controversy and “strongly” urged against using the algorithm in question until the matter could be resolved. The public comments period will close Nov. 6.
Even without more explicit confirmation that the weaknesses in the algorithm and standard constitute a backdoor, Kocher and Schneier believe they do.
“It is extraordinarily bad cryptography,” says Kocher. “If you look at the NSA’s role in creating standards [over the years] and its general cryptographic sophistication, none of it makes sense if there isn’t a backdoor in this.”
Schneier agrees and says the NSA has done too many other things for him to think, when he sees government-mandated crypto that’s weak, that it’s just by accident.
“If we were living in a kinder world, that would be a plausible explanation,” he says. “But we’re living in a very malicious world, it turns out.”
He adds that the uncertainty around the algorithm and standard is the worst part of the whole matter.
“This is the worst problem that the NSA has done,” Schneier says. “They have so undermined the fundamental trust in the internet, that we don’t know what to trust. We have to suspect everything. We’re never sure. That’s the greatest damage.”

Tuesday, September 24, 2013

NSA Job Opening

The NSA is looking for a Civil Liberties & Privacy Officer. It appears to be an internal posting.
The NSA Civil Liberties & Privacy Officer (CLPO) is conceived as a completely new role, combining the separate responsibilities of NSA's existing Civil Liberties and Privacy (CL/P) protection programs under a single official. The CLPO will serve as the primary advisor to the Director of NSA for ensuring that privacy is protected and civil liberties are maintained by all of NSA's missions, programs, policies and technologies. This new position is focused on the future, designed to directly enhance decision making and to ensure that CL/P protections continue to be baked into NSA's future operations, technologies, tradecraft, and policies. The NSA CLPO will consult regularly with the Office of the Director of National Intelligence CLPO, privacy and civil liberties officials from the Department of Defense and the Department of Justice, as well as other U.S. government, private sector, public advocacy groups and foreign partners.

Monday, September 23, 2013

Metadata Equals Surveillance

Back in June, when the contents of Edward Snowden's cache of NSA documents were just starting to be revealed and we learned about the NSA collecting phone metadata of every American, many people -- including President Obama -- discounted the seriousness of the NSA's actions by saying that it's just metadata.
Lots and lots of people effectively demolished that trivialization, but the arguments are generally subtle and hard to convey quickly and simply. I have a more compact argument: metadata equals surveillance.
Imagine you hired a detective to eavesdrop on someone. He might plant a bug in their office. He might tap their phone. He might open their mail. The result would be the details of that person's communications. That's the "data."
Now imagine you hired that same detective to surveil that person. The result would be details of what he did: where he went, who he talked to, what he looked at, what he purchased -- how he spent his day. That's all metadata.
When the government collects metadata on people, the government puts them under surveillance. When the government collects metadata on the entire country, they put everyone under surveillance. When Google does it, they do the same thing. Metadata equals surveillance; it's that simple.

Sunday, September 22, 2013

Virus Types:

What is a Computer Virus ? A potentially damaging computer programme capable of reproducing itself causing great harm to files or other programs without permission or knowledge of the user.
Virus - A program that when run, has the ability to self-replicate by infecting other programs and files on your computer. These programs can have many effects ranging from wiping your hard drive, displaying a joke in a small box, or doing nothing at all except to replicate itself. These types of infections tend to be localized to your computer and not have the ability to spread to another computer on their own. The word virus has incorrectly become a general term that encompasses trojans, worms, and viruses.
Types of viruses :-
The different types of viruses are as follows-
1) Boot Sector Virus :- Boot sector viruses infect either the master boot record of the hard disk or the floppy drive. The boot record program responsible for the booting of operating system is replaced by the virus. The virus either copies the master boot program to another part of the hard disk or overwrites it. They infect a computer when it boots up or when it accesses the infected floppy disk in the floppy drive. i.e. Once a system is infected with a boot-sector virus, any non-write-protected disk accessed by this system will become infected.

Examples of boot- sector viruses are Michelangelo and Stoned.

2) File or Program Viruses :-Some files/programs, when executed, load the virus in the memory and perform predefined functions to infect the system. They infect program files with extensions like .EXE, .COM, .BIN, .DRV and .SYS .

Some common file viruses are Sunday, Cascade.

3) Multipartite Viruses :-A multipartite virus is a computer virus that infects multiple different target platforms, and remains recursively infective in each target. It attempts to attack both the boot sector and the executable, or programs, files at the same time. When the virus attaches to the boot sector, it will in turn affect the system’s files, and when the virus attaches to the files, it will in turn infect the boot sector.
This type of virus can re-infect a system over and over again if all parts of the virus are not eradicated.
Ghostball was the first multipartite virus, discovered by Fridrik Skulason in October 1989.
Other examples are Invader, Flip, etc.

4) Stealth Viruses :-These viruses are stealthy in nature means it uses various methods for hiding themselves to avoid detection. They sometimes remove themselves from the memory temporarily to avoid detection by antivirus. They are somewhat difficult to detect. When an antivirus program tries to detect the virus, the stealth virus feeds the antivirus program a clean image of the file or boot sector.

5) Polymorphic Viruses :-
Polymorphic viruses have the ability to mutate implying that they change the viral code known as the signature each time they spread or infect. Thus an antivirus program which is scanning for specific virus codes unable to detect it's presense.

6) Macro Viruses :- A macro virus is a computer virus that "infects" a Microsoft Word or similar application and causes a sequence of actions to be performed automatically when the application is started or something else triggers it. Macro viruses tend to be surprising but relatively harmless.A macro virus is often spread as an e-mail virus. Well-known examples are Concept Virus and Melissa Worm.

If you use a computer, read the newspaper, or watch the news, you will know about computer viruses or other malware. These are those malicious programs that once they infect your machine will start causing havoc on your computer. What many people do not know is that there are many different types of infections that are categorized in the general category of Malware.

Malware - Malware is programming or files that are developed for the purpose of doing harm. Thus, malware includes computer viruses, worms, Trojan horses, spyware, hijackers, and certain type of adware.
This article will focus on those malware that are considered viruses, trojans, worms, and viruses, though this information can be used to remove the other types of malware as well. We will not go into specific details about any one particular infection, but rather provide a broad overview of how these infections can be removed. For the most part these instructions should allow you to remove a good deal of infections, but there are some that need special steps to be removed and these won't be covered under this tutorial.
Before we continue it is important to understand the generic malware terms that you will be reading about.
Backdoor- A program that allows a remote user to execute commands and tasks on your computer without your permission. These types of programs are typically used to launch attacks on other computers, distribute copyrighted software or media, or hack other computers.
Hijackers- A program that attempts to hijack certain Internet functions like redirecting your start page to the hijacker's own start page, redirecting search queries to a undesired search engine, or replace search results from popular search engines with their own information.
Spyware- A program that monitors your activity or information on your computer and sends that information to a remote computer without your Knowledge.
Adware- A program that generates popups on your computer or displays advertisements. It is important to note that not all adware programs are necessarily considered malware.
There are many legitimate programs that are given for free that display ads in their programs in order to generate revenue. As long as this information is provided up front then they are generally not considered malware.
Dialler - A program that typically dials a premium rate number that has per minute charges over and above the typical call charge. These calls are with the intent of gaining access to pornographic material.
Trojan- A program that has been designed to appear innocent but has been intentionally designed to cause some malicious activity or to provide a backdoor to your system.
Worm- A program that when run, has the ability to spread to other computers on its own using either mass-mailing techniques to email addresses found on your computer or by using the Internet to infect a remote computer using known security holes.
Intrusion Detection System (IDS):

A system that tries to identify attempts to hack or break into a computer system or to misuse it. IDSs may monitor packets passing over the network, monitor system files, monitor log files, or set up deception systems that attempt to trap hackers.

Computer systems have become more vulnerable to intrusions than ever. Intrusion Detection is a security technology that allows not only the detection of attacks, but
 also attempts to provide notification of new attacks unforeseen by other
 components. Intrusion detection is an important component of a security system,
and it complements other security technologies.

>>How does an IDS work?

While there are several types of IDSs, the most common types work the same.
They analyze network traffic and log files for certain patterns. What kind of
patterns you may ask? While a firewall will continually block a hacker from connecting to a network, most firewalls never alert an administrator.

The administrator may notice if he/she checks the access log of the firewall, but
that could be weeks or even months after the attack. This is where an IDS comes
into play. The attempts to pass through the firewall are logged, and IDS will analyze its log. At some point in the log there will be a large number of request-reject
entries. An IDS will flag the events and alert an administrator. The administrator
can then see what is happening right after or even while the attacks are still taking place. This gives an administrator the advantage of being able to analyze the techniques being used, source of attacks, and methods used by the hacker.

>>Following are the types of intrusion detection systems :-

1)Host-Based Intrusion Detection System (HIDS): Host-based intrusion detection
systems or HIDS are installed as agents on a host. These intrusion detection systems can look into system and application log files to detect any intruder activity.

2)Network-Based Intrusion Detection System (NIDS): These IDSs detect attacks by capturing and analyzing network packets. Listening on a network segment or
switch, one network-based IDS can monitor the network traffic affecting multiple
hosts that are connected to the network segment, thereby protecting those hosts. Network-based IDSs often consist of a set of single-purpose sensors or hosts placed
at various points in a network. These units monitor network traffic, performing local analysis of that traffic and reporting attacks to a central management console.

 >>Some important topics comes under intrusion detection are as follows :-

1)Signatures: Signature is the pattern that you look for inside a data packet. A signature is used to detect one or multiple types of attacks. For example, the
presence of “scripts/iisadmin” in a packet going to your web server may indicate
an intruder activity. Signatures may be present in different parts of a data packet depending upon the nature of the attack.

2)Alerts: Alerts are any sort of user notification of an intruder activity. When an IDS detects an intruder, it has to inform security administrator about this using alerts.
Alerts may be in the form of pop-up windows, logging to a console, sending e-mail and so on. Alerts are also stored in log files or databases where they can be viewed later on by security experts.

3)Logs: The log messages are usually saved in file.Log messages can be saved
either in text or binary format.

4)False Alarms: False alarms are alerts generated due to an indication that is not
an intruder activity. For example, misconfigured internal hosts may sometimes broadcast messages that trigger a rule resulting in generation of a false alert.
Some routers, like Linksys home routers, generate lots of UPnP related alerts. To
avoid false alarms, you have to modify and tune different default rules. In some
cases you may need to disable some of the rules to avoid false alarms.

5)Sensor: The machine on which an intrusion detection system is running is also called the sensor in the literature because it is used to “sense” the network.


Snort is a very flexible network intrusion detection system that has a large set of pre-configured rules. Snort also allows you to write your own rule set. There are several mailing lists on the internet where people share new snort rules that can counter the latest attacks.

Snort is a modern security application that can perform the following three functions :

* It can serve as a packet sniffer.
* It can work as a packet logger.

* It can work as a Network-Based Intrusion Detection System (NIDS).

Smooth-Sec 3.0 Intrusion Detection System

Smooth-Sec is a lightweight and fully-ready IDS/IPS (Intrusion Detection/Prevention System) Linux distribution based on Debian 7 (wheezy), available for 32 and 64 bit architecture. The distribution includes the latest version of Snorby, Snort, Suricata, PulledPork and Pigsty. An easy setup process allows to deploy a complete IDS/IPS System within minutes, even for security beginners with minimal Linux experience.
  •     Debian 7 Wheezy based
  •     32 and 64 bit iso available. Snorby V 2.6.2
  •     Snort V
  •     Suricata V 1.4.3
  •     Pigsty V 0.1.0
  •     PulledPork V 0.6.1


32-Bit – smoothsec-3.0-i386.iso
64-Bit – smoothsec-3.0-amd64.iso

Saturday, September 21, 2013

Payment terminals allow for remote PIN capture and card cloning

Plastic cards are an increasingly popular means of payment all over the world. Payment credentials come in different flavors ranging from credit cards of globally operating brands (Visa, Mastercard, AmEx), to national payment schemes (i.e., German EC cards) and store-issued gift cards. The underlying technologies range from magnetic stripes, to microprocessors (EMV, Chip&PIN) and virtual cards stored as software on smartphones (PayPass, payWave).
All card flavors and technologies have in common that they regularly interact with point-of-sale payment terminals, today’s Achilles Heel of cash-less payment.
Hacking Payment Terminals
An analysis of the most widely deployed payment terminal in Germany found serious weaknesses.
A. Remote exploitation. The device’s network stack contains buffer overflows that can be used to execute code at system level.
B. Local compromise. There are at least two interfaces over which the device can be exploited locally:
  1. Serial. Some versions of the terminal software are vulnerable to a buffer overflow that gains code execution through the readily accessible serial interface.
  2. JTAG. The JTAG interface of the application processor is accessible without opening the device. It allows full debugging control over the device.
These attacks target the terminal’s application processor. The security of the cryptographic module (HSM) has not yet been investigated as far as key extraction attacks are concerned. However, a design or implementation shortcoming in the HSM enables control over display and PIN pad from the application processor.
Abuse scenarios
Once exploited, the terminal under the control of an adversary can be used for fraud:
  • Card cloning. Collect credit/EC card data and PIN numbers
  • Alter transactions. Change transaction –  including EMV transactions – in type (debit vs. credit), value, or other fields
  • Fake transactions. Spoof transactions towards the payment back-end or the cash register (i.e., falsely signal a successful transaction)
Software vulnerabilities demand software patches. Fortunately, the payment terminals are patchable, often remotely by their connected payment back-ends. This post will be updated with information on which software versions mitigate the software attacks described herein.
Hardware-level vulnerabilities are harder to mitigate. The device’s application processor, for instance, does not provide configuration settings for JTAG to be switched off. Deployed devices will likely stay vulnerable to local attacks, potentially undermining trust in cash-less payment considerably for a long time. Unfortunately, the world-wide payment infrastructure’s planned updates to EMV do not protect from compromised terminals adding one more bit of concern about the EMV standard that others have criticized for its protocol imperfections (PDF, PDF).

How to Install Encfs and Encrypt Files in Mac OS X

Encfs is an open-source software that is widely used to create encrypted filesystem. It is particularly useful for encrypting files that you want to store in the cloud, such as Dropbox or Google Drive. Using encfs in Linux and Windows is pretty straightforward as there are installers for both platforms. However, for Mac OS X, installing encfs is not as easy as it should be. Here is how you can install and use encfs in Mac OS X.

Installing encfs in Mac OS X

There are a number of ways to install encfs in Mac OS X. In this tutorial, we will show you the homebrew and OSXfuse installation method. It will require the use of terminal, so be prepared to get your hands dirty.
Homebrew is a useful package manager for Mac OS X. It allows you to install plenty of applications with a single command.
1. Open Terminal in Mac OS X. If you are not aware of where it is located, you can find it at “Applications -> Utilities -> Terminal”. Type the command:
ruby -e "$(curl -fsSL"
This will install Homebrew in your system. If you have already installed Homebrew, you can ignore this step.
2. Go to the Fuse for OS X site and download the osxfuse.dmg file. Install osxfuse in your system.
3. Next, download the encfs.macosx package file to your Desktop and extract it.
In the terminal, “cd” to the encfs.macosx folder and execute the installer script.
cd ~/Desktop/encfs.macosx-master
This script will download the encfs copy from the web, install it with Homebrew and patch it to make it compatible with osxfuse.
Once the installation is completed, you will have encfs running in your system.

Using encfs in Mac OS X

Still in the terminal, you can create an encrypted filesystem in Dropbox using this command:
encfs ~/Dropbox/Private ~/Private

 Go through the setup process. You will be prompted to enter a master password. Make sure you use a strong password and remember it. Once completed, you should see a “Private” folder in your Home directory. Any file you place in this folder will be encrypted and stored in the “Dropbox -> Private” folder.

Automount encrypted directory on startup

It is very troublesome if you have to mount the encrypted directory everytime you login. Use the steps below to automount the encrypted directory when you login.
1. Open “Keychain Access”. We will be adding your encfs master password to Keychain so the script can automount the encrypted directory without prompting you for password. Add a new entry. Enter “encfs” for both the “Item name” and “Account Name” field. Once you have added the password, you can close Keychain Access.
2. Open a text editor and copy the following text into it.
security find-generic-password -ga encfs 2>&1 >/dev/null | cut -d'"' -f2 | "$ENCFS" -S "$ENCDIR" "$DECDIR"
Save the file as “” in your User directory.
Make the script executable:
chmod a+x
3. Next, open AppleScript editor and paste the line:
do shell script "$HOME/"
and save it as an application in your User directory.
4. Lastly, go to “System Preferences -> Users & Groups”, click on your User account and select “Login items”. Add the encfslogin application to the startup list.
Log out and login again. Your encrypted directory should be auto-mounted now.


How Bacteria Terraform a Squid:

The bacterium Vibrio fischeri is a squid terraformer. Although it can live independently in seawater, it also colonises the body of the adorable Hawaiian bobtail squid. The squid nourishes the bacteria with nutrients and the bacteria, in turn, act as an invisibility cloak. They produce a dim light that matches the moonlight shining down from above, masking the squid's silhouette from predators watching from below. With its light-emitting microbes, the squid becomes less visible. Margaret McFall-Ngai from the University of Wisconsin has been studying this partnership for almost 25 years and her team, led by postdoc Natacha Kremer, have now uncovered its very first moments. They've shown how the incoming bacteria activate the squid's genes to create a world that's more suitable for their kind. And remarkably, it takes just five of these microbial pioneers to start the terraforming (teuthoforming?) process.
As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Legally Justifying NSA Surveillance of Americans

 Just to challenge ourselves, we'll ignore the several statutory provisions and other doctrines that allow for spying without court oversight, such as urgent collection, gathering information not considered protected by the Fourth Amendment, the wartime spying provision, or the president's "inherent authority" for warrantless spying. Let's also ignore the fact that we have general wiretaps ala the Verizon order on phone metadata and Internet traffic that we can fish through in secret. Let's actually try to get this by the FISA Court under 50 U.S.C. §§ 1801-1805 for electronic surveillance or § 1861 for documents and records.

 The NSA's Spying Powers: Reading the Statute

In the midst of confusion over the NSA's spying powers, even members of Congress who voted for the applicable laws claim surprise at how they are playing out in practice. With defenders of spying saying to “read the statute” to understand its privacy protections, I thought I'd do just that.
Say I'm the NSA and I want to legally justify a court order giving me access to private emails of Occupy activists (so I can join in the FBI and DHS surveillance of peaceful protesters, for example). It's a domestic political movement, so that sounds as if it should be pretty hard, right? Let's see...
Just to challenge ourselves, we'll ignore the several statutory provisions and other doctrines that allow for spying without court oversight, such as urgent collection, gathering information not considered protected by the Fourth Amendment, the wartime spying provision, or the president's "inherent authority" for warrantless spying. Let's also ignore the fact that we have general wiretaps ala the Verizon order on phone metadata and Internet traffic that we can fish through in secret. Let's actually try to get this by the FISA Court under 50 U.S.C. §§ 1801-1805 for electronic surveillance or § 1861 for documents and records.
First Hurdle: I need "probable cause" to believe the "target" is a "foreign power" or "agent of a foreign power." This is great - I don't need probable cause of any crime, just something relating to the identity of the "target." And if the "target" of my investigation meets those criteria, I can slurp up all sorts of data about US people, subject only to toothless "minimization" requirements I'll discuss in step 2. To obtain stored records such as emails, it's even easier. The court is instructed to presume that I am entitled to an order to get those records if I can just show "reasonable grounds to believe" the records are "relevant to" investigating a foreign power or an agent of a foreign power or someone "known to" a suspected agent of a foreign power.
So, can I consider "Occupy" itself to be a foreign power? Believe it or not, any foreign-based political organization qualifies unless it is "substantially composed" of US persons. So all the Occupy branches in other parts of the world, and their agents, are valid "targets" for surveillance (as well as AdBusters, the Canadian organization that first called for an occupation of Wall Street). That's a great start. I bet a lot of the domestic Occupiers are within one or two links of a person directly communicating with a "foreign power" or one of their "agents," so I'll ask for their communications as part of my "targeting" the foreigners. Actually, some of the foreign-run banks and corporations they're protesting might qualify as valid foreign targets, too. I'll "target" them... by reading emails of people talking about their actions, and maybe their private intelligence about the protesters.
Second Hurdle: I'm going to have to "minimize" the data I collect about US persons. But wait! I don't have to minimize if it's evidence of a "crime" that has been or might be committed. All of Occupy's civil disobedience organizing is fair game for surveillance. I bet I can find evidence of drug crimes in here, too, and who knows what else? That'll give the state some leverage in case these uppity protesters get out of hand. I also have the general "national security" and "foreign affairs" exceptions to minimization, which might help if the protesters plan to demonstrate at sites relevant to national security or at diplomatic summits. Of course, the court can require me to minimize even in those circumstances, but they don't have to, and no one will ever know one way or the other. Besides, the secret "minimization" procedures may sound good to laypeople, but anyone who follows privacy research knows that it's really easy to re-identify people from anonymized records if you have other databases to correlate data against, and boy do we ever!
Third Hurdle: Maybe tech companies won't like it. But I have a court order, and I already beat Yahoo in court, so there's nothing they can do, and I'll pay them well for their time. They've already built me these nice PRISM systems to streamline the data acquisition process for me, so let's get spying!
Fourth Hurdle: Some Senators are whining about the invasive spying. Solution: Send in Director of National Intelligence James Clapper to lie to our Congressional overseers about what we do.
The most common form of lying that has been exposed is giving specialized meanings to English words that do not match their common meanings, then using those words misleadingly. The Electronic Frontier Foundation has summarized many such word games, and above I discussed some of the ambiguities in the "targeting" and "minimization" terms. I didn't even have to break the letter of the law today to spy on these domestic political activists. (Breaking the law is for tomorrow, after the companies have handed over the data and there's no chance I'll ever have to justify myself in court, even one as favorable as the FISA Court.)
--- That's it. I spent just an hour and a half cooking up this analysis, while the intelligence/law enforcement apparatus has teams of lawyers who consider it part of their job to justify expansive surveillance and who have been doing this for years. The FISA Court has the power to reject the broad interpretations of statutory authority and close the “minimization” loopholes I outlined above. Given what we've seen in recent leaks, though, that doesn't seem to be the court's approach.

Friday, September 20, 2013

Google knows nearly every Wi-Fi password in the world

If an Android device (phone or tablet) has ever logged on to a particular Wi-Fi network, then Google probably knows the Wi-Fi password. Considering how many Android devices there are, it is likely that Google can access most Wi-Fi passwords worldwide.
Recently IDC reported that 187 million Android phones were shipped in the second quarter of this year. That multiplies out to 748 million phones in 2013, a figure that does not include Android tablets.
Many (probably most) of these Android phones and tablets are phoning home to Google, backing up Wi-Fi passwords along with other assorted settings. And, although they have never said so directly, it is obvious that Google can read the passwords.
Sounds like a James Bond movie.
Android devices have defaulted to coughing up Wi-Fi passwords since version 2.2. And, since the feature is presented as a good thing, most people wouldn't change it. I suspect that many Android users have never even seen the configuration option controlling this. After all, there are dozens and dozens of system settings to configure.
And, anyone who does run across the setting can not hope to understand the privacy implication. I certainly did not.
  • In Android 2.3.4, go to Settings, then Privacy. On an HTC device, the option that gives Google your Wi-Fi password is "Back up my settings". On a Samsung device, the option is called "Back up my data". The only description is "Back up current settings and application data". No mention is made of Wi-Fi passwords. 
  • In Android 4.2, go to Settings, then "Backup and reset". The option is called "Back up my data". The description says "Back up application data, Wi-Fi passwords, and other settings to Google servers". 
Needless to say "settings" and "application data" are vague terms. A longer explanation of this backup feature in Android 2.3.4 can be found in the Users Guide on page 374:
Check to back up some of your personal data to Google servers, with your Google Account. If you replace your phone, you can restore the data you’ve backed up, the first time you sign in with your Google Account. If you check this option, a wide variety of you personal data is backed up, including your Wi-Fi passwords, Browser bookmarks, a list of the applications you’ve installed, the words you’ve added to the dictionary used by the onscreen keyboard, and most of the settings that you configure with the Settings application. Some third-party applications may also take advantage of this feature, so you can restore your data if you reinstall an application. If you uncheck this option, you stop backing up your data to your account, and any existing backups are deleted from Google servers.
A longer explanation for Android 4.0 can be found on page 97 of the Galaxy Nexus phone users Guide:
If you check this option, a wide variety of your personal data is backed up automatically, including your Wi-Fi passwords, Browser bookmarks, a list of the apps you've installed from the Market app, the words you've added to the dictionary used by the onscreen keyboard, and most of your customized settings. Some third-party apps may also take advantage of this feature, so you can restore your data if you reinstall an app. If you uncheck this option, your data stops getting backed up, and any existing backups are deleted from Google servers.
Sounds great. Backing up your data/settings makes moving to a new Android device much easier. It lets Google configure your new Android device very much like your old one.
What is not said, is that Google can read the Wi-Fi passwords.
And, if you are reading this and thinking about one Wi-Fi network, be aware that Android devices remember the passwords to every Wi-Fi network they have logged on to. The Register writes
The list of Wi-Fi networks and passwords stored on a device is likely to extend far beyond a user's home, and include hotels, shops, libraries, friends' houses, offices and all manner of other places. Adding this information to the extensive maps of Wi-Fi access points built up over years by Google and others, and suddenly fandroids face a greater risk to their privacy if this data is scrutinised by outside agents.
The good news is that Android owners can opt out just by turning off the checkbox.
Update: Sept 15, 2013: Even if Google deletes every copy of your backed up data, they may already have been compelled to share it with others. And, Google will continue to have a copy of the password until every Android device that has ever connected to the network turns off the backing up of settings/data. 
The bad news is that, like any American company, Google can be compelled by agencies of the U.S. government to silently spill the beans.
When it comes to Wi-Fi, the NSA, CIA and FBI may not need hackers and cryptographers. They may not need to exploit WPS or UPnP. If Android devices are offering up your secrets, WPA2 encryption and a long random password offer no protection.
I doubt that Google wants to rat out their own customers. They may simply have no choice. What large public American company would? Just yesterday, Marissa Mayer, the CEO of Yahoo, said executives faced jail if they revealed government secrets. Lavabit felt there was a choice, but it was a single person operation.
This is not to pick on Google exclusively. After all, Dropbox can read the files you store with them. So too, can Microsoft read files stored in SkyDrive. And, although the Washington Post reported back in April that Apple’s iMessage encryption foils law enforcement, cryptographer Matthew Green did a simple experiment that showed that Apple can read your iMessages.
In fact, Green's experiment is pretty much the same one that shows that Google can read Wi-Fi passwords. He describes it:
First, lose your iPhone. Now change your password using Apple's iForgot service ... Now go to an Apple store and shell out a fortune buying a new phone. If you can recover your recent iMessages onto a new iPhone -- as I was able to do in an Apple store this afternoon -- then Apple isn't protecting your iMessages with your password or with a device key. Too bad.
Similarly, a brand new Android device can connect to Wi-Fi hotspots it is seeing for the very first time.
Back in June 2011, writing for TechRepublic, Donovan Colbert described stumbling across this on a new ASUS Eee PC Transformer tablet:
I purchased the machine late last night after work. I brought it home, set it up to charge overnight, and went to bed. This morning when I woke I put it in my bag and brought it to the office with me. I set up my Google account on the device, and then realized I had no network connection ...  I pulled out my Virgin Mobile Mi-Fi 2200 personal hotspot and turned it on. I searched around Honeycomb looking for the control panel to select the hotspot and enter the encryption key. To my surprise, I found that the Eee Pad had already found the Virgin hotspot, and successfully attached to it ... As I looked further into this puzzling situation, I noticed that not only was my Virgin Hotspot discovered and attached, but a list of other hotspots ... were also listed in the Eee Pad's hotspot list. The only conclusion that one can draw from this is obvious - Google is storing not only a list of what hotspots you have visited, but any private encryption keys necessary to connect to those hotspots ...
Micah Lee, staff technologist at the EFF, CTO of the Freedom of the Press Foundation and the maintainer of HTTPS Everywhere, blogged about the same situation back in July.
When you format an Android phone and set it up on first run, after you login to your Google account and restore your backup, it immediately connects to wifi using a saved password. There’s no sort of password hash that your Android phone could send your router to authenticate besides the password itself.
Google stores the passwords in a manner such that they can decrypt them, given only a Gmail address and password.
Shortly after Lee's blog, Ars Technica picked up on this (see Does NSA know your Wi-Fi password? Android backups may give it to them). A Google spokesperson responded to the Ars article with a prepared statement.
Our optional ‘Backup my data’ feature makes it easier to switch to a new Android device by using your Google Account and password to restore some of your previous settings. This helps you avoid the hassle of setting up a new device from scratch.  At any point, you can disable this feature, which will cause data to be erased. This data is encrypted in transit, accessible only when the user has an authenticated connection to Google and stored at Google data centers, which have strong protections against digital and physical attacks.
Sean Gallagher, who wrote the Ars article, added "The spokesperson could not speak to how ... the data was secured at rest."
Lee responded to this with:
... it’s great the backup/restore feature is optional. It’s great that if you turn it off Google will delete your data. It’s great that the data is encrypted in transit between the Android device and Google’s servers, so that eavesdroppers can’t pull your backup data off the wire. And it’s great they they have strong security, both digital and physical, at their data centers. However, Google’s statement doesn’t mention whether or not Google itself has access to the plaintext backup data (it does)... [The issue is] Not how easy it is for an attacker to get at this data, but how easy it is for an authorized Google employee to get at it as part of their job. This is important because if Google has access to this plaintext data, they can be compelled to give it to the US government.
Google danced around the issue of whether they can read the passwords because they don't want people like me writing blogs like this. Maybe this is why Apple, so often, says nothing.
Eventually Lee filed an official Android feature request, asking Google to offer backups that are stored in such a way that only the end user (you and I) can access the data. The request was filed about two months ago and has been ignored by Google.
I am not revealing anything new here. All this has been out in the public before. Below is a partial list of previous articles.
However, this story has, on the whole, flown under the radar. Most tech outlets didn't cover it (Ars Technica and The Register being exceptions) for reasons that escape me.
1) Google knows where you've been and they might be holding your encryption keys. June 21, 2011 by Donovan Colbert for TechRepublic. This is the first article I was able to find on the subject. Colbert was not happy, writing:
 ... my corporate office has a public, protected wireless access point. The idea that every Android device that connects with that access point shares our private corporate access key with Google is pretty unacceptable ... This isn't just a trivial concern. The fact that my company can easily lose control of their own proprietary WPA2 encryption keys just by allowing a user with an Android device to use our wireless network is significant. It illustrates a basic lack of understanding on the ethics of dealing with sensitive corporate and personal data on the behalf of the engineers, programmers and leadership at Google. Honestly, if there is any data that shouldn't be harvested, stored and synched automatically between devices, it is encryption keys, passcodes and passwords.
2) Storage of credentials on the company servers Google by Android smartphones  (translated from German). July 8, 2013. The University of Passau in Germany tells the university community to turn off Android backups because disclosing passwords to third parties is prohibited. They warn that submitting your password to any third party lets unauthorised people access University services under your identity. They also advise changing all passwords stored on Android devices.
3)  Use Android? You’re Probably Giving Google All Your Wifi Passwords  July 11, 2013 by Micah Lee. Where I first ran into this story.
4) Android and its password problems open doors for spies July 16, 2013 by The H Security in Germany. Excerpt:
Tests ... at heise Security showed that after resetting an Android phone to factory settings and then synchronising with a Google account, the device was immediately able to connect to a heise test network secured using WPA2. Anyone with access to a Google account therefore has access to its Wi-Fi passwords. Given that Google maintains a database of Wi-Fi networks throughout the world for positioning purposes, this is a cause for concern in itself, as the backup means that it also has the passwords for these networks. In view of Google's generosity in sharing data with the NSA, this now looks even more troubling ... European companies are unlikely to be keen on the idea of this backup service, activated by default, allowing US secret services to access their networks with little effort.
5)  Does NSA know your Wi-Fi password? Android backups may give it to them July 17, 2013 by Sean Gallagher for Ars Technica. This is the article referred to earlier. After this one story, Ars dropped the issue, which I find strange since they must have realized the implications.
6) Android backup sends unencrypted Wi-Fi passwords to Google July 18, 2013 by Zeljka  Zorz for
7) Would you tell Google your Wi-Fi password? You probably already did... July 18, 2013 by Paul Ducklin writing for the Sophos Naked Security blog. Ducklin writes
... the data is encrypted in transit, and Google (for all we know) probably stores it encrypted at the other end. But it's not encrypted in the sense of being inaccessible to anyone except you ...  Google can unilaterally recover the plaintext of your Wi-Fi passwords, precisely so it can return those passwords to you quickly and conveniently ...
8) Android Backups Could Expose Wi-Fi Passwords to NSA July 19, 2013 by Ben Weitzenkorn of TechNewsDaily. This same story also appeared at and
9) Despite Google’s statement, they still have access to your wifi passwords July 19, 2013 by Micah Lee on his personal blog. Lee rebuts the Google spokesperson response to the Ars Technica article.
10) Oi, Google, you ate all our Wi-Fi keys - don't let the spooks gobble them too July 23, 2013 by John Leyden for The Register. Leyden writes: "Privacy experts have urged Google to allow Android users' to encrypt their backups in the wake of the NSA PRISM surveillance flap."
11) Google: Keep Android Users' Secure Network Passwords Secure August 5, 2013 by Micah Lee and David Grant of the EFF. They write
Fixing the flaw is more complicated than it might seem. Android is an open source operating system developed by Google. Android Backup Service is a proprietary service offered by Google, which runs on Android. Anyone can write new code for Android, but only Google can change the Android Backup Service.
To conclude on a Defensive Computing note, those that need Wi-Fi at home should consider using a router offering a guest network.
Make sure that Android devices accessing the private network are not backing up settings to Google. This is not realistic for the guest network, but you can enable the guest network only when needed and then shut it down afterwards. Also, you can periodically change the password of the guest network without impacting your personal wireless devices.
At this point, everybody should probably change their Wi-Fi password.