The Question the NSA is Not Answering

The NSA is claiming that the data that they grabbed in essentially warrant-less Hoovering up of telephone and Internet metadata has helped in certain investigations.

I have no doubt that is probably true.

But that is not the right way to frame the problem.  The real issue is:  Did being able to data mine metadata for all Americans help solve the case better and faster than had they been required to seek specific probable cause warrants for data from specific people?

To make clear the distinction, let's suppose I were trying to justify stealing a copy of every book in Barnes & Noble.  I might be able to accurately say that those books helped me writing a good Napoleon paper for school.   But could I have achieved the same goal - writing a paper on Napoleon - by purchasing individual books as needed via legal shopping processes?  The answer is probably "yes."  Having all the books pre-stolen only contributed in that it saved me the hassle of going down to the store and finding a specific book I needed.

In the same way, I suspect that having this data base merely saved FBI and others the hassle of filling out some paperwork in each case.  I am not sure incremental success rates in a few cases is enough justification to rip up the Constitution, but I am sure that laziness is not.


  1. John Moore:

    Seriously? Getting the meta-data allows them to do all sorts of analysis to identify a target, while getting a warrant requires that the target already be identified.

    While this may be fine for most terrorists, the first time one carrying a loose Paki nuke (or Iranian nuke) crosses the border, it would be really sorta nice if the NSA could detect it.

  2. ErikTheRed:

    It also depends on what their definition of "terrorist attack" is. We know of 22 attempts in the US since 9/11. One was in New York and was stopped by alert citizens. One was the Boston Marathon bombing which they failed to detect or prevent, despite being handed information on the perpetrators on a silver platter (perhaps if they weren't drowning in data it might have been followed up on).

    The other twenty consisted of the FBI finding some mouthy malcontents, supplying them with weapons or explosives, and then goading them into some violent act - basically manufacturing "terrorism" in order to justify all of their own work. Until they come up with some really, really good proof that they're doing something worthwhile - and after all of the lying (err... "least untruthful" statements - a new low in governmental newspeak) - it can probably be assumed that these "prevented attacks" happened right next to the Weapons of Mass Destruction in Iraq.

  3. ErikTheRed:

    I'm going to go out on a limb here and suggest that maybe if they'd stop killing Pakistani civilians and children that hypothetical loose Pakistani nukes would be less of an issue. Just a random thought.

  4. Benjamin Cole:

    If you give a government the power to tax, it will abuse that power, and try to extend that power.

    If you give a government the okay for a standing and mobilized military, it will abuse that power and try to extend that power.

    If you give a government the right to spy on citizens, it will abuse that power, and try to extend that power.

    If you give a government the right to kill citizens without a trial---well, then the law is an ass, and so are you.

    The NSA says it thwarted 50 plots. But no sense of the scale, or development of those plots. Are we talking plots that would kill 100, 1,000 or 10,000? And really---since there have been almost no successful acts of terrorism since 9/11 on US soil, are we to assume the NSA is batting 1000? They catch all the plots? Does this not strain credulity?

    Add on: Remember, 30,000 Americans die in auto accidents every year, and 12,000 by non-terrorist gunshot homicides. Since 9/11, about half a million Americans have died in auto accidents and gunshots.

    But we accept automobiles and private ownership of guns. We pay a price for mobility and freedom. Happily so, too.

    So be it with terrorism. I do not want to give up any freedoms or comfort to fight terrorism. I do not think it is a serious threat. I think the airlines can take steps to make sure their airplanes are not commandeered, and indeed likely would be forced to by insurers. The private market would fix that problem.

    As for other kinds of terrorists, maybe we should start regulating the sale of cooking utensils. At least get photos and fingerprints of everyone who buys a pressure cooker.

  5. Gil:

    In Libertarian terms there's no real right to privacy - if someone over hears a conversation you make in public then they broke no rules. Then again some overestimate what "theirs". Some think they own their medical records when in usually it's the private property of the medical institution that they have made about you. Chances are your conservations over phone lines is the property of the phone providers and your emails you transmit over the internet are the property of the internet provider. One straightforward way to not need a warrant is to ask the property owner for permission to enter and search. If phone and internet companies agreed to share their information with the government then it's all nice and legal (unless they stated otherwise in your contract but how many of us just click "Agree" and continue?).

  6. John:

    Using your analogy, wouldn't that assume they knew to go to Barnes & Noble specifically for a book on Napoleon? What if they are looking for something more generic like calls being made to certain countries.

  7. GeneralKnowledge:

    You, sir, have absolutely no idea what you are talking about. My medical records are mine, my phone calls, my eMails, my letters and other communications as well.

    Bonus question: Let's assume you own a car and bring it into the dealership for a small repair job. Who owns the car: You or the mechanic (since the car is now in his garage)?

  8. Can You hear Me Now?:

    NSA did have a warrant. The larger question is why has the Supreme Court become so generous for lack of a better word in the interpretation of the Constitution. The Fourth Amendment has room for interpretation, but the First Amendment "make no law"is as definitive as can be yet we have things like McCain-Fiengold on the books.

  9. Can You hear Me Now?:

    Care to explain how meta-data pinpoints the bad guys other than the ones stupid enough to use the same phones again and again? If you are really afraid President Obama is going to shove a Hellfire missile up your butt, you have a bunch of prepaid cell phones that you use once and throw away. With cell phones country and area codes become meaningless. I can buy a bunch of prepaid cell phones in West Podunk and use them any where.

  10. NewEnglandDevil:

    MIT did a study recently that indicated that even with as few as four pieces of information in one year (e.g. four independent hits that give: date, time of day, cell tower location ping(s)) you can be individually identified. Then the number is associated with you. Then, any place that number moves to (cell tower pinging) becomes associated with you. (edit) as to your "single-use phone" theory, NSA is legitimately hoovering data and convos for calls made to overseas locales. Having one number overseas that receives multiple calls from many phones which are only used once is its own pattern that can be identified.

  11. NL7:

    They want all available information so they can create an algorithm based on people's behavior - communications, transactions, travel, reading habits. Since they expressly want to know all which is knowable, they could never meet any reasonable warrant standard.

  12. NL7:

    Better pay cash for those burners, Stringer, or the credit card records will haunt you.

    As data storage and bandwidth get ever cheaper, I won't be shocked if NSA or some agency is able to start collecting video streams from drug stores. And given that even Facebook is already pretty good at recognizing faces, we aren't terribly far off from NSA being able to recognize you as paying cash for your burners. And the gummint could always institute a rule to make vendors require a driver's license for every burner they sell, then to report the new phone's number and your DL to a database.

    At this point, it's hard to imagine up some data collection the government would not partake of, if it could.

  13. marque2:

    Yes and all because we refuse to control our borders to the point we issue visas to folks who openly proclaim hate of the US, don't bother searching when student visa holders from places of known terrorism go AWOL.

    Control who comes in and you won't have to survey everyone with this metadata "theft"

  14. marque2:

    Well if you read the document you sign at your doctors office, it pretty much limits anyone you would want to have your records from getting them but lets law enforcement have them all if they ask nicely.

    I once had a argument with a lawyer on this board about it. He called a dope - but then his reference material basically showed that it only prevents people you would want to have your info (your wife for instance) and any public official, if they want to do surveys, or collect metadata, or law enforcement can just walk up and get it.

    In the end the only thing that prevents the government from abusing our medical records - per HIPAA is a trust that the government will "do the right thing"

  15. marque2:

    Seems like the FBI and NSA were getting blanket warrants so they didn't have to look at folks specifically.

  16. marque2:

    I think everyone knows if you go to just about any retail store and you buy a non-contract phone, and refill card for cash, you can't really be traced. Yes you need to change phones every so often. But with this knowledge already out there, only the really stupid "terrorists" who probably couldn't harm us anyway will get caught.

  17. Nehemiah:

    Ever hear of a mechanic's lien?

  18. marque2:

    Yes they are looking at calls made by foreign entities - not just overseas calls - when they have a certainty of 51% that it is an overseas entity. Now that is a pretty good certainty level to protect our civil rights - about a flip of a coin.

  19. mlhouse:

    Big deal. IF someone wanted to go to the trouble of "identifying you" with such information they have wasted a lot of time and resources. And, what they have really identified is a "profile", not you.

  20. NewEnglandDevil:

    They haven't "wasted" people's time; perhaps they're wasting computer cycles or HD space; and whether or not it is a waste is in the eye of the beholder. You could use the data to track networking and see how people are associated. At this point, it may be moot with all the social media info we give away for free (also hoovered). However, this information could be used to identify personal habits that you wanted kept secret or private. Perhaps something illegal (we all break 3 laws per day, unknowingly), or something you're hiding from your spouse that could be used to blackmail you. What if information gleaned from this program was used to identify a non-public proclivity of a politician or justice? Sudden changes in position on an issue, or surprise judgements could be precipitated illigitimately by use of this type of information.

  21. mlhouse:

    People who make statements like yours only demonstrates you have no clue what is possible with the data. You watch too many movies were the all powerful government can find out all of this information (and actually wants to).

    For example, if someone truly wanted to know how people are "associating" there are much easier and more accurate ways of doing that then creating a model based on telephone metadata. Any analysis done with telephone metadata of this kind is not going to give very many meaningful conclusions about anything unique about an individual.

    If you think that it can be used to predict illegal activity or be used to determine you are hiding something from your spouse for purposes of blackmail, then you are simply ignorant.

  22. enjayboy:

    An unrelated question I'd to have answered by the NSA et al:
    Have any non-terrorist related suspects been identified through any of these programs?
    Drug trafficking? Money Laundering? Tax evasion?

  23. Gil:

    If someone finds information about you without trespassing or breaking and entering then how have you being violated? If you hold a conservation in public and others hear what you said how is what they heard "yours". Do you have a right to threaten people to forget what they heard because you don't want them to repeat it? As said how many have read the agreements they signed or clicked on and know for sure the information a company has will be kept confidential?

    On the other hand, Libertarian keep citing the 4th Amendment as though warrants are needed for everything when not all searches require a warrant by the U.S. Constitution (Wikipedia lists 8 exemptions). It's similar to when Libertarian think reading the Constitution makes the Fed illegal and we should all be on a gold standard when they haven't found a masterstroke flaw at all.

  24. NewEnglandDevil:

    Actually it's pretty clear what is possible with the data. I would have said something similar to you a few months ago; and then I learned about the IRS asking about membership in groups, family of members, associates of members, whether any of these members, their family or their associates were currently, previously, or might in the future be interested in public office; reading lists; contents of prayers; etc. and actively worked to restrict 1st amendment rights to speech. So, methinks thou dost protest too much.
    Furthermore, it isn't necessarily about determining anything specific in the aggregate. It's about identifying individuals of interest, and once identified, identifying patterns in their life (calls made, locations) that can then be used to direct resources efficiently to find out compromising information. I no longer need to follow someone 24 hours a day. I can account for their location and occupation 95% of the time. But what is it that they're doing once or twice a week around lunchtime off exit 17? Why are they calling an escort service? For men? When they're married to a woman? Or that specializes in 'under-age' service providers (Menendez, anyone?)?
    It's possible that this could be done. We now know for a fact that several different departments of the federal government have targeted conservatives for discriminatory treatment in their interactions with the federal government, in some cases, illegal treatment. And they demonstrated that they specifically DO WANT TO KNOW ABOUT YOUR ASSOCIATIONS. It's called oppo research. So it is also possible that this kind of meta-data could be used in very specific cases as an aid to additional research, to find non-public information that could be used to blackmail compliance. For instance, one of the answers in the recent hearings indicated that congresspersons information was also subject to this meta-data collection. This would also apply (in all likelihood) to all court justices.
    But your blanket assertions that this could never happen, and your assertion of authority and knowledge in excess of my own, have truly reassured me. I will *trust* my betters. I will *love* my betters. I will *serve* my betters. LOL.

  25. Gil:

    If what you say is true - the doctors office is quite clearly stating your medical records are their property alone - information they compiled from your visits. Thus if the office volunteers to share information with law enforcement officers then they don't need to get a warrant.

  26. mlhouse:

    Obviously, IRS data and phone metadata are not the same. A reasonable person would conclude that since the data is not the same what can be done analytically with the data might be just a bit different.

    The analysis of the IRS data was pretty simple: target any group that applied with the terms "Tea Party", "Constitution", etc, etc. This was an abuse of power because it was specific and targeted.

    The metadata is far from that. Unfortunately, you have zero understanding of the data and what can be done with it. Thus, your conspiracy theory brain cells are being activated.

  27. NewEnglandDevil:

    I took the liberty of going back and looking at some of your previous comments. You routinely assert that 4th amendment protections do not apply; however, there is a very specific reason for the language of the 4th amendment. The constitution does not authorize a general warrant to allow the government to obtain information about your person or your property for the specific reason, that the government is not authorized to hunt through an individual's life looking for occassion where they transgressed the law and thereby detain, arrest, or otherwise encumber an individual and their liberty.
    In a country which, at the federal level only, publishes 3,000 regulations a year, comprising approximately 82,000 pages, there is with certainty some act in the past which could be found to be illegal. Are you saying that you have no compassion for someone who is not in compliance with every rule and/or regulatory change, change of interpretation, or change in enforcement disposition of a prosecuting or regulatory authority? Because these acts are illegal and therefore that person must be a bad person? For using a black coating on their roof instead of a white coating? or because they flew a flag in their front yard? Or because it rained too much (and thus, their property produced a volumetric discharge that was in excess of permitted amounts)? Your premise is absurd.

  28. mlhouse:

    "that the government is not authorized to hunt through an individual's life looking for occassion where they transgressed the law and thereby detain, arrest, or otherwise encumber an individual and their liberty."

    They aren't. It is that simple. This is no further of a burden against "liberty" than a cop walking a beat or patrolling a neighborhood. Maybe in a simplistic world like yours were there is seemingly no negative human interaction, your perfect liberty model would be appropriate. But we do not live in such a world.

    AS for government regulations, it has no bearing. I support reasonable governmental regulations. My definition of "reasonable" may be different from yours, but in each case I evaluate them on their merits: are they effective, are they appropriate, does the cost/benefit of the regulation make sense, is it a legitimate function of government, is it the proper level of government.

  29. NewEnglandDevil:

    "They aren't. It is that simple."

    You don't know that. They could be. They did target conservatives in the EPA and IRS. In one case it is claimed that a party was targeted by the IRS, the EPA, DOJ, ATF and OSHA. We didn't know that the IRS was targeting conservatives and people just like you said, "They aren't. It is that simple." It doesn't seem that simple anymore. If they're going to collect this information, there should be a transparent audit as to how it is used. Say, every three months, who accessed the data. For what purpose. To what end. If the program was so successful and useful, why didn't they catch the Boston bombers? or the Ft. Hood shooter?

  30. NewEnglandDevil:

    LMAO. "You have zero understanding...." Yes all-knowing Carnac. Thou art great, clearly in thine own mind.

    The IRS data is different. Which is why they then asked for membership lists, family members, and other associations. All of these associations, which were requested by the IRS, could also be identified using phone meta-data. WOW - you don't say?

    Just because hoovering the data is unspecific and untargeted does not mean that the use of the data will be unspecific and untargeted. The IRS collected "meta-data" on all groups seeking tax emempt status. They then used that meta-data to identify and target what they deemed to be political opposition. Phone meta-data could be used to the same purpose.

  31. marque2:

    Many of those exemptions are holes poked into the 4th amendment by the liberal courts that should not have done so.

    I also think it bizarre that a phone call over a copper wire has a right to privacy expectation, but a cell phone or voip call doesn't. Seems like the Supreme court is being awfully generous giving away our rights. What we need is for congress to re-affirm our rights, but congress folk like the control they get, and love to tell is us it is to fight crime and or save the kids, - so they won't act to save us.

  32. marque2:

    Well that is nice of my doctor, isn't it. I thought the Supreme Court decided that what we do between us and our doctor falls under the right to privacy. Didn't we hear this in Roe vs Wade?

  33. mlhouse:

    "The IRS collected "meta-data" on all groups seeking tax emempt status."

    Here is a clue to your cluelessness. No they didn't. The data they collected was not "meta-data".

    Next, they did not use that "meta-data" to identify and target anything. THey already targeted the political opposition. The collection of data was secondary to their purposes, although they were obviously interested in donors. Instead, asking for most of the data was pure delay, harassment, and obfuscation.

  34. marque2:

    "For example, if someone truly wanted to know how people are "associating" there are much easier and more accurate ways of doing that then creating a model based on telephone metadata."

    Great point, so why are they collecting the meta data again?

  35. mlhouse:

    "Just because hoovering the data is unspecific and untargeted does not mean that the use of the data will be unspecific and untargeted. "

    Yes it does. You are pretending that data is some sort of all powerful thing. From decades of experience in developing analytical models using some of the most expensive and extensive databases in the world, the most sophisticated technology, and advanced statistical methods, I can very certainly tell you that the use of phone metadata of this type will give you very unspecific and very untargeted results.

    Most of the analytic value would be found in a very simple way: identifying phone numbers that called already identified phone numbers and then looking at relationships between the other telephone numbers called to/from the identified numbers.

    As I have stated previously in other discussion, once such a diffuse pattern is identified there are very legitimate questions as to what should be the next step. In some cases, such as those that involve certain corroborating evidence, further investigative activity should be warranted. What the standards should be is a legitimate debate. Clearly, if there is other corroborating evidence then more extensive investigation could be necessary. If not, further analytical work may be the only step, although my guess is that would be virtually impossible because it would be beyond the manpower capacity of any investigative agency.

  36. mlhouse:

    Two reasons.

    1. Terrorist involved in terroristic plans "associate" much differently than normal people.

    2. Such analysis could potentially degrade a terrorist group's ability to communicate.

  37. NewEnglandDevil:

    "Next, they did not use that "meta-data" to identify and target anything."
    Oh yes they did. They targeted people's businesses, their personal tax returns. They leaked personal data to groups opposed to the Tea Party (and farming). Someone makes a group. The GROUP is the meta data. The specifics are the membership and C- level or Board; with that knowledge, the persons and their businesses are targeted.

  38. NewEnglandDevil:

    "I can very certainly tell you that the use of phone metadata of this type will give you very unspecific and very untargeted results."
    Well, scientists at MIT say you're 100% wrong.

  39. mlhouse:

    LOL....again, you do not even know what you are talking about.

  40. mlhouse:

    Again, LOL. That is not "meta-data". That is DIRECT DATA. They did not have to arrive at this identification from statistical analysis of data. It was there.

    Your position is one of ignorance because you have no clue as to what the data is, what can be done with it, what methods would or could be used, and how effective such data is in making any types of real analysis. Because of your ignorance, you fill in the gaps with your own superstition and bias.

  41. NewEnglandDevil:

    And I've routinely demonstrated that your mere assertions are falacious (e.g. the gov't is interested in my associations, they are using databased information to persecute political dissent, it is happening in multiple agencies). You also routinely assert positions to me which I have not taken: "You are pretending that data is some sort of all powerful thing." No, in fact I am not. I am simply recognizing that it is a tool. One tool. Perhaps a crude tool, but with enough computing power and the ability to associate that data with data located in other gov't databases, it could be used effectively in any number of illegal, illicit ways.

  42. NewEnglandDevil:


    "Two reasons.

    1. Tea partiers involved in tea party plans "associate" much differently than normal people.

    2. Such analysis could potentially degrade a tea party group's ability to communicate."
    Then again, seeing as DHS and other federal agencies specifically identified Tea Partiers (and pro-lifers) as terrorists in their official documents, I didn't really fix or change anything that you originally said. Hmmmm.

  43. mlhouse:

    That government can use data for malicious, illegal and illicit ways is true. You are correct that the use of IRS data is such an example. I have totally agreed with that.

    BUT, as you have also agreed, the telephone meta-data and the IRS data are not the same. The ability to use a list of donors to a rival political group for nefarious actions is MUCH MORE DIFFERENT than being able to identify similar actionable analysis from the phone metadata. Using statistical terms, the R-squared of the IRS data is close to 1 (there would be some typo errors and other issues that make it imperfect). On the other hand, any analysis using the telephone meta-data would have an R-squared several magnitudes lower. Because of this, the predictive ability of such data is very limited. And because of such, the actions you can do are much more limited.

  44. mlhouse:

    Your comments are foolish. If you don't think so, to what address do the terrorist send their membership dues?

  45. marque2:

    I find this doubtful.

    I think a better plan would be to keep the terrorists out of the country in the first place. Boston bombers we were warned about several times through intelligence channels. And did you note how many of their friends were on student visas and not attending college, or even living close to the college they intended on going to? Also the mosque off limit thing should come to an end.

    Then we would have no need to monitor for terrorists because we kept people who hate us out. Seems like we let anyone in who wants to run over the border, and if they come from countries where the folks hate us, we will immediately grant them Visas, and then we have to monitor every person in the USA because we refuse to be selective about who comes in, so Democrats can get more votes.

  46. marque2:

    Seems like much of the spying and harassment has been linked to trying to discredit T party groups or intimidate reporters who were looking into government malfeasance. AP, Fox News, CBS news all spied upon. And then you tell us the NSA program is just for looking at terrorists, and any doubts about the governments pure intentions are "foolish." Hmm.

  47. NewEnglandDevil:

    The fact that the data itself doesn't produce a high R-squared for PREDICTIVE uses is inconsequential. Rather, pair that data with other data in other databases and use the data to fill in holes in information. Don't use the meta-data as a predictor. Hold the meta-data of all so that you will have access to meta-data of individuals. You're only thinking of one use of the data; your limiting parameters have no basis in what might be done with the data in fact.

  48. marque2:

    Seems like you can add the Gibson Guitar wood case to the list. Liberal guitar makers who imported the same wood were not harassed and did not have the wood confiscated to limit the ability to produce guitars. What seemed like just government overreach, now seems like another case of government intimidation.

  49. NewEnglandDevil:

    It is no more DIRECT than a phone number is. No one else has my phone number. It is as individual to me as my SS# or my DNA.
    "You're ignorant." "You're clueless." Any other insults you feel the need to throw my way since you can't win the argument on the merits?