I Am Starting To Believe Most Studies Are Crap
I spent years, before I burned out on the task, picking over bad climate studies, and at the time reached the conclusion that there was something about the climate science field that was anomalous, tolerating so much bad science, bad sampling methodology, and bad statistical approaches.
However, now I am coming to the conclusion that perhaps most studies in every field are dominated by this same crap. Here is an example, from the NTSB on busses.
steve:
I work in engineering. Not exactly a study but I think it is relevant. In the commercial world, we were told or suggested what was wanted, would estimate the time and cost, and a decision would be made if it made sense to go forward.
In the government world, we were told or suggested what was wanted, then a couple of university types were added to the team to suggest a few more things. Then an estimate of the time and cost was made. This estimate was ruthlessly and yes fraudulently paired down to magically fit the amount of dollars in the available contract without altering any of the goals whatsoever. We eventually delivered some portion of the original goals once the money was gone. By commercial standards, every project was a failure. However, sometimes those pieces of a project that actually got delivered were interesting and useful. These constituted a success in the government world.
Government corrupts everything it touches with its funding. Science, engineering, day care centers, etc.
May 7, 2013, 10:30 amAugust:
This is true in nutrition. Additionally, even in tests related to DNA research, there is this troubling tendency to test against a computer model rather than actual DNA. Someone did their thesis on this- I think she found the models had a 30% error rate. So, in addition to all the faux sciences, we've even got one I thought was pretty solidly anchored in reality that probably needs everything thrown out and rebooted. Have to chunk most of academia too, since the diversity movement did a pretty good job of excluding anyone smart enough to see how stupid this stuff is. A lot of them really think they are doing good. Some of them actually think they are scientists.
May 7, 2013, 11:00 amMatthew Slyfield:
Power corrupts, absolute power corrupts absolutely.
Government = Power
Therefore
Government corrupts, absolute government corrupts absolutely.
:)
May 7, 2013, 11:51 amAugust:
I am not that big of a fan of this saying because I think the abuses of power that we see come from the nature of being human. The corruption is not in the power, but in the man. An invulnerable being would have a different set of incentives than humans do- zero need to defend oneself, for instance, but possibly a higher interest in things that make the galaxy more interesting. Consciousness makes the galaxy more interesting because we conscious being are slightly less predictable than non-conscious things.
May 7, 2013, 12:42 pmcal_culus:
Immigration, homosexuality, and gun control are other victims of bad statistical and social science to add to your list.
May 7, 2013, 12:50 pmMatthew Slyfield:
" An invulnerable being would have a different set of incentives than humans do- zero need to defend oneself"
An invulnerable being would be immune to retaliation for abusing others for it's own amusement.
"higher interest in things that make the galaxy more interesting"
That which such a being would consider interesting does not necessarily make the galaxy a better place for others. It could for instance find your suffering to be intensely interesting.
Absolute power would tend to remove any incentive to consider the well being of others.
May 7, 2013, 1:15 pmMatthew Slyfield:
It is very different to consider a being that naturally possesses such power from the effect on the morality of a mortal being that is suddenly handed such power. The latter is what the saying refers to.
May 7, 2013, 1:25 pmMatthew Slyfield:
"Immigration, homosexuality, and gun control"
Pretty much everything in the social and political science arenas. Plus at least 3/4ths of psychology.
May 7, 2013, 1:31 pmcal_culus:
Every ones belief system comes from "what we were when." My dad still keeps a wad of bills in his socks from being raised during the Great Depression. Humans are skeptical of some information but need little proof for core beliefs, future consequences be damned. The dismal science, economics, is probably the worst.
May 7, 2013, 1:45 pmcal_culus:
You don't make me feel good about our nuclear weapons since they are computer modeled and we haven't done any real testing in decades.
May 7, 2013, 1:52 pmMatthew Slyfield:
Oh, yes, I forgot economics. Silly me.
May 7, 2013, 2:14 pmJoshua Vanderberg:
Yeah, but those models are written by real scientist who actually fact-check against reality. And we've got a heck of a lot of experience with blowing up real bombs.
May 7, 2013, 2:16 pmjimc5499:
When you have a Senator who has the power of Schumer wanting a study that shows something to be dangerous, you make sure that the study says that it is dangerous. Otherwise your funding might go someplace else. It used to be that the NTSB had a rep for staying out of politics, that went away during the Clinton Administration. The NHTSA has been owned by MADD for years. I don't take anything they say seriously.
I'm pretty sure that you can track Global Warming back to Al Gore when he was VP. I believe that the VP is the head of NASA and the National Science Foundation. That position controls a lot of the funding for research and these scientists know which side their bread is buttered on.
May 7, 2013, 2:21 pmcal_culus:
The Federal Reserve has a hundred years of experience blowing up the economy, but can't figure it out. I'm skeptical they ever will, no matter what model they use.
May 7, 2013, 3:10 pmMingoV:
I've posted here and elsewhere about the poor quality of most scientific and medical studies. Other fields had and have similar problems. Anyone who knows statistics can see that many studies screw up data analyses and draw incorrect conclusions. (I'm remembering the idiot physicists who thought they had found "cold fusion.")
Government studies or government-commissioned studies almost always 'prove' the exact thing that the pols want. The recent Heritage Foundation report on providing amnesty for illegal immigrants arrived at a total cost to governments of over six trillion dollars. The Heritage Foundation 'forgot' that many of those costs (such as schooling and hospital care) already are being borne by governments. They also forgot that providing the immigrants with legal status will enable them to get jobs that pay more. (Many are being paid under-the-table for less than their work is worth. Others want to work but are afraid of ICE.) Heritage Foundation is known to we right-wing, and it generated exactly what right-wing politicians wanted.
May 7, 2013, 3:49 pmAndrew_M_Garland:
( http://www.lhup.edu/~DSIMANEK/cargocul.htm )
Cargo Cult Science - 1974 by Richard P. Feynman - Commencement speech at The California Institute of Technology
The speech is a must-read. It explains the difference between mimicking the language and process of science compared to the real thing. It explains what "cargo cult" means applied to our present lives. [edited excerpts]
=== ===
I often talked to the people in the psychology department at Cornell. One of the students wanted to do an experiment. Others had found that under certain circumstances X, rats did something A. If she changed the circumstances to Y, would they still do A?
I explained that she should first repeat the experiment of the other person. Create condition X to see if she would also get result A. Then change to Y and see if A changed. She would know that the real difference was the thing she thought she had under control.
She proposed this to her professor. He replied no, you cannot do that, because the experiment has already been done and you would be wasting time. This was in about 1947. It seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happens.
[Later, Feynman tells about the studies done by Mr. Young which determined how perceptive rats really are when trying to test their behavior]
Young did an A-number-one experiment from a scientific standpoint. His research made rat-running experiments sensible, because it uncovered the clues that the rat is really using, not what you think it's using. And that tells exactly what conditions you have to use in order to be careful and control everything in an experiment with rat-running.
I looked into the subsequent history of this research. The next experiment, and the one after that, never referred to Mr. Young. They never used any of his criteria of putting the corridor on sand, or of being very careful. They just went right on running rats in the same old way, and paid no attention to the great discoveries of Mr. Young.
His papers are not referred to, because he didn't discover anything about rats. In fact, he discovered all the things you have to do to discover something about rats. But not paying attention to experiments like that is a characteristic of cargo cult science.
=== ===
What is the similarity between our government and some superstitious Pacific islanders during World War II?
May 7, 2013, 5:40 pmCargo Cult Economics
steve:
There are some people in those places with real morality. What happens when one of those people produces a study that says something unwanted is that it just gets shelved and not released.
May 7, 2013, 8:01 pmcal_culus:
The physical quantities modeled by physics and engineering exist independent of the observer. Those
May 7, 2013, 8:23 pmmodeled by socialists exist only in the mind of the observer.
annonerz:
Mass transit denier!
May 8, 2013, 5:15 amAugust:
Destruction is finite, since it ends when whatever is being destroyed is destroyed. Consciousness, creation, innovation- these are the interesting things. Now, there could be quite a large difference between what others thought might be abuse or well being, but at least in the aggregate, I suspect the incentive is to have more conscious beings around and at least well enough to have more children. A universe with no conscious beings in it would be very boring.
May 8, 2013, 8:22 amdc:
kudos for the feynman reference - the man was a true genius
May 8, 2013, 9:08 amdc:
"cant figure it out" is also an assumption ;)
considered it possible that they know *exactly* what they're doing???
May 8, 2013, 9:10 ambigmaq1980:
Excellent link!
May 8, 2013, 9:14 amirandom:
"Yet, in holding scientific research and discovery in respect, as we
should, we must also be alert to the equal and opposite danger that
public policy could itself become the captive of a
scientific-technological elite."
Eisenhower
May 8, 2013, 9:32 amZachriel:
Arrhenius published his findings on greenhouse warming and amplification in 1896.
Arrhenius, On the Influence of Carbonic Acid in the Air Upon the Temperature of the Ground, Philosophical Magazine 1896 {Carbonic acid
May 8, 2013, 12:04 pmmeans CO2 in the historical context}.
Matthew Slyfield:
Nonsense. An omnipotent being could destroy you, then resurrect you just to do it all over again. We are not talking about a naturally omnipotent being, but a mortal that acquires such power as a mature individual. Such a person could be very self destructive and take the rest of us with them. There is absolutely no force or incentive to drive such a being in positive directions.
May 8, 2013, 5:21 pmZachriel:
Coyote Blog: I Am Starting To Believe Most Studies Are Crap
Of course. Nothing new about that.
May 8, 2013, 5:56 pmAugust:
He could do all that you say, but the quiet little disincentive to do that is that it is boring. I am finite; there are a myriad of ways to kill me, but it all ends the same way, and the whole experience is finite. The incentive to foster life is novelty. More conscious beings creating and innovating means more new things/beings/experiences. There is no force barring anyone from doing what is boring, but the discontent in the mind eventually encourages us to do other things.
May 9, 2013, 7:03 amsteve:
Just as a side not, in the commercial world most of the infighting was about how to resolve various technical issues, i.e. my way or your way. In the government world, most of the infighting was about which of the goals should receive priority, especially by the university types, knowing that most of the goals wouldn't be accomplished, and they had papers to write.
May 9, 2013, 7:07 amrxc:
If this is what the NTSB is producing these days, then we really are in a mess of trouble. They used to be the quintisential(sp?) non-political, technical, no BS agency in the govt. They just do the technical analyses of transportation accidents, and present the results to the people who have to fix the problems.
I used to be in charge of performing and evaluating nuclear reactor safety analyses in the govt, and I can say that I was never pressured to produce results that met the expectations of any politicians or even senior agency management. However, this sort of job does lead one to be skeptical about analyses done in other fields. I have seen lots of organizations do very "creative" things to make the results come out the way they wanted them, sometimes even when there was no money at stake - only the ego of the analysts to be able to model an experiment.
It can be very humbling to try to model nature.
So I disbelieve, off-the-bat, just about every analysis that comes out of any field with the word "science" in its title. The social scientists and econimists are mis-using numbers to screw with our minds, while the climate scientists are using clouds (literally) to try to drive us back into caves.
And the educational system is trying to ensure that the children are as ignorant of all of this as possible, so that they will be easier to control when they become adults.
Progressives have a lot to answer for.
May 10, 2013, 7:52 amJoe_Da:
Many of the environmental studies suffer from several biases including political biases.
For example, numerous studies show that there is an increase in premature mortality when exposed to increases in ground level ozone. One of the premier studies of 100 US cities showing increased morality based on 10ppb increase in ground level ozone. This and many of the other ozone studies suffer from lack of a control, data collection bias, etc.
May 11, 2013, 12:18 pmrxc:
I am occasionally tempted to accumulate all of these reports about the number of "excess deaths" that are attributed to one environ threat or another, and add them all up. I have a feeling that the total would be larger than the number of people who are actually dying each year. That would be something to write a paper about.
The big scandal with these studies is that they are used to justify squeezing the economy to make life "safer" for "the most vulnerable". It is a progressive trick to kill western society.
May 19, 2013, 12:35 amJoe_Da:
Another good example are the recent studies showing the reduction of premature deaths supporting the EPA's proposed reduction in small particle and mercury standards, The reduction in annual premature deaths account for 20-25% of the total respiritory deaths per year.
May 19, 2013, 2:54 amMogumbo Gono:
Arrhenius was wrong, because he did not account for convection — a basic error. Convection happens in the planetary atmosphere, not in a laboratory beaker.
Planet Earth herself is falsifying the "carbon" scare: for the past 16+ years, global warming has STOPPED — despite steadily rising CO2 levels. The endless predictions by the alarmist crowd of runaway global warming have been thoroughly debunked by the ultimate Authority: Planet Earth. Global warming is simply not happening.
But don't expect people who have invested their entire belief systems and egos into their personal demonization of 'carbon' to admit that they were completely wrong all along. Human nature does not operate like that. And the Scientific Method is routinely ignored by the climate alarmist crowd.
Fortunately, the public is beginning to see that the global warming scare is driven by federal grant money, not by science. And taxpayers are getting really fed up with that scam.
'Global warming' is a money-fed scare that has made Algore and a few others rich, at the expense of the rest of us. But it is a false alarm, as anyone with common sense can see.
June 5, 2013, 9:42 am