Want to Make Your Reputation in Academia? Here is an Important Class of Problem For Which We Have No Solution Approach
Here is the problem: There exists a highly dynamic, multi- multi- variable system. One input is changed. How much, and in what ways, did that change affect the system?
Here are two examples:
- The government makes a trillion dollars in deficit spending to try to boost the economy. Did it do so? By how much? (This Reason article got me thinking about it)
- Man's actions increase the amount of CO2 in the atmosphere. We are fairly confident that this has some warming effect, but how how much? There are big policy differences between the response to a lot and a little.
The difficulty, of course, is that there is no way to do a controlled study, and while one's studied variable is changing, so are thousands, even millions of others. These two examples have a number of things in common:
- We know feedbacks play a large role in the answer, but the system is so hard to pin down that we are not even sure of the sign, much less the magnitude, of the feedback. Do positive feedbacks such as ice melting and cloud formation multiply CO2 warming many times, or is warming offset by negative feedback from things like cloud formation? Similarly in the economy, does deficit spending get multiplied many times as the money gets respent over and over, or is it offset by declines in other categories of spending like business investment?
- In both examples, we have recent cases where the system has not behaved as expected (at least by some). The economy remained at best flat after the recent stimulus. We have not seen global temperatures increase for 15-20 years despite a lot of CO2 prodcution. Are these evidence that the hypothesized relationship between cause and effect does not exist (or is small), or simply evidence that other effects independently drove the system in the opposite direction such that, for example, the economy would have been even worse without the stimulus or the world would have cooled without CO2 additions.
- In both examples, we use computer models not only to predict the future, but to explain the past. When the government said that the stimulus had worked, they did so based on a computer model whose core assumptions were that stimulus works. In both fields, we get this sort of circular proof, with the output of computer models that assume a causal relationship being used to prove the causal relationship
So, for those of you who may think that we are at the end of math (or science), here is a class of problem that is clearly, just from these two examples, enormously important. And we cannot solve it -- we can't even come close, despite the hubris of Paul Krugman or Michael Mann who may argue differently. We are explaining fire with Phlogiston.
I have no idea where the solution lies. Perhaps all we can hope for is a Goedel to tell us the problem is impossible to solve so stop trying. Perhaps the seeds of a solution exist but they are buried in another discipline (God knows the climate science field often lacks even the most basic connection to math and statistics knowledge).
Maybe I am missing something, but who is even working on this? By "working on it" I do not mean trying to build incrementally "better" economics or climate models. Plenty of folks doing that. But who is working on new approaches to tease out relationships in complex multi-variable systems?
jon:
One title "Human Action." Of course, I don't think Mises claimed he could know everything either. But he puts forth a good effort. Mises states that you cannot know by history you need to figure it out through praxeology.
February 18, 2014, 2:00 pmIgnoramus:
Logical thinking, with occasional laps of inspired thinking, tempered by common sense, is how. Humans can still out do computers at this. Math is foundational to logical thinking but doesn't have all the answers.
February 18, 2014, 2:11 pmI've been a skeptic of AGW for the following reasons:
1) CO2 is a trace gas. It would have to have truly magical thermodynamic properties to do what Warmists claim.
2) We've had much higher levels of CO2 in the atmosphere than now, and the atmosphere didn't boil over. Positive feedback loops are rare in nature -- most that we are familiar with (explosions and nuclear fusion) are man-made manipulations of nature and short-lived.
3) Earth has had large swings in temperature in the past. 10,000 years ago, from where I sit, I'd be writing this under a mile of ice. We've had documented swings in just the last 10,000 years. All this before the Industrial Age.
4) In trying to establish historical temperatures, AGW theory is based on relatively crude measuring sticks. High school chemistry 101: Your conclusions can't pretend to be more accurate than your tools.
5) AGW theory fails to account for variations related to the Sun's output, and excludes alternative theories such as the effect of solar wind variation on water vapor in the atmosphere.
6) Correlation doesn't prove causation.
7) A computer model isn't an experiment, especially when the model's predictions don't fit the actual data. AGW models don't even have correlation.
8) Al Gore
So AGW violates nearly all the foundational principles of science. It's amazing to me that we have so many college graduates -- including from elite universities -- believe this crock of shit.
Jesse:
According to the people implementing the solutions, we already have the answers. Just assume them and everything else flows forth from there.
February 18, 2014, 2:27 pmIgnoramus:
As to the Stimulus bill, many voices without Power, myself included, predicted that it wouldn't stimulate. ie. the multiplier would be less than one.
We could have dropped 100s from helicopters and had a higher multiplier. Stimulus money was used as payoffs to cronies, or went to lower income people who bought Chinese-made flat screens with their payroll tax cuts. Even Michael "Hockey Stick" Mann got a Stimulus grant. Nothing went to long-term investment. Infrastructure wasn't shovel ready.
February 18, 2014, 2:29 pmBusinesses read into this and other early Obama & Co initiatives and decided to hunker down and cut costs to be profitable. Hence the low multiplier.
You could see this coming.
DirtyJobsGuy:
Really complex systems are almost unmodellable. Pattern recognition and matching are really the best you can do. For climate the past (over the human past of about 50000 years) has been in a period of glaciation. Our understanding of really cold periods is based on a pretty slim data base of both the climate and potential forcings. For national economies we can similarly rely on both prior history and individual initiative. Only for things like currencies can we model it at all
February 18, 2014, 2:31 pmRuss R.:
Ignoramus,
1) Your "Trace Gas" argument is meaningless. The quantity isn't the issue, it's the effect on whether infrared radiation is transmitted or absorbed that matters.
February 18, 2014, 2:32 pm2) Agree.
3) The advance and retreat of glaciers over millennia is well explained by Milankovitch cycles. Forcing from GHGs is additive to that.
4) The degree of accuracy, and resolution decreases the further you go back, but recent historical temperature data, especially in the satellite era, are very good.
5) Then go ahead and demonstrate how the variations in the sun's output affect climate... or that these "alternative theories" have any predictive skill.
6) Agree. But what's your point?
7) A computer model is a model. It is not expected to ever be "right", but should have predictive skill. Some models have more skill than others. You can't assert than they all "don't even have correlation".
8) Ad hominem.
Daublin:
Why does every question have to have an answer?
For the two examples you mention, I believe the most sailent point is that there is no model with predictive power right now. As such, vigorous action based on those lines of arguments is futile.
February 18, 2014, 3:02 pmIgnoramus:
1) How can you say that the amount of CO2 in the atmosphere is irrelevant. That’s unscientific and defies common sense. I agree that CO2 captures heat. So every time I exhale I help make Earth a little bit warmer. The real question is how much of an effect CO2 has compared to other effects. As with nearly all natural phenomena, satiation and negative feedbacks are more likely to come into play than positive feedbacks.
3) Milankovitch cycles drive the Earths’s temperatures, not just glaciers. You say that GHGs add to this, the question is how much. Somehow Earth went from being hotter with more CO2 to colder with less CO2, without human intervention. Again, reason says that the absolute amount
of CO2 should be relevant to this,
5) It’s your theory not mine. At this stage you’ve got a hypothesis, called into question by my points 1 – 8, and belied by recent data trends vs. the models. We shouldn’t be surprised if we’re entering a 20 to 30 year cold snap, given recent falloff in the Sun’s output. Assuming academic freedom here (not a given right now), better science will emerge.
4/6) AGW modeling is based on inferring that because perceived increases in temperature happened when CO2 was going up so that CO2 must be the cause. Correlation isn’t causation. By necessity, AGW theory must use proxies for older time periods, which haven't been proven to be sufficiently accurate. Even if recent measuring techniques have gotten accurate, you don’t have a sufficient timeline to make assertions about long-term temperatures.
7 The models predictions have already been disappointing, no? QED, almost.
8 Al Gore isn’t ad hominem. Follow the money.
February 18, 2014, 3:12 pmjohncunningham:
predictive skill is exactly what the IPCC's computer models lack. they all failed to predict the flat global temps of the past 16 yrs. none of them accurately hindcast--you cannot start running a model is say 1930, and produce the recorded temps of 1940, 1950, 1960, etc.
February 18, 2014, 3:13 pmAlan in RI:
"But who is working on new approaches to tease out relationships in complex multi-variable systems?"
The Santa Fe Institute: http://www.santafe.edu
They have been working at it for 30 years.
My educated guess is that these systems are inherently impossibe to solve analytically due to the nonlinear nature of the coupled differential equations.
Moore's law will likely allow us to run sufficiently high fidelity simulations faster than real time, but I remain skeptical that we will ever be able to measure the the initial conditions accurately enough to ensure an accurate result.
February 18, 2014, 4:42 pmmarque2:
Your assertion on #4 isn't exactly true considering how the NOAA and NASA go out of their way to goose the satellite and other temperature data to show global warming when the sensors don't. Also satellites do not measure the poles well at all spnthe data has to be discarded - and I believe they don't measure ground temperature.
February 18, 2014, 5:14 pmmlhouse:
If you posit a relationship between a dependent and independent variable, build a forecast model that shows essentially a linear trend, and claim that the science is "settled", when the out of sample results come back that do not show the predicted trend, you are WRONG. I developed statistical models for a living and if my models would have been as bad as the climate models I would have been ashamed, and then fired.
Sure, there is noise in forecast models. Maybe there is some cyclical dimension that you missed. But you have no credibility and the chutzpah of the climate "scientists" and their fellow travelers is something to behold.
February 18, 2014, 6:08 pmNukeE:
Both of these problems, and most 'models' can be developed, they are just way too big and complex to permit current technology to translate the theoretical basis for the model into a usable tool. Conceptually, you build a set of theoretical relationships with unknown quantities in the variables and load a matrix with enough data to solve the for the unknowns. Then you put those variables back in and use the now complete model to extrapolate to the future.
The old linear equation fit is a very simple application where mx+b=y. If you know two points, you can solve it for any other point. The generalized problem is not new - the method has been around for a long time. It is used in things like nuclear reactor core and systems models (where I learned and promptly forgot the details - but I do have the books and notes - grin), structural analyses, all those pretty fluid flow models where they simulate temperatures and pressures in rockets and stuff, etc.
The guys in Santa Fe were doing economic models years ago, a bunch of us considered it in grad school, but you get into how to decide what data you need to look at to build a good model, e.g. is weather in Borneo that affects coffee prices significant enough to model the price of Hilton hotels stock since they provide coffee to many of their guests? And what is the appropriate time scale to model - how big of chunks of time must I consider to get a good model, do I need to calculate every month, every day, every minute, etc. How do you get all the data into a suitable format that you can process? Heck, how to I get all that data period (think of every possible stock price, every market index, every T number for every nation, currency rates, interest rates, etc. ad nauseum and how do you know you have the right information to make it work? (a. See below - you test it.)
The current climate applications are relatively straightforward compared to econ issues. The climate model is built the same way, but physical relationships allow some simplification of sorts. Basically the atmosphere is modeled as a bunch of little boxes which affect each other over time. Wind blows and air moves from box to box, heat moves with the air and with conduction. Don't forget radiant heat transfer, too. Pollutants and gases diffuse into the boxes and out of the boxes. Moisture moves in and out, etc. It gets real complex real quick. (I read of one climate model that was being touted, that had boxes so big that Japan disappeared into the pacific when the boxes were laid onto a map.) And remember you have to consider boundary conditions - the outer space 'boundary' and perhaps more importantly the surface under the atmosphere. Or maybe space is more important, and we just are more familiar with the surface where we live. Again - what size time steps do we use, and how often do we need data to get a convergent answer that will really represent the future (i.e. provide valid unknowns based on the historical data). And oh yeah, in a good model there are usually some little fudge-factors tossed in appropriately to account for relationships you don't recognize, but suspect exist.
Finally - a part the warming folks completely miss - you have to benchmark the results against a known outcome and demonstrate predictability. It is a pain, but without it you have nothing more than a lot of warm computers and wasted time. In the nuke business we do benchmarking on _every_ machine we run a model on - that means each and every individual PC is tested to verify the expected outcome. Not just a class or model of machine.
Anyway fun stuff. Available at your local engineering graduate school - typically called something like numerical methods or numeral analysis of complex systems, or transport theory and applications. Someday we may have enough good data available real-time to feed a good model in a timely manner that can then be solved by a really good computer (why did you think they build those big massively parallel machines on the hill in NM?) Even with 'static' physical data it is daunting to analyze the behavior that will occur. Lots of progress in this arena in the computer era, more to come, I'm sure.
Now back to the important study of some beer - simpler math and I can predict the outcome relatively easy without any core time. (You whippersnappers out there who came thru school within the last 20 years can correct my imprecise description, but I can't figure out how to explain it better right now.)
February 18, 2014, 6:52 pmNormD:
I would love to be a billionaire so I could hire people to work on really complex problems.
Protein folding
Economic modeling
Climate modeling
Control of gene expression
Understanding protein interactions
I have been concerned for 20 or so years that the quality of scientific research is dropping precipitously. There are three problems:
1. Researchers avoid complex problems because they take a long time to study and the chance of failure is very high. This means fewer or perhaps no publications.
2. The people who make it into research have followed a path the requires extreme conformity. There are very few odd-balls. Many breakthroughs are made by oddballs.
3. Attacking complex problems is intellectually exhausting. Its much easier to attack simple problems.
On a recent Econtalk podcast Judith Curry lamented that she has a very difficult time hiring staff to work on the really hard climate problems. Most departments are full of people who want to run a model and predict that the grape crop in CA will drop off as the world warms. Its easy to do. You publish lots of papers. You get accolades from the press and professional groups.
Why aren't economists modeling an economy from the bottom up on a computer building millions of semi-intelligent agents and see what happens when they impose Marxist or Keynesian or Libertarian solutions?
February 18, 2014, 8:42 pmjohn mcginnis:
For a model to be useful it has to be reliable and able to project both future events and PAST events. So far all the climate models fail on both scores.
February 18, 2014, 9:39 pmdon anderson:
It may be simplistic to say that at some point a problems complexity takes it beyond the effective reach of "Hard Science". Climate prediction is obviously one such example. Consider that we can successfully model a raindrop, including its interaction with its environment & other drops due to electrical, aerodynamic, plus elastic-surface-tension and hydrodynamic behavior when they collide -- all beautifully defined; but that does not help us predict whether our Saturday picnic will get rained out! We tend to work the complex problems by careful experimental and clinical observation, keeping notes in jargon that has grown specialized with the field. Results are subjective rather than primarily mathematical. Medical science has reached a growth point where we can now use bio-physics results from micro and molecular scale studies to gain deeper understanding of a disease and to fashion treatment -- discerning the link between micro & macro behavior. Although the physical body is enormously complex and variations between individuals are overwhelming, there is certainly great progress in medical science. But climate explanations, despite the mathematical models, will probably remain in the category "Observe-record-extrapolate (subjectively-with-high-fog-index-vocabulary)" for a long time yet.
February 18, 2014, 11:14 pmmikehaseler:
But there is a difference - if the chief economist told the public that they were "unequivocal" that the economy would grow because they had super-dooper models they knew were right, and then it did not grow, and it was found that the chief economist's figures were not based on their model, but were just estimates not from the model (which he knew were wrong) but were a bunch of estimates he and a bunch of his buddies had dreamt up .... and he very carefully hid this fact so the government would continue spending billions even though it was known the models were wrong ... and it was known the chief economist was getting a huge amount of money in grants because their models had said they could stop economic recession. But this was clearly hog-wash and it was obvious there was a clear personal interest in them hiding the failure of their models.
The public, press, politicians, academics, ... everyone would be tearing them to shreds. And no doubt the police would be called in.
But when it happens in climate - they award themselves a nobel prize and no one says a thing.
February 19, 2014, 12:33 amIain:
Didn't Hari Seldon do this already? If I remember, it needed a group fudging the results over the long term....
February 19, 2014, 2:08 am