Every other macro blogger seems to be taking a crack at this question. I like what they have to say. But I have a much simpler theory.
Let's suppose you wanted to design an experiment to test the effects of monetary and fiscal policy. And suppose you had the power to do whatever you wanted, and couldn't care less about getting clearance from the Research Ethics Board. What experiment would you design?
Probably something like: get 100 countries, then for each country toss two dice, one for monetary policy, the other for fiscal policy, then watch what happens for a couple of decades. That should settle the question.
Now, just for a laugh, imagine you wanted the worst possible experiment design. An experiment design so bad that no researcher would ever be able to figure out the effects of monetary and fiscal policy by looking at the data. What would you do?
Probably something like: make monetary policy negatively correlated with fiscal policy, and negatively correlated with any other shocks you observed hitting the economy. So it would be impossible for the researcher to disentangle the effects of monetary policy, fiscal policy, and any other observed shocks. If you were feeling really mean, you would do your best to try to set monetary policy and/or fiscal policy so that nothing ever did happen to the economy. So all the researcher would ever observe would be a few small fluctuations in the economy due to shocks you didn't observe (and he probably can't observe either), plus a few small fluctuations because you over-compensated or under-compensated for the shocks you did observe, and even here the researcher won't be able to figure out even the sign of the effect of monetary policy because he won't know whether tight money caused inflation or whether you just didn't tighten monetary policy enough in response to an inflationary shock.
That's the world we live in. We live in something very close to the worst possible experiment design for testing macroeconomics. We live in a house controlled by Milton Friedman's Thermostat. It's sometimes a wonky thermostat, that under- or over-compensates for the shocks it can see, but it's still a thermostat. For all its faults, it does not play dice with the amount of oil going into the furnace. Even if it did play dice, you could never be sure that it was playing dice, or just responding as best it could to some shock it sees that you don't see.
You want us macroeconomists to figure stuff out better? Sure. No problem. Just lend us 100 countries for a couple of decades, and let us play dice with monetary and fiscal policy.
Worthwhile Canadian Initiative 
Nick:
This isn’t magic, it’s science – so how about approaching this question scientifically?
It’s much easier to score macroeconomists than to give them 100 economies to play with for a couple of decades: simply take their past predictions about growth, unemployment, inflation and check how well their models worked in the real world.
Put macroeconomists who produce no testable predictions in the middle of the list.
If you do that then people like Paul Murphy end up near the bottom of the list – while people like Paul Krugman end up near the top of the list.
Reality is good enough of a “decider”, no need for other macroeconomists to agree to accept reality. If our universe has an apparent liberal bias then that’s your problem really.
Anon: tell me what testable predictions there would be in a world like my worst experimental design.
Anon, scientifically what you propose becomes yet another non-causal correlation. i.e. people can get lucky.
Empirical study is hard in macroeconomics, but I think the greatest hope lies with progress in computer simulation and new technology overall. Just as technology has finally given neuroscience the tools it needs to really experiment empirically (and is thus ushering in a golden age of neurophys study), one day someone will probably be able to come up with something similar for economies. Until then, it is just a waiting game I suppose.
I dunno. Much of the DSGE literature is just that: simulated economies that are judged by how well they fit some basic stylized facts. The problem is that there are an infinity of models that can reproduce a finite set of stylized facts – if you doubt that, read the DSGE literature.
I think Nick is on the right track here. There are brutal identification problems in macro. And you can’t test to see if a given identification rule is correct, you can only try to persuade people that it’s plausible.
tl;dr: it’s not a real science.
[Urban dictionary translates “tl;dr” as “too long; didn’t read” – SG]
Thankfully in the real world monetary and fiscal policy don’t act perfectly like that giving us some insight. And while there are always an infinite set of theories that can fit any finite set of facts, there’s also an infinite set of theories which do not fit those facts and the profession, or subgroup of the profession, at times, is overly reluctant to move a theory from the former to the later set.
Nick,
Why do macroeconomists disagree?
I don’t think macroeconomists disagree on most cause and effect issues. I think that macroeconomists disagree on objectives.
Nick is right. There are a few natural experiments out there, like North Korea vs. South Korea, and East Germany vs. West Germany, but only a few. And the dimensions they differ along are not monetary or fiscal policy, but in more fundamental institutions, like private property vs socialism, central planning vs market economy.
To the extent that we have these natural experiments, it’s pretty clear that capitalism works better than socialism, but that’s about all they tell us unambiguously.
However, there is a halo effect: when two economists who differ along the capitalism socialism dimension also disagree about something else, I tend to disregard the socialist on the grounds that he’s already shown himself incapable of learning from evidence. Milton Friedman vs Paul Samuelson? Which one had the bad judgement to extol the virtues of the Soviet economy long after it was obvious that it didn’t work?
Anybody who believed North Korea and East Germany were socialist is way too credulous for me to pay attention to.
I’m a little surprised to not see more experimental macroeconomics using small economies in a lab setting. I’m sure that won’t answer many interesting questions, but it should answer quite a few.
This post is as concise an explanation of why I don’t like the “economist as prophet” view that is so often used by popularises of macroeconomic theories (including the ruthlessly aprioristic Austrian School!) i.e. such famous prophecies might be interesting if they were based on numbers derived from a formal model derived from the theory in question, but that is so rarely the case…
Instead, predictive accuracy serves two modest but useful roles in macroeconomics: (1) clipping off some of the truly muddle-headed theories and (2) telling us that the world macroeconomy is such that if you build a theoretically coherent forecasting model, it will probably get all the big events wrong even if it gets the more mundane events right.
Sciences that inform us of the limits of our knowledge are disappointing, but no less scientific for that. Similarly, it’s disappointing to find out that the Earth is a tiny sub-dot in an unimaginably huge universe, but that doesn’t make astronomy unscientific. (Yes, this does mean that Carl Menger and F. A. Hayek are economics’s Coperniucs and Galileo.)
I like this post, but would like to add a little bit of digging I did a while ago via Email correspondence.
Here is GREG MANKIW from “Modern Macro: Its origin, development, and current state” textbook…
On Clinton: “My reaction to President Clinton’s speech [17 February 1993] is that I don’t think we need the fiscal stimulus that he is proposing. Recovery is already on its way. It wasn’t a very deep recession to start off with, so I’m not terribly shocked that there is a mild recovery. It will take the fiscal stimulus a while to get people employed. I am happy that he is worried about the budget deficit, as low national saving is an important macro problem in the long term in the USA. Yet I am disappointed that he is putting so much emphasis on tax increases rather than spending cuts. That is really a view not so much about macroeconomics as about the size of government. I am also disappointed that he is giving no attention to the low rate of private saving in the USA. I would recommend tax reforms to remove the present disincentives toward saving. So I give him a mixed review.”
To emphasize, his most important views on policy are “NOT SO MUCH ABOUT MACROECONOMICS AS ABOUT THE SIZE OF GOVERNMENT” – the size of government is a moral hazard, for Mankiw, nothing more…
Economists could make a lot more progress by having more objective debates aimed at converging and resolving precise points of disagreement. It’s no good quoting models at each other, since all models do is obscure the key views driving the conclusion. Economics is about the real world, so it should always be possible to explain the mechanistic logic for a particular view, and when it becomes clear that a particular point is the source of the difference, it should be possible to resort to evidence, perhaps econometric, to resolve the issue and move on.
For example, I have been left none the wiser about perhaps the greatest debate of our time – how fiscal policy works. I thought Fama made a perfectly reasonable argument in his debate with Krugman, about where the actual resources to stimulate come from, but Krugman resorted to rhetoric – “dark age” etc – rather than actually addressing the point.
Given that there are so many with political agendas eager to latch on to arguments that support their position, the economics profession should disdain those “academics” who become unobjective advocates of one side of a debate, like Krugman, DeLong or Scott Sumner. They certainly should not be teaching and evaluating vulnerable students.
I think you are saying that macro economists disagree because they don’t know anything and have no way of finding out.
Seems a fair view. But perhaps they should be more open to the Wittgenstein view “if you don’t know, shut up.”
Follow-up to Stephen Gordon’s comment.
He mentioned “identification” (more generally, I think he meant the field of systems identification and control theory), which is an alternative to your proposals for experiment design. Better and real-time economic data would help; possibly an Act of government to compel credit card issuers and capital markets players to provide their data for economic analysis as it is already being collected. Ditto for other government services like land registery etc.
Nick,
What is interesting to me is that those will a greater understanding in economics than those inside the academic bubble, people like George Soros, likely completely disagree with you. They do not view economics as a science and they do not hold Friedman’s view that we live in a house controlled by Milton Friedman’s Thermostat.
Soros describes our economy in entirely different terms. For him, an economy, as a social process, is a “reflexive process in which the participants’ biased decisions interact with a reality that is beyond their comprehension.” No models or experiments about such a process are possible.
What good does it do to observe someone who is pushing down on the gas pedal who is blind?
The greatest practitioner of Macro Economics was Roosevelt, but not because he got his macro right. No. Roosevelt instead focused on protecting all in society for whom the economy was not working. The lessons that economics teaches us are three: (1) attack rent seeking; (2) and build as strong a safety net as possible, as social insurance against what we don’t know about; and (3) learn a little from Adam Smith about the super importance of insurance. The ultimate result of the reflexive process is the creation of risk. Macro economics ought to concentrate on how to manage that risk, which was what Keynes was doing (the idea that the Gov’t should save in good times and spend in bad is, at its heart, insurance).
Why would actual macro experiments need to be conducted at a whole country level? And done randomly? Is this really the way controlled trials work in medicine or other fields? Has anyone in economics seriously looked at what expense, time, rule changes, etc., would be necessary to set up sufficiently-robust-to-be-meaningful macroeconomic experiments? I’d love to see some posts on what would be involved. For years, in the U.S., people have described the states as “laboratories for Democracy.” This always made me curious, even back in the 1980s–couldn’t disputed issues like whether or not raising or lowering capital gains taxes enhances economic growth or provides other benefits be possible at the state level (pick a state out of a hat)? Or if people object to this, doing such tests on a volunteer level, in which the participants are compensated for serving as guinea pigs (and tracked exhaustively, via the use of say, debit cards, for ALL financial transactions). I’m sure this could be done, and I’ve wondered if the world and politicians aren’t simply afraid to really learn many of the answers that such experiments would provide. It would dent ideological positions, to be sure.
A very good econometrician used to have a maxim: “estimate, don’t test”.
this is why.
good post Nick.
Re: worst possible experimental design….how about throwing out 99% of the data we have and just focusing on US data? No….wait….let’s throw out even more by only looking at “recent” data?
….and let’s introduce lots of unobserved variables that play key roles in our models! Like “gaps” or “marginals” or “expected” or…..
“…. get 100 countries, then for each country toss two dice, one for monetary policy, the other for fiscal policy, then watch what happens for a couple of decades. That should settle the question.”
Actually, we’ve got 3 decades or more of macroeconomic data for literally dozens of countries….with quite a bit of cross-country variation in policy….and macroeconomists (as opposed to international economists) rarely use the cross-country dimension to improve identification. Some macroeconomists are prone to argue something along the lines of “the US is exceptional” or “Japan is different” or “Germany is a special case”, etc. etc. This continues to puzzle me in the context of macroeconomic models that are supposed to have firm theoretical foundations, as I thought the aim of such modeling exercises was to find fundamental and broadly applicable truths about macroeconomic behaviour. How do you think about that problem, Nick?
I think macroeconomists disagree becaused the interrelationships in the macroeconomy are so complex that different theories can be justified for different reasons, based on different pieces of the evidence. For any compelling theory, there is a mountain of evidence supporting it. For every theory’s shortcomings, there is a mountain of evidence explaining why the failure in question was not really a failure, but the absence of ceteris paribus.
This is a great explanation for someone in the biz: it absolves macroeconomists of any guilt for still debating basic issues that have long been settled. Yes, it’s structural barriers on research, and nothing else! No corruption to examine here! Let’s all go drink champagne.
But back in the world of people who search for the truth, a good understanding of economics shows that it’s an inherently self-contradictory field. Economists know that people are motivated by incentives, and the incentive to get macro right (pride… that’s about it) is less than the incentive to get macro wrong (lucrative book and speaking deals, sinecures at ideological think tanks, advisory positions for ideology-driven politicians). Couple that with a desire to teach all current theory as if it’s equally valid and to publish all theory as if it’s equally valid, a gentleman’s agreement among macroeconomists to never suggest that their colleagues might be corrupt, and universities that remain unwilling to push the field towards truth-seeking, and there’s no incentive to get macro right.
We don’t need perfect experimental conditions to know that tripling the money supply doesn’t invariably triple prices. Yet we can’t get macroeconomists to definitively agree to that simple statement. Blaming structural barriers to absolutely perfect research is intellectual laziness at best, deceptively covering up corruption at worst.
Because macroeconomics is the religion of the 21st century. You have a number of competing schools of thought which can all find backing somewhere in the text/Bible/data but ultimately, because of the points made above, nothing can be “proved” and therefore it relies more on faith than anything else. In the absence of actual scientific rigor, people will manage to “prove” what they “believe” and “believe” that they “proved” it because they use math, the same way that religious sects use different close readings of religious sects to support their own closely held views as dogma. The various sects will therefore never come to any agreement over how old the Earth is or how many angels can dance on the head of a pin.
No offense intended to the economists out there, but that’s just what the last decade looks like to me. There will never be any better data than the past 10 years, yet we “know” nothing more than we did before hand. The different schools of though have varying levels of credibility (to me, at least), but the final truth is that the economy is people and people act stupidly, especially in packs, and nothing I’ve seen accommodates appropriately for that. Human society is not particle physics.
100 countries will be as bad as today. You need the same world 100 times and a time machine.
What if … since macroeconomics is not a natural science any dominant theory will be arbitraged by the “market” which will always finds profitable theoretical gaps to exploit. Like you start targeting consumer inflation, the market will create asset inflation and financial sector to exploit it.
Nick,
It seems to me that you’re conflating two very different questions here. The first one is why macroeconomists don’t know more than they do, and on this point your argument is indeed convincing. However, your title suggests a different question, namely why there is so much loud, passionate, and self-assured disagreement among macroeconomists. This second question is much more unpleasant, and it can’t be answered by appealing to the difficulty of the problem.
Mere inability to figure out things should ideally produce a humble consensus that we don’t know much, or at most a tentative and respectful disagreement in which everyone admits that they may well be wrong. Yet this is not what we normally see in the public discourse among economists. (Recognizing, of course, the exceptionally high quality of discourse in some venues like this one.) So it seems to me that there are deeper and worse problems involved.
Sergei: that sounds like the Lucas critique.
John: “The greatest practitioner of Macro Economics was Roosevelt, but not because he got his macro right. No. Roosevelt instead focused on protecting all in society for whom the economy was not working.”
Roosevelt took an empirical approach. He tried things out, rejected what did not seem to be working, and kept what did seem to be working. I’ll take that approach any day over experts who claim, “We understand the economy, and nothing works.” 😉
Nick, I think your argument ignores a fundamental and important problems with macroeconomics; macroeconomists frequently disagree about the facts.
As a case in point, remember the arguments back in October about whether or not financial crises are typically followed by unusually deep recessions or unusually fast recoveries. This kind of question would not appear to be subject to the identification problems that you stress. However, when considerable attention was focused on the issue of US macroeconomic performance, prominent economists (e.g. John Taylor, Michael Bordo, Glenn Hubbard, Paul Krugman, Ken Rogoff and many others) were unable to agree whether the historical record showed the financial crises had a positive or a negative effect…but both camps felt that the empirical evidence was clear and that there was no significant identification problem.
Yes, the data is crappy. I like to give my students this example. Suppose two different worlds: (i) Prices are perfectly flexible, business cycles are driven by various shocks to fundamentals, and the central bank cares about price stability; (ii) Prices are sticky, the same shocks are working in the background, and the central bank has enough information to elmiminate output gaps (assume the zero lower bound never binds). You can construct the example in such a way that the data in (i) and (ii) will be exactly the same – prices look sticky, and the shocks appear to be driving the business cycle. What are we supposed to conclude by looking at the data? This is why we have econometricians. The way economists use statistics is much more sophisticated than almost anything you see in other sciences, for good reason.
Erik,
Well, you’re right that economics isn’t particle physics – its a lot harder, precisely because it looks at the interaction of sentient people rather than inanimate (at least so far as we know ;)) particles.
Moreover, I think we have to be careful about dismissing economics as a modern religion on the basis of the nloud and occasionally obnoxious disagreements between competing theoretical camps. After all the same behaviour is not uncommon between different theoretical schools in the “hard” sciences where conclusive emprical evidence to refute one camp or the other (or both) is unavailable. All that distinquishes economics from the “hard” sciences is that the more arcane – and therefore the nastiest – disputes in the “hard” sciences don’t play out on the op-ed pages of the New York Times and string theory isn’t a common topic of discussion at fashionable dinner parties. The only reason your allegation isn’t levied at theoretical physicists is that only theoretical physicists know or care what they’re talking about, whereas the audience for economic disputes is much broader.
Its only when such disputes continue in the face of conclusive empirical evidence (or when the parties stop trying to provide conclusive evidence one way or the other and start excomunicating heretics) that we can properly characterize a field of study as a religion.
Its only when such disputes continue in the face of conclusive empirical evidence (or when the parties stop trying to provide conclusive evidence one way or the other and start excomunicating heretics) that we can properly characterize a field of study as a religion.
I posit that the relationship between Austrian, Mainstream and Post-Keynesian economists shows that the excommunications have already happened.
Bob Smith,
It is certainly true that most people have a “treasure-chest” view of how the sciences develop: knowledge gets discussed a bit, then becomes uncontroversial and put into the treasure chest, and what goes in the treasure chest stays in the treasure chest. When sciences get the cold light of day put upon them, their publics reputations suffer.
Determinant,
I would go further: the creed was never set down. There were some doctrinal debates in the 1930s, and the Neo-Keynesians nearly swallowed up the (non-Marxist) profession, but even at the height of the Neo-Keynesian hegemony there were institutions like Cambridge, Chicago, and Brooklyn Polytechnic where very ideas could be encountered. Given how well Neo-Keynesianism turned out, this diversity was a good thing. Scientific consensus is no good if it’s wrong! Better knowing that economists don’t know it all, than the “New Economics” thinking of the 1960s and the associated destructive delusions of grandeur.
Alex, Lucas critique? Hm. Sounds like it is if we give enough credit to Lucas that his critique covers all possible cases and especially expand them onto theories. Which to my understanding was not his idea. But look what we have as a matter of macro facts. Exchange rate targeting and current account. Inflation targeting and asset inflation. Fiscal demand management and consumer inflation. Maybe some other cases as well but I did not really think about it. Probably not so much but we are talking decades and a couple of centuries is all we have anyways. Now the talk is going more to the NGDP targeting and somehow I am sure that in 1-2-3 decades the private sector will come up with a sure way to arbitrage this policy and milk central bank like it always did. Yes, not the whole private sector. But all we need is one Soros and one Bank of England and the mission will be accomplished.
Vladimir @2.06PM makes a good point. I’m not quite sure how to respond. Maybe like this: economists (like all scientists and dentists and policymakers) will always find something to argue about. That’s our job. If we figure out the answer to one question and stop arguing about it there will always be something else we don’t know, where there’s more than one way to interpret “the facts”, so we will start arguing about that instead. What’s surprising though is that macroeconomists disagree about the big stuff. I think my theory explains that.
Simon: I think there’s a lot to be said for looking at as wide a possible range of countries and years.
For example, one of the natural experiments would be Spanish inflation after bringing in gold from the Americas. It is very implausible, as some other economist once said, that Columbus set sail to America to try to find the gold that would be needed during the inflation he forecast.
As I understand it, the whole point of Friedman and Schwartz’s Monetary History of the US was to try to use historian’s methods to try to identify causes of monetary changes that were plausibly exogenous and could be treated as much like a roll of the dice.
Another good natural experiment is when somebody turns down the dial on the thermostat. Like maybe the 1982 recession (in e.g. Canada).
Some of the best evidence I have that monetary policy affects inflation is that when the Bank of Canada said it was going to use monetary policy to keep inflation at 2% it succeeded (roughly). It is just too big a fluke to believe that inflation just happened to average 2% over the last 20 years. But someone just looking at the data would say (and I have heard people say this): “Look! There was almost no correlation between inflation and unemployment over the last 20 years! And the Bank of Canada kept changing interest rates but nothing happened to inflation!”
But we also lose something when we extend the range like that. Because we have to hope that all countries at all times respond the same way, at least qualitatively.
Adam P: Great to see you back!
Steve: I agree. “This is why we have econometricians. The way economists use statistics is much more sophisticated than almost anything you see in other sciences, for good reason. ”
But I still can’t figure out how econometricians can actually solve this problem. For example, when an econometrician claims to have identified “monetary policy shocks”, how can he know that the central bank isn’t responding to something that is in the Bank’s information set but not in the econometrician’s data set? (My old post on this.)
Good comments all. I’m not responding to all of them because I don’t have much to add.
Another little issue is the paucity of data. I find it hard to argue that we can extend US macro data earlier than (say) 1982, after the Volker anti-inflationary period, and after the (then still partial) resuscitation of global capital markets and at least the start of interstate banking and a greater penetration of money market funds and other such inventions. If we go before 1975, unit banking still prevailed in much of the US, trade as a share of GDP was trivial, international capital flows even smaller … and so on. The net is that we’d have 30 years of data, 120 quarters. , even if we have sufficient exogenous variation to identify parameters (which requires agreeing on what’s exogenous!). As Simon Van Norden noted, panel data may help, but institutions vary sufficiently from country to country that it may not help when it comes to macro policy.
With only 120 data points, and recessions and inflations (thankfully) rare events, there are neither enough events nor enough degrees of freedom for coming to agreement on much of anything. The old Cowles Foundation approach, cf. Ray Fair’s continued work at Yale, offers another angle for lessening the data limitations. But that’s out of favor — macroeconomists can’t even decide how to text a model — so we’re in a world where it’s hard for empirics to decide the debate.
Now in micro, well all know that minimum wages … uh. Maybe this issue is generic to the social sciences.
Great post Nick. I think maybe the way to address Vladimir’s interesting distinction is to say that really smart guys can have big egos, and so if there is no way to definitively refute somebody (through repeated, controlled experiments) then the schools of thought reign. I imagine the same thing happens in philosophy.
“But we also lose something when we extend the range like that. Because we have to hope that all countries at all times respond the same way, at least qualitatively.”
I don’t follow you.
I had thought that, for identification, we want the opposite. Furthermore, if we’re deriving our models from “first principles”, they should be generally applicable, right?
Bob, Erik: macro and modern physics actually have quite a lot in common in that neither can do experiments to figure out the stuff they really would like to figure out. We can’t go poke around a black hole, nor can we probe the Planck scale any more than we can do a gold standard study on the effects of macro policy by replicating this universe, changing nothing put policy, and starting the clock (though perhaps a better understanding of physics might help in this area 🙂
Mike’s point on the paucity of data is a great one. That some very fascinating work in economic history has been done to try to put together time series GDP and price data for developed countries in the 19th century or that we’re currently having intelligent discussions on such basic issues of how we measure productivity/capital (which has been recently discussed in Canada) or inflation demonstrates the point that, on one level, we’re still in the process of collecting/identifying the most basic data.
Patrick, if I’m the “Bob” to whom you refer, then I actually do know a decent amount about physics. (But maybe you’re were referring to a different Bob, since I didn’t mention physics.) Anyway, the point works there too: There are rivalries with some of his colleagues e.g. thinking Brian Greene is a prima donna on superstring theory because he can’t be falsified anytime soon, right?
Here’s one for you Nick, from the Washington Post today:
“JP Morgan’s Michael Feroli estimates that the tax hikes and spending cuts that have survived the cliff deal could shave at least 1 percentage point off U.S. economic growth in 2013. We’ll see how that prediction holds up.”
How, exactly, will we see that?
“How, exactly, will we see that?”
Well, if the economy tanks, he was spot on. If the economy booms, it would have grown faster but for the tax hikes and spending cuts, and he was still spot on. Good work if you can get it.
Gregor and Bob Smith: spot on!
Simon: well, we would hope that the signs of the effects of monetary and fiscal policy would be the same across countries and times, but the elasticities could be very different.
Bob Murphy: thanks. Yep, and when you have real policies depending on it, that will affect the wealth and well-being of a lot of people, you could see how people would get even more exercised about it.
@Patrick, re:”Bob, Erik: macro and modern physics actually have quite a lot in common in that neither can do experiments to figure out the stuff they really would like to figure out…”
LOL, you’re joking, right?
I’m an astrophysicist. One reason that banks hire physicists rather than economists for economic modelling is that economists try to pretend to be physicists, whereas physicists know physics so they know that when economists try to pretend to be physicists, they aren’t doing physics.
One real problem with physics especially astrophysics, is that you often cannot do repeatable experiments. The big bang happened once, and you can’t make it happen again. You can only observe stars. You can’t produce them. The other thing that happens is that the rules change. Adding heavy elements to stars greatly changes their behavior, and so recently created stars just act differently than earlier ones. Also while you can classify stars and planets into groups, every star and every planet is different from every other one, to the point where for certain types of stars you can show a spectrum to someone and they can tell you what star it is.
The fact that “things change” means that you just can’t take observations from one era and blindly apply them to another. Physicists seem to be more aware of that than economists. What annoys me are people that assume that there is some sacred scientific method when in fact there are some ad-hoc rules that people use, and it gets interesting because different physicists often have different philosophical foundations and it usually doesn’t matter (i.e. I don’t think string theory is really physics and I’m not alone in thinking that).
Also since my background is computational astrophysics, I think people need to understand the use of simulation in science. You don’t simulate to create an exact copy. You use computer simulations so that you can figure out that if you assume X, Y, and Z then N happens. If it turns out that N is totally at odds with reality, that’s GOOD, because then you figure out which assumption went bad.