Enlarge /. Senator John Cornyn (middle, black suit) takes a photo with the 2017 astronaut class.
On Friday, Texas Senator John Cornyn posted some advice for scientists on Twitter: Models are not part of the scientific method. Scientists have responded with a mix of confusion and anger. And Cornyn's misunderstanding is common enough – and important enough – that it's worth exploring.
After the crisis in #COVID ー 19, could we discuss in good faith the use and abuse of "modeling" to predict the future? Everything from public health to economic to climate predictions. It's not the scientific method, people. https://t.co/OYBm3CIUxX
– Senator John Cornyn (@JohnCornyn) April 10, 2020
Cornyn's beef with models reflects a point of discussion that is often raised by people who want to reject inconvenient conclusions from systems science. In reality, "you can get a model to say anything you want" is as strong an argument as "all swans are white". The latter is either an insincere argument, or you have an embarrassingly limited familiarity with swans.
Models are not perfect. You can generate inaccurate predictions. You can generate highly uncertain predictions when science is uncertain. And some models can be really bad and provide useless and poorly supported predictions. However, the idea that models are not central to science is profoundly wrong. It is true that criticism usually focuses on mathematical simulations, but this is only one type of model in a spectrum – and there is no science without models.
What is a model to do?
Scientific thinking has something fundamental – and indeed most of the things we control in everyday life: the conceptual model. This is the picture that exists in your head of how something works. Regardless of whether you're testing a bacterium or microwaving a burrito, refer to your conceptual model to get what you're looking for. Conceptual models can be extremely simple (turnkey, engine start) or extremely detailed (basic knowledge of all components in your car's ignition system), but are useful in both cases.
Since science is a knowledge-seeking endeavor, it is about developing better and better conceptual models. While the interaction of model and data can take many forms, most of us learn a kind of laboratory-oriented scientific method that consists of hypotheses, experiments, data and revised hypotheses.
In a now famous lecture, quantum physicist Richard Feynman similarly described to his students the process of discovering a new law in physics: “First we guess it. Then we calculate the consequences of the presumption to see what (…) this would mean. And then we compare these calculation results with nature (…) If it doesn't match the experiment, it's wrong. The key to science lies in this simple statement. "
To “calculate the consequences of the presumption”, you need a model. A good conceptual model will suffice for some phenomena. For example, one of the basic principles taught to young geologists is T.C. Chamberlin's "Method of Multiple Working Hypotheses". He advised all geologists in the field to consider more than one hypothesis – built into full conceptual models – when walking around and making observations.
In this way, rather than simply summarizing all observations that match your preferred hypothesis, the data can more objectively highlight those that are closer to reality. The more detailed your conceptual model is, the easier it is for an observation to show that it is wrong. If you know where a particular layer of rock should appear and it does not exist, there is a problem with your hypothesis.
It's about math
But at some point the system under investigation becomes too complex for a human being to calculate the consequences in his own head. Enter the mathematical model. This can be as simple as a single equation that is solved in a table, or as complex as a multi-layer global simulation that takes supercomputer time to run.
And here is the modeler's saying, made by George E.P. Box comes in: "All models are wrong, but some are useful." Any mathematical model is necessarily a simplification of reality and therefore probably not complete and perfect in every respect. But perfection is not their job. Its job is to be more useful than no model.
Consider an example from a science that produces few party-political arguments: hydrogeology. Imagine a leak was discovered in a storage tank under a gas station. The groundwater level here is close enough to the surface that petrol has contaminated the groundwater. This contamination must be mapped to see how far it has traveled and (ideally) to facilitate cleaning.
If money and effort didn't matter, you could drill a thousand surveillance holes into a grid to find out where it was going. Obviously nobody does that. Instead, you could drill three wells near the tank to determine the properties of the soil or bedrock, the direction of the groundwater flow, and the concentration of contaminants near the source. This information can be incorporated into a groundwater model that is simple enough to run on your laptop. It simulates likely flow rates, chemical reactions, and microbial activities that degrade the contaminants, and so on, spitting out the likely location and extent of the contamination. That's just too much math to do everything in your head, but we can quantify the relevant physics and chemistry and let the computer do the heavy lifting.
A truly perfect model prediction would more or less require knowledge of the location of each grain of sand and quarry below the station. However, a simplified model can generate a helpful hypothesis that can easily be tested with just a few more monitoring holes – certainly more effective than drilling on a hunch.
Don't shoot the modeler
Of course, Senator Cornyn probably had no groundwater models in mind. The tweet was triggered by working with epidemiological models that project the effects of COVID-19 in the United States. Recent models that incorporate the social distancing, testing, and treatment measures used to date predict fewer deaths than previous projections. Instead of welcoming this sign of progress, some have inexplicably attacked the models, claiming these downward corrections show that previous warnings have exaggerated the threat and led to excessive economic effects.
This argument ignores a startlingly obvious fact: previous projections showed what would happen if we didn't take a strong response (as well as other scenarios), while new projections show where our current path leads us to. The downward correction does not mean that the models were bad. it means we did something.
Often the social value of the scientific "what if?" Model is that we want to change the "if". Calculating how quickly your bank account goes to zero when you buy new pants every day can change your overly ambitious wardrobe procurement plan. That's why you cracked the numbers in the first place.
Unfortunately, complaints about “exaggerated models” are predictable. All the excitement about a hole in the ozone layer and it turned out to be no longer growing! (Because we have banned the production of the responsible pollutants.) Acid rain should be a disaster, but I haven't heard from it in years! (Because we needed pollution control for sulfur-emitting chimneys.) The worst scenario of climate change was over 4 ° C by 2100, and now they're getting closer to 3 ° C! (Because we took steps to reduce emissions.)
These complaints seem to view models as crystal balls or psychic visions of a future event. But they are not. Models simply take a scenario or hypothesis that you are interested in and "calculate the consequences of the presumption". The result can be used to promote scientific understanding of how it works or to make important decisions.
After all, what is the alternative? Could science reject models in favor of another method? Imagine what would happen if NASA eyed Mars in a telescope, aimed at the missile, pressed the start button and hoped for the best. Or maybe humanity could base its response to climate change on someone waving their hands at the atmosphere and saying, "I don't know, 600 ppm carbon dioxide doesn't sound like much."
Obviously, these are not alternatives that a sensible person should seriously consider.
The spread of COVID-19 is an incredibly complex process and difficult to predict. It depends on some things that are well studied (how pathogens can spread between people), some that are partially understood (like the properties of the SARS-CoV-2 virus and its lethality), and some that are not known (like the exact movements and actions of each individual American). And it has to be simulated across the country on a relatively small scale if we want to understand the ability of hospitals to meet local care needs.
Without computer models, we would be reduced to spit-balling on the back of the envelope – and even that would require conceptual and mathematical models for individual variables. The reality is that great science requires big models. Those who pretend otherwise are not defending a "pure" scientific method. You just don't understand science.
We cannot undress the science of models more than knowledge.