This might be a bit of derivative post because I am just deciding to come back to blogging.
This topic has been resonating in my head over the last couple of weeks as I hear people saying the words “there is proof”. The difference between “proof” and evidence doesn’t appear to be an obvious one so let me try to clarify.
There are two kinds of proofs. Mathematical proofs, and legal proofs. Mathematical proofs are axioms which are internally coherent, there are many levels of sophistication in mathematics and some higher level proofs may not be deemed worthy of the title of “proof” by the purest of mathematicians. Legal proofs on the other hand are not nearly as related to internal coherence. It is merely the nature of the law to categorize things into binary states (guilty or not guilty), it is not meant to indicate a perfectly coherent internal structure but is meant to classify people so that decisions can be made.
The important thing to note here is that science and proofs are unrelated. Science is a negative endeavor, it never deals with saying anything is true, it works only in calling out untruths. When you read any paper that says “we show that there is a relationship between…” what they mean to say is “we cannot reject the fact that there is a relationship between these things…” You cannot show that something is true you can only show that it is untrue. If you do an experiment within a closed system a million times, it is STILL not proven.
You might think that all this means that the correct question is then: “at what point is the evidence strong enough so that we can lean on it to make decisions?” While the volume of evidence does matter, what could matter to a much greater degree is what it implies.
Let’s take global warming as an example. There is some correlational evidence but it is hardly convincing given that we know spurious correlations arise naturally. There are also some simulations but simulations can be made to show anything and always miss some real world nuances (given chaos theory, a half realistic simulation would take years to run on the best super computers). Though everyone treats human driven global warming as “fact” it is far from it. This however is irrelevant, the consequences of global warming, perhaps eventually making the planet uninhabitable to humans are so gargantuan that the standard of evidence we need is as low as it could be. In other words, the consequences of overreacting are comparatively minimal to the consequences of underreacting. Indeed when there is some probability, however small, that the world will end, the burden of proof falls to the other side.
The opposite example is GMO’s (Genetically modified organisms). The question is simple, should we genetically modify food on a mass scale? Well it is still unclear what such modification will do to humans and it could have strange long term interactions (which will almost never be fully controlled for in an experimental setting). Even if the evidence is unclear, this is grossly insufficient because the effects could take any form and if a large population of people were to be involved, the consequences could explode. The burden of proof quite obviously falls on the people pushing GMO’s. Ninety ninth percentile significance levels is not a metric that warrants application, if ninety nine percent of flights were safe, there would be over ten thousand people dying a day from plane accidents.
Technical note: When using Bayesian statistics it is easier to objectively apply this logic because we can just set extreme priors.