When do we need science?

I often get into arguments with people and invariably I end up invoking science in a way to present evidence. Often this invocation of mine is countered with an accusation of, ‘you must admit science isn’t perfect’, where I am forced to admit that this is indeed the case. What is quite frustrating is that I more often find myself arguing against the blind application of science rather than for it, so when this accusation is levied at me, I find that a gross misunderstanding has occurred. With this in mind, I wish to attempt to clarify when the application of science is or is not appropriate or necessary.

Science is NOT appropriate for living, intuition dominates daily life. We do not need science to tell us how much pressure to put on the pencil so the tip does not break, this will be learned by trial and error. Similarly like Nassim Nicholas Taleb likes to say, we do need to ‘lecture birds how to fly’, birds can fly without understanding aerodynamics and the complicated physical laws that accompany them. A snake does not need to understand the mechanics of friction or momentum to learn how to slither. In this very same way science will very often shed light on things that we can already perfectly navigate through without this formalized mumbo jumbo. Religion would also fall in the intuitive domain.

Even science that is ridiculously close to the truth can often be insufficient for taking action due to the Precautionary Principle (a previous topic). Just because we find that something is unlikely does not make it less important, acting upon a hypothesis cannot be weighted without taking into account the magnitude of its implications. It follows from this that  it is worthwhile to invest in preventative measures when the consequences of an event are devastating even if the event is highly unlikely. An application of this is that it could be worth overreacting(a misnomer) to disease contagions such as Ebola and NOT to airplane accidents because the former does have a higher propagation effect (the probability of Ebola killing a million people is much higher than the probability of an airplane accident killing a million people).

With these devastating critiques of science, one might think I’m as anti-science as they come. However, this conclusion would be blatantly false; science is the most systematic way for humanity to approach objective truth. This truth need not imply anything about how we structure society or our lives. Yes, it may be misused but so can every possible freedom afforded to society, indeed this is the very definition of freedom. We avoid restricting freedoms of the masses based on what some people may do , we do not quarantine every individual because some people may be murderers. Free inquiry is a fundamental principle of every open society, the goal of science is to attempt to codify our knowledge, not to tell us how to use it. The fact that each person may infer from it what he wills is the very basis of an individualist society and why we need robust Kantian moral imperatives. This principled stance is vital to the open society so that when an inconvenient truth emerges, we have no need to change our social behavior; This is because our behavior was not derived from scientific principles in the first place but from irrefutable morality.

Science is not perfect and many hypothesis we take as temporarily right (a hypothesis can never be proven) end up being false; however when we are dealing with ideology and things away from intuition, science is how we put our biases in check. Inherently when one discusses ideology based on hypothesis which are out of the intuition of daily life and are contradicted by many other individual experiences, science must be invoked. The very first step to the scientific method outlined by Popper, before ANY empirical work is conducted is to clean up your hypothesis. If your hypothesis predicts the same observations an existing hypothesis predicts then your theory isn’t precise enough to be meaningful, much in the same way the hypothesis of ‘shit happens’ is not a credible theory. Any theory that cannot be disproven is fundamentally flawed and must be discarded or at least not believed in until it can be fleshed out to make observable predictions.

Technical point: The production of evidence is often a function of a function. What this means is that we cannot ignore the bias or effort that goes through the production process. If for instance there are ten thousand ideologically biased researchers who are searching to prove hypothesis x, then by pure randomness we can conjecture that at the 99% level, then there will be 100 studies showing an effect when none exists.

Finally, often we can come up with numerous theories that fit the same observations; invariably this happens in every field, so how do we choose one? The most philosophically consistent position to adopt in such situations is to choose the hypothesis that makes the least assumptions. This is often called Occam’s razor and it is possibly one of the most important elements in the selection of any scientific theory.

 

Evidence vs proof. How to think about Global Warming and GMO’S.

This might be a bit of derivative post because I am just deciding to come back to blogging.

This topic has been resonating in my head over the last couple of weeks as I hear people saying the words “there is proof”. The difference between “proof” and evidence doesn’t appear to be an obvious one so let me try to clarify.

There are two kinds of proofs. Mathematical proofs, and legal proofs. Mathematical proofs are axioms which are internally coherent, there are many levels of sophistication in mathematics and some higher level proofs may not be deemed worthy of the title of “proof” by the purest of mathematicians. Legal proofs on the other hand are not nearly as related to internal coherence. It is merely the nature of the law to categorize things into binary states (guilty or not guilty), it is not meant to indicate a perfectly coherent internal structure but is meant to classify people so that decisions can be made.

The important thing to note here is that science and proofs are unrelated. Science is a negative endeavor, it never deals with saying anything is true, it works only in calling out untruths. When you read any paper that says “we show that there is a relationship between…” what they mean to say is “we cannot reject the fact that there is a relationship between these things…” You cannot show that something is true you can only show that it is untrue. If you do an experiment within a closed system a million times, it is STILL not proven.

You might think that all this means that the correct question is then: “at what point is the evidence strong enough so that we can lean on it to make decisions?” While the volume of evidence does matter, what could matter to a much greater degree is what it implies.

Let’s take global warming as an example. There is some correlational evidence but it is hardly convincing given that we know spurious correlations arise naturally. There are also some simulations but simulations can be made to show anything and always miss some real world nuances (given chaos theory, a half realistic simulation would take years to run on the best super computers). Though everyone treats human driven global warming as “fact” it is far from it. This however is irrelevant, the consequences of global warming, perhaps eventually making the planet uninhabitable to humans are so gargantuan that the standard of evidence we need is as low as it could be. In other words, the consequences of overreacting are comparatively minimal to the consequences of underreacting. Indeed when there is some probability, however small, that the world will end, the burden of proof falls to the other side.

The opposite example is GMO’s (Genetically modified organisms). The question is simple, should we genetically modify food on a mass scale? Well it is still unclear what such modification will do to humans and it could have strange long term interactions (which will almost never be fully controlled for in an experimental setting). Even if the evidence is unclear, this is grossly insufficient because the effects could take any form and if a large population of people were to be involved, the consequences could explode. The burden of proof quite obviously falls on the people pushing GMO’s. Ninety ninth percentile significance levels is not a metric that warrants application, if ninety nine percent of flights were safe, there would be over ten thousand people dying a day from plane accidents.

Technical note: When using Bayesian statistics it is easier to objectively apply this logic because we can just set extreme priors.

Do hospitals make you sicker?

Selection bias is often a very destructive force when trying to determine what works and what doesn’t, it’s often impossible to evaluate if programmes are a success or not unless you’re under something extreme like a totalitarian regime.

Let’s take hospitalization for instance; if you were to ask people coming out of the hospital how their health status was, it would probably be worse for people coming out of the hospital rather than randomly selected people from the population.

So let’s assign some simple terms here then.  First let’s have the treatment dummy:

So its 0 if they don’t get treated and 1 if they do get treated. Then we observe the actual outcome (health status) for each individual with Y.

So to actually measure how much of an effect treatment has we need to compare people receiving treatment to those not receiving it.

Yet the world presents problems since what we actually observe in the real world is:

So there is no one number that represents what actually happens. We can measure the, the average effect of treatment on the treated (ATT), average effect of treatment on the untreated (ATU) or the average treatment effect (ATE).

In public institutions there is this top down selection problem and the more private an institution is the more bias comes from self-selection.

The ATT can tell us the effect of going to the hospital on the people who went to the hospital. The first part of the equation is the observed part, where we measure the health status of individuals who went to the hospital had they went to the hospital. The second part is unobserved because it measures the potential health status of patients who went to the hospital had they not gone to the hospital.

ATT can show us how much people gain from going to the hospital. In the real world private enterprises are much more likely to survive if people notice that there is a positive effect and so the market eliminates a low output hospital. A government hospital on the other hand might have trouble eliminating waste because it will receive customers who might not necessarily think the hospital is any good but will still go just because it is free.

The ATU tells us the expected effect of going to the hospital on someone who did not go to the hospital. The unobserved portion is the effect of the hospital on those who did not go. Second part which we observe is the effect of the hospital on those who went to the hospital.

So it seems pretty obvious that in the real world ATU and ATT are very close to impossible to measure accurately. However what we can measure accurately is the average treatment effect. This is represented by:

To accurately measure this we must make sure to apply a RCT (randomized control trial), in other words randomly allocate if people will go to the hospital or not, and here a paradox arises. We can understand if something works if we randomly allocate it, but if we randomly allocate it we are not maximizing the use of the hospital since we are sending in healthy people. Yet if we don’t randomly allocate it we cannot observe if the hospital is working.

This also is the case with control areas if we decide to do something differently in one country/state/city and leave the rest untouched we can then compare them to see if the area where we applied the new method is better off. So what’s the next step? If they are better off then we mass produce this method to the control areas and we can no longer see if it’s working(think long term effects), and if we don’t mass produce it then those control areas are not benefiting from this new method that could be improving their standard of living.

So the paradox here is between knowing something works versus making it work for everyone. Whether we apply a randomized control trial or use control groups, chances are we are not helping out the best we can, and if we do not apply these methods then we are ignorant as to whether the program is helping people at that specific period in time. To properly understand treatment effects we need a sacrificial lamb to exist. Though what’s generally done is to assume that if something worked in the past, it will keep doing so.