Part 1. Section 4b

[click here to go to Section 4a]

4.3 Argument from ignorance, inability to be falsified, post hoc ergo propter hoc, biased samples

 

Argument from ignorance

Suppose that a person notes that there is no support for a given claim - which means informally that there are no stated reasons why the claim is true. When this occurs it is tempting to make an argument from ignorance fallacy. To illustrate this, suppose for example that you are evaluating the use of 'cupping' as a method to relieve muscle pain and relax muscles. You note that there are no real arguments either for or against the practice. Emily is inclined to think it works and make the following argument in its favor.

Since there is no evidence showing that cupping doesn't work, the we are safe to assume it does.

However John, who is rather skeptical uses the same lack of evidence about cupping to make the following counter-argument:

Since there is no evidence showing that cupping works, we can assume it does not.

Notice that the absence of evidence about cupping is used to support a conclusion and to argue against it at the same time. This particular fallacy, when it occurs is known as the argument from ignorance.

argument from ignorance

Definition: When a claim is said to be true or false because the claim has not been proven, or from the absence of any evidence concerning the truth of the claim, then the resulting error in reasoning is known as the argument from ignorance fallacy.

 


Inability to be falsified

The fact that we argue for or against certain claims is an indication that there are conditions that make the claim true and other conditions that make the claim false. To argue for the truth of a claim then is to argue that the conditions that make it true have actually occurred in some sense of the word.

With this is mind, let us re-visit the valid argument form modus tollens:

 

If p then q

not q

Therefore not p

 

To help make our explanation more clear, let's consider an actual instance of modus tollens with replacement instances for both p and q.

 

If a creature like big-foot exists, then there will be ample physical evidence of the creature's existence.

There is not ample physical evidence of the creature's existence.

Therefore big-foot does not exist.

Now suppose someone makes the following (invalid) argument:

 

If a creature like big-foot exists, then there will be ample physical evidence of the creature's existence.

There is not ample physical evidence of the creature's existence.

But big-foot exists anyway.

Notice that the argument ignores the evidence that big-foot does not exist, which is implied by the two premises, and instead refuses to accept that the big-foot hypothesis is false. When this happens in the extreme, such that no evidence is accepted that shows the falsity of some claim, the resulting fallacy is known as the inability to be falsified fallacy.

Here is an another example:

Tara Marie is certain that she has psychic powers, and claims she can predict the future with 100% accuracy. And indeed, sometimes her predictions seem to come close to what actually happens, but most of the time they don't. However, this fact does not disuade Tara Marie from her belief in her ability to predict the future. If she predicts something that does not come to pass, then she dismisses the saying by claiming it was not a prediction anyway, or states that it will still happen, but in a more distant future.

Notice that in this case, the evidence that a prediction did not happen is evidence that the prediction is false. We could roughly re-word this observation the following way:

 

If Tara Marie is able to predict the future with 100% accuracy, then those predictions will always come true.

Those predictions do not always come true.

Hence Tara Marie is not able to predict the future with 100% accuracy.

 

But Tara Maries rejects this line of reasoning and argues more or less this way:

 

If I am able to predict the future with 100% accuracy, then those predictions will always come true.

Those predictions do not always come true.

(excuses given)

So I am still able to predict the future with 100% accuracy.

inability to be falsified

Definition: When an inference ignores evidence that shows that a claim is false and asserts that the claim is true under all falsifying cases, then the resulting error in reasoning is known as the inability to be falsified fallacy.

 


Post hoc ergo propter hoc

Consider the following generally true set of claims:

 

In each case a certain action or event is said to lead to another event. In these cases, the word 'then' in the conditional statement has a temporal meaning also, and could be replaced by the phrase 'then after some time'. Conditional statements of this type are said to be causal statements. Such statements can be reformulated to have the general abstract form:

 

'X is a cause of Y'

Such that the first statement would be reformulated to be:

 

'Laying out in the sun too long without sun-screen protection is a cause of getting sunburn.'

 

The causes of events are an important part of reasoning, and trying to understand which events cause other events is a major goal in applied statistics. As important as causal reasoning is, it should come as no surprise that there are fallacies related to causality. To motivate our examination of just one of these fallacies, let's consider the list of statements above. In each case the cause of an event always happens before the event itself. This is so grounded in our thinking, that there exists a statement whose truth seems to be self evident: 'CAUSES PRECEDE EFFECTS', which is just another way of saying that if X is the cause of Y, then X must come before Y in time. However we know that temporal order alone is not enough to establish a causal relationship. Consider the following arguments.

"We had a very normally developed child, meeting all the marks as a child – walking, eye contact … and after his 18-month vaccination we had a very different child,"

 

Bexar County District Attorney Nico LaHood: 'Vaccinations Can and Do Cause Autism'" San Antonio Express-News. N.p., n.d. Web. 29 Aug. 2016. <http://www.mysanantonio.com/news/local/article/Bexar-County-District-Attorney-Nico-LaHood-9190563.php>

 

 

The rain and thunder stopped right after we shot bottle-rockets into the air and banged the drums for five minutes, hence we caused the the thunder and rain to stop by our actions.

 

In each case an error in reasoning occurs when one assigns the cause X of some effect Y on the basis of that X happens before Y alone. Such errors of reasoning are know as post hoc ergo propter hoc (lat. after this, therefore caused by this) fallacies.

 

post hoc ergo propter hoc

Definition: When X is asserted to be the cause of an event Y on the sole basis that X comes before Y in time, then the resulting error in reasoning is known as the post hoc ergo propter hoc (or just post hoc) fallacy.

 


Biased sample fallacies

In a previous section we noted that the principle of induction is an invaluable tool for reasoning. The principle proceeds from the observations of many cases and makes an inference that future cases will be similar. Strong induction assumes that under appropriate circumstances the resulting inference is strong. It is now time to discuss a small list of cases where such inferences are inappropriate. Let's consider a fictitious story which illustrates this:

Suppose you are a worker for WHO (the World Health Organization), and you are tasked to go to villages in a distant country with both a language and customs unfamiliar to you and assess the health status of the general population by doing a survey which poses written questions about the health of those who take the survey. You discover that in each village there is usually one building where people congregate on a daily basis, and you decide to make your job easier by taking all of your survey questions in this village locale. The end result of your survey is that a large majority of the villagers are quite unhealthy as almost all reported being unwell at the time of the survey. You send your report to back to the WHO headquarters - who seem skeptical about the results, since other surveys in the past have not reported such high rates of illness. They send a person to investigate who knows the language and culture well, and - as it happens - the village gathering place where you conducted all of your surveys turns out to be a local health clinic. Since people go to health clinics for reasons of ill health, it is no surprise that your survey data reflected this fact. But the inference from the data taken there to the entire population is a bad inference, since there was a bias in the data resulting in the fact that by its very nature and purpose, people in the locale were more likely than not to be sick or to be accompanying someone who is sick.

The above story introduces the concept of a bias in the data used to draw inductive inferences. Here a bias is any condition which causes the data used in an inductive inference not to be representative of the whole. Think of biases as 'filters' - where filters can remove or even add properties which are not characteristic of the whole. In the above example the filter was the village locale was a health clinic which filtered out healthy members of the village population.

Another famous real-life example of bias deals with the unexpected results of the 1948 Presidential Election, where the bias in this case was human intervention in the form of quotas which ignored important questions such as whether the candidate interviewed was a likely voter, the following is a brief description of the famous Dewey-Truman surprise!

Case Study 2: The 1948 Presidential Election

Soon after the 1936 fiasco, the Literary Digest went out of the business of polling. In fact, they went out of business altogether. At the same time, the practice of using public opinion polls to measure the pulse of the American electorate was thriving. By 1948 there were several major polls competing for the big prize, that of accurately predicting the outcome of presidential elections. The best known was the Gallup poll, and its two main competitors were the Roper poll and the Crossley poll.

By this time, all major polls were using what was believed to be a much more scientific method for choosing their samples called quota sampling. Quota sampling had been introduced by George Gallup as early as 1935 and had been successfully used by him to predict the winner of the 1936,1940 and 1944 elections. Quota sampling is nothing more than a systematic effort to force the sample to fit a certain national profile by using quotas: The sample should have so many women, so many men, so many blacks, so many whites, so many under 40, so many over 40 etc. The numbers in each category are taken to represent the same proportions in the sample as are in the electorate at large.

If we assume that every important characteristic of the population is taken into account when setting up the quotas, it is reasonable to expect that quota sampling will produce a good cross-section of the population and therefore lead to accurate predictions. For the 1948 election between Thomas Dewey and Harry Truman, Gallup conducted a poll with a sample size of about 3250. Each individual in the sample was interviewed in person by a professional interviewer to minimize nonresponse bias, and each interviewer was given a very detailed set of quotas to meet. For example, an interviewer could have been given the following quotas: seven white males under 40 living in a rural area, five black males under 40 living in a rural area, six black females under 40 living in a rural area, etc. Other than meeting these quotas the ultimate choice of who was interviewed was left to each interviewer.

Based on the results of this poll, Gallup predicted a victory for Dewey, the Republican candidate. The predicted breakdown of the vote was 50% for Dewey, 44% for Truman, and 6% for third-party candidates Strom Thurmond and Henry Wallace. The actual results of the election turned out to be almost exactly reversed: 50% for Truman, 45% for Dewey, and 5% for third-party candidates.

Truman's victory was a great surprise to the nation as a whole. So convinced was the Chicago Tribune of Dewey's victory that it went to press on its early edition for November 4, 1948 with the headline:

"Dewey defeats Truman" -- a blunder that led to Truman's famous retort "Ain't the way I heard it." The picture of Truman holding aloft a copy of the Tribune has become part of our national folklore. To pollsters and statisticians, the results of this election were a clear indication that as a method for selecting a representative sample, quota sampling can have some serious flaws.

The basic idea of quota sampling is on the surface a good one: Force the sample to be a representative cross-section of the population by having each important characteristic of the population proportionally represented in the sample. Since income is an important factor in determining how people vote, the sample should have all income groups represented in the same proportion as the population at large. Ditto for sex, race, age, etc. Right away we can see one of the potential problems: Where do we stop? No matter how careful one might be, there is always the possibility that some criterion that would affect the way people vote might be missed and the sample could be deficient in this regard.

An even more serious flaw in the method of quota sampling is the fact that ultimately the choice of who is in the sample is left to the human element. Recall that other than meeting the quotas the interviewers were free to choose whom they interviewed. Looking back over the history of quota sampling, one can see a clear tendency to overestimate the Republican vote. In 1936, using quota sampling, Gallup predicted the Republican candidate would get 44% of the vote, but the actual number was 38%. In 1940 the prediction was 48% and the actual vote was 45%; in 1944 the prediction was 48% and the actual vote was 46%. But in spite of the errors, Gallup was able to predict the winner correctly in 1936, 1940 and 1944. This was merely due to luck -- the spread between the candidates was large enough to cover the error. In 1948 Gallup, and all the other pollsters, simply ran out of luck. It was time to ditch quota sampling.

The failure of quota sampling as a method for getting representative samples has a moral: Even with the most carefully laid plans, human intervention in choosing the sample is always subject to bias.

 

DeTurck, Dennis. "Case Study 2: The 1948 Presidential Election." Case Study 2: The 1948 Presidential Election. University of Pennsylvania, n.d. Web. 20 Aug. 2016. <https://www.math.upenn.edu/~deturck/m170/wk4/lecture/case2.html>.

 

There are many biases which can skew the data used in making inductive inferences. As stated we will focus on just two. To do so, reconsider the following argument taken from Section 2.

 It is a good idea to make sure you have working fire alarms in your house. Just look at the family whose house burned down on Christmas. They lost everything in their house, and almost lost their daughter who nearly died of smoke inhalation. Their house did not have working fire alarms, and for that reason, the fire itself went unnoticed while the family slept. They were saved only by the chance occurrence of a neighbor's teenage son arriving home late from a Christmas-eve date who noticed the fire and woke the family up.

We now pose the following two questions:

  1. Suppose someone notices an extremely rare event, and draws a general conclusion about the rare event, are such inferences strong inferences?
  2. Does the above argument provide good reasons to accept the conclusion that it is a good idea to make sure you have working fire alarms in your house?

If you said 'no' to question 1 and 'yes' to question 2 then you are certainly not alone, but these answers are inconsistent. First observe that the conclusion that it is a good idea to make sure you have working gire alarms in your house is supported by a rare instance (since house fires are rare as a percentage of the general population). Given this fact, why then answer the second question as 'yes'? The most probable reason is that most accepted the conclusion about having fire alarms in one's house as being true before reading the argument and then took the argument as a confirmation of what they already believed to be true. When this happens the error in reasoning is known as confirmation bias.

The second type of bias (also present in the above argument) has to do with the sample size of an inductive inference. Suppose you go to a new country and see one family eating grapes fried in butter, and conclude that eating butter-fried grapes is a common practice in that county - then you are using one piece of data to draw and inference concerning what is true of most of the inhabitants of the country. This type of bias - based on small percentages of the whole population - are called hasty generalization.

biased samples: confirmation bias and hasty generalization

Definition: When an inductive inference is made from a biased sample (a set of data not representative of the whole), then the error in reasoning is known as a biased sample fallacy. In particular when the bias is caused by previous convictions about what is true or false, the fallacy is known as confirmation bias. On the other hand if the bias is caused by looking at only a few samples and drawing a conclusion about the whole based on those few samples, the fallacy is known as a hasty generalization.

 


Test your ability to recognize the fallacies covered in Section 4.3 by taking the following quiz:

quiz: fallacies 4

 

Choose the fallacy.