Joel Pust – Induction, Focused Sampling and the Law of Small Numbers

October 28, 2012 — Leave a comment

Notes from: Joel Pust – Induction, Focused Sampling and the Law of Small Numbers

This paper is a response to Kornblith‘s defense of the use of the Law of Small Numbers (Judgmental bias which occurs when it is assumed that the characteristics of a sample population can be estimated from a small number of observations or data points). Pust is claiming that this argument fails for the following reasons: the sort of inferences Kornblith seeks to justify are not really inductive inferences based on small samples, but rather knowledge based deductive inferences, and secondly that Dorrit Billman’s computational model upon which Kornblith builds some of his argument is not sufficient for this purpose.

Firstly, Kornblith’s defense of the use of the Law of Small Numbers is characterized as reliabilist.

“We have built-in biases in our processing of visual information and such presuppositions bring perceptual errors with them. However, the simple fact that our perceptual mechanisms are biased has no implications for their reliability without the further claim that such biases are inappropriate for our environment.” (Pust, p90)

Our inferential reasoning, like our perceptual mechanisms, whilst prone to making mistakes is nevertheless generally quite relaible. Our inferential mechanisms depend upon the assumption that the world contains natural kinds. With regard to the Law of Small Numbers, Kornblith cites Tversky and Kahneman (K, p90) claiming that we tend to draw inductive inferences on the basis of extraordinarily small samples. The key question becomes how to evaluate this tendency (K, p90). Tversky and Kahneman assert that this tendency is inappropriate, whereas Kornblith points out the question of reliability is far more subtle and complex.

1) Is presdiction from a small sample always unreliable?
2) Is the logic of statistical inference a reasonable standard against which to measure the appropriateness of our inferences?

” […] when a population is uniform with respect to a certain property, a generalisation based on a single case will be reliable indeed.” (Pust, p92)

Imagine that you are observing an unknown species of bird lay an egg. Based on this one observation, and the background knowledge that all birds in existence that have been observed thus far lay eggs, you can infer based on one very small sample that this unknown bird will always lay eggs. This ties in with Quine’s notion of projection. Projection seems to tie in with background knowledge to provide a solid base upon which to infer based on small samples.

Kornblith: we do have “a sensitivity to those features in objects which tend to reside in homeostatic clusters; and a tendency to project those characteristics which are indeed essential to the real kinds in nature.” (K, p95)

This also ties in with the ‘naturalisation’ of Kornblith’s epistemology. Kornblith’s reliabilist argument rejects the notion that the only justified inferential procedures are those that are relaible in any possible world. The justifiedness of a belief is a function of the actual world: actual world reliability of the process (evolution) that produced the justified inferential procedures.


Pust’s summary of Kornblith’s argument:

“The Aim: To provide a reliabilist defense of TLSN.
(1) Inductive generalizations based on small samples (or the single case) will be reliable if the features selected do, in fact, generally co-occur.
(2) In order for such an inferential tendency to be reliable, then, we must possess a sensitivity to those properties of natural kinds that are highly correlated. In other words, we must be able to detect what property correlations obtain.
(3) Though some experimental data shows that we are rather poor at detecting covariation when a single pair of properties covaries, Billman’s research on focused sampling shows that we are good at detecting property covariation when the properties in question also covary with a number of others that jointly covary. In short, when properties are ‘clustered’, we are quite adept at detecting their correlations by engaging in focused sampling.” (Pust, p95)

The main criticism, referring to the bird example above, is that Kornblith;s assertion of small sample sizes misses altogether crucial premises. i.e ‘new species of bird lays an egg’ (based on 1 observation) needs to be a conjunct with ‘all members of a biological species reproduce in the same manner’, such that:

premise: An observed new species of bird lays an egg.
premise: All members of a biological species reproduce in the same manner.
concl: New species of bird will always reproduce via laying eggs.

This is a deductive inference, even if the premise themselves were arrived at inductively; knowledge based deduction. This could be construed as ‘prior knowledge’ plus ‘observations’ equals ‘new knowledge’.

Kornblith’s descriptors ‘sensitivity’ and ‘intuitive grasp’ for his assertion of our evolved inductive reasoning mechanisms in this new light possibly, and most likely, correlate with background knowledge. Pust redesign’s Kornblith’s argument such that we have inductive inferences drawn from the law of large numbers (we develop knowledge of the world based on large scale observations) and then based on this background knowledge are able to make deductive inferences on the basis small samples.


No Comments

Be the first to start the conversation!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s