40 / 48
AIQS
News
75
40
in answer to the question of whether
an agent can still “think for himself”
after indoctrination. And what is it
to be able to think for ourselves? We
must be capable of being “moved by
reasons”; that is, we must be reasonable
and accessible to rational persuasion,
the introduction of new evidence, and
further considerations. If we are more
or less impervious to experiences that
ought to influence us, our capacity has
been diminished.
The only evidence that HAL might be
in such a partially disabled state is the
much-remarked-upon fact that he has
actually made a mistake, even though
the series 9000 computer is supposedly
utterly invulnerable to error. This
is, to my mind, the weakest point in
Clarke’s narrative. The suggestion that
a computer could be both a heuristically
programmed algorithmic computer
and “by any practical definition of the
words, foolproof and incapable of error”
verges on self-contradiction. The whole
point of heuristic programming is that
it defies the problem of combinatorial
explosion—which
we
cannot
mathematically solve by sheer increase
in computing speed and size—by taking
risky chances, truncating its searches
in ways that must leave it open to error,
however low the probability. The saving
clause, “by any practical definition of the
words,” restores sanity. HALmay indeed
be ultra-reliable without being literally
foolproof, a fact whose importance
Alan Turing pointed out in 1946, at the
dawn of the computer age, thereby
“prefuting” Roger Penrose’s 1989
criticisms of artificial intelligence.**
(See my
Darwin’s Dangerous Idea
, chapter
15, for the details.)
The Monolith
**The verb prefute, coined in 1990,
was inspired by the endearing
tendencyof psichologist TonyMarcel
to interrupt conference talks by
leaping to this feet and exclaiming,
“I can see where your argument is
heading and here is what is wrong
with what you’re going to say”
Marcel is the master of prefutation,
but he is not its only practitioner.