AI, Armageddon, and Alice’s House of Cards

​

“No, no!” said the Queen. “Sentence first; verdict afterwards.”

“Stuff and nonsense!” said Alice loudly.

“The idea of having the sentence first!”

​

     Recently, a report on the dangers of AI commissioned by the U.S. State Department made a profound assertion. As AI “approaches human- and superhuman levels of capability across a wide (but as-yet unknown) range of tasks,” it warned, “it may become uncontrollable….” This will occur, it cautioned, “if its capabilities exceed a certain (currently unknown) threshold.” The parentheses are in the original. The recommendation was to take immediate steps to curtail the questionable emergence of these unknown capabilities beforehand.

​

     This is otherwise known in the philosophical community as “a pig in a poke” or, as the Queen of Hearts put it, “Sentence first!”.

​

     We’re not sure exactly what AI may become capable of doing that’s beyond what humans can do, but it certainly shouldn’t. We have no idea how that might occur, but it certainly mustn’t. This unknown and unpredictable danger lies in the acquisition of AGI—Artificial General Intelligence—by AI beings. AGI is an artificial form of consciousness (self-awareness) and sentience (ability to sense the world) that humans and many other animals possess. It requires an intimate connection between the organism and its environment.

​

     AI developers are in three camps of opinion on this: AGI is a) an imminent achievement, b) a very distant possibility, or c) never achievable. Only 1% agree with the last option. I believe the others read a lot of science fiction, and the 1% have a pretty decent understanding of biology.

​

     But all of this aside, the report argues that our government

should put up safeguards to keep the development of AGI

from happening until we know how to control it. We don’t know

exactly how to do this, since we can’t say how such a develop-

ment could happen and what it would look like if it did.

​

“Off with her head!” shouted the Queen.

“You’re nothing but a pack of cards,” said Alice.

​

     At this, the whole pack rose up into the air and came flying down upon her….

And Alice woke up in the real world.

​

Ron Wetherington