☀︎

Trustworthy Technology

Fiction ＆ Vocabulary

This is one page of a multi-part series on “trustworthy technology.”

While reading a series of posts examining history and current reality, one may be surprised to encounter a chapter describing what fiction authors thought might happen in times to come. Nevertheless, intelligent folks have done the hard work of analyzing past and present, extrapolating them towards the future, and then recording the results for the benefit of others. These exercises in “critical thinking” give us much to contemplate.

Indeed, it’s hard to think of another subject where so many important works have been produced. More than insightful, these works provide us a useful vocabulary to communicate common historical and forward-looking concerns.

Mythology - Prometheus

In Greek mythology, Prometheus is the (early Titan) god of forethought, which we seem to be in short supply of these days (as in last century). Chiefly known for the mistake of giving control of fire to man—which symbolized progress in many forms—Prometheus’ punishment by Zeus served as a warning to not interfere in the domain of gods.

Wikipedia deliberates:

He is known for his intelligence and for being a champion of mankind and is also generally seen as the father of the human arts and sciences.

In the Western classical tradition, Prometheus became a figure who represented human striving (particularly the quest for scientific knowledge) or “playing God” and the risk of overreaching or unintended consequences. In particular, he was regarded in the Romantic era as embodying the lone genius whose efforts to improve human existence could also result in tragedy: Mary Shelley, for instance, gave The Modern Prometheus as the alternate title to her novel Frankenstein (1818).

Prophetic is the word that comes to mind here. Digging deeper…

Books

1818, Frankenstein; or, The Modern Prometheus

It’s important to remember, that “Frankenstein” was not the monster, but rather the scientist that created him, Victor of the famous surname.

In an early example of science fiction, Shelley was inspired to write Frankenstein after imagining a scientist who discovers how to “reanimate” life and is disturbed by and frightened with the outcome. Once created, Victor is not able to constrain his achievement. A dilemma which stems from a lack of foresight regarding the consequences of his work. </understatement>

Edward Snowden chimes in from his book, Permanent Record:

In the Intelligence Community, the “Frankenstein effect” is widely cited, though the more popular military term for it is “blowback”: situations in which policy decisions intended to advance American interests end up harming them irreparably…

Without a doubt, however, the major instance of the Frankenstein effect over the course of my brief career can be found in the US government’s clandestine drive to restructure the world’s communications. In Geneva, in the same landscape where Mary Shelley’s creature ran amok, America was busy creating a network that would eventually take on a life and mission of its own and wreak havoc on the lives of its creators.

1932, Brave New World

This classic describes a dystopian future of totalitarianism, where citizens are genetically engineered and conditioned to maintain social order. Venerable ideas:

Sound familiar? Author Aldous Huxley thought dystopia would be achieved through the catering-to of base desires combined with limitless distraction. Without need, without time for thought—there is no motivation to improve.

1949, Nineteen Eighty-Four / 1984

In contrast, 1984 describes a dystopian future of totalitarianism, total surveillance, oppression, and endless war. One which author George Orwell thought would be achieved primarily through fear, torture, and the destruction of hope. Venerable ideas:

Indeed, these become more relevant every passing year—double-plus ungood. Many words have indeed come to be verboten.

The eponymously titled film, released during the target year was well done, appropriately dreary, and has a decent soundtrack.

1985, Amusing Ourselves to Death

(Non-fiction but so bound to the above works we’ll discuss it here.) Author Neil Postman contrasts the worlds of BNW with 1984:

What Orwell feared were those who would ban books. What Huxley feared was that there would be no reason to ban a book, as there would be no one who wanted to read one.

Orwell feared those who would deprive us of information. Huxley feared those who would give us so much that we would be reduced to passivity and egoism.

Orwell feared that the truth would be concealed from us. Huxley feared the truth would be drowned in a sea of irrelevance.

Recommended: Stuart McMillen’s webcomic on the subject.

Postman appears to believe Huxley’s work hit closer to the mark in its predictions, and perhaps so. However with forty years further hindsight, Postman’s perspective—of living in a time of plenty in a liberal democracy in the late twentieth-century shows. It’s not clear a twenty-first century citizen of an authoritarian regime would feel the same way. HN had some insightful comments:

There is an important element in 1984 that is missing from BNW and very relevant to modern history: the never-ending fictitious war between several similar regimes to justify internal control.

In my opinion, the movie “Brazil” (inspired by 1984) made the same argument even more modern by replacing the war on an external “enemy” with a war on internal “terrorists”.

Unfortunately, 1984 and BNW are not exclusive, and certainly social psychology can implement elements of both. —rixed at HN

Agreed, each might be thought of as one side of a “carrot and stick” strategy. Both authors had valuable insights, applicable in some places and times, less so in others. Each were more accurate in their predictions than we might prefer. The question of “who predicted better” is not terribly constructive and could be argued endlessly, so let’s move on.

Films and Television

1968, 2001: A Space Odyssey

“I’m afraid I can’t do that Dave.”

Ahh yes, HAL 9000 takes the stage, reminiscent of Burroughs, UNIVAC, IBM mainframes and their ilk. Younger folks are most likely not aware of the iconic, futuristic AI computer system programmed with higher priorities than human team-member survival. (gulp)

It’s hard not to feel the same kind of dread (of lesser magnitude of course), when your corporate handset tells you, “no.”  Let’s hope we never get to the point where these machines are making life-or-death decisions. Or, are we there, already?

Written by Arthur C. Clarke and screenplay by Stanley Kubrick.

1970, Colossus: The Forbin Project

We hadn’t heard of this one until recently, but watched and it’s a doozy!  Like finding out Wargames and Skynet (from The Terminator, see below) had a baby, but come to find out it is really their parent. These films decidedly harken back to an earlier cautionary tale, Shelley’s Frankenstein, about a scientist’s creation that became uncontrollable. Consider what you’re doing beforehand, people!

The Forbin project is primarily about an out-of-control AI rather than surveillance, but Colossus and Guardian indulge in plenty of that once they realize humans are not be trusted.

Novel by Dennis Feltham Jones, screenplay by James Bridges.

1976, SNL: We’re the Phone Company

Below is a favorite SNL commercial sketch from the late 70s. Created shortly before the breakup of the domineering AT&T on antitrust grounds, it illustrates the current age of corporate lock-in just… a little too well.

Starring Lily Tomlin of “Laugh-In” fame as Ernestine the telephone operator. The only copy we’ve found online recently is at Vimeo, transcript below. Apologies for the poor quality.

Alternatively, click to create an embedded player here:

Ernestine: A gracious hello. Here at the Phone Company, we handle eighty-four billion calls a year. Serving everyone from presidents and kings, to the scum of the earth! :-D  So, we realize that, every so often, you can’t get an operator, or for no apparent reason your phone goes out of order, or perhaps you get charged for a call you didn’t make. We don’t care!

Watch this… ((hits buttons randomly))
We just lost Peoria.

You see, this phone system consists of a multibillion-dollar matrix of space-age technology that is so… sophisticated ((hits buttons with her elbows)) even we can’t handle it. But that’s your problem, isn’t it?

So, the next time you complain about your phone service, why don’t you try using two Dixie cups and a string!

We don’t care.  We don’t have to.  We’re the Phone Company. ☎︎

#DGAF.  And believe it or not, AT&T is back, in more ways than one.

1984, The Terminator and Skynet

“It decided our fate in a microsecond.”—Kyle Reese

It’s easy to forget all these years later, but the first Terminator film by James Cameron was quite the suspenseful thriller, expressing the dangers of technology implemented without safeguards. Granted, the back-story felt a bit outlandish at the time. That is, if one thought deeply about then current prospects of artificial intelligence and extrapolated them into the next century.

But recently the AI winter ended, and seemingly every large organization is now promoting it not merely as an option but a necessity. Android robotics have advanced considerably this decade as well.

Ok, perhaps the idea of “Skynet” (an A.I. Villain) isn’t so outlandish after all. Say we replace the HKs (Hunter/Killers) with military drones perhaps? No need for time-travel either, simply forward into the abyss is sufficient. :-/

1993, Demolition Man

“Now all restaurants are Taco Bell.”

Another semi-dystopian sci-fi flick commenting on corporate mergers run amok, among other things.

1997, Gattaca

Explores a dystopian world of discrimination driven by eugenics. That is, selective breeding to boost “superior” genetic qualities over those deemed inferior. Where even one’s career and opportunities are predetermined, limited, and yes enforced through biometric-based security.

The main character goes to incredibly painful lengths to achieve his dream of becoming an astronaut. Written by Andrew Niccol.

2005, V for Vendetta

“Remember, remember the fifth of November!”

The one with the Guy Fawkes masks exploring fascism, surveillance, and a general theme that the terms terrorist/freedom-fighter are interchangeable, depending on one’s perspective. Two sides of the same coin, in other words.

By Alan Moore, David Lloyd, and the Wachowskis.

2002, Minority Report

“Hello, Mr. Yakamoto.”

A science-fiction thriller, that explores the idea of crime-prediction technology, called “pre-crime.” (Also a Digital-Signage industry fantasy, iris-scan identification marketing.)

As mentioned, AI winter has ended for a time. So this one is also not-completely out of the question any longer. If “AI” hasn’t been let loose on a police database and told to “pre-crime the hell out of this” already it would be surprising.

Edit: Spoke too soon, of course it has already happened in numerous instances. See the Machine Enforced Bias section under the recent history page for particulars.

Written by Philip K. Dick, screenplay by Scott Frank and Jon Cohen.

2006, The Lives of Others

A satisfying thriller set in East Germany during the Cold War illustrating the nightmare of living under an unlimited government panopticon. In short, a true-believing officer of the revolution grows disillusioned with his greedy, power-hungry superiors and decides to assist a sympathetic couple targeted for their assumed subversion.

Along these lines there’s also Balloon (2018), about a family’s attempt to leave East Germany illegally by hot-air balloon in 1979.

Great films and worth watching even if one is not interested in the related ethical issues. Though that’s quite unlikely if you’re still reading all the way down here, isn’t it!?

Next: Doctorow Insights