An article on cnet.com recently described how Google’s new
smart assistant, called Google Nest Hub Max, uses facial recognition technology
to tell who is talking with it. This
feature has raised privacy concerns, as Google has admitted that they reserve
the right to upload facial data from it to the cloud to help improve “product
experience.” But whatever Google
does legitimately, a hacker might be able to do too, and so we are approaching a
time when the telescreens of George Orwell’s dystopian fantasy novel Nineteen
Eighty-Four have become a reality—not because of the unilateral command of
a totalitarian government (at least not in the U. S.), but because we want what
they can do.
For those unfamiliar with the novel, Orwell’s book was a
warning to the free world to beware of what a dictatorship could do with
communications technologies of the future.
Telescreens were two-way televisions on which propaganda by a dictator
known only as Big Brother is transmitted, and through which images of whoever
is watching are transmitted back to the party’s central headquarters. Orwell was simply extrapolating the efforts
of regimes such as the Nazis of the 1930s and the Soviet Union of the 1940s to
spy on their populace twenty-four hours a day to enforce total obedience to the
At the time the novel was published in 1949, no one took the
telescreen-spying idea very seriously, because it would take a huge number of
human monitors to spy on a significant number of people. Carried to its logical extreme, the only way the
government could watch everybody would be if half the population spied on the
other half, and then took turns.
But neither Orwell nor anybody else at the time reckoned on
the development of advanced artificial-intelligence (AI) systems using facial
recognition technology. In China, the government
is deploying many thousands of cameras and taking facial data from millions of
people with the intention of developing a Social Credit rating that measures
how well you measure up to the regime’s model of the ideal citizen. If you have been caught on camera by
computers going to suspicious places or meetings, your Social Credit score could
go in the tank, making it hard to travel, get a job, or even stay out of jail.
None of that is happening in the U. S., but the fact that a
large corporation will now have electronic access to views in millions of
private residences should at least give us pause.
Leaving the hardware aside for the moment, let’s examine the
difference in motives between a government, such as in Nineteen Eighty-Four,
spying on its citizens for the purposes of controlling behavior, and a
commercial entity such as Google using images to sell both its own services and
advertising for others. The government spying
is motivated by suspicion and fear of what people might be doing while the
government isn’t watching them. Whatever
the regime sets out as an ideal of behavior, it watches for deviations from
that ideal, and punishes those who deviate from it. Participation is not voluntary, and people
have to go to great lengths to avoid being spied on.
Now contrast that with a benign-looking thing such as the
Google Nest Hub Max. Nobody is going to
make you buy one. And if you do, there
are ways of turning off the facial-recognition feature, though it will be less
convenient to use. And the device is
intended to serve you, not the other way around. It’s sold with the vision portrayed in so
many TV ads of people happily using it to make their lives better, not for
means of social control like Orwell’s telescreens.
But maybe the differences are not as great as they first
appear. Both the telescreen and the Nest
Hub Max are intended to change behavior.
If they don’t, they have failed.
True, the ideal behavior that a totalitarian government wants and the
ideal behavior that a company wants are two different things. But neither ideal is the way the
citizen-consumer was before the screen or Nest Hub shows up, namely, unwatched
and unbenefited by the products or services that the company wants to sell.
Nobody should read this blog and then go around saying
“Ahh, Stephan’s saying Google is Big Brother and they’re trying to take
over our lives!” That’s not the
comparison I’m making. My point is that
the mere fact of being watched by someone, or something that can inform someone
about us, is going to change our behavior.
And that change by itself is significant.
Now, the change may not necessarily be bad. Already, virtual audio assistant devices such
as Alexa have been used in criminal cases when bad actors set them off, either
by accident or on purpose, and the data thus generated has proved to be
incriminating. Though this is ancient
history, I am told that in the days when some middle-class and upper-class
people had servants, families tended to behave better when the servants were
around, although I’m sure there were exceptions. Alexa isn’t Jeeves the butler, but as virtual
assistants play a more significant role in domestic life, it’s not beyond
imagination to think that some of the worst behavior in homes—domestic abuse,
for example—might be mitigated if the victim could call 911 by just shouting it
instead of having to pull out a phone.
I’m not necessarily crying doom and gloom here. Millions of people are already using virtual
assistants with few if any problems, and adding two-way video to the mix will
only increase the devices’ capabilities.
But we are entering a new territory of connectivity here, and it’s bound
to have some effects that nobody has predicted yet. Perhaps it’s not too helpful to predict that
there will be unpredictable effects, but right now that’s all I can do at the
moment. Let’s hope that the security
features of the Nest Home Max are good enough to prevent nefarious use, and
that people who buy them are truly happier with them than they were