Some digital technologies are so dangerous that they should probably come with a government health warning. Rarely does this hazard occur by chance alone but rather is the result of an insatiable drive for success, regardless of social cost.
Former employees of several tech companies have admitted what many suspected, that social media software was engineered to be habit forming. It was tacitly understood by them that the business model required users to become addicted. As one engineer explained,
“In order to get the next round of funding, in order to get your stock price up, the amount of time that people spend on your app has to go up.”
It’s not only social media that has pernicious effects on the people, video games have long been associated with poor mental health outcomes and even death.
In an extraordinarily candid talk, game designer Teut Weidemann described how online video games are designed to entice people and keep them playing by appealing to their shortcomings, brazenly confessing to
“monetizing all the weakness of people.”
He then proceeded to address the seven biblical sins, showing how game design can monetize each while exploiting the worst of human failings. What chutzpah!
Smartphone apps are a recent example of addiction by design. Walk down Main street in any American city and you will see examples of mainly young people, heads down staring blankly, fingers gently stroking their precious devices. They are having their brains sucked out for profit. Get them hooked early like a drug pusher, then you have them for life. A generation is being lost before our eyes.
Of course, parents in Silicon Valley are wise to what’s going on and many are raising their kid’s technology free.
“You can’t put your face in a device and expect to develop a long-term attention span”
It’s not only mental abilities that are being short-changed, some children need physical therapy when they start school as a result of playing with technology. One mother ruefully describes the difficulties her child experienced when learning to write:
“he was gripping his pencil like cavemen held sticks.”
Surveillance is the new threat
Designing interfaces that intentionally cause addiction, steal people’s attention and leave them with a host of mental and physical problems in the name of profit, is itself a moral weakness.
It’s a failing though that pales in comparison to the efforts underway to build the surveillance state. Tech giants have been vacuuming up huge amounts of user data for years, originally with the intent to target them with ads. It must be said that much of this data was offered freely by users in exchange for access to whatever dopamine rush the technologists were offering.
Mark Zuckerberg has built his Facebook fortune on a cynical attitude to his customer base. We know this from his Harvard days when he considered people who trusted him with their personal information as
Technology has advanced well beyond hoovering up people’s data to package and sell. The hot new thing is “people analytics”, a sweet name for AI enabled surveillance capitalism. This is where everything you do (or don’t do) at work is recorded, analyzed and compared against metrics. Even your emotions will be fair game for detection by facial analysis. Don’t like your boss, thinking of quitting? Nothing will be hidden from the algorithmic dictatorship.
Surveillance is also pushing further and further into regular life outside work. Who you’re friends are may affect your credit rating, or whatever aspect of your life some data scientist thinks it correlates with. Every purchase you make, place you visit, person you talk with, could be captured in a database and scrutinized for inferences.
Many people will no doubt see safety with this blanket surveillance of our lives. After all, it could help stop crime and may identify terrorists before they can act. However, where does the line get drawn? How about predictive policing, where you can get apprehended before committing a crime? Is that acceptable? What will be considered a crime in the future?
Could a tyrant seize power and identify his opposition as criminals? Not too far from reality when we see people in high places calling the press, the “enemy of the people”. Benjamin Franklin once warned us about this,
“They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.”
Without some concerted effort on the part of concerned citizens, current developments are leading us to the surveillance state. Perhaps it’s inevitable. The technology will be built and used, just because it can. Profit will be used as the rationale. Technologists will just say they were following orders.
The Tech Industry is incapable of policing itself
The general population is at a huge disadvantage when it comes to living with pervasive surveillance technology. This information asymmetry is insurmountable and we are at the mercy of business practices which are not in our favor. Because of this, regulation is needed to protect our interests and ensure that we are not subject to exploitation and abuse.
Regulation needs to be written by people who actually understand the threat and can identify the red lines that companies and government should not cross. Significant financial interests in a company creating AI software should be grounds for disqualification from the process.
What’s really needed though is independent testing and verification of all software that makes potentially life-changing decisions. This includes facial recognition, crime prediction, employment, healthcare, and personal finance. Independent experts need to be able to determine whether these systems perform in a fair and equitable way.
Citizens need to be given the ability to opt-out of surveillance systems and companies need to be held accountable for respecting that choice.
Recently we have seen employees protest and, in some cases, quit their employers over what they see as unethical uses of technology. Employees taking a stand may be the best hope we have for avoiding the surveillance state.