Eyes in the Sky: How to Quantify Your Life

China has a notorious (yet, contrary to the popular opinion, not quite deployed) system that combines total surveillance, AI-based facial recognition, and a complex system of social credit that looks like an episode of Black Mirror came to life. Still, as frightening as it seems, is this system actually that scary? Or is it worse than it looks? Plus, China isn’t the only country out there to use systems like that. The United States of America have their own social credits, total surveillance, and other dystopian feats that would have been no less frightening had they been as obvious as those in China.

It’s a very nice feeling to walk down your block at night knowing that nothing bad will happen to you, even if you try your best by waving a fat pack of dollars above your head. Why would you ever do that? Well, maybe because you, as well as all thugs in your neighborhood, are well aware that every square inch of the street is under continuing video surveillance, and there’s virtually no chance for a criminal to mug you, or do something even worse, and get away with it.

That’s probably the most obvious advantage of CCTV cameras sticking out of every lamp post and street corner. People in countries like the U.S. or the U.K. that both had very brief episodes of oppressive governments a long while ago, are generally fine with all that stuff, and mostly believe that it’s no big deal as they don’t do anything wrong. But, as anyone coming from a country with less fortunate history would tell you, it’s not big a deal right until it is.

Renowned works of literature and various sci-fi endeavors led us to believe that if there’s anybody capable of taking your freedoms from you, it’s government. Whether it’s mind-bending nightmare of 1984, or a consumerist utopia of Brave New World, where the entire planet is one big real-life show about the Kardashians, it’s always the government who secretly (or not) pulls the strings. Still, it looks like being a government is not compulsory to turn into a behind-the-scenes eye in the sky, as recent massive privacy breaches by privately-owned corporations obviously suggest.

With the advent of new technologies, like AI, neural networks, facial recognition, and other pipe dreams of sci-fi pundits of the sixties, the world is quite ready to become brave and new. And while the notion of “new” is hard to argue, its braveness is quite questionable. It’s fairly easy to get depressed after reading about China’s social credit and Sesame credit systems combined with AI-powered mass surveillance systems that look like a monstrous meeting point for George Orwell, Aldous Huxley, and whoever is that sick freak who came up with Black Mirror. However, it is a grave mistake to think that it’s necessarily China who will lead the world into that dystopian nightmare with mass surveillance and hi-tech profiling. In the West, things aren’t as different as one might think. There is an Eye in the Sky in the East. But, whatever it is, it’s not a cyclops.

Eastern Eye in the Sky: Social Credit

Before judging China’s social credit and labelling it as the harbinger of the new era of digital authoritarianism, it is important to have at least a glance at the cultural and historical context in which the bedevilled system was conceived.

In fact, the country’s social control and self-policing tradition dates back to the Middle Ages, and to the Song dynasty in particular. In the 11th century, the Chinese government devised a community-based law enforcement and civil control system called baojia aimed to decrease the government’s dependance on mercenaires by delegating the power to local authorities. Baojia also included an older concept of lianzuofa, or “sequential accusation law”, which implied collective liability for wrongdoing and encouraged mutual observation. In such a way, people within a community were very much interested to make sure that all of their peers are “good” hard-working citizens, and to exclude those who could do or did something “wrong”. In one way or another, this system has been used by China’s rulers throughout the nation’s entire history. A bit more recently, in 1949, when the Communist Party came to power, they tested their own methods of social control and collectivization that in a way resonated with that well-rooted model of mutual responsibility and reporting bad actors to the authorities.

Another important reason behind the social credit system is the distrust plaguing China’s society. According to the survey conducted by the Chinese Communist Party newspaper People’s Tribune, the “crisis of credibility” tops the list of 10 most burning concerns in the country. Moreover, according to the same survey, 51% of those surveyed believe the “unethical behavior that has gone unpunished” is the main reason for the crisis. While it is arguable whether the survey conducted by the Party’s leading newspaper is unbiased, and whether the respondents gave candid answers, the actual picture of a distrust problem seems fairly believable. There is a slightly dusty story about an elderly lady who has fallen on the street and then tried to sue the kids that helped her to get back up. It appears that after the incident, people think twice before helping out senior strangers out of fear that their benevolent actions may be exploited.

While the state-owned media began acknowledging the aforementioned crisis, the government faced the need to come up with some sort of a solution and bring the harmony back to the people. What they eventually came up with was the concept of a technological, data-driven solution that would keep tabs on citizens’ behavior, effectively recognizing some actions as “good”, and some as “bad”.

Thus, in 2014, the Chinese government rolled out the outline for the nationwide social credit system, calling it the “State Council Notice concerning Issuance of the Planning Outline for the Construction of a Social Credit System (2014–2020).” According to the plan, the system coordinated by the Central Leading Group for Comprehensively Deepening Reforms is expected to be fully deployed by 2020. The outline describes the main goal of the system as “raising the awareness for integrities and the level of credibility within society.” The system itself is supposed to standardize the procedure of evaluating one’s economic or social reputation, concisely called “credit.”

In 2015, the People’s Bank of China authorized eight private companies to try out their pilot versions of the social credit system:

• Sesame Credit Management Co., Ltd.
• Tencent Credit Co., Ltd.
• Shenzhen Qianhai Credit Information Center Co., Ltd.
• Pengyuan Credit Co., Ltd.
• China Chengxin Credit Co., Ltd.
• Zhongzhicheng Credit Information Co., Ltd.
• Lakara Credit Management Co., Ltd.
• Beijing Huadao Credit Co., Ltd.

So, numerous tech companies eagerly collaborated with the government and its institutions to create the necessary software, though, amongst them all, only Sesame (Zhima) Credit Management, which is operated by Ant Financial, proved to be the most frequent target for the Western media. It’s hardly surprising, considering the fact that shortly after the licenses were issued, Zhima Credit scoring system was embedded in Alipay, a popular mobile payments app. While Zhima obviously isn’t the only social credit system out there, it still can be seen as a typical specimen of such infrastructure.

Basically, it uses algorithms to analyze user behaviour by a vast number of criteria and boils everything down to a three-digit number somewhere between 350 and 950. And “vast” is quite a conservative word for what the system collects. Apart from checking if you’ve repaid all your bills on time, it takes note of your purchases, your insurance claims, your education and academic degrees, your friends’ scores, your encounters with courts and fines, and even your dates.

As the company puts it, the system devours the data “from more than 300 million real-name registered users and 37 million small businesses that buy and sell on Alibaba Group marketplaces,” as well as from numerous public documents, such as one’s official ID and financial records. In addition, the system has access to the data on businesses and the messages of 850 million of WeChat users.

The actual inner workings of Zhima Credit are kept dark, but it is known that it puts the collected data into five categories:

• Credit history, which reflects the user’s past payment history and indebtedness, for example credit card repayment and utility bill payments.
• Behavior and preference, which reveal the user’s online behavior on the websites they visit, the product categories they shop, etc.
• Fulfillment capacity, which shows the user’s ability to fulfill his or her contractual obligations. Indicators of this category include the use of financial products and services, as well as Alipay account balances.
• Personal characteristics, which examine the extent and accuracy of personal information, for example home address and the duration of residence, mobile phone numbers, etc.
• Interpersonal relationships, which reflect the online characteristics of the user’s friends and the interactions between the user and his or her friends.

Based on the individual metrics for each of these criteria, the system determines the final score. Those who have their score high enough can get certain tangible privileges, such as renting a car without a deposit, and a bit more abstract benefits, such as more favorable position when applying for a job. According to Wired, those who have a score above 750 can skip security checks at Beijing Capital Airport. Another article suggests that the scores have already become a part of a mating ritual as people who have high social credit numbers don’t hesitate to show them off on local social media. However, if anything goes wrong, like the rented car gets crashed, or a passenger is caught smuggling prohibited items aboard a plane, it all immediately goes to the system, and their social credit number starts plummeting.

Notably, the social credit system already brings certain positive changes to people’s lives, such as cars yielding to pedestrians at a crosswalk. However, there are also outcries from those who have internet access:

Another Reddit user blames the system for invading people’s privacy and for spreading the responsibility to friends and family:

Some people even shared their knowledge of the technical and organizational aspects of the system:

Others shared their experience with the system’s punishments:

And some have noticed the uncanny resemblance between the Chinese social credit and the credit score system in the United States:

This resemblance is indeed hard to ignore, so the comparison looks almost inevitable. Still, the cultural, historical, and ethical differences between the East and the West make the matters much more complicated than a simple comparison could possibly point out.

Western Eye in the Sky: Credit Score

In the U.S. people have been worshiping another three-digit number for a while. This number is called credit score.

Basically speaking, one’s credit score is a numerical expression of their creditworthiness, or, simply put, a number reflecting how likely a person is to repay their loans. The score, which is a grade given to one’s credit report, is calculated based on the data from credit reports provided by three Credit Reporting Agencies (CRA’s): Equifax, TransUnion, and Experian.

While credit reporting was a thing more than a century ago in a form of retail merchants trading financial information about their customers, the actual concept of credit scoring in its modern meaning was created in the late 1950s by engineer William Fair and mathematician Earl Isaac. The two founded a company called FICO and began pitching America’s largest credit grantors with their new concept. Credit scoring was adopted and soon, in the 1960s, people noticed that credit reports were affecting their access to numerous services not connected to getting a loan, for example, landing a job.

Given such a sensitive nature of credit reports, the newly formed industry required proper regulation. Thus, in 1970, the Fair Credit Reporting Act (FCRA) was passed. The act authorised the Federal Trade Commission to regulate access to people’s financial information, accuracy of this information, its privacy, and the purposes it was collected for.

In 1975 FICO developed its first behaviour scoring system aimed to assess and predict credit risks of the existing customers. The first general-purpose FICO scoring model was introduced in 1989.

Yet, only in 2001 the FCRA was amended to allow people access their credit reports directly, but at a “fair and reasonable” cost. Subsequently, the act was amended more than once and by now it gives people the right for one free report annually, the right to purge “inaccurate and certain derogatory” information from their history, and the right to restrict access to their information.

Today, the most commonly used models for credit scoring are VantageScore 3.0 and FICO 8. These models give out a score somewhere between 300 and 850, which is quite similar to the range of 350 to 950 used in China. It is impossible to draw a precise line between “good” and “bad” scores, as each lender is free to choose their own threshold. However, generally the scores are perceived as follows:

• 300 to 329 is Bad;
• 630 to 689 is Fair or “average credit”
• 690 to 719 is Good;
• 720+ is Excellent.

Notably, both FICO and VantageScore are based on the same factors. In both systems the final score depends on the following criteria: 35% payment history, 30% debt amount, 15% age of the record, 10% new credit, and 10% types of credit used.

As reported by the Federal Reserve, “nearly all mortgage, credit card, and auto loan originations involve lenders checking loan applicants’ credit scores.” And those with low scores get their loans applications declined or face higher interest rates and lower credit limits.

Considering these unpleasant consequences, people are willing to keep their scores in shape. Only between 2012 and 2017 the Consumer Financial Protection Bureau’s consumer complaint database got 1,1 million complaints bigger, and 185,717, or about 18%, of them concerned credit reporting agencies.

Complaints may continue, but the cultural phenomenon of credit in the U.S. won’t go anywhere regardless of the scoring system flaws. By the way, China’s media aren’t screaming about an inhumane scoring system in “the land of the free.”

Spending the money you don’t possess isn’t only the theme for Shania Twain’s 2003 hit or the fundamental reason for the Great Recession, it’s also a modus operandi for lots of people out there. In fact, that’s one of the reasons why credit scores have gained that much traction as a quantifier for a person’s trustworthiness. It looks like making a living out of your own money without contracting a debt may seem too incredible to be true to some people, as this story of a Reddit user named angrydeuce suggests:

The ridiculousness of using credit reports as a representation of a person’s trustworthiness was also pointed out by none other than the great and terrible John Oliver in a segment of his iconic Last Week Tonight show on credit reports.

“Just look on Craigslist, because you’ll find credit checks are required in all sorts of job listings from managing a Benihana in Cincinnati to this one which reads ‘Who runs those fireworks tents? It could be you!… Applications with good credit check required,’ which is clearly ridiculous. They don’t need to bring credit into that job, they just need to ask the question: ‘What do you think of fireworks?’”

Most users are obviously unhappy about having to depend on three digits in too many areas, and some of them even see this system as something that may cause serious problems in the security area:

Still, the U.S. Consumer Financial Protection Bureau believes that credit reports are a given, and should be treated respectively.

“Credit scores are central to a consumer’s financial life and people deserve honest and accurate information about them,” CFPB Director Richard Cordray said in January 2017.

Most notably, the statement came as a comment as to the $23 million fine imposed on TransUnion and Equifax for lying to consumers about their credit scores. A killing joke could be here, yet the circumstances and the context actually make Mr. Cordray’s statement look like one.

Beam in the Eye in the Sky

While in China the big brother is yet to become really big, though he’s definitely headed in that direction, there is no big brother in the West. Instead, there are hundreds of not so big brothers that do pretty much the same job without a single umbrella infrastructure for them to feed data to. Credit scores in the U.S. bear a striking resemblance to social credit in China, the main difference being it’s actually run by three big privately-owned companies that reportedly make huge mistakes while also failing to ensure sufficient security for the data they operate.

This privatization and certain decentralization are the key features of the Western Eye in the Sky. After all, why should the government bother to rate its citizens using some metrics coming from a machine if people themselves are quite happy to rate each other? We already rate restaurants, museums, and hotels. In Uber, we rate our drivers, and the drivers rate us in their turn. If a user gets too low a rating, no Uber will ever go to them. If a driver is rated badly, he or she won’t ever get called.

This sounds convenient and reasonable to some extent. Of course, no driver will be happy to give a ride to a person who is known for serial exuberant puking on the backseat, while there’s hardly any passenger willing to go for a ride with a driver famous for incessant talks about burning feminists at a stake, as well as several well-documented attempts to do so.

However, credit scoring in the U.S. was intended to give a quantitative expression of the likelihood of a loan being repaid. It was never intended as a measure of a person’s trustworthiness. Still, it’s is widely used as one. Is there anything that can stop employers or landlords from checking your Uber rating to make sure you’ll behave decently, work hard, and pay your rent on time? The only possible reason why they don’t do that is that because it’s downright stupid, and your Uber rating has nothing to do with any of that. Yet, so does your credit score, but here we are.

Meanwhile, the recent case of Irvine Company who fed their customer’s license plate numbers to a company selling data to ICE, is yet another example that the government doesn’t even need to do the dirty job if there are private companies who are happy to get involved. If there is one firm who does that, who’s to guarantee there’s no other one? Or that there are thousands of them who spy on you without you having any idea about that?

After all, you know Facebook is spying on you. You know your privacy is a matter of really big money for really big corporations. You know Google has been reading your emails for years, and even now, when they reportedly stopped eavesdropping, they still allow third-party developers to do the same. Most notably, while Google used software to do that, in case of third-developers it’s actual people of flesh and blood who read your correspondence. And it’s you who give them the right to do it.

What do all those things have in common, and how are they different from the system that is to be rolled out in China? In China, it’s a system that might become mandatory, and that has been devised in the corridors of power, has a centralized implementation, and a centralized vision. It’s a good old dystopia with an omnipotent government who uses every tech piece at its disposal to tighten the control over the society. It cares about which video games you play, which movies you watch, and what messenger app you use. In China, they may shame you into submission. On the other hand, the system’s core concept is deeply rooted in the mindset of the people, which, again, is quite different from that of the Westerners, and therefore cannot be fully comprehended by them.

Still, in the West, there’s no need to create such a complex government-run system. In full accordance with the underlying meaning of Huxley’s Brave New World, if you get people to love their cages, they won’t be willing to get out of them even if you leave the doors open. Instead of a nationwide surveillance system, the West has thousands of local networks, even privately owned, that still feed the data to the law enforcement. There are lots of ways to rate your fellow human beings, from credit scores to Uber rating or Facebook likes, and most people seem to enjoy it. Who cares that Facebook violates your privacy if it’s nearly the only reliable way to see the newest memes or play your favorite match-3 timekiller without having to surf the internet?

Essentially, this decentralized Eye in the Sky does the same thing as its centralized peer in the East: it gamifies the process of taking your freedoms from you. A person doesn’t need to worry about being controlled and spied upon 24/7 if there are no spies to be seen and everything looks like a game with scores and stuff, and, most importantly, with great prizes for playing good and observing the rules.

So, the future indeed looks like Huxley’s Brave New World borrowed a few things from Orwell’s 1984. This symbiosis of technological and social infrastructures could become the end of any freedom and human rights in the hands of an authoritarian government, to which no country in the world is immune these days. So, is our freedom really on a deathbed? Or is it all just a horror bedtime story that libertarian parents tell their children if they spill their biodynamic shiraz on an organic carpet? Well, as long as we’re able to talk about that publicly without fearing prosecution for speaking our minds, saying that freedom is about to die might be a bit preposterous.

Or maybe they just let the birds sing in their cages all they want. Who knows.

---

This post originally appeared at https://lawless.tech/eyes-in-the-sky-how-to-quantify-your-life/

lawless.tech is an online magazine devoted to covering the ongoing regulatory attempts to oversee and control the newest technologies

Join our Telegram channel, follow us on Twitter and Facebook to explore how regulations will impact the latest technological advances.