Interpreting a spoken request isn’t magic, rather it has taken a team of underpaid, subcontracted linguists to make the technology possible
“Do you believe in magic?” Google asked attendees of its annual developer conference this May, playing the seminal Lovin’ Spoonful tune as an introduction. Throughout the three-day event, company executives repeatedly answered yes while touting new features of the Google Assistant, the company’s version of Alexa or Siri, that can indeed feel magical. The tool can book you a rental car, tell you what the weather is like at your mother’s house, and even interpret live conversations across 26 languages.
But to some of the Google employees responsible for making the Assistant work, the tagline of the conference – “Keep making magic” – obscured a more mundane reality: the technical wizardry relies on massive data sets built by subcontracted human workers earning low wages.
“It’s smoke and mirrors if anything,” said a current Google employee who, as with the others quoted in this story, spoke on condition of anonymity because they were not authorized to speak to the press. “Artificial intelligence is not that artificial; it’s human beings that are doing the work.”
The Google employee works on Pygmalion, the team responsible for producing linguistic data sets that make the Assistant work. And although he is employed directly by Google, most of his Pygmalion co-workers are subcontracted temps who have for years been routinely pressured to work unpaid overtime, according to seven current and former members of the team.
These employees, some of whom spoke to the Guardian because they said efforts to raise concerns internally were ignored, alleged that the unpaid work was a symptom of the workplace culture put in place by the executive who founded Pygmalion. That executive was fired by Google in March following an internal investigation.
But current and former employees also identified Google’s broad reliance on approximately 100,000 temps, vendors and contractors (known at Google as TVCs) for large amounts of the company’s work as a culprit. Google does not directly employ the workers who collect or create the data required for much of its technology, be they the drivers who capture photos for Google Maps’ Street View, the content moderators training YouTube’s filters to catch prohibited material, or the scanners flipping pages to upload the contents of libraries into Google Books.
Having these two tiers of workers – highly paid full-time Googlers and often low-wage and precarious workers contracted through staffing firms – is “corrosive”, “highly problematic”, and “permissive of exploitation”, the employees said.
“It’s like a white-collar sweatshop,” said one current Google employee. “If it’s not illegal, it’s definitely exploitative. It’s to the point where I don’t use the Google Assistant, because I know how it’s made, and I can’t support it.”
An ‘army’ of linguists
The study of language is at the very heart of current advancements in computing. For decades, people have had to work to learn the language of computers, whether they were trying to program a VCR or writing software. Technology such as the Google Assistant reverses the equation: the computer understands natural human speech, in all its variations.
Take, for example, the straightforward task of asking the Assistant to set a timer to go off in five minutes, a former employee on Pygmalion explained. There are infinite ways that users could phrase that request, such as “Set a timer for five minutes”; “Can you ring the buzzer in five minutes?”; or “Configurar una alarma para cinco minutos.” The Assistant has to be able to convert the spoken request into text, then interpret the user’s intended meaning to produce the desired outcome, all practically instantaneously.
The technology that makes this possible is a form of machine learning. For a machine learning model to “understand” a language, it needs vast amounts of text that has been annotated by linguists to teach it the building blocks of human language, from parts of speech to syntactic relationships.
Enter Pygmalion. The team was born in 2014, the brainchild of longtime Google executive Linne Ha, to create the linguistic data sets required for Google’s neural networks to learn dozens of languages. The “painstaking” nature of the labor required to create this “handcrafted” data was featured in a 2016 article about Pygmalion’s “massive team of PhD linguists” by Wired. (Ha did not respond to a request for comment.)
From the beginning, Google planned to build the team with just a handful of full-time employees while outsourcing the vast majority of the annotation work to an “army” of subcontracted linguists around the world, documents reviewed by the Guardian and interviews with staff show.
The appetite for Pygmalion’s hand-labeled data, and the size of the team, has only increased over the years. Today, it includes 40 to 50 full-time Googlers and approximately 200 temporary workers contracted through agencies, including Adecco, a global staffing firm. The contract workers include associate linguists, who are tasked with annotation, and project managers, who oversee their work.
All of the contract workers have at least a bachelor’s degree in linguistics, though many have master’s degrees and some have doctorates. In addition to annotating data, the temp workers write “grammars” for the Assistant, complex and technical work that requires considerable expertise and involves Google’s code base. Their situation is comparable to adjunct professors on US college campuses: they are highly educated and highly skilled, performing work crucial to the company’s mission, and shut out of the benefits and security that come with a tenured position.
“Imagine going from producing PhD-level research and pushing forward the state of knowledge in the world to going to an annotation type job, where all you’re doing all day is annotating data; it’s very click, click, click,” said a former project manager on Pygmalion. “Everyone was trying to prove themselves because everyone was trying to work for Google. The competitive edge that happened among colleagues as TVCs was severe.”
‘The definition of wage theft’
This dynamic created the incentive for temps to perform unpaid work. Managers took advantage by making it clear they wouldn’t approve overtime for contract workers, while also assigning unrealistic amounts of work, current and former employees said.
The pressure to complete assignments was “immense”, said one Googler. “In this mixed stream of messages, I think a lot of people had to make their own calls, and given the pressure, I think people made different calls.”
The Googler described the overall effect as “gaslighting”, and recalled receiving messages from management such as, “If the TVCs want to work more, let them work more.” All seven current and former employees interviewed by the Guardian said they had either experienced or witnessed contract workers performing unpaid overtime.
“To my knowledge, no one ever said, you need to work TVCs above their contracts, but it was set up so that it was the only way to get the expected work done, and if anyone raised concerns they would be openly mocked and belittled,” said another current Googler.
“The 40-hour thing was just not respected,” said a former associate linguist. “It was made clear to us that we were never to log more than 40 hours, but we were never told not to work more than 40 hours.
“The work that they assign often takes more than hours hours,” they added. “Every week you fill out a timesheet. One person one time did submit overtime, and they were chastised. No punishment, but definitely told not to work overtime.”
A spokeswoman for Google said that it was company policy that temp workers must be paid for all hours worked, even if overtime was not approved in advance.
“Working off the clock is the very definition of wage theft,” said Beth Ross, a longtime labor and employment attorney. Ross said that both Google and Adecco could face liability for unpaid wages and damages under federal and state law.
‘They dangle that carrot’
The associate linguist was one of several who said that they took the position at Google in hopes that they could eventually convert to a full-time position. Several members of Pygmalion are former contract workers, including the current head of the team, who took over when Ha, the executive who founded the team, moved on to another project.
“People did [unpaid overtime] because they were dangled the opportunity of becoming a full-time employee, which is against company policy,” a current Googler said. “There’s a particular leveraging of people’s desire to become full time,” said another.
“When I was hired, I was very explicitly told that there is no ladder,” a current contract worker said. “‘This is not a temp-to-hire position. There is no moving up’ … But the reality on the team is very much one where there is clearly a ladder. A certain percentage of the associate linguists will get project manager. A certain percentage of project managers get converted to full time. We watch it happen, and they dangle that carrot.”
One Googler who successfully converted to a full-time position after working as a temp on Pygmalion said that at times the bargain was even made explicit. In April 2017, they recalled, Ha attended a meeting of outsourced Pygmalion project managers in London and “explain[ed] that the position was designed for conversion and that we should be proactive in asking for more work in order to achieve this”.
The Google spokeswoman said that it is company policy not to make any commitment about employment or conversion to temps, and that Googlers who manage temps are required to take a mandatory training on this and other policies related to TVCs.
‘Why do it?’
The disparity in wages and benefits between Google employees and contract workers is stark. Alphabet recently reported median pay of $246,804, and employees enjoy perks such as free meals, on-site yoga classes, free massages and generous benefits.
Amid increasing activism by Googlers and contract workers, Google recently announced improved minimum standards for US-based contract workers, including a minimum of eight paid sick days, “comprehensive” health insurance, and a minimum wage of at least $15 an hour by 2020. (A full-time job at that wage pays $31,200 a year; by comparison, Google charges its own employees $38,808 a year to place an infant in its onsite daycare facilities.)
Wages for contract workers on the Pygmalion team are well above the new minimum standard, usually starting around $25 an hour for associate linguists and going up to $35 an hour for project managers. But contractors complain about subpar benefits and other indignities.
The former project manager described Adecco’s benefits plan as “the worst health insurance I have ever had”. A current contract worker earning less than $60,000 annually said they were paying $180 each month in premiums for an individual plan with a $6,000 deductible. For families, the deductible is $12,000, according to documents reviewed by the Guardian. Google declined to comment on Adecco’s pay and benefits.
Googlers earn significantly more, and those on individual plans contribute between $0 and $53 for their health insurance and have a much lower deductible ($1,350), according to documents reviewed by the Guardian. Googlers with families pay up to $199 every two weeks, with a $2,700 deductible.
Others complained of a lack of trust and respect. In 2018, Google revoked the ability for contractors on Pygmalion to work while riding Google’s wifi-equipped commuter buses, creating frustration for those who spent three to four hours a day traveling to the company’s Mountain View campus and could no longer work and count that time toward their shift. Google said it works to ensure that temps, vendors and contractors do not have over-broad access to sensitive internal information for security reasons.
“Why do it?” a former associate linguist said of working unpaid overtime under these conditions. “I didn’t want to lose the job. Having Google on your résumé is important to a career … Later on, I came to find out that you can’t say ‘Google’ on your résumé. You have to say ‘Google by Adecco’.”
A weekend assignment
Both Google and Adecco recently launched investigations into the allegations of unpaid overtime in Pygmalion.
“Our policy is clear that all temporary workers must be paid for any overtime worked,” said Eileen Naughton, Google’s vice-president of people operations, in a statement to the Guardian. “If we find that anyone isn’t properly paid, we make sure they are compensated appropriately and take action against any Google employee who violates this policy.”
The current investigation was initiated after the company received a report of a possible policy violation in February 2019, the Google spokeswoman said. The company will provide appropriate compensation if need be and will take action up to and including terminations if policy violations are found, she added.
The spokeswoman also acknowledged that concerns about unpaid overtime were raised to human resources in 2017, but said that the company investigated and did not find any such cases at the time.
“We are committed to ensuring all employees are compensated for all time worked,” said Mary Beth Waddill, a spokeswoman for Adecco. “Our longstanding policy is that every employee is required to report time accurately – even if that time isn’t pre-approved – and they should feel encouraged to do so by their managers. If we learn that this is not the case, we will work with Google to take appropriate action.”
On Friday 17 May, Adecco sent emails to current and former Pygmalion temps. Recipients were asked whether they reported all the hours they worked, and, if not, to estimate how many hours they worked unpaid. The emails requested a response by Monday 20 May, though a Google spokeswoman said this week that the deadline has been extended.
A Google employee reacted: “They’re asking people to work on the weekend to recall unbilled overwork. It seems like it’s designed to discourage people from responding.”
Indeed, one former contract worker who left the company many months ago said they received the email but did not bother to respond. “After I left, I didn’t keep records of the hours I worked,” they said. “Even if I wanted to report overtime now, how could I?”
(Source: The Guardian)
“Do you believe in magic?” Google asked attendees of its annual developer conference this May, playing the seminal Lovin’ Spoonful tune as an introduction. Throughout the three-day event, company executives repeatedly answered yes while touting new features of the Google Assistant, the company’s version of Alexa or Siri, that can indeed feel magical. The tool can book you a rental car, tell you what the weather is like at your mother’s house, and even interpret live conversations across 26 languages.
But to some of the Google employees responsible for making the Assistant work, the tagline of the conference – “Keep making magic” – obscured a more mundane reality: the technical wizardry relies on massive data sets built by subcontracted human workers earning low wages.
“It’s smoke and mirrors if anything,” said a current Google employee who, as with the others quoted in this story, spoke on condition of anonymity because they were not authorized to speak to the press. “Artificial intelligence is not that artificial; it’s human beings that are doing the work.”
The Google employee works on Pygmalion, the team responsible for producing linguistic data sets that make the Assistant work. And although he is employed directly by Google, most of his Pygmalion co-workers are subcontracted temps who have for years been routinely pressured to work unpaid overtime, according to seven current and former members of the team.
These employees, some of whom spoke to the Guardian because they said efforts to raise concerns internally were ignored, alleged that the unpaid work was a symptom of the workplace culture put in place by the executive who founded Pygmalion. That executive was fired by Google in March following an internal investigation.
But current and former employees also identified Google’s broad reliance on approximately 100,000 temps, vendors and contractors (known at Google as TVCs) for large amounts of the company’s work as a culprit. Google does not directly employ the workers who collect or create the data required for much of its technology, be they the drivers who capture photos for Google Maps’ Street View, the content moderators training YouTube’s filters to catch prohibited material, or the scanners flipping pages to upload the contents of libraries into Google Books.
 |
| The workers who make Google Assistant possible are underpaid and overworked. Photograph: Dado Ruvić/Reuters |
Having these two tiers of workers – highly paid full-time Googlers and often low-wage and precarious workers contracted through staffing firms – is “corrosive”, “highly problematic”, and “permissive of exploitation”, the employees said.
“It’s like a white-collar sweatshop,” said one current Google employee. “If it’s not illegal, it’s definitely exploitative. It’s to the point where I don’t use the Google Assistant, because I know how it’s made, and I can’t support it.”
An ‘army’ of linguists
The study of language is at the very heart of current advancements in computing. For decades, people have had to work to learn the language of computers, whether they were trying to program a VCR or writing software. Technology such as the Google Assistant reverses the equation: the computer understands natural human speech, in all its variations.
Take, for example, the straightforward task of asking the Assistant to set a timer to go off in five minutes, a former employee on Pygmalion explained. There are infinite ways that users could phrase that request, such as “Set a timer for five minutes”; “Can you ring the buzzer in five minutes?”; or “Configurar una alarma para cinco minutos.” The Assistant has to be able to convert the spoken request into text, then interpret the user’s intended meaning to produce the desired outcome, all practically instantaneously.
The technology that makes this possible is a form of machine learning. For a machine learning model to “understand” a language, it needs vast amounts of text that has been annotated by linguists to teach it the building blocks of human language, from parts of speech to syntactic relationships.
Enter Pygmalion. The team was born in 2014, the brainchild of longtime Google executive Linne Ha, to create the linguistic data sets required for Google’s neural networks to learn dozens of languages. The “painstaking” nature of the labor required to create this “handcrafted” data was featured in a 2016 article about Pygmalion’s “massive team of PhD linguists” by Wired. (Ha did not respond to a request for comment.)
From the beginning, Google planned to build the team with just a handful of full-time employees while outsourcing the vast majority of the annotation work to an “army” of subcontracted linguists around the world, documents reviewed by the Guardian and interviews with staff show.
The appetite for Pygmalion’s hand-labeled data, and the size of the team, has only increased over the years. Today, it includes 40 to 50 full-time Googlers and approximately 200 temporary workers contracted through agencies, including Adecco, a global staffing firm. The contract workers include associate linguists, who are tasked with annotation, and project managers, who oversee their work.
All of the contract workers have at least a bachelor’s degree in linguistics, though many have master’s degrees and some have doctorates. In addition to annotating data, the temp workers write “grammars” for the Assistant, complex and technical work that requires considerable expertise and involves Google’s code base. Their situation is comparable to adjunct professors on US college campuses: they are highly educated and highly skilled, performing work crucial to the company’s mission, and shut out of the benefits and security that come with a tenured position.
“Imagine going from producing PhD-level research and pushing forward the state of knowledge in the world to going to an annotation type job, where all you’re doing all day is annotating data; it’s very click, click, click,” said a former project manager on Pygmalion. “Everyone was trying to prove themselves because everyone was trying to work for Google. The competitive edge that happened among colleagues as TVCs was severe.”
‘The definition of wage theft’
This dynamic created the incentive for temps to perform unpaid work. Managers took advantage by making it clear they wouldn’t approve overtime for contract workers, while also assigning unrealistic amounts of work, current and former employees said.
The pressure to complete assignments was “immense”, said one Googler. “In this mixed stream of messages, I think a lot of people had to make their own calls, and given the pressure, I think people made different calls.”
 |
| Behind the technology that makes the Google Assistant work is an army of Google-contracted linguists. Photograph: Samuel Gibbs/The Guardian |
The Googler described the overall effect as “gaslighting”, and recalled receiving messages from management such as, “If the TVCs want to work more, let them work more.” All seven current and former employees interviewed by the Guardian said they had either experienced or witnessed contract workers performing unpaid overtime.
“To my knowledge, no one ever said, you need to work TVCs above their contracts, but it was set up so that it was the only way to get the expected work done, and if anyone raised concerns they would be openly mocked and belittled,” said another current Googler.
“The 40-hour thing was just not respected,” said a former associate linguist. “It was made clear to us that we were never to log more than 40 hours, but we were never told not to work more than 40 hours.
“The work that they assign often takes more than hours hours,” they added. “Every week you fill out a timesheet. One person one time did submit overtime, and they were chastised. No punishment, but definitely told not to work overtime.”
A spokeswoman for Google said that it was company policy that temp workers must be paid for all hours worked, even if overtime was not approved in advance.
“Working off the clock is the very definition of wage theft,” said Beth Ross, a longtime labor and employment attorney. Ross said that both Google and Adecco could face liability for unpaid wages and damages under federal and state law.
‘They dangle that carrot’
The associate linguist was one of several who said that they took the position at Google in hopes that they could eventually convert to a full-time position. Several members of Pygmalion are former contract workers, including the current head of the team, who took over when Ha, the executive who founded the team, moved on to another project.
“People did [unpaid overtime] because they were dangled the opportunity of becoming a full-time employee, which is against company policy,” a current Googler said. “There’s a particular leveraging of people’s desire to become full time,” said another.
“When I was hired, I was very explicitly told that there is no ladder,” a current contract worker said. “‘This is not a temp-to-hire position. There is no moving up’ … But the reality on the team is very much one where there is clearly a ladder. A certain percentage of the associate linguists will get project manager. A certain percentage of project managers get converted to full time. We watch it happen, and they dangle that carrot.”
 |
| Google employees enjoy perks such as free meals, on-site yoga classes, free massages and generous benefits packages. Photograph: Lucy Nicholson/REUTERS |
One Googler who successfully converted to a full-time position after working as a temp on Pygmalion said that at times the bargain was even made explicit. In April 2017, they recalled, Ha attended a meeting of outsourced Pygmalion project managers in London and “explain[ed] that the position was designed for conversion and that we should be proactive in asking for more work in order to achieve this”.
The Google spokeswoman said that it is company policy not to make any commitment about employment or conversion to temps, and that Googlers who manage temps are required to take a mandatory training on this and other policies related to TVCs.
‘Why do it?’
The disparity in wages and benefits between Google employees and contract workers is stark. Alphabet recently reported median pay of $246,804, and employees enjoy perks such as free meals, on-site yoga classes, free massages and generous benefits.
Amid increasing activism by Googlers and contract workers, Google recently announced improved minimum standards for US-based contract workers, including a minimum of eight paid sick days, “comprehensive” health insurance, and a minimum wage of at least $15 an hour by 2020. (A full-time job at that wage pays $31,200 a year; by comparison, Google charges its own employees $38,808 a year to place an infant in its onsite daycare facilities.)
Wages for contract workers on the Pygmalion team are well above the new minimum standard, usually starting around $25 an hour for associate linguists and going up to $35 an hour for project managers. But contractors complain about subpar benefits and other indignities.
The former project manager described Adecco’s benefits plan as “the worst health insurance I have ever had”. A current contract worker earning less than $60,000 annually said they were paying $180 each month in premiums for an individual plan with a $6,000 deductible. For families, the deductible is $12,000, according to documents reviewed by the Guardian. Google declined to comment on Adecco’s pay and benefits.
 |
| Google workers have walked out over the company’s handling of sexual misconduct claims and its treatment of temporary workers. Photograph: Noah Berger/AP |
Others complained of a lack of trust and respect. In 2018, Google revoked the ability for contractors on Pygmalion to work while riding Google’s wifi-equipped commuter buses, creating frustration for those who spent three to four hours a day traveling to the company’s Mountain View campus and could no longer work and count that time toward their shift. Google said it works to ensure that temps, vendors and contractors do not have over-broad access to sensitive internal information for security reasons.
“Why do it?” a former associate linguist said of working unpaid overtime under these conditions. “I didn’t want to lose the job. Having Google on your résumé is important to a career … Later on, I came to find out that you can’t say ‘Google’ on your résumé. You have to say ‘Google by Adecco’.”
A weekend assignment
Both Google and Adecco recently launched investigations into the allegations of unpaid overtime in Pygmalion.
“Our policy is clear that all temporary workers must be paid for any overtime worked,” said Eileen Naughton, Google’s vice-president of people operations, in a statement to the Guardian. “If we find that anyone isn’t properly paid, we make sure they are compensated appropriately and take action against any Google employee who violates this policy.”
The current investigation was initiated after the company received a report of a possible policy violation in February 2019, the Google spokeswoman said. The company will provide appropriate compensation if need be and will take action up to and including terminations if policy violations are found, she added.
The spokeswoman also acknowledged that concerns about unpaid overtime were raised to human resources in 2017, but said that the company investigated and did not find any such cases at the time.
“We are committed to ensuring all employees are compensated for all time worked,” said Mary Beth Waddill, a spokeswoman for Adecco. “Our longstanding policy is that every employee is required to report time accurately – even if that time isn’t pre-approved – and they should feel encouraged to do so by their managers. If we learn that this is not the case, we will work with Google to take appropriate action.”
On Friday 17 May, Adecco sent emails to current and former Pygmalion temps. Recipients were asked whether they reported all the hours they worked, and, if not, to estimate how many hours they worked unpaid. The emails requested a response by Monday 20 May, though a Google spokeswoman said this week that the deadline has been extended.
A Google employee reacted: “They’re asking people to work on the weekend to recall unbilled overwork. It seems like it’s designed to discourage people from responding.”
Indeed, one former contract worker who left the company many months ago said they received the email but did not bother to respond. “After I left, I didn’t keep records of the hours I worked,” they said. “Even if I wanted to report overtime now, how could I?”
(Source: The Guardian)
No comments:
Post a Comment