Google will not renew Pentagon contract that upset employees

Many of Google's top AI researchers worried that the contract was the first step toward using the nascent technology in advanced weapons.
Many of Google's top AI researchers worried that the contract was the first step toward using the nascent technology in advanced weapons.PHOTO: REUTERS

SAN FRANCISCO (NYTIMES) - Google, hoping to head off a rebellion by employees upset that the technology they were working on could be used for lethal purposes, will not renew a contract with the Pentagon for artificial intelligence work when a current deal expires next year.

Diane Greene, who is the head of the Google Cloud business that won a contract with the Pentagon's Project Maven, said during a weekly meeting with employees on Friday (June 1) that the company was backing away from its AI work with the military, according to a person familiar with the discussion but not permitted to speak publicly about it.

Google's work with the Defence Department on the Maven programme, which uses artificial intelligence to interpret video images and could be used to improve the targeting of drone strikes, roiled the Internet giant's workforce. Many of the company's top AI researchers, in particular, worried that the contract was the first step toward using the nascent technology in advanced weapons.

But it is not unusual for Silicon Valley's big companies to have deep military ties. And the internal dissent over Maven stands in contrast to Google's biggest competitors for selling cloud-computing services - Amazon.com and Microsoft - which have aggressively pursued Pentagon contracts without pushback from their employees.

Google's self-image is different - it once had a motto of "don't be evil." A number of its top technical talent said the Internet company was betraying its idealistic principles, even as its business-minded officials worried that the protests would damage its chances to secure more business from the Defence Department.

About 4,000 Google employees signed a petition demanding "a clear policy stating that neither Google nor its contractors will ever build warfare technology." A handful of employees resigned in protest, while some were openly calling on the company to cancel the Maven contract.

Months before the deal became public, senior Google officials were worried about how the Maven contract would be perceived inside and outside the company, The New York Times reported this week. By courting business with the Pentagon, they risked angering a number of the company's highly regarded AI researchers, who had vowed that their work would not become militarised.

James Mattis, the US Defence Secretary, had reached out to tech companies and sought their support and cooperation as the Pentagon makes artificial intelligence a centrepiece of its weapons strategy. The decision made by Google on Friday is a setback to that outreach.

But if Google drops out of some or all of the competition to sell the software that will guide future weaponry, the Pentagon will likely find plenty of other companies happy to take the lucrative business. A Defence Department spokeswoman did not reply to a request for comment on Friday.

Greene's comments were reported earlier by Gizmodo.

The money for Google in the Project Maven contract was never large by the standards of a company with revenue of US$110 billion (S$147 billion) last year - US$9 million, one official told employees, or a possible US$15 million over 18 months, according to an internal e-mail.

But some company officials saw it as an opening to much greater revenue down the road. In an e-mail last September (2017), a Google official in Washington told colleagues she expected Maven to grow into a US$250 million-a-year project, and eventually it could have helped open the door to contracts worth far more; notably a multiyear, multibillion-dollar cloud computing project called JEDI, or Joint Enterprise Defence Infrastructure.

Whether Google's Maven decision is a short-term reaction to employee protests and adverse news coverage or reflects a more sweeping strategy not to pursue military work is unclear. The question of whether a particular contract contributes to warfare does not always have a simple answer.

When the Maven work came under fire inside Google, company officials asserted that it was not "offensive" in nature. But Maven is using the company's artificial intelligence software to improve the sorting and analysis of imagery from drones, and some drones rely on such analysis to identify human targets for lethal missile shots.

Google management had told employees that it would produce a set of principles to guide its choices in the use of artificial intelligence for defence and intelligence contracting. At Friday's meeting, Greene said the company was expected to announce those guidelines next week.

Google has already said that the new artificial intelligence principles under development precluded the use of AI in weaponry. But it was unclear how such a prohibition would be applied in practice and whether it would affect Google's pursuit of the JEDI contract.

Defence Department officials are themselves wrestling with the complexity of their move into cloud computing and artificial intelligence. Critics have questioned the proposal to give the entire JEDI contract, which could extend for 10 years, to a single vendor. This week, officials announced they were slowing down the contracting process.

Dana White, the Pentagon spokeswoman, said this week that the JEDI contract had drawn "incredible interest" and more than 1,000 responses to a draft request for proposals. But she said officials wanted to take their time.

"So, we are working on it, but it's important that we don't rush toward failure," White said. "This is different for us. We have a lot more players in it. This is something different from some of our other acquisition programs because we do have a great deal of commercial interest."

Greene said the company probably would not have sought the Maven work if company officials had anticipated the criticism, according to notes on Greene's remarks taken by a Google employee and shared with The Times.

Another person who watched the meeting added that Greene said that Maven had been "terrible for Google" and that the decision to pursue the contract was done when Google was more aggressively going after military work.

Google does other, more innocuous business with the Pentagon, including military advertising on Google properties and Google's ad platform, as well as providing web apps like email.

Meredith Whittaker, a Google AI researcher who was openly critical of the Maven work, wrote on Twitter that she was "incredibly happy about this decision" and had "a deep respect for the many people who worked and risked to make it happen. Google should not be in the business of war."

Even though the internal protest has carried on for months, there was no indication that employee criticism of the deal was dying down.

Earlier this week, one Google engineer - on the company's internal message boards - proposed the idea of employees' protesting Google Cloud's conference at the Moscone Center in San Francisco in July with a campaign called "Occupy Moscone Center," fashioned after the Occupy Wall Street protests.

That engineer resigned from the company this week in protest of Maven and planned for Friday to be his last day. But he said he was told Friday morning to leave immediately, according to an e-mail viewed by The Times.

Peter Singer, who studies war and technology at New America, a Washington research group, said many of the tools the Pentagon was seeking were "neither inherently military nor inherently civilian." He added, "This is not cannons and ballistic missiles."

The same software that speeds through video shot with armed drones can be used to study customers in fast-food restaurants or movements on a factory floor.

Singer also said he thought Google employees who denounced Maven were somewhat naïve, because Google's search engine and the video platform of its YouTube division have been used for years by warriors of many countries, as well as al-Qaida and the Islamic State.

"They may want to act like they're not in the business of war, but the business of war long ago came to them," said Singer, author of LikeWar, a book examining such issues which is scheduled for publication later this year.