Leaked messages show early challenges for Amazon’s big AI product and concern about losing customers to Microsoft

Visitors at an AWS conference in Seoul, South Korea.

Amazon Q, one of the company’s most important new AI products, became publicly available in late April. Four months into its launch, Q is facing a host of teething problems.

Some of these concerns were shared in an internal Slack channel in August, where a number of Amazon Web Services employees voiced their honest opinions about how Q has been doing in its early rollout phase, according to the messages and an internal memo seen by B-17.

They said Q lacks certain features common to competing products. It’s also sometimes expensive for end users and has trouble integrating with other software. Worst, Q may be losing customers to Microsoft’s Copilot.

“My view is that Q is fit for demos and very tightly controlled simple use cases right now,” one of the AWS employees wrote in the Slack channel.

Q’s early struggles illustrate the challenges facing business software makers in the age of AI. Buzz, hype, and hope have fueled intense competition that forces tech companies to rush new product launches, often with incomplete features, flawed internal tracking systems, and unintended IT impacts. Amazon competitors, such as Microsoft and Google, have also experienced problems with early AI launches.

For Amazon, this is yet another test of its AI prowess. The company expects several billion dollars in revenue from AI services this year. But AWS’s AI chips that compete with Nvidia’s GPUs are seeing slow adoption rates, while efforts to upgrade the Alexa voice assistant with ChatGPT-like technology are facing headwinds.

Internally, Amazon employees are worried that Q’s shortcomings are causing customers to switch to Microsoft’s Copilot.

One common customer complaint is Q’s inability to process images embedded in PDF files, according to the Slack messages and a separate internal memo viewed by B-17. As a result, Q’s answers are “not on par” with Microsoft Copilot’s, and customers have to wait until AWS’s re:Invent conference in December, when upgrades are expected, one of the AWS employees wrote in Slack.

Another issue is image creation. The same AWS employee noted that Q doesn’t support “multi-modal capabilities” or the ability to handle both text and image files, making it less appealing. One customer switched to Copilot after finding out that Q couldn’t create a marketing campaign that includes both an image and text, this staffer said.

Some customers are complaining about Q’s high data integration cost, according to the internal memo and Slack messages. For example, one customer found Amazon Q didn’t have the tools to selectively choose which email content to ingest when integrating with Microsoft’s Exchange for enhanced email search. The initial cost estimate for this customer was approximately $400 per user per inbox. Microsoft’s Copilot, meanwhile, has “out of the box functionality” to support this use case, the employee noted.

“Is this product even ready for adoption at scale?” this person wrote in Slack.

Some Amazon employees raised red flags before Q’s launch, saying the release felt “rushed,” with little testing and heavy reliance on human reviewers, B-17 previously reported.

This is part of a broader culture at Amazon that encourages employees to speak up honestly about issues, so they can be fixed sooner. Other major technology rollouts by the company have faced early headwinds, only to steadily improve and succeed later. Amazon Q could be one of these products.

In an email to B-17, AWS spokesperson Patrick Neighorn said the open discussions are part of Amazon’s culture of encouraging employees to be “vocally self-critical” about their work. He added Amazon Q is seeing strong growth across the board.

“Being vocally self-critical is core to how we earn the trust of customers, and we seek and value builder feedback,” Neighorn wrote. “We already see large customers like Bayer, Smartsheet, and National Australia Bank using Amazon Q today, and we continue to use Amazon Q across our businesses, including the AWS sales organization and Prime Video.”

‘Fire drill’

Some AWS employees in the Slack channel pointed out the company’s ambitious marketing language for Q and asked whether the company needed to “tone it down” so customers don’t lose trust.

“It’s a fire drill waiting to happen,” one of these AWS employees wrote in Slack. “All we can do is plan on picking up the pieces once marketing is done slamming our customers against the wall of what Q can’t do.”

Neighorn, AWS’s spokesperson, told B-17 that Amazon has clear pricing for Amazon Q, and customers can use the AWS Pricing Calculator to calculate Amazon Q charges. He added Amazon Q gives customers control over what documents are indexed and how they are ingested.

“Amazon Q only launched in April, and we continue to add new features based on customer feedback,” Neighorn said. “Amazon Q has a rapidly growing customer base that is using Q for a variety of use cases.”

Amazon Web Services CEO Matt Garman

Disappointing early sales data

Internal data also hint at disappointing sales for Q.

One part of AWS’s sales team, with more than 3,000 people, missed a sales target for Q that it was supposed to hit by July, according to internal data obtained by B-17. The data, which was tracked by Amazon’s most senior leadership team, showed that Q’s sales fell short of its goals across all regions, most notably the North American market.

Amazon has also struggled to keep track of how this new AI product is selling.

AWS spokesperson Neighorn said the internal sales data obtained by B-17 came from “inaccurate preliminary figures based on an incomplete methodology that we fixed weeks ago.” B-17 knows the specific sales numbers and targets from the internal data, but is not publishing them due to Amazon correcting its methodology.

“Customers are excited about Amazon Q, which is seeing rapid customer adoption since its launch only four months ago and is already close to meeting our ambitious sales goals,” Neighorn said.

$260 million in efficiency gains

AWS CEO Matt Garman instructed a part of his sales team with roughly 6,500 employees to undergo a mandatory half-day training for Amazon Q earlier this year, according to an internal message seen by B-17.

The training, designed to build the “confidence and skills” needed to sell Q, covered everything from Amazon Q’s branding and security features to supply chain and coding overview, the message said.

Neighorn, AWS’s spokepseron, told B-17 that sales trainings are “standard practice” for enterprise technology companies, and any suggestion that it’s unusual is “false.”

Garman appears to have high hopes for AI assistants like Amazon Q. At an internal talk in June, he said software engineers may soon stop coding because of the advancements in these types of AI tools, B-17 previously reported. Amazon is also working on a separate AI chatbot, internally codenamed Metis.

For now, Amazon CEO Andy Jassy may be the biggest salesperson for Amazon Q. Last month, he wrote on LinkedIn that Amazon’s internal use of Q has helped the company gain “significant efficiencies,” adding that it saved the company “4,500 developer-years of work” and an estimated $260 million in “annualized efficiency gains.”

“It’s been a game changer for us, and not only do our Amazon teams plan to use this transformation capability more, but our Q team plans to add more transformations for developers to leverage,” Jassy wrote.