Imagine receiving a customer support email meant for an entirely different company—one with the same name but a completely unrelated business. This is exactly what happened to the team behind the note-taking app Obsidian, and it’s a cautionary tale about the pitfalls of relying too heavily on AI in customer service. But here’s where it gets controversial: could this be a sign of a larger issue with how companies are using off-the-shelf AI tools to handle sensitive customer inquiries? Let’s dive in.
Steph Ango, CEO of the company behind the Obsidian note-taking app, recently shared on social media that his team’s support inbox received an email from a user of Obsidian Entertainment, the well-known video game developer. The user was trying to provide feedback on Outer Worlds 2 but was blocked by the game’s two-factor authentication (2FA) system. In response, Obsidian Entertainment’s support team sent the user an email—which Ango later shared on X—instructing them to contact a ‘specialized account security team.’ The catch? The email provided the support address for Ango’s company, the wrong Obsidian.
The email reads: ‘Since your issue is with the Obsidian account system and involves account security, this is something our specialized account security team needs to handle directly. They’re the only ones who can verify your ownership and safely disable the authenticator app link without a current code.’ It also asks the user to explain their issue and provide any proof of account ownership needed. Ango suspects the entire email, including the incorrect address, was AI-generated—though this couldn’t be confirmed.
And this is the part most people miss: When customers are directed to the wrong support team, they might unwittingly share personal or sensitive data in an attempt to prove their account ownership. This not only creates frustration but also poses a potential security risk. Ango highlighted this concern in his post, stating, ‘The perils of trusting an LLM with your customer support.’ He also questioned whether this was a reinforcement learning issue, suggesting that if the AI’s goal is to minimize support ticket interactions, passing the buck to another company might be seen as a ‘success’ by the system.
Obsidian Entertainment responded to Ango on X, apologizing for the mix-up and explaining that a team member had missed an attachment from the user. They assured the public they were working to resolve the issue and prevent future errors. While this incident was resolved, it raises broader questions about the reliability of AI in customer service. Are companies sacrificing accuracy and security for efficiency? And at what cost to the user experience?
Here’s the controversial question: Is it ethical for companies to use AI tools that might misdirect customers or mishandle sensitive information, even if it reduces support ticket volume? Let us know your thoughts in the comments—we’d love to hear whether you think this is a minor oversight or a symptom of a bigger problem in AI-driven customer service.
