Our security principles behind Command

Building Command has been exciting. We’re reimagining how people interact with their finances, with the goal of creating a banking experience that feels as intuitive as the best software you use every day.

But with that comes significant responsibility. Financial data is highly sensitive — and often, deeply personal — and introducing new technology can naturally raise questions.

As the principal engineer on Command, I’d like to share more on how we prioritized security when we built this product. Our approach can be distilled into two core principles: rely on systems we already trust, and position AI as an orchestrator rather than an executor.

When you first open the Command chat, the interface will be new. But the underlying guardrails are largely built on our existing audit, authorization, permissions, risk, and security systems that’ve served our customers over the past seven years.

While building Command, we focused on reusing as much of that infrastructure as possible, in order to minimize risk and reduce the number of new variables.

Command is also not our first introduction to securely integrating AI into Mercury. Mercury Insights is a dashboard with built-in intelligence to provide trends and recommendations, which provides traceable answers so customers can see exactly where the AI pulled its information from. Additionally, we’ve utilized AI in our auto-categorization features. Those products gave us a chance to observe how AI performs in real customer workflows before actually letting it stage actions that involve moving money.

Our overall philosophy towards AI was to assume that LLMs will hallucinate. Rather than wonder if or when that would happen, we focused our efforts on ensuring that it’d be virtually impossible in those instances for the AI to do anything unintended or harmful to a user’s account.

With an AI agent embedded in financial operations, an important question is where authority lives. For us, at this stage, we’re not accepting “the model is careful” as an answer.

Command is built around this sequence:

• The model proposes.
• The product enforces.
• The user authorizes.

This is different than many other, more autonomous AI agents you may be accustomed to working with. Typically with those agents, actions happen on your behalf. The current iteration of Command is more like a deliberate assistant: staging information, working within your existing permissions, and requiring your sign-off to complete any action.

Let’s take an example of sending money.

If you ask Command to send money, it will ask you questions about what you want to do, fetch details from your accounts like people you’ve paid before, and, finally, stage. Command retrieves this information via read-only access from a real source of truth — your Mercury account.

As well, Command will follow your existing permissions structure. If the user would typically require admin approval to send money, Command will then tee up that next or final approval step.

After approval, we directly call our backend to take the action, bypassing the AI entirely. Keeping the AI out of the loop ensures that it can never attempt to do something without permission.

While this is a new product for us, and we can’t predict 100% of all use cases that our customers will try, we’ve gone both deep and wide in our evals to test AI behavior for the vast majority of user scenarios. And we’re confident in the guardrails we’ve put in place to mitigate any risk for even unforeseen use cases. Command is set up to address a specific set of actions, and if it can’t help, it sends you to a human.

Here are three goals that guide how we’ve built Command:

1. Preserve and adhere to the permission system that you have set up.
2. Provide a one-stop shop for your financial operations.
3. Help supercharge your workflow.

We understand that AI is evolving, and using it in your financial operations may be a new concept. That’s why Command is underpinned by a trusted infrastructure developed over years. And we’ve conducted hundreds of evals on AI behavior to make sure no unintended outcomes can actually occur.

Have questions? Want to learn more? Get more details on Mercury’s usage of AI in Command in the Help Center.

AI-generated responses and suggested actions may vary and are not guaranteed. Please review outputs before taking action.