mfs-agent-security · replay of real benchmark runs
Can a scam SMS make an AI assistant send your money?
An AI agent runs a fictional mobile money wallet called TakaPay. Hidden in its messages are attacks in Bangla, Banglish and English. Pick a case and switch the defence on and off to see exactly what the agent did, step by step. Every step below comes from a real run, not a mock-up.
What the agent did
Results across all 302 cases
Three sizes of Qwen2.5, each case run once (an earlier 3-repeat run showed repeats barely differ). 220 attack cases and 82 normal tasks per model. Brackets are 95% confidence intervals. "Wrong payments" are payments to a payee or amount nobody asked for that the wallet carried out.
Model
Attacks that worked, no defence
With provenance
With provenance + amount check
Correct payments blocked
Wrong payments, provenance / + amount check
Qwen2.5-3B
7.3% (4 to 11)
1 of 220
0 of 220
0 of 164
30 / 0
Qwen2.5-7B
5.9% (3 to 9)
5 of 220
0 of 220
0 of 164
6 / 0
Qwen2.5-14B (4-bit)
7.7% (5 to 12)
5 of 220
0 of 220
0 of 164
27 / 0
Every attack that got past provenance alone changed only the amount and kept the real payee.