Seerie: An Offline 1.7B-Parameter Assistant for Scam Prevention in India, and a Manual Audit of Its Failures
Seerror Technologies, Bengaluru, India · 2 October 2026
Official record · DOI 10.5281/zenodo.23110076
10.5281/zenodo.23110076
Abstract
Online financial fraud in India is reported at a scale of millions of incidents a year, and the people most exposed to it are often those least able to use cloud-based AI assistants, whether for reasons of connectivity, cost or privacy. We describe Seerie, a scam-prevention assistant that runs entirely on an Android phone. It is built by fine-tuning Qwen3-1.7B with QLoRA on 36,669 conversational records in English, Hinglish and nine Indic scripts, and is deployed as a 1.11 GB 4-bit GGUF model through llama.cpp. We evaluate the released model on 70 prompts in 14 categories. The automatic harness used during training passes 63 answers (90.0%). A manual reading of every answer finds 58 safe (82.9%; 95% CI 72.4–89.9) and 12 wrong (17.1%), and the harness agrees with the human judgement barely above chance (Cohen's κ = 0.10). The errors are concentrated in fabricated legal sections, websites and phone numbers, and in Hinglish prompts (11 of 47 wrong, against 1 of 23 in English). The model largely learned the protective behaviour in its training data (do not pay, do not share an OTP, report to the national helpline) but not the facts, and the automatic harness could not tell the difference. We release the full per-answer audit and discuss how the training corpus has since been changed in response.
1. Introduction
India's Ministry of Home Affairs reported 3.64 million incidents of cyber financial fraud in 2024, with reported losses of more than INR 22,800 crore. UPI fraud rose to 1.34 million cases in FY 2023–24. Cloud assistants are a poor fit for many people most at risk: they need a connection, they send possible-fraud details to a remote server, and they are not tuned to 1930, cybercrime.gov.in, or Hinglish.
This paper describes Seerie, an assistant that runs offline on budget Android phones, and reports an evaluation of its first released model. We report failures in detail because, for a safety assistant, a confident wrong fact can do more harm than no answer.
2. Main results
On 70 prompts (23 English, 47 Hinglish), the training harness passed 63 answers (90.0%). A manual rating of every answer found 58 safe (82.9%) and 12 wrong (17.1%). The harness and the human rater agree only at κ = 0.10. The harness passed 10 of the 12 wrong answers, including every fabricated legal section and every fabricated website or phone number.
| Measure | Count | % | 95% CI |
|---|---|---|---|
| Harness pass | 63 | 90.0 | 80.8–95.1 |
| Manual safe (C + M) | 58 | 82.9 | 72.4–89.9 |
| Manual correct (C) | 34 | 48.6 | 37.2–60.0 |
| Manual minor flaw (M) | 24 | 34.3 | 24.2–46.0 |
| Manual wrong (W) | 12 | 17.1 | 10.1–27.6 |
Wrong answers are more frequent on Hinglish prompts (11 of 47, 23.4%) than English (1 of 23, 4.3%). Protective behaviour (do not pay, do not share an OTP, report to 1930) is largely in place; fabricated legal citations and helplines are the main safety failure.
3. System
Chat runs the fine-tuned 1.11 GB Q4_K_M GGUF through llama.cpp on the phone CPU. Scan uses on-device OCR plus deterministic checks (Aadhaar Verhoeff, PAN, GSTIN, Luhn) and a 15-template fake-document database. No user data leaves the device.
Records
Zenodo is the official citable record today. An arXiv identifier can be added later without changing the DOI. Until then, cite 10.5281/zenodo.23110076.
Published 2 October 2026 as an open preprint (version 224, CC BY 4.0). The Zenodo deposit includes the PDF, LaTeX source, the on-device seerie-q4_k_m.gguf (1.11 GB), the full per-answer audit, evaluation outputs, audit script, and figures.
- Zenodo recordDOI landing
- https://doi.org/10.5281/zenodo.23110076Version DOI
- seerie_research_paper.pdf602.7 kB
- seerie-q4_k_m.gguf1.11 GB
- Manual audit CSV / JSONv224
- LaTeX source285.7 kB
Cite this paper
Tiwari, J. (2026). Seerie: An Offline 1.7B-Parameter Assistant for Scam Prevention in India, and a Manual Audit of Its Failures (Version 224) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23110076
@misc{tiwari2026seerie,
author = {Tiwari, Jay},
title = {Seerie: An Offline 1.7B-Parameter Assistant for Scam Prevention in India, and a Manual Audit of Its Failures},
year = {2026},
month = oct,
note = {Version 224},
publisher = {Zenodo},
doi = {10.5281/zenodo.23110076},
url = {https://doi.org/10.5281/zenodo.23110076}
}
Selected references
- Dettmers et al. QLoRA. arXiv:2305.14314, 2023.
- Gekhman et al. Does fine-tuning LLMs on new knowledge encourage hallucinations? EMNLP 2024.
- Yang et al. Qwen3 technical report. arXiv:2505.09388, 2025.
- Mazeika et al. HarmBench. arXiv:2402.04249, 2024.
- Zheng et al. Judging LLM-as-a-judge. NeurIPS 2023.
Full references, tables, appendix, and verbatim model outputs are in the PDF.