AI BriefWire / Use Cases

Hybrid Semantic and Keyword Search for a Local AI Email Agent

A private local AI email agent named Llamail uses a hybrid retrieval approach combining semantic search via ChromaDB embeddings and exact keyword search via SQLite FTS5 to enable efficient and accurate email search and question answering on a personal mailbox of 18,000+ emails. The system merges normalized scores from both search methods with weighted fusion to balance conceptual understanding and exact token matching, providing fast (~3 seconds for search, ~7 seconds for Q&A) and relevant results on consumer hardware. The agent supports natural language queries and follow-ups via Telegram, with local LLM calls for answer generation based on retrieved email context.

May 11, 2026, 5:44 PM

StagePRODUCTION

Priority score8

Verification score10

Back to Use Cases Open source discussion

Executive Summary

ResultThe hybrid search approach dramatically improves retrieval relevance compared to semantic-only or keyword-only methods, enabling practical search and RAG-style Q&A on a...

Implementation Complexity-

Best forChromaDB (semantic search), SQLite FTS5 (keyword search), llama.cpp (LLM), Nomic Embed v2 MoE (embedding model) / Sviatoslav Barbutsa • Dev.to

Primary Outcome3seconds

The hybrid search approach dramatically improves retr...

8/10Priority score

10/10Verification score

PRODUCTIONStage

Verdict

High-value case for teams facing a similar - problem. Implementation effort is -, so it is worth prioritizing when the workflow pain is recurring, measurable, and owned by a team that can execute.

Should You Care?

Yes, if

Worth considering if this workflow is already losing value to this problem.
Move faster if operational value is measurable in your current operation.
Relevant when the task is close to: Enable accurate, fast, and context-aware email search and question answering by c...

No / wait, if

Pause if this limitation applies: Does not embed every chunk of long emails individually, only a master summary embedding; li...
Wait if ownership, compliance, or implementation capacity is unclear.

Implementation Complexity-

Estimated deployment: Not specified

Deployment timeline

ResearchPilotProductionScaling

Best Deployment Fit

Production teamsSimilar industryOwner teamChromaDB (semantic search), SQLite FTS5 (keyword search),...Local-only / low-volume operation

Implementation Risks

Does not embed every chunk of long emails individually, only a master summary embedding
limited context window (~8k tokens) restricts full mailbox visibility during Q&A
aggregate queries over entire mailbox are approximated with limited sample
punctuation-heavy queries may require escaping for FTS5

Source context

Sviatoslav Barbutsa • Dev.to

Who used AI

Sviatoslav Barbutsa (individual developer)

Industry

Role

Tool / model

ChromaDB (semantic search), SQLite FTS5 (keyword search), llama.cpp (LLM), Nomic Embed v2 MoE (embedding model)

Maturity

Repeatable

ROI type

Implementation effort

Context

Personal email management with a large mailbox (18,000+ emails) on consumer hardware, accessed via Telegram interface.

Task solved

Enable accurate, fast, and context-aware email search and question answering by combining semantic and keyword search methods locally without cloud dependencies.

Tools

ChromaDB for vector semantic search, SQLite FTS5 for keyword search, llama.cpp for local LLM inference, Nomic Embed v2 MoE embedding model, FastAPI backend, Telegram frontend, SQLite database for storage and chat history.

Result

The hybrid search approach dramatically improves retrieval relevance compared to semantic-only or keyword-only methods, enabling practical search and RAG-style Q&A on a large personal mailbox with response times suitable for phone use (~3 seconds for search, ~7 seconds for full Q&A)
The system handles exact matches (e.g., invoice numbers, sender emails) and conceptual queries (e.g., budget discussions) effectively, with a simple weighted score fusion
The local setup avoids complex infrastructure and cloud dependencies, providing a private, efficient AI email assistant.

Analyst Notes

Main challenge: Does not embed every chunk of long emails individually, only a master summary embedding; limited context window (~8k tokens) restricts full mailbox visibility during Q&A; aggregat...
Implementation effort: The technical piece is only part of the work; the harder question is whether ChromaDB for vector semantic search, SQLite FTS5 for keyword search, llama.cpp for local LLM inference, Nomic Embed v2 MoE embedding model, FastAPI backend, Telegram frontend, SQLite database for storage and chat history. can be owned, monitored, and reconciled in production.
Practical read: Best read as a - operational change with ROI upside when the pain is already measurable.

Source review

Open the original discussion for implementation details, constraints, and team context.

Open source discussionPublished: May 11, 2026, 5:44 PM

Opening the operator briefing

Hybrid Semantic and Keyword Search for a Local AI Email Agent

Yes, if

No / wait, if