01 /AI / MULTIMODAL RAG
OFFLINE MULTIMODAL RAG.
Ask a question and it searches every file, even photos and voice notes, then shows where each answer came from. It works with no internet, so private data never leaves the computer.
How it works
Every file. No internet. Cited answers.
Retrieval with the network unplugged. Text, OCR, CLIP image embeddings and Whisper transcripts land in FAISS, Neo4j and MongoDB; a query fans out to all three in parallel, Reciprocal Rank Fusion merges the rankings, and a local GGUF model answers with citations down to the page, image region or audio timestamp.
The hard parts
- Running embeddings, vector search and the LLM with no network access, for restricted settings like government and research.
- Searching across text, images and audio in one query.
- Combining vector and graph results into one ranked context.
Results
- Smart India Hackathon 2025 National Finalist.
- Text-to-image, image-to-text and audio-to-text search in one interface.
- Answers cite the exact page, image snippet or audio timestamp.
Stack
Reach the end of the file to complete this operation
