Your team’s last two years of architecture decisions, incident postmortems, and “why we ditched that vendor” threads are sitting in a Slack export ZIP nobody can query. In 2026 the free-tier 90-day limit means the messages that mattered are already gone from search, and the export you did pull is a directory of dated JSON files where threads are shredded across days, user IDs are opaque hashes, and half the messages are join/leave subtypes and bot noise. Naive chunking on that produces a retriever that confidently surfaces a 2023 decision that was reversed in a thread three weeks later — and a private-channel message in an answer to someone who was never in that channel.
This is for developers who ship Python and have already built at least one working RAG pipeline — you know what an embedding is, you’ve used a vector store, and you can read a FastAPI route without hand-holding. Assumes comfort with JSON wrangling, async ingestion, and Docker. Out of scope: Slack app OAuth and Enterprise Grid Discovery API pulls (this works from a standard workspace export), fine-tuning, agent frameworks beyond retrieval, and any managed RAG-as-a-service platform — you’re running LlamaIndex and Qdrant yourself.
Honest framing: retrieval over conversational data is genuinely harder than retrieval over docs, and the models are unevenly good at it. LLMs are strong at summarizing a reconstructed thread into a coherent answer and at generating your evaluation question set. They are bad at knowing which of two contradicting messages is current, bad at inferring who was allowed to see what, and bad at judging whether sarcasm in a channel was a real decision. Permission mapping and the recency/supersession logic are where human review is non-negotiable — you validate the channel-to-audience map by hand before this touches a production bot, every time.
What This Guide Covers
- How to read an export’s structure so you know what you actually have before writing a line of parser code
- A normalization approach that turns opaque user and channel IDs into readable, filterable metadata
- Which message subtypes to drop, which to keep, and why the wrong call here quietly poisons your index
- Thread reconstruction that reassembles reply chains split across daily files into chunks that read as conversations
- Capturing the signals plain text loses — shared files, links, reactions, and pins — as ranking-useful metadata
- A custom LlamaIndex node parser built for conversational data instead of prose paragraphs
- Qdrant collection design with named vectors, payload indexes, and quantization sized for real message volume
- Hybrid dense-plus-sparse retrieval so exact terms like error codes, service names, and ticket IDs still hit
- Time-decay reranking that stops superseded decisions from outranking the thread that reversed them
- A permission-aware retrieval layer that makes private-channel leakage structurally impossible, not merely unlikely
- Incremental re-indexing so new messages land without a full rebuild every week
- Building a golden question set and scoring it on hit rate and MRR, so tuning is measured rather than vibes
- Full cost math for a 2M-message workspace, with a line-by-line comparison against commercial search vendors
- Shipping the thing: a FastAPI service, a Slack bot front end, and drift monitoring that tells you when quality slips
Instant online access the moment checkout completes — the full guide is available immediately, no waiting on an email, no upsell, no follow-on tier to buy.
Reviews
There are no reviews yet.