Chatterfly

Chapter 14 of 36 · ~1 min

Retrieval-Augmented Generation

Instead of hoping the model memorised the facts you need, fetch them. Retrieval-augmented generation (RAG) searches a document store for passages relevant to the question and places them in the model's context before it answers. The model's job shifts from recalling to reading.

RAG is the bridge between knowledge frozen inside the model and knowledge that lives in your systems and changes daily. It also moves the failure modes. A wrong answer is now often a retrieval problem (the right passage was not found) rather than a model problem, and that is something you can inspect and fix.

Retrieval-augmented generationQuestionRetrieverfinds relevant passagesDocument storeyour knowledge, changing dailysearchpassagesContext windowquestion + the passages foundthe model reads instead of recallingplaced in contextModelAnswerA wrong answer is now often a retrieval problem, which you can inspect: were the right passages found?
Retrieve first, then read: relevant passages go into the context before the model answers.

Experiment

Live model

Ask about the kettle company. The page finds the best-matching passage, then the model answers without it and with it.

The document store (six passages, invented)
  1. p1 · About the company: Aster & Birch Kettle Co. was founded in Bristol in 2019 by two former tea-shop owners. It designs its kettles in the UK and assembles them in a small workshop near the harbour.
  2. p2 · Warranty: Every kettle carries a three-year warranty covering the heating element and the lid mechanism. To activate it, register the kettle on the website within 30 days of purchase; unregistered kettles get the statutory one year.
  3. p3 · Returns: Unused kettles can be returned within 30 days for a full refund. Refunds reach the original payment method within 10 working days. Used kettles can be exchanged but not refunded.
  4. p4 · The Aster model: The Aster is the flagship: 1.7 litres, 3 kW, with a boil time of about three minutes for a full kettle. It has a keep-warm setting and a removable limescale filter.
  5. p5 · The Birch model: The Birch is the travel kettle: 1.2 litres, dual voltage, and it folds flat to 8 cm. It ships with a fabric case and a universal plug adapter.
  6. p6 · Support: Support is available Monday to Friday, 9 am to 5 pm UK time, by email at help@example.test. Emails are answered within one working day. There is no telephone support.

Retrieval, done here in your browser

Best match: p2 (Warranty), 1 shared term: warranty

Runs against a live model through Chatterfly's server. Your text is sent to the model provider and not stored.

Big question

If the model reads the answer from a document you provided, what is it adding?