Context Windows, Lost in the Middle, and Practical Limits

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ai-foundations-context-windows-final1-creators

1. A developer wants to include both usage instructions and a system prompt in a single request to a long-context model. Based on the lost-in-the-middle phenomenon, where should the most critical instructions be placed?

   1. At the beginning only, since that's where models focus attention
   2. In a separate follow-up message after the main document
   3. Only in the middle where the model processes it most thoroughly
   4. At the very beginning and the very end of the input

2. A student inserts a unique sentence at three positions in a 30,000-word document: the beginning, middle, and end. When querying the model, which position is most likely to be recalled accurately?

   1. All three positions equally
   2. The middle position
   3. The beginning position
   4. The end position

3. A company is building a system that needs to answer questions about a large policy document. Which approach would likely be most accurate according to research on long-context models?

   1. Retrieving relevant passages and feeding only those to the model
   2. Splitting the document into equal sections and including all of them
   3. Asking the model to summarize the entire document first
   4. Pasting the entire policy document into the context window

4. What is a key limitation of models with million-token context windows that users should be aware of?

   1. They automatically discount information older than one week
   2. They can only process text, not images or code
   3. They require internet connectivity to function
   4. They cannot treat a 1M-token context as perfect, uniform memory

5. A developer is charged $2 and waits 3 minutes for a single 1M-token request to complete. What does this illustrate about very long context usage?

   1. The model's accuracy has improved substantially
   2. There are significant cost and time implications for processing very long contexts
   3. The API is malfunctioning and needs a refund
   4. The model has reached its processing limit

6. Why is chunking a document into smaller sections considered a best practice when working with long contexts?

   1. Because smaller chunks are always cheaper to process
   2. Because the model can only read one chunk at a time
   3. Because chunking increases the total context window size
   4. Because it helps the model navigate and retrieve relevant information more reliably

7. A user includes multiple sections in a long document with clear headers like 'Background,' 'Methods,' and 'Results.' What benefit do these structured headers provide?

   1. They automatically improve the model's reasoning ability
   2. They increase the total token count
   3. They help the model navigate long inputs more effectively
   4. They make the document look more professional

8. What does the term 'lost in the middle' refer to in the context of large language models?

   1. The model loses connection to the internet mid-generation
   2. The model forgets information from previous conversations
   3. Performance degrades for content placed in the middle of long inputs
   4. Information at the start and end of inputs gets deleted

9. A user wants to verify that the model correctly captured specific facts deep within a lengthy input. What strategy would be most effective?

   1. Including a direct verification query that references the specific facts
   2. Asking the model to confirm it read every single word
   3. Asking a general question about the document's contents
   4. Assuming the model has perfect recall since it has the full context

10. Which statement accurately describes a trade-off between using retrieval (RAG) versus stuffing an entire corpus into the context window?

    1. RAG is slower but more accurate than full document stuffing
    2. RAG is faster, cheaper, and typically more accurate for production cases
    3. There is no difference between the two approaches
    4. RAG requires more tokens, making it more expensive

11. A developer assumes that because a model supports 1 million tokens, it will equally attend to every part of a 1M-token input. What assumption is this?

    1. An incorrect assumption that ignores the lost-in-the-middle effect
    2. A minor inaccuracy that doesn't affect results
    3. A correct assumption about how modern models work
    4. A reasonable assumption based on model marketing

12. A user submits a 500,000-token document to a model and notices it takes several minutes to generate a response. What is the primary reason for this delay?

    1. The document is being stored in a database
    2. The model is waiting for human approval
    3. The model is thinking about the answer deeply
    4. The computational cost of processing very long contexts is substantial

13. A researcher wants to test whether a model actually attended to a specific sentence in a long document. What approach would demonstrate this?

    1. Checking how many tokens the document contained
    2. Asking the model to summarize the entire document
    3. Asking the model if it read the document
    4. Running a retrieval query that targets that specific sentence and comparing results across different positions

14. What can users never know with certainty when working with long-context models?

    1. The exact position of a given sentence in the input
    2. The total cost of the request
    3. Which specific sentence the model attended to in producing an answer
    4. The length of the response

15. Even with models that advertise million-token context windows, what fundamental memory limitation persists?

    1. Models forget everything after generating a response
    2. Models can only hold 100,000 tokens in active memory
    3. Models do not treat long contexts as perfect, uniform memory
    4. Models cannot process text longer than 10,000 tokens