Natural-Language Code Search: Replacing Grep with an LLM Index

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-ai-coding-LLM-code-search-natural-language-creators

1. A developer wants to find every location where a variable named 'userId' is used across a million-line codebase. Which search method is appropriate?

   1. Semantic LLM search, because it's faster for large codebases
   2. Grep, because it produces embeddings for faster retrieval
   3. Semantic LLM search, because it understands the intent behind variable usage
   4. Grep, because it guarantees finding every literal occurrence of the string

2. An engineer joins a new team and needs to understand where the payment processing logic lives in a large codebase. What capability of semantic search would be most helpful?

   1. Finding exact matches of 'payment' in filenames
   2. Running automated tests on payment modules
   3. Exploring the codebase conversationally using natural language
   4. Converting payment functions to a different programming language

3. A team is about to rename a function called 'processData' to 'handleUserData' across their entire codebase. Why must they use grep instead of semantic search?

   1. Semantic search is too slow for large rename operations
   2. Semantic search will refuse to make changes
   3. Grep automatically commits changes to version control
   4. Grep finds every literal occurrence, which is required to update all references

4. What problem does re-indexing an LLM code search index solve?

   1. It automatically fixes syntax errors in code
   2. It ensures the search index reflects the latest changes in the codebase
   3. It makes grep searches return results faster
   4. It generates new unit tests for recent changes

5. If your embedding index is more than a day behind the main branch, how should you treat its search results?

   1. As hints that need verification from actual code
   2. As ground truth that can be trusted completely
   3. As errors that should be reported to the team
   4. As suggestions that should be ignored entirely

6. In the context of code search, what does 'fuzzy product description' refer to?

   1. A detailed specification document
   2. An imprecise or incomplete description of what functionality you're trying to locate
   3. A bug report with incomplete information
   4. A search query containing regular expressions

7. A developer wants to find the main authorization function but doesn't know its exact name. They search for 'how we check if a user is logged in.' What concept allows this search to work?

   1. Code comments being indexed
   2. Function overloading detection
   3. Semantic search understanding intent rather than exact strings
   4. Grep's pattern matching capabilities

8. What is a 'canonical handler' in the context of code search?

   1. A file that cannot be deleted
   2. The primary or most authoritative version of similar code functions
   3. A broken piece of code that needs fixing
   4. A function that handles errors in search

9. Why can semantic LLM search not find code that was just written and not yet indexed?

   1. The code has syntax errors
   2. The LLM model is too old
   3. The code is in a different branch
   4. The embedding index hasn't been updated to include the new code

10. In semantic code search, what is an 'embedding'?

    1. A file name in a project
    2. A function's return value
    3. A comment in source code
    4. A numerical representation of code that captures its meaning

11. A developer finds code using semantic search and wants to document their search process for a code review. Where should they save both their semantic and grep queries?

    1. In an email to the team
    2. In the commit message
    3. In the pull request description
    4. In a personal text file on their desktop

12. What does 'code-navigation' mean in the context of AI-powered developer tools?

    1. Tools and methods for exploring and moving through a codebase
    2. The process of commenting out lines of code
    3. The way git branches are organized
    4. Writing documentation for functions

13. When semantic search surfaces 'near-duplicates' in code, what does it mean?

    1. It found multiple similar implementations of the same functionality
    2. It found broken code with syntax errors
    3. It found files with identical content
    4. It found code that has no tests

14. What does it mean for a search index to be 'stale'?

    1. The index contains only commented-out code
    2. The index is outdated and doesn't reflect current codebase state
    3. The index is encrypted and cannot be read
    4. The index is stored on a slow server

15. A developer needs to connect a UI string 'Processing payment...' to the backend function that generates it. Which search approach is best?

    1. Binary search in compiled files
    2. Semantic search, which can connect UI strings to backend functions
    3. Grep for the exact string 'Processing payment...'
    4. Git history search