LoRA and Fine-Tuning: When Prompting Is Not Enough

Students should know when to prompt, when to use RAG, and when a small adapter or fine-tune is actually justified. In local AI, the model family is only one part of the system. The runtime, file format, serving path, hardware budget, evaluation set, and safety policy decide whether the model becomes useful.

Current source signal

Build the small version

Make a decision tree that chooses prompting, RAG, LoRA, or full fine-tuning for different failure modes.

1. 1Define the user task in one sentence.
2. 2Choose the smallest model and runtime that might pass that task.
3. 3Run one happy-path prompt and one failure-path prompt.
4. 4Record speed, memory pressure, output quality, and the exact reason for any failure.
5. 5Write the operating rule you would give a non-expert user.

adaptation_decision:
  if model_lacks_current_facts: use_RAG
  if output_style_is_wrong: improve_prompt_or_examples
  if repeated_format_task_with_many_examples: consider_LoRA
  if broad_capability_gap: choose_better_base_model

never: tune_without_eval_set

The big idea: tune only with evals. A local model app is not done when the model answers once; it is done when the whole workflow can be installed, measured, trusted, and recovered.