Fine-Tuning (LoRA & QLoRA)
💡 Quick Definition: Low-Rank Adaptation technique that trains lightweight adapter weights on base LLMs using minimal GPU VRAM.
Detailed Explanation & Workplace Application
Instead of retraining all 70 billion parameters of an LLM, LoRA freezes base model weights and trains small low-rank decomposition matrices, reducing GPU memory requirements by 80% with near-zero quality loss.
Practical Syntax / Framework Formula
from peft import LoraConfig, get_peft_model config = LoraConfig(r=8, lora_alpha=32, target_modules=["q_proj", "v_proj"])