🌾 Builder Series #4: Fine-Tuning a Domain-Specific LLM with LoRA on AgriQA

“This was not a lab experiment. This was built in the wild — with free tools, local hardware, and real farmer data.”

🌟 Overview

We fine-tuned a lightweight LLM on the AgriQA dataset using LoRA — a memory-efficient technique — to produce accurate responses for Indian agriculture-based questions.

Despite failed cloud attempts, we succeeded using a 12GB consumer GPU and published a working adapter on Hugging Face.

---

🧠 Why LoRA?

Traditional LLM fine-tuning is GPU-heavy. LoRA (Low-Rank Adaptation) changes that:

• Freezes base model

• Adds small trainable matrices to attention layers

• Reduces memory & compute by over 90%

✅ Result: Efficient fine-tuning on consumer-grade machines.

---

🔧 Stack & Setup

• Model: TinyLlama-1.1B-Chat

• Dataset: AgriQA (farming Q&A)

• Libraries: HuggingFace transformers, peft, bitsandbytes

• Platform: Local GPU (12GB RTX)

• Precision: 4-bit quantization via BitsAndBytesConfig

---

💥 What Didn’t Work First (Kaggle + Colab Woes)

PlatformIssueKaggleSession expired mid-runColabGPU reset, runtime disconnects

After repeated frustration, we moved to a fully local setup.

---

💻 Local Setup (with CUDA)

• Installed CUDA Toolkit + NVIDIA drivers

• Verified GPU access with torch.cuda.is_available()

• Enabled bfloat16 + 4-bit quantization

• Training time per epoch: ~8 hrs

---

🗂️ Data Formatting (Instruction Tuning)

We used the SFT format (Supervised Fine-Tuning), common in instruction-tuned models:

### Instruction: What is the ideal fertilizer for tomato crops? 
### Response: Use a balanced NPK fertilizer in a 10-10-10 ratio.

We also expanded prompts using paraphrasing and regional language influences.

---

🔄 Epoch Simulation Trick

We split the dataset into 200–300 sample slices to simulate longer epochs on small RAM and VRAM environments.

---

🔬 Base vs Fine-Tuned: Real Prompt Examples

🌾 Prompt 1: When should paddy be planted in Tamil Nadu?

🔸 Base Model Output:

Paddy should be planted during the monsoon season.

🔹 Fine-Tuned LoRA Output:

In Tamil Nadu, paddy cultivation is typically done during the Kuruvai season (June–July) and the Samba season (August–November), depending on the region and water availability.

✅ The fine-tuned model provides regional accuracy and includes local terminology — a huge improvement in context.

---

🐛 Prompt 2: How to control aphids in brinjal crops?

🔸 Base Model Output:

Use pesticides or insecticides as recommended by agricultural experts.

🔹 Fine-Tuned LoRA Output:

To control aphids in brinjal crops, apply neem oil spray at 5 ml per liter of water weekly and use yellow sticky traps to monitor infestation levels.

✅ More actionable, natural, and grounded in practical advice. The base response was too vague.

---

💧 Prompt 3: What is drip irrigation and how is it useful for sugarcane?

🔸 Base Model Output:

Drip irrigation is a method of watering crops.

🔹 Fine-Tuned LoRA Output:

Drip irrigation is a micro-irrigation technique that delivers water directly to the roots of the sugarcane plant, improving water efficiency and reducing weed growth.

✅ Better structure and explanation. The LoRA model understands both what and why — while the base model only outputs a definition.

---

---

🤗 Try the Model on Hugging Face

🔗 AgriQA TinyLlama LoRA Adapter

Model Specs

• Base: TinyLlama/TinyLlama-1.1B-Chat

• Type: LoRA

• Size: ~4.5MB

• Dataset: shchoi83/agriQA

• Task: Instruction-based agri-Q&A

• Trained by: @theone049

---

💡 Key Takeaways

• LoRA democratizes LLM fine-tuning — it works even on mid-tier GPUs

• Instruction formatting is critical

• Cloud ≠ control — local machines gave us resilience

• LoRA adapters can produce high-utility, low-latency outputs

---

🚀 What’s Next

• Try QLoRA on larger agri or legal corporation

• Build a chatbot UI around the adapter

• Set up automated evaluation and deployment pipeline

---

Thanks for reading Builder Series #4. Built with friction. Deployed with pride.
🛠️ More builds coming soon.