Parameter-Efficient Fine-Tuning (PEFT) for large language models has become one of the most discussed topics in artificial intelligence. The rapid growth of AI has come with skyrocketing costs, and training large language models (LLMs) has become so expensive that only corporations with billion-dollar budgets can afford it. Research data shows that the development of ChatGPT-4 cost between forty-one million and seventy-eight million dollars, while Google’s Gemini 1 reached nearly two hundred million dollars.