Fine-tuning LLMs encourages hallucinations

[@thankqwerty]

Thanks for sharing the paper.
In my experience with using Llama3-8B, in my benchmark dataset, I noticed that LLM has learned an incorrect fact or in contradiction with my application. I tried to clarify that in the prompt, but noticed the LLM is actually quite stubborn, and lead to quite fragile responses, i.e. the LLM sometimes get it right sometimes get it wrong with minimal changes in the prompt, could be as small as adding spaces.
I wonder if you have come across similar situation or papers that discuss this behavior.
Thanks.

[@VivekHaldar]

Yes that kind of brittleness is a common issue unfortunately.

[@gilinachum]

But why is the paper's fine tuning different than the original pre-training and alignment fine tuning that came before it. All expose the model to a mix of existing and new data...

[@VivekHaldar]

You are correct -- in principle fine-tuning works the same way as pre-training (updating weights), so FT can be thought of as continued PT.
Difference is in data used. One will FT when they have a domain-specific set of data that's very different from the PT data.

[@willtipton1698]

Nice video ty

[@hosseinmohammadi4574]

Interesting! Tnx