this post was submitted on 05 Aug 2023
17 points (100.0% liked)

LocalLLaMA

2235 readers
13 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS
 

I want to train, or more likely fine-tune, a model on about 20 years worth of email and text data that I've collected.

The goal would be to train it how to respond like me in simple cases.

It's there a particular base model I should start with?

I'm also interested in anyone's experience in doing this kind of thing themselves.

top 1 comments
sorted by: hot top controversial new old
[–] [email protected] 4 points 1 year ago

I'm inclined to fine tune/make a Lora for Llama (2? I'm not sure how well it's supported yet), you definitely won't want to train from scratch. The big thing seems to be formating the dataset properly, although I haven't actually done any of this myself yet. this blog seems to have a few good articles about it that you might be interested in reading.