Forging Light with AI
AI is the most powerful tool to come out in an era. It will undoubtedly change the world. Let’s work... View more
Organizers:
- Organized by
-
-
Revisiting FLCCC LLM
-
Revisiting FLCCC LLM
With the inspiring Brighteon.ai release (https://healthranger.substack.com/p/we-are-building-the-infrastructure) and some recent advancements and improvements for localized training, we are revisiting the FLCCC LLM.
We are a small team and have many, many big initiatives that take up our time (like the big conferences). Projects like these tend to get put on the back burner for months at a time.
Here is a quick sampler of how much of a difference a single round of training and the right persona can make.
Today, I’m going to write a quick tutorial on how you can start training your own FLCCC LLM and even help us continue and improve upon this training.
This example will be a small dataset compared to the wealth of knowledge we would like to process. Brighteon is right when they talk about how time-consuming and expensive it is to train models with significant amounts of data. The first challenge is just gathering all the data you want to train on and putting it in a format that is ideal for training.
I’ve put together a minimal sample of FLCCC training materials. This sample needs to be significantly increased, especially with our 3rd successful conference and the great content presented there.
Download flccc-training-txt-zip
I only did a single round of training with this data, just to demo the impact. You can download the generated LORA bundle here:
The Guide
I am using Text Generation WebUI, found here:
https://github.com/oobabooga/text-generation-webui
If you are running it locally, I recommend being set up with conda environments. Anaconda is a practical solution for that.
If you don’t have the hardware to run locally, you can use RunPods, which is what I used to do the first LLM experiment dubbed Wilbur AI.
With Text Generation WebUI running, you need to grab the right type of model. I am a fan of Stability AI, but in this case, the Llama 2 GPT is more compatible with limited hardware training.
https://huggingface.co/TheBloke/Llama-2-7B-Chat-GPTQ
Click the copy model icon next to the model’s name and paste that into the model box in Text Gen WebUI, then click Download.
Once the model is downloaded, then refresh your model list and select it.
There are two recommended ways to load the model. For good performance while using the model, I recommend ExLlamav2_HF. However, for training, you should use the Transformers loader.
In my case, I had to enable the cache_4bit flag for both loaders. Additionally, for Transformers training, you need to disable exllama kernels.
if your model has loaded successfully, you should see a message similar to this one.
You can now go over to Chat tab and test that you can generate responses. If you can’t generate responses, you won’t be able to do training. This is also where you can form a baseline for how the default model with default characters will respond to your test prompts.
Next, head over to the Training tab.
The training text you downloaded above, you want to extract to a folder in text-generation-webui/training/datasets/flccc-training-txt/. Once there, you can refresh the Text file dropdown and select that data source.
Optionally, you can extract the downloaded LORA file to text-generation-webui/loras/FLCCC. You can refresh the “copy parameters from” and select that LORA. Make sure you name it the same as well. If your settings match the existing LORA, your training will be added to it.
You can also train from scratch with your own data set, or our data set + yours.
Once you start training, it can take many, many hours. This single round took me about 8 hours. The more iterations you do, the better the training. The larger the dataset, the longer the training sessions.
Finally, before you test your newly trained data, you’ll want to create a good persona for it. Here is an example of an FLCCC agent.
With your training complete and your character made, you need to reload the model into ExLlamav2_HF and apply your LORAs. It’s essential you reload the base model, as training can cause issues with the current loaded model. You’ll want to switch to a loader that performs better during chat anyway, as Transformers is painfully slow with response generation.
Now test your results! See how different the generated responses can be, with even just a little bit of training and prompt engineering.
Fin!
If you have other tips, resources, training data, or other insights, please share them here! LLMs are a powerful tool, but take an astounding number of resources to do properly.
healthranger.substack.com
The Health Ranger's newsletter | Mike Adams | Substack
Health freedom and a pro-human future. Click to read The Health Ranger's newsletter, by Mike Adams, a Substack publication with thousands of subscribers.
-
2 Replies
-
For those wanting to use RunPod and Text Gen Web UI, here is a good guide. The use of Text Gen WebUI locally is similar, so there is learning overlap there.
-
Text Gen Web UI team also has a pretty detailed guide on configuring your training settings.
https://github.com/oobabooga/text-generation-webui/wiki/05-%E2%80%90-Training-Tab
github.com
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models. - oobabooga/text-generation-webui
Log in to reply.