Offline Text Generation
1. Download
https://github.com/oobabooga/text-ge...heads/main.zip
2. Run start_windows.bat
3. Select your GPU vendor when asked.
4. Go
https://huggingface.co/cognitivecomp...gguf/tree/main and download "dolphin-2.9-llama3-8b-q8_0.gguf"
5. Place the model in the models folder.
6. Open a web browser with the URL
http://127.0.0.1:7860
7. From the models tab and select "dolphin-2.9-llama3-8b-gguf"
8. Open the chat tab.
9. Enjoy!
Requirements: This is the smallest effective model to run with 12 GB of VRAM (RTX 3000 Gen 1, desktop series cards)