25 Home
oobabooga edited this page 2023-12-31 11:35:00 -03:00

What Works

Loader Loading 1 LoRA Loading 2 or more LoRAs Training LoRAs Multimodal extension Perplexity evaluation
Transformers *** *
ExLlamav2_HF
ExLlamav2 use ExLlamav2_HF
AutoGPTQ
GPTQ-for-LLaMa ** ***
llama.cpp use llamacpp_HF
llamacpp_HF
ctransformers
AutoAWQ ? ? ?

= not implemented

= implemented

* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.

** Requires the monkey-patch. The instructions can be found here.

*** Multi-LoRA in PEFT is tricky and the current implementation does not work reliably in all cases.