History

PanQiWei be0c894d89 do not use fast tokenizer by default		2023-04-20 18:33:49 +08:00
..
dataset	add quant_with_alpaca.py and update README.md	2023-04-17 01:27:37 +08:00
basic_usage.py	update README.md and add some examples	2023-04-16 22:45:28 +08:00
quant_with_alpaca.py	do not use fast tokenizer by default	2023-04-20 18:33:49 +08:00
README.md	add quant_with_alpaca.py and update README.md	2023-04-17 01:27:37 +08:00

Examples

To run example scripts in this folder, one must first install auto_gptq as described in this

Basic Usage

Run the following code to execute basic_usage.py:

python basic_usage.py

To Run this script, one also need to install datasets via pip install datasets.

Then Execute quant_with_alpaca.py using command like this:

CUDA_VISIBLE_DEVICES=0 python quant_with_alpaca.py --pretrained_model_dir "facebook/opt-125m"

The alpaca dataset used in here is a cleaned version provided by gururise in AlpacaDataCleaned