19 lines
653 B
Markdown
19 lines
653 B
Markdown
# Examples
|
|
|
|
To run example scripts in this folder, one must first install `auto_gptq` as described in [this](../README.md)
|
|
|
|
## Basic Usage
|
|
Run the following code to execute `basic_usage.py`:
|
|
```shell
|
|
python basic_usage.py
|
|
```
|
|
|
|
## Quantize with Alpaca
|
|
To Run this script, one also need to install `datasets` via `pip install datasets`.
|
|
|
|
Then Execute `quant_with_alpaca.py` using command like this:
|
|
```shell
|
|
CUDA_VISIBLE_DEVICES=0 python quant_with_alpaca.py --pretrained_model_dir "facebook/opt-125m"
|
|
```
|
|
|
|
The alpaca dataset used in here is a cleaned version provided by **gururise** in [AlpacaDataCleaned](https://github.com/gururise/AlpacaDataCleaned)
|