qwopqwop200
|
eb5a27f48c
|
Update basic_usage_wikitext2.py
|
2023-04-29 18:18:35 +09:00 |
|
qwopqwop200
|
3b74c9758e
|
change save name
|
2023-04-29 18:18:21 +09:00 |
|
qwopqwop200
|
b5eb906ac9
|
change save name
|
2023-04-29 18:17:59 +09:00 |
|
qwopqwop200
|
05733ae482
|
change save the name
|
2023-04-29 18:17:33 +09:00 |
|
qwopqwop200
|
1792cd1111
|
change save the name
|
2023-04-29 18:16:48 +09:00 |
|
ZXED
|
24a371d14a
|
use the same Optional style as in other params
|
2023-04-29 09:52:11 +03:00 |
|
ZXED
|
c22770188d
|
allow user to set trust_remote_code flag manually
|
2023-04-29 09:52:11 +03:00 |
|
ZXED
|
b3f19a7ba7
|
support custom model name when loading the model
|
2023-04-29 09:52:11 +03:00 |
|
ZXED
|
ea8ab73343
|
support custom quantize_config when loading the model
|
2023-04-29 09:51:50 +03:00 |
|
PanQiWei
|
16d8dd200f
|
remove non-parameters module from MOSSGPTQForCausalLM.outside_layer_modules
|
2023-04-29 10:58:29 +08:00 |
|
PanQiWei
|
d9e7363fa8
|
update README.md
|
2023-04-29 01:47:40 +08:00 |
|
PanQiWei
|
b490ab004e
|
remove override of _resize_attention_mask for llama and opt
|
2023-04-28 23:08:42 +08:00 |
|
潘其威(William)
|
1d91fded6c
|
Merge pull request #31 from PanQiWei/add-raise-exception-and-gpt2-xl-example-add
Add example to use with GPT2-XL and raise an error when infeature and outfeature are not be divisible by 256
|
2023-04-28 22:52:18 +08:00 |
|
qwopqwop200
|
ae8b1a22a3
|
change global to local
|
2023-04-28 23:18:39 +09:00 |
|
qwopqwop200
|
e914b9b1bd
|
update support 256 not div
|
2023-04-28 22:48:23 +09:00 |
|
qwopqwop200
|
c9215a1b5b
|
change div num
|
2023-04-28 22:42:29 +09:00 |
|
qwopqwop200
|
7a38c2a6ef
|
add basic_usage_gpt_xl
|
2023-04-28 22:32:46 +09:00 |
|
qwopqwop200
|
19f167e58b
|
add raise-exception
|
2023-04-28 22:24:44 +09:00 |
|
潘其威(William)
|
1e353a8dc5
|
Merge pull request #24 from PanQiWei/speedup_quantization
Offloading and Multiple devices quantization/inference
|
2023-04-28 18:50:12 +08:00 |
|
PanQiWei
|
5055d785b6
|
update README.md
|
2023-04-28 18:44:06 +08:00 |
|
PanQiWei
|
3f761568a8
|
update example code
|
2023-04-28 18:26:16 +08:00 |
|
PanQiWei
|
bdb713b5a3
|
add batch_size to model.quant() api
|
2023-04-28 18:26:07 +08:00 |
|
PanQiWei
|
41564a48db
|
make data_utils.py as global utils
|
2023-04-28 18:08:58 +08:00 |
|
PanQiWei
|
e189d91004
|
update example code
|
2023-04-28 17:56:52 +08:00 |
|
PanQiWei
|
3dfc87bec3
|
return module in .to function
|
2023-04-28 17:20:46 +08:00 |
|
PanQiWei
|
789f821e6c
|
update example code
|
2023-04-28 17:17:38 +08:00 |
|
PanQiWei
|
a69a73a22c
|
fix device mismatch when directly using model to inference after quantization
|
2023-04-28 16:41:46 +08:00 |
|
PanQiWei
|
892eeb40e0
|
update example code
|
2023-04-28 16:23:02 +08:00 |
|
潘其威(William)
|
092bd502d0
|
Merge pull request #29 from PanQiWei/fix-bug-speedup_quant
Fix bug speedup quant and support gpt2
|
2023-04-28 12:02:55 +08:00 |
|
qwopqwop200
|
c4f82ac628
|
Merge pull request #30 from PanQiWei/add-gpt2
Add gpt2
|
2023-04-28 09:19:20 +09:00 |
|
qwopqwop200
|
329a64ed40
|
support conv1d,conv2d
|
2023-04-28 09:15:42 +09:00 |
|
qwopqwop200
|
bb9afe8b61
|
support conv1d,conv2d
|
2023-04-28 09:15:13 +09:00 |
|
qwopqwop200
|
c1b7c7647d
|
support conv1d
|
2023-04-28 09:14:44 +09:00 |
|
qwopqwop200
|
ac41f68532
|
add gpt2
|
2023-04-28 09:14:05 +09:00 |
|
qwopqwop200
|
dad249990c
|
add gpt2
|
2023-04-28 09:13:22 +09:00 |
|
qwopqwop200
|
435eebee4b
|
support conv1d,conv2d
|
2023-04-28 09:13:00 +09:00 |
|
qwopqwop200
|
cc0f71a568
|
add gpt2
|
2023-04-28 09:11:50 +09:00 |
|
qwopqwop200
|
3f90a22632
|
fix bug
|
2023-04-28 08:26:58 +09:00 |
|
qwopqwop200
|
9c38393e31
|
fix bug about wf meta device
|
2023-04-28 08:26:11 +09:00 |
|
PanQiWei
|
d0cd5af5d3
|
make code more robust
|
2023-04-28 01:29:12 +08:00 |
|
PanQiWei
|
51d2e53130
|
add support to cpu offloading and multi gpus inference on quantized model
|
2023-04-28 00:53:57 +08:00 |
|
PanQiWei
|
9c44365836
|
Merge remote-tracking branch 'origin/speedup_quantization' into speedup_quantization
|
2023-04-28 00:31:03 +08:00 |
|
PanQiWei
|
48737db0ff
|
update setup.py
|
2023-04-28 00:30:51 +08:00 |
|
PanQiWei
|
3aac76e71a
|
align 'from_pretrained' api
|
2023-04-28 00:30:05 +08:00 |
|
PanQiWei
|
b14dca9207
|
disk offload assertion
|
2023-04-27 21:31:53 +08:00 |
|
PanQiWei
|
7a3397e7ba
|
add cpu offload when doing quantization
|
2023-04-27 21:25:24 +08:00 |
|
PanQiWei
|
ac3f7054e0
|
big fix
|
2023-04-27 19:33:25 +08:00 |
|
PanQiWei
|
498de923f2
|
support multi gpus quantization
|
2023-04-27 18:48:43 +08:00 |
|
潘其威(William)
|
a21205f609
|
Merge pull request #23 from PanQiWei/add-option
add option
|
2023-04-27 16:51:33 +08:00 |
|
qwopqwop200
|
8b6ee04aee
|
add option
|
2023-04-27 17:29:36 +09:00 |
|