潘其威(William)
15db2cdc44
Update _base.py
...
fix problem that recursively adding file extension to model_base_name
2023-05-30 07:26:42 +08:00
潘其威(William)
cfa7271617
Update _base.py
...
fix variable not found error
2023-05-30 07:22:10 +08:00
潘其威(William)
e5771fb206
Update _base.py
...
fix key mismatch
2023-05-30 06:44:45 +08:00
潘其威(William)
61a4ea035f
Update auto.py
...
add back save_dir for backward compatible
2023-05-30 06:43:00 +08:00
潘其威(William)
ea74e15199
Update _base.py
...
add model_name_or_path and model_file_base_name to BaseQuantizeConfig for better model file management; add back save_dir to .from_quantized() for backward compatible
2023-05-30 06:40:31 +08:00
潘其威(William)
0021417050
Update README.md
2023-05-30 05:56:46 +08:00
潘其威(William)
243bb2d56e
Update README_zh.md
...
fix typo
2023-05-30 05:56:04 +08:00
潘其威(William)
9a10b8496a
Update README_zh.md
...
merge the example code of downloading from and uploading to HF Hub into simplest usage code above to keep README compact.
2023-05-30 05:53:27 +08:00
潘其威(William)
17db71491f
Update README.md
...
merge the example code of downloading from and uploading to HF Hub into simplest usage code above to keep README compact.
2023-05-30 05:49:29 +08:00
TheBloke
b7bb50b4d5
Fix bug added after merge
2023-05-25 07:05:51 +01:00
Tom Jobbins
492255b400
Merge branch 'main' into TheBloke_support-HF-download
2023-05-25 07:02:13 +01:00
潘其威(William)
18c7ce5875
Merge pull request #100 from PanQiWei/improve_cpu_offload
...
Improve CPU offload
2023-05-24 18:48:37 +08:00
PanQiWei
c341a6df2f
update tutorial
2023-05-24 18:48:19 +08:00
PanQiWei
ac14180946
update tutorial
2023-05-24 18:31:59 +08:00
PanQiWei
065fd1de35
update README
2023-05-24 18:26:47 +08:00
PanQiWei
e6ba062c08
update basic usage example code
2023-05-24 17:58:01 +08:00
PanQiWei
94ef4d5ada
update basic usage example code
2023-05-24 17:56:46 +08:00
PanQiWei
c89bb6450c
correct typo of function name
2023-05-24 17:43:38 +08:00
PanQiWei
10347fdd7b
remove full_cpu_offload argument and unify model dispatch strategy
2023-05-24 17:41:04 +08:00
PanQiWei
379f24c2a5
remove add_align_logits_hook_to_model
2023-05-24 17:01:57 +08:00
PanQiWei
749dba1a7e
disable add_align_logits_hook_to_model for now
2023-05-24 13:42:06 +08:00
PanQiWei
58c1b509f0
support add_align_logits_hook_to_model
2023-05-24 12:50:30 +08:00
PanQiWei
21ab7c435a
make comments more readable
2023-05-24 11:38:29 +08:00
PanQiWei
c31b370228
make_sure_not_tensor_in_meta_device before load checkpoint
2023-05-24 11:32:45 +08:00
PanQiWei
63f1b4e073
remove comment
2023-05-24 11:23:07 +08:00
PanQiWei
057c39e3f2
fix meta device bug when use low_cpu_mem_usage
2023-05-24 11:19:59 +08:00
PanQiWei
e2e7809a1f
always to enable QuantLinear bias to make compatible with model quantized from other frameworks
2023-05-24 10:56:31 +08:00
PanQiWei
8e034b28bc
remove duplicate code
2023-05-23 23:48:15 +08:00
PanQiWei
4373d6b29c
Merge branch 'main' into improve_cpu_offload
2023-05-23 23:47:33 +08:00
PanQiWei
191da8141e
fix device mismatch
2023-05-23 23:22:52 +08:00
PanQiWei
e4e90e8b0a
add warmup_triton method
2023-05-23 23:18:46 +08:00
PanQiWei
ed14d3a786
fix save quantized model failed when load pretrained model using CPU offload
2023-05-23 23:17:11 +08:00
潘其威(William)
7820322089
Merge pull request #66 from LexSong/main
...
Fix CUDA out of memory error in qlinear_old.py
2023-05-23 23:04:45 +08:00
PanQiWei
6476ee4235
add options: 'low_cpu_mem_usage' and 'full_cpu_offload'
2023-05-23 22:51:00 +08:00
PanQiWei
c63959365a
update setup.py
2023-05-23 19:30:47 +08:00
PanQiWei
1b2159bd4c
add more help functions
2023-05-23 19:30:28 +08:00
PanQiWei
db63c0876a
half out
2023-05-23 16:08:28 +08:00
潘其威(William)
1bb7be3dd3
Update issue templates
2023-05-23 15:55:48 +08:00
潘其威(William)
a85d65e915
Update issue templates
2023-05-23 15:53:07 +08:00
Lex Song
f2ab4fab46
Fix CUDA out of memory error in qlinear_old.py
...
Add a missing line from qlinear.py to qlinear_old.py to convert the output tensor.
This resolves a CUDA out of memory error that occurred without this line.
2023-05-20 21:10:11 +08:00
TheBloke
bf633c298e
Clean up some unused params
2023-05-20 10:32:27 +01:00
潘其威(William)
d4011d29c6
Merge pull request #92 from PanQiWei/fix_triton_integration_bugs
...
fix ImportError when triton is not installed
2023-05-20 17:01:14 +08:00
潘其威(William)
809efa6fcb
Update README_zh.md
2023-05-20 16:53:27 +08:00
潘其威(William)
082e76713e
Update README.md
2023-05-20 16:52:43 +08:00
潘其威(William)
0ca1752a9b
Merge pull request #93 from TheBloke/TheBloke_rename-quant_cuda2
...
Rename 'quant_cuda' to 'autogptq_cuda' to avoid conflicts with existing GPTQ-for-LLaMa installations.
2023-05-20 16:44:02 +08:00
PanQiWei
b803369719
update quant_with_alpaca.py
2023-05-20 16:43:21 +08:00
PanQiWei
f78f074409
update quant_with_alpaca.py
2023-05-20 16:42:34 +08:00
TheBloke
898f1ef62d
Rename 'quant_cuda' to 'autogptq_cuda' to avoid conflicts with existing GPTQ-for-LLaMa installations.
2023-05-20 09:33:51 +01:00
PanQiWei
73b5952f5e
fix not return directly when triton is not installed
2023-05-20 16:21:52 +08:00
PanQiWei
86b3b52c63
fix ImportError when triton is not installed
2023-05-20 16:15:20 +08:00
