simcop2387/AutoGPTQ
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity
538 commits 7 branches 13 tags 8.1 MiB
Commit graph

110 commits

Author SHA1 Message Date
PanQiWei
ab6faa6496 implement gptj attention and mlp fused ops injection logic 2023-08-06 14:55:06 +08:00
PanQiWei
bacac399d3 abandon change trainable mode after model is loaded; support specify customized AttentionOp 2023-08-06 14:15:43 +08:00
PanQiWei
c71f5cdf12 add '_fuse_attention' and '_fuse_mlp' abstract static methods 2023-08-06 12:45:08 +08:00
Felix Marty
1f99b94ae2 fix revision 2023-07-31 15:03:33 +00:00
PanQiWei
ff1f100ded remove argument 'save_dir' in method from_quantized 2023-07-26 17:58:04 +08:00
潘其威(William)
bbc4a7c455
Merge pull request #208 from TheBloke/TB_Add_SafeTensors_Metadata 
Add Safetensors metadata saving, with some values saved to each .safetensor file
 2023-07-26 11:54:47 +08:00
TheBloke
2647c92743 safetensors_metadata: add conversion to str() for input metadata to avoid errors from save_safe. Warn if this results in keys being overwritten. 2023-07-25 21:14:21 +00:00
TheBloke
ee7d80945b Add version to metadata using new value 2023-07-25 14:25:24 +00:00
TheBloke
eeaf5ebc53 Extend huggingface_hub features to AutoGPTQForCausalLM.from_pretrained() so models can be quantised from the hub including using a private token and revision/branch etc 2023-07-25 13:26:37 +00:00
TheBloke
c9124e3fc7 Fix revision and other huggingface_hub args for .from_quantized(), which were not being passed through 2023-07-25 12:48:33 +00:00
TheBloke
3f359fc778 Add support for Safetensors metadata 2023-07-25 11:30:39 +00:00
潘其威(William)
b4fdd8d264
Merge branch 'main' into peft_integration 2023-06-02 19:11:59 +08:00
PanQiWei
ec6603d0ab support older version python 2023-05-31 22:11:16 +08:00
qwopqwop200
c381958a5f
add warning 2023-05-30 23:53:33 +09:00
潘其威(William)
defc96ff04
Merge pull request #91 from TheBloke/TheBloke_support-HF-download 
Add support for HF Hub download, and `push_to_hub`
 2023-05-30 07:37:15 +08:00
潘其威(William)
15db2cdc44
Update _base.py 
fix problem that recursively adding file extension to model_base_name
 2023-05-30 07:26:42 +08:00
潘其威(William)
cfa7271617
Update _base.py 
fix variable not found error
 2023-05-30 07:22:10 +08:00
潘其威(William)
e5771fb206
Update _base.py 
fix key mismatch
 2023-05-30 06:44:45 +08:00
潘其威(William)
ea74e15199
Update _base.py 
add model_name_or_path and model_file_base_name to BaseQuantizeConfig for better model file management; add back save_dir to .from_quantized() for backward compatible
 2023-05-30 06:40:31 +08:00
PanQiWei
86f060c74b Merge branch 'main' into peft_integration 2023-05-28 16:23:38 +08:00
qwopqwop200
4d5b4fa5c6
add dtype 2023-05-27 07:49:28 +09:00
PanQiWei
f7e705848a move peft compatible model injection to the last step 2023-05-26 14:29:33 +08:00
PanQiWei
2b532f9453 add trainable mode 2023-05-26 13:11:30 +08:00
PanQiWei
cfd27e8caa refactor file structure of qlinears 2023-05-26 07:18:16 +08:00
PanQiWei
4d157a3b64 add hack of __getattr__ 2023-05-25 15:10:33 +08:00
TheBloke
b7bb50b4d5 Fix bug added after merge 2023-05-25 07:05:51 +01:00
Tom Jobbins
492255b400
Merge branch 'main' into TheBloke_support-HF-download 2023-05-25 07:02:13 +01:00
PanQiWei
096749fe9d generalize QuantLinear 2023-05-25 13:33:09 +08:00
PanQiWei
94ef4d5ada update basic usage example code 2023-05-24 17:56:46 +08:00
PanQiWei
c89bb6450c correct typo of function name 2023-05-24 17:43:38 +08:00
PanQiWei
10347fdd7b remove full_cpu_offload argument and unify model dispatch strategy 2023-05-24 17:41:04 +08:00
PanQiWei
379f24c2a5 remove add_align_logits_hook_to_model 2023-05-24 17:01:57 +08:00
PanQiWei
749dba1a7e disable add_align_logits_hook_to_model for now 2023-05-24 13:42:06 +08:00
PanQiWei
58c1b509f0 support add_align_logits_hook_to_model 2023-05-24 12:50:30 +08:00
PanQiWei
21ab7c435a make comments more readable 2023-05-24 11:38:29 +08:00
PanQiWei
c31b370228 make_sure_not_tensor_in_meta_device before load checkpoint 2023-05-24 11:32:45 +08:00
PanQiWei
63f1b4e073 remove comment 2023-05-24 11:23:07 +08:00
PanQiWei
057c39e3f2 fix meta device bug when use low_cpu_mem_usage 2023-05-24 11:19:59 +08:00
PanQiWei
e2e7809a1f always to enable QuantLinear bias to make compatible with model quantized from other frameworks 2023-05-24 10:56:31 +08:00
PanQiWei
191da8141e fix device mismatch 2023-05-23 23:22:52 +08:00
PanQiWei
e4e90e8b0a add warmup_triton method 2023-05-23 23:18:46 +08:00
PanQiWei
ed14d3a786 fix save quantized model failed when load pretrained model using CPU offload 2023-05-23 23:17:11 +08:00
PanQiWei
6476ee4235 add options: 'low_cpu_mem_usage' and 'full_cpu_offload' 2023-05-23 22:51:00 +08:00
TheBloke
bf633c298e Clean up some unused params 2023-05-20 10:32:27 +01:00
PanQiWei
86b3b52c63 fix ImportError when triton is not installed 2023-05-20 16:15:20 +08:00
潘其威(William)
13defe253a
Merge pull request #84 from TheBloke/TheBloke_forward-positional-args 
Forward position args to allow `model(tokens)` syntax
 2023-05-20 15:04:27 +08:00
潘其威(William)
1ef0af824a
Merge pull request #80 from PanQiWei/user_customized_device_map 
Support users customize `device_map`
 2023-05-20 15:00:05 +08:00
TheBloke
e5c8479100 Remove debugging print line 2023-05-19 17:50:48 +01:00
TheBloke
735f7df4cc Add push_to_hub for HF hub uploading 2023-05-19 17:10:57 +01:00
TheBloke
908b338436 Initial support for model loading from HF hub 2023-05-19 15:57:05 +01:00