student686
|
c1a3013c45
|
import exllama QuantLinear instead of exllamav2's
|
2023-09-27 11:05:13 +08:00 |
|
Marc Sun
|
c912bf361a
|
exllamav2 integration
|
2023-09-25 16:51:18 +00:00 |
|
Marc Sun
|
04b321da89
|
fix type
|
2023-08-31 14:07:16 -04:00 |
|
qwopqwop200
|
45a1ee4d84
|
install check qigen
|
2023-08-31 14:37:39 +09:00 |
|
qwopqwop200
|
6a9d80eddc
|
Merge remote-tracking branch 'qwopqwop200/main' into main
|
2023-08-25 18:06:03 +09:00 |
|
qwopqwop200
|
dafdd6189a
|
duplicate code remove
|
2023-08-25 14:59:13 +09:00 |
|
Felix Marty
|
04730ac66c
|
expose api to set exllama max length
|
2023-08-24 11:22:15 +00:00 |
|
qwopqwop200
|
b8a42911a6
|
qigen refactoring
|
2023-08-17 15:22:16 +09:00 |
|
qwopqwop200
|
051f3facc7
|
change arguments name
|
2023-08-11 16:10:32 +09:00 |
|
qwopqwop200
|
c591d6a1e1
|
change name make_quant_cpu to make_quant_qigen
|
2023-08-11 15:12:33 +09:00 |
|
qwopqwop200
|
2c1afc2ad9
|
chang name make_quant_cpu to make_quant_qigen
|
2023-08-11 15:04:58 +09:00 |
|
qwopqwop200
|
870be83bea
|
Merge branch 'PanQiWei:main' into main
|
2023-08-10 22:48:30 +09:00 |
|
qwopqwop200
|
7ba78af3ae
|
support cpu
|
2023-08-10 22:48:04 +09:00 |
|
Felix Marty
|
4af7ea619d
|
patch for transformers compatiblity
|
2023-08-09 14:23:59 +00:00 |
|
Felix Marty
|
38447262c0
|
fix fused attn
|
2023-07-31 13:46:32 +00:00 |
|
Felix Marty
|
760667dccc
|
cleaning
|
2023-07-31 11:58:10 +00:00 |
|
Felix Marty
|
179776bd1d
|
exllama kernel
|
2023-07-31 11:50:45 +00:00 |
|
PanQiWei
|
86f060c74b
|
Merge branch 'main' into peft_integration
|
2023-05-28 16:23:38 +08:00 |
|
qwopqwop200
|
277809381b
|
fix bug
|
2023-05-27 08:53:47 +09:00 |
|
qwopqwop200
|
c14b4c1567
|
change find layer algorithm
|
2023-05-27 07:48:50 +09:00 |
|
PanQiWei
|
2b532f9453
|
add trainable mode
|
2023-05-26 13:11:30 +08:00 |
|
PanQiWei
|
c89bb6450c
|
correct typo of function name
|
2023-05-24 17:43:38 +08:00 |
|
PanQiWei
|
10347fdd7b
|
remove full_cpu_offload argument and unify model dispatch strategy
|
2023-05-24 17:41:04 +08:00 |
|
PanQiWei
|
379f24c2a5
|
remove add_align_logits_hook_to_model
|
2023-05-24 17:01:57 +08:00 |
|
PanQiWei
|
58c1b509f0
|
support add_align_logits_hook_to_model
|
2023-05-24 12:50:30 +08:00 |
|
PanQiWei
|
057c39e3f2
|
fix meta device bug when use low_cpu_mem_usage
|
2023-05-24 11:19:59 +08:00 |
|
PanQiWei
|
e2e7809a1f
|
always to enable QuantLinear bias to make compatible with model quantized from other frameworks
|
2023-05-24 10:56:31 +08:00 |
|
PanQiWei
|
ed14d3a786
|
fix save quantized model failed when load pretrained model using CPU offload
|
2023-05-23 23:17:11 +08:00 |
|
PanQiWei
|
6476ee4235
|
add options: 'low_cpu_mem_usage' and 'full_cpu_offload'
|
2023-05-23 22:51:00 +08:00 |
|
PanQiWei
|
1b2159bd4c
|
add more help functions
|
2023-05-23 19:30:28 +08:00 |
|
PanQiWei
|
2273f9ef39
|
refactor file structure for triton kernels
|
2023-05-14 11:49:10 +08:00 |
|
PanQiWei
|
d718d63e9c
|
add import_utils.py for commonly used module importation
|
2023-05-12 19:58:48 +08:00 |
|
PanQiWei
|
4bb10fda49
|
groupsize -> group_size
|
2023-05-12 13:37:52 +08:00 |
|
qwopqwop200
|
3ff6ab18cb
|
Merge branch 'main' into faster-llama
|
2023-05-06 00:20:29 +09:00 |
|
TheBloke
|
f61ce12271
|
Change 'groupsize' to 'group_size' everywhere. Turns out this is easier than 'groupsize' due to dependencies in other files.
|
2023-05-05 13:36:00 +01:00 |
|
TheBloke
|
f64c71e779
|
Change referenes to 'group_size' to 'groupsize' to match rest of this file
|
2023-05-05 13:21:13 +01:00 |
|
PanQiWei
|
1c6bb69fae
|
fix attribute name error
|
2023-05-04 22:10:33 +08:00 |
|
潘其威(William)
|
771b650a7c
|
Merge pull request #38 from PanQiWei/faster-cuda-no-actorder
Faster cuda no actorder
|
2023-05-04 21:47:19 +08:00 |
|
qwopqwop200
|
b19c59541b
|
fix bug
|
2023-05-04 13:17:10 +09:00 |
|
qwopqwop200
|
b14d42e68a
|
bug fix
|
2023-05-04 13:03:38 +09:00 |
|
qwopqwop200
|
cc992c21bd
|
Merge branch 'faster-cuda-no-actorder' into faster-llama
|
2023-05-04 09:09:09 +09:00 |
|
qwopqwop200
|
c8504f0660
|
support faster and model load strict
|
2023-05-04 09:06:52 +09:00 |
|
qwopqwop200
|
41f2379850
|
bug fix
|
2023-05-02 20:38:17 +09:00 |
|
qwopqwop200
|
d2f48e5311
|
bug fix
|
2023-05-02 20:36:53 +09:00 |
|
qwopqwop200
|
709bd7594f
|
Merge pull request #44 from PanQiWei/fix-bug-cuda
Fix bug cuda
|
2023-05-02 19:50:59 +09:00 |
|
TheBloke
|
593a0b28bb
|
Fix typo: 'hole' -> 'whole'
|
2023-05-01 10:25:18 +01:00 |
|
qwopqwop200
|
5a69e22a93
|
add qlinear_old
|
2023-05-01 13:04:47 +09:00 |
|
qwopqwop200
|
435eebee4b
|
support conv1d,conv2d
|
2023-04-28 09:13:00 +09:00 |
|
PanQiWei
|
ac3f7054e0
|
big fix
|
2023-04-27 19:33:25 +08:00 |
|
PanQiWei
|
498de923f2
|
support multi gpus quantization
|
2023-04-27 18:48:43 +08:00 |
|