5 commits

Author	SHA1	Message	Date
PanQiWei	26dc6852fe	support inherit one of the three fused attention class and customize attn_bias building logic	2023-08-07 18:59:04 +08:00
PanQiWei	e5f874e5af	add fused attention injection logic to llama	2023-08-07 13:45:37 +08:00
PanQiWei	1f9717af7f	change classes default values	2023-08-06 18:24:23 +08:00
PanQiWei	f67b512cee	add 'training' argument	2023-08-06 14:54:34 +08:00
PanQiWei	8b19122775	add fused attentions	2023-08-04 19:11:43 +08:00