simcop2387/text-generation-webui-mirror
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2025-06-07 14:17:09 -04:00
Code Issues Projects Releases Packages Wiki Activity

ExLlamaV3_HF: Change max_chunk_size to 256

Browse source
This commit is contained in:
oobabooga 2025-05-04 20:37:15 -07:00
parent df7bb0db1f
commit f3da45f65d
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
modules/exllamav3_hf.py
View file

@ -119,7 +119,7 @@ class Exllamav3HF(PreTrainedModel, GenerationMixin):
reset = True
# Maximum number of tokens to process in a single forward pass
max_chunk_size = 2048
max_chunk_size = 256
# Make the forward call
if labels is None: