simcop2387/text-generation-webui-mirror
1
0
Fork 0
mirror of https://github.com/oobabooga/text-generation-webui.git synced 2025-06-07 22:25:54 -04:00
Code Issues Projects Releases Packages Wiki Activity

Fix loading Llama-3_3-Nemotron-Super-49B-v1 and similar models (closes #7012)

Browse source
This commit is contained in:
oobabooga 2025-05-25 17:19:26 -07:00
parent 7f6579ab20
commit bae1aa34aa
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
modules/models_settings.py
View file

@ -335,7 +335,7 @@ def estimate_vram(gguf_file, gpu_layers, ctx_size, cache_type):
if key.endswith('.block_count'): if key.endswith('.block_count'):
n_layers = value n_layers = value
elif key.endswith('.attention.head_count_kv'): elif key.endswith('.attention.head_count_kv'):
n_kv_heads = value n_kv_heads = max(value) if isinstance(value, list) else value
elif key.endswith('.embedding_length'): elif key.endswith('.embedding_length'):
embedding_dim = value embedding_dim = value