18 commits

Author	SHA1	Message	Date
oobabooga	ed42154c78	Revert "llama.cpp: close the connection immediately on 'Stop'" ... This reverts commit 5fdebc554b.	2025-04-19 05:32:36 -07:00
oobabooga	5fdebc554b	llama.cpp: close the connection immediately on 'Stop'	2025-04-19 04:59:24 -07:00
oobabooga	6589ebeca8	Revert "llama.cpp: new optimization attempt" ... This reverts commit e2e73ed22f.	2025-04-18 21:16:21 -07:00
oobabooga	e2e73ed22f	llama.cpp: new optimization attempt	2025-04-18 21:05:08 -07:00
oobabooga	e2e90af6cd	llama.cpp: don't include --rope-freq-base in the launch command if null	2025-04-18 20:51:18 -07:00
oobabooga	9f07a1f5d7	llama.cpp: new attempt at optimizing the llama-server connection	2025-04-18 19:30:53 -07:00
oobabooga	f727b4a2cc	llama.cpp: close the connection properly when generation is cancelled	2025-04-18 19:01:39 -07:00
oobabooga	b3342b8dd8	llama.cpp: optimize the llama-server connection	2025-04-18 18:46:36 -07:00
oobabooga	2002590536	Revert "Attempt at making the llama-server streaming more efficient." ... This reverts commit 5ad080ff25.	2025-04-18 18:13:54 -07:00
oobabooga	71ae05e0a4	llama.cpp: Fix the sampler priority handling	2025-04-18 18:06:36 -07:00
oobabooga	5ad080ff25	Attempt at making the llama-server streaming more efficient.	2025-04-18 18:04:49 -07:00
oobabooga	4fabd729c9	Fix the API without streaming or without 'sampler_priority' (closes #6851)	2025-04-18 17:25:22 -07:00
oobabooga	5135523429	Fix the new llama.cpp loader failing to unload models	2025-04-18 17:10:26 -07:00
oobabooga	caa6afc88b	Only show 'GENERATE_PARAMS=...' in the logits endpoint if use_logits is True	2025-04-18 09:57:57 -07:00
oobabooga	d00d713ace	Rename get_max_context_length to get_vocabulary_size in the new llama.cpp loader	2025-04-18 08:14:15 -07:00
oobabooga	c1cc65e82e	Lint	2025-04-18 08:06:51 -07:00
oobabooga	a0abf93425	Connect --rope-freq-base to the new llama.cpp loader	2025-04-18 06:53:51 -07:00
oobabooga	ae54d8faaa	New llama.cpp loader (#6846)	2025-04-18 09:59:37 -03:00