Update docs and design a bit, I think most of the things needed are ready and now its just the boilerplate for the server
Some checks failed
ci/woodpecker/push/author-tests Pipeline failed
Some checks failed
ci/woodpecker/push/author-tests Pipeline failed
This commit is contained in:
parent
68c7f96bf7
commit
303b3cd37d
2 changed files with 56 additions and 11 deletions
1
.vstags
1
.vstags
|
@ -9895,6 +9895,7 @@ OpenAIAsync::Client .build/t7Cb8f47yj/blib/lib/OpenAIAsync/Client.pm 1;" p
|
|||
OpenAIAsync::Client .build/t7Cb8f47yj/lib/OpenAIAsync/Client.pm 1;" p
|
||||
OpenAIAsync::Client OpenAIAsync-0.01/lib/OpenAIAsync/Client.pm 1;" p
|
||||
OpenAIAsync::Client lib/OpenAIAsync/Client.pm 1;" p
|
||||
OpenAIAsync::Server lib/OpenAIAsync/Server.pm 1;" p
|
||||
OpenAIAsync::Types .build/0T4wbFlmwf/blib/lib/OpenAIAsync/Types.pm 1;" p
|
||||
OpenAIAsync::Types .build/0T4wbFlmwf/lib/OpenAIAsync/Types.pm 1;" p
|
||||
OpenAIAsync::Types .build/2oNz8Mp68u/blib/lib/OpenAIAsync/Types.pm 1;" p
|
||||
|
|
|
@ -72,8 +72,13 @@ it will properly suspend the execution of your program and do something else con
|
|||
All the methods that you provide must be async, this is so that multiple requests can be easily handled internally with ease.
|
||||
|
||||
You subclass the ::Server class and implement your own methods for handling the limited number of requests, and provide some
|
||||
configuration information for the servers to start in the init() method
|
||||
configuration information for the servers to start in the configure() method. The reason to put this into a method is so that
|
||||
The server object can handle reloading itself without being recreated. This way any sockets that are already listening don't have
|
||||
to be closed and reopened to be reconfigured (assuming that the new configuration keeps them open).
|
||||
|
||||
Streaming from ::Server is still being designed, I'll publish this system WITHOUT streaming support the first time around
|
||||
since I need to write at least a new http client module that supports it in order to test things properly, and the make OpenAIAsync::Client
|
||||
work with streaming events anyway.
|
||||
|
||||
=head1 Methods
|
||||
|
||||
|
@ -91,7 +96,44 @@ TODO FILL IN
|
|||
|
||||
=back
|
||||
|
||||
=head2 auth_check($key, $http_req)
|
||||
=head2 init($loop)
|
||||
|
||||
The intialization phase, this happens before the configure() method is called, so that you can setup any local data stores and validate
|
||||
the environment. This will only be called once, but configure() may be called multiple times, such as from a signal handler (SIGHUP or SIGUSR1?)
|
||||
to reload any configuration for the HTTP endpoints.
|
||||
|
||||
Not completely sure if I should have these separated but I kept not liking doing it all in one method for some reason. Maybe fold this into BUILD {} instead?
|
||||
|
||||
=back
|
||||
|
||||
=head2 configure($loop)
|
||||
|
||||
The configuration phase, returns a list of the arguments to be given to Net::Async::HTTP::Server
|
||||
|
||||
TODO bring in some docuemtation, this is derived from C<IO::Async::Loop> 's listen method, and we also need to refer to the connect method for more details
|
||||
basically, just C<< listen => [addr => '::1', port => 8080, socktype => 'stream'] >> as a basic example.
|
||||
|
||||
We handle multiple of these so we can have this proxy listen on multiple ports, and then handle any other behavior for it
|
||||
|
||||
The context key will be passed to all of the methods doing the handling here, so you can use it for static data based on which
|
||||
connection actually was used. This should help handle multiple networks/vlans or auth patterns a bit easier from a single daemon.
|
||||
|
||||
I think this should also let you pass in a listening handle from another module like IO::Socket::SSL or something to do native SSL
|
||||
But I'm not going to be testing that for some time myself, and would recommend using something else like nginx, haproxy, etc. as a
|
||||
terminator for TLS connections
|
||||
|
||||
[
|
||||
{
|
||||
listen => {addrs => ['::1', ...], ...},
|
||||
context => {...}
|
||||
on_something => code
|
||||
},
|
||||
...
|
||||
]
|
||||
|
||||
=back
|
||||
|
||||
=head2 auth_check($key, $ctx, $http_req)
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
|
@ -101,43 +143,45 @@ if $key is undef then you can use the $http_req object to get a look at the full
|
|||
|
||||
By default all OpenAI compatible APIs are expecting ONLY an API-Key type Authorization header so doing anytyhing else isn't strictly compatible but there's no reason that this server shouldn't be able to do more if you're customizing things
|
||||
|
||||
=head2 completion (deprecated)
|
||||
=head2 async completion($ctx, $completion)
|
||||
|
||||
=head3 DEPRECATED
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Handle a completion request, takes in a request object, must return a response object
|
||||
Handle a completion request, takes in a request object, must return a response object.
|
||||
|
||||
=head2 chat
|
||||
=head2 async chat($ctx, $chat_completion)
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Handle a chat completion request
|
||||
|
||||
=head2 embedding
|
||||
=head2 async embedding($ctx, $embedding)
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Handle an embedding request
|
||||
|
||||
=head2 image_generate
|
||||
=head2 async image_generate
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Unimplemented, but once present will be used to generate images with Dall-E (or for self hosted, stable diffusion).
|
||||
|
||||
=head2 text_to_speech
|
||||
=head2 async text_to_speech
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Unimplemented, but can be used to turn text to speech using whatever algorithms/models are supported.
|
||||
|
||||
=head2 speech_to_text
|
||||
=head2 async speech_to_text
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
Unimplemented. The opposite of the above.
|
||||
|
||||
=head2 vision
|
||||
=head2 async vision
|
||||
|
||||
This method requres async keyword.
|
||||
|
||||
|
@ -161,7 +205,7 @@ Ryan Voots, ... etc.
|
|||
|
||||
=cut
|
||||
|
||||
class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params) {
|
||||
class OpenAIAsync::Server :repr(HASH) :isa(IO::Async::Notifier) :strict(params) {
|
||||
use JSON::MaybeXS qw//;
|
||||
use Net::Async::HTTP;
|
||||
use Feature::Compat::Try;
|
||||
|
|
Loading…
Add table
Reference in a new issue