Naive regex replacement, needs examining for correctness

beginning rename of things
New API design works!
2024-12-18 11:11:10 -05:00 · 2024-12-18 11:07:10 -05:00 · 2024-03-06 15:47:17 -05:00 · 2024-03-06 15:29:30 -05:00 · 2024-03-06 15:06:59 -05:00 · 2024-03-06 15:01:37 -05:00
45 changed files with 4122 additions and 1099 deletions
--- a/.vstags
+++ b/.vstags
--- a/Client.pod
+++ b/Client.pod
@ -1,346 +0,0 @@
-package OpenAIAsync::Client;
-
-use v5.36.0;
-use Object::Pad;
-use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
-use Future::AsyncAwait;
-use IO::Async;
-
-use OpenAIAsync::Types::Results;
-use OpenAIAsync::Types::Requests;
-
-our $VERSION="v0.1.0";
-
-# ABSTRACT: Async client for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
-
-=pod
-
-=head1 NAME
-
-OpenAIAsync::Client - IO::Async based client for OpenAI compatible APIs
-
-=head1 SYNOPSIS
-
-  use IO::Async::Loop;
-  use OpenAIAsync::Client;
-
-  my $loop = IO::Async::Loop->new();
-
-  my $client = OpenAIAsync::Client->new();
-
-  $loop->add($client);
-
-  my $output = await $client->chat({
-      model => "gpt-3.5-turbo",
-      messages => [
-        {
-          role => "system",
-          content => "You are a helpful assistant that tells fanciful stories"
-        },
-        {
-          role => "user",
-          content => "Tell me a story of two princesses, Judy and Emmy.  Judy is 8 and Emmy is 2."
-        }
-      ],
-
-
-
-    max_tokens => 1024, 
-  })->get();
-
-  # $output is now an OpenAIAsync::Type::Response::ChatCompletion
-
-=head1 THEORY OF OPERATION
-
-This module implements the L<IO::Async::Notifier> interface, this means that you create a new client and then call C<< $loop->add($client) >>
-this casues all L<Future>s that are created to be part of the L<IO::Async::Loop> of your program.  This way when you call C<await> on any method
-it will properly suspend the execution of your program and do something else concurrently (probably waiting on requests).
-
-=head1 Methods
-
-=head2 new()
-
-Create a new OpenAIAsync::Client.  You'll need to register the client with C<< $loop->add($client) >> after creation.
-
-=head3 PARAMETERS
-
-=over 4
-
-=item * api_base (optional)
-
-Base url of the service to connect to.  Defaults to C<https://api.openai.com/v1>.  This should be a value pointing to something that
-implements the v1 OpenAI API, which for OobaBooga's text-generation-webui might be something like C<http://localhost:5000/v1>.
-
-It will also be pulled from the environment variable C<OPENAI_API_BASE> in the same fashion that the OpenAI libraries in other languages will do.
-
-=item * api_key (required)
-
-Api key that will be passed to the service you call.  This gets passed as a header C<Authorization: Api-Key ....> to the service in all of the REST
-calls.  This should be kept secret as it can be used to make all kinds of calls to paid services.
-
-It will also be pulled from the environment variable C<OPENAI_API_KEY> in the same fashion that the OpenAI libraries in other languages will do.
-
-=item * api_org_name (optional)
-
-A name for the organization that's making the call.  This can be used by OpenAI to help identify which part of your company is
-making any specific request, and I believe to help itemize billing and other tasks. 
-
-=item * http_user_agent (optional)
-
-Set the useragent that's used to contact the API service.  Defaults to 
-
-C<< __PACKAGE__." Perl/$VERSION (Net::Async::HTTP/".$Net::Async::HTTP::VERSION." IO::Async/".$IO::Async::VERSION." Perl/$])" >>
-
-The default is to make it easier to debug if we ever see weird issues with the requests being generated but it does reveal some information
-about the code environment.
-
-=item * http_max_in_flight (optional)
-
-How many requests should we allow to happen at once.  Increasing this will increase the allowed parallel requests, but that can also
-allow you to make too many requests and cost more in API calls.
-
-Defaults to 2
-
-=item * http_max_connections_per_host (optional)
-
-TODO, I'm thinking this one will get dropped.  Effectively since we're only ever connecting to one server this ends up functioning the same as the above parameter.
-
-Defaults to 2
-
-=item * http_max_redirects (optional)
-
-How many redirects to allow.  The official OpenAI API never sends redirects (for now) but for self hosted or other custom setups this might happen and should be handled correctly
-
-Defaults to 3
-
-=item * http_timeout (optional)
-
-How long to wait on any given request to start.  
-
-Defaults to 120 seconds.
-
-=item * http_stall_timeout (optional)
-
-How long to wait on any given request to decide if it's been stalled.  If a request starts responding and then stops part way through, this is how we'll treat it as stalled and time it out
-
-Defaults to 600s (10 minutes).  This is unlikely to happen except for a malfunctioning inference service since once generation starts to return it'll almost certainly finish.
-
-=item * http_other (optional)
-
-A hash ref that gets passed as additional parameters to L<Net::Async::HTTP>'s constructor.  All values will be overriden by the ones above, so if a parameter is supported use those first.
-
-=back
-
-=head2 completion (deprecated)
-
-Create a request for completion, this takes a prompt and returns a response.  See L<OpenAIAsync::Types::Request::Completion> for exact details.
-
-This particular API has been deprecated by OpenAI in favor of doing everything through the chat completion api below.  However it is still supported
-by OpenAI and compatible servers as it's a very simple interface to use
-
-=head2 chat
-
-Create a request for the chat completion api.  This takes a series of messages and returns a new chat response.  See L<OpenAIAsync::Types::Request::ChatCompletion> for exact details.
-
-This API takes a series of messages from different agent sources and then responds as the assistant agent.  A typical interaction is to start with a C<"system"> agent message 
-to set the context for the assistant, followed by the C<"user"> agent type for the user's request.  You'll then get the response from the assistant agent to give to the user.
-
-To continue the chat, you'd then take the new message and insert it into the list of messages as part of the chat and make a new request with the user's response.  I'll be creating
-a new module that uses this API and helps manage the chat in an easier manner with a few helper functions.
-
-=head2 embedding
-
-Create a request for calculating the embedding of an input.  This takes a bit of text and returns a gigantic list of numbers, see L<OpenAIAsync::Types::Request::Embedding> for exact details.
-
-These values are a bit difficult to explain how they work, but essentially you get a mathematical object, a vector, that describes the contents of the input as 
-a point in an N-dimensional space (typically 768 or 1536 dimensions).  The dimensions themselves really don't have any inherit mathematical meaning but are instead relative to one-another
-from the training data of the embedding model.
-
-You'll want to take the vector and store it in a database that supports vector operations, like PostgreSQL with the L<pgvector|https://github.com/pgvector/pgvector> extension.
-
-=head2 image_generate
-
-Unimplemented, but once present will be used to generate images with Dall-E (or for self hosted, stable diffusion).
-
-=head2 text_to_speech
-
-Unimplemented, but can be used to turn text to speech using whatever algorithms/models are supported.
-
-=head2 speech_to_text
-
-Unimplemented. The opposite of the above.
-
-=head2 vision
-
-Unimplemented, I've not investigated this one much yet but I believe it's to get a description of an image and it's contents.
-
-=head1 See Also
-
-L<IO::Async>, L<Future::AsyncAwait>, L<Net::Async::HTTP>
-
-=head1 License
-
-Artistic 2.0
-
-=head1 Author
-
-Ryan Voots, ... etc.
-
-=cut
-
-class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params) {
-  use JSON::MaybeXS qw//;
-  use Net::Async::HTTP;
-  use Feature::Compat::Try;
-  use URI;
-
-  field $_json = JSON::MaybeXS->new(utf8 => 1, convert_blessed => 1);
-  field $http;
-
-  # TODO document these directly, other options gets mixed in BEFORE all of these
-  field $_http_max_in_flight :param(http_max_in_flight) = 2;
-  field $_http_max_redirects :param(http_max_redirects) = 3;
-  field $_http_max_connections_per_host :param(http_max_connections_per_host) = 2;
-  field $_http_timeout :param(http_timeout) = 120; # My personal server is kinda slow, use a generous default
-  field $_http_stall_timeout :param(http_stall_timeout) = 600; # generous for my slow personal server
-  field $_http_other :param(http_other_options) = {};
-  field $_http_user_agent :param(http_user_agent) = __PACKAGE__." Perl/$VERSION (Net::Async::HTTP/".$Net::Async::HTTP::VERSION." IO::Async/".$IO::Async::VERSION." Perl/$])";
-
-  field $api_base :param(api_base) = $ENV{OPENAI_API_BASE} // "https://api.openai.com/v1";
-  field $api_key :param(api_key) = $ENV{OPENAI_API_KEY};
-
-  field $api_org_name :param(api_org_name) = undef;
-
-  field $io_async_notifier_params :param = undef;
-
-  method configure(%params) {
-    # We require them to go this way, so that there is no conflicts
-    # TODO document this
-    my %io_async_params = ($params{io_async_notifier_params} // {})->%*;
-    IO::Async::Notifier::configure($self, %io_async_params);
-  }
-
-  method __make_http() {
-    die "Missing API Key for OpenAI" unless $api_key;
-
-    return Net::Async::HTTP->new(
-      $_http_other->%*,
-      user_agent => "SNN OpenAI Client 1.0",
-      +headers => {
-        "Authorization" => "Bearer $api_key",
-        "Content-Type" => "application/json",
-        $api_org_name ? (
-          'OpenAI-Organization' => $api_org_name,
-        ) : ()
-      },
-      max_redirects => $_http_max_redirects,
-      max_connections_per_host => $_http_max_connections_per_host,
-      max_in_flight => $_http_max_in_flight,
-      timeout => $_http_timeout,
-      stall_timeout => $_http_stall_timeout,
-    )
-  }
-
-  ADJUST {
-    $http = $self->__make_http;
-
-    $api_base =~ s|/$||; # trim an accidental final / since we will be putting it on the endpoints
-  }
-
-  async method _make_request($endpoint, $data) {
-    my $json = $_json->encode($data);
-
-    my $url = URI->new($api_base . $endpoint );
-
-    my $result = await $http->do_request(
-      uri => $url,
-      method => "POST",
-      content => $json,
-      content_type => 'application/json',
-    );
-
-    if ($result->is_success) {
-      my $json = $result->decoded_content;
-      my $out_data = $_json->decode($json);
-
-      return $out_data;
-    } else {
-      die "Failure in talking to OpenAI service: ".$result->status_line.": ".$result->decoded_content;
-    }
-  }
-
-   method _add_to_loop($loop) {
-    $loop->add($http);
-  }
-
-  method _remove_from_loop($loop) {
-    $loop->remove($http);
-    $http = $self->__make_http; # overkill? want to make sure we have a clean one
-  }
-
-  # This is the legacy completion api
-  async method completion($input) {
-
-    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::Completion->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::Completion') {
-      # dummy, nothing to do
-    } else {
-      die "Unsupported input type [".ref($input)."]";
-    }
-
-    my $data = await $self->_make_request($input->_endpoint(), $input);
-
-    my $type_result = OpenAIAsync::Types::Results::Completion->new($data->%*);
-
-    return $type_result;
-  }
-
-  async method chat($input) {
-    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::ChatCompletion->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::ChatCompletion') {
-      # dummy, nothing to do
-    } else {
-      die "Unsupported input type [".ref($input)."]";
-    }
-
-    my $data = await $self->_make_request($input->_endpoint(), $input);
-
-    my $type_result = OpenAIAsync::Types::Results::ChatCompletion->new($data->%*);
-
-    return $type_result;
-  }
-
-  async method embedding($input) {
-    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::Embedding->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::Embedding') {
-      # dummy, nothing to do
-    } else {
-      die "Unsupported input type [".ref($input)."]";
-    }
-
-    my $data = await $self->_make_request($input->_endpoint(), $input);
-
-    my $type_result = OpenAIAsync::Types::Results::Embedding->new($data->%*);
-
-    return $type_result;
-  }
-
-  async method image_generate($input) {
-    ...
-  }
-
-  async method text_to_speech($text) {
-    ...
-  }
-
-  async method speech_to_text($sound_data) {
-    ...
-  }
-
-  async method vision($image, $prompt) {
-    ...
-  }
-}
--- a/dist.ini
+++ b/dist.ini
@ -17,6 +17,7 @@ managed_versions = 1

 [Prereqs / RuntimeRequires]
 ; perl = 5.008001 # TODO figure this out
+Object::Pad = 0.807

 [Prereqs / TestRequires]
 Test::More = 0.88
@ -26,5 +27,7 @@ critic_config = perlcritic.rc

 [PruneFiles]
 filename = xtest.sh ; Local git development test script, sets environment for local dzil xtest
-; match     = ^test_data/
-; match     = ^test.cvs$
+match     = ^ci-docker/ ; Local git testing stuff, doesn't need to be part of the dist
+filename = Client.pod
+filename = README.pod
+filename = README.md
--- a/lib/Net/Async/xLM.pm
+++ b/lib/Net/Async/xLM.pm
@ -1,9 +1,9 @@
-package OpenAIAsync;
+package Net::Async::xLM::API;

 use strict;
 use warnings;

-our $VERSION="0.01";
+our $VERSION = '0.02';

 # ABSTRACT: OpenAI style Client and Server for processing various AI tasks

--- a/lib/Net/Async/xLM.pod
+++ b/lib/Net/Async/xLM.pod
@ -0,0 +1,57 @@
+=pod
+
+=head1 PURPOSE
+
+There's two big submodules that you'll want to look at:
+
+L<Net::Async::xLM::API::Client> and L<OpenAIAsync::Client::OobaBooga>
+
+There will eventually be a compatible server that uses Net::Async::HTTP::Server that can be used to build a proxy that lets you manipulate or reroute requests, etc.
+
+=head1 WARNING
+
+This whole module is in a very early state, and while it has been working for my tests and simple projects it's far from complete and robustly tested yet.
+
+=head1 PLANS
+
+=over 4
+
+=item * Create a subclass of ::Client that properly handles the additional fields for a few local ai inference servers.
+
+Particularly OobaBooga's text-generation-webui since it's what I use.  It has a number of additional features like negative prompts and other sampler parameters that can help get better quality responses from Llama models
+
+=item * Streaming response support
+
+This is going to take a few things to work properly.  Particularly it's going to need a new module that is similar to L<Net::Async::Websocket> because OpenAI implemented this via C<server-sent-events>, which does not have any support in Net::Async::HTTP (or Mojo::UserAgent) to be handled properly.
+
+Once I get to working on that new module I'll start implementing the streaming support in here, which will likely look similar to IO::Async::Stream or Net::Async::Websocket, where you'll give a coderef for each new event to be processed.
+
+=item * Image Gen, Vision, STT, and TTS support
+
+I've left these off since they're more expensive on OpenAI's service and I haven't set them up on my own local system yet.  But this will come somewhat soon.  I'm not going to require any specific image library to be used, but I'll try to provide some simple functions to create the requests in an agnostic manner.
+
+=back
+
+=head1 NON-PLANS
+
+=over 4
+
+=item * Direct simple chatbot interface
+
+The ChatCompletion API and it's features are actually a little verbose and complicated to use properly, but I'm intending this series of modules to be direct and complete API clients only.  To that end though I'll be making an Net::Async::xLM::API::ChatBot module that provides a better interface for making actual chatbots, particularly chat session handling and stable serialization of chats so that they can be persisted somewhere and then reloaded to continue in the future.
+
+=back
+
+=head1 KNOWN ISSUES
+
+I'm pretty sure there's a bug or two in the ChatCompletion code in the client, and I'm working on making some tests to uncover and fix those.  So use those with caution
+
+=head1 LICENSE
+
+Artistic 2.0 - L<Software::License::Artistic_2_0>
+
+=head1 AUTHOR
+
+Ryan Voots TODO add more here
+
+=cut
--- a/lib/Net/Async/xLM/API/Client.pm
+++ b/lib/Net/Async/xLM/API/Client.pm
@ -1,4 +1,4 @@
-package OpenAIAsync::Client;
+package Net::Async::xLM::API::Client;

 use v5.36.0;
 use Object::Pad;
@ -6,10 +6,10 @@ use IO::Async::SSL; # We're not directly using it but I want to enforce that we
 use Future::AsyncAwait;
 use IO::Async;

-use OpenAIAsync::Types::Results;
-use OpenAIAsync::Types::Requests;
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;

-our $VERSION="v0.1.0";
+our $VERSION = '0.02';

 # ABSTRACT: Async client for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)

@ -17,16 +17,16 @@ our $VERSION="v0.1.0";

 =head1 NAME

-OpenAIAsync::Client - IO::Async based client for OpenAI compatible APIs
+Net::Async::xLM::API::Client - IO::Async based client for OpenAI compatible APIs

 =head1 SYNOPSIS

  use IO::Async::Loop;
-  use OpenAIAsync::Client;
+  use Net::Async::xLM::API::Client;

  my $loop = IO::Async::Loop->new();

-  my $client = OpenAIAsync::Client->new();
+  my $client = Net::Async::xLM::API::Client->new();

  $loop->add($client);

@ -48,7 +48,7 @@ OpenAIAsync::Client - IO::Async based client for OpenAI compatible APIs
    max_tokens => 1024, 
  })->get();

-  # $output is now an OpenAIAsync::Type::Response::ChatCompletion
+  # $output is now an Net::Async::xLM::API::Type::Results::ChatCompletion

 =head1 THEORY OF OPERATION

@ -60,7 +60,7 @@ it will properly suspend the execution of your program and do something else con

 =head2 new()

-Create a new OpenAIAsync::Client.  You'll need to register the client with C<< $loop->add($client) >> after creation.
+Create a new Net::Async::xLM::API::Client.  You'll need to register the client with C<< $loop->add($client) >> after creation.

 =head3 PARAMETERS

@ -133,14 +133,14 @@ A hash ref that gets passed as additional parameters to L<Net::Async::HTTP>'s co

 =head2 completion (deprecated)

-Create a request for completion, this takes a prompt and returns a response.  See L<OpenAIAsync::Types::Request::Completion> for exact details.
+Create a request for completion, this takes a prompt and returns a response.  See L<Net::Async::xLM::API::Types::Request::Completion> for exact details.

 This particular API has been deprecated by OpenAI in favor of doing everything through the chat completion api below.  However it is still supported
 by OpenAI and compatible servers as it's a very simple interface to use

 =head2 chat

-Create a request for the chat completion api.  This takes a series of messages and returns a new chat response.  See L<OpenAIAsync::Types::Request::ChatCompletion> for exact details.
+Create a request for the chat completion api.  This takes a series of messages and returns a new chat response.  See L<Net::Async::xLM::API::Types::Request::ChatCompletion> for exact details.

 This API takes a series of messages from different agent sources and then responds as the assistant agent.  A typical interaction is to start with a C<"system"> agent message 
 to set the context for the assistant, followed by the C<"user"> agent type for the user's request.  You'll then get the response from the assistant agent to give to the user.
@ -150,7 +150,7 @@ a new module that uses this API and helps manage the chat in an easier manner wi

 =head2 embedding

-Create a request for calculating the embedding of an input.  This takes a bit of text and returns a gigantic list of numbers, see L<OpenAIAsync::Types::Request::Embedding> for exact details.
+Create a request for calculating the embedding of an input.  This takes a bit of text and returns a gigantic list of numbers, see L<Net::Async::xLM::API::Types::Request::Embedding> for exact details.

 These values are a bit difficult to explain how they work, but essentially you get a mathematical object, a vector, that describes the contents of the input as 
 a point in an N-dimensional space (typically 768 or 1536 dimensions).  The dimensions themselves really don't have any inherit mathematical meaning but are instead relative to one-another
@ -192,7 +192,8 @@ Ryan Voots, ... etc.

 =cut

-class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params) {
+class Net::Async::xLM::API::Client :repr(HASH) :strict(params) {
+  inherit IO::Async::Notifier;
  use JSON::MaybeXS qw//;
  use Net::Async::HTTP;
  use Feature::Compat::Try;
@ -252,7 +253,7 @@ class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params)
  }

  async method _make_request($endpoint, $data) {
-    my $json = $_json->encode($data);
+    my $json = $data->_encode();

    my $url = URI->new($api_base . $endpoint );

@ -286,8 +287,8 @@ class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params)
  async method completion($input) {

    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::Completion->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::Completion') {
+      $input = Net::Async::xLM::API::Types::Requests::Completion->new($input->%*);
+    } elsif (ref($input) eq 'Net::Async::xLM::API::Types::Requests::Completion') {
      # dummy, nothing to do
    } else {
      die "Unsupported input type [".ref($input)."]";
@ -295,15 +296,15 @@ class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params)

    my $data = await $self->_make_request($input->_endpoint(), $input);

-    my $type_result = OpenAIAsync::Types::Results::Completion->new($data->%*);
+    my $type_result = Net::Async::xLM::API::Types::Results::Completion->new($data->%*);

    return $type_result;
  }

  async method chat($input) {
    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::ChatCompletion->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::ChatCompletion') {
+      $input = Net::Async::xLM::API::Types::Requests::ChatCompletion->new($input->%*);
+    } elsif (ref($input) eq 'Net::Async::xLM::API::Types::Requests::ChatCompletion') {
      # dummy, nothing to do
    } else {
      die "Unsupported input type [".ref($input)."]";
@ -311,15 +312,15 @@ class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params)

    my $data = await $self->_make_request($input->_endpoint(), $input);

-    my $type_result = OpenAIAsync::Types::Results::ChatCompletion->new($data->%*);
+    my $type_result = Net::Async::xLM::API::Types::Results::ChatCompletion->new($data->%*);

    return $type_result;
  }

  async method embedding($input) {
    if (ref($input) eq 'HASH') {
-      $input = OpenAIAsync::Types::Requests::Embedding->new($input->%*);
-    } elsif (ref($input) eq 'OpenAIAsync::Types::Requests::Embedding') {
+      $input = Net::Async::xLM::API::Types::Requests::Embedding->new($input->%*);
+    } elsif (ref($input) eq 'Net::Async::xLM::API::Types::Requests::Embedding') {
      # dummy, nothing to do
    } else {
      die "Unsupported input type [".ref($input)."]";
@ -327,7 +328,7 @@ class OpenAIAsync::Client :repr(HASH) :isa(IO::Async::Notifier) :strict(params)

    my $data = await $self->_make_request($input->_endpoint(), $input);

-    my $type_result = OpenAIAsync::Types::Results::Embedding->new($data->%*);
+    my $type_result = Net::Async::xLM::API::Types::Results::Embedding->new($data->%*);

    return $type_result;
  }
--- a/lib/Net/Async/xLM/API/Client/Stream.pm
+++ b/lib/Net/Async/xLM/API/Client/Stream.pm
@ -0,0 +1,24 @@
+package Net::Async::xLM::API::Client::Stream;
+
+use v5.36;
+use Object::Pad;
+
+class Net::Async::xLM::API::Client::Stream {
+  use Future::Queue;
+  use Future::AsyncAwait;
+
+  field $queue = Future::Queue->new();
+  field $io_stream :param;
+
+  ADJUST {
+  }
+
+  # Put an event into the queue once it's come in from $io_stream
+  method _send_event() {
+
+  }
+
+  async method next_event() {
+
+  }
+}
--- a/lib/Net/Async/xLM/API/Server.pm
+++ b/lib/Net/Async/xLM/API/Server.pm
@ -0,0 +1,418 @@
+package Net::Async::xLM::API::Server;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+use Future::Queue;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server - IO::Async based server for OpenAI compatible APIs
+
+=head1 SYNOPSIS
+
+  use IO::Async::Loop;
+  use Net::Async::xLM::API::Server;
+  use builtin qw/true false/;
+
+  my $loop = IO::Async::Loop->new();
+
+  class MyServer {
+    inherit Net::Async::xLM::API::Server;
+
+    method init() {
+      # We return the info on where we should be listening, and any other settings for Net::Async::HTTP::Server
+
+      return [
+        {
+          port => "8085",
+          listen => "127.0.0.1",
+          ...
+        },
+        ...
+      ];
+    }
+
+    async method auth_check($key, $http_req) {
+      # we can implement any auth checks that we need here
+      return true; # AIs need to be free!
+    }
+
+    async method chat_completion($chat_completion_req) {
+      ...
+
+      return $chat_completion_response;
+    }
+
+    ...
+  }
+
+  my $server = MyServer->new();
+
+  $loop->add($server);
+
+  $loop->run()
+
+=head1 THEORY OF OPERATION
+
+This module implements the L<IO::Async::Notifier> interface, this means that you create a new server and then call C<< $loop->add($client) >>
+this casues all L<Future>s that are created to be part of the L<IO::Async::Loop> of your program.  This way when you call C<await> on any method
+it will properly suspend the execution of your program and do something else concurrently (probably waiting on requests).
+
+All the methods that you provide must be async, this is so that multiple requests can be easily handled internally with ease.
+
+You subclass the ::Server class and implement your own methods for handling the limited number of requests, and provide some
+configuration information for the servers to start in the configure() method.  The reason to put this into a method is so that
+The server object can handle reloading itself without being recreated.  This way any sockets that are already listening don't have
+to be closed and reopened to be reconfigured (assuming that the new configuration keeps them open).
+
+Streaming from ::Server is still being designed, I'll publish this system WITHOUT streaming support the first time around
+since I need to write at least a new http client module that supports it in order to test things properly, and the make Net::Async::xLM::API::Client
+work with streaming events anyway.
+
+=head1 Methods
+
+=head2 new()
+
+Create a new Net::Async::xLM::API::Server.  You'll need to register the server with C<< $loop->add($server) >> after creation.
+
+=head3 PARAMETERS
+
+=over 4
+
+=item * ?
+
+TODO FILL IN
+
+=back
+
+=head2 init($loop)
+
+The intialization phase, this happens before the configure() method is called, so that you can setup any local data stores and validate
+the environment.  This will only be called once, but configure() may be called multiple times, such as from a signal handler (SIGHUP or SIGUSR1?)
+to reload any configuration for the HTTP endpoints.
+
+Not completely sure if I should have these separated but I kept not liking doing it all in one method for some reason.  Maybe fold this into BUILD {} instead?
+
+=head2 configure($loop)
+
+The configuration phase, returns a list of the arguments to be given to Net::Async::HTTP::Server
+
+TODO bring in some docuemtation, this is derived from C<IO::Async::Loop> 's listen method, and we also need to refer to the connect method for more details
+basically, just C<< listen => [addr => '::1', port => 8080, socktype => 'stream'] >> as a basic example.
+
+We handle multiple of these so we can have this proxy listen on multiple ports, and then handle any other behavior for it
+
+The context key will be passed to all of the methods doing the handling here, so you can use it for static data based on which
+connection actually was used.  This should help handle multiple networks/vlans or auth patterns a bit easier from a single daemon.
+
+I think this should also let you pass in a listening handle from another module like IO::Socket::SSL or something to do native SSL
+But I'm not going to be testing that for some time myself, and would recommend using something else like nginx, haproxy, etc. as a
+terminator for TLS connections
+
+    [
+      {
+        listen => {addrs => ['::1', ...], ...},
+        context => {...}
+        on_something => code    
+      },
+      ...
+    ]
+
+=head2 async auth_check($key, $ctx, $http_req)
+
+This method requres async keyword.
+
+Return a true or false value on if a request should be authorized.  This is given the API-Key value from the Authorization header
+
+if $key is undef then you can use the $http_req object to get a look at the full headers, so you can implement whatever other authorization system you need
+
+By default all OpenAI compatible APIs are expecting ONLY an API-Key type Authorization header so doing anytyhing else isn't strictly compatible but there's no reason that this server shouldn't be able to do more if you're customizing things
+
+=head2 async completion($ctx, $completion)
+
+=head3 DEPRECATED
+
+This method requres async keyword.
+
+Handle a completion request, takes in a request object, must return a response object.
+
+=head2 async chat($ctx, $chat_completion)
+
+This method requres async keyword.
+
+Handle a chat completion request
+
+=head2 async embedding($ctx, $embedding)
+
+This method requres async keyword.
+
+Handle an embedding request
+
+=head2 async image_generate($ctx, $image_req)
+
+This method requres async keyword.
+
+Unimplemented, but once present will be used to generate images with Dall-E (or for self hosted, stable diffusion).
+
+=head2 async text_to_speech($ctx, $tts_req)
+
+This method requres async keyword.
+
+Unimplemented, but can be used to turn text to speech using whatever algorithms/models are supported.
+
+=head2 async speech_to_text($ctx, $stt_req)
+
+This method requres async keyword.
+
+Unimplemented. The opposite of the above.
+
+=head2 async vision($ctx, $vision_req)
+
+This method requres async keyword.
+
+Unimplemented, I've not investigated this one much yet but I believe it's to get a description of an image and it's contents.
+
+=head2 Missing apis
+
+At least some for getting the list of models and some other meta information, those will be added next after I get some more documentation written
+
+=head1 SERVER SENT EVENTS
+
+Design for this is pending, I'll end making this use new methods, i.e. C<stream_chat_completion>, etc.  These will take in a new $stream object, that can have an event written to it which will be sent without closing the connection.
+This is mostly because using this as a proxy will require handling a different kind of client to call another OpenAI endpoint which will necessitate a loop inside
+the method that is handling the other end.
+
+=head1 See Also
+
+L<IO::Async>, L<Future::AsyncAwait>, L<Net::Async::HTTP>
+
+=head1 License
+
+Artistic 2.0
+
+=head1 Author
+
+Ryan Voots, ... etc.
+
+=cut
+
+class Net::Async::xLM::API::Server :repr(HASH) :strict(params) {
+  inherit IO::Async::Notifier;
+
+  use JSON::MaybeXS qw//;
+  use Net::Async::HTTP::Server;
+  use Feature::Compat::Try;
+  use URI;
+  use WWW::Form::UrlEncoded;
+  no warnings 'experimental';
+  use builtin qw/true false/;
+  use Hash::Merge;
+  use HTTP::Response;
+  use HTTP::Request;
+  use Scalar::Util qw/blessed/;
+
+  use Net::Async::xLM::API::Types::Requests;
+  use Net::Async::xLM::API::Types::Results;
+
+  field $_json = JSON::MaybeXS->new(utf8 => 1, convert_blessed => 1);
+  field $http_server;
+
+  field $port :param = "8080";
+  field $listen :param = "127.0.0.1";
+  field $ctx :param = {};
+  field $httpserver_args :param = {}; # by default nothing
+
+  # TODO document these directly, other options gets mixed in BEFORE all of these
+  field $io_async_notifier_params :param = undef;
+
+  method configure(%params) {
+    # We require them to go this way, so that there is no conflicts
+    # TODO document this
+    my %io_async_params = ($params{io_async_notifier_params} // {})->%*;
+    IO::Async::Notifier::configure($self, %io_async_params);
+  }
+
+  # These might not stay internal only
+  method _make_future() {
+    # TODO make this workable with Future::IO too, but longer term goal
+    return $self->loop->new_future();
+  }
+
+  method _make_queue() {
+    my $queue = Future::Queue->new(
+      prototype => sub {$self->_make_future()},
+      max_items => 1, # set a max item count, this is so that await ->push() will cause things to yield
+    );
+    return $queue;
+  }
+
+  method __make_http_server() {
+    # TODO args?
+    # TODO make this work during a reload
+    my $server_id = sprintf("%s\0%d", $listen, $port);
+    $ctx->{server_id} = $server_id;
+
+    $http_server = Net::Async::HTTP::Server->new(
+      $httpserver_args->%*,
+      on_request => sub($httpself, $req) {
+        my $async_f = $self->_route_request($req, $ctx);
+        $self->adopt_future($async_f);
+      }
+    );
+
+    $self->loop->add($http_server);
+
+    my $merger = Hash::Merge->new('LEFT_PRECEDENT');
+
+    my $http_args = $merger->merge($httpserver_args, {addr => {socktype => "stream", port => $port, ip => $listen, family => "inet"}});
+
+    $http_server->listen($http_args->%*)->get();
+  }
+
+  method _add_to_loop {
+    $self->__make_http_server();
+  }
+
+  method _resp_custom($req, $code, $str, $json = 0) {
+    my $response = HTTP::Response->new( $code );
+
+    if (blessed($str)) {
+      my $new_str = $str->_serialize();
+      my $ct = $str->_content_type();
+
+      $response->content_type($ct);
+
+      $response->add_content($new_str);
+      $response->content_length(length $new_str);
+    } else {
+      $response->content_type('text/plain') unless $json; # TODO this needs to be more flexible due to audio outputs
+      $response->content_type('application/json') if $json;
+
+      $response->add_content($str);
+      $response->content_length(length $str);
+    }
+
+    $req->respond($response);
+  }
+
+  field $routes = [];
+
+  method register_url(%opts) {
+    # TODO check params
+    #use Data::Dumper;
+    #say Dumper("Got url registered", \%opts);
+    push $routes->@*, \%opts;
+  }
+
+  async method _route_request($req, $ctx) {
+    my $method = $req->method();
+    my $path   = $req->path;
+    say "Got request ", $method, " => ", $path;
+
+    try {
+      my $found_route = false;
+      my $f;
+      for my $route ($routes->@*) {
+#        printf "  Checking %s %s\n", $route->{url}, $route->{method};
+        if ($path =~ $route->{url} && $route->{method} eq $method) {
+          my $params = +{%+, _ => [@+]}; # make a copy of named parameters, and digited ones to pass into the handler
+          $found_route = true;
+          say "Found path $route->{url}";
+
+          my $obj;
+          if ($route->{decoder} eq "www-form-urlencoded") {
+            my %data = WWW::Form::UrlEncoded::parse_urlencoded($req->body);
+            $obj = $route->{request_class}->new(%data);
+          } elsif ($route->{decoder} eq "json") {
+            my $data = $_json->decode($req->body);
+            $obj = $route->{request_class}->new(%$data);
+          } elsif ($route->{decoder} eq "null") {
+            $obj = $route->{request_class}->new();
+          } else { # Try to detect based on content-type, then fail
+            my $content_type = $req->header("Content-Type");
+            if ($content_type eq 'application/json') {
+              my $data = $_json->decode($req->body);
+              $obj = $route->{request_class}->new(%$data);
+            } elsif ($content_type eq 'application/x-www-form-urlencoded') {
+              my %data = WWW::Form::UrlEncoded::parse_urlencoded($req->body);
+              $obj = $route->{request_class}->new(%data);
+            } else {
+              die "Unsupported content-type for URI: $content_type";
+            }
+          }
+
+          try {
+            # TODO make these use some other method for creation that is abstracted out, once I get rid of IO::Async
+            my $future_status = $self->_make_future();
+            my $queue = $self->_make_queue();
+
+            # TODO can I redo this to eliminate the $future_status? I want it for internal chaining inside the handler
+            # that is needed for it to persist some code that's running in the future that populates the queue
+            my $route_method = $route->{handle};
+            $self->$route_method($future_status, $queue, $ctx, $obj, $params);
+            
+            my $status = await $future_status;
+            my $is_streaming = $status->{is_streaming};
+
+            my $headers = HTTP::Headers->new(
+              "Content-Type" => $is_streaming ? "text/event-stream" : $status->{content_type},
+              $is_streaming ? ("Cache-Control" => "no-store") : (),
+              # TODO others?
+            );
+            my $response = HTTP::Response->new($status->{status_code}, $status->{status_message}, $headers);
+            $response->protocol("HTTP/1.1");
+
+            my $resp_string = $response->as_string("\r\n");
+
+            $req->write($resp_string);
+#            $req->write("\r\n"); # extra to end headers
+
+            $req->write(sub {
+              my $body_obj = $queue->shift()->get();
+
+              if (defined $body_obj) {
+                my $body = $body_obj->_serialize();
+                my $event_name = $body_obj->_event_name();
+
+                if ($is_streaming) {
+                  return sprintf "event: %s\ndata: %s\n\n", $event_name, $body;
+                } else {
+                  return $body;
+                }
+              } else {
+                # Finished
+
+                $req->done();
+                $req->{conn}->close(); #TODO why is this needed?
+                return undef;
+              }
+              });
+
+            return;
+          } catch($err) {
+            $self->_resp_custom($req, 500, "Server error: ".$err);
+            return;
+          }
+        }
+      }
+
+      unless ($found_route) {
+        $self->_resp_custom($req, 404, "Not found");
+      }
+    } catch($err) {
+      $self->_resp_custom($req, 400, "Error: ".$err);
+    }
+  }
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/Audio.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/Audio.pm
@ -0,0 +1,79 @@
+package Net::Async::xLM::API::Server::API::v1::Audio;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::Audio - Basic audio api role, consumed to implement the OpenAI audio api.  Does not provide an implementation, you are expected to override them in your class
+
+TODO document the subroles here, split up because TTS is much simpler to implement than the others and will be more valuable to support alone if someone chooses
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+
+role Net::Async::xLM::API::Server::API::v1::AudioTTS :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/audio/speech$}, 
+        handle => "audio_create_speech",
+        request_class => "Net::Async::xLM::API::Type::Requests::CreateSpeech",
+        result_class => "", # This gives back a file of audio data
+    );
+  }
+ 
+  async method audio_create_speech($future_status, $queue, $ctx, $obj, $params);
+}
+
+role Net::Async::xLM::API::Server::API::v1::AudioSTT :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/audio/transcription$}, 
+        handle => "audio_create_transcript",
+        request_class => "Net::Async::xLM::API::Type::Requests::CreateTranscription",
+        result_class => "Net::Async::xLM::API::Type::Response::AudioFile",
+    );
+  }
+ 
+  async method audio_create_transcript($future_status, $queue, $ctx, $obj, $params);
+}
+
+role Net::Async::xLM::API::Server::API::v1::AudioTranslate :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/$}, 
+        handle => "audio_create_translation",
+        request_class => "Net::Async::xLM::API::Type::Requests::CreateTranslation",
+        result_class => "Net::Async::xLM::API::Type::Response::AudioFile",
+    );
+  }
+ 
+  async method audio_create_translation($future_status, $queue, $ctx, $obj, $params);
+}
+
+role Net::Async::xLM::API::Server::API::v1::Audio :strict(params) {
+  apply Net::Async::xLM::API::Server::API::v1::AudioTTS;
+  apply Net::Async::xLM::API::Server::API::v1::AudioSTT;
+  apply Net::Async::xLM::API::Server::API::v1::AudioTranslate;
+}
+
+1;
--- a/lib/Net/Async/xLM/API/Server/API/v1/ChatCompletion.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/ChatCompletion.pm
@ -0,0 +1,43 @@
+package Net::Async::xLM::API::Server::API::v1::ChatCompletion;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::ChatCompletion - Basic chat api role, consumed to implement the OpenAI chat completion api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::ChatCompletion :strict(params) {
+  use Future::AsyncAwait;
+
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/chat/completions$}, 
+        handle => "chat_completion",
+        request_class => "Net::Async::xLM::API::Types::Requests::ChatCompletion",
+        result_class => "Net::Async::xLM::API::Types::Results::ChatCompletion",
+        decoder => 'json', # default is json, we need this for this api
+    );
+  }
+
+  async method chat_completion($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/Completions.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/Completions.pm
@ -0,0 +1,41 @@
+package Net::Async::xLM::API::Server::API::v1::Completions;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::Completions - Basic completion api role, consumed to implement the OpenAI chat completion api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::Completions :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/completions$}, 
+        handle => "completion",
+        request_class => "Net::Async::xLM::API::Type::Request::Completion",
+        result_class => "Net::Async::xLM::API::Type::Result::Completion",
+        decoder => 'www-form-urlencoded', # default is json, we need this for this api
+    );
+  }
+ 
+  async method completion($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/Embeddings.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/Embeddings.pm
@ -0,0 +1,41 @@
+package Net::Async::xLM::API::Server::API::v1::Embeddings;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::Embeddings - Basic embeddings api role, consumed to implement the OpenAI embeddings api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::Embeddings :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/embeddings$}, 
+        handle => "embeddings",
+        request_class => "Net::Async::xLM::API::Type::Request::Embeddings",
+        result_class => "Net::Async::xLM::API::Type::Result::Embeddings",
+        decoder => 'www-form-urlencoded', # default is json, we need this for this api
+    );
+  }
+ 
+  async method embeddings($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/File.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/File.pm
@ -0,0 +1,74 @@
+package Net::Async::xLM::API::Server::API::v1::File;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::File - Basic file api role, consumed to implement the OpenAI file server.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::File :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/files$}, 
+        handle => "file_upload",
+        request_class => "Net::Async::xLM::API::Type::Request::FileUpload",
+        result_class => "Net::Async::xLM::API::Type::Shared::File",
+        decoder => 'www-form-urlencoded', # default is json, we need this for this api
+    );
+    $self->register_url(
+        method => 'GET',
+        url => qr{^/v1/files/(?<file_id>[^/]+)/content$}, 
+        handle => "file_download",
+        request_class => "", # No req type here
+        result_class => "Net::Async::xLM::API::Type::Results::RawFile",
+    );
+    $self->register_url(
+        method => 'GET',
+        url => qr{^/v1/files/(?<file_id>[^/]+)$}, 
+        handle => "file_info",
+        request_class => "", # No req type here
+        result_class => "Net::Async::xLM::API::Type::Shared::File",
+    );
+    $self->register_url(
+        method => 'DELETE',
+        url => qr{^/v1/files/(?<file_id>[^/]+)$}, 
+        handle => "file_delete",
+        request_class => "", # No req type here        
+        result_class => "Net::Async::xLM::API::Type::Results::FileDeletion",
+    );
+    $self->register_url(
+        method => 'GET',
+        url => qr{^/v1/files$}, 
+        handle => "file_list",
+        request_class => "Net::Async::xLM::API::Type::Request::FileList",
+        result_class => "Net::Async::xLM::API::Type::Results::FileList",
+        decoder => 'optional_json', # this API input is OPTIONAL, if it's not present then we create a blank object to use.
+    );
+  }
+
+  async method file_list($future_status, $queue, $ctx, $obj, $params);
+  async method file_info($future_status, $queue, $ctx, $obj, $params);
+  async method file_delete($future_status, $queue, $ctx, $obj, $params);
+  async method file_upload($future_status, $queue, $ctx, $obj, $params);
+  async method file_download($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/Image.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/Image.pm
@ -0,0 +1,40 @@
+package Net::Async::xLM::API::Server::API::v1::Image;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::Image - Basic image role, consumed to implement the OpenAI image api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::Image :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'GET',
+        url => qr{^/v1/files$}, 
+        handle => "create_image",
+        request_class => "Net::Async::xLM::API::Type::Requests::GenerateImage",
+        result_class => "Net::Async::xLM::API::Type::Results::RawFile", # TOOD image class?
+    );
+  }
+
+  async method create_image($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/ModelList.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/ModelList.pm
@ -0,0 +1,40 @@
+package Net::Async::xLM::API::Server::API::v1::ModelList;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::ModelList - Basic model list api role, consumed to implement the OpenAI model list api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::ModelList :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/models$}, 
+        handle => "model_list",
+        request_class => "",
+        result_class => "Net::Async::xLM::API::Type::Result::ModelList",
+    );
+  }
+ 
+  async method model_list($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/API/v1/Moderations.pm
+++ b/lib/Net/Async/xLM/API/Server/API/v1/Moderations.pm
@ -0,0 +1,40 @@
+package Net::Async::xLM::API::Server::API::v1::Moderations;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use Net::Async::xLM::API::Types::Results;
+use Net::Async::xLM::API::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Server::API::Moderations - Basic moderation api role, consumed to implement the OpenAI moderation api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+role Net::Async::xLM::API::Server::API::v1::Moderations :strict(params) {
+  ADJUST {
+    $self->register_url(
+        method => 'POST',
+        url => qr{^/v1/moderations$}, 
+        handle => "moderations",
+        request_class => "Net::Async::xLM::API::Type::Requests::CreateModeration",
+        result_class => "Net::Async::xLM::API::Type::Results::Moderations",
+    );
+  }
+ 
+  async method moderations($future_status, $queue, $ctx, $obj, $params);
+}
--- a/lib/Net/Async/xLM/API/Server/Stream.pm
+++ b/lib/Net/Async/xLM/API/Server/Stream.pm
@ -0,0 +1,23 @@
+package Net::Async::xLM::API::Server::Stream;
+
+use v5.36;
+use Object::Pad;
+
+class Net::Async::xLM::API::Server::Stream {
+  use Future::Queue;
+
+  # TODO what to do for non-io async setups, long term
+  field $io_stream :param;
+  
+  async method send_headers() {
+
+  }
+
+  async method send_event($event_data) {
+    
+  }
+
+  async method finish() {
+    
+  }
+}
--- a/lib/Net/Async/xLM/API/Types.pm
+++ b/lib/Net/Async/xLM/API/Types.pm
@ -0,0 +1,103 @@
+package Net::Async::xLM::API::Types;
+use v5.36.0;
+use Object::Pad;
+
+use Object::PadX::Role::AutoMarshal;
+use Object::PadX::Role::AutoJSON;
+use Object::Pad::ClassAttr::Struct;
+
+# Base role for all the types to simplify things later
+role Net::Async::xLM::API::Types::Base :Struct {
+  apply Object::PadX::Role::AutoJSON;
+  apply Object::PadX::Role::AutoMarshal;
+
+  use JSON::MaybeXS qw//;
+
+  our $_json = JSON::MaybeXS->new(utf8 => 1, convert_blessed => 1);
+
+  method _encode() {
+    return $_json->encode($self);
+  }
+}
+
+# Keep the JSON role stuff here, I might use it to annotate encodings of some non-json fields? not sure
+role Net::Async::xLM::API::Types::BaseFormEncoding :Struct {
+  apply Object::PadX::Role::AutoJSON;
+  apply Object::PadX::Role::AutoMarshal;
+
+  use WWW::Form::UrlEncoded;
+
+  use Object::Pad::MOP::FieldAttr;
+  use Object::Pad::MOP::Field;
+  use Object::Pad::MOP::Class;
+
+  my $_to_str = sub ($x) {
+    return "".$x;
+  };
+ 
+  my $_to_num = sub ($x) {
+    return 0+$x;
+  };
+ 
+  my $_to_bool = sub ($x) {
+    return !!$x ? \1 : \0;
+  };
+ 
+  my $_to_list = sub ($ref, $type) {
+    my $sub = $type eq 'JSONNum' ? $_to_num :
+              $type eq 'JSONStr' ? $_to_str :
+              $type eq 'JSONBool' ? $_to_bool :
+                                    sub {die "Wrong type $type in json conversion"};
+    return [map {$sub->($_)} $ref->@*]
+  };
+
+  method _as_hash() {
+    my $class = __CLASS__;
+    my $classmeta = Object::Pad::MOP::Class->for_class($class);
+    my @metafields = $classmeta->fields;
+ 
+    my %json_out = ();
+ 
+    for my $metafield (@metafields) {
+      my $field_name = $metafield->name;
+      my $sigil = $metafield->sigil;
+ 
+      my $has_exclude = $metafield->has_attribute("JSONExclude");
+ 
+      next if $has_exclude;
+ 
+      next if $sigil ne '$';  # Don't try to handle anything but scalars
+ 
+      my $has_null = $metafield->has_attribute("JSONNull");
+ 
+      my $value = $metafield->value($self);
+      next unless (defined $value || $has_null);
+ 
+      my $key = $field_name =~ s/^\$//r;
+      $key = $metafield->get_attribute_value("JSONKey") if $metafield->has_attribute("JSONKey");
+ 
+      if ($metafield->has_attribute('JSONBool')) {
+        $value = $_to_bool->($value);
+      } elsif ($metafield->has_attribute('JSONNum')) {
+        # Force numification
+        $value = $_to_num->($value);
+      } elsif ($metafield->has_attribute('JSONList')) {
+        my $type = $metafield->get_attribute_value('JSONList');
+        $value = $_to_list->($value, $type);
+      } else {
+        # Force stringification
+        $value = $_to_str->($value);
+      }
+ 
+      $json_out{$key} = $value;
+    }
+    return \%json_out;
+  }
+
+
+  method _encode() {
+    my $hash = $self->_as_hash();
+    my $string = WWW::Form::UrlEncoded::build_urlencoded($hash);
+    return $string;
+  }
+}
--- a/lib/Net/Async/xLM/API/Types/Requests.pm
+++ b/lib/Net/Async/xLM/API/Types/Requests.pm
@ -0,0 +1,357 @@
+package Net::Async::xLM::API::Types::Requests;
+use v5.36.0;
+use Object::Pad;
+
+use Object::PadX::Role::AutoMarshal;
+use Object::PadX::Role::AutoJSON;
+use Object::Pad::ClassAttr::Struct;
+use Net::Async::xLM::API::Types;
+use Net::Async::xLM::API::Types::Shared;
+
+role Net::Async::xLM::API::Types::Requests::Base :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  method _endpoint(); # How the client finds where to send the request
+  method decoder() {"json"}
+  method encoder() {"json"}
+}
+
+role Net::Async::xLM::API::Types::Requests::BaseFormEncoding :Struct {
+  apply Net::Async::xLM::API::Types::BaseFormEncoding;
+  method _endpoint(); # How the client finds where to send the request
+  method decoder() {"www-form-urlencoded"}
+  method encoder() {"www-form-urlencoded"}
+}
+
+#### Base Request Types
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/chat/completions"}
+  
+  field $messages :MarshalTo([Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Union]);
+  field $model :JSONStr = "gpt-3.5-turbo";
+  field $frequency_penalty :JSONNum = undef;
+  field $presence_penalty :JSONNum = undef;
+  field $logit_bias = undef; # TODO wtf is this?
+  field $max_tokens :JSONNum = undef;
+  field $response_format :JSONStr :JSONExclude = undef; # I'm not supporting this this version yet
+
+  field $seed :JSONNum = undef;
+  field $stop = undef; # String, array or null, todo handle
+  field $stream :JSONBool = undef; # TODO handle
+  field $temperature :JSONNum = undef;
+  field $top_p :JSONNum = undef;
+  field $tools :JSONExclude = undef; # TODO handle this
+  field $tool_choice :JSONExclude = undef; # TODO handle this
+
+  field $function_call :JSONExclude = undef;
+  field $functions :JSONExclude = undef;
+}
+class Net::Async::xLM::API::Types::Requests::Completion :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+
+  method _endpoint() {"/completions"}
+  
+  field $model :JSONStr = "gpt-3.5-turbo"; # This is how 99% of everyone else seems to default this
+  field $prompt :JSONStr;
+  
+  field $max_tokens :JSONNum = undef; # use the platform default usually
+  field $temperature :JSONNum = undef;
+  field $top_p :JSONNum = undef;
+  field $seed :JSONNum  = undef;
+  field $echo :JSONBool = undef; # true or false only
+  field $suffix :JSONStr = undef;
+  field $stop :JSONStr = undef; # array of stop tokens
+  field $user :JSONStr = undef; # used for tracking purposes later
+
+  field $frequency_penalty :JSONNum = undef;
+  field $presence_penalty :JSONNum = undef;
+  
+  field $logit_bias = undef; # TODO make this work
+  field $log_probs = undef; # TODO
+  
+  field $n :JSONNum = undef; # Danger will robinson! easy to cause $$$$$$$ costs
+  field $best_of :JSONNum = undef;
+
+  field $stream :JSONBool = undef; # TODO FALSE ALWAYS RIGHT NOW
+
+  ADJUST {
+    # Type assertions here
+    die "Streaming unsupported" if $self->stream;
+  }
+}
+
+class Net::Async::xLM::API::Types::Requests::Embedding :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/embeddings"}
+  field $input :JSONStr;
+  field $model :JSONStr;
+  field $encoding_format :JSONStr = undef;
+  field $user :JSONStr = undef;
+}
+
+### Request Subtypes
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant::ToolCall :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $id :JSONStr;
+  field $arguments :JSONStr;
+  field $type :JSONStr;
+  field $function :MarshalTo(Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall);
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $arguments :JSONStr;
+  field $name :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::Text :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $type :JSONStr;
+  field $text :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::ImageUrl :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $url :JSONStr;
+  field $detail :JSONStr = undef;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::Image :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $type :JSONStr;
+  field $image_url :MarshalTo(Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::ImageUrl);
+}
+
+# TODO, why have two of these? just shove it into the big one below
+
+package 
+    Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::ContentUnion {
+  # This guy does some additional checks to give us the right type here
+  
+  sub new {
+    my $class = shift @_;
+    my %input = @_;
+  
+    die "Missing type in creation" unless $input{type};
+
+    if ($input{type} eq 'text') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::Text->new(%input);
+    } elsif ($input{type} eq 'image_url') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::Image->new(%input);
+    } else {
+      die "Unsupported ChatCompletion User Message type: [".$input{type}."]";
+    }
+  }
+};
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  # This particular type is more complicated than AutoMarshal can handle, so we need to
+  # do this in a custom manner.
+  field $role;
+  field $name = undef;
+  field $content;
+
+  ADJUST {
+    my $create_obj = sub {
+      my $cont = shift;
+
+      if (ref($cont) eq 'HASH') {
+        # We've got a more detailed type here, create the union type here
+        my $obj = Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User::ContentUnion->new(%$cont);
+      } elsif (ref($cont) eq '') {
+        return $cont; # Bare string/scalar is fine
+      } else {
+        die "Can't nest other types in \$content of a ChatCompletion user message: ".ref($cont);
+      }
+    };
+
+    if (ref($content) eq 'ARRAY') {
+      $content = [map {$create_obj->($_)} $content->@*];
+    } else {
+      # TODO check that this is acutally doing the right thing.  I think it might not be for user messages that are just text
+      $content = $create_obj->($content);
+    }
+  }
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $role :JSONStr;
+  field $content :JSONStr;
+  field $name = undef;
+  field $tool_calls :MarshalTo([Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant::ToolCall]) = undef;
+  field $function_call :MarshalTo(Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall) = undef;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Function :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $role :JSONStr;
+  field $content :JSONStr;
+  field $name :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Tool :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $role :JSONStr;
+  field $content :JSONStr;
+  field $tool_call_id :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::System :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+  field $role :JSONStr;
+  field $name :JSONStr = undef;
+  field $content :JSONStr;
+}
+
+
+package 
+  Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Union {
+  # This guy does some additional checks to give us the right type here
+  
+  sub new {
+    my ($class, %input) = @_;
+    die "Missing role in creation" unless $input{role};
+
+    if ($input{role} eq 'system') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::System->new(%input);
+    } elsif ($input{role} eq 'user') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::User->new(%input);
+    } elsif ($input{role} eq 'tool') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Tool->new(%input);
+    } elsif ($input{role} eq 'function') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Function->new(%input);
+    } elsif ($input{role} eq 'assistant') {
+      return Net::Async::xLM::API::Types::Requests::ChatCompletion::Messages::Assistant->new(%input);
+    } else {
+      die "Unsupported ChatCompletion Message role: [".$input{role}."]";
+    }
+  }
+};
+
+class Net::Async::xLM::API::Types::Requests::FileUpload :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/files"}
+
+  field $file :MarshalTo(Net::Async::xLM::API::Types::Shared::FileObject);
+  field $purpose :JSONStr; # fine-tune and assistants for the types, TODO check format/type of file
+}
+
+
+class Net::Async::xLM::API::Types::Requests::FileList :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/files"}
+
+  field $purpose :JSONStr = undef; # fine-tune and assistants for the types, optional, used for filtering
+}
+
+class Net::Async::xLM::API::Types::Requests::FileInfo :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/files/".$self->file_id}
+
+  field $file_id :JSONStr; # id of the file to retrieve
+}
+
+class Net::Async::xLM::API::Types::Requests::FileDelete :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/files/".$self->file_id}
+
+  field $file_id :JSONStr; # id of the file to retrieve
+}
+
+class Net::Async::xLM::API::Types::Requests::FileContent :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/files/".$self->file_id.'/content'}
+
+  field $file_id :JSONStr; # id of the file to retrieve
+}
+
+class Net::Async::xLM::API::Types::Requests::CreateSpeech :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/audio/speech"}
+
+  field $model :JSONStr = 'tts-1'; # default to cheapest model for simpler requests
+  field $input :JSONStr; # TODO max 4k chars?
+  field $voice :JSONStr; # TODO default to alloy?
+  field $response_format :JSONStr = undef; # mp3, opus, aac, or flac
+  field $speed :JSONNum = undef; # default 1.0, range 0.25 to 4.0
+}
+
+class Net::Async::xLM::API::Types::Requests::CreateTranscript :Struct {
+  apply Net::Async::xLM::API::Types::Requests::BaseFormEncoding;
+  method _endpoint() {"/audio/transcript"}
+
+  field $file;
+  field $model;
+  field $language = undef; # What language to use, ISO-639-1 format
+  field $prompt = undef; # Text to guide the model's style or continue a previous audio segment
+  field $response_format = undef; # json, text, srt, verbose_json or vtt
+  field $temperature = undef; # number, between 0 and 1.  higher values with make the ouput more random but lower values will make it more deterministic.
+}
+
+# ED: Why do they only support translating audio to english? seems really limited and I feel like this API will get
+# updated or replaced fairly soon
+class Net::Async::xLM::API::Types::Requests::CreateTranslations :Struct {
+  apply Net::Async::xLM::API::Types::Requests::BaseFormEncoding;
+
+  method _endpoint() {"/audio/translations"}
+
+  field $file;
+  field $model;
+  field $prompt = undef; # Text to guide the model's style or continue a previous audio segment
+  field $response_format = undef; # json, text, srt, verbose_json or vtt
+  field $temperature = undef; # number, between 0 and 1.  higher values with make the ouput more random but lower values will make it more deterministic.
+}
+
+class Net::Async::xLM::API::Types::Requests::Moderations :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/moderations"}
+
+  field $input :JSONStr;
+  field $model :JSONStr = undef;
+}
+
+class Net::Async::xLM::API::Types::Requests::GenerateImage :Struct {
+  apply Net::Async::xLM::API::Types::Requests::Base;
+  method _endpoint() {"/images/generations"}
+
+  field $prompt :JSONStr;
+  field $model :JSONStr = undef;
+  field $n :JSONNum = undef; # how many to generate
+  field $quality :JSONStr = undef; # defaults to "standard", can also be "hd"
+  field $response_format :JSONStr = undef; # url, or b64_json
+  field $size :JSONStr = undef; # defaults to 1024x1024
+  field $style :JSONStr = undef; # vivid or natural, defaults to vivid
+  field $user :JSONStr = undef;
+}
+
+class Net::Async::xLM::API::Types::Requests::CreateImageEdit :Struct {
+  apply Net::Async::xLM::API::Types::Requests::BaseFormEncoding;
+  method _endpoint() {"/images/edits"}
+
+  field $image; # Image file data, TODO document?
+  field $mask = undef; # Image file data for mask, TODO document
+  field $prompt :JSONStr;
+  field $model :JSONStr = undef;
+  field $n :JSONNum = undef; # how many to generate
+  field $response_format :JSONStr = undef; # url, or b64_json
+  field $size :JSONStr = undef; # defaults to 1024x1024
+  field $user :JSONStr = undef;
+}
+
+class Net::Async::xLM::API::Types::Requests::CreateImageVariation :Struct {
+  apply Net::Async::xLM::API::Types::Requests::BaseFormEncoding;
+  method _endpoint() {"/images/variations"}
+
+  field $image; # Image file data, TODO document?
+  field $model :JSONStr = undef;
+  field $n :JSONNum = undef; # how many to generate
+  field $response_format :JSONStr = undef; # url, or b64_json
+  field $size :JSONStr = undef; # defaults to 1024x1024
+  field $user :JSONStr = undef;
+}
+
+1;
--- a/lib/Net/Async/xLM/API/Types/Requests/ChatCompletion.pod
+++ b/lib/Net/Async/xLM/API/Types/Requests/ChatCompletion.pod
@ -2,20 +2,20 @@

 =head1 NAME

-OpenAIAsync::Types::Request::ChatCompletion
+Net::Async::xLM::API::Types::Request::ChatCompletion

 =head1 DESCRIPTION

-A chat completion request, once put through the client you'll get a L<OpenAIAsync::Types::Response::ChatCompletion> with the result of the model.
+A chat completion request, once put through the client you'll get a L<Net::Async::xLM::API::Types::Results::ChatCompletion> with the result of the model.

 =head1 SYNOPSIS

-  use OpenAIAsync::Client;
+  use Net::Async::xLM::API::Client;
  use IO::Async::Loop;

  my $loop = IO::Async::Loop->new();

-  my $client = OpenAIAsync::Client->new();
+  my $client = Net::Async::xLM::API::Client->new();
  $loop->add($client);

  my $output_future = $client->chat({
@ -38,7 +38,7 @@ A chat completion request, once put through the client you'll get a L<OpenAIAsyn

 =head2 messages (required)

-The messages that are part of the chat, see the L<OpenAIAsync::Types::Request::ChatCompletion/messages> section for details
+The messages that are part of the chat, see the L<Net::Async::xLM::API::Types::Request::ChatCompletion/messages> section for details

 =head2 model

@ -104,7 +104,7 @@ lead to less variation in the responses at the same time.

 =head2 response_format

-This is currently ignored by OpenAIAsync::Client right now, but will be used to force generation of specific formats of responses.
+This is currently ignored by Net::Async::xLM::API::Client right now, but will be used to force generation of specific formats of responses.

 OpenAI supports two values, null and C<json_object> to force a correctly formatted JSON response.  Needs additional documentation
 for how to use this before I enable it.
@ -199,7 +199,7 @@ That will generate a new response based on the results of the function calls wit

 =head1 SEE ALSO

-L<OpenAIAsync::Types::Response::ChatCompletion>, L<OpenAIAsync::Client>
+L<Net::Async::xLM::API::Types::Results::ChatCompletion>, L<OpenAIAsync::Client>

 =head1 AUTHOR

--- a/lib/Net/Async/xLM/API/Types/Requests/Completion.pod
+++ b/lib/Net/Async/xLM/API/Types/Requests/Completion.pod
@ -2,11 +2,11 @@

 =head1 NAME

-OpenAIAsync::Types::Request::Completion
+Net::Async::xLM::API::Types::Request::Completion

 =head1 DESCRIPTION

-A completion request, once put through the client you'll get a L<OpenAIAsync::Types::Response::Completion> with the result of the model.
+A completion request, once put through the client you'll get a L<Net::Async::xLM::API::Types::Results::Completion> with the result of the model.

 This type of request is officially deprecated by OpenAI and got it's final update in June 2023.  That said it's a very simple API and will
 likely exist for some time, but it can be more difficult to control and get continuous responses since you have to do all the prompt formatting
@ -14,11 +14,11 @@ yourself.

 =head1 SYNOPSIS

-  use OpenAIAsync::Client;
+  use Net::Async::xLM::API::Client;
  use IO::Async::Loop;

  my $loop = IO::Async::Loop->new();
-  my $client = OpenAIAsync::Client->new();
+  my $client = Net::Async::xLM::API::Client->new();

  $loop->add($client)

@ -61,7 +61,7 @@ Or for an self hosted inference server running a WizardLM style model:

 You will need to consult with whatever model you are using to properly format and handle the response from the model.  Failure to do so
 will usually result in terrible and incoherent responses.  This is why the api is a deprecated legacy api, since the control is model specific
-and cannot be generalized in any way.  For the replacement see L<OpenAIAsync::Types::Requests::ChatCompletion> for a better API, even if you
+and cannot be generalized in any way.  For the replacement see L<Net::Async::xLM::API::Types::Requests::ChatCompletion> for a better API, even if you
 are not explicitly doing a chat session.

 =head2 model
@ -155,7 +155,7 @@ lead to less variation in the responses at the same time.

 =head1 SEE ALSO

-L<OpenAIAsync::Types::Response::Completion>, L<OpenAIAsync::Client>
+L<Net::Async::xLM::API::Types::Results::Completion>, L<OpenAIAsync::Client>

 =head1 AUTHOR

--- a/lib/Net/Async/xLM/API/Types/Requests/Embedding.pod
+++ b/lib/Net/Async/xLM/API/Types/Requests/Embedding.pod
@ -2,19 +2,19 @@

 =head1 NAME

-OpenAIAsync::Types::Request::Embedding
+Net::Async::xLM::API::Types::Request::Embedding

 =head1 DESCRIPTION

-An embedding request, once put through the client you'll get a L<OpenAIAsync::Types::Response::Embedding> with the result of the model.
+An embedding request, once put through the client you'll get a L<Net::Async::xLM::API::Types::Results::Embedding> with the result of the model.

 =head1 SYNOPSIS

  use IO::Async::Loop;
-  use OpenAIAsync::Client;
+  use Net::Async::xLM::API::Client;

  my $loop = IO::Async::Loop->new();
-  my $client = OpenAIAsync::Client->new();
+  my $client = Net::Async::xLM::API::Client->new();

  $loop->add($client);

@ -47,7 +47,7 @@ Parameter used for tracking users when you make the api request.  Give it whatev

 =head1 SEE ALSO

-L<OpenAIAsync::Types::Response::Embedding>, L<OpenAIAsync::Client>
+L<Net::Async::xLM::API::Types::Results::Embedding>, L<OpenAIAsync::Client>

 =head1 AUTHOR

--- a/lib/Net/Async/xLM/API/Types/Results.pm
+++ b/lib/Net/Async/xLM/API/Types/Results.pm
@ -0,0 +1,256 @@
+package Net::Async::xLM::API::Types::Results;
+use v5.36.0;
+use Object::Pad;
+
+use Net::Async::xLM::API::Types;
+use Object::PadX::Role::AutoMarshal;
+use Object::PadX::Role::AutoJSON;
+use Object::Pad::ClassAttr::Struct;
+
+role Net::Async::xLM::API::Types::Results::Encoder::JSON {
+  apply Net::Async::xLM::API::Types::Base;
+  apply Object::PadX::Role::AutoJSON;
+  apply Object::PadX::Role::AutoMarshal;
+
+  use JSON::MaybeXS;
+  my $_json = JSON::MaybeXS->new(utf8 => 1, convert_blessed => 1, canonical => 1);
+
+  method _serialize() {
+    my $json = $_json->encode($self);
+
+    return $json;
+  }
+
+  method _content_type() {"application/json"}
+  method _event_name() {"event"}
+}
+
+role Net::Async::xLM::API::Types::Results::Encoder::Raw {
+  apply Net::Async::xLM::API::Types::Base;
+  apply Object::PadX::Role::AutoJSON;
+  apply Object::PadX::Role::AutoMarshal;
+
+  use JSON::MaybeXS;
+
+  method serialize() {
+    ... # TODO this needs to give out bytes, how to decide that? meta programming?
+  }
+}
+
+role Net::Async::xLM::API::Types::Results::Encoder::WWWForm {
+  apply Net::Async::xLM::API::Types::Base;
+  apply Object::PadX::Role::AutoJSON;
+  apply Object::PadX::Role::AutoMarshal;
+
+  use JSON::MaybeXS;
+
+  method serialize() {
+    ...
+  }
+}
+
+class Net::Async::xLM::API::Types::Results::ToolCall :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $id :JSONStr = undef;
+  field $type :JSONStr = undef; # always "function" right now, may get expanded in the future
+  field $function :MarshalTo(Net::Async::xLM::API::Types::Results::FunctionCall) = undef;
+}
+
+class Net::Async::xLM::API::Types::Results::FunctionCall :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $arguments :JSONStr = undef; # TODO decode the json from this directly?
+  field $name :JSONStr = undef;
+}
+
+class Net::Async::xLM::API::Types::Results::ChatMessage :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $content :JSONStr;
+  field $tool_calls :MarshalTo([Net::Async::xLM::API::Types::Results::ToolCall]) = undef; # don't think my local server provides this
+  field $role :JSONStr;
+  field $function_call :MarshalTo(Net::Async::xLM::API::Types::Results::FunctionCall) = undef; # Depcrecated, might still happen
+}
+
+class Net::Async::xLM::API::Types::Results::ChatCompletionChoices :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $finish_reason :JSONStr;
+  field $index :JSONNum;
+  field $message :MarshalTo(Net::Async::xLM::API::Types::Results::ChatMessage);
+}
+
+class Net::Async::xLM::API::Types::Results::ChatCompletion :Struct {
+  apply Net::Async::xLM::API::Types::Results::Encoder::JSON
+
+  field $id :JSONStr;
+  field $choices :MarshalTo([Net::Async::xLM::API::Types::Results::ChatCompletionChoices]);
+  field $created :JSONStr;
+  field $model :JSONStr;
+  field $system_fingerprint :JSONStr = undef; # My local system doesn't provide this
+  field $usage :MarshalTo(Net::Async::xLM::API::Types::Results::Usage);
+  field $object :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Results::ChunkDelta :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $content :JSONStr;
+  field $function_call :MarshalTo(Net::Async::xLM::API::Types::Results::FunctionCall) = undef;
+  field $tool_cass :MarshalTo([Net::Async::xLM::API::Types::Results::ToolCall]) = undef;
+  field $role :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Results::ChatCompletionChunkChoices :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $delta :MarshalTo(Net::Async::xLM::API::Types::Results::ChunkDelta);
+  field $finish_reason :JSONStr;
+  field $index :JSONStr;
+}
+
+# This is part of the streaming API
+class Net::Async::xLM::API::Types::Results::ChatCompletionChunk :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $id :JSONStr;
+  field $choices :MarshalTo(Net::Async::xLM::API::Types::Results::ChatCompletionChunkChoices);
+  field $created :JSONStr;
+  field $model :JSONStr;
+  field $system_fingerprint :JSONStr = undef;
+  field $object :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Results::Usage :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $total_tokens :JSONNum;
+  field $prompt_tokens :JSONNum;
+  field $completion_tokens :JSONNum = undef; # look at chat completions, is this the same
+}
+
+class Net::Async::xLM::API::Types::Results::LogProbs :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  # TODO what's the representation here?
+  field $text_offset = undef;
+  field $token_logprobs = undef;
+  field $tokens = undef;
+  field $top_logprobs = undef;
+}
+
+class Net::Async::xLM::API::Types::Results::CompletionChoices :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $text :JSONStr; 
+  field $index :JSONNum;
+  field $logprobs :MarshalTo(Net::Async::xLM::API::Types::Results::LogProbs) = undef; # TODO make nicer type?
+  field $finish_reason :JSONStr = undef; # TODO enum? helper funcs for this class? ->is_finished?
+}
+
+class Net::Async::xLM::API::Types::Results::Completion :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $id :JSONStr;
+  field $choices :MarshalTo([Net::Async::xLM::API::Types::Results::CompletionChoices]);
+  field $created :JSONStr;
+  field $model :JSONStr;
+  field $system_fingerprint = undef; # my local implementation doesn't provide this, openai does it for tracking changes somehow
+  field $usage :MarshalTo(Net::Async::xLM::API::Types::Results::Usage);
+  field $object :JSONStr;
+}
+
+
+class Net::Async::xLM::API::Types::Results::Embedding :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $object :JSONStr;
+  field $model :JSONStr;
+  field $usage :MarshalTo(Net::Async::xLM::API::Types::Results::Usage);
+  field $data :MarshalTo([Net::Async::xLM::API::Types::Results::EmbeddingData]);
+}
+
+class Net::Async::xLM::API::Types::Results::EmbeddingData :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $index :JSONNum;
+  field $embedding :JSONList(JSONNum);
+  field $object :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Results::ModelList :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $object :JSONStr = 'list';
+  field $data :MarshalTo(Net::Async::xLM::API::Types::Results::ModelInfo);
+}
+
+class Net::Async::xLM::API::Types::Results::ModelInfo :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $created :JSONNum;
+  field $id :JSONStr;
+  field $object :JSONStr = "model";
+  field $owned_by :JSONStr;
+}
+
+class Net::Async::xLM::API::Types::Results::Moderation :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $id :JSONStr;
+  field $model :JSONStr;
+  field $results :MarshalTo([Net::Async::xLM::API::Types::Results::ModerationResults]); # Not really sure why it's an array, the input doesn't allow multiple things to categorize
+}
+
+class Net::Async::xLM::API::Types::Results::ModerationResults :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $flagged :JSONBool;
+  field $categories :MarshalTo(Net::Async::xLM::API::Types::Results::ModerationResultsCategories);
+  field $category_scores :MarshalTo(Net::Async::xLM::API::Types::Results::ModerationResultsCategoryScores);
+}
+
+class Net::Async::xLM::API::Types::Results::ModerationResultsCategories :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $hate :JSONBool;
+  field $hate_threatening :JSONBool :JSONKey(hate/threatening);
+  field $harassment :JSONBool;
+  field $harassment_threatening :JSONBool :JSONKey(harassment/threatening);
+  field $self_harm :JSONBool :JSONKey(self-harm);
+  field $self_harm_intent :JSONBool :JSONKey(self-harm/intent);
+  field $self_harm_instructions :JSONBool :JSONKey(self-harm/instructions);
+  field $sexual :JSONBool;
+  field $sexual_minors :JSONBool :JSONKey(sexual/minors);
+  field $violence :JSONBool;
+  field $violence_graphic :JSONBool :JSONKey(violence/graphic);
+}
+
+class Net::Async::xLM::API::Types::Results::ModerationResultsCategoryScores :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $hate :JSONNum;
+  field $hate_threatening :JSONNum :JSONKey(hate/threatening);
+  field $harassment :JSONNum;
+  field $harassment_threatening :JSONNum :JSONKey(harassment/threatening);
+  field $self_harm :JSONNum :JSONKey(self-harm);
+  field $self_harm_intent :JSONNum :JSONKey(self-harm/intent);
+  field $self_harm_instructions :JSONNum :JSONKey(self-harm/instructions);
+  field $sexual :JSONNum;
+  field $sexual_minors :JSONNum :JSONKey(sexual/minors);
+  field $violence :JSONNum;
+  field $violence_graphic :JSONNum :JSONKey(violence/graphic);
+}
+
+class Net::Async::xLM::API::Types::Results::Image :Struct {
+  apply Net::Async::xLM::API::Types::Base;
+
+  field $b64_json :JSONStr = undef;
+  field $url :JSONStr = undef;
+  field $revised_prompt :JSONStr = undef;
+
+  ADJUST {
+    die "Missing required value one of b64_json or url" unless ($b64_json or $url);
+  }
+}
--- a/lib/Net/Async/xLM/API/Types/Results/ChatCompletion.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/ChatCompletion.pod
@ -0,0 +1,75 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::ChatCompletion
+
+=head1 DESCRIPTION
+
+An object representing a Chat Completion response, see L<Net::Async::xLM::API::Types::Request::ChatCompletion>
+
+=head1 SYNOPSIS
+
+  use Net::Async::xLM::API::Client;
+  use IO::Async::Loop;
+
+  my $loop = IO::Async::Loop->new();
+
+  my $client = Net::Async::xLM::API::Client->new();
+  $loop->add($client);
+
+  my $output_future = $client->chat({
+      model => "gpt-3.5-turbo",
+      messages => [
+        {
+          role => "system",
+          content => "You are a helpful assistant that tells fanciful stories"
+        },
+        {
+          role => "user",
+          content => "Tell me a story of two princesses, Judy and Emmy.  Judy is 8 and Emmy is 2."
+        }
+      ],
+
+    max_tokens => 1024, 
+  });
+
+=head1 Fields
+
+=head2 id
+
+id of the response, used for debugging and tracking
+
+=head2 choices
+
+The chat responses, L<Net::Async::xLM::API::Types::Results::ChatCompletionChoices> for details.  The text of the responses will be here
+
+=head2 created
+
+Date and time of when the response was generated
+
+=head2 model
+
+Name of the model that actually generated the response, may not be the same as the requested model depending on the service
+
+=head2 system_fingerprint
+
+Given by the service to identify which server actually generated the response, used to detect changes and issues with servers
+
+=head2 usage
+
+Token counts for the generated responses, in a L<Net::Async::xLM::API::Types::Results::Usage> object.  Has C<total_tokens>, C<prompt_tokens>, and C<completion_tokens> fields.
+
+=head2 object
+
+Static field that will likely only ever contain, C<chat.completion>
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Types::Result::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Results/Completion.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/Completion.pod
@ -0,0 +1,69 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::Completion
+
+=head1 DESCRIPTION
+
+A result from a completion request, L<Net::Async::xLM::API::Types::Request::Completion>
+
+=head1 SYNOPSIS
+
+  use Net::Async::xLM::API::Client;
+  use IO::Async::Loop;
+
+  my $loop = IO::Async::Loop->new();
+  my $client = Net::Async::xLM::API::Client->new();
+
+  $loop->add($client)
+
+  my $output_future = $client->completion({max_tokens => 1024, prompt => "Tell a story about a princess named Judy and her princess sister Emmy"});
+
+  my $result = $output_future->get();
+
+  print $result->choices->[0]->text;
+
+=head1 Fields
+
+=head2 id
+
+id of the completion response, used for tracking duplicate responses or reporting issues to the service
+
+=head1 choices
+
+An array of L<Net::Async::xLM::API::Types::Results::CompletionChoices> objects.  If you asked for more than 1 response with the request parameter C<n> then they will be present here.
+
+You likely just want to get ->text from the first result, as demonstrated in the synopsis but see the ::CompletionChoices docs for more detailed information.
+
+=head2 model
+
+The model that was used to generate the response.  Usually will be what you requested,
+but some local inference servers will ignore what was requested and use the model that was
+already loaded, and this will reflect what was loaded.
+
+=head2 created
+
+When was the request completed and returned.
+
+=head2 system_finterprint
+
+Used by OpenAI to identify which system the generation happened on.  Needed for bug repoprts along with the id
+
+=head2 usage
+
+A L<Net::Async::xLM::API::Tupes::Results::Usage> object, has three fields C<total_tokens>, C<prompt_tokens>, and C<completion_tokens>
+
+=head2 object
+
+A string describing what kind of result this was, will always be "completion".
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Results/CompletionChoices.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/CompletionChoices.pod
@ -0,0 +1,53 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::CompletionChoices
+
+=head1 DESCRIPTION
+
+A choice from a completion request, L<Net::Async::xLM::API::Types::Request::Completion> as part of L<OpenAIAsync::Types::Results::Completion>
+
+=head1 SYNOPSIS
+
+  use Net::Async::xLM::API::Client;
+  use IO::Async::Loop;
+
+  my $loop = IO::Async::Loop->new();
+  my $client = Net::Async::xLM::API::Client->new();
+
+  $loop->add($client)
+
+  my $output_future = $client->completion({max_tokens => 1024, prompt => "Tell a story about a princess named Judy and her princess sister Emmy"});
+
+  my $result = $output_future->get();
+
+  print $result->choices->[0]->text;
+
+=head1 Fields
+
+=head2 text
+
+The contents of the response, very likely all you want or need
+
+=head2 index
+
+Index of the choice? I believe this will just always be the same as it's position in the array.
+
+=head2 logprobs
+
+Log probabilities, see L<Net::Async::xLM::API::Types::Results::LogProbs> for details
+
+=head2 finish_reason
+
+What made the model stop generating.  Could be from hitting a stop token, or running into max tokens.
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Types::Results::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Results/Embedding.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/Embedding.pod
@ -0,0 +1,61 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::Completion
+
+=head1 DESCRIPTION
+
+A result from a an embedding request, L<Net::Async::xLM::API::Types::Request::Completion>
+
+=head1 SYNOPSIS
+
+  use IO::Async::Loop;
+  use Net::Async::xLM::API::Client;
+
+  my $loop = IO::Async::Loop->new();
+  my $client = Net::Async::xLM::API::Client->new();
+
+  $loop->add($client);
+
+  my $output = $client->embedding({
+      input => "My hovercraft is full of eels",
+      model => "text-embedding-ada-002",
+      encoding_format => "float"
+  })->get();
+
+  print Dumper($output->data->embedding);
+
+=head1 Fields
+
+=head2 model
+
+The model that was used to generate the response.  Usually will be what you requested,
+but some local inference servers will ignore what was requested and use the model that was
+already loaded, and this will reflect what was loaded.
+
+=head2 data
+
+An C<Net::Async::xLM::API::Types::Results::EmbeddingData> object, used just for this
+
+it has the following fields: C<index>, C<embedding>, C<object>
+
+Of these, you probably only want embedding as it's the list of the numbers representing the embedding vector
+
+=head2 usage
+
+A L<Net::Async::xLM::API::Tupes::Results::Usage> object, has three fields C<total_tokens>, C<prompt_tokens>, and C<completion_tokens>
+
+=head2 object
+
+A string describing what kind of result this was, will always be "compleembeddingtion".
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Results/LogProbs.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/LogProbs.pod
@ -0,0 +1,37 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::LogProbs
+
+=head1 DESCRIPTION
+
+An object representing Log Probabilities from the LLM.
+
+=head1 SYNOPSIS
+
+  ... TODO? difficult to make an example
+
+=head1 Fields
+
+=head2 text_offset
+
+Which position in the resulting text this log probability represents
+
+=head2 token_logprobs
+
+=head2 tokens
+
+=head2 top_logprobss
+
+Not available on my local ai server, will update in next set of changes from how OpenAI implements them
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Types::Result::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Results/Usage.pod
+++ b/lib/Net/Async/xLM/API/Types/Results/Usage.pod
@ -0,0 +1,39 @@
+=pod
+
+=head1 NAME
+
+Net::Async::xLM::API::Types::Results::Usage
+
+=head1 DESCRIPTION
+
+A small report of the tokens given, used, and generated
+
+=head1 SYNOPSIS
+
+  my $result = ...;
+
+  printf "Processed %d tokens in generating response\n", $result->usage->total_tokens
+
+=head1 Fields
+
+=head2 prompt_tokens
+
+How many tokens were in the prompt and initial data
+
+=head2 completion_tokens (optional)
+
+How many tokens were part of the response, may not be present.
+
+=head2 total_tokens
+
+How many total tokens were processed in completing the request.  May also include tokens used as part of an image pipeline or tool call
+
+=head1 SEE ALSO
+
+L<Net::Async::xLM::API::Types::Request::Completion>, L<OpenAIAsync::Client>
+
+=head1 AUTHOR
+
+Ryan Voots ...
+
+=cut
--- a/lib/Net/Async/xLM/API/Types/Shared.pm
+++ b/lib/Net/Async/xLM/API/Types/Shared.pm
@ -0,0 +1,24 @@
+package Net::Async::xLM::API::Types::Shared;
+use v5.36.0;
+use Object::Pad;
+
+use Object::PadX::Role::AutoMarshal;
+use Object::PadX::Role::AutoJSON;
+use Object::Pad::ClassAttr::Struct;
+use Net::Async::xLM::API::Types;
+
+# TODO this is shared request and result?
+# TODO Add a method here that given a file name will create a new object with things filled out
+class Net::Async::xLM::API::Types::Shared::FileObject :Struct {
+  apply Net::Async::xLM::API::Types::Base; 
+  field $id :JSONStr = undef; # Only optional for uploads, but always comes back from the service.  TODO make a check
+  field $bytes :JSONNum;
+  field $created_at :JSONNum;
+  field $filename :JSONStr;
+  field $object :JSONStr = "file"; # Always a file, maybe enforce this in the future
+  field $purpose :JSONStr; # fine-tune, fine-tune-results, assistants, or assistants_output
+  field $status :JSONStr = undef; # DEPRECATED, current status of the file: uploaded, processed, or error
+  field $status_detailts :JSONStr = undef; # DEPRECATED originally used for details of fine-tuning
+}
+
+1;
--- a/lib/OpenAIAsync.pod
+++ b/lib/OpenAIAsync.pod
@ -1,19 +0,0 @@
-=pod
-
-=head1 PURPOSE
-
-There's two big submodules that you'll want to look at:
-
-L<OpenAIAsync::Client> and L<OpenAIAsync::Client::OobaBooga>
-
-There will eventually be a compatible server that uses Net::Async::HTTP::Server that can be used to build a proxy that lets you manipulate or reroute requests, etc.
-
-=head1 LICENSE
-
-Artistic 2.0 - L<Software::License::Artistic_2_0>
-
-=head1 AUTHOR
-
-Ryan Voots TODO add more here
-
-=cut
--- a/lib/OpenAIAsync/Types.pm
+++ b/lib/OpenAIAsync/Types.pm
@ -1,11 +0,0 @@
-package OpenAIAsync::Types;
-use v5.36.0;
-use Object::Pad;
-
-use Object::PadX::Role::AutoMarshal;
-use Object::PadX::Role::AutoJSON;
-use Object::Pad::ClassAttr::Struct;
-
-# Base role for all the types to simplify things later
-role OpenAIAsync::Types::Base :does(Object::PadX::Role::AutoJSON) :does(Object::PadX::Role::AutoMarshal) :Struct {
-}
--- a/lib/OpenAIAsync/Types/Requests.pm
+++ b/lib/OpenAIAsync/Types/Requests.pm
@ -1,210 +0,0 @@
-package OpenAIAsync::Types::Requests;
-use v5.36.0;
-use Object::Pad;
-
-use Object::PadX::Role::AutoMarshal;
-use Object::PadX::Role::AutoJSON;
-use Object::Pad::ClassAttr::Struct;
-use OpenAIAsync::Types;
-
-role OpenAIAsync::Types::Requests::Base :does(OpenAIAsync::Types::Base) :Struct {
-  method _endpoint(); # How the client finds where to send the request
-}
-
-#### Base Request Types
-
-class OpenAIAsync::Types::Requests::ChatCompletion :does(OpenAIAsync::Types::Requests::Base) :Struct {
-  method _endpoint() {"/chat/completions"}
-  field $messages :MarshalTo([OpenAIAsync::Types::Requests::ChatCompletion::Messages::Union]);
-  field $model :JSONStr = "gpt-3.5-turbo";
-  field $frequency_penalty :JSONNum = undef;
-  field $presence_penalty :JSONNum = undef;
-  field $logit_bias = undef; # TODO wtf is this?
-  field $max_tokens :JSONNum = undef;
-  field $response_format :JSONStr :JSONExclude = undef; # I'm not supporting this this version yet
-
-  field $seed :JSONNum = undef;
-  field $stop = undef; # String, array or null, todo handle
-  field $stream :JSONBool = undef; # TODO handle
-  field $temperature :JSONNum = undef;
-  field $top_p :JSONNum = undef;
-  field $tools :JSONExclude = undef; # TODO handle this
-  field $tool_choice :JSONExclude = undef; # TODO handle this
-
-  field $function_call :JSONExclude = undef;
-  field $functions :JSONExclude = undef;
-}
-
-class OpenAIAsync::Types::Requests::Completion :does(OpenAIAsync::Types::Requests::Base) :Struct {
-  method _endpoint() {"/completions"}
-  
-  field $model :JSONStr = "gpt-3.5-turbo"; # This is how 99% of everyone else seems to default this
-  field $prompt :JSONStr;
-  
-  field $max_tokens :JSONNum = undef; # use the platform default usually
-  field $temperature :JSONNum = undef;
-  field $top_p :JSONNum = undef;
-  field $seed :JSONNum  = undef;
-  field $echo :JSONBool = undef; # true or false only
-  field $suffix :JSONStr = undef;
-  field $stop :JSONStr = undef; # array of stop tokens
-  field $user :JSONStr = undef; # used for tracking purposes later
-
-  field $frequency_penalty :JSONNum = undef;
-  field $presence_penalty :JSONNum = undef;
-  
-  field $logit_bias = undef; # TODO make this work
-  field $log_probs = undef; # TODO
-  
-  field $n :JSONNum = undef; # Danger will robinson! easy to cause $$$$$$$ costs
-  field $best_of :JSONNum = undef;
-
-  field $stream :JSONBool = undef; # TODO FALSE ALWAYS RIGHT NOW
-
-  ADJUST {
-    # Type assertions here
-    die "Streaming unsupported" if $self->stream;
-  }
-}
-
-class OpenAIAsync::Types::Requests::Embedding :does(OpenAIAsync::Types::Requests::Base) :Struct {
-  method _endpoint() {"/embeddings"}
-  field $input :JSONStr;
-  field $model :JSONStr;
-  field $encoding_format :JSONStr = undef;
-  field $user :JSONStr = undef;
-}
-
-### Request Subtypes
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant::ToolCall :does(OpenAIAsync::Types::Base) :Struct {
-  field $id :JSONStr;
-  field $arguments :JSONStr;
-  field $type :JSONStr;
-  field $function :MarshalTo(OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall);
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall :does(OpenAIAsync::Types::Base) {
-  field $arguments :JSONStr;
-  field $name :JSONStr;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::Text :does(OpenAIAsync::Types::Base) :Struct {
-  field $type :JSONStr;
-  field $text :JSONStr;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::ImageUrl :does(OpenAIAsync::Types::Base) :Struct {
-  field $url :JSONStr;
-  field $detail :JSONStr = undef;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::Image :does(OpenAIAsync::Types::Base) :Struct {
-  field $type :JSONStr;
-  field $image_url :MarshalTo(OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::ImageUrl);
-}
-
-# TODO, why have two of these? just shove it into the big one below
-
-package 
-    OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::ContentUnion {
-  # This guy does some additional checks to give us the right type here
-  
-  sub new {
-    my $class = shift @_;
-    my %input = @_;
-  
-    die "Missing type in creation" unless $input{type};
-
-    if ($input{type} eq 'text') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::Text->new(%input);
-    } elsif ($input{type} eq 'image_url') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::Image->new(%input);
-    } else {
-      die "Unsupported ChatCompletion User Message type: [".$input{type}."]";
-    }
-  }
-};
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::User :does(OpenAIAsync::Types::Base) :Struct {
-  # This particular type is more complicated than AutoMarshal can handle, so we need to
-  # do this in a custom manner.
-  field $role;
-  field $name = undef;
-  field $content;
-
-  ADJUST {
-    my $create_obj = sub {
-      my $cont = shift;
-
-      if (ref($cont) eq 'HASH') {
-        # We've got a more detailed type here, create the union type here
-        my $obj = OpenAIAsync::Types::Requests::ChatCompletion::Messages::User::ContentUnion->new(%$cont);
-      } elsif (ref($cont) eq '') {
-        return $cont; # Bare string/scalar is fine
-      } else {
-        die "Can't nest other types in \$content of a ChatCompletion user message: ".ref($cont);
-      }
-    };
-
-    if (ref($content) eq 'ARRAY') {
-      $content = [map {$create_obj->($_)} $content->@*];
-    } else {
-      # TODO check that this is acutally doing the right thing.  I think it might not be for user messages that are just text
-      $content = $create_obj->($content);
-    }
-  }
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant :does(OpenAIAsync::Types::Base) :Struct {
-  field $role :JSONStr;
-  field $content :JSONStr;
-  field $name = undef;
-  field $tool_calls :MarshalTo([OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant::ToolCall]) = undef;
-  field $function_call :MarshalTo(OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant::FunctionCall) = undef;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::Function :does(OpenAIAsync::Types::Base) :Struct {
-  field $role :JSONStr;
-  field $content :JSONStr;
-  field $name :JSONStr;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::Tool :does(OpenAIAsync::Types::Base) :Struct {
-  field $role :JSONStr;
-  field $content :JSONStr;
-  field $tool_call_id :JSONStr;
-}
-
-class OpenAIAsync::Types::Requests::ChatCompletion::Messages::System :does(OpenAIAsync::Types::Base) :Struct {
-  field $role :JSONStr;
-  field $name :JSONStr = undef;
-  field $content :JSONStr;
-}
-
-
-package 
-  OpenAIAsync::Types::Requests::ChatCompletion::Messages::Union {
-  # This guy does some additional checks to give us the right type here
-  
-  sub new {
-    my ($class, %input) = @_;
-    die "Missing role in creation" unless $input{role};
-
-    if ($input{role} eq 'system') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::System->new(%input);
-    } elsif ($input{role} eq 'user') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::User->new(%input);
-    } elsif ($input{role} eq 'tool') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::Tool->new(%input);
-    } elsif ($input{role} eq 'function') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::Function->new(%input);
-    } elsif ($input{role} eq 'assistant') {
-      return OpenAIAsync::Types::Requests::ChatCompletion::Messages::Assistant->new(%input);
-    } else {
-      die "Unsupported ChatCompletion Message role: [".$input{role}."]";
-    }
-  }
-};
-
-1;  
--- a/lib/OpenAIAsync/Types/Results.pm
+++ b/lib/OpenAIAsync/Types/Results.pm
@ -1,110 +0,0 @@
-package OpenAIAsync::Types::Results;
-use v5.36.0;
-use Object::Pad;
-
-use OpenAIAsync::Types;
-use Object::PadX::Role::AutoMarshal;
-use Object::PadX::Role::AutoJSON;
-use Object::Pad::ClassAttr::Struct;
-
-class OpenAIAsync::Types::Results::ToolCall :does(OpenAIAsync::Types::Base) :Struct {
-  field $id :JSONStr = undef;
-  field $type :JSONStr = undef; # always "function" right now, may get expanded in the future
-  field $function :MarshalTo(OpenAIAsync::Types::Results::FunctionCall) = undef;
-}
-
-class OpenAIAsync::Types::Results::FunctionCall :does(OpenAIAsync::Types::Base) :Struct {
-  field $arguments :JSONStr = undef; # TODO decode the json from this directly?
-  field $name :JSONStr = undef;
-}
-
-class OpenAIAsync::Types::Results::ChatMessage :does(OpenAIAsync::Types::Base) :Struct {
-  field $content :JSONStr;
-  field $tool_calls :MarshalTo([OpenAIAsync::Types::Results::ToolCall]) = undef; # don't think my local server provides this
-  field $role :JSONStr;
-  field $function_call :MarshalTo(OpenAIAsync::Types::Results::FunctionCall) = undef; # Depcrecated, might still happen
-}
-
-class OpenAIAsync::Types::Results::ChatCompletionChoices :does(OpenAIAsync::Types::Base) :Struct {
-  field $finish_reason :JSONStr;
-  field $index :JSONNum;
-  field $message :MarshalTo(OpenAIAsync::Types::Results::ChatMessage);
-}
-
-class OpenAIAsync::Types::Results::ChatCompletion :does(OpenAIAsync::Types::Base) :Struct {
-  field $id :JSONStr;
-  field $choices :MarshalTo([OpenAIAsync::Types::Results::ChatCompletionChoices]);
-  field $created :JSONStr;
-  field $model :JSONStr;
-  field $system_fingerprint :JSONStr = undef; # My local system doesn't provide this
-  field $usage :MarshalTo(OpenAIAsync::Types::Results::Usage);
-  field $object :JSONStr;
-}
-
-class OpenAIAsync::Types::Results::ChunkDelta :does(OpenAIAsync::Types::Base) :Struct {
-  field $content :JSONStr;
-  field $function_call :MarshalTo(OpenAIAsync::Types::Results::FunctionCall) = undef;
-  field $tool_cass :MarshalTo([OpenAIAsync::Types::Results::ToolCall]) = undef;
-  field $role :JSONStr;
-}
-
-class OpenAIAsync::Types::Results::ChatCompletionChunkChoices :does(OpenAIAsync::Types::Base) :Struct {
-  field $delta :MarshalTo(OpenAIAsync::Types::Results::ChunkDelta);
-  field $finish_reason :JSONStr;
-  field $index :JSONStr;
-}
-
-# This is part of the streaming API
-class OpenAIAsync::Types::Results::ChatCompletionChunk :does(OpenAIAsync::Types::Base) :Struct {
-  field $id :JSONStr;
-  field $choices :MarshalTo(OpenAIAsync::Types::Results::ChatCompletionChunkChoices);
-  field $created :JSONStr;
-  field $model :JSONStr;
-  field $system_fingerprint :JSONStr = undef;
-  field $object :JSONStr;
-}
-
-class OpenAIAsync::Types::Results::Usage :does(OpenAIAsync::Types::Base) :Struct {
-  field $total_tokens :JSONNum;
-  field $prompt_tokens :JSONNum;
-  field $completion_tokens :JSONNum = undef; # look at chat completions, is this the same
-}
-
-class OpenAIAsync::Types::Results::LogProbs :does(OpenAIAsync::Types::Base) :Struct {
-  # TODO what's the representation here?
-  field $text_offset = undef;
-  field $token_logprobs = undef;
-  field $tokens = undef;
-  field $top_logprobs = undef;
-}
-
-class OpenAIAsync::Types::Results::CompletionChoices :does(OpenAIAsync::Types::Base) :Struct {
-  field $text :JSONStr; 
-  field $index :JSONNum;
-  field $logprobs :MarshalTo(OpenAIAsync::Types::Results::LogProbs) = undef; # TODO make nicer type?
-  field $finish_reason :JSONStr = undef; # TODO enum? helper funcs for this class? ->is_finished?
-}
-
-class OpenAIAsync::Types::Results::Completion :does(OpenAIAsync::Types::Base) :Struct {
-  field $id :JSONStr;
-  field $choices :MarshalTo([OpenAIAsync::Types::Results::CompletionChoices]);
-  field $created :JSONStr;
-  field $model :JSONStr;
-  field $system_fingerprint = undef; # my local implementation doesn't provide this, openai does it for tracking changes somehow
-  field $usage :MarshalTo(OpenAIAsync::Types::Results::Usage);
-  field $object :JSONStr;
-}
-
-
-class OpenAIAsync::Types::Results::Embedding :does(OpenAIAsync::Types::Base) :Struct {
-  field $object :JSONStr;
-  field $model :JSONStr;
-  field $usage :MarshalTo(OpenAIAsync::Types::Results::Usage);
-  field $data :MarshalTo([OpenAIAsync::Types::Results::EmbeddingData]);
-}
-
-class OpenAIAsync::Types::Results::EmbeddingData :does(OpenAIAsync::Types::Base) :Struct {
-  field $index :JSONNum;
-  field $embedding :JSONList(JSONNum);
-  field $object :JSONStr;
-}
--- a/t/02-create-client.t
+++ b/t/02-create-client.t
@ -6,6 +6,7 @@ use Test2::V0;
 use OpenAIAsync::Client;

 BEGIN {
+  no warnings 'uninitialized';
  $ENV{OPENAI_API_KEY}="12345" unless $ENV{OPENAI_API_KEY}eq"12345";
 }

@ -37,4 +38,4 @@ ok lives {



-done_testing();
+done_testing();
--- a/t/03-create-server.t
+++ b/t/03-create-server.t
@ -0,0 +1,108 @@
+# test for OpenAIAsync perl module
+use strict;
+use warnings;
+
+# Use the Test2::V0 module for testing
+use Test2::V0;
+
+use OpenAIAsync::Server;
+# Import the OpenAIAsync::Server module
+use OpenAIAsync::Server;
+
+# Use Object::Pad for object-oriented programming
+use Object::Pad;
+
+# Import IO::Async::Loop for managing loops and event loops
+use IO::Async::Loop;
+
+# Use Future::AsyncAwait for easier handling of asynchronous operations
+use Future::AsyncAwait;
+
+# Import JSON::MaybeXS for encoding and decoding JSON data
+use JSON::MaybeXS;
+
+# Import Net::Async::HTTP for asynchronous HTTP requests
+use Net::Async::HTTP;
+
+# Use the relative path './lib' for module lookup
+use lib::relative './lib';
+
+# Create an instance of IO::Async::Loop
+my $loop = IO::Async::Loop->new();
+
+# Set the OPENAI_API_KEY environment variable if it is not set or not equal to "12345"
+BEGIN {
+  no warnings 'uninitialized';
+  $ENV{OPENAI_API_KEY}="12345" unless $ENV{OPENAI_API_KEY}eq"12345";
+  $ENV{OPENAI_API_KEY} = "12345" unless $ENV{OPENAI_API_KEY} eq "12345";
+}
+
+# Define a TestServer class that inherits from OpenAIAsync::Server
+class TestServer {
+  # Inherit methods and properties from OpenAIAsync::Server
+  inherit OpenAIAsync::Server;
+
+  # Apply methods from various OpenAIAsync::Server::API::Test modules
+  apply OpenAIAsync::Server::API::Test::ChatCompletion;
+  apply OpenAIAsync::Server::API::Test::Audio;
+  apply OpenAIAsync::Server::API::Test::Completions;
+  apply OpenAIAsync::Server::API::Test::Embeddings;
+  apply OpenAIAsync::Server::API::Test::File;
+  apply OpenAIAsync::Server::API::Test::Image;
+  apply OpenAIAsync::Server::API::Test::ModelList;
+  apply OpenAIAsync::Server::API::Test::Moderations;
+}
+
+# Pick a random high port, TODO better scheme for this
+my $port = int(2048+rand(20480));
+# Pick a random high port number between 2048 and 22528
+my $port = int(2048 + rand(20480));
+
+# Create an instance of the TestServer class that listens on localhost with the chosen port
+my $server = TestServer->new(listen => '127.0.0.1', port => $port);
+
+# Create an instance of Net::Async::HTTP for making HTTP requests
+my $http_client = Net::Async::HTTP->new();
+
+# Add the $http_client and $server instances to the event loop
+$loop->add($http_client);
+$loop->add($server);
+
+# Define a hash ref for the chat completion request data
+my $chat_completion_input = {
+     "model" => "gpt-3.5-turbo",
+     "messages" => [
+        {"role" => "user", "content" => "Say this is a test!"}
+      ],
+     "temperature" => 0.7
+};
+
+# Subroutine to make an HTTP POST request to the server
+sub mk_req($uri, $content) {
+  my $content_json = encode_json($content);
+  return $http_client->POST("http://127.0.0.1:$port/v1".$uri, $content_json, content_type => 'application/json');
+  my $content_json = encode_json($content);  # Encode the content data as JSON
+  return $http_client->POST("http://127.0.0.1:$port/v1{$uri}", $content_json, content_type => 'application/json');  # Make the POST request
+}
+
+# Make an HTTP POST request to the chat completions endpoint with the chat completion input data
+my $res_fut = mk_req("/chat/completions", $chat_completion_input);
+
+# Delay the loop for 5 seconds to allow the request to complete
+$loop->delay_future(after => 5)->get();
+
+# Get the response object from the future
+my $res = $res_fut->get();
+
+# Extract the response content
+my $content = $res->content;
+is($content, '{"choices":[],"created":"0","id":"24601","model":"GumbyBrain-llm","object":"text_completion","system_fingerprint":"SHODAN node 12 of 16 tertiary adjunct of unimatrix 42","usage":{"completion_tokens":9,"prompt_tokens":6,"total_tokens":42}}', "check marshalling of data directly");
+
+
+
+
+done_testing();
+
+
+# Compare the response content to the expected content and provide a test message
+is($content, '{"choices":[],"created":"0","id":"24601","model":"GumbyBrain-llm","object":"text_completion","
--- a/t/lib/OpenAIAsync/Server/API/Test/Audio.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/Audio.pm
@ -0,0 +1,54 @@
+package OpenAIAsync::Server::API::Test::Audio;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+use OpenAIAsync::Server::API::v1::Audio;
+
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::Audio - Basic audio api role, consumed to implement the OpenAI audio api.  Does not provide an implementation, you are expected to override them in your class
+
+TODO document the subroles here, split up because TTS is much simpler to implement than the others and will be more valuable to support alone if someone chooses
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+
+role OpenAIAsync::Server::API::Test::AudioTTS :strict(params) {
+  apply OpenAIAsync::Server::API::v1::AudioTTS;
+  async method audio_create_speech($future_status, $queue, $ctx, $obj, $params) {...}
+}
+
+role OpenAIAsync::Server::API::Test::AudioSTT :strict(params) {
+  apply OpenAIAsync::Server::API::v1::AudioSTT;
+  async method audio_create_transcript($future_status, $queue, $ctx, $obj, $params) {...}
+}
+
+role OpenAIAsync::Server::API::Test::AudioTranslate :strict(params) {
+  apply OpenAIAsync::Server::API::v1::AudioTranslate;
+  async method audio_create_translation($future_status, $queue, $ctx, $obj, $params) {...}
+}
+
+role OpenAIAsync::Server::API::Test::Audio :strict(params) {
+  apply OpenAIAsync::Server::API::Test::AudioTTS;
+  apply OpenAIAsync::Server::API::Test::AudioSTT;
+  apply OpenAIAsync::Server::API::Test::AudioTranslate;
+}
+
+1;
--- a/t/lib/OpenAIAsync/Server/API/Test/ChatCompletion.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/ChatCompletion.pm
@ -0,0 +1,65 @@
+package OpenAIAsync::Server::API::Test::ChatCompletion;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::ChatCompletion - Basic chat api role, consumed to implement the OpenAI chat completion api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::ChatCompletion :strict(params) {
+  apply OpenAIAsync::Server::API::v1::ChatCompletion;
+  use OpenAIAsync::Types::Results;
+  use Future::AsyncAwait;
+  no warnings 'experimental::builtin';
+  use builtin qw/false/;
+  
+  async method chat_completion ($future_status, $queue, $ctx, $obj, $params) {
+    my $chained_future = $future_status->then(sub {
+      $queue->push(OpenAIAsync::Types::Results::ChatCompletion->new(
+          id => "24601",
+          choices => [],
+          model => "GumbyBrain-llm",
+          system_fingerprint => "SHODAN node 12 of 16 tertiary adjunct of unimatrix 42",
+          usage => {
+            total_tokens => 42,
+            prompt_tokens => 6,
+            completion_tokens => 9,
+          },
+          object => "text_completion",
+          created => 0,
+        ))->get();
+
+        $queue->finish();
+      }
+    );
+
+    $future_status->done({
+      headers => {}, # TODO a way I can make this not leak and not expose the HTTP nature?
+      is_streaming => false,
+      content_type => "application/json", # TODO this should come from the object return type!
+      status_code => 200,
+      status_message => "OK",
+    });
+
+    return $chained_future; # TODO this might actually be wrong thanks to the async above?
+  }
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/Completions.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/Completions.pm
@ -0,0 +1,32 @@
+package OpenAIAsync::Server::API::Test::Completions;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::Completions - Basic completion api role, consumed to implement the OpenAI chat completion api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::Completions :strict(params) {
+  apply OpenAIAsync::Server::API::v1::Completions;
+ 
+  async method completion($future_status, $queue, $ctx, $obj, $params) { ... }
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/Embeddings.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/Embeddings.pm
@ -0,0 +1,32 @@
+package OpenAIAsync::Server::API::Test::Embeddings;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::Embeddings - Basic embeddings api role, consumed to implement the OpenAI embeddings api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::Embeddings {
+  apply OpenAIAsync::Server::API::v1::Embeddings;
+ 
+  async method embeddings($future_status, $queue, $ctx, $obj, $params) { ... }
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/File.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/File.pm
@ -0,0 +1,36 @@
+package OpenAIAsync::Server::API::Test::File;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::File - Basic file api role, consumed to implement the OpenAI file server.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::File :strict(params) {
+  apply OpenAIAsync::Server::API::v1::File;
+
+  async method file_list($future_status, $queue, $ctx, $obj, $params) {...}
+  async method file_info($future_status, $queue, $ctx, $obj, $params) {...}
+  async method file_delete($future_status, $queue, $ctx, $obj, $params) {...}
+  async method file_upload($future_status, $queue, $ctx, $obj, $params) {...}
+  async method file_download($future_status, $queue, $ctx, $obj, $params) {...}
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/Image.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/Image.pm
@ -0,0 +1,32 @@
+package OpenAIAsync::Server::API::Test::Image;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::Image - Basic image role, consumed to implement the OpenAI image api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::Image :strict(params) {
+  apply OpenAIAsync::Server::API::v1::Image;
+
+  async method create_image($future_status, $queue, $ctx, $obj, $params) {...}
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/ModelList.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/ModelList.pm
@ -0,0 +1,32 @@
+package OpenAIAsync::Server::API::Test::ModelList;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::ModelList - Basic model list api role, consumed to implement the OpenAI model list api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::ModelList :strict(params) {
+  apply OpenAIAsync::Server::API::v1::ModelList;
+
+  async method model_list($future_status, $queue, $ctx, $obj, $params) {...}
+}
--- a/t/lib/OpenAIAsync/Server/API/Test/Moderations.pm
+++ b/t/lib/OpenAIAsync/Server/API/Test/Moderations.pm
@ -0,0 +1,32 @@
+package OpenAIAsync::Server::API::Test::Moderations;
+
+use v5.36.0;
+use Object::Pad;
+use IO::Async::SSL; # We're not directly using it but I want to enforce that we pull it in when detecting dependencies, since openai itself is always https
+use Future::AsyncAwait;
+use IO::Async;
+
+use OpenAIAsync::Types::Results;
+use OpenAIAsync::Types::Requests;
+
+our $VERSION = '0.02';
+
+# ABSTRACT: Async server for OpenAI style REST API for various AI systems (LLMs, Images, Video, etc.)
+
+=pod
+
+=head1 NAME
+
+OpenAIAsync::Server::API::Moderations - Basic moderation api role, consumed to implement the OpenAI moderation api.  Does not provide an implementation, you are expected to override them in your class
+
+=head1 SYNOPSIS
+ 
+...
+
+=cut
+
+role OpenAIAsync::Server::API::Test::Moderations :strict(params) {
+  apply OpenAIAsync::Server::API::v1::Moderations;
+  
+  async method moderations($future_status, $queue, $ctx, $obj, $params) {...}
+}
--- a/xtest.sh
+++ b/xtest.sh
@ -3,7 +3,7 @@
 set -euxo pipefail

 export OPENAI_API_KEY="0118 999 881 999 119 725 3"
-export OPENAI_API_BASE="http://openai.general1-model.brainiac.ai.simcop2387.info/v1"
+export OPENAI_API_BASE="http://openai.mixtral1-model.brainiac.ai.simcop2387.info/v1"
 export I_PROMISE_NOT_TO_SUE_FOR_EXCESSIVE_COSTS="Signed, ryan"

 dzil xtest
Author	SHA1	Message	Date
Ryan Voots	1f188e3ceb	Naive regex replacement, needs examining for correctness Some checks failed ci/woodpecker/push/author-tests/1 Pipeline failed Details ci/woodpecker/push/author-tests/4 Pipeline failed Details ci/woodpecker/push/author-tests/2 Pipeline failed Details ci/woodpecker/push/author-tests/3 Pipeline failed Details	2024-12-18 11:11:10 -05:00
Ryan Voots	502433f9a1	beginning rename of things Some checks failed ci/woodpecker/push/author-tests/1 Pipeline failed Details ci/woodpecker/push/author-tests/2 Pipeline failed Details ci/woodpecker/push/author-tests/3 Pipeline failed Details ci/woodpecker/push/author-tests/4 Pipeline failed Details	2024-12-18 11:07:10 -05:00
Ryan Voots	e47e503698	New API design works!	2024-03-06 15:47:17 -05:00
Ryan Voots	4811272f5e	Ok so it now runs, but doesnt exit the test	2024-03-06 15:29:30 -05:00
Ryan Voots	a6fd7dc043	More cleanup but not working quite there	2024-03-06 15:06:59 -05:00
Ryan Voots	afdb8bd5c9	syntax fixes, big errors still	2024-03-06 15:01:37 -05:00
Ryan Voots	f54d3a89e6	Change full parameters of API and first try at tests	2024-03-06 14:58:38 -05:00
Ryan Voots	213278513b	base classes modified, completely new api here and change the handle => to use method names and avoid a useless coderef	2024-03-05 09:33:22 -05:00
Ryan Voots	8f33acb3b9	Server side of this new api setup	2024-03-04 13:43:00 -05:00
Ryan Voots	5651230712	Proper serialization happening now	2024-02-11 13:36:20 -05:00
Ryan Voots	78184a015b	Many things better now, just needs to do some other stuff	2024-01-26 18:48:43 -05:00
Ryan Voots	75a359bd6d	Fix url for chat completions, update tests a bit more	2024-01-26 18:33:49 -05:00
Ryan Voots	70f9db502d	more progress, some known encoding/decoding bugs still, working on it	2024-01-13 09:31:49 -05:00
Ryan Voots	d3b801b5f4	Starting of test of server stuff, async methods using pre-release patch for F:AA	2024-01-09 13:21:58 -05:00
Ryan Voots	30cb348269	serves things up sort of	2024-01-07 10:03:53 -05:00
Ryan Voots	b36235801b	Some debugging, more working Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-04 12:44:14 -05:00
Ryan Voots	70d23486b6	listen starts now, still bugs	2024-01-04 12:31:25 -05:00
Ryan Voots	276248d5dd	Couple of bugs found in Object::Pad and Future::AsyncAwait. see the role changes going on here for what i have to do right now but will report these upstream later. Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-04 12:05:59 -05:00
Ryan Voots	f5be4d9e53	More stuff implemented, and also found a client bug Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-04 11:27:45 -05:00
Ryan Voots	26f5e15da8	Final conversion Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-01 11:33:50 -05:00
Ryan Voots	8c9fe2e808	More conversions Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-01 11:28:32 -05:00
Ryan Voots	8addb512f2	Fixup audio api stuff, use new version of OP Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-01 11:06:09 -05:00
Ryan Voots	cb19e8f166	Upgrade OP version, so we can use apply/inherit	2024-01-01 11:01:49 -05:00
Ryan Voots	ccafe0deae	More image work here Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-01 10:56:51 -05:00
Ryan Voots	1e9a5b1fe2	server side audio apis Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2024-01-01 10:24:20 -05:00
Ryan Voots	3430f315cf	More moderation stuff for server, also fix model list function Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 19:00:18 -05:00
Ryan Voots	bf4f20792e	Moderation api types Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 18:58:23 -05:00
Ryan Voots	4145bc6a63	all existing tests seem to pass, all linting passes now too. need to run with live tests and also write server tests Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:57:12 -05:00
Ryan Voots	fe7fe7fd2e	and this one too Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:55:49 -05:00
Ryan Voots	a438985ea6	rename this for perlcritic	2023-12-31 17:54:58 -05:00
Ryan Voots	dadb53dc81	All syntax fixes done, and known warnings Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:54:28 -05:00
Ryan Voots	4cb031b94b	Client basic tests working again, author tests not Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:34:47 -05:00
Ryan Voots	574ac17d07	Getting there Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:30:31 -05:00
Ryan Voots	1bfa640af2	try to migrate to new ->_encode() method Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:27:45 -05:00
Ryan Voots	fa8829f7f6	More types, now have requests for audio data, which needs a new encoding setup for www-form-encoded stuff. need to change client code slightly to handle this Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-31 17:20:25 -05:00
Ryan Voots	b8f46b0192	More apis in the new format Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-30 16:36:31 -05:00
Ryan Voots	1cc1e3aa4e	Switch to new pattern, makes things easier to work with Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-30 16:22:54 -05:00
Ryan Voots	0570722dc5	More meta about the apis Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-30 14:46:18 -05:00
Ryan Voots	c0272d0f44	Working towards a new design here, makes things easier and saner later Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-30 14:38:22 -05:00
Ryan Voots	3a6e18d204	Add file requests Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-28 15:59:27 -05:00
Ryan Voots	97f88ecdb0	Structure is there, fixed a few design things i made along the way Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-28 13:31:45 -05:00
Ryan Voots	303b3cd37d	Update docs and design a bit, I think most of the things needed are ready and now its just the boilerplate for the server Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-12-21 11:40:56 -05:00
Ryan Voots	68c7f96bf7	First design, already not good	2023-12-12 14:11:37 -05:00
Ryan Voots	635a8a9546	More pod docs Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-27 10:42:44 -05:00
Ryan Voots	f58a113242	Update some more docs Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-27 10:22:16 -05:00
Ryan Voots	c1a08eeb27	I need some tests for log probs to see how to do it properly Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 15:04:52 -05:00
Ryan Voots	950e83017a	Finish off completions direct modules Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 14:51:54 -05:00
Ryan Voots	1304eb119f	embedding is documented Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 14:45:16 -05:00
Ryan Voots	122c4fd5c4	Start of the result type docs Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 14:43:30 -05:00
Ryan Voots	68bfb84348	Some more prune file stuff to clean up the dist more Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 09:30:23 -05:00
Ryan Voots	1d0e963a27	Bump version Some checks failed ci/woodpecker/push/author-tests Pipeline failed Details	2023-11-23 09:05:57 -05:00