OpenAI
Based on OpenAI-DotNet
A OpenAI package for the Unity to use though their RESTful API. Independently developed, this is not an official library and I am not affiliated with OpenAI. An OpenAI API account is required.
All copyrights, trademarks, logos, and assets are the property of their respective owners.
This repository is available to transfer to the OpenAI organization if they so choose to accept it.
Installing
Requires Unity 2021.3 LTS or higher.
The recommended installation method is though the unity package manager and OpenUPM.
Via Unity Package Manager and OpenUPM
- Open your Unity project settings
- Add the OpenUPM package registry:
- Name:
OpenUPM
- URL:
https://package.openupm.com
- Scope(s):
com.openai
com.utilities
- Name:
- Open the Unity Package Manager window
- Change the Registry from Unity to
My Registries
- Add the
OpenAI
package
Via Unity Package Manager and Git url
- Open your Unity Package Manager
- Add package from git url:
https://github.com/RageAgainstThePixel/com.openai.unity.git#upm
Note: this repo has dependencies on other repositories! You are responsible for adding these on your own.
Documentation
Check out our new api docs!
https://rageagainstthepixel.github.io/OpenAI-DotNet :new:
Table of Contents
- Authentication
- Azure OpenAI
- OpenAI API Proxy
- Models
- Assistants :warning: :construction:
- List Assistants
- Create Assistant
- Retrieve Assistant
- Modify Assistant
- Delete Assistant
- Assistant Streaming :warning: :construction:
- Threads :warning: :construction:
- Create Thread
- Create Thread and Run
- Streaming :warning: :construction:
- Retrieve Thread
- Modify Thread
- Delete Thread
- Thread Messages
- Thread Runs
- List Runs
- Create Run
- Streaming :warning: :construction:
- Retrieve Run
- Modify Run
- Submit Tool Outputs to Run
- List Run Steps
- Retrieve Run Step
- Cancel Run
- Vector Stores
- Chat
- Audio
- Images :warning: :construction:
- Files
- Fine Tuning
- Batches
- Embeddings
- Moderations
Authentication
There are 4 ways to provide your API keys, in order of precedence:
[!WARNING] We recommended using the environment variables to load the API key instead of having it hard coded in your source. It is not recommended use this method in production, but only for accepting user credentials, local testing and quick start scenarios.
- Pass keys directly with constructor :warning:
- Unity Scriptable Object :warning:
- Load key from configuration file
- Use System Environment Variables
You use the OpenAIAuthentication
when you initialize the API as shown:
Pass keys directly with constructor
[!WARNING] We recommended using the environment variables to load the API key instead of having it hard coded in your source. It is not recommended use this method in production, but only for accepting user credentials, local testing and quick start scenarios.
var api = new OpenAIClient("sk-apiKey");
Or create a OpenAIAuthentication
object manually
var api = new OpenAIClient(new OpenAIAuthentication("sk-apiKey", "org-yourOrganizationId", "proj_yourProjectId"));
Unity Scriptable Object
You can save the key directly into a scriptable object that is located in the Assets/Resources
folder.
You can create a new one by using the context menu of the project pane and creating a new OpenAIConfiguration
scriptable object.
[!WARNING] Beware checking this file into source control, as other people will be able to see your API key. It is recommended to use the OpenAI-DotNet-Proxy and authenticate users with your preferred OAuth provider.
Load key from configuration file
Attempts to load api keys from a configuration file, by default .openai
in the current directory, optionally traversing up the directory tree or in the user's home directory.
To create a configuration file, create a new text file named .openai
and containing the line:
[!NOTE] Organization and project id entries are optional.
Json format
{
"apiKey": "sk-aaaabbbbbccccddddd",
"organizationId": "org-yourOrganizationId",
"projectId": "proj_yourProjectId"
}
Deprecated format
OPENAI_API_KEY=sk-aaaabbbbbccccddddd
OPENAI_ORGANIZATION_ID=org-yourOrganizationId
OPENAI_PROJECT_ID=proj_yourProjectId
You can also load the configuration file directly with known path by calling static methods in OpenAIAuthentication
:
- Loads the default
.openai
config in the specified directory:
var api = new OpenAIClient(new OpenAIAuthentication().LoadFromDirectory("path/to/your/directory"));
- Loads the configuration file from a specific path. File does not need to be named
.openai
as long as it conforms to the json format:
var api = new OpenAIClient(new OpenAIAuthentication().LoadFromPath("path/to/your/file.json"));
Use System Environment Variables
Use your system's environment variables specify an api key and organization to use.
- Use
OPENAI_API_KEY
for your api key. - Use
OPENAI_ORGANIZATION_ID
to specify an organization. - Use
OPENAI_PROJECT_ID
to specify a project.
var api = new OpenAIClient(new OpenAIAuthentication().LoadFromEnvironment());
Azure OpenAI
You can also choose to use Microsoft's Azure OpenAI deployments as well.
You can find the required information in the Azure Playground by clicking the View Code
button and view a URL like this:
https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/chat/completions?api-version={api-version}
your-resource-name
The name of your Azure OpenAI Resource.deployment-id
The deployment name you chose when you deployed the model.api-version
The API version to use for this operation. This follows the YYYY-MM-DD format.
To setup the client to use your deployment, you'll need to pass in OpenAISettings
into the client constructor.
var auth = new OpenAIAuthentication("sk-apiKey");
var settings = new OpenAISettings(resourceName: "your-resource-name", deploymentId: "deployment-id", apiVersion: "api-version");
var api = new OpenAIClient(auth, settings);
Azure Active Directory Authentication
Authenticate with MSAL as usual and get access token, then use the access token when creating your OpenAIAuthentication
. Then be sure to set useAzureActiveDirectory to true when creating your OpenAISettings
.
Tutorial: Desktop app that calls web APIs: Acquire a token
// get your access token using any of the MSAL methods
var accessToken = result.AccessToken;
var auth = new OpenAIAuthentication(accessToken);
var settings = new OpenAISettings(resourceName: "your-resource", deploymentId: "deployment-id", apiVersion: "api-version", useActiveDirectoryAuthentication: true);
var api = new OpenAIClient(auth, settings);
OpenAI API Proxy
Using either the OpenAI-DotNet or com.openai.unity packages directly in your front-end app may expose your API keys and other sensitive information. To mitigate this risk, it is recommended to set up an intermediate API that makes requests to OpenAI on behalf of your front-end app. This library can be utilized for both front-end and intermediary host configurations, ensuring secure communication with the OpenAI API.
Front End Example
In the front end example, you will need to securely authenticate your users using your preferred OAuth provider. Once the user is authenticated, exchange your custom auth token with your API key on the backend.
Follow these steps:
- Setup a new project using either the OpenAI-DotNet or com.openai.unity packages.
- Authenticate users with your OAuth provider.
- After successful authentication, create a new
OpenAIAuthentication
object and pass in the custom token with the prefixsess-
. - Create a new
OpenAISettings
object and specify the domain where your intermediate API is located. - Pass your new
auth
andsettings
objects to theOpenAIClient
constructor when you create the client instance.
Here's an example of how to set up the front end:
var authToken = await LoginAsync();
var auth = new OpenAIAuthentication($"sess-{authToken}");
var settings = new OpenAISettings(domain: "api.your-custom-domain.com");
var api = new OpenAIClient(auth, settings);
This setup allows your front end application to securely communicate with your backend that will be using the OpenAI-DotNet-Proxy, which then forwards requests to the OpenAI API. This ensures that your OpenAI API keys and other sensitive information remain secure throughout the process.
Back End Example
In this example, we demonstrate how to set up and use OpenAIProxy
in a new ASP.NET Core web app. The proxy server will handle authentication and forward requests to the OpenAI API, ensuring that your API keys and other sensitive information remain secure.
- Create a new ASP.NET Core minimal web API project.
- Add the OpenAI-DotNet nuget package to your project.
- Powershell install:
Install-Package OpenAI-DotNet-Proxy
- Manually editing .csproj:
<PackageReference Include="OpenAI-DotNet-Proxy" />
- Powershell install:
- Create a new class that inherits from
AbstractAuthenticationFilter
and override theValidateAuthentication
method. This will implement theIAuthenticationFilter
that you will use to check user session token against your internal server. - In
Program.cs
, create a new proxy web application by callingOpenAIProxy.CreateWebApplication
method, passing your customAuthenticationFilter
as a type argument. - Create
OpenAIAuthentication
andOpenAIClientSettings
as you would normally with your API keys, org id, or Azure settings.
public partial class Program
{
private class AuthenticationFilter : AbstractAuthenticationFilter
{
public override void ValidateAuthentication(IHeaderDictionary request)
{
// You will need to implement your own class to properly test
// custom issued tokens you've setup for your end users.
if (!request.Authorization.ToString().Contains(TestUserToken))
{
throw new AuthenticationException("User is not authorized");
}
}
public override async Task ValidateAuthenticationAsync(IHeaderDictionary request)
{
await Task.CompletedTask; // remote resource call