OpenAI Scala 客户端 🤖
这是一个支持所有可用端点和参数的无废话异步 Scala 客户端 包括流式传输,最新的 聊天补全,视觉和语音例程(如这里定义),提供在一个名为 OpenAIService的单一便利服务中。支持的调用有:
- 模型: listModels,和 retrieveModel
- 补全: createCompletion
- 聊天补全: createChatCompletion(也支持 GPT 视觉!),createChatFunCompletion(已弃用),和 createChatToolCompletion
- 编辑: createEdit(已弃用)
- 图像: createImage,createImageEdit,和 createImageVariation
- 嵌入: createEmbeddings
- 批次: createBatch,retrieveBatch,cancelBatch,和 listBatches
- 音频: createAudioTranscription,createAudioTranslation,和 createAudioSpeech
- 文件: listFiles,uploadFile,deleteFile,retrieveFile,和 retrieveFileContent
- 微调: createFineTune,listFineTunes,retrieveFineTune,cancelFineTune,listFineTuneEvents,listFineTuneCheckpoints,和 deleteFineTuneModel
- 审核: createModeration
- 助手: createAssistant,listAssistants,retrieveAssistant,modifyAssistant,和 deleteAssistant
- 线程: createThread,retrieveThread,modifyThread,和 deleteThread
- 线程消息: createThreadMessage,retrieveThreadMessage,modifyThreadMessage,listThreadMessages,retrieveThreadMessageFile,和 listThreadMessageFiles
- 运行(🔥 新): createRun,createThreadAndRun,listRuns,retrieveRun,modifyRun,submitToolOutputs,和 cancelRun
- 运行步骤(🔥 新): listRunSteps,和 retrieveRunStep
- 向量商店(🔥 新): createVectorStore,listVectorStores,retrieveVectorStore,modifyVectorStore,和 deleteVectorStore
- 向量商店文件(🔥 新): createVectorStoreFile,listVectorStoreFiles,retrieveVectorStoreFile,和 deleteVectorStoreFile
- 向量商店文件批次(🔥 新): createVectorStoreFileBatch,retrieveVectorStoreFileBatch,cancelVectorStoreFileBatch,和 listVectorStoreBatchFiles
请注意,为了与 OpenAI API 命名一致,服务函数名称与 API 端点标题/描述完全匹配,采用驼峰命名。
另外,我们的目标是使库尽可能自包含并减少依赖,因此我们最终只使用了两个库 play-ahc-ws-standalone
和 play-ws-standalone-json
(在顶层)。此外,如果需要依赖注入,我们也使用 scala-guice
库。
除了 OpenAI API,此库还支持 API 兼容的提供商,如:
- Azure OpenAI - 基于云,使用 OpenAI 模型但具有更低的延迟
- Azure AI - 基于云,提供广泛的开源模型
- Anthropic - 基于云的 OpenAI 主要竞争对手,拥有专有/闭源模型如 Claude3 - Haiku, Sonnet 和 Opus
- Google Vertex AI(🔥 新) - 基于云,提供专有/闭源模型如 Gemini 1.5 Pro 和 flash
- Groq - 基于云的提供商,以其超快速的 LPU 推理闻名
- Fireworks AI - 基于云的提供商
- OctoAI - 基于云的提供商
- TogetherAI(🔥 新)- 基于云的提供商
- Mistral(🔥 新) - 基于云,领先的开源 LLM 公司
- Ollama - 本地运行,作为包括 LLaMA3, dbrx 和 Command-R 在内开源 LLM 的综合工具
- FastChat - 本地运行,作为包括 Vicuna, Alpaca 和 FastChat-T5 在内开源 LLM 的综合工具
有关更多详情,请参见示例。
👉 有关背景信息,请阅读 Medium 上关于库/客户端的文章。
也请尝试我们的 Pinecone 向量数据库的 Scala 客户端,或者同时使用两个客户端!这个演示项目展示了如何生成和存储 OpenAI 嵌入(使用 text-embedding-ada-002
模型)到 Pinecone 并随后查询它们。OpenAI + Pinecone 组合通常用于自主 AI 代理,如 babyAGI 和 AutoGPT。
✔️ 注意: 这是一个"社区维护"的库,因此与 OpenAI 公司没有关系。
安装 🚀
当前支持的 Scala 版本是 2.12, 2.13,和 3。
要安装此库,请将以下依赖项添加到您的 build.sbt 中
"io.cequence" %% "openai-scala-client" % "1.1.0.RC.1"
或添加到 pom.xml(如果您使用 maven)
<dependency>
<groupId>io.cequence</groupId>
<artifactId>openai-scala-client_2.12</artifactId>
<version>1.1.0.RC.1</version>
</dependency>
如果你需要流媒体支持,请使用 "io.cequence" %% "openai-scala-client-stream" % "1.1.0.RC.1"
。
配置 ⚙️
- 环境变量:
OPENAI_SCALA_CLIENT_API_KEY
和可选的OPENAI_SCALA_CLIENT_ORG_ID
(如果你有的话) - 文件配置(默认): openai-scala-client.conf
使用 👨🎓
I. 获取 OpenAIService
首先你需要提供一个隐式的执行上下文以及 akka 的材质器,例如:
implicit val ec = ExecutionContext.global
implicit val materializer = Materializer(ActorSystem())
然后你可以通过以下方式之一获取服务。
- 默认配置(需要设置环境变量,如配置章节中定义)
val service = OpenAIServiceFactory()
- 自定义配置
val config = ConfigFactory.load("path_to_my_custom_config")
val service = OpenAIServiceFactory(config)
- 无配置
val service = OpenAIServiceFactory(
apiKey = "your_api_key",
orgId = Some("your_org_id") // 如果你有的话
)
- 使用 API Key 进行 Azure 配置
val service = OpenAIServiceFactory.forAzureWithApiKey(
resourceName = "your-resource-name",
deploymentId = "your-deployment-id", // 通常是模型名,例如 "gpt-35-turbo"
apiVersion = "2023-05-15", // 最新版本
apiKey = "your_api_key"
)
- 最小化的
OpenAICoreService
支持listModels
、createCompletion
、createChatCompletion
和createEmbeddings
调用 - 例如由 FastChat 服务在端口 8000 运行
val service = OpenAICoreServiceFactory("http://localhost:8000/v1/")
OpenAIChatCompletionService
仅提供createChatCompletion
val service = OpenAIChatCompletionServiceFactory(
coreUrl = "https://api.groq.com/openai/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("GROQ_API_KEY")}"))
)
- Azure AI - 例如 Cohere R+ 模型
val service = OpenAIChatCompletionServiceFactory.forAzureAI(
endpoint = sys.env("AZURE_AI_COHERE_R_PLUS_ENDPOINT"),
region = sys.env("AZURE_AI_COHERE_R_PLUS_REGION"),
accessToken = sys.env("AZURE_AI_COHERE_R_PLUS_ACCESS_KEY")
)
- Anthropic (需要
openai-scala-anthropic-client
库)
val service = AnthropicServiceFactory.asOpenAI()
val service = OpenAIChatCompletionServiceFactory(
coreUrl = "https://api.fireworks.ai/inference/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("FIREWORKS_API_KEY")}"))
)
val service = OpenAIChatCompletionServiceFactory(
coreUrl = "https://text.octoai.run/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("OCTOAI_TOKEN")}"))
)
val service = OpenAIChatCompletionServiceFactory(
coreUrl = "http://localhost:11434/v1/"
)
- 提供额外流媒体支持的服务 -
createCompletionStreamed
和createChatCompletionStreamed
由 OpenAIStreamedServiceExtra 提供(需要openai-scala-client-stream
库)
import io.cequence.openaiscala.service.StreamedServiceTypes.OpenAIStreamedService
import io.cequence.openaiscala.service.OpenAIStreamedServiceImplicits._
val service: OpenAIStreamedService = OpenAIServiceFactory.withStreaming()
同样对于 chat-completion 服务
import io.cequence.openaiscala.service.OpenAIStreamedServiceImplicits._
val service = OpenAIChatCompletionServiceFactory.withStreaming(
coreUrl = "https://api.fireworks.ai/inference/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("FIREWORKS_API_KEY")}"))
)
或者仅需要流媒体播放时
val service: OpenAIChatCompletionStreamedServiceExtra =
OpenAIChatCompletionStreamedServiceFactory(
coreUrl = "https://api.fireworks.ai/inference/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("FIREWORKS_API_KEY")}"))
)
- 通过依赖注入(需要
openai-scala-guice
库)
class MyClass @Inject() (openAIService: OpenAIService) {...}
II. 调用功能
每个调用及其相应的输入和设置的完整文档在 OpenAIService 提供。由于所有调用都是异步的,它们返回 Future
包装的响应。
🔥 新:有一个新项目 openai-scala-client-examples,你可以在其中找到很多现成的示例!
- 列出模型
service.listModels.map(models =>
models.foreach(println)
)
- 检索模型
service.retrieveModel(ModelId.text_davinci_003).map(model =>
println(model.getOrElse("N/A"))
)
- 创建完成
val text = """Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""".stripMargin
service.createCompletion(text).map(completion =>
println(completion.choices.head.text)
)
- 带有自定义设置的创建完成
val text = """Extract the name and mailing address from this email:
|Dear Kelly,
|It was great to talk to you at the seminar. I thought Jane's talk was quite good.
|Thank you for the book. Here's my address 2111 Ash Lane, Crestview CA 92002
|Best,
|Maya
""".stripMargin
service.createCompletion(
text,
settings = CreateCompletionSettings(
model = ModelId.gpt_3_5_turbo_16k,
max_tokens = Some(1500),
temperature = Some(0.9),
presence_penalty = Some(0.2),
frequency_penalty = Some(0.2)
)
).map(completion =>
println(completion.choices.head.text)
)
- 带有流媒体播放及自定义设置的创建完成
val source = service.createCompletionStreamed(
prompt = "Write me a Shakespeare poem about two cats playing baseball in Russia using at least 2 pages",
settings = CreateCompletionSettings(
model = ModelId.text_davinci_003,
max_tokens = Some(1500),
temperature = Some(0.9),
presence_penalty = Some(0.2),
frequency_penalty = Some(0.2)
)
)
source.map(completion =>
println(completion.choices.head.text)
).runWith(Sink.ignore)
要使其工作,你需要使用 openai-scala-client-stream
库中的 OpenAIServiceStreamedFactory
。
- 创建对话的完成
val createChatCompletionSettings = CreateChatCompletionSettings(
model = ModelId.gpt_3_5_turbo
)
val messages = Seq(
SystemMessage("You are a helpful assistant."),
UserMessage("Who won the world series in 2020?"),
AssistantMessage("The Los Angeles Dodgers won the World Series in 2020."),
UserMessage("Where was it played?"),
)
service.createChatCompletion(
messages = messages,
settings = createChatCompletionSettings
).map { chatCompletion =>
println(chatCompletion.choices.head.message.content)
}
- 为函数创建对话的完成
val messages = Seq(
SystemMessage("You are a helpful assistant."),
UserMessage("What's the weather like in San Francisco, Tokyo, and Paris?")
)
// 作为参数类型我们可以使用 "number"、"string"、"boolean"、"object"、"array" 和 "null"
val tools = Seq(
FunctionSpec(
name = "get_current_weather",
description = Some("Get the current weather in a given location"),
parameters = Map(
"type" -> "object",
"properties" -> Map(
"location" -> Map(
"type" -> "string",
"description" -> "The city and state, e.g. San Francisco, CA"
),
"unit" -> Map(
"type" -> "string",
"enum" -> Seq("celsius", "fahrenheit")
)
),
"required" -> Seq("location")
)
)
)
// 如果我们想强制模型使用上述函数作为响应
// 我们可以通过传递:responseToolChoice = Some("get_current_weather")`来实现
service.createChatToolCompletion(
messages = messages,
tools = tools,
responseToolChoice = None, // 表示 "auto"
settings = CreateChatCompletionSettings(ModelId.gpt_3_5_turbo_1106)
).map { response =>
val chatFunCompletionMessage = response.choices.head.message
val toolCalls = chatFunCompletionMessage.tool_calls.collect {
case (id, x: FunctionCallSpec) => (id, x)
}
println(
"工具调用ID : " + toolCalls.map(_._1).mkString(", ")
)
println(
"函数/工具调用名称 : " + toolCalls.map(_._2.name).mkString(", ")
)
println(
"函数/工具调用参数 : " + toolCalls.map(_._2.arguments).mkString(", ")
)
}
- 🔥 新:在调用
createChatCompletions
或createChatFunCompletions
之前计算预期使用的令牌,这可以帮助你选择合适的模型例如gpt-3.5-turbo
或gpt-3.5-turbo-16k
并降低成本。这是一个实验性功能,可能不适用于所有模型。需要openai-scala-count-tokens
库。
一个如何计算消息令牌的示例:
import io.cequence.openaiscala.service.OpenAICountTokensHelper
import io.cequence.openaiscala.domain.{AssistantMessage, BaseMessage, FunctionSpec, ModelId, SystemMessage, UserMessage}
class MyCompletionService extends OpenAICountTokensHelper {
def exec = {
val model = ModelId.gpt_4_turbo_2024_04_09
// 要发送给 OpenAI 的消息
val messages: Seq[BaseMessage] = Seq(
SystemMessage("You are a helpful assistant."),
UserMessage("Who won the world series in 2020?"),
AssistantMessage("The Los Angeles Dodgers won the World Series in 2020."),
UserMessage("Where was it played?"),
)
val tokenCount = countMessageTokens(model, messages)
}
}
一个在涉及函数时如何计算消息令牌的示例:
import io.cequence.openaiscala.service.OpenAICountTokensHelper
import io.cequence.openaiscala.domain.{BaseMessage, FunctionSpec, ModelId, SystemMessage, UserMessage}
这个是一个英文到中文的翻译,请提供该源文本的中文翻译。
请不要提供任何解释或文本,除了翻译之外。
源文本如下,由XML标签<SOURCE_TEXT></SOURCE_TEXT>分隔。
<SOURCE_TEXT>
class MyCompletionService extends OpenAICountTokensHelper {
def exec = { val model = ModelId.gpt_4_turbo_2024_04_09
// 发送到OpenAI的消息
val messages: Seq[BaseMessage] =
Seq(
SystemMessage("你是一个有帮助的助手。"),
UserMessage("旧金山、东京和巴黎的天气如何?")
)
// 将被调用的函数
val function: FunctionSpec = FunctionSpec(
name = "getWeather",
parameters = Map(
"type" -> "object",
"properties" -> Map(
"location" -> Map(
"type" -> "string",
"description" -> "要获取天气的城市"
),
"unit" -> Map("type" -> "string", "enum" -> List("摄氏度", "华氏度"))
)
)
)
val tokenCount = countFunMessageTokens(model, messages, Seq(function), Some(function.name))
} }
**✔️ 重要**: 使用完服务后,应调用`service.close`将其关闭。否则,底层资源/线程将不会被释放。
---
**三、使用适配器**
为OpenAI服务(聊天完成、核心或全面)提供的适配器由[OpenAIServiceAdapters](./openai-core/src/main/scala/io/cequence/openaiscala/service/adapter/OpenAIServiceAdapters.scala)提供。适配器用于在多个服务之间分配负载、在瞬时错误时重试、路由或提供其他功能。详见[示例](./openai-examples/src/main/scala/io/cequence/openaiscala/examples/adapter)了解更多细节。
注意,适配器可以任意组合/堆叠。
- **循环轮询**负载分配
```scala
val adapters = OpenAIServiceAdapters.forFullService
val service1 = OpenAIServiceFactory("your-api-key1")
val service2 = OpenAIServiceFactory("your-api-key2")
val service = adapters.roundRobin(service1, service2)
- 随机顺序负载分配
val adapters = OpenAIServiceAdapters.forFullService
val service1 = OpenAIServiceFactory("your-api-key1")
val service2 = OpenAIServiceFactory("your-api-key2")
val service = adapters.randomOrder(service1, service2)
- 记录函数调用
val adapters = OpenAIServiceAdapters.forFullService
val rawService = OpenAIServiceFactory()
val service = adapters.log(
rawService,
"openAIService",
logger.log
)
- 重试瞬时错误(例如速率限制错误)
val adapters = OpenAIServiceAdapters.forFullService
implicit val retrySettings: RetrySettings = RetrySettings(maxRetries = 10).constantInterval(10.seconds)
val service = adapters.retry(
OpenAIServiceFactory(),
Some(println(_)) // 简单日志记录
)
- 重试特定函数使用RetryHelpers直接
class MyCompletionService @Inject() (
val actorSystem: ActorSystem,
implicit val ec: ExecutionContext,
implicit val scheduler: Scheduler
)(val apiKey: String)
extends RetryHelpers {
val service: OpenAIService = OpenAIServiceFactory(apiKey)
implicit val retrySettings: RetrySettings =
RetrySettings(interval = 10.seconds)
def ask(prompt: String): Future[String] =
for {
completion <- service
.createChatCompletion(
List(MessageSpec(ChatRole.User, prompt))
)
.retryOnFailure
} yield completion.choices.head.message.content
}
- 路由基于模型的聊天完成调用
val adapters = OpenAIServiceAdapters.forFullService
// OctoAI
val octoMLService = OpenAIChatCompletionServiceFactory(
coreUrl = "https://text.octoai.run/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("OCTOAI_TOKEN")}"))
)
// Anthropic
val anthropicService = AnthropicServiceFactory.asOpenAI()
// OpenAI
val openAIService = OpenAIServiceFactory()
val service: OpenAIService =
adapters.chatCompletionRouter(
// OpenAI服务是默认的所以无需在此指定其模型
serviceModels = Map(
octoMLService -> Seq(NonOpenAIModelId.mixtral_8x22b_instruct),
anthropicService -> Seq(
NonOpenAIModelId.claude_2_1,
NonOpenAIModelId.claude_3_opus_20240229,
NonOpenAIModelId.claude_3_haiku_20240307
)
),
openAIService
)
- 聊天到完成适配器
val adapters = OpenAIServiceAdapters.forCoreService
val service = adapters.chatToCompletion(
OpenAICoreServiceFactory(
coreUrl = "https://api.fireworks.ai/inference/v1/",
authHeaders = Seq(("Authorization", s"Bearer ${sys.env("FIREWORKS_API_KEY")}"))
)
)
常见问题 🤔
-
Wen Scala 3?
2023年2月。你说得对;我们选择了最短的一个月份来完成 :)已完成! -
我遇到了超时异常。如何更改超时设置?
你可以通过将
timeouts
参数传递给OpenAIServiceFactory
来更改,也可以通过在配置文件中添加如下内容:
openai-scala-client {
timeouts {
requestTimeoutSec = 200
readTimeoutSec = 200
connectTimeoutSec = 5
pooledConnectionIdleTimeoutSec = 60
}
}
-
我遇到了一个异常,类似于
com.typesafe.config.ConfigException$UnresolvedSubstitution: openai-scala-client.conf @ jar:file:.../io/cequence/openai-scala-client_2.13/0.0.1/openai-scala-client_2.13-0.0.1.jar!/openai-scala-client.conf: 4: Could not resolve substitution to a value: ${OPENAI_SCALA_CLIENT_API_KEY}
。我该怎么办?设置环境变量
OPENAI_SCALA_CLIENT_API_KEY
。如果没有,可以在这里注册。 -
一切看起来很酷。我想与你讨论你的研究和开发?
发送电子邮件到openai-scala-client@cequence.io联系我们。
许可证 ⚖️
根据MIT许可证的条款,此库作为开源软件发布。
贡献者 🙏
这个项目是开源的,欢迎任何贡献或反馈(这里)。
此库的开发得到了 - Cequence.io - '合同的未来' 的支持和维护。
由Peter Banda创建和维护。 </SOURCE_TEXT>