382 lines
32 KiB
JSON
382 lines
32 KiB
JSON
{
|
||
"apiVersion": "yingce.plugin/v2",
|
||
"id": "doubao-streaming-tts",
|
||
"name": "豆包音频生成",
|
||
"version": "2.0.0",
|
||
"author": "Volcengine / 影策",
|
||
"description": "豆包音频生成 独立请求协议插件。",
|
||
"documentation": "# 豆包音频生成\n\n该目录是独立官方协议插件源码。后端从生成的 `doubao-streaming-tts.yingce-plugin` 包加载，不依赖系统内置 `host:` 适配器。\n\n完整接口见 [docs/interface.md](docs/interface.md)。\n\n---\n\n# 豆包音频生成 接口字段\n\n## 协议身份\n\n- 插件 ID：`doubao-streaming-tts`。\n- Provider ID：`doubao-streaming-tts`。\n- 能力：`audio`。\n- 默认 Base URL：`https://openspeech.bytedance.com`。\n- 鉴权驱动：`header`。\n- 创建：`POST /api/v3/tts/create`。\n- 生命周期：同步响应。\n\n## 配置字段\n\n| 字段 | 类型 | 必填 | 含义 |\n| --- | --- | --- | --- |\n| `apiKey` | secret | 是 | API Key |\n\n## 统一字段映射\n\n| 统一字段 | 类型 | 必填 | 上游映射 | 说明 |\n| --- | --- | --- | --- | --- |\n| `model` | string | 是 | `model` | 音频模型 ID。 |\n| `prompt` | string | 是 | `input` | 待合成文本。 |\n| `providerOptions` | object | 否 | `provider-specific fields` | 插件命名空间内的厂商扩展字段。 |\n\n## 上游请求模板逐字段清单\n\n下表由插件请求模板生成，覆盖 body、query、headers 和 multipart 文件声明中的每个字段。\n\n| 上游位置 | 值或转换表达式 |\n| --- | --- |\n| `create.method` | `\"POST\"` |\n| `create.path` | `\"/api/v3/tts/create\"` |\n| `create.contentType` | `\"application/json\"` |\n| `create.body.model` | `{\"$ref\":\"request.model\"}` |\n| `create.body.text_prompt` | `{\"$ref\":\"request.prompt\"}` |\n| `create.body.references` | `{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$gt\":[{\"$len\":{\"$ref\":\"request.images\"}},0]},\"then\":{\"$map\":{\"from\":{\"$ref\":\"request.images\"},\"as\":\"media\",\"in\":{\"image_url\":{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"media.source.type\"},\"url\"]},\"then\":{\"$ref\":\"media.url\"},\"else\":null}}},\"image_data\":{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"media.source.type\"},\"data\"]},\"then\":{\"$dataPayload\":{\"$ref\":\"media.value\"}},\"else\":null}}}}}},\"else\":{\"$if\":{\"condition\":{\"$gt\":[{\"$len\":{\"$ref\":\"request.audios\"}},0]},\"then\":{\"$concatArrays\":[{\"$map\":{\"from\":{\"$ref\":\"request.audios\"},\"as\":\"media\",\"in\":{\"audio_url\":{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"media.source.type\"},\"url\"]},\"then\":{\"$ref\":\"media.url\"},\"else\":null}}},\"audio_data\":{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"media.source.type\"},\"data\"]},\"then\":{\"$dataPayload\":{\"$ref\":\"media.value\"}},\"else\":null}}}}}},{\"$if\":{\"condition\":{\"$and\":[{\"$ref\":\"request.extra.audioVoice\"},{\"$lt\":[{\"$len\":{\"$ref\":\"request.audios\"}},3]}]},\"then\":[{\"speaker\":{\"$ref\":\"request.extra.audioVoice\"}}],\"else\":[]}}]},\"else\":{\"$if\":{\"condition\":{\"$ref\":\"request.extra.audioVoice\"},\"then\":[{\"speaker\":{\"$ref\":\"request.extra.audioVoice\"}}],\"else\":null}}}}}}}` |\n| `create.body.audio_config.format` | `{\"$coalesce\":[{\"$ref\":\"request.extra.audioFormat\"},\"mp3\"]}` |\n| `create.body.audio_config.speech_rate` | `{\"$omitEmpty\":{\"$if\":{\"condition\":{\"$ne\":[{\"$toFloat\":{\"$ref\":\"request.extra.audioSpeed\"}},0]},\"then\":{\"$add\":[{\"$multiply\":[{\"$toFloat\":{\"$ref\":\"request.extra.audioSpeed\"}},100]},-100]},\"else\":null}}}` |\n| `create.headers.X-Api-Request-Id` | `{\"$ref\":\"request.extra.idempotencyKey\"}` |\n\n## Provider 扩展键\n\n- 无额外扩展键。\n\n动态模型或工作流允许使用文档声明的完整 `parameters/input/extra_body` 对象；该对象是协议本身的开放 schema，不会被宿主裁剪。\n\n## 响应映射逐字段清单\n\n| 映射位置 | 上游路径或转换表达式 |\n| --- | --- |\n| `response.errorPaths[0]` | `\"code\"` |\n| `response.messagePaths[0]` | `\"message\"` |\n| `response.status` | `\"succeeded\"` |\n| `response.resultKind` | `\"audio\"` |\n| `response.audios` | `{\"$if\":{\"condition\":{\"$ref\":\"response.url\"},\"then\":{\"url\":{\"$ref\":\"response.url\"},\"ephemeral\":true},\"else\":{\"$if\":{\"condition\":{\"$ref\":\"response.audio\"},\"then\":{\"dataUrl\":{\"$ref\":\"response.audio\"},\"mimeType\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"request.extra.audioFormat\"},\"wav\"]},\"then\":\"audio/wav\",\"else\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"request.extra.audioFormat\"},\"pcm\"]},\"then\":\"audio/pcm\",\"else\":{\"$if\":{\"condition\":{\"$eq\":[{\"$ref\":\"request.extra.audioFormat\"},\"ogg_opus\"]},\"then\":\"audio/ogg\",\"else\":\"audio/mpeg\"}}}}}}},\"else\":null}}}}` |\n\n## 响应与错误\n\n插件把上游 task/status/text/media/usage 映射为统一结果。临时媒体 URL 标记为 ephemeral，由宿主立即下载持久化。HTTP 错误、业务 code 和 error object 保持失败语义，不包装成成功。\n\n## 兼容边界\n\n火山引擎 seed-audio-1.0 非流式音频生成。POST /api/v3/tts/create。不传 references 为纯文本生成；连接音频时写入 references 的 audio_url 或 audio_data，最多 3 段，参考音频少于 3 段且指定了音色时再追加一个 speaker；连接图片时写入 image_url 或 image_data，最多 1 张。图片和音频不能同时使用。没有参考素材时，speaker 只在用户明确指定音色时发送，且官方仅接受语音合成 2.0 音色或复刻音色。text_prompt 引用参考音频时使用 @Audio1、@Audio2。响应是单个 JSON，audio 为 Base64，url 两小时过期。\n\n<!-- YINGCE_MANIFEST_CONTRACT_START -->\n## Manifest 完整接口定义\n\n以下 JSON 与插件包内实际 `manifest.json` 逐字段一致，覆盖插件身份、权限、配置、鉴权、参数、校验、创建、Agent、查询、取消、结果下载、响应和 Agent 响应映射。`documentation` 字段的值就是当前完整文档；为避免文档在自身内部无限递归，JSON 中仅用等义占位文本表示正文。\n\n```json\n{\n \"apiVersion\": \"yingce.plugin/v2\",\n \"id\": \"doubao-streaming-tts\",\n \"name\": \"豆包音频生成\",\n \"version\": \"2.0.0\",\n \"author\": \"Volcengine / 影策\",\n \"description\": \"豆包音频生成 独立请求协议插件。\",\n \"documentation\": \"<当前插件的完整 documentation，由 README.md 与 docs/interface.md 拼接而成；为避免 JSON 递归，此处不重复展开正文。>\",\n \"permissions\": [\n \"generation.run\",\n \"media.read\"\n ],\n \"configuration\": {\n \"fields\": [\n {\n \"name\": \"apiKey\",\n \"type\": \"secret\",\n \"label\": \"API Key\",\n \"required\": true\n }\n ]\n },\n \"contributes\": {\n \"providers\": [\n {\n \"id\": \"doubao-streaming-tts\",\n \"label\": \"豆包音频生成\",\n \"capabilities\": [\n \"audio\"\n ],\n \"scopes\": [\n \"admin.system-channel\",\n \"user.custom-channel\",\n \"canvas\",\n \"creation\",\n \"agent\"\n ],\n \"baseUrl\": \"https://openspeech.bytedance.com\",\n \"requiresPublicMediaUrls\": false,\n \"auth\": {\n \"type\": \"header\",\n \"field\": \"apiKey\",\n \"header\": \"X-Api-Key\"\n },\n \"parameters\": [\n {\n \"name\": \"model\",\n \"type\": \"string\",\n \"required\": true,\n \"mapping\": \"model\",\n \"description\": \"音频模型 ID。\"\n },\n {\n \"name\": \"prompt\",\n \"type\": \"string\",\n \"required\": true,\n \"mapping\": \"input\",\n \"description\": \"待合成文本。\"\n },\n {\n \"name\": \"providerOptions\",\n \"type\": \"object\",\n \"required\": false,\n \"mapping\": \"provider-specific fields\",\n \"description\": \"插件命名空间内的厂商扩展字段。\"\n }\n ],\n \"create\": {\n \"method\": \"POST\",\n \"path\": \"/api/v3/tts/create\",\n \"contentType\": \"application/json\",\n \"body\": {\n \"model\": {\n \"$ref\": \"request.model\"\n },\n \"text_prompt\": {\n \"$ref\": \"request.prompt\"\n },\n \"references\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$gt\": [\n {\n \"$len\": {\n \"$ref\": \"request.images\"\n }\n },\n 0\n ]\n },\n \"then\": {\n \"$map\": {\n \"from\": {\n \"$ref\": \"request.images\"\n },\n \"as\": \"media\",\n \"in\": {\n \"image_url\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"media.source.type\"\n },\n \"url\"\n ]\n },\n \"then\": {\n \"$ref\": \"media.url\"\n },\n \"else\": null\n }\n }\n },\n \"image_data\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"media.source.type\"\n },\n \"data\"\n ]\n },\n \"then\": {\n \"$dataPayload\": {\n \"$ref\": \"media.value\"\n }\n },\n \"else\": null\n }\n }\n }\n }\n }\n },\n \"else\": {\n \"$if\": {\n \"condition\": {\n \"$gt\": [\n {\n \"$len\": {\n \"$ref\": \"request.audios\"\n }\n },\n 0\n ]\n },\n \"then\": {\n \"$concatArrays\": [\n {\n \"$map\": {\n \"from\": {\n \"$ref\": \"request.audios\"\n },\n \"as\": \"media\",\n \"in\": {\n \"audio_url\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"media.source.type\"\n },\n \"url\"\n ]\n },\n \"then\": {\n \"$ref\": \"media.url\"\n },\n \"else\": null\n }\n }\n },\n \"audio_data\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"media.source.type\"\n },\n \"data\"\n ]\n },\n \"then\": {\n \"$dataPayload\": {\n \"$ref\": \"media.value\"\n }\n },\n \"else\": null\n }\n }\n }\n }\n }\n },\n {\n \"$if\": {\n \"condition\": {\n \"$and\": [\n {\n \"$ref\": \"request.extra.audioVoice\"\n },\n {\n \"$lt\": [\n {\n \"$len\": {\n \"$ref\": \"request.audios\"\n }\n },\n 3\n ]\n }\n ]\n },\n \"then\": [\n {\n \"speaker\": {\n \"$ref\": \"request.extra.audioVoice\"\n }\n }\n ],\n \"else\": []\n }\n }\n ]\n },\n \"else\": {\n \"$if\": {\n \"condition\": {\n \"$ref\": \"request.extra.audioVoice\"\n },\n \"then\": [\n {\n \"speaker\": {\n \"$ref\": \"request.extra.audioVoice\"\n }\n }\n ],\n \"else\": null\n }\n }\n }\n }\n }\n }\n },\n \"audio_config\": {\n \"format\": {\n \"$coalesce\": [\n {\n \"$ref\": \"request.extra.audioFormat\"\n },\n \"mp3\"\n ]\n },\n \"speech_rate\": {\n \"$omitEmpty\": {\n \"$if\": {\n \"condition\": {\n \"$ne\": [\n {\n \"$toFloat\": {\n \"$ref\": \"request.extra.audioSpeed\"\n }\n },\n 0\n ]\n },\n \"then\": {\n \"$add\": [\n {\n \"$multiply\": [\n {\n \"$toFloat\": {\n \"$ref\": \"request.extra.audioSpeed\"\n }\n },\n 100\n ]\n },\n -100\n ]\n },\n \"else\": null\n }\n }\n }\n }\n },\n \"headers\": {\n \"X-Api-Request-Id\": {\n \"$ref\": \"request.extra.idempotencyKey\"\n }\n }\n },\n \"response\": {\n \"errorPaths\": [\n \"code\"\n ],\n \"messagePaths\": [\n \"message\"\n ],\n \"status\": \"succeeded\",\n \"resultKind\": \"audio\",\n \"audios\": {\n \"$if\": {\n \"condition\": {\n \"$ref\": \"response.url\"\n },\n \"then\": {\n \"url\": {\n \"$ref\": \"response.url\"\n },\n \"ephemeral\": true\n },\n \"else\": {\n \"$if\": {\n \"condition\": {\n \"$ref\": \"response.audio\"\n },\n \"then\": {\n \"dataUrl\": {\n \"$ref\": \"response.audio\"\n },\n \"mimeType\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"request.extra.audioFormat\"\n },\n \"wav\"\n ]\n },\n \"then\": \"audio/wav\",\n \"else\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"request.extra.audioFormat\"\n },\n \"pcm\"\n ]\n },\n \"then\": \"audio/pcm\",\n \"else\": {\n \"$if\": {\n \"condition\": {\n \"$eq\": [\n {\n \"$ref\": \"request.extra.audioFormat\"\n },\n \"ogg_opus\"\n ]\n },\n \"then\": \"audio/ogg\",\n \"else\": \"audio/mpeg\"\n }\n }\n }\n }\n }\n }\n },\n \"else\": null\n }\n }\n }\n }\n }\n }\n ]\n }\n}\n```\n<!-- YINGCE_MANIFEST_CONTRACT_END -->\n",
|
||
"permissions": [
|
||
"generation.run",
|
||
"media.read"
|
||
],
|
||
"configuration": {
|
||
"fields": [
|
||
{
|
||
"name": "apiKey",
|
||
"type": "secret",
|
||
"label": "API Key",
|
||
"required": true
|
||
}
|
||
]
|
||
},
|
||
"contributes": {
|
||
"providers": [
|
||
{
|
||
"id": "doubao-streaming-tts",
|
||
"label": "豆包音频生成",
|
||
"capabilities": [
|
||
"audio"
|
||
],
|
||
"scopes": [
|
||
"admin.system-channel",
|
||
"user.custom-channel",
|
||
"canvas",
|
||
"creation",
|
||
"agent"
|
||
],
|
||
"baseUrl": "https://openspeech.bytedance.com",
|
||
"requiresPublicMediaUrls": false,
|
||
"auth": {
|
||
"type": "header",
|
||
"field": "apiKey",
|
||
"header": "X-Api-Key"
|
||
},
|
||
"parameters": [
|
||
{
|
||
"name": "model",
|
||
"type": "string",
|
||
"required": true,
|
||
"mapping": "model",
|
||
"description": "音频模型 ID。"
|
||
},
|
||
{
|
||
"name": "prompt",
|
||
"type": "string",
|
||
"required": true,
|
||
"mapping": "input",
|
||
"description": "待合成文本。"
|
||
},
|
||
{
|
||
"name": "providerOptions",
|
||
"type": "object",
|
||
"required": false,
|
||
"mapping": "provider-specific fields",
|
||
"description": "插件命名空间内的厂商扩展字段。"
|
||
}
|
||
],
|
||
"create": {
|
||
"method": "POST",
|
||
"path": "/api/v3/tts/create",
|
||
"contentType": "application/json",
|
||
"body": {
|
||
"model": {
|
||
"$ref": "request.model"
|
||
},
|
||
"text_prompt": {
|
||
"$ref": "request.prompt"
|
||
},
|
||
"references": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$gt": [
|
||
{
|
||
"$len": {
|
||
"$ref": "request.images"
|
||
}
|
||
},
|
||
0
|
||
]
|
||
},
|
||
"then": {
|
||
"$map": {
|
||
"from": {
|
||
"$ref": "request.images"
|
||
},
|
||
"as": "media",
|
||
"in": {
|
||
"image_url": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "media.source.type"
|
||
},
|
||
"url"
|
||
]
|
||
},
|
||
"then": {
|
||
"$ref": "media.url"
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
},
|
||
"image_data": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "media.source.type"
|
||
},
|
||
"data"
|
||
]
|
||
},
|
||
"then": {
|
||
"$dataPayload": {
|
||
"$ref": "media.value"
|
||
}
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"else": {
|
||
"$if": {
|
||
"condition": {
|
||
"$gt": [
|
||
{
|
||
"$len": {
|
||
"$ref": "request.audios"
|
||
}
|
||
},
|
||
0
|
||
]
|
||
},
|
||
"then": {
|
||
"$concatArrays": [
|
||
{
|
||
"$map": {
|
||
"from": {
|
||
"$ref": "request.audios"
|
||
},
|
||
"as": "media",
|
||
"in": {
|
||
"audio_url": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "media.source.type"
|
||
},
|
||
"url"
|
||
]
|
||
},
|
||
"then": {
|
||
"$ref": "media.url"
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
},
|
||
"audio_data": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "media.source.type"
|
||
},
|
||
"data"
|
||
]
|
||
},
|
||
"then": {
|
||
"$dataPayload": {
|
||
"$ref": "media.value"
|
||
}
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
{
|
||
"$if": {
|
||
"condition": {
|
||
"$and": [
|
||
{
|
||
"$ref": "request.extra.audioVoice"
|
||
},
|
||
{
|
||
"$lt": [
|
||
{
|
||
"$len": {
|
||
"$ref": "request.audios"
|
||
}
|
||
},
|
||
3
|
||
]
|
||
}
|
||
]
|
||
},
|
||
"then": [
|
||
{
|
||
"speaker": {
|
||
"$ref": "request.extra.audioVoice"
|
||
}
|
||
}
|
||
],
|
||
"else": []
|
||
}
|
||
}
|
||
]
|
||
},
|
||
"else": {
|
||
"$if": {
|
||
"condition": {
|
||
"$ref": "request.extra.audioVoice"
|
||
},
|
||
"then": [
|
||
{
|
||
"speaker": {
|
||
"$ref": "request.extra.audioVoice"
|
||
}
|
||
}
|
||
],
|
||
"else": null
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"audio_config": {
|
||
"format": {
|
||
"$coalesce": [
|
||
{
|
||
"$ref": "request.extra.audioFormat"
|
||
},
|
||
"mp3"
|
||
]
|
||
},
|
||
"speech_rate": {
|
||
"$omitEmpty": {
|
||
"$if": {
|
||
"condition": {
|
||
"$ne": [
|
||
{
|
||
"$toFloat": {
|
||
"$ref": "request.extra.audioSpeed"
|
||
}
|
||
},
|
||
0
|
||
]
|
||
},
|
||
"then": {
|
||
"$add": [
|
||
{
|
||
"$multiply": [
|
||
{
|
||
"$toFloat": {
|
||
"$ref": "request.extra.audioSpeed"
|
||
}
|
||
},
|
||
100
|
||
]
|
||
},
|
||
-100
|
||
]
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"headers": {
|
||
"X-Api-Request-Id": {
|
||
"$ref": "request.extra.idempotencyKey"
|
||
}
|
||
}
|
||
},
|
||
"response": {
|
||
"errorPaths": [
|
||
"code"
|
||
],
|
||
"messagePaths": [
|
||
"message"
|
||
],
|
||
"status": "succeeded",
|
||
"resultKind": "audio",
|
||
"audios": {
|
||
"$if": {
|
||
"condition": {
|
||
"$ref": "response.url"
|
||
},
|
||
"then": {
|
||
"url": {
|
||
"$ref": "response.url"
|
||
},
|
||
"ephemeral": true
|
||
},
|
||
"else": {
|
||
"$if": {
|
||
"condition": {
|
||
"$ref": "response.audio"
|
||
},
|
||
"then": {
|
||
"dataUrl": {
|
||
"$ref": "response.audio"
|
||
},
|
||
"mimeType": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "request.extra.audioFormat"
|
||
},
|
||
"wav"
|
||
]
|
||
},
|
||
"then": "audio/wav",
|
||
"else": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "request.extra.audioFormat"
|
||
},
|
||
"pcm"
|
||
]
|
||
},
|
||
"then": "audio/pcm",
|
||
"else": {
|
||
"$if": {
|
||
"condition": {
|
||
"$eq": [
|
||
{
|
||
"$ref": "request.extra.audioFormat"
|
||
},
|
||
"ogg_opus"
|
||
]
|
||
},
|
||
"then": "audio/ogg",
|
||
"else": "audio/mpeg"
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"else": null
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
]
|
||
}
|
||
}
|