feat(editor): AI 编辑助手悬浮面板、插件权限体系与时间线渲染管线(M4–M6.3) (#411)
* feat(editor): 编辑器实施 M0-M1 - 剪辑入口占位与插件 SDK v2 垂直切片 - detail.tsx 新增剪辑成片视图(editor tab,常驻),概览页插入剪辑步骤卡 - plugin-types.ts 新增 v2 类型(EditorSlotKind/EditorPluginPermission/PluginManifestV2),修复权限数组交叉归约 bug - editor-slot-registry.ts 插槽注册表(优先级排序、幂等注册、订阅机制) - plugin-registry.ts 注册器 v1/v2 分支校验,v1 路径零改动 - editor-shell.tsx 垂直切片插件:timeline-panel 插槽真实渲染,验证插件链路全通 - 测试:插槽注册表 5 用例 + 命令测试骨架,v1 插件回归 21 例零回归 * feat(editor): 命令协议黄金文件 - editor-commands 纯函数 reducer + 8 个黄金 op * feat(editor): 快照撤销历史 - editor-history 有界 200 层 undo/redo * feat(editor): 命令状态机 store - dispatch/手势预览/1.5s 防抖保存/串行保存链 * fix(editor): createEditorStore 返回 zustand hook 类型修复 typecheck * feat(editor): 手势 echo 最小件 - 时间线面板拖拽移动/裁剪 + store 接线与本地持久化 * docs(editor): M2 完成状态记录 * feat(editor): M3.1 时间线面板选中联动 - store 选中状态 + 片段高亮/点击取消 * feat(editor): M3.2 预览监视器 - preview-renderer 插槽 + 真实媒体帧/播放头/时间码 * feat(editor): M3.3 片段检查器 - inspector 插槽 + 属性编辑入队可撤销 * feat(editor): M3.4 asset-ingest 素材库插件(addClip 命令入轨) * feat(editor): M3.4 补充 - host 上下文(项目信息+资产)、editor.tsx 接线、asset-ingest 纯函数、死代码清理 * feat(editor): M3.5 subtitle-tools - SRT import/export + rebuild from node snapshot * feat(editor): M3.6 transcription - deterministic mock ASR to subtitle track * feat(editor): M3.7 export - render plan + ffmpeg.wasm fallback export * feat(editor): M3.8 - render all 8 editor slots in three-pane layout + ai-assistant placeholder * docs(editor): M3 完成状态 - 8 插件 + 全插槽接线,冒烟待确认 * feat(editor): 编辑器媒体与时间线打磨 - 资产来源分组筛选、时间线全宽布局、拖拽末片段修复、OSS 降级本地存储、Concat 风格控制条与检查器 * fix(plugins): 合并上游后修复插件分类筛选对 v2 清单的类型兼容 * feat(editor): 时间线工具态与轨道操作打磨 - 工具按钮选中仅图标变色、监视器 Concat 式控制条、removeTrack 黄金命令与轨道行头移除按钮、maskClosable 迁移 * fix(api): 资源上传超 MaxBytesReader 上限时报中文大小限制而非英文裸 400 * fix(editor): 监视器控制条 - 修复播放按钮 JSX 模板闭合花括号缺失导致的编译错误 * feat(api): 本地媒体导入分片化对齐 Concat - >50MB 视频不再失败 - 分片会话三接口(POST开始/PUT单片/complete合并)、前端超50MB自动切8MB分片并失败整传重试、默认配额放宽(日200MB→2GB、总2GB→20GB)、UploadResourceFile 豁免单文件限 * feat(editor): 轨道行头新增可见性/静音开关按钮 - 复用现有图标资源,按轨道类型显隐;命令层新增 setTrackFlag 黄金命令(flag 白名单 fail-closed)并补单测 * docs(progress): 记录轨道可见/静音开关改动与复测项 * docs(editor): M4 转写与渲染任务设计 - 复用 Task 表与状态机、独立创建 API、worker 领取类型过滤、whisper HTTP provider(OpenAI 兼容 /v1/audio) * fix(api): OSS 上传失败自动降级本地存储时 body 已被 Transport 关闭 - newOSSRequest 用 NopCloser 包装请求体,服务端提前返回(签名 403)时不再关闭调用方持有的文件,降级分支 Seek 重读生效;补回归测试 * docs(progress): 记录媒体导入根治(分片+OSS 降级修复)复测项 * feat(api): 视频播放副本转码 H.265→H.264 + variant=playback 服务与启动回填 - 上传视频后异步探测 codec,H.265/AV1 走 moov 快速探测并转码 H.264 播放副本;GET /api/user-data/...?variant=playback 就绪时服务副本(独立 ETag :pb),未就绪回退原件并撤销后缀防 304 黑屏;storeResource 触发、启动幂等回填存量;前端播放失败自动轮询副本就绪后刷新 * fix(editor): 轨道至少保留一条 - 不可移除轨道点击 X 时居中浮现原因提示(2.2s 自动消失, aria-live);修正可见性 toggle 传参 (visible===false 语义) ;轨道行内移除按钮改为常驻(不可移除时点击给提示) * feat(timeline): M4.1 字幕转写后台任务 - whisper.cpp 本地执行 - 新增 timeline_transcription 任务类型,worker 按类型分叉独立执行器(不走计费与模型渠道);输入 resourceId+language,读取用户资源后 ffmpeg 预处理 16k 单声道 wav,POST 本地 whisper.cpp /inference,结果写回 segments+srt+language;未配置 CANVAS_WHISPER_BASE_URL 等失败均落明确终态;feature 门控;进度分阶段;含解析/SRT/快速失败单元测试 * feat(media): 素材卡片时长与媒体面板打磨 - 导入媒体解析时长(前端 media-metadata + 后端 project_asset durationMs 透出)随卡片展示;媒体面板/监视器/时间线相关样式细节收尾 * docs(progress): 阶段 C 监视器播放复检清单 - H.265 播放副本/真实播放驱动/时长链路待浏览器确认 * feat(timeline): M4.1 转写任务创建 API - POST /api/timeline/transcriptions - service CreateTimelineTranscriptionTask:IsDraining 守护 + feature 门控 + 资源归属与可转写类型校验 + 排队计 active 限额(含任务级/项目级存储估算);local/whisper.cpp 标记,任务不经过模型路由与计费;createTaskWithinStorageQuota 复用与 ErrActiveTaskLimit 用户可读映射;测试覆盖排队成功(owner/type/provider 断言)、跨用户资源拒绝、非音视频拒绝 * feat(timeline): M4.3 字幕转写前端接线 - 真实任务替换 mock - 新增 services/api/timeline-tasks.ts(POST /timeline/transcriptions,复用 apiClient+request);EditorTranscription 改为 创建任务→waitForGenerationTask 轮询(25min)→结果 segments 映射 SrtEntry[]→rebuildSubtitleClips 写入字幕轨道;实时展示 stage/progress,失败/空识别/无字幕轨道分别提示;素材选择支持音频与视频(取 storageKey 中的 resource id 提交) * fix(media): 播放副本状态收敛与轮询终态 - 无ffmpeg/远端存储/编码不可处理落 none 避免重复探测;前端 none 停轮询回原生播放并加 processing 上限 * feat(timeline): M4.2 时间线渲染任务 - POST /api/timeline/renders 创建(buildRenderPlan 校验含可渲染媒体、配额入队、local/ffmpeg、60min 超时);worker 顶部按类型分叉 processTimelineRender;ffmpeg 合成产物写资源存储并经 ResultJSON 记 resourceId;render 计划/结果测试 PASS * fix(resource): 转码原子抢占与 OSS 请求体加固 - ClaimPlaybackTranscode 空/none→processing 单实例抢占避免重复转码,runPlaybackTranscode defer recover 落 failed;If-Range 与带 :pb 后缀的 serveETag 比较;无请求体的 OSS 请求不再包 NopCloser(nil)(防 Go 1.26 body 探测 nil 解引用) * style(editor): 修正字幕回写面板缩进 * docs(timeline): M4 设计文档按实现回写 - 单一 worker 池 + processClaimedTask 顶部按类型分叉(决策 3/4),端点统一为复数 /api/timeline/transcriptions 与 /api/timeline/renders(注册于 RegisterTaskRoutes),ResultJSON/feature gate/ffmpeg env 同步;Runbook M4 表格与状态行更新;auth.go API 前缀白名单补 timeline * fix(service): 修复上游 v1.2.5 公告配图删除保护自锁与测试夹具缺口 上游 #385 资源引用走查器对公告配图/草稿做了硬性引用检查,导致三个问题: - UpdateAnnouncement 在更新公告前对旧配图走查,必然命中当前公告自身引用而自锁, 公告替换配图功能实际不可用;DiscardAnnouncementImage 同理被草稿自身引用阻塞。 修复:ensureResourceHasNoBusinessReferences 支持忽略指定引用(当前公告/当前草稿), 其余引用仍按原语义拦截。 - admin DeleteAdminResources 将公告草稿引用视为阻塞,但删除事务会级联清理草稿与 资源绑定,service 层永不进入事务,回滚审计注入测试无法到达预期路径。修复:service 预检时过滤公告草稿引用,与 repository 级联语义对齐。 - project_settings_test.go 夹具未建 ResourceReferenceSnapshot 走查的 Task/Session/ Message/TaskLog/Result/Announcement/AnnouncementImageDraft 表,报 no such table。 修复:夹具 AutoMigrate 补齐。 后端全量 go test ./... 通过。 * feat(editor): 成片导出接入后端渲染任务(M4.2 前端接线)- timeline-tasks 增 createTimelineRenderTask;editor-export 主路径改 POST /timeline/renders 轮询取 resourceId 内嵌预览/下载,ffmpeg.wasm 降级保留 * feat(editor): M5 插件权限执行(fail-closed)- plugin-permission-check 新建(8 插槽最小权限表/判定/断言);SlotStack 渲染前过滤缺权限插槽改诊断条;editor-shell manifest 补 timeline.command/export.run;测试 10 用例含 editor-shell 真实回归 * feat(editor): M6.1 AI 命令 schema + M6.2 时间线摘要 - ai-command-schema(parse/dry-run 校验/12-op 提示词契约/版本化+golden);timeline-summary(确定性中文摘要/折叠上限);各 20/10 用例全绿 * docs(progress): M6 质检清单 - M6.1 schema/M6.2 摘要标记完成(含测试计数),M6.3 待模型源确认;清理 M5.1 重复条目 * feat(editor): AI 命令 schema 支持只读问答 - commands 空数组合法化(无法执行/纯提问时 reasoning 直接作答),新增 AI_EDITING_MAX_COMMANDS=8 上限防失控编辑,提示词同步 * feat(editor): AI 助手重构为右上悬浮对话面板(M6.3)+ 素材入库原子化收尾 - editor-ai-assistant.tsx: 占位面板替换为 OpenAI Canvas 风格右上悬浮对话面板; 面板内文本模型选择器(effective.textModels,选择即持久化并覆写请求模型), 消息流 user/assistant/error/preview 四类气泡 + 思考中占位,空态引导示例 chips 一键发送,底部输入条;顶部工具栏常驻 Sparkles 入口(aria-pressed 切换)。 修复上一版重复/截断函数声明(appendToEntry 双定义、settleAnswer 三声明等)。 宿主链路:buildAiEditingSystemPrompt(summarizeTimeline(project)) 喂上下文 → parseAiCommandPlan → validateAiCommandBatch dry-run(任一非法整批拒绝)→ ≤3 条 dispatch 直执行 / >3 条预览确认卡片;宿主拒绝时 failTurn 回填 FAILED_MARK 可重试。 - editor-asset-ingest/editor-preview-monitor/detail/editor.tsx: 配套接线与打磨 (编辑器顶栏 AI 入口、素材导入面板、预览监视、项目页顶栏整理)。 - backend: LinkProjectAsset 事务内原子创建资产 + 项目链接(OnConflict 幂等), 资产首次入库不再预落库,失败整体回滚,消除"有资产无链接"孤儿记录。 - docs: pending-test.mdx / runbook M6.3 状态更新(tsc + 测试 + 构建通过, 浏览器冒烟待真实文本渠道)。
This commit is contained in:
1 parent
33c589fd8e
commit
05d1bce876
106 files changed
+10857
-119
No files matched your search
@@ -69,7 +69,18 @@ func (s *Service) DeleteAdminResources(actor *model.User, req AdminResourceDelet
|
||||
return nil, snapshotErr
|
||||
}
|
||||
for resourceID, references := range adminResourceReferences(snapshot, userResources) {
|
||||
blockedByID[resourceID] = AdminResourceDeleteBlocked{ID: resourceID, Reason: "资源仍被业务数据引用", References: references}
|
||||
blocking := make([]AdminResourceReferenceView, 0, len(references))
|
||||
for _, reference := range references {
|
||||
if reference.Kind == "公告草稿" {
|
||||
// 公告草稿及其资源绑定在删除事务内级联清理,不构成阻塞引用。
|
||||
continue
|
||||
}
|
||||
blocking = append(blocking, reference)
|
||||
}
|
||||
if len(blocking) == 0 {
|
||||
continue
|
||||
}
|
||||
blockedByID[resourceID] = AdminResourceDeleteBlocked{ID: resourceID, Reason: "资源仍被业务数据引用", References: blocking}
|
||||
}
|
||||
}
|
||||
announcementReferences, err := s.repo.AnnouncementResourceReferences(resourceIDs)
|
||||
|
||||
@@ -125,7 +125,7 @@ func (s *Service) UpdateAnnouncement(actor *model.User, id string, req UpdateAnn
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if err := s.ensureResourceHasNoBusinessReferences(oldResource); err != nil {
|
||||
if err := s.ensureResourceHasNoBusinessReferences(oldResource, repository.ResourceDirectReference{Kind: "公告", ID: announcement.ID}); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sharedCount, countErr := s.repo.ResourceStorageReferenceCount(oldResource, []string{oldResource.ID})
|
||||
@@ -316,7 +316,7 @@ func (s *Service) discardAnnouncementImageDraft(userID string, resourceID string
|
||||
}
|
||||
return err
|
||||
}
|
||||
if err := s.ensureResourceHasNoBusinessReferences(resource); err != nil {
|
||||
if err := s.ensureResourceHasNoBusinessReferences(resource, repository.ResourceDirectReference{Kind: "公告草稿", ID: resource.ID}); err != nil {
|
||||
return err
|
||||
}
|
||||
sharedCount, err := s.repo.ResourceStorageReferenceCount(resource, []string{resource.ID})
|
||||
@@ -340,7 +340,7 @@ func (s *Service) discardAnnouncementImageDraft(userID string, resourceID string
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Service) ensureResourceHasNoBusinessReferences(resource *model.Resource) error {
|
||||
func (s *Service) ensureResourceHasNoBusinessReferences(resource *model.Resource, ignoredDirect ...repository.ResourceDirectReference) error {
|
||||
if resource == nil {
|
||||
return NotFound("资源不存在")
|
||||
}
|
||||
@@ -348,7 +348,18 @@ func (s *Service) ensureResourceHasNoBusinessReferences(resource *model.Resource
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(snapshot.Direct) > 0 {
|
||||
hasBlocking := false
|
||||
DirectLoop:
|
||||
for _, direct := range snapshot.Direct {
|
||||
for _, ignored := range ignoredDirect {
|
||||
if direct.Kind == ignored.Kind && direct.ID == ignored.ID {
|
||||
continue DirectLoop
|
||||
}
|
||||
}
|
||||
hasBlocking = true
|
||||
break
|
||||
}
|
||||
if hasBlocking {
|
||||
return BadAuthRequest("公告配图仍被其他业务数据引用,已停止删除")
|
||||
}
|
||||
resourceIDs := map[string]struct{}{resource.ID: {}}
|
||||
|
||||
@@ -14,23 +14,25 @@ import (
|
||||
const featureAvailabilitySettingKey = "feature_availability"
|
||||
|
||||
const (
|
||||
FeatureShortDrama = "shortDrama"
|
||||
FeatureTaskCenter = "taskCenter"
|
||||
FeatureCredits = "credits"
|
||||
FeatureCustomChannels = "customChannels"
|
||||
FeatureFrontendModels = "frontendModels"
|
||||
FeaturePluginCenter = "pluginCenter"
|
||||
FeatureSystemPlugins = "systemPluginsVisibleToUsers"
|
||||
FeatureShortDrama = "shortDrama"
|
||||
FeatureTaskCenter = "taskCenter"
|
||||
FeatureCredits = "credits"
|
||||
FeatureCustomChannels = "customChannels"
|
||||
FeatureFrontendModels = "frontendModels"
|
||||
FeaturePluginCenter = "pluginCenter"
|
||||
FeatureSystemPlugins = "systemPluginsVisibleToUsers"
|
||||
FeatureTimelineTranscription = "timelineTranscription"
|
||||
)
|
||||
|
||||
type FeatureAvailability struct {
|
||||
ShortDramaEnabled bool `json:"shortDramaEnabled"`
|
||||
TaskCenterEnabled bool `json:"taskCenterEnabled"`
|
||||
CreditsEnabled bool `json:"creditsEnabled"`
|
||||
CustomChannelsEnabled bool `json:"customChannelsEnabled"`
|
||||
FrontendModelsEnabled bool `json:"frontendModelsEnabled"`
|
||||
PluginCenterEnabled bool `json:"pluginCenterEnabled"`
|
||||
SystemPluginsVisibleToUsers bool `json:"systemPluginsVisibleToUsers"`
|
||||
ShortDramaEnabled bool `json:"shortDramaEnabled"`
|
||||
TaskCenterEnabled bool `json:"taskCenterEnabled"`
|
||||
CreditsEnabled bool `json:"creditsEnabled"`
|
||||
CustomChannelsEnabled bool `json:"customChannelsEnabled"`
|
||||
FrontendModelsEnabled bool `json:"frontendModelsEnabled"`
|
||||
PluginCenterEnabled bool `json:"pluginCenterEnabled"`
|
||||
SystemPluginsVisibleToUsers bool `json:"systemPluginsVisibleToUsers"`
|
||||
TimelineTranscriptionEnabled bool `json:"timelineTranscriptionEnabled"`
|
||||
}
|
||||
|
||||
type PublicFeatureAvailability struct {
|
||||
@@ -44,13 +46,14 @@ type PublicFeatureAvailability struct {
|
||||
func defaultFeatureAvailability() FeatureAvailability {
|
||||
// 缺少配置代表尚未由运维接管;前台模型需要明确配置后才开放。
|
||||
return FeatureAvailability{
|
||||
ShortDramaEnabled: true,
|
||||
TaskCenterEnabled: true,
|
||||
CreditsEnabled: true,
|
||||
CustomChannelsEnabled: true,
|
||||
FrontendModelsEnabled: false,
|
||||
PluginCenterEnabled: true,
|
||||
SystemPluginsVisibleToUsers: true,
|
||||
ShortDramaEnabled: true,
|
||||
TaskCenterEnabled: true,
|
||||
CreditsEnabled: true,
|
||||
CustomChannelsEnabled: true,
|
||||
FrontendModelsEnabled: false,
|
||||
PluginCenterEnabled: true,
|
||||
SystemPluginsVisibleToUsers: true,
|
||||
TimelineTranscriptionEnabled: true,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -114,6 +117,8 @@ func (s *Service) FeatureEnabled(feature string) (bool, error) {
|
||||
return value.PluginCenterEnabled, nil
|
||||
case FeatureSystemPlugins:
|
||||
return value.SystemPluginsVisibleToUsers, nil
|
||||
case FeatureTimelineTranscription:
|
||||
return value.TimelineTranscriptionEnabled, nil
|
||||
default:
|
||||
return false, errors.New("未知功能开放配置")
|
||||
}
|
||||
@@ -142,6 +147,8 @@ func (s *Service) RequireFeature(feature string) error {
|
||||
return Forbidden("插件中心暂未开放")
|
||||
case FeatureSystemPlugins:
|
||||
return Forbidden("系统插件暂未向普通用户展示")
|
||||
case FeatureTimelineTranscription:
|
||||
return Forbidden("字幕转写暂未开放")
|
||||
default:
|
||||
return Forbidden("该功能暂未开放")
|
||||
}
|
||||
|
||||
@@ -2,9 +2,12 @@ package service
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"infinite-canvas/backend/internal/model"
|
||||
)
|
||||
|
||||
@@ -12,6 +15,8 @@ type LinkProjectAssetRequest struct {
|
||||
AssetID string `json:"assetId"`
|
||||
Category string `json:"category"`
|
||||
FolderID *string `json:"folderId"`
|
||||
// Title 媒体导入场景下由前端携带原始文件名;已有资产时忽略。
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
type UpdateProjectAssetRequest struct {
|
||||
@@ -37,8 +42,10 @@ type ProjectAssetSummary struct {
|
||||
FolderID string `json:"folderId,omitempty"`
|
||||
Position int `json:"position"`
|
||||
StorageKey string `json:"storageKey,omitempty"`
|
||||
DurationMs int64 `json:"durationMs,omitempty"`
|
||||
PreviewText string `json:"previewText,omitempty"`
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Character *CharacterCardSummary `json:"character,omitempty"`
|
||||
}
|
||||
|
||||
@@ -99,9 +106,17 @@ func (s *Service) LinkProjectAsset(userID string, projectID string, req LinkProj
|
||||
}
|
||||
assetID := strings.TrimSpace(req.AssetID)
|
||||
asset, err := s.repo.AssetForUser(userID, assetID)
|
||||
if err != nil {
|
||||
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return ProjectAssetSummary{}, err
|
||||
}
|
||||
if asset == nil {
|
||||
// 媒体导入只落 resources 表;首次链接时按资源元数据合成资产记录,
|
||||
// 避免“资源存在但无资产记录”导致导入永远失败。
|
||||
asset, err = s.assetFromUploadedResource(userID, assetID, req.Title)
|
||||
if err != nil {
|
||||
return ProjectAssetSummary{}, err
|
||||
}
|
||||
}
|
||||
category := model.AssetCategory(strings.TrimSpace(req.Category))
|
||||
if category == "" {
|
||||
category = asset.Category
|
||||
@@ -156,6 +171,62 @@ func (s *Service) LinkProjectAsset(userID string, projectID string, req LinkProj
|
||||
return s.projectAssetSummary(userID, projectID, asset)
|
||||
}
|
||||
|
||||
// assetFromUploadedResource 把媒体导入上传的 Resource 合成为资产记录(内存构造,不落库)。
|
||||
// 资产创建与项目链接由 LinkProjectAsset 事务原子提交,避免失败留下孤立资产。
|
||||
func (s *Service) assetFromUploadedResource(userID string, resourceID string, title string) (*model.Asset, error) {
|
||||
resource, err := s.repo.ResourceForUser(userID, resourceID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resource.Kind == "" {
|
||||
return nil, BadAuthRequest("资源缺少媒体类型,无法导入")
|
||||
}
|
||||
payload, err := json.Marshal(map[string]any{
|
||||
"data": map[string]any{
|
||||
"storageKey": "resource:" + resource.ID,
|
||||
"mimeType": resource.MimeType,
|
||||
"kind": resource.Kind,
|
||||
"size": resource.Size,
|
||||
"width": resource.Width,
|
||||
"height": resource.Height,
|
||||
"durationMs": resource.DurationMs,
|
||||
"status": resource.Status,
|
||||
"mediaType": resource.Kind,
|
||||
"source": "uploaded",
|
||||
},
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
title = strings.TrimSpace(title)
|
||||
if title == "" {
|
||||
title = mediaTitleFallback(resource)
|
||||
}
|
||||
asset := &model.Asset{
|
||||
ID: resource.ID,
|
||||
UserID: userID,
|
||||
Kind: resource.Kind,
|
||||
Category: model.NormalizeAssetCategory("", resource.Kind),
|
||||
Status: model.AssetVersionStatusConfirmed,
|
||||
Title: title,
|
||||
PayloadJSON: string(payload),
|
||||
CreatedAt: resource.CreatedAt,
|
||||
UpdatedAt: resource.UpdatedAt,
|
||||
}
|
||||
// 不在此预落库:资产创建交给 LinkProjectAsset 的事务统一提交,
|
||||
// 事务失败时与链接一起回滚,避免留下“有资产无链接”的孤儿记录。
|
||||
return asset, nil
|
||||
}
|
||||
|
||||
// mediaTitleFallback 资源无文件名时的展示标题:未命名 + 媒体类型 + 分辨率。
|
||||
func mediaTitleFallback(resource *model.Resource) string {
|
||||
base := "未命名" + resource.Kind
|
||||
if resource.Width > 0 && resource.Height > 0 {
|
||||
base += fmt.Sprintf(" %dx%d", resource.Width, resource.Height)
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
func (s *Service) UnlinkProjectAsset(userID string, projectID string, assetID string) error {
|
||||
if _, err := s.repo.ProjectForUser(userID, projectID); err != nil {
|
||||
return err
|
||||
@@ -482,8 +553,8 @@ func (s *Service) projectAssetSummary(userID string, projectID string, asset *mo
|
||||
if err != nil {
|
||||
return ProjectAssetSummary{}, err
|
||||
}
|
||||
storageKey, previewText := projectAssetPreview(asset.PayloadJSON)
|
||||
summary := ProjectAssetSummary{ID: asset.ID, Title: asset.Title, MediaType: asset.Kind, Category: asset.Category, Status: asset.Status, PrimaryVersionID: asset.PrimaryVersionID, VersionCount: len(versions), Usages: usages, FolderID: link.FolderID, Position: link.Position, StorageKey: storageKey, PreviewText: previewText, UpdatedAt: asset.UpdatedAt}
|
||||
storageKey, previewText, source, durationMs := projectAssetPreview(asset.PayloadJSON)
|
||||
summary := ProjectAssetSummary{ID: asset.ID, Title: asset.Title, MediaType: asset.Kind, Category: asset.Category, Status: asset.Status, PrimaryVersionID: asset.PrimaryVersionID, VersionCount: len(versions), Usages: usages, FolderID: link.FolderID, Position: link.Position, StorageKey: storageKey, PreviewText: previewText, DurationMs: durationMs, UpdatedAt: asset.UpdatedAt, Source: source}
|
||||
if asset.Category == model.AssetCategoryCharacter && asset.PrimaryVersionID != "" {
|
||||
card, cardErr := s.characterCard(userID, asset)
|
||||
if cardErr != nil {
|
||||
@@ -494,19 +565,22 @@ func (s *Service) projectAssetSummary(userID string, projectID string, asset *mo
|
||||
return summary, nil
|
||||
}
|
||||
|
||||
func projectAssetPreview(payloadJSON string) (string, string) {
|
||||
func projectAssetPreview(payloadJSON string) (string, string, string, int64) {
|
||||
var payload struct {
|
||||
Data struct {
|
||||
StorageKey string `json:"storageKey"`
|
||||
Content string `json:"content"`
|
||||
Source string `json:"source"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
} `json:"data"`
|
||||
}
|
||||
if json.Unmarshal([]byte(payloadJSON), &payload) != nil {
|
||||
return "", ""
|
||||
return "", "", "", 0
|
||||
}
|
||||
previewRunes := []rune(strings.TrimSpace(payload.Data.Content))
|
||||
if len(previewRunes) > 240 {
|
||||
previewRunes = append(previewRunes[:240], '…')
|
||||
}
|
||||
return strings.TrimSpace(payload.Data.StorageKey), string(previewRunes)
|
||||
return strings.TrimSpace(payload.Data.StorageKey), string(previewRunes), strings.TrimSpace(payload.Data.Source), payload.Data.DurationMs
|
||||
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
"infinite-canvas/backend/internal/repository"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func newProjectAssetLinkTestService(t *testing.T) (*Service, *gorm.DB) {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+newID()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(
|
||||
&model.Project{},
|
||||
&model.ProjectAssetLink{},
|
||||
&model.ProjectAssetFolder{},
|
||||
&model.Asset{},
|
||||
&model.AssetVersion{},
|
||||
&model.AssetRepresentation{},
|
||||
&model.Resource{},
|
||||
&model.Shot{},
|
||||
&model.ShotAssetReference{},
|
||||
); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return &Service{repo: repository.New(db)}, db
|
||||
}
|
||||
|
||||
// TestLinkProjectAssetCreatesAssetFromUploadedResource 覆盖媒体导入场景:
|
||||
// 上传只落 resources 表,首次链接时服务端应按资源元数据合成资产记录,
|
||||
// 否则“资源存在但无资产记录”会让导入永远失败。
|
||||
func TestLinkProjectAssetCreatesAssetFromUploadedResource(t *testing.T) {
|
||||
service, db := newProjectAssetLinkTestService(t)
|
||||
project := model.Project{ID: "project-1", UserID: "user-1", Name: "短剧", Status: model.ProjectStatusActive}
|
||||
resource := model.Resource{
|
||||
ID: "resource-1",
|
||||
UserID: "user-1",
|
||||
Kind: "video",
|
||||
Status: model.ResourceStatusReady,
|
||||
MimeType: "video/mp4",
|
||||
Size: 1024,
|
||||
Width: 1920,
|
||||
Height: 1080,
|
||||
}
|
||||
for _, item := range []any{&project, &resource} {
|
||||
if err := db.Create(item).Error; err != nil {
|
||||
t.Fatalf("seed %T: %v", item, err)
|
||||
}
|
||||
}
|
||||
|
||||
summary, err := service.LinkProjectAsset("user-1", "project-1", LinkProjectAssetRequest{
|
||||
AssetID: "resource-1",
|
||||
Category: "",
|
||||
Title: "开场.mp4",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("LinkProjectAsset: %v", err)
|
||||
}
|
||||
if summary.ID != "resource-1" {
|
||||
t.Fatalf("summary.ID = %q, want resource-1", summary.ID)
|
||||
}
|
||||
|
||||
var asset model.Asset
|
||||
if err := db.First(&asset, "id = ?", "resource-1").Error; err != nil {
|
||||
t.Fatalf("asset not created: %v", err)
|
||||
}
|
||||
if asset.Title != "开场.mp4" {
|
||||
t.Errorf("asset.Title = %q, want 开场.mp4", asset.Title)
|
||||
}
|
||||
if asset.Category != model.AssetCategoryMaterial {
|
||||
t.Errorf("asset.Category = %q, want material", asset.Category)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLinkProjectAssetMissingResourceKeepsFailingClosed 资源与资产都不存在时仍应报错,
|
||||
// 不允许凭空链接。
|
||||
func TestLinkProjectAssetMissingResourceKeepsFailingClosed(t *testing.T) {
|
||||
service, db := newProjectAssetLinkTestService(t)
|
||||
project := model.Project{ID: "project-2", UserID: "user-2", Name: "短剧", Status: model.ProjectStatusActive}
|
||||
if err := db.Create(&project).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if _, err := service.LinkProjectAsset("user-2", "project-2", LinkProjectAssetRequest{
|
||||
AssetID: "no-such-resource",
|
||||
}); err == nil {
|
||||
t.Fatal("LinkProjectAsset succeeded for missing resource, want error")
|
||||
}
|
||||
}
|
||||
@@ -21,6 +21,8 @@ func newProjectSettingsTestService(t *testing.T) (*Service, *gorm.DB) {
|
||||
&model.Project{}, &model.ProjectUnit{}, &model.Resource{}, &model.Asset{}, &model.AssetVersion{}, &model.AssetRepresentation{},
|
||||
&model.CanvasProject{}, &model.StyleProfile{}, &model.ProjectAssetCandidate{}, &model.WorkflowInstance{}, &model.WorkflowStepInstance{},
|
||||
&model.Shot{}, &model.ShotArtifact{}, &model.VoiceProfile{},
|
||||
&model.Task{}, &model.Session{}, &model.Message{}, &model.TaskLog{}, &model.Result{},
|
||||
&model.Announcement{}, &model.AnnouncementImageDraft{},
|
||||
); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
_ "image/jpeg"
|
||||
_ "image/png"
|
||||
"io"
|
||||
"log"
|
||||
"mime"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
@@ -308,7 +309,43 @@ func (s *Service) UploadResource(userID string, header *multipart.FileHeader, ki
|
||||
return resource, err
|
||||
}
|
||||
|
||||
func detectUploadedMimeType(file multipart.File, fileName string, declared string) string {
|
||||
// UploadResourceFile 接收已完整落盘的本地文件(分片上传合并后调用),语义与 UploadResource 一致;
|
||||
// 唯一差异是豁免“单文件大小上限”——分片会话已在 handler 按片校验,此处只受日上传与账号存储总量约束。
|
||||
func (s *Service) UploadResourceFile(userID string, fileName string, size int64, kind string, width int, height int, durationMs int64, file io.ReadSeeker, uploadIdentity ...string) (*model.Resource, error) {
|
||||
if file == nil || size <= 0 {
|
||||
return nil, BadAuthRequest("请选择要上传的文件")
|
||||
}
|
||||
uploadKey := normalizedResourceUploadKey(uploadIdentity)
|
||||
existing, err := s.resourceForUploadKey(userID, uploadKey)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if existing != nil && existing.Status == model.ResourceStatusReady {
|
||||
return existing, nil
|
||||
}
|
||||
if existing != nil && existing.Status == model.ResourceStatusPending {
|
||||
return nil, resourceUploadInProgress()
|
||||
}
|
||||
mimeType := detectUploadedMimeType(file, fileName, "")
|
||||
if existing != nil {
|
||||
return s.retryStoredResource(userID, existing, kind, mimeType, size, file)
|
||||
}
|
||||
day, err := s.reserveChunkedUploadQuota(userID, size)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resource, stored, err := s.storeResource(userID, kind, fileName, mimeType, size, width, height, durationMs, file, uploadKey)
|
||||
if err != nil {
|
||||
s.releaseUserUploadQuota(userID, day, size)
|
||||
} else if stored {
|
||||
s.commitUserUploadQuota(userID, size)
|
||||
} else {
|
||||
s.releaseUserUploadQuota(userID, day, size)
|
||||
}
|
||||
return resource, err
|
||||
}
|
||||
|
||||
func detectUploadedMimeType(file io.ReadSeeker, fileName string, declared string) string {
|
||||
declared = strings.TrimSpace(strings.Split(declared, ";")[0])
|
||||
if declared != "" && declared != "application/octet-stream" {
|
||||
return declared
|
||||
@@ -492,12 +529,7 @@ func (s *Service) storeResource(userID string, kind string, fileName string, mim
|
||||
return nil, false, err
|
||||
}
|
||||
var etag string
|
||||
if provider == "local" {
|
||||
filePath := filepath.Join(s.dataDir, "resources", filepath.FromSlash(objectKey))
|
||||
err = writeLocalResourceObject(filePath, body)
|
||||
} else {
|
||||
etag, err = putOSSObject(setting, objectKey, mimeType, size, body)
|
||||
}
|
||||
etag, err = s.storeResourceObject(&resource, fileName, body)
|
||||
resource.UpdatedAt = time.Now()
|
||||
if err != nil {
|
||||
resource.Status = model.ResourceStatusFailed
|
||||
@@ -527,6 +559,7 @@ func (s *Service) storeResource(userID string, kind string, fileName string, mim
|
||||
return nil, true, errors.Join(err, fmt.Errorf("清理已上传资源对象失败:%w", cleanupErr))
|
||||
}
|
||||
s.recordActivity(userID, "resource", 1)
|
||||
s.maybeStartPlaybackTranscode(&resource)
|
||||
return &resource, true, nil
|
||||
}
|
||||
|
||||
@@ -546,6 +579,48 @@ func writeLocalResourceObject(filePath string, body io.Reader) error {
|
||||
return closeErr
|
||||
}
|
||||
|
||||
// storeResourceObject 写入资源物理对象。对象存储不可用(配置错误、密钥失效、网络
|
||||
// 故障、设置被删)时自动降级为本地存储并同步改写资源记录,保证上传写路径不因外部
|
||||
// 存储故障整体失败。对象存储失败后 body 会被重新读取,须支持 Seek。
|
||||
func (s *Service) storeResourceObject(resource *model.Resource, fileName string, body io.Reader) (string, error) {
|
||||
if resource == nil {
|
||||
return "", errors.New("资源不存在")
|
||||
}
|
||||
if resource.Provider == "local" {
|
||||
return "", writeLocalResourceObject(filepath.Join(s.dataDir, "resources", filepath.FromSlash(resource.ObjectKey)), body)
|
||||
}
|
||||
setting, settingErr := s.ossSettingForResource(resource.UserID, resource)
|
||||
var etag string
|
||||
var putErr error
|
||||
if settingErr == nil {
|
||||
etag, putErr = putOSSObject(setting, resource.ObjectKey, resource.MimeType, resource.Size, body)
|
||||
}
|
||||
if putErr == nil && settingErr == nil {
|
||||
return etag, nil
|
||||
}
|
||||
fallbackErr := putErr
|
||||
if fallbackErr == nil {
|
||||
fallbackErr = settingErr
|
||||
}
|
||||
if seeker, ok := body.(io.Seeker); ok {
|
||||
if _, seekErr := seeker.Seek(0, io.SeekStart); seekErr != nil {
|
||||
return "", errors.Join(fallbackErr, fmt.Errorf("降级本地存储时重置读取位置失败:%w", seekErr))
|
||||
}
|
||||
}
|
||||
localKey := localObjectKey(resource.UserID, resource.Kind, fileName, resource.MimeType, time.Now())
|
||||
resource.Provider = "local"
|
||||
resource.ObjectKey = localKey
|
||||
resource.Endpoint = ""
|
||||
resource.Bucket = ""
|
||||
resource.StorageSettingID = ""
|
||||
resource.ETag = ""
|
||||
if localErr := writeLocalResourceObject(filepath.Join(s.dataDir, "resources", filepath.FromSlash(localKey)), body); localErr != nil {
|
||||
return "", errors.Join(fallbackErr, fmt.Errorf("降级本地存储失败:%w", localErr))
|
||||
}
|
||||
log.Printf("object storage upload degraded to local storage: resource=%s error=%v", resource.ID, fallbackErr)
|
||||
return "", nil
|
||||
}
|
||||
|
||||
func (s *Service) retryStoredResource(userID string, resource *model.Resource, kind string, mimeType string, size int64, body io.Reader) (*model.Resource, error) {
|
||||
if resource == nil {
|
||||
return nil, errors.New("资源不存在")
|
||||
@@ -585,15 +660,7 @@ func (s *Service) retryStoredResource(userID string, resource *model.Resource, k
|
||||
return nil, err
|
||||
}
|
||||
var etag string
|
||||
if resource.Provider == "local" {
|
||||
err = writeLocalResourceObject(filepath.Join(s.dataDir, "resources", filepath.FromSlash(resource.ObjectKey)), body)
|
||||
} else {
|
||||
var setting ossSettingValue
|
||||
setting, err = s.ossSettingForResource(userID, resource)
|
||||
if err == nil {
|
||||
etag, err = putOSSObject(setting, resource.ObjectKey, resource.MimeType, resource.Size, body)
|
||||
}
|
||||
}
|
||||
etag, err = s.storeResourceObject(resource, "", body)
|
||||
resource.UpdatedAt = time.Now()
|
||||
if err != nil {
|
||||
s.releaseRetryUploadQuota(userID, day, size)
|
||||
@@ -1489,7 +1556,17 @@ func newOSSRequest(method string, setting ossSettingValue, objectKey string, con
|
||||
return nil, err
|
||||
}
|
||||
baseURL.Path = strings.TrimRight(baseURL.Path, "/") + "/" + escapeObjectKey(objectKey)
|
||||
req, err := http.NewRequest(method, baseURL.String(), body)
|
||||
// 请求体必须用 no-op close 包装:服务端提前返回(如 OSS 签名 403)时
|
||||
// http.Transport 会关闭未发完的 Request.Body,若直接传入 *os.File 等
|
||||
// 调用方持有的文件,后续“降级本地存储”的 Seek 重读将因 file already
|
||||
// closed 失败。NopCloser 让 Transport 的关闭成为空操作,底层文件保持可用。
|
||||
// GET/HEAD/DELETE 等无请求体的调用传入 nil body;NopCloser(nil) 会产生非 nil 的
|
||||
// Body 包装 nil reader,Go 1.26 发送前 body 探测会直接 nil 解引用崩溃。
|
||||
var reqBody io.Reader
|
||||
if body != nil {
|
||||
reqBody = io.NopCloser(body)
|
||||
}
|
||||
req, err := http.NewRequest(method, baseURL.String(), reqBody)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
"infinite-canvas/backend/internal/repository"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
func newResourceFallbackTestService(t *testing.T) (*Service, *gorm.DB) {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+newID()+"?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.AutoMigrate(
|
||||
&model.Resource{},
|
||||
&model.UserOSSSetting{},
|
||||
&model.SystemSetting{},
|
||||
&model.StorageLocation{},
|
||||
&model.UserDailyActivity{},
|
||||
); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return &Service{repo: repository.New(db), dataDir: t.TempDir()}, db
|
||||
}
|
||||
|
||||
// TestStoreResourceDegradesToLocalWhenOSSUnavailable 覆盖对象存储不可用时的降级:
|
||||
// 配置了 OSS(enabled=1、endpoint 不可达)时上传应回退本地存储成功,
|
||||
// 而不是把上传标记为失败——本地媒体导入不应因外部存储故障整体失败。
|
||||
func TestStoreResourceDegradesToLocalWhenOSSUnavailable(t *testing.T) {
|
||||
service, db := newResourceFallbackTestService(t)
|
||||
seedOSSEnabled(t, db, "user-1", "http://127.0.0.1:1")
|
||||
|
||||
resource, created, err := service.storeResource(
|
||||
"user-1", "video", "intro.mp4", "video/mp4", 1024,
|
||||
1920, 1080, 0, bytes.NewReader([]byte("fake-mp4-bytes")), nil,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("storeResource: %v", err)
|
||||
}
|
||||
if !created {
|
||||
t.Fatal("storeResource returned created=false, want true")
|
||||
}
|
||||
if resource.Status != model.ResourceStatusReady {
|
||||
t.Fatalf("resource.Status = %q, want %q", resource.Status, model.ResourceStatusReady)
|
||||
}
|
||||
if resource.Provider != "local" {
|
||||
t.Fatalf("resource.Provider = %q, want local (degraded from OSS)", resource.Provider)
|
||||
}
|
||||
if resource.Endpoint != "" || resource.Bucket != "" || resource.StorageSettingID != "" {
|
||||
t.Fatalf("resource storage binding not cleared after degrade: endpoint=%q bucket=%q setting=%q",
|
||||
resource.Endpoint, resource.Bucket, resource.StorageSettingID)
|
||||
}
|
||||
payload, err := os.ReadFile(filepath.Join(service.dataDir, "resources", filepath.FromSlash(resource.ObjectKey)))
|
||||
if err != nil {
|
||||
t.Fatalf("local object not written: %v", err)
|
||||
}
|
||||
if string(payload) != "fake-mp4-bytes" {
|
||||
t.Fatalf("local object content = %q, want fake-mp4-bytes", payload)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStoreResourceLocalPathUnaffectedByOSS 未启用 OSS 时上传仍走本地存储,不受其他用户 OSS 设置影响。
|
||||
func TestStoreResourceLocalPathUnaffectedByOSS(t *testing.T) {
|
||||
service, db := newResourceFallbackTestService(t)
|
||||
seedOSSEnabled(t, db, "user-other", "http://127.0.0.1:1")
|
||||
|
||||
resource, created, err := service.storeResource(
|
||||
"user-2", "video", "local.mp4", "video/mp4", 512,
|
||||
1280, 720, 0, bytes.NewReader([]byte("local-bytes")), nil,
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("storeResource: %v", err)
|
||||
}
|
||||
if !created {
|
||||
t.Fatal("storeResource returned created=false, want true")
|
||||
}
|
||||
if resource.Provider != "local" {
|
||||
t.Fatalf("resource.Provider = %q, want local", resource.Provider)
|
||||
}
|
||||
payload, err := os.ReadFile(filepath.Join(service.dataDir, "resources", filepath.FromSlash(resource.ObjectKey)))
|
||||
if err != nil {
|
||||
t.Fatalf("local object not written: %v", err)
|
||||
}
|
||||
if string(payload) != "local-bytes" {
|
||||
t.Fatalf("local object content = %q, want local-bytes", payload)
|
||||
}
|
||||
}
|
||||
|
||||
func seedOSSEnabled(t *testing.T, db *gorm.DB, userID string, endpoint string) {
|
||||
t.Helper()
|
||||
setting := model.UserOSSSetting{
|
||||
UserID: userID,
|
||||
Enabled: true,
|
||||
ValueJSON: `{"provider":"aliyun","endpoint":"` + endpoint + `","bucket":"test-bucket","accessKeyId":"ak-test","accessKeySecret":"sk-plaintext","region":"cn-shenzhen"}`,
|
||||
}
|
||||
if err := db.Create(&setting).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
@@ -95,6 +97,47 @@ func TestAliyunOSSUploadRequestStillUsesEndpointWhenCDNConfigured(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewOSSRequestBodyCloseDoesNotCloseCallerFile(t *testing.T) {
|
||||
// 服务端提前返回 403(不读 body)时,http.Transport 会关闭 Request.Body。
|
||||
// newOSSRequest 必须用 no-op close 包装请求体,否则调用方持有的 *os.File
|
||||
// 会被关掉,OSS 失败后的“降级本地存储”Seek 重读将报 file already closed。
|
||||
f, err := os.CreateTemp(t.TempDir(), "merged")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer f.Close()
|
||||
if _, err := f.WriteString("payload-bytes"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req, err := newOSSRequest(http.MethodPut, ossSettingValue{
|
||||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com",
|
||||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||||
}, "users/u-1/video/clip.mp4", "video/mp4", f)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if req.Body == nil {
|
||||
t.Fatal("newOSSRequest() body is nil")
|
||||
}
|
||||
// 模拟 Transport 在服务端提前响应后关闭请求体:
|
||||
if err := req.Body.Close(); err != nil {
|
||||
t.Fatalf("close request body: %v", err)
|
||||
}
|
||||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||||
t.Fatalf("caller file was closed by transport: %v", err)
|
||||
}
|
||||
got, err := io.ReadAll(f)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(got) != "payload-bytes" {
|
||||
t.Fatalf("payload after body close = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignedOSSObjectURLUsesTencentCOSCDNBaseURLWithoutCOSSignature(t *testing.T) {
|
||||
value, err := signedOSSObjectURL(ossSettingValue{
|
||||
Provider: tencentCOSProvider, Region: "ap-guangzhou", Bucket: "private-bucket-1250000000",
|
||||
@@ -962,7 +1005,7 @@ func TestPersistGeneratedMediaAppliesStoredFileQuota(t *testing.T) {
|
||||
_, err := svc.persistGeneratedMediaResult("user-1", map[string]interface{}{
|
||||
"image": map[string]interface{}{"dataUrl": "data:image/png;base64,YQ=="},
|
||||
})
|
||||
if err == nil || !strings.Contains(err.Error(), "2GB 上限") {
|
||||
if err == nil || !strings.Contains(err.Error(), "20GB 上限") {
|
||||
t.Fatalf("persistGeneratedMediaResult() error = %v", err)
|
||||
}
|
||||
}
|
||||
@@ -103,8 +103,8 @@ func defaultRuntimePolicy() RuntimePolicySetting {
|
||||
ResourceUploadMB: 50,
|
||||
SessionUploadMB: 32,
|
||||
GeneratedFileMB: 64,
|
||||
DailyUploadMB: 200,
|
||||
StoredFileGB: 2,
|
||||
DailyUploadMB: 2048,
|
||||
StoredFileGB: 20,
|
||||
StructuredDataMB: 256,
|
||||
TaskDataGB: 1,
|
||||
AssetCount: 2_000,
|
||||
|
||||
@@ -0,0 +1,200 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
)
|
||||
|
||||
const renderFfmpegEnv = "CANVAS_FFMPEG_PATH"
|
||||
|
||||
// processTimelineRender 执行时间线渲染:按快照把引用的媒体落盘 →
|
||||
// ffprobe 探测音轨 → ffmpeg concat 合成 → 产物写入资源存储。
|
||||
// 渲染是本地重编码,不经模型路由与计费;失败一律落明确终态。
|
||||
func (w *taskWorkerCoordinator) processTimelineRender(task *model.Task, ctx context.Context) error {
|
||||
s := w.service
|
||||
ffmpegBin, err := renderFfmpegBinary()
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", err.Error())
|
||||
}
|
||||
var input timelineRenderInput
|
||||
if err := json.Unmarshal([]byte(task.InputJSON), &input); err != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", "任务缺少有效的时间线快照")
|
||||
}
|
||||
plan := buildRenderPlan(input.Timeline)
|
||||
if !plan.HasMedia {
|
||||
return w.failTimelineTask(task, "渲染失败", "时间线没有可渲染的媒体片段")
|
||||
}
|
||||
|
||||
if err := w.progress(task, "准备媒体…", 10); err != nil {
|
||||
return err
|
||||
}
|
||||
workDir, cleanup, err := materializeRenderSources(ctx, s, task.UserID, &plan)
|
||||
if cleanup != nil {
|
||||
defer cleanup()
|
||||
}
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", err.Error())
|
||||
}
|
||||
for _, seg := range plan.Segments {
|
||||
if seg.Source == nil || seg.Kind == "image" {
|
||||
continue
|
||||
}
|
||||
hasAudio, probeErr := probeHasAudioStream(ctx, seg.Source.Path)
|
||||
if probeErr != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", probeErr.Error())
|
||||
}
|
||||
seg.Source.HasAudio = hasAudio
|
||||
}
|
||||
|
||||
args := buildRenderFFmpegArgs(plan, filepath.Join(workDir, "render-output.mp4"))
|
||||
if len(args) == 0 {
|
||||
return w.failTimelineTask(task, "渲染失败", "无法生成渲染命令")
|
||||
}
|
||||
if err := w.progress(task, "正在渲染…", 30); err != nil {
|
||||
return err
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, ffmpegBin, args...)
|
||||
cmd.Dir = workDir
|
||||
output, runErr := cmd.CombinedOutput()
|
||||
if runErr != nil {
|
||||
detail := strings.TrimSpace(string(output))
|
||||
if len(detail) > 400 {
|
||||
detail = detail[len(detail)-400:]
|
||||
}
|
||||
return w.failTimelineTask(task, "渲染失败", fmt.Sprintf("ffmpeg 渲染失败:%s", detail))
|
||||
}
|
||||
|
||||
if err := w.progress(task, "写入资源…", 85); err != nil {
|
||||
return err
|
||||
}
|
||||
renderedPath := filepath.Join(workDir, "render-output.mp4")
|
||||
file, err := os.Open(renderedPath)
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", "读取渲染产物失败")
|
||||
}
|
||||
defer file.Close()
|
||||
stat, err := file.Stat()
|
||||
if err != nil || stat.Size() == 0 {
|
||||
return w.failTimelineTask(task, "渲染失败", "渲染产物为空")
|
||||
}
|
||||
durationMs := planDurationMs(plan)
|
||||
fileName := fmt.Sprintf("timeline-render-%s.mp4", time.Now().Format("20060102-150405"))
|
||||
resource, _, err := s.storeResource(task.UserID, "media", fileName, "video/mp4", stat.Size(), renderWidth, renderHeight, durationMs, file, nil)
|
||||
if err != nil || resource == nil {
|
||||
return w.failTimelineTask(task, "渲染失败", "保存渲染产物失败")
|
||||
}
|
||||
|
||||
result := timelineRenderResult{
|
||||
ResourceID: resource.ID,
|
||||
FileName: fileName,
|
||||
Size: stat.Size(),
|
||||
DurationMs: durationMs,
|
||||
SubtitleSRT: plan.SubtitleSRT,
|
||||
}
|
||||
payload, err := json.Marshal(result)
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "渲染失败", "渲染结果序列化失败")
|
||||
}
|
||||
task.Status = model.TaskStatusSucceeded
|
||||
task.Stage = "渲染完成"
|
||||
task.Progress = 100
|
||||
task.ResultJSON = string(payload)
|
||||
completedAt := time.Now()
|
||||
task.CompletedAt = &completedAt
|
||||
if err := s.repo.SaveTaskCompletion(task, model.TaskStatusRunning, nil, nil, nil); err != nil {
|
||||
return fmt.Errorf("写入渲染完成态失败: %w", err)
|
||||
}
|
||||
s.logInfo(task.UserID, task.ID, fmt.Sprintf("时间线渲染完成,时长 %.1fs", float64(durationMs)/1000), "")
|
||||
return nil
|
||||
}
|
||||
|
||||
func planDurationMs(plan renderPlan) int64 {
|
||||
var total int64
|
||||
for _, seg := range plan.Segments {
|
||||
total += seg.DurationMs
|
||||
}
|
||||
return total
|
||||
}
|
||||
|
||||
func renderFfmpegBinary() (string, error) {
|
||||
if configured := strings.TrimSpace(os.Getenv(renderFfmpegEnv)); configured != "" {
|
||||
return configured, nil
|
||||
}
|
||||
path, err := exec.LookPath("ffmpeg")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("渲染依赖未安装(需要 ffmpeg,可通过 %s 指定)", renderFfmpegEnv)
|
||||
}
|
||||
return path, nil
|
||||
}
|
||||
|
||||
// materializeRenderSources 把计划中每个媒体片段对应的资源下载到临时目录,
|
||||
// 同资源复用同一份本地文件;返回工作目录与清理函数。
|
||||
func materializeRenderSources(ctx context.Context, s *Service, userID string, plan *renderPlan) (string, func(), error) {
|
||||
tmpDir, err := os.MkdirTemp("", "yingce-render-*")
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("创建临时目录失败: %w", err)
|
||||
}
|
||||
cleanup := func() { _ = os.RemoveAll(tmpDir) }
|
||||
cache := map[string]string{}
|
||||
for i := range plan.Segments {
|
||||
seg := &plan.Segments[i]
|
||||
resourceID, ok := mediaResourceID(seg.Clip)
|
||||
if !ok {
|
||||
// 无可用媒体引用时渲染为黑场段,保证时间轴连续。
|
||||
continue
|
||||
}
|
||||
path, cached := cache[resourceID]
|
||||
if !cached {
|
||||
_, reader, err := s.OpenResource(userID, resourceID)
|
||||
if err != nil || reader == nil {
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("无法读取时间线引用的媒体,可能已被删除")
|
||||
}
|
||||
ext := extForMime("video/mp4")
|
||||
path = filepath.Join(tmpDir, fmt.Sprintf("src-%d%s", len(cache), ext))
|
||||
file, err := os.Create(path)
|
||||
if err != nil {
|
||||
reader.Close()
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("写入临时媒体失败: %w", err)
|
||||
}
|
||||
if _, err := io.Copy(file, reader); err != nil {
|
||||
file.Close()
|
||||
reader.Close()
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("读取时间线媒体失败: %w", err)
|
||||
}
|
||||
file.Close()
|
||||
reader.Close()
|
||||
cache[resourceID] = path
|
||||
}
|
||||
seg.Source = &renderSource{ResourceID: resourceID, Clip: seg.Clip, Path: path, Ext: filepath.Ext(path)}
|
||||
}
|
||||
return tmpDir, cleanup, nil
|
||||
}
|
||||
|
||||
// probeHasAudioStream 用 ffprobe 判断媒体是否含音轨,供滤镜图选择静音回退。
|
||||
func probeHasAudioStream(ctx context.Context, path string) (bool, error) {
|
||||
bin, err := exec.LookPath("ffprobe")
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("媒体探测依赖未安装(需要 ffprobe)")
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, bin,
|
||||
"-v", "error", "-select_streams", "a:0", "-show_entries", "stream=index",
|
||||
"-of", "csv=p=0", path)
|
||||
output, runErr := cmd.Output()
|
||||
if runErr != nil {
|
||||
// 探测失败按无音轨处理,渲染仍可产出静音视频。
|
||||
return false, nil
|
||||
}
|
||||
return strings.TrimSpace(string(output)) != "", nil
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
)
|
||||
|
||||
// renderTestProject 构造一个可渲染的 v2 快照:一条可见视频轨 + 一个
|
||||
// 经 directMedia.storageKey(resource:<id>)引用后端资源的片段。
|
||||
func renderTestProject(storageKey string) renderProject {
|
||||
visible := true
|
||||
return renderProject{
|
||||
Version: 2,
|
||||
DurationMs: 2000,
|
||||
Tracks: []renderTrack{{ID: "track-v1", Kind: "video", Visible: &visible}},
|
||||
Clips: []renderClip{{
|
||||
ID: "clip-v1",
|
||||
Kind: "video",
|
||||
TrackID: "track-v1",
|
||||
StartMs: 0,
|
||||
DurationMs: 2000,
|
||||
SourceStartMs: 0,
|
||||
Volume: 1,
|
||||
DirectMedia: &struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
StorageKey string `json:"storageKey"`
|
||||
}{ID: "asset-v1", Kind: "video", StorageKey: storageKey},
|
||||
}},
|
||||
}
|
||||
}
|
||||
|
||||
func renderInputJSON(t *testing.T, project renderProject) string {
|
||||
t.Helper()
|
||||
raw, err := json.Marshal(timelineRenderInput{ProjectID: "prj-render", Timeline: project})
|
||||
if err != nil {
|
||||
t.Fatalf("marshal render input: %v", err)
|
||||
}
|
||||
return string(raw)
|
||||
}
|
||||
|
||||
func seedRunningRenderTask(t *testing.T, db *gorm.DB, inputJSON string) *model.Task {
|
||||
t.Helper()
|
||||
task := &model.Task{
|
||||
ID: fmt.Sprintf("tsk-render-%d", time.Now().UnixNano()),
|
||||
UserID: "usr-render-test",
|
||||
Type: model.TaskTypeTimelineRender,
|
||||
Status: model.TaskStatusRunning,
|
||||
Stage: "已领取",
|
||||
InputJSON: inputJSON,
|
||||
}
|
||||
if err := db.Create(task).Error; err != nil {
|
||||
t.Fatalf("seed task: %v", err)
|
||||
}
|
||||
return task
|
||||
}
|
||||
|
||||
func TestCreateTimelineRenderTaskQueues(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
|
||||
task, err := svc.CreateTimelineRenderTask("usr-render-test", TimelineRenderCreateRequest{
|
||||
ProjectID: "prj-render",
|
||||
Timeline: renderTestProject("resource:res-1"),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if task.Type != model.TaskTypeTimelineRender || task.Status != model.TaskStatusQueued {
|
||||
t.Fatalf("task = %s/%s, want %s/queued", task.Type, task.Status, model.TaskTypeTimelineRender)
|
||||
}
|
||||
if task.Provider != "local" || task.Model != "ffmpeg" {
|
||||
t.Fatalf("provider/model = %s/%s, want local/ffmpeg", task.Provider, task.Model)
|
||||
}
|
||||
|
||||
var stored model.Task
|
||||
if err := db.First(&stored, "id = ?", task.ID).Error; err != nil {
|
||||
t.Fatalf("load task: %v", err)
|
||||
}
|
||||
if stored.ProjectID != "prj-render" || stored.UserID != "usr-render-test" {
|
||||
t.Fatalf("stored owner/project mismatch: %s/%s", stored.UserID, stored.ProjectID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTimelineRenderTaskRejectsNoMedia(t *testing.T) {
|
||||
svc, _ := newTimelineTaskTestService(t)
|
||||
|
||||
visible := true
|
||||
textOnly := renderProject{
|
||||
Version: 2,
|
||||
DurationMs: 1000,
|
||||
Tracks: []renderTrack{{ID: "track-t1", Kind: "text", Visible: &visible}},
|
||||
Clips: []renderClip{{
|
||||
ID: "clip-t1", Kind: "text", TrackID: "track-t1", StartMs: 0,
|
||||
DurationMs: 1000, Text: "字幕",
|
||||
}},
|
||||
}
|
||||
_, err := svc.CreateTimelineRenderTask("usr-render-test", TimelineRenderCreateRequest{Timeline: textOnly})
|
||||
if err == nil || !strings.Contains(err.Error(), "可渲染") {
|
||||
t.Fatalf("err = %v, want mention of 没有可渲染的媒体片段", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimelineRenderFailsFastWhenSourceUnreadable(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
// 指向不存在的资源;ffmpeg 路径设为无效值不影响断言——本测试在
|
||||
// 素材落盘(materialize)阶段即失败,且该失败早于任何外部命令执行。
|
||||
task := seedRunningRenderTask(t, db, renderInputJSON(t, renderTestProject("resource:res-missing")))
|
||||
|
||||
t.Setenv(renderFfmpegEnv, "/no/such/ffmpeg")
|
||||
w := newTaskWorkerCoordinator(svc)
|
||||
if err := w.processTimelineRender(task, context.Background()); err != nil {
|
||||
t.Fatalf("process: want nil (task terminal handled internally), got %v", err)
|
||||
}
|
||||
|
||||
var stored model.Task
|
||||
if err := db.First(&stored, "id = ?", task.ID).Error; err != nil {
|
||||
t.Fatalf("load task: %v", err)
|
||||
}
|
||||
if stored.Status != model.TaskStatusFailed {
|
||||
t.Fatalf("status = %q, want failed", stored.Status)
|
||||
}
|
||||
if !strings.Contains(stored.Error, "时间线引用的媒体") {
|
||||
t.Fatalf("error = %q, want mention of 无法读取时间线引用的媒体", stored.Error)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
"infinite-canvas/backend/internal/repository"
|
||||
)
|
||||
|
||||
// timeline 转写任务输入,由画布提交,字段与前端契约一致。
|
||||
type timelineTranscriptionInput struct {
|
||||
ResourceID string `json:"resourceId"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
const whisperLangEnv = "CANVAS_WHISPER_BASE_URL"
|
||||
|
||||
// processTimelineTranscription 执行时间线字幕转写:
|
||||
// 读取资源 -> 本地 ffmpeg 预处理为 16k 单声道 wav -> 本地 whisper.cpp 识别 ->
|
||||
// 结果写回任务(segments + srt)。任何失败都落到任务终态并携带用户可读原因。
|
||||
func (w *taskWorkerCoordinator) processTimelineTranscription(task *model.Task, ctx context.Context) error {
|
||||
s := w.service
|
||||
baseURL := strings.TrimSpace(os.Getenv(whisperLangEnv))
|
||||
if baseURL == "" {
|
||||
return w.failTimelineTask(task, "转写失败", "未配置本地转写服务:请设置 CANVAS_WHISPER_BASE_URL 指向 whisper.cpp 服务")
|
||||
}
|
||||
var input timelineTranscriptionInput
|
||||
if err := json.Unmarshal([]byte(task.InputJSON), &input); err != nil || strings.TrimSpace(input.ResourceID) == "" {
|
||||
return w.failTimelineTask(task, "转写失败", "任务缺少有效的资源引用")
|
||||
}
|
||||
if err := s.RequireFeature(FeatureTimelineTranscription); err != nil {
|
||||
return w.failTimelineTask(task, "转写失败", "字幕转写暂未开放")
|
||||
}
|
||||
s.logInfo(task.UserID, task.ID, "时间线转写任务开始", "")
|
||||
|
||||
resource, reader, err := s.OpenResource(task.UserID, input.ResourceID)
|
||||
if err != nil || reader == nil {
|
||||
return w.failTimelineTask(task, "转写失败", "无法读取待转写媒体,可能已被删除")
|
||||
}
|
||||
defer reader.Close()
|
||||
if resource == nil || !isTranscribableMime(resource.MimeType) {
|
||||
return w.failTimelineTask(task, "转写失败", "仅支持音视频文件转写")
|
||||
}
|
||||
|
||||
if err := w.progress(task, "等待转写服务", 15); err != nil {
|
||||
return err
|
||||
}
|
||||
wavPath, cleanup, err := prepareWhisperWav(ctx, reader, resource.MimeType)
|
||||
if cleanup != nil {
|
||||
defer cleanup()
|
||||
}
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "转写失败", err.Error())
|
||||
}
|
||||
if err := w.progress(task, "正在转写…", 40); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
client := newWhisperClient(baseURL)
|
||||
segments, language, err := client.transcribe(ctx, wavPath, input.Language)
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "转写失败", err.Error())
|
||||
}
|
||||
if err := w.progress(task, "整理字幕…", 80); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
result := timelineTranscriptionResult{
|
||||
Segments: segments,
|
||||
SRT: buildTimelineSRT(segments),
|
||||
Language: language,
|
||||
}
|
||||
payload, err := json.Marshal(result)
|
||||
if err != nil {
|
||||
return w.failTimelineTask(task, "转写失败", "转写结果序列化失败")
|
||||
}
|
||||
task.Status = model.TaskStatusSucceeded
|
||||
task.Stage = "转写完成"
|
||||
task.Progress = 100
|
||||
task.ResultJSON = string(payload)
|
||||
completedAt := time.Now()
|
||||
task.CompletedAt = &completedAt
|
||||
if err := s.repo.SaveTaskCompletion(task, model.TaskStatusRunning, nil, nil, nil); err != nil {
|
||||
// 冲突/租约已失效时不覆盖他人终态,交由上层判定。
|
||||
return fmt.Errorf("写入转写完成态失败: %w", err)
|
||||
}
|
||||
s.logInfo(task.UserID, task.ID, fmt.Sprintf("时间线转写完成,段落 %d", len(segments)), "")
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *taskWorkerCoordinator) failTimelineTask(task *model.Task, stage string, message string) error {
|
||||
s := w.service
|
||||
done, err := s.repo.UpdateTaskTerminalState(task.ID, model.TaskStatusRunning, model.TaskStatusFailed, stage, message, time.Now())
|
||||
if err != nil {
|
||||
return fmt.Errorf("写入转写失败态失败: %w", err)
|
||||
}
|
||||
s.logInfo(task.UserID, task.ID, fmt.Sprintf("时间线转写失败: %s", message), "")
|
||||
if !done {
|
||||
return fmt.Errorf("时间线转写已失败: %s", message)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *taskWorkerCoordinator) progress(task *model.Task, stage string, progress int) error {
|
||||
if err := w.service.repo.UpdateTaskProgress(task.ID, stage, progress); err != nil {
|
||||
return fmt.Errorf("更新转写进度失败: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Service) logInfo(userID string, taskID string, message string, extra string) {
|
||||
s.log(userID, taskID, "info", message, extra)
|
||||
}
|
||||
|
||||
func isTranscribableMime(mime string) bool {
|
||||
return strings.HasPrefix(mime, "video/") || strings.HasPrefix(mime, "audio/")
|
||||
}
|
||||
|
||||
// prepareWhisperWav 将媒体流经本地 ffmpeg 转为 whisper.cpp 期望的
|
||||
// 16k 单声道 PCM wav;返回临时文件路径与清理函数。
|
||||
func prepareWhisperWav(ctx context.Context, reader io.Reader, mime string) (string, func(), error) {
|
||||
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||
return "", nil, fmt.Errorf("音频预处理依赖未安装(需要 ffmpeg)")
|
||||
}
|
||||
tmpDir, err := os.MkdirTemp("", "yingce-whisper-*")
|
||||
if err != nil {
|
||||
return "", nil, fmt.Errorf("创建临时目录失败: %w", err)
|
||||
}
|
||||
cleanup := func() { _ = os.RemoveAll(tmpDir) }
|
||||
inPath := filepath.Join(tmpDir, "input"+extForMime(mime))
|
||||
inFile, err := os.Create(inPath)
|
||||
if err != nil {
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("写入待转写媒体失败: %w", err)
|
||||
}
|
||||
if _, err := io.Copy(inFile, reader); err != nil {
|
||||
inFile.Close()
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("读取待转写媒体失败: %w", err)
|
||||
}
|
||||
if err := inFile.Close(); err != nil {
|
||||
cleanup()
|
||||
return "", nil, fmt.Errorf("关闭临时文件失败: %w", err)
|
||||
}
|
||||
wavPath := filepath.Join(tmpDir, "audio16k.wav")
|
||||
cmd := exec.CommandContext(ctx, "ffmpeg", "-nostdin", "-y", "-i", inPath, "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le", wavPath)
|
||||
output, runErr := cmd.CombinedOutput()
|
||||
if runErr != nil {
|
||||
cleanup()
|
||||
detail := strings.TrimSpace(string(output))
|
||||
if len(detail) > 400 {
|
||||
detail = detail[len(detail)-400:]
|
||||
}
|
||||
return "", nil, fmt.Errorf("音频预处理失败(ffmpeg): %s", detail)
|
||||
}
|
||||
return wavPath, cleanup, nil
|
||||
}
|
||||
|
||||
func extForMime(mime string) string {
|
||||
switch {
|
||||
case strings.HasPrefix(mime, "video/mp4"), strings.HasPrefix(mime, "audio/mp4"):
|
||||
return ".mp4"
|
||||
case strings.HasPrefix(mime, "video/webm"), strings.HasPrefix(mime, "audio/webm"):
|
||||
return ".webm"
|
||||
case strings.HasPrefix(mime, "video/quicktime"):
|
||||
return ".mov"
|
||||
case strings.HasPrefix(mime, "audio/mpeg"), strings.HasPrefix(mime, "audio/mp3"):
|
||||
return ".mp3"
|
||||
case strings.HasPrefix(mime, "audio/wav"), strings.HasPrefix(mime, "audio/x-wav"), strings.HasPrefix(mime, "audio/wave"):
|
||||
return ".wav"
|
||||
case strings.HasPrefix(mime, "audio/flac"):
|
||||
return ".flac"
|
||||
case strings.HasPrefix(mime, "audio/aac"):
|
||||
return ".aac"
|
||||
case strings.HasPrefix(mime, "audio/ogg"), strings.HasPrefix(mime, "video/ogg"):
|
||||
return ".ogg"
|
||||
default:
|
||||
return ".bin"
|
||||
}
|
||||
}
|
||||
|
||||
// TimelineTranscriptionCreateRequest 是画布提交字幕转写任务的入参。
|
||||
type TimelineTranscriptionCreateRequest struct {
|
||||
ResourceID string `json:"resourceId"`
|
||||
Language string `json:"language"`
|
||||
ProjectID string `json:"projectId"`
|
||||
}
|
||||
|
||||
// CreateTimelineTranscriptionTask 创建时间线字幕转写任务:转写由本地
|
||||
// whisper.cpp 执行,不经模型路由与计费;创建时校验功能门控、资源归属
|
||||
// 与可转写类型,排队计入 active 限额。
|
||||
func (s *Service) CreateTimelineTranscriptionTask(userID string, req TimelineTranscriptionCreateRequest) (*model.Task, error) {
|
||||
if s.IsDraining() {
|
||||
return nil, &AppError{Status: 503, Code: 503, Message: "服务正在维护,暂不接受新的生成任务", Retryable: true}
|
||||
}
|
||||
if err := s.RequireFeature(FeatureTimelineTranscription); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resourceID := strings.TrimSpace(req.ResourceID)
|
||||
if resourceID == "" {
|
||||
return nil, BadAuthRequest("必须指定待转写媒体")
|
||||
}
|
||||
resource, err := s.Resource(userID, resourceID)
|
||||
if err != nil || resource == nil {
|
||||
return nil, BadAuthRequest("无法读取待转写媒体,可能已被删除")
|
||||
}
|
||||
if !isTranscribableMime(resource.MimeType) {
|
||||
return nil, BadAuthRequest("仅支持音视频文件转写")
|
||||
}
|
||||
policy, err := s.RuntimePolicy()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input := timelineTranscriptionInput{ResourceID: resourceID, Language: strings.TrimSpace(req.Language)}
|
||||
inputJSON, _ := json.Marshal(input)
|
||||
task := model.Task{
|
||||
ID: newID(), UserID: userID, ProjectID: req.ProjectID,
|
||||
Type: model.TaskTypeTimelineTranscription, Status: model.TaskStatusQueued,
|
||||
Stage: "等待队列调度", Progress: 5, Prompt: "字幕转写",
|
||||
Provider: "local", Model: "whisper.cpp", InputJSON: string(inputJSON),
|
||||
}
|
||||
if err := s.createTaskWithinStorageQuota(&task, nil, policy); err != nil {
|
||||
if errors.Is(err, repository.ErrActiveTaskLimit) {
|
||||
return nil, BadAuthRequest(fmt.Sprintf("同时排队或运行的任务最多 %d 个,请等待已有任务完成", policy.Task.ActiveTaskLimit))
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
s.recordActivity(userID, "task", 1)
|
||||
_ = s.log(userID, task.ID, "info", "字幕转写任务已进入队列", "")
|
||||
return taskForOutput(task), nil
|
||||
}
|
||||
|
||||
// TimelineRenderCreateRequest 是画布提交时间线渲染任务的入参;
|
||||
// Timeline 为前端 TimelineProject 快照(v2:tracks/clips 平铺)。
|
||||
// 片段通过 directMedia.storageKey=resource:<id> 引用后端资源。
|
||||
type TimelineRenderCreateRequest struct {
|
||||
ProjectID string `json:"projectId"`
|
||||
Timeline renderProject `json:"timeline"`
|
||||
}
|
||||
|
||||
type timelineRenderInput struct {
|
||||
ProjectID string `json:"projectId"`
|
||||
Timeline renderProject `json:"timeline"`
|
||||
}
|
||||
|
||||
type timelineRenderResult struct {
|
||||
ResourceID string `json:"resourceId"`
|
||||
FileName string `json:"fileName"`
|
||||
Size int64 `json:"size"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
SubtitleSRT string `json:"subtitleSrt,omitempty"`
|
||||
}
|
||||
|
||||
// CreateTimelineRenderTask 创建时间线渲染任务:本地 ffmpeg 合成,不经模型
|
||||
// 路由与计费;创建时校验快照至少包含一个可渲染媒体片段。
|
||||
func (s *Service) CreateTimelineRenderTask(userID string, req TimelineRenderCreateRequest) (*model.Task, error) {
|
||||
if s.IsDraining() {
|
||||
return nil, &AppError{Status: 503, Code: 503, Message: "服务正在维护,暂不接受新的生成任务", Retryable: true}
|
||||
}
|
||||
plan := buildRenderPlan(req.Timeline)
|
||||
if !plan.HasMedia {
|
||||
return nil, BadAuthRequest("时间线没有可渲染的媒体片段")
|
||||
}
|
||||
policy, err := s.RuntimePolicy()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input := timelineRenderInput{ProjectID: strings.TrimSpace(req.ProjectID), Timeline: req.Timeline}
|
||||
inputJSON, _ := json.Marshal(input)
|
||||
task := model.Task{
|
||||
ID: newID(), UserID: userID, ProjectID: strings.TrimSpace(req.ProjectID),
|
||||
Type: model.TaskTypeTimelineRender, Status: model.TaskStatusQueued,
|
||||
Stage: "等待队列调度", Progress: 5, Prompt: "时间线渲染",
|
||||
Provider: "local", Model: "ffmpeg", InputJSON: string(inputJSON),
|
||||
}
|
||||
if err := s.createTaskWithinStorageQuota(&task, nil, policy); err != nil {
|
||||
if errors.Is(err, repository.ErrActiveTaskLimit) {
|
||||
return nil, BadAuthRequest(fmt.Sprintf("同时排队或运行的任务最多 %d 个,请等待已有任务完成", policy.Task.ActiveTaskLimit))
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
s.recordActivity(userID, "task", 1)
|
||||
_ = s.log(userID, task.ID, "info", "时间线渲染任务已进入队列", "")
|
||||
return taskForOutput(task), nil
|
||||
}
|
||||
@@ -0,0 +1,150 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gorm.io/driver/sqlite"
|
||||
|
||||
"gorm.io/gorm"
|
||||
"infinite-canvas/backend/internal/database"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
"infinite-canvas/backend/internal/repository"
|
||||
)
|
||||
|
||||
func newTimelineTaskTestService(t *testing.T) (*Service, *gorm.DB) {
|
||||
t.Helper()
|
||||
// 每个测试独立的 sqlite 内存库,避免共享缓存串库。
|
||||
dsn := fmt.Sprintf("file:%s_%d?mode=memory&cache=shared", t.Name(), time.Now().UnixNano())
|
||||
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
|
||||
t.Helper()
|
||||
if err != nil {
|
||||
t.Fatalf("open sqlite: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(database.Models()...); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
return &Service{repo: repository.New(db)}, db
|
||||
}
|
||||
|
||||
func seedRunningTimelineTask(t *testing.T, db *gorm.DB, inputJSON string) *model.Task {
|
||||
t.Helper()
|
||||
task := &model.Task{
|
||||
ID: fmt.Sprintf("tsk-timeline-%d", time.Now().UnixNano()),
|
||||
UserID: "usr-timeline-test",
|
||||
Type: model.TaskTypeTimelineTranscription,
|
||||
Status: model.TaskStatusRunning,
|
||||
Stage: "已领取",
|
||||
InputJSON: inputJSON,
|
||||
}
|
||||
if err := db.Create(task).Error; err != nil {
|
||||
t.Fatalf("seed task: %v", err)
|
||||
}
|
||||
return task
|
||||
}
|
||||
|
||||
func TestTimelineTranscriptionFailsFastWhenWhisperUnconfigured(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
task := seedRunningTimelineTask(t, db, `{"resourceId":"res-1","language":""}`)
|
||||
w := newTaskWorkerCoordinator(svc)
|
||||
|
||||
t.Setenv(whisperLangEnv, "")
|
||||
err := w.processTimelineTranscription(task, context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("process: want nil (task terminal handled internally), got %v", err)
|
||||
}
|
||||
var stored model.Task
|
||||
if err := db.First(&stored, "id = ?", task.ID).Error; err != nil {
|
||||
t.Fatalf("load task: %v", err)
|
||||
}
|
||||
if stored.Status != model.TaskStatusFailed {
|
||||
t.Fatalf("status = %q, want failed", stored.Status)
|
||||
}
|
||||
if !strings.Contains(stored.Error, "CANVAS_WHISPER_BASE_URL") {
|
||||
t.Fatalf("error = %q, want mention of CANVAS_WHISPER_BASE_URL", stored.Error)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimelineTranscriptionRejectsMissingResourceRef(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
task := seedRunningTimelineTask(t, db, `{"resourceId":" "}`)
|
||||
w := newTaskWorkerCoordinator(svc)
|
||||
|
||||
t.Setenv(whisperLangEnv, "http://127.0.0.1:9999")
|
||||
err := w.processTimelineTranscription(task, context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("process: want nil (task terminal handled internally), got %v", err)
|
||||
}
|
||||
var stored model.Task
|
||||
if err := db.First(&stored, "id = ?", task.ID).Error; err != nil {
|
||||
t.Fatalf("load task: %v", err)
|
||||
}
|
||||
if stored.Status != model.TaskStatusFailed {
|
||||
t.Fatalf("status = %q, want failed", stored.Status)
|
||||
}
|
||||
if !strings.Contains(stored.Error, "资源引用") {
|
||||
t.Fatalf("error = %q, want mention of 资源引用", stored.Error)
|
||||
}
|
||||
}
|
||||
|
||||
func seedResource(t *testing.T, db *gorm.DB, id string, userID string, mime string) {
|
||||
t.Helper()
|
||||
if err := db.Create(&model.Resource{ID: id, UserID: userID, Kind: "media", Status: model.ResourceStatusReady, Provider: "local", MimeType: mime, Size: 1024}).Error; err != nil {
|
||||
t.Fatalf("seed resource: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTimelineTranscriptionTaskQueues(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
seedResource(t, db, "res-video-1", "usr-create-test", "video/mp4")
|
||||
|
||||
task, err := svc.CreateTimelineTranscriptionTask("usr-create-test", TimelineTranscriptionCreateRequest{ResourceID: "res-video-1", Language: "zh", ProjectID: "prj-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("create: %v", err)
|
||||
}
|
||||
if task.Type != model.TaskTypeTimelineTranscription {
|
||||
t.Fatalf("type = %q, want timeline_transcription", task.Type)
|
||||
}
|
||||
if task.Status != model.TaskStatusQueued {
|
||||
t.Fatalf("status = %q, want queued", task.Status)
|
||||
}
|
||||
var stored model.Task
|
||||
if err := db.First(&stored, "id = ?", task.ID).Error; err != nil {
|
||||
t.Fatalf("load task: %v", err)
|
||||
}
|
||||
if stored.UserID != "usr-create-test" || stored.ProjectID != "prj-1" {
|
||||
t.Fatalf("owner/project mismatch: %q / %q", stored.UserID, stored.ProjectID)
|
||||
}
|
||||
if stored.Provider != "local" || stored.Model != "whisper.cpp" {
|
||||
t.Fatalf("provider/model = %q/%q, want local/whisper.cpp", stored.Provider, stored.Model)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTimelineTranscriptionTaskRejectsForeignResource(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
seedResource(t, db, "res-owner-a", "usr-a", "video/mp4")
|
||||
|
||||
_, err := svc.CreateTimelineTranscriptionTask("usr-b", TimelineTranscriptionCreateRequest{ResourceID: "res-owner-a"})
|
||||
if err == nil || !strings.Contains(err.Error(), "媒体") {
|
||||
t.Fatalf("want 无法读取待转写媒体 error, got %v", err)
|
||||
}
|
||||
// 另一用户自己的音视频资源可正常创建。
|
||||
seedResource(t, db, "res-b", "usr-b", "audio/mpeg")
|
||||
if _, err := svc.CreateTimelineTranscriptionTask("usr-b", TimelineTranscriptionCreateRequest{ResourceID: "res-b"}); err != nil {
|
||||
t.Fatalf("own audio resource should create: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTimelineTranscriptionTaskRejectsNonTranscribable(t *testing.T) {
|
||||
svc, db := newTimelineTaskTestService(t)
|
||||
seedResource(t, db, "res-img", "usr-img", "image/png")
|
||||
|
||||
_, err := svc.CreateTimelineTranscriptionTask("usr-img", TimelineTranscriptionCreateRequest{ResourceID: "res-img"})
|
||||
if err == nil || !strings.Contains(err.Error(), "音视频") {
|
||||
t.Fatalf("want 仅支持音视频 error, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -130,6 +130,15 @@ func (w *taskWorkerCoordinator) processClaimedTask(task *model.Task) error {
|
||||
return terminal.handleAlreadyCancelled(*latest)
|
||||
}
|
||||
|
||||
// 时间线转写由本地 whisper.cpp 执行,不经计费与模型渠道路由,
|
||||
// 在进入通用生成流程前按类型分叉到独立执行器。
|
||||
if task.Type == model.TaskTypeTimelineTranscription {
|
||||
return w.processTimelineTranscription(task, ctx)
|
||||
}
|
||||
if task.Type == model.TaskTypeTimelineRender {
|
||||
return w.processTimelineRender(task, ctx)
|
||||
}
|
||||
|
||||
task.Stage = "调用生成模型"
|
||||
task.Progress = 35
|
||||
if taskUsesUpstreamReportedProgress(task.Type) {
|
||||
@@ -231,6 +240,12 @@ func taskExecutionTimeoutWithPolicy(taskType string, policy RuntimeTaskPolicy) t
|
||||
return time.Duration(policy.AudioTimeoutMinutes) * time.Minute
|
||||
case strings.HasPrefix(taskType, "canvas_text"):
|
||||
return time.Duration(policy.TextTimeoutMinutes) * time.Minute
|
||||
case taskType == model.TaskTypeTimelineTranscription:
|
||||
// 本地转写受媒体时长影响,给出独立宽松超时。
|
||||
return 20 * time.Minute
|
||||
case taskType == model.TaskTypeTimelineRender:
|
||||
// 渲染是整条时间线的重编码,耗时随长度线性增长。
|
||||
return 60 * time.Minute
|
||||
default:
|
||||
return time.Duration(policy.DefaultTimeoutMinutes) * time.Minute
|
||||
}
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// 时间线渲染:把前端 TimelineProject 快照(tracks/clips 平铺,见
|
||||
// web/src/types/timeline.ts TimelineProject v2)展开为可执行的 ffmpeg 步骤。
|
||||
// 与前端 timeline-to-ffmpeg.ts 保持同一数据流:trim(按 sourceStartMs 裁切)
|
||||
// → 空隙补黑场静音 → concat。字幕不烧录(依赖 libass),单独产出 SRT。
|
||||
//
|
||||
// 输入布局统一为「每个片段两个输入:视频源 + 音频源」,空隙展开为独立
|
||||
// 黑场片段,因此第 i 个片段的视频输入为 2i、音频输入为 2i+1。
|
||||
|
||||
type renderClip struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
TrackID string `json:"trackId"`
|
||||
StartMs int64 `json:"startMs"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
SourceStartMs int64 `json:"sourceStartMs"`
|
||||
SourceDurationMs int64 `json:"sourceDurationMs"`
|
||||
Volume float64 `json:"volume"`
|
||||
Text string `json:"text"`
|
||||
DirectMedia *struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
StorageKey string `json:"storageKey"`
|
||||
} `json:"directMedia"`
|
||||
}
|
||||
|
||||
type renderTrack struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Visible *bool `json:"visible"`
|
||||
Muted bool `json:"muted"`
|
||||
}
|
||||
|
||||
type renderProject struct {
|
||||
Version int `json:"version"`
|
||||
Tracks []renderTrack `json:"tracks"`
|
||||
Clips []renderClip `json:"clips"`
|
||||
DurationMs int64 `json:"durationMs"`
|
||||
}
|
||||
|
||||
type renderSource struct {
|
||||
ResourceID string
|
||||
Clip renderClip
|
||||
Path string
|
||||
Ext string
|
||||
// HasAudio 由 ffprobe 探测得到;无音轨的媒体改为静音源,避免 map 失败。
|
||||
HasAudio bool
|
||||
}
|
||||
|
||||
// renderSegment 是渲染序列中的一段;Kind 为 video/image 表示媒体片段,
|
||||
// gap 表示补黑场静音的空隙段(无 Source)。
|
||||
type renderSegment struct {
|
||||
Kind string
|
||||
DurationMs int64
|
||||
GapMs int64
|
||||
Clip renderClip
|
||||
Source *renderSource
|
||||
}
|
||||
|
||||
type renderPlan struct {
|
||||
Segments []renderSegment
|
||||
SubtitleSRT string
|
||||
HasMedia bool
|
||||
}
|
||||
|
||||
// buildRenderPlan 挑选可见视频/图片轨片段按 startMs 排序,并把片段之间的
|
||||
// 空隙展开为黑场段,使渲染序列在时间轴上连续。
|
||||
func buildRenderPlan(project renderProject) renderPlan {
|
||||
visible := map[string]bool{}
|
||||
for _, track := range project.Tracks {
|
||||
visible[track.ID] = track.Visible == nil || *track.Visible
|
||||
}
|
||||
clips := make([]renderClip, 0, len(project.Clips))
|
||||
for _, clip := range project.Clips {
|
||||
if !visible[clip.TrackID] {
|
||||
continue
|
||||
}
|
||||
if clip.Kind != "video" && clip.Kind != "image" {
|
||||
continue
|
||||
}
|
||||
if clip.DurationMs <= 0 {
|
||||
continue
|
||||
}
|
||||
clips = append(clips, clip)
|
||||
}
|
||||
sort.SliceStable(clips, func(i, j int) bool {
|
||||
if clips[i].StartMs == clips[j].StartMs {
|
||||
return clips[i].ID < clips[j].ID
|
||||
}
|
||||
return clips[i].StartMs < clips[j].StartMs
|
||||
})
|
||||
|
||||
plan := renderPlan{SubtitleSRT: buildRenderSubtitleSRT(project)}
|
||||
cursor := int64(0)
|
||||
for _, clip := range clips {
|
||||
if gap := clip.StartMs - cursor; gap > 0 {
|
||||
plan.Segments = append(plan.Segments, renderSegment{Kind: "gap", DurationMs: gap, GapMs: gap})
|
||||
}
|
||||
plan.Segments = append(plan.Segments, renderSegment{Kind: clip.Kind, DurationMs: clip.DurationMs, Clip: clip})
|
||||
cursor = clip.StartMs + clip.DurationMs
|
||||
}
|
||||
for _, seg := range plan.Segments {
|
||||
if _, ok := mediaResourceID(seg.Clip); ok {
|
||||
plan.HasMedia = true
|
||||
break
|
||||
}
|
||||
}
|
||||
return plan
|
||||
}
|
||||
|
||||
// mediaResourceID 从片段的 directMedia.storageKey(resource:<id>)还原资源 ID。
|
||||
func mediaResourceID(clip renderClip) (string, bool) {
|
||||
if clip.DirectMedia == nil {
|
||||
return "", false
|
||||
}
|
||||
key := strings.TrimSpace(clip.DirectMedia.StorageKey)
|
||||
if !strings.HasPrefix(key, "resource:") {
|
||||
return "", false
|
||||
}
|
||||
id := strings.TrimSpace(strings.TrimPrefix(key, "resource:"))
|
||||
return id, id != ""
|
||||
}
|
||||
|
||||
func buildRenderSubtitleSRT(project renderProject) string {
|
||||
subtitle := make([]renderClip, 0, len(project.Clips))
|
||||
for _, clip := range project.Clips {
|
||||
if clip.Kind != "subtitle" || strings.TrimSpace(clip.Text) == "" || clip.DurationMs <= 0 {
|
||||
continue
|
||||
}
|
||||
subtitle = append(subtitle, clip)
|
||||
}
|
||||
sort.SliceStable(subtitle, func(i, j int) bool {
|
||||
if subtitle[i].StartMs == subtitle[j].StartMs {
|
||||
return subtitle[i].ID < subtitle[j].ID
|
||||
}
|
||||
return subtitle[i].StartMs < subtitle[j].StartMs
|
||||
})
|
||||
if len(subtitle) == 0 {
|
||||
return ""
|
||||
}
|
||||
var out strings.Builder
|
||||
for i, clip := range subtitle {
|
||||
fmt.Fprintf(&out, "%d\n%s --> %s\n%s\n\n",
|
||||
i+1,
|
||||
formatSRTTimestamp(clip.StartMs),
|
||||
formatSRTTimestamp(clip.StartMs+clip.DurationMs),
|
||||
strings.TrimSpace(clip.Text))
|
||||
}
|
||||
return out.String()
|
||||
}
|
||||
|
||||
const (
|
||||
renderWidth = 1920
|
||||
renderHeight = 1080
|
||||
renderFPS = 30
|
||||
renderSampleRate = 44100
|
||||
)
|
||||
|
||||
// buildRenderFFmpegArgs 依据渲染计划生成 ffmpeg 参数(工作目录内相对路径)。
|
||||
// 每段固定两个输入:视频源与音频源,末尾 concat 为单路输出。
|
||||
func buildRenderFFmpegArgs(plan renderPlan, target string) []string {
|
||||
if len(plan.Segments) == 0 {
|
||||
return nil
|
||||
}
|
||||
scale := fmt.Sprintf("scale=%d:%d:force_original_aspect_ratio=decrease,pad=%d:%d:(ow-iw)/2:(oh-ih)/2",
|
||||
renderWidth, renderHeight, renderWidth, renderHeight)
|
||||
args := []string{"-nostdin", "-y"}
|
||||
for _, seg := range plan.Segments {
|
||||
seconds := fmt.Sprintf("%.3f", float64(seg.DurationMs)/1000)
|
||||
switch seg.Kind {
|
||||
case "gap":
|
||||
args = append(args,
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("color=c=black:s=%dx%d:r=%d", renderWidth, renderHeight, renderFPS),
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("anullsrc=r=%d:cl=stereo", renderSampleRate),
|
||||
)
|
||||
case "image":
|
||||
if seg.Source == nil {
|
||||
args = append(args,
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("color=c=black:s=%dx%d:r=%d", renderWidth, renderHeight, renderFPS),
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("anullsrc=r=%d:cl=stereo", renderSampleRate),
|
||||
)
|
||||
continue
|
||||
}
|
||||
args = append(args,
|
||||
"-loop", "1", "-t", seconds, "-i", seg.Source.Path,
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("anullsrc=r=%d:cl=stereo", renderSampleRate),
|
||||
)
|
||||
default:
|
||||
if seg.Source == nil {
|
||||
args = append(args,
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("color=c=black:s=%dx%d:r=%d", renderWidth, renderHeight, renderFPS),
|
||||
"-f", "lavfi", "-t", seconds,
|
||||
"-i", fmt.Sprintf("anullsrc=r=%d:cl=stereo", renderSampleRate),
|
||||
)
|
||||
continue
|
||||
}
|
||||
if seg.Clip.SourceStartMs > 0 {
|
||||
args = append(args, "-ss", fmt.Sprintf("%.3f", float64(seg.Clip.SourceStartMs)/1000))
|
||||
}
|
||||
args = append(args, "-t", seconds, "-i", seg.Source.Path)
|
||||
// 音频输入复用同一媒体文件;无音轨时 ffmpeg 会因 map 失败,
|
||||
// 由调用方探测后回退为静音源(见 renderAudioFallback)。
|
||||
if seg.Clip.SourceStartMs > 0 {
|
||||
args = append(args, "-ss", fmt.Sprintf("%.3f", float64(seg.Clip.SourceStartMs)/1000))
|
||||
}
|
||||
args = append(args, "-t", seconds, "-i", seg.Source.Path)
|
||||
}
|
||||
}
|
||||
|
||||
filters := make([]string, 0, len(plan.Segments)*2+1)
|
||||
labels := make([]string, 0, len(plan.Segments)*2)
|
||||
for i, seg := range plan.Segments {
|
||||
videoIn := 2 * i
|
||||
audioIn := 2*i + 1
|
||||
videoLabel := fmt.Sprintf("v%d", i)
|
||||
audioLabel := fmt.Sprintf("a%d", i)
|
||||
if seg.Source != nil && seg.Kind != "image" && !seg.Source.HasAudio {
|
||||
// 无音轨媒体:改用静音源,避免 map 失败。
|
||||
audioIn = videoIn
|
||||
filters = append(filters,
|
||||
fmt.Sprintf("[%d:v]fps=%d,%s,setsar=1[%s]", videoIn, renderFPS, scale, videoLabel))
|
||||
silentLabel := fmt.Sprintf("silent%d", i)
|
||||
filters = append(filters,
|
||||
fmt.Sprintf("anullsrc=r=%d:cl=stereo[%s]", renderSampleRate, silentLabel))
|
||||
filters = append(filters,
|
||||
fmt.Sprintf("[%s]atrim=0:%.3f,asetpts=N/SR/TB[%s]", silentLabel, float64(seg.DurationMs)/1000, audioLabel))
|
||||
} else {
|
||||
volume := 1.0
|
||||
if seg.Clip.Volume > 0 {
|
||||
volume = seg.Clip.Volume
|
||||
}
|
||||
filters = append(filters,
|
||||
fmt.Sprintf("[%d:v]fps=%d,%s,setsar=1[%s]", videoIn, renderFPS, scale, videoLabel))
|
||||
filters = append(filters,
|
||||
fmt.Sprintf("[%d:a]aformat=sample_fmts=fltp:sample_rates=%d:channel_layouts=stereo,volume=%.3f,apad[%s]",
|
||||
audioIn, renderSampleRate, volume, audioLabel))
|
||||
}
|
||||
labels = append(labels, videoLabel, audioLabel)
|
||||
}
|
||||
var concatInputs strings.Builder
|
||||
for _, label := range labels {
|
||||
fmt.Fprintf(&concatInputs, "[%s]", label)
|
||||
}
|
||||
filters = append(filters, fmt.Sprintf("%sconcat=n=%d:v=1:a=1[vout][aout]", concatInputs.String(), len(plan.Segments)))
|
||||
|
||||
args = append(args,
|
||||
"-filter_complex", strings.Join(filters, ";"),
|
||||
"-map", "[vout]", "-map", "[aout]",
|
||||
"-c:v", "libx264", "-preset", "veryfast", "-crf", "23", "-pix_fmt", "yuv420p",
|
||||
"-c:a", "aac", "-b:a", "128k", "-movflags", "+faststart",
|
||||
"-t", fmt.Sprintf("%.3f", planTotalSeconds(plan)),
|
||||
target,
|
||||
)
|
||||
return args
|
||||
}
|
||||
|
||||
func planTotalSeconds(plan renderPlan) float64 {
|
||||
var total int64
|
||||
for _, seg := range plan.Segments {
|
||||
total += seg.DurationMs
|
||||
}
|
||||
return float64(total) / 1000
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func renderClipFixture(id string, kind string, trackID string, startMs int64, durationMs int64, storageKey string) renderClip {
|
||||
clip := renderClip{ID: id, Kind: kind, TrackID: trackID, StartMs: startMs, DurationMs: durationMs, Volume: 1}
|
||||
if storageKey != "" {
|
||||
clip.DirectMedia = &struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
StorageKey string `json:"storageKey"`
|
||||
}{ID: "media-" + id, Kind: kind, StorageKey: storageKey}
|
||||
}
|
||||
return clip
|
||||
}
|
||||
|
||||
func TestBuildRenderPlanExpandsGapAndSorts(t *testing.T) {
|
||||
project := renderProject{
|
||||
Version: 2,
|
||||
Tracks: []renderTrack{{ID: "track-video-1", Kind: "video"}},
|
||||
Clips: []renderClip{
|
||||
renderClipFixture("clip-b", "video", "track-video-1", 3000, 2000, "resource:res-b"),
|
||||
renderClipFixture("clip-a", "video", "track-video-1", 0, 2000, "resource:res-a"),
|
||||
renderClipFixture("clip-sub", "subtitle", "track-subtitle-1", 0, 1000, ""),
|
||||
},
|
||||
}
|
||||
plan := buildRenderPlan(project)
|
||||
if !plan.HasMedia {
|
||||
t.Fatal("HasMedia = false, want true")
|
||||
}
|
||||
// 前 0–3000ms 被 clip-a(0-2000) 覆盖后仍有 1000ms 空隙,应展开为黑场段。
|
||||
if len(plan.Segments) != 3 {
|
||||
t.Fatalf("segments = %d, want 3 (clip-a, gap, clip-b)", len(plan.Segments))
|
||||
}
|
||||
if plan.Segments[0].Kind != "video" || plan.Segments[0].Clip.ID != "clip-a" {
|
||||
t.Fatalf("seg0 = %s/%s, want video/clip-a", plan.Segments[0].Kind, plan.Segments[0].Clip.ID)
|
||||
}
|
||||
if plan.Segments[1].Kind != "gap" || plan.Segments[1].DurationMs != 1000 {
|
||||
t.Fatalf("seg1 = %s/%dms, want gap/1000ms", plan.Segments[1].Kind, plan.Segments[1].DurationMs)
|
||||
}
|
||||
if plan.Segments[2].Clip.ID != "clip-b" {
|
||||
t.Fatalf("seg2 = %s, want clip-b (sorted by startMs)", plan.Segments[2].Clip.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRenderPlanSkipsHiddenTrack(t *testing.T) {
|
||||
hidden := false
|
||||
project := renderProject{
|
||||
Version: 2,
|
||||
Tracks: []renderTrack{{ID: "track-hidden", Kind: "video", Visible: &hidden}},
|
||||
Clips: []renderClip{renderClipFixture("clip-hidden", "video", "track-hidden", 0, 1000, "resource:res-x")},
|
||||
}
|
||||
plan := buildRenderPlan(project)
|
||||
if len(plan.Segments) != 0 {
|
||||
t.Fatalf("segments = %d, want 0 (hidden track skipped)", len(plan.Segments))
|
||||
}
|
||||
if plan.HasMedia {
|
||||
t.Fatal("HasMedia = true, want false for hidden-only timeline")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRenderSubtitleSRT(t *testing.T) {
|
||||
project := renderProject{
|
||||
Version: 2,
|
||||
Clips: []renderClip{
|
||||
{ID: "sub-b", Kind: "subtitle", StartMs: 2000, DurationMs: 1000, Text: "第二条"},
|
||||
{ID: "sub-a", Kind: "subtitle", StartMs: 0, DurationMs: 1000, Text: "第一条"},
|
||||
{ID: "sub-blank", Kind: "subtitle", StartMs: 5000, DurationMs: 1000, Text: " "},
|
||||
},
|
||||
}
|
||||
srt := buildRenderSubtitleSRT(project)
|
||||
want := "1\n00:00:00,000 --> 00:00:01,000\n第一条\n\n2\n00:00:02,000 --> 00:00:03,000\n第二条\n\n"
|
||||
if srt != want {
|
||||
t.Fatalf("srt mismatch:\n got %q\nwant %q", srt, want)
|
||||
}
|
||||
if buildRenderSubtitleSRT(renderProject{Version: 2}) != "" {
|
||||
t.Fatal("srt for empty timeline: want empty string")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRenderFFmpegArgsLayout(t *testing.T) {
|
||||
plan := buildRenderPlan(renderProject{
|
||||
Version: 2,
|
||||
Tracks: []renderTrack{{ID: "track-video-1", Kind: "video"}},
|
||||
Clips: []renderClip{
|
||||
renderClipFixture("clip-a", "video", "track-video-1", 0, 2000, "resource:res-a"),
|
||||
renderClipFixture("clip-b", "video", "track-video-1", 5000, 1000, "resource:res-b"),
|
||||
},
|
||||
})
|
||||
plan.Segments[0].Source = &renderSource{ResourceID: "res-a", Path: "src-0.mp4", HasAudio: true}
|
||||
plan.Segments[1].Source = &renderSource{ResourceID: "res-a", Path: "src-0.mp4", HasAudio: true}
|
||||
plan.Segments[2].Source = &renderSource{ResourceID: "res-b", Path: "src-1.mp4", HasAudio: true}
|
||||
|
||||
args := buildRenderFFmpegArgs(plan, "render-output.mp4")
|
||||
joined := strings.Join(args, " ")
|
||||
if len(args) == 0 {
|
||||
t.Fatal("args empty, want ffmpeg arguments")
|
||||
}
|
||||
for _, want := range []string{
|
||||
"-filter_complex",
|
||||
"concat=n=3:v=1:a=1",
|
||||
"[vout]", "[aout]",
|
||||
"libx264", "aac",
|
||||
"color=c=black:s=1920x1080:r=30",
|
||||
"anullsrc=r=44100:cl=stereo",
|
||||
"render-output.mp4",
|
||||
} {
|
||||
if !strings.Contains(joined, want) {
|
||||
t.Fatalf("args missing %q: %s", want, joined)
|
||||
}
|
||||
}
|
||||
// 每段两个输入:视频 + 音频;末尾输入索引应为 5(3 段 × 2)。
|
||||
if !strings.Contains(joined, "[5:a]") {
|
||||
t.Fatalf("args missing final audio input label [5:a]: %s", joined)
|
||||
}
|
||||
if buildRenderFFmpegArgs(renderPlan{}, "out.mp4") != nil {
|
||||
t.Fatal("args for empty plan: want nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildRenderFFmpegArgsSilentFallbackUniqueLabels(t *testing.T) {
|
||||
plan := renderPlan{Segments: []renderSegment{
|
||||
{Kind: "video", DurationMs: 1000, Source: &renderSource{Path: "src-0.mp4", HasAudio: false}},
|
||||
{Kind: "video", DurationMs: 1000, Source: &renderSource{Path: "src-1.mp4", HasAudio: false}},
|
||||
}}
|
||||
args := buildRenderFFmpegArgs(plan, "out.mp4")
|
||||
joined := strings.Join(args, " ")
|
||||
// 两段均无音轨:静音源标签必须按片段唯一,否则 filter_complex 报错。
|
||||
if !strings.Contains(joined, "[silent0]") || !strings.Contains(joined, "[silent1]") {
|
||||
t.Fatalf("silent labels not unique: %s", joined)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMediaResourceID(t *testing.T) {
|
||||
clip := renderClipFixture("a", "video", "track-1", 0, 1000, "resource:res-42")
|
||||
if id, ok := mediaResourceID(clip); !ok || id != "res-42" {
|
||||
t.Fatalf("mediaResourceID = %q/%v, want res-42/true", id, ok)
|
||||
}
|
||||
if _, ok := mediaResourceID(renderClip{}); ok {
|
||||
t.Fatal("mediaResourceID for clip without directMedia: want false")
|
||||
}
|
||||
if _, ok := mediaResourceID(renderClipFixture("a", "video", "track-1", 0, 1000, "https://example.com/a.mp4")); ok {
|
||||
t.Fatal("mediaResourceID for http storage key: want false")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// 本地 whisper.cpp HTTP 服务客户端。配置项 CANVAS_WHISPER_BASE_URL
|
||||
// 指向自建 whisper.cpp server(/inference,多部件 file 上传)。
|
||||
// 语音数据不出本机,未配置该地址时任务在进入转写前明确失败。
|
||||
|
||||
type timelineTranscriptionSegment struct {
|
||||
StartMs int64 `json:"startMs"`
|
||||
EndMs int64 `json:"endMs"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
type timelineTranscriptionResult struct {
|
||||
Segments []timelineTranscriptionSegment `json:"segments"`
|
||||
SRT string `json:"srt"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
type whisperSegment struct {
|
||||
Start float64 `json:"start"`
|
||||
End float64 `json:"end"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
type whisperVerboseJSON struct {
|
||||
Segments []whisperSegment `json:"segments"`
|
||||
Language string `json:"language"`
|
||||
}
|
||||
|
||||
type whisperClient struct {
|
||||
baseURL string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
func newWhisperClient(baseURL string) *whisperClient {
|
||||
return &whisperClient{
|
||||
baseURL: strings.TrimRight(strings.TrimSpace(baseURL), "/"),
|
||||
http: &http.Client{Timeout: 20 * time.Minute},
|
||||
}
|
||||
}
|
||||
|
||||
// transcribe 将本地 wav 送至 whisper.cpp /inference,返回段落与语言。
|
||||
func (c *whisperClient) transcribe(ctx context.Context, wavPath string, language string) ([]timelineTranscriptionSegment, string, error) {
|
||||
if c.baseURL == "" {
|
||||
return nil, "", fmt.Errorf("本地转写服务未配置:请设置 CANVAS_WHISPER_BASE_URL")
|
||||
}
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
audio, err := os.Open(wavPath)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("读取待转写音频失败: %w", err)
|
||||
}
|
||||
defer audio.Close()
|
||||
part, err := writer.CreateFormFile("file", filepath.Base(wavPath))
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if _, err := io.Copy(part, audio); err != nil {
|
||||
return nil, "", fmt.Errorf("上传音频失败: %w", err)
|
||||
}
|
||||
if err := writer.WriteField("response_format", "verbose_json"); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if strings.TrimSpace(language) != "" {
|
||||
if err := writer.WriteField("language", strings.TrimSpace(language)); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/inference", &body)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("无法连接本地转写服务(whisper.cpp): %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 2048))
|
||||
return nil, "", fmt.Errorf("本地转写服务返回 %d: %s", resp.StatusCode, strings.TrimSpace(string(msg)))
|
||||
}
|
||||
payload, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("读取转写结果失败: %w", err)
|
||||
}
|
||||
segments, languageOut, err := decodeWhisperVerboseJSON(payload)
|
||||
if err != nil {
|
||||
return nil, "", err
|
||||
}
|
||||
if len(segments) == 0 {
|
||||
return nil, "", fmt.Errorf("本地转写服务未识别出语音内容")
|
||||
}
|
||||
return segments, languageOut, nil
|
||||
}
|
||||
|
||||
// decodeWhisperVerboseJSON 解析 whisper.cpp verbose_json 响应为内部段落,
|
||||
// 过滤空文本,秒精度取整为毫秒。
|
||||
func decodeWhisperVerboseJSON(payload []byte) ([]timelineTranscriptionSegment, string, error) {
|
||||
var parsed whisperVerboseJSON
|
||||
if err := json.Unmarshal(payload, &parsed); err != nil {
|
||||
return nil, "", fmt.Errorf("转写结果解析失败: %w", err)
|
||||
}
|
||||
segments := make([]timelineTranscriptionSegment, 0, len(parsed.Segments))
|
||||
for _, seg := range parsed.Segments {
|
||||
if strings.TrimSpace(seg.Text) == "" {
|
||||
continue
|
||||
}
|
||||
segments = append(segments, timelineTranscriptionSegment{
|
||||
StartMs: int64(math.Round(seg.Start * 1000)),
|
||||
EndMs: int64(math.Round(seg.End * 1000)),
|
||||
Text: strings.TrimSpace(seg.Text),
|
||||
})
|
||||
}
|
||||
return segments, parsed.Language, nil
|
||||
}
|
||||
|
||||
// formatSRTTimestamp 将毫秒时间格式化为 SRT 时间戳 hh:mm:ss,mmm。
|
||||
func formatSRTTimestamp(ms int64) string {
|
||||
if ms < 0 {
|
||||
ms = 0
|
||||
}
|
||||
hours := ms / 3600000
|
||||
minutes := (ms % 3600000) / 60000
|
||||
seconds := (ms % 60000) / 1000
|
||||
millis := ms % 1000
|
||||
return fmt.Sprintf("%02d:%02d:%02d,%03d", hours, minutes, seconds, millis)
|
||||
}
|
||||
|
||||
// buildTimelineSRT 由段落生成 SRT 字幕文本,供前端直接消费或导出。
|
||||
func buildTimelineSRT(segments []timelineTranscriptionSegment) string {
|
||||
var out strings.Builder
|
||||
for i, seg := range segments {
|
||||
fmt.Fprintf(&out, "%d\n%s --> %s\n%s\n\n", i+1, formatSRTTimestamp(seg.StartMs), formatSRTTimestamp(seg.EndMs), seg.Text)
|
||||
}
|
||||
return out.String()
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDecodeWhisperVerboseJSON(t *testing.T) {
|
||||
payload := []byte(`{
|
||||
"task": "transcribe",
|
||||
"language": "zh",
|
||||
"duration": 6.12,
|
||||
"text": " 大家好 ",
|
||||
"segments": [
|
||||
{"id": 0, "start": 0.0, "end": 2.015, "text": " 大家好 "},
|
||||
{"id": 1, "start": 2.5, "end": 6.12, "text": " "},
|
||||
{"id": 2, "start": 4.12, "end": 6.121, "text": "今天天气不错"}
|
||||
]
|
||||
}`)
|
||||
segments, language, err := decodeWhisperVerboseJSON(payload)
|
||||
if err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if language != "zh" {
|
||||
t.Fatalf("language = %q, want zh", language)
|
||||
}
|
||||
// 空文本段被过滤,毫秒取整(0.0→0,2.5→2500)。
|
||||
if len(segments) != 2 {
|
||||
t.Fatalf("segments = %d, want 2 (blank segment filtered)", len(segments))
|
||||
}
|
||||
if segments[0].StartMs != 0 || segments[0].EndMs != 2015 {
|
||||
t.Fatalf("seg0 = %d-%d, want 0-2015", segments[0].StartMs, segments[0].EndMs)
|
||||
}
|
||||
if segments[0].Text != "大家好" {
|
||||
t.Fatalf("seg0 text = %q, want 大家好 (trimmed)", segments[0].Text)
|
||||
}
|
||||
if segments[1].StartMs != 4120 || segments[1].EndMs != 6121 {
|
||||
t.Fatalf("seg1 = %d-%d, want 4120-6121", segments[1].StartMs, segments[1].EndMs)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeWhisperVerboseJSONInvalid(t *testing.T) {
|
||||
if _, _, err := decodeWhisperVerboseJSON([]byte(`{not json`)); err == nil {
|
||||
t.Fatal("decode invalid payload: want error")
|
||||
}
|
||||
// 空段落由 decode 正常返回空切片,守卫在 transcribe 层(避免写出空字幕)。
|
||||
segments, _, err := decodeWhisperVerboseJSON([]byte(`{"text":"","segments":[]}`))
|
||||
if err != nil {
|
||||
t.Fatalf("decode empty segments: unexpected error %v", err)
|
||||
}
|
||||
if len(segments) != 0 {
|
||||
t.Fatalf("decode empty segments: got %d, want 0", len(segments))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatSRTTimestamp(t *testing.T) {
|
||||
cases := []struct {
|
||||
ms int64
|
||||
want string
|
||||
}{
|
||||
{0, "00:00:00,000"},
|
||||
{2015, "00:00:02,015"},
|
||||
{61001, "00:01:01,001"},
|
||||
{3661001, "01:01:01,001"},
|
||||
{-5, "00:00:00,000"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := formatSRTTimestamp(c.ms); got != c.want {
|
||||
t.Fatalf("formatSRTTimestamp(%d) = %q, want %q", c.ms, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildTimelineSRT(t *testing.T) {
|
||||
segments := []timelineTranscriptionSegment{
|
||||
{StartMs: 0, EndMs: 2015, Text: "大家好"},
|
||||
{StartMs: 2500, EndMs: 6121, Text: "今天天气不错"},
|
||||
}
|
||||
srt := buildTimelineSRT(segments)
|
||||
want := "1\n00:00:00,000 --> 00:00:02,015\n大家好\n\n2\n00:00:02,500 --> 00:00:06,121\n今天天气不错\n\n"
|
||||
if srt != want {
|
||||
t.Fatalf("srt mismatch:\n got %q\nwant %q", srt, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtForMimeAndTranscribable(t *testing.T) {
|
||||
transcribable := []string{"video/mp4", "audio/wav", "audio/mpeg", "video/webm", "audio/flac", "video/quicktime", "audio/aac", "audio/ogg"}
|
||||
for _, mime := range transcribable {
|
||||
if !isTranscribableMime(mime) {
|
||||
t.Fatalf("isTranscribableMime(%q) = false, want true", mime)
|
||||
}
|
||||
if strings.TrimPrefix(extForMime(mime), ".") == "bin" {
|
||||
t.Fatalf("extForMime(%q) fell back to .bin", mime)
|
||||
}
|
||||
}
|
||||
for _, mime := range []string{"image/png", "text/plain", ""} {
|
||||
if isTranscribableMime(mime) {
|
||||
t.Fatalf("isTranscribableMime(%q) = true, want false", mime)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,16 @@ func (s *Service) reserveUserUploadQuota(userID string, size int64) (string, err
|
||||
return s.reserveUserStoredFileQuota(userID, size, megabytes(policy.Resource.ResourceUploadMB), megabytes(policy.Resource.DailyUploadMB), gigabytes(policy.Resource.StoredFileGB), fmt.Sprintf("单个上传文件必须小于 %dMB", policy.Resource.ResourceUploadMB))
|
||||
}
|
||||
|
||||
// reserveChunkedUploadQuota 用于分片上传完成时预留额度:单文件上限对分片会话不适用(每片已独立校验),
|
||||
// 仅受“今日上传”与“账号存储总量”约束;singleFileLimit 传 size+1 使单文件上限永不命中。
|
||||
func (s *Service) reserveChunkedUploadQuota(userID string, size int64) (string, error) {
|
||||
policy, err := s.RuntimePolicy()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return s.reserveUserStoredFileQuota(userID, size, size+1, megabytes(policy.Resource.DailyUploadMB), gigabytes(policy.Resource.StoredFileGB), "")
|
||||
}
|
||||
|
||||
func (s *Service) reserveSessionUploadQuota(userID string, size int64) (string, error) {
|
||||
policy, err := s.RuntimePolicy()
|
||||
if err != nil {
|
||||
|
||||
@@ -17,12 +17,15 @@ func TestReserveUserUploadQuotaRejectsSingleFileAtLimit(t *testing.T) {
|
||||
|
||||
func TestReserveUserUploadQuotaRejectsDailyTotalAtLimit(t *testing.T) {
|
||||
svc := newResourceTestService(t)
|
||||
for range 4 {
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", 49<<20); err != nil {
|
||||
daily := megabytes(defaultRuntimePolicy().Resource.DailyUploadMB)
|
||||
chunk := int64(49 << 20)
|
||||
for used := int64(0); used+chunk <= daily; used += chunk {
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", chunk); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", 4<<20); err == nil || !strings.Contains(err.Error(), "小于 200MB") {
|
||||
// 单文件限(50MB)未命中、今日额度已满 → 拒绝并提示每日上限。
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", chunk); err == nil || !strings.Contains(err.Error(), "小于 2GB") {
|
||||
t.Fatalf("reserveUserUploadQuota() error = %v", err)
|
||||
}
|
||||
}
|
||||
@@ -63,8 +66,7 @@ func TestReserveUserUploadQuotaRejectsTotalStoredFilesAtLimit(t *testing.T) {
|
||||
if err := svc.repo.Create(&model.Resource{ID: "resource-1", UserID: "user-1", Status: model.ResourceStatusReady, Size: gigabytes(defaultRuntimePolicy().Resource.StoredFileGB) - 1}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", 1); err == nil || !strings.Contains(err.Error(), "2GB 上限") {
|
||||
t.Fatalf("reserveUserUploadQuota() error = %v", err)
|
||||
if _, err := svc.reserveUserUploadQuota("user-1", 1); err == nil || !strings.Contains(err.Error(), "20GB 上限") {
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,314 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"infinite-canvas/backend/internal/model"
|
||||
)
|
||||
|
||||
// 播放副本转码:HEVC/H.265 原片在 Chrome/Firefox 等无法解码(<video> 黑屏),
|
||||
// 上传就绪后若探测到 hvc1/hev1 且本机可用 ffmpeg,则异步转 H.264/AAC 到本地
|
||||
// playback 目录,供 file 端点 variant=playback 读取。转码只作用于本地原件;
|
||||
// OSS 原件不转码(避免每次上传拉取远端)。
|
||||
|
||||
// ErrPlaybackNotReady 表示资源没有可用的浏览器兼容播放副本(未转码/转码中/失败),
|
||||
// file 端点应回退 serve 原件。
|
||||
var ErrPlaybackNotReady = errors.New("播放副本尚未就绪")
|
||||
|
||||
const (
|
||||
playbackDirName = "playback"
|
||||
videoCodecH264 = "h264"
|
||||
videoCodecH265 = "h265"
|
||||
videoCodecAV1 = "av1"
|
||||
videoCodecVP9 = "vp9"
|
||||
videoCodecMPEG4 = "mpeg4"
|
||||
probeMaxMoovSize = 128 << 20
|
||||
)
|
||||
|
||||
// probeVideoCodec 解析本地 mp4 的 stsd 首个视频 sample entry fourcc,返回 h264/h265 等。
|
||||
// 非 mp4 容器或解析失败返回空串(调用方按“无需转码”处理,前端仍可用原件)。
|
||||
func probeVideoCodec(path string) string {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// 顶层 box 遍历(跳过 mdat 数据体),定位 moov。
|
||||
var moovSize int64
|
||||
var moovData []byte
|
||||
pos := int64(0)
|
||||
for {
|
||||
boxType, size, err := readMP4BoxHeaderAt(f, pos)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
switch boxType {
|
||||
case "moov":
|
||||
if size > probeMaxMoovSize {
|
||||
return ""
|
||||
}
|
||||
moovData = make([]byte, size-8)
|
||||
if _, err := f.ReadAt(moovData, pos+8); err != nil {
|
||||
return ""
|
||||
}
|
||||
moovSize = size
|
||||
}
|
||||
pos += size
|
||||
if moovSize != 0 {
|
||||
break
|
||||
}
|
||||
if size < 8 {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
return codecFromMoov(moovData)
|
||||
}
|
||||
|
||||
// readMP4BoxHeaderAt 定位文件 pos 处的 box(8/16 字节头),返回类型与总大小。
|
||||
func readMP4BoxHeaderAt(r io.ReaderAt, pos int64) (string, int64, error) {
|
||||
var hdr [16]byte
|
||||
if _, err := r.ReadAt(hdr[:8], pos); err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
size := int64(binary.BigEndian.Uint32(hdr[:4]))
|
||||
boxType := string(hdr[4:8])
|
||||
if size == 1 {
|
||||
if _, err := r.ReadAt(hdr[8:16], pos+8); err != nil {
|
||||
return "", 0, err
|
||||
}
|
||||
size = int64(binary.BigEndian.Uint64(hdr[8:16]))
|
||||
}
|
||||
if size < 8 {
|
||||
return "", 0, fmt.Errorf("invalid box size %d", size)
|
||||
}
|
||||
return boxType, size, nil
|
||||
}
|
||||
|
||||
// codecFromMoov 在 moov 子树中找出所有 stsd,取首个视频 sample entry fourcc。
|
||||
func codecFromMoov(moov []byte) string {
|
||||
for _, stsdBody := range boxBodies(moov, "stsd") {
|
||||
// stsd = fullbox(4) + entry_count(4) + entries…
|
||||
// 首个 sample entry:entry_size(4) + fourcc(4) → fourcc 在 body+12。
|
||||
if stsdBody+16 > len(moov) {
|
||||
continue
|
||||
}
|
||||
fourcc := string(moov[stsdBody+12 : stsdBody+16])
|
||||
switch fourcc {
|
||||
case "avc1":
|
||||
return videoCodecH264
|
||||
case "hvc1", "hev1":
|
||||
return videoCodecH265
|
||||
case "av01":
|
||||
return videoCodecAV1
|
||||
case "vp09":
|
||||
return videoCodecVP9
|
||||
case "mp4v":
|
||||
return videoCodecMPEG4
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// boxBodies 在 data 中递归查找类型为 want 的 box,返回其 body 起始偏移。
|
||||
// 只下钻容器 box(moov/trak/mdia/minf/stbl),避免误入 sample entry 内部。
|
||||
func boxBodies(data []byte, want string) []int {
|
||||
var out []int
|
||||
var walk func(start, end int)
|
||||
walk = func(start, end int) {
|
||||
pos := start
|
||||
for pos+8 <= end {
|
||||
size := int(binary.BigEndian.Uint32(data[pos : pos+4]))
|
||||
boxType := string(data[pos+4 : pos+8])
|
||||
hdr := 8
|
||||
if size == 1 {
|
||||
if pos+16 > end {
|
||||
return
|
||||
}
|
||||
size = int(binary.BigEndian.Uint64(data[pos+8 : pos+16]))
|
||||
hdr = 16
|
||||
}
|
||||
if size < 8 || pos+size > end {
|
||||
return
|
||||
}
|
||||
if boxType == want {
|
||||
out = append(out, pos+hdr)
|
||||
}
|
||||
switch boxType {
|
||||
case "moov", "trak", "mdia", "minf", "stbl":
|
||||
walk(pos+hdr, pos+size)
|
||||
}
|
||||
pos += size
|
||||
}
|
||||
}
|
||||
walk(0, len(data))
|
||||
return out
|
||||
}
|
||||
|
||||
// maybeStartPlaybackTranscode 由上传就绪路径调用;H.265 且 ffmpeg 可用时置
|
||||
// processing 并异步转码。H.264 直接标记 none,避免每次上传重复探测。
|
||||
func (s *Service) maybeStartPlaybackTranscode(resource *model.Resource) {
|
||||
if resource == nil || resource.Kind != "video" {
|
||||
return
|
||||
}
|
||||
// 远端存储(OSS 等)不落本地副本:标 none 表示无需转码,前端按原生播放/直链处理,
|
||||
// 避免状态留空被当成 processing 无限轮询(markPlaybackNone 幂等)。
|
||||
if resource.Provider != "local" || resource.Status != model.ResourceStatusReady {
|
||||
if resource.Provider != "local" {
|
||||
markPlaybackNone(s, resource)
|
||||
}
|
||||
return
|
||||
}
|
||||
if resource.PlaybackStatus != "" && resource.PlaybackStatus != model.PlaybackStatusNone {
|
||||
return
|
||||
}
|
||||
// 不可转码场景(无 ffmpeg)落 none:状态为空会被前端当成 processing 无限轮询,
|
||||
// 也会让启动回填每次重试探测。none 表示"无需转码",按原生播放处理。
|
||||
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||
markPlaybackNone(s, resource)
|
||||
return
|
||||
}
|
||||
src := filepath.Join(s.dataDir, "resources", filepath.FromSlash(resource.ObjectKey))
|
||||
switch probeVideoCodec(src) {
|
||||
case videoCodecH265:
|
||||
// 原子抢占(空/none → processing):并发上传 + 回填、多实例同抢时
|
||||
// 仅一个能成功置位,其余直接返回,避免重复转码。
|
||||
claimed, err := s.repo.ClaimPlaybackTranscode(resource.ID)
|
||||
if err != nil || !claimed {
|
||||
return
|
||||
}
|
||||
resource.PlaybackStatus = model.PlaybackStatusProcessing
|
||||
go s.runPlaybackTranscode(resource.UserID, resource.ID, src)
|
||||
case videoCodecH264, videoCodecAV1, videoCodecVP9, videoCodecMPEG4, "":
|
||||
// H.264 浏览器可直接解码;AV1/VP9/MPEG4 暂不转码;探针读不出编码(非 mp4 /
|
||||
// moov 在尾部 / 加密容器)也无法处理 —— 均标 none,避免重复探测与前端无限轮询。
|
||||
markPlaybackNone(s, resource)
|
||||
}
|
||||
}
|
||||
|
||||
// markPlaybackNone 将资源标记为无需播放副本(幂等,写失败不影响上传主流程)。
|
||||
func markPlaybackNone(s *Service, resource *model.Resource) {
|
||||
if resource.PlaybackStatus == model.PlaybackStatusNone {
|
||||
return
|
||||
}
|
||||
resource.PlaybackStatus = model.PlaybackStatusNone
|
||||
_ = s.repo.SaveResource(resource)
|
||||
}
|
||||
|
||||
// runPlaybackTranscode 转码本地原件到 playback/<id>.mp4 并回写状态(幂等按 id 重载)。
|
||||
func (s *Service) runPlaybackTranscode(userID string, resourceID string, src string) {
|
||||
// 转码 goroutine 意外 panic 时把状态落 failed,避免 processing 卡死到下次重启。
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
if res, err := s.repo.ResourceForUser(userID, resourceID); err == nil && res != nil {
|
||||
res.PlaybackStatus = model.PlaybackStatusFailed
|
||||
res.PlaybackError = clipText(fmt.Sprintf("转码 panic:%v", r), 1000)
|
||||
_ = s.repo.SaveResource(res)
|
||||
}
|
||||
}
|
||||
}()
|
||||
status := model.PlaybackStatusFailed
|
||||
objectKey := ""
|
||||
var errText string
|
||||
dst := filepath.Join(s.dataDir, playbackDirName, resourceID+".mp4")
|
||||
// 副本已存在且为 H.264(上次转码完成但回写随进程崩溃丢失):直接置 ready 不重转。
|
||||
if probeVideoCodec(dst) == videoCodecH264 {
|
||||
status = model.PlaybackStatusReady
|
||||
objectKey = resourceID + ".mp4"
|
||||
} else if err := runH264Transcode(src, dst); err != nil {
|
||||
errText = clipText(err.Error(), 1000)
|
||||
} else {
|
||||
status = model.PlaybackStatusReady
|
||||
objectKey = resourceID + ".mp4"
|
||||
}
|
||||
res, err := s.repo.ResourceForUser(userID, resourceID)
|
||||
if err != nil || res == nil {
|
||||
return
|
||||
}
|
||||
res.PlaybackStatus = status
|
||||
res.PlaybackObjectKey = objectKey
|
||||
res.PlaybackError = errText
|
||||
_ = s.repo.SaveResource(res)
|
||||
}
|
||||
|
||||
// runH264Transcode 用 ffmpeg 将任意输入转为 H.264/AAC mp4(faststart、yuv420p、偶数尺寸)。
|
||||
func runH264Transcode(src string, dst string) error {
|
||||
if err := os.MkdirAll(filepath.Dir(dst), 0o750); err != nil {
|
||||
return err
|
||||
}
|
||||
cmd := exec.Command("ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
|
||||
"-i", src,
|
||||
"-c:v", "libx264", "-preset", "veryfast", "-crf", "23",
|
||||
"-pix_fmt", "yuv420p", "-vf", "scale=trunc(iw/2)*2:trunc(ih/2)*2",
|
||||
"-c:a", "aac", "-b:a", "128k", "-movflags", "+faststart",
|
||||
dst)
|
||||
var stderr bytes.Buffer
|
||||
cmd.Stderr = &stderr
|
||||
if err := cmd.Run(); err != nil {
|
||||
msg := strings.TrimSpace(stderr.String())
|
||||
if msg == "" {
|
||||
msg = err.Error()
|
||||
}
|
||||
return fmt.Errorf("ffmpeg 转码失败:%s", clipText(msg, 800))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func clipText(s string, max int) string {
|
||||
if len(s) <= max {
|
||||
return s
|
||||
}
|
||||
return s[:max]
|
||||
}
|
||||
|
||||
// OpenResourcePlaybackRange 打开浏览器兼容播放副本(本地 ffmpeg 转码的 H.264)。
|
||||
// 仅当资源为本地存储且副本 ready 时可用;否则返回 ErrPlaybackNotReady,调用方回退原件。
|
||||
func (s *Service) OpenResourcePlaybackRange(userID string, resourceID string) (*ResourceStream, error) {
|
||||
resource, err := s.repo.ResourceForUser(userID, resourceID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if resource == nil || resource.Status != model.ResourceStatusReady || resource.Provider != "local" ||
|
||||
resource.PlaybackStatus != model.PlaybackStatusReady || resource.PlaybackObjectKey == "" {
|
||||
return nil, ErrPlaybackNotReady
|
||||
}
|
||||
body, err := os.Open(filepath.Join(s.dataDir, playbackDirName, filepath.FromSlash(resource.PlaybackObjectKey)))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
playback := *resource
|
||||
playback.MimeType = "video/mp4"
|
||||
playback.ObjectKey = filepath.Join(playbackDirName, resource.PlaybackObjectKey)
|
||||
size := int64(0)
|
||||
if st, err := body.Stat(); err == nil {
|
||||
size = st.Size()
|
||||
}
|
||||
return &ResourceStream{Resource: &playback, Body: body, StatusCode: http.StatusOK, ContentLength: size, AcceptRanges: "bytes"}, nil
|
||||
}
|
||||
|
||||
// BackfillPlaybackTranscodes 在服务启动后扫描存量本地视频:未判定 codec 的补判定,
|
||||
// H.265 触发转码、H.264 标记 none。幂等:maybeStartPlaybackTranscode 先置
|
||||
// processing/none 再入库,重复扫描不会重复转码。
|
||||
func (s *Service) BackfillPlaybackTranscodes() {
|
||||
// 上次进程可能崩溃在转码中途(状态卡 processing),先重置为待判定。
|
||||
_ = s.repo.ResetStuckPlaybackTranscodes()
|
||||
for {
|
||||
resources, err := s.repo.PlaybackPendingVideos(20)
|
||||
if err != nil || len(resources) == 0 {
|
||||
return
|
||||
}
|
||||
for i := range resources {
|
||||
s.maybeStartPlaybackTranscode(&resources[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// stsdBox 构造仅含 first-sample-entry fourcc 的最小 stsd box(置于 moov 切片内)。
|
||||
func stsdBoxWithFourcc(fourcc string) []byte {
|
||||
b := make([]byte, 24)
|
||||
binary.BigEndian.PutUint32(b[0:4], 24)
|
||||
copy(b[4:8], "stsd")
|
||||
// body: version/flags(4) + entry_count(4)=1 + entry_size(4) + fourcc(4)
|
||||
binary.BigEndian.PutUint32(b[12:16], 1)
|
||||
binary.BigEndian.PutUint32(b[16:20], 8)
|
||||
copy(b[20:24], fourcc)
|
||||
return b
|
||||
}
|
||||
|
||||
func TestCodecFromMoov(t *testing.T) {
|
||||
cases := []struct {
|
||||
fourcc string
|
||||
want string
|
||||
}{
|
||||
{"hvc1", videoCodecH265},
|
||||
{"hev1", videoCodecH265},
|
||||
{"avc1", videoCodecH264},
|
||||
{"av01", videoCodecAV1},
|
||||
{"vp09", videoCodecVP9},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := codecFromMoov(stsdBoxWithFourcc(c.fourcc)); got != c.want {
|
||||
t.Errorf("codecFromMoov(%s) = %q, want %q", c.fourcc, got, c.want)
|
||||
}
|
||||
}
|
||||
if got := codecFromMoov([]byte{0, 1, 2, 3}); got != "" {
|
||||
t.Errorf("garbage moov codec = %q, want empty", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeVideoCodecReadsRealFile(t *testing.T) {
|
||||
// 构造 ftyp + moov(含 hvc1 stsd) 的最小 mp4 文件,验证 probeVideoCodec 走文件读取路径。
|
||||
stsd := stsdBoxWithFourcc("hvc1")
|
||||
moov := make([]byte, 8+len(stsd))
|
||||
binary.BigEndian.PutUint32(moov[0:4], uint32(len(moov)))
|
||||
copy(moov[4:8], "moov")
|
||||
copy(moov[8:], stsd)
|
||||
full := make([]byte, 0, 16+len(moov))
|
||||
ftyp := make([]byte, 16)
|
||||
binary.BigEndian.PutUint32(ftyp[0:4], 16)
|
||||
copy(ftyp[4:8], "ftyp")
|
||||
copy(ftyp[8:12], "isom")
|
||||
full = append(full, ftyp...)
|
||||
full = append(full, moov...)
|
||||
|
||||
path := filepath.Join(t.TempDir(), "clip.mp4")
|
||||
if err := os.WriteFile(path, full, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := probeVideoCodec(path); got != videoCodecH265 {
|
||||
t.Errorf("probeVideoCodec = %q, want h265", got)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user