* feat(editor): 编辑器实施 M0-M1 - 剪辑入口占位与插件 SDK v2 垂直切片 - detail.tsx 新增剪辑成片视图(editor tab,常驻),概览页插入剪辑步骤卡 - plugin-types.ts 新增 v2 类型(EditorSlotKind/EditorPluginPermission/PluginManifestV2),修复权限数组交叉归约 bug - editor-slot-registry.ts 插槽注册表(优先级排序、幂等注册、订阅机制) - plugin-registry.ts 注册器 v1/v2 分支校验,v1 路径零改动 - editor-shell.tsx 垂直切片插件:timeline-panel 插槽真实渲染,验证插件链路全通 - 测试:插槽注册表 5 用例 + 命令测试骨架,v1 插件回归 21 例零回归 * feat(editor): 命令协议黄金文件 - editor-commands 纯函数 reducer + 8 个黄金 op * feat(editor): 快照撤销历史 - editor-history 有界 200 层 undo/redo * feat(editor): 命令状态机 store - dispatch/手势预览/1.5s 防抖保存/串行保存链 * fix(editor): createEditorStore 返回 zustand hook 类型修复 typecheck * feat(editor): 手势 echo 最小件 - 时间线面板拖拽移动/裁剪 + store 接线与本地持久化 * docs(editor): M2 完成状态记录 * feat(editor): M3.1 时间线面板选中联动 - store 选中状态 + 片段高亮/点击取消 * feat(editor): M3.2 预览监视器 - preview-renderer 插槽 + 真实媒体帧/播放头/时间码 * feat(editor): M3.3 片段检查器 - inspector 插槽 + 属性编辑入队可撤销 * feat(editor): M3.4 asset-ingest 素材库插件(addClip 命令入轨) * feat(editor): M3.4 补充 - host 上下文(项目信息+资产)、editor.tsx 接线、asset-ingest 纯函数、死代码清理 * feat(editor): M3.5 subtitle-tools - SRT import/export + rebuild from node snapshot * feat(editor): M3.6 transcription - deterministic mock ASR to subtitle track * feat(editor): M3.7 export - render plan + ffmpeg.wasm fallback export * feat(editor): M3.8 - render all 8 editor slots in three-pane layout + ai-assistant placeholder * docs(editor): M3 完成状态 - 8 插件 + 全插槽接线,冒烟待确认 * feat(editor): 编辑器媒体与时间线打磨 - 资产来源分组筛选、时间线全宽布局、拖拽末片段修复、OSS 降级本地存储、Concat 风格控制条与检查器 * fix(plugins): 合并上游后修复插件分类筛选对 v2 清单的类型兼容 * feat(editor): 时间线工具态与轨道操作打磨 - 工具按钮选中仅图标变色、监视器 Concat 式控制条、removeTrack 黄金命令与轨道行头移除按钮、maskClosable 迁移 * fix(api): 资源上传超 MaxBytesReader 上限时报中文大小限制而非英文裸 400 * fix(editor): 监视器控制条 - 修复播放按钮 JSX 模板闭合花括号缺失导致的编译错误 * feat(api): 本地媒体导入分片化对齐 Concat - >50MB 视频不再失败 - 分片会话三接口(POST开始/PUT单片/complete合并)、前端超50MB自动切8MB分片并失败整传重试、默认配额放宽(日200MB→2GB、总2GB→20GB)、UploadResourceFile 豁免单文件限 * feat(editor): 轨道行头新增可见性/静音开关按钮 - 复用现有图标资源,按轨道类型显隐;命令层新增 setTrackFlag 黄金命令(flag 白名单 fail-closed)并补单测 * docs(progress): 记录轨道可见/静音开关改动与复测项 * docs(editor): M4 转写与渲染任务设计 - 复用 Task 表与状态机、独立创建 API、worker 领取类型过滤、whisper HTTP provider(OpenAI 兼容 /v1/audio) * fix(api): OSS 上传失败自动降级本地存储时 body 已被 Transport 关闭 - newOSSRequest 用 NopCloser 包装请求体,服务端提前返回(签名 403)时不再关闭调用方持有的文件,降级分支 Seek 重读生效;补回归测试 * docs(progress): 记录媒体导入根治(分片+OSS 降级修复)复测项 * feat(api): 视频播放副本转码 H.265→H.264 + variant=playback 服务与启动回填 - 上传视频后异步探测 codec,H.265/AV1 走 moov 快速探测并转码 H.264 播放副本;GET /api/user-data/...?variant=playback 就绪时服务副本(独立 ETag :pb),未就绪回退原件并撤销后缀防 304 黑屏;storeResource 触发、启动幂等回填存量;前端播放失败自动轮询副本就绪后刷新 * fix(editor): 轨道至少保留一条 - 不可移除轨道点击 X 时居中浮现原因提示(2.2s 自动消失, aria-live);修正可见性 toggle 传参 (visible===false 语义) ;轨道行内移除按钮改为常驻(不可移除时点击给提示) * feat(timeline): M4.1 字幕转写后台任务 - whisper.cpp 本地执行 - 新增 timeline_transcription 任务类型,worker 按类型分叉独立执行器(不走计费与模型渠道);输入 resourceId+language,读取用户资源后 ffmpeg 预处理 16k 单声道 wav,POST 本地 whisper.cpp /inference,结果写回 segments+srt+language;未配置 CANVAS_WHISPER_BASE_URL 等失败均落明确终态;feature 门控;进度分阶段;含解析/SRT/快速失败单元测试 * feat(media): 素材卡片时长与媒体面板打磨 - 导入媒体解析时长(前端 media-metadata + 后端 project_asset durationMs 透出)随卡片展示;媒体面板/监视器/时间线相关样式细节收尾 * docs(progress): 阶段 C 监视器播放复检清单 - H.265 播放副本/真实播放驱动/时长链路待浏览器确认 * feat(timeline): M4.1 转写任务创建 API - POST /api/timeline/transcriptions - service CreateTimelineTranscriptionTask:IsDraining 守护 + feature 门控 + 资源归属与可转写类型校验 + 排队计 active 限额(含任务级/项目级存储估算);local/whisper.cpp 标记,任务不经过模型路由与计费;createTaskWithinStorageQuota 复用与 ErrActiveTaskLimit 用户可读映射;测试覆盖排队成功(owner/type/provider 断言)、跨用户资源拒绝、非音视频拒绝 * feat(timeline): M4.3 字幕转写前端接线 - 真实任务替换 mock - 新增 services/api/timeline-tasks.ts(POST /timeline/transcriptions,复用 apiClient+request);EditorTranscription 改为 创建任务→waitForGenerationTask 轮询(25min)→结果 segments 映射 SrtEntry[]→rebuildSubtitleClips 写入字幕轨道;实时展示 stage/progress,失败/空识别/无字幕轨道分别提示;素材选择支持音频与视频(取 storageKey 中的 resource id 提交) * fix(media): 播放副本状态收敛与轮询终态 - 无ffmpeg/远端存储/编码不可处理落 none 避免重复探测;前端 none 停轮询回原生播放并加 processing 上限 * feat(timeline): M4.2 时间线渲染任务 - POST /api/timeline/renders 创建(buildRenderPlan 校验含可渲染媒体、配额入队、local/ffmpeg、60min 超时);worker 顶部按类型分叉 processTimelineRender;ffmpeg 合成产物写资源存储并经 ResultJSON 记 resourceId;render 计划/结果测试 PASS * fix(resource): 转码原子抢占与 OSS 请求体加固 - ClaimPlaybackTranscode 空/none→processing 单实例抢占避免重复转码,runPlaybackTranscode defer recover 落 failed;If-Range 与带 :pb 后缀的 serveETag 比较;无请求体的 OSS 请求不再包 NopCloser(nil)(防 Go 1.26 body 探测 nil 解引用) * style(editor): 修正字幕回写面板缩进 * docs(timeline): M4 设计文档按实现回写 - 单一 worker 池 + processClaimedTask 顶部按类型分叉(决策 3/4),端点统一为复数 /api/timeline/transcriptions 与 /api/timeline/renders(注册于 RegisterTaskRoutes),ResultJSON/feature gate/ffmpeg env 同步;Runbook M4 表格与状态行更新;auth.go API 前缀白名单补 timeline * fix(service): 修复上游 v1.2.5 公告配图删除保护自锁与测试夹具缺口 上游 #385 资源引用走查器对公告配图/草稿做了硬性引用检查,导致三个问题: - UpdateAnnouncement 在更新公告前对旧配图走查,必然命中当前公告自身引用而自锁, 公告替换配图功能实际不可用;DiscardAnnouncementImage 同理被草稿自身引用阻塞。 修复:ensureResourceHasNoBusinessReferences 支持忽略指定引用(当前公告/当前草稿), 其余引用仍按原语义拦截。 - admin DeleteAdminResources 将公告草稿引用视为阻塞,但删除事务会级联清理草稿与 资源绑定,service 层永不进入事务,回滚审计注入测试无法到达预期路径。修复:service 预检时过滤公告草稿引用,与 repository 级联语义对齐。 - project_settings_test.go 夹具未建 ResourceReferenceSnapshot 走查的 Task/Session/ Message/TaskLog/Result/Announcement/AnnouncementImageDraft 表,报 no such table。 修复:夹具 AutoMigrate 补齐。 后端全量 go test ./... 通过。 * feat(editor): 成片导出接入后端渲染任务(M4.2 前端接线)- timeline-tasks 增 createTimelineRenderTask;editor-export 主路径改 POST /timeline/renders 轮询取 resourceId 内嵌预览/下载,ffmpeg.wasm 降级保留 * feat(editor): M5 插件权限执行(fail-closed)- plugin-permission-check 新建(8 插槽最小权限表/判定/断言);SlotStack 渲染前过滤缺权限插槽改诊断条;editor-shell manifest 补 timeline.command/export.run;测试 10 用例含 editor-shell 真实回归 * feat(editor): M6.1 AI 命令 schema + M6.2 时间线摘要 - ai-command-schema(parse/dry-run 校验/12-op 提示词契约/版本化+golden);timeline-summary(确定性中文摘要/折叠上限);各 20/10 用例全绿 * docs(progress): M6 质检清单 - M6.1 schema/M6.2 摘要标记完成(含测试计数),M6.3 待模型源确认;清理 M5.1 重复条目 * feat(editor): AI 命令 schema 支持只读问答 - commands 空数组合法化(无法执行/纯提问时 reasoning 直接作答),新增 AI_EDITING_MAX_COMMANDS=8 上限防失控编辑,提示词同步 * feat(editor): AI 助手重构为右上悬浮对话面板(M6.3)+ 素材入库原子化收尾 - editor-ai-assistant.tsx: 占位面板替换为 OpenAI Canvas 风格右上悬浮对话面板; 面板内文本模型选择器(effective.textModels,选择即持久化并覆写请求模型), 消息流 user/assistant/error/preview 四类气泡 + 思考中占位,空态引导示例 chips 一键发送,底部输入条;顶部工具栏常驻 Sparkles 入口(aria-pressed 切换)。 修复上一版重复/截断函数声明(appendToEntry 双定义、settleAnswer 三声明等)。 宿主链路:buildAiEditingSystemPrompt(summarizeTimeline(project)) 喂上下文 → parseAiCommandPlan → validateAiCommandBatch dry-run(任一非法整批拒绝)→ ≤3 条 dispatch 直执行 / >3 条预览确认卡片;宿主拒绝时 failTurn 回填 FAILED_MARK 可重试。 - editor-asset-ingest/editor-preview-monitor/detail/editor.tsx: 配套接线与打磨 (编辑器顶栏 AI 入口、素材导入面板、预览监视、项目页顶栏整理)。 - backend: LinkProjectAsset 事务内原子创建资产 + 项目链接(OnConflict 幂等), 资产首次入库不再预落库,失败整体回滚,消除"有资产无链接"孤儿记录。 - docs: pending-test.mdx / runbook M6.3 状态更新(tsc + 测试 + 构建通过, 浏览器冒烟待真实文本渠道)。
1012 lines
41 KiB
Go
1012 lines
41 KiB
Go
package service
|
||
|
||
import (
|
||
"bytes"
|
||
"encoding/json"
|
||
"errors"
|
||
"hash/crc64"
|
||
"io"
|
||
"net/http"
|
||
"net/http/httptest"
|
||
"net/url"
|
||
|
||
"os"
|
||
"strconv"
|
||
"strings"
|
||
"testing"
|
||
"testing/iotest"
|
||
"time"
|
||
|
||
"infinite-canvas/backend/internal/model"
|
||
"infinite-canvas/backend/internal/repository"
|
||
|
||
"gorm.io/driver/sqlite"
|
||
"gorm.io/gorm"
|
||
)
|
||
|
||
func TestSignedOSSObjectURLUsesExpiringQuerySignature(t *testing.T) {
|
||
expiresAt := time.Unix(1800000000, 0)
|
||
value, err := signedOSSObjectURL(ossSettingValue{
|
||
Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/image/test image.png", expiresAt)
|
||
if err != nil {
|
||
t.Fatalf("signedOSSObjectURL() error = %v", err)
|
||
}
|
||
parsed, err := url.Parse(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
query := parsed.Query()
|
||
if parsed.Host != "private-bucket.oss-cn-test.aliyuncs.com" || query.Get("OSSAccessKeyId") != "access-id" || query.Get("Expires") != "1800000000" || query.Get("Signature") == "" {
|
||
t.Fatalf("signed URL = %q", value)
|
||
}
|
||
if strings.Contains(value, "secret-value") {
|
||
t.Fatalf("signed URL leaked access key secret: %q", value)
|
||
}
|
||
}
|
||
|
||
func TestSignedOSSObjectURLSupportsTencentCOS(t *testing.T) {
|
||
value, err := signedOSSObjectURL(ossSettingValue{
|
||
Provider: tencentCOSProvider, Region: "ap-guangzhou", Bucket: "private-bucket-1250000000",
|
||
AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, "users/u-1/image/test image.png", time.Now().Add(time.Hour))
|
||
if err != nil {
|
||
t.Fatalf("signedOSSObjectURL() error = %v", err)
|
||
}
|
||
parsed, err := url.Parse(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
query := parsed.Query()
|
||
if parsed.Host != "private-bucket-1250000000.cos.ap-guangzhou.myqcloud.com" || query.Get("q-sign-algorithm") != "sha1" || query.Get("q-ak") != "secret-id" || query.Get("q-signature") == "" {
|
||
t.Fatalf("signed COS URL = %q", value)
|
||
}
|
||
if strings.Contains(value, "secret-key") {
|
||
t.Fatalf("signed COS URL leaked secret key: %q", value)
|
||
}
|
||
}
|
||
|
||
func TestSignedOSSObjectURLUsesAliyunCDNBaseURLForDownloads(t *testing.T) {
|
||
value, err := signedOSSObjectURL(ossSettingValue{
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", CDNBaseURL: "https://media.example.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/image/test image.png", time.Now().Add(time.Hour))
|
||
if err != nil {
|
||
t.Fatalf("signedOSSObjectURL() error = %v", err)
|
||
}
|
||
parsed, err := url.Parse(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if parsed.Host != "media.example.com" || parsed.Path != "/users/u-1/image/test image.png" || parsed.RawQuery != "" || !strings.Contains(value, "test%20image.png") {
|
||
t.Fatalf("Aliyun OSS CDN URL = %q", value)
|
||
}
|
||
}
|
||
|
||
func TestAliyunOSSUploadRequestStillUsesEndpointWhenCDNConfigured(t *testing.T) {
|
||
req, err := newOSSRequest(http.MethodPut, ossSettingValue{
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", CDNBaseURL: "https://media.example.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/image/test.png", "image/png", strings.NewReader("payload"))
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if req.URL.Host != "private-bucket.oss-cn-test.aliyuncs.com" || req.URL.Path != "/users/u-1/image/test.png" {
|
||
t.Fatalf("Aliyun OSS upload URL = %q", req.URL.String())
|
||
}
|
||
}
|
||
|
||
func TestNewOSSRequestBodyCloseDoesNotCloseCallerFile(t *testing.T) {
|
||
// 服务端提前返回 403(不读 body)时,http.Transport 会关闭 Request.Body。
|
||
// newOSSRequest 必须用 no-op close 包装请求体,否则调用方持有的 *os.File
|
||
// 会被关掉,OSS 失败后的“降级本地存储”Seek 重读将报 file already closed。
|
||
f, err := os.CreateTemp(t.TempDir(), "merged")
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
defer f.Close()
|
||
if _, err := f.WriteString("payload-bytes"); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
req, err := newOSSRequest(http.MethodPut, ossSettingValue{
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/video/clip.mp4", "video/mp4", f)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if req.Body == nil {
|
||
t.Fatal("newOSSRequest() body is nil")
|
||
}
|
||
// 模拟 Transport 在服务端提前响应后关闭请求体:
|
||
if err := req.Body.Close(); err != nil {
|
||
t.Fatalf("close request body: %v", err)
|
||
}
|
||
if _, err := f.Seek(0, io.SeekStart); err != nil {
|
||
t.Fatalf("caller file was closed by transport: %v", err)
|
||
}
|
||
got, err := io.ReadAll(f)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if string(got) != "payload-bytes" {
|
||
t.Fatalf("payload after body close = %q", got)
|
||
}
|
||
}
|
||
|
||
func TestSignedOSSObjectURLUsesTencentCOSCDNBaseURLWithoutCOSSignature(t *testing.T) {
|
||
value, err := signedOSSObjectURL(ossSettingValue{
|
||
Provider: tencentCOSProvider, Region: "ap-guangzhou", Bucket: "private-bucket-1250000000",
|
||
CDNBaseURL: "https://media.example.com", AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, "users/u-1/image/test image.png", time.Now().Add(time.Hour))
|
||
if err != nil {
|
||
t.Fatalf("signedOSSObjectURL() error = %v", err)
|
||
}
|
||
parsed, err := url.Parse(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if parsed.Host != "media.example.com" || parsed.Path != "/users/u-1/image/test image.png" || parsed.RawQuery != "" || !strings.Contains(value, "test%20image.png") {
|
||
t.Fatalf("signed COS CDN URL = %q", value)
|
||
}
|
||
}
|
||
|
||
func TestPutOSSObjectSupportsTencentCOS(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
payload := []byte("cos upload payload")
|
||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||
if r.Method != http.MethodPut || r.URL.Path != "/users/u-1/image/test.png" {
|
||
t.Errorf("request = %s %s", r.Method, r.URL.Path)
|
||
}
|
||
if r.Header.Get("Content-Type") != "image/png" {
|
||
t.Errorf("Content-Type = %q", r.Header.Get("Content-Type"))
|
||
}
|
||
authorization := r.Header.Get("Authorization")
|
||
if !strings.Contains(authorization, "q-sign-algorithm=sha1") || !strings.Contains(authorization, "q-ak=secret-id") {
|
||
t.Errorf("Authorization = %q", authorization)
|
||
}
|
||
data, err := io.ReadAll(r.Body)
|
||
if err != nil {
|
||
t.Error(err)
|
||
}
|
||
if !bytes.Equal(data, payload) {
|
||
t.Errorf("body = %q", data)
|
||
}
|
||
w.Header().Set("ETag", `"cos-etag"`)
|
||
w.Header().Set("x-cos-hash-crc64ecma", strconv.FormatUint(crc64.Checksum(data, crc64.MakeTable(crc64.ECMA)), 10))
|
||
w.WriteHeader(http.StatusOK)
|
||
}))
|
||
defer server.Close()
|
||
|
||
etag, err := putOSSObject(ossSettingValue{
|
||
Provider: tencentCOSProvider, Endpoint: server.URL, Bucket: "private-bucket-1250000000",
|
||
CDNBaseURL: "https://media.example.com", AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, "users/u-1/image/test.png", "image/png", int64(len(payload)), bytes.NewReader(payload))
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if etag != "cos-etag" {
|
||
t.Fatalf("ETag = %q", etag)
|
||
}
|
||
}
|
||
|
||
func TestGetOSSObjectRangeSupportsTencentCOS(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||
if r.Header.Get("Range") != "bytes=0-3" {
|
||
t.Errorf("Range = %q", r.Header.Get("Range"))
|
||
}
|
||
authorization := r.Header.Get("Authorization")
|
||
if !strings.Contains(authorization, "q-sign-algorithm=sha1") || !strings.Contains(authorization, "q-ak=secret-id") {
|
||
t.Errorf("Authorization = %q", authorization)
|
||
}
|
||
w.Header().Set("Accept-Ranges", "bytes")
|
||
w.Header().Set("Content-Range", "bytes 0-3/7")
|
||
w.WriteHeader(http.StatusPartialContent)
|
||
_, _ = w.Write([]byte("data"))
|
||
}))
|
||
defer server.Close()
|
||
|
||
stream, err := getOSSObjectRange(ossSettingValue{
|
||
Provider: tencentCOSProvider, Endpoint: server.URL, Bucket: "private-bucket-1250000000",
|
||
AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, "users/u-1/image/test.png", "bytes=0-3")
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
defer stream.body.Close()
|
||
data, err := io.ReadAll(stream.body)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if stream.statusCode != http.StatusPartialContent || stream.contentRange != "bytes 0-3/7" || string(data) != "data" {
|
||
t.Fatalf("stream = %#v, data = %q", stream, data)
|
||
}
|
||
}
|
||
|
||
func TestGetOSSObjectRangeUsesTencentCOSCDNBaseURL(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||
if r.Header.Get("Range") != "bytes=0-3" {
|
||
t.Errorf("Range = %q", r.Header.Get("Range"))
|
||
}
|
||
if r.Header.Get("Authorization") != "" || r.URL.RawQuery != "" {
|
||
t.Errorf("Tencent CDN request should not carry COS authentication: header %q, query %q", r.Header.Get("Authorization"), r.URL.RawQuery)
|
||
}
|
||
w.Header().Set("Accept-Ranges", "bytes")
|
||
w.Header().Set("Content-Range", "bytes 0-3/7")
|
||
w.WriteHeader(http.StatusPartialContent)
|
||
_, _ = w.Write([]byte("data"))
|
||
}))
|
||
defer server.Close()
|
||
|
||
stream, err := getOSSObjectRange(ossSettingValue{
|
||
Provider: tencentCOSProvider, Endpoint: "https://cos.ap-guangzhou.myqcloud.com", CDNBaseURL: server.URL,
|
||
Bucket: "private-bucket-1250000000", AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, "users/u-1/image/test.png", "bytes=0-3")
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
defer stream.body.Close()
|
||
data, err := io.ReadAll(stream.body)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if stream.statusCode != http.StatusPartialContent || stream.contentRange != "bytes 0-3/7" || string(data) != "data" {
|
||
t.Fatalf("stream = %#v, data = %q", stream, data)
|
||
}
|
||
}
|
||
|
||
func TestGetOSSObjectRangeUsesAliyunCDNBaseURLWithoutOSSSignature(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||
if r.Header.Get("Range") != "bytes=0-3" {
|
||
t.Errorf("Range = %q", r.Header.Get("Range"))
|
||
}
|
||
if r.Header.Get("Authorization") != "" || r.URL.RawQuery != "" {
|
||
t.Errorf("Aliyun CDN request should not carry OSS authentication: header %q, query %q", r.Header.Get("Authorization"), r.URL.RawQuery)
|
||
}
|
||
w.Header().Set("Accept-Ranges", "bytes")
|
||
w.Header().Set("Content-Range", "bytes 0-3/7")
|
||
w.WriteHeader(http.StatusPartialContent)
|
||
_, _ = w.Write([]byte("data"))
|
||
}))
|
||
defer server.Close()
|
||
|
||
stream, err := getOSSObjectRange(ossSettingValue{
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", CDNBaseURL: server.URL,
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/image/test.png", "bytes=0-3")
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
defer stream.body.Close()
|
||
data, err := io.ReadAll(stream.body)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if stream.statusCode != http.StatusPartialContent || stream.contentRange != "bytes 0-3/7" || string(data) != "data" {
|
||
t.Fatalf("stream = %#v, data = %q", stream, data)
|
||
}
|
||
}
|
||
|
||
func TestTencentCOSSettingDerivesEndpointAndDoesNotReuseAliyunSecret(t *testing.T) {
|
||
normalized := normalizeOSSSetting(ossSettingValue{Provider: tencentCOSProvider, Region: "ap-shanghai"})
|
||
if normalized.Endpoint != "https://cos.ap-shanghai.myqcloud.com" {
|
||
t.Fatalf("Endpoint = %q", normalized.Endpoint)
|
||
}
|
||
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
_, err := ossSettingFromRequest(OSSSettingRequest{
|
||
Enabled: true, Provider: tencentCOSProvider, Endpoint: server.URL, Bucket: "private-bucket-1250000000", AccessKeyID: "secret-id",
|
||
}, ossSettingValue{Provider: aliyunOSSProvider, AccessKeySecret: "aliyun-secret"})
|
||
if err == nil || !strings.Contains(err.Error(), "访问密钥 SecretKey") {
|
||
t.Fatalf("ossSettingFromRequest() error = %v", err)
|
||
}
|
||
next, err := ossSettingFromRequest(OSSSettingRequest{
|
||
Enabled: true, Provider: tencentCOSProvider, Endpoint: server.URL, CDNBaseURL: server.URL,
|
||
Bucket: "private-bucket-1250000000", AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
}, ossSettingValue{})
|
||
if err != nil || next.CDNBaseURL != server.URL {
|
||
t.Fatalf("Tencent COS CDN setting = %#v, %v", next, err)
|
||
}
|
||
}
|
||
|
||
func TestAliyunOSSSettingKeepsCDNBaseURL(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
next, err := ossSettingFromRequest(OSSSettingRequest{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: server.URL, CDNBaseURL: server.URL,
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, ossSettingValue{})
|
||
if err != nil || next.CDNBaseURL != server.URL {
|
||
t.Fatalf("Aliyun OSS CDN setting = %#v, %v", next, err)
|
||
}
|
||
}
|
||
|
||
func TestQiniuKodoSettingAllowsMissingCDNBaseURL(t *testing.T) {
|
||
next, err := ossSettingFromRequest(OSSSettingRequest{
|
||
Enabled: true, Provider: qiniuKodoProvider, Region: "z0", Endpoint: "https://up-z0.qiniup.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, ossSettingValue{})
|
||
if err != nil {
|
||
t.Fatalf("ossSettingFromRequest() error = %v", err)
|
||
}
|
||
if next.CDNBaseURL != "" {
|
||
t.Fatalf("CDNBaseURL = %q, want empty", next.CDNBaseURL)
|
||
}
|
||
}
|
||
|
||
func TestSignedQiniuS3ObjectURL(t *testing.T) {
|
||
value, err := signedQiniuObjectURL(ossSettingValue{
|
||
Provider: qiniuKodoProvider, Endpoint: "https://up-z0.qiniup.com", Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}, "users/u-1/image/test image.png", time.Now().Add(time.Hour))
|
||
if err != nil {
|
||
t.Fatalf("signedQiniuObjectURL() error = %v", err)
|
||
}
|
||
parsed, err := url.Parse(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
query := parsed.Query()
|
||
if parsed.Host != "private-bucket.s3.cn-east-1.qiniucs.com" || parsed.Path != "/users/u-1/image/test image.png" || query.Get("X-Amz-Algorithm") != "AWS4-HMAC-SHA256" || query.Get("X-Amz-Credential") == "" || query.Get("X-Amz-Signature") == "" || query.Get("X-Amz-Expires") == "" {
|
||
t.Fatalf("signed Qiniu S3 URL = %q", value)
|
||
}
|
||
if strings.Contains(value, "secret-value") {
|
||
t.Fatalf("signed URL leaked access key secret: %q", value)
|
||
}
|
||
}
|
||
|
||
func TestOSSCDNBaseURLRejectsNonDomainParts(t *testing.T) {
|
||
for _, value := range []string{"ftp://media.example.com", "https://media.example.com/assets", "https://media.example.com?token=value", "https://user@media.example.com"} {
|
||
if _, err := ossCDNBaseURL(value); err == nil {
|
||
t.Fatalf("ossCDNBaseURL(%q) should fail", value)
|
||
}
|
||
}
|
||
}
|
||
|
||
func TestPlatformProviderSwitchKeepsHistoricalCredentials(t *testing.T) {
|
||
current := ossSettingValue{Provider: aliyunOSSProvider, AccessKeyID: "aliyun-id", AccessKeySecret: "aliyun-secret"}
|
||
next := archiveOSSProviderCredentials(ossSettingValue{Provider: tencentCOSProvider, AccessKeyID: "cos-id", AccessKeySecret: "cos-secret"}, current)
|
||
historical, err := ossSettingForProvider(next, aliyunOSSProvider)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if historical.Provider != aliyunOSSProvider || historical.AccessKeyID != "aliyun-id" || historical.AccessKeySecret != "aliyun-secret" {
|
||
t.Fatalf("historical setting = %#v", historical)
|
||
}
|
||
if _, ok := next.ArchivedCredentials[tencentCOSProvider]; ok {
|
||
t.Fatalf("active provider credentials were archived: %#v", next.ArchivedCredentials)
|
||
}
|
||
}
|
||
|
||
func TestArchivedProviderCredentialsAreEncryptedAtRest(t *testing.T) {
|
||
svc := &Service{dataDir: t.TempDir()}
|
||
value := ossSettingValue{
|
||
Provider: tencentCOSProvider, AccessKeyID: "cos-id", AccessKeySecret: "cos-secret",
|
||
ArchivedCredentials: map[string]ossProviderCredentials{
|
||
aliyunOSSProvider: {AccessKeyID: "aliyun-id", AccessKeySecret: "aliyun-secret"},
|
||
},
|
||
}
|
||
stored, err := svc.encryptOSSSettingSecrets(value)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if !strings.HasPrefix(stored.AccessKeySecret, encryptedSettingPrefix) || !strings.HasPrefix(stored.ArchivedCredentials[aliyunOSSProvider].AccessKeySecret, encryptedSettingPrefix) {
|
||
t.Fatalf("stored credentials are not encrypted: %#v", stored)
|
||
}
|
||
if _, err := svc.decryptOSSSettingSecrets(&stored); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if stored.AccessKeySecret != "cos-secret" || stored.ArchivedCredentials[aliyunOSSProvider].AccessKeySecret != "aliyun-secret" {
|
||
t.Fatalf("decrypted credentials = %#v", stored)
|
||
}
|
||
}
|
||
|
||
func TestDirectResourceURLChecksOwnershipAndSignsOSSResource(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: "aliyun", Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-direct", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: "aliyun", Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
ObjectKey: "users/user-1/image/direct.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
value, err := svc.DirectResourceURL("user-1", resource.ID)
|
||
if err != nil || !strings.Contains(value, "Signature=") {
|
||
t.Fatalf("DirectResourceURL() = %q, %v", value, err)
|
||
}
|
||
if _, err := svc.DirectResourceURL("other-user", resource.ID); err == nil {
|
||
t.Fatal("DirectResourceURL() allowed another user's resource")
|
||
}
|
||
}
|
||
|
||
func TestPrepareResourceDeliveryPrefersConfiguredCDN(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: tencentCOSProvider, Endpoint: "https://cos.ap-shanghai.myqcloud.com", CDNBaseURL: "https://media.example.com",
|
||
Bucket: "private-bucket-1250000000", AccessKeyID: "secret-id", AccessKeySecret: "secret-key",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-cdn-delivery", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: tencentCOSProvider, Endpoint: "https://cos.ap-shanghai.myqcloud.com", Bucket: "private-bucket-1250000000",
|
||
ObjectKey: "users/user-1/image/test image.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery("user-1", resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if delivery.Resource == nil || delivery.Resource.ID != resource.ID || delivery.RedirectURL != "https://media.example.com/users/user-1/image/test%20image.png" {
|
||
t.Fatalf("PrepareResourceDelivery() = %#v", delivery)
|
||
}
|
||
}
|
||
|
||
func TestPrepareResourceDeliveryAllowsExplicitProxyWithCDN(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", CDNBaseURL: "https://media.example.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-cdn-proxy", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
ObjectKey: "users/user-1/image/proxy.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery("user-1", resource.ID, ResourceDeliveryOptions{ForceProxy: true})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if delivery.Resource == nil || delivery.Resource.ID != resource.ID || delivery.RedirectURL != "" {
|
||
t.Fatalf("PrepareResourceDelivery(force proxy) = %#v", delivery)
|
||
}
|
||
}
|
||
|
||
func TestPrepareResourceDeliverySignsPrivateQiniuCDNURL(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: qiniuKodoProvider, Endpoint: "https://up-z0.qiniup.com", CDNBaseURL: "https://media.example.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-qiniu-private-cdn", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: qiniuKodoProvider, Endpoint: "https://up-z0.qiniup.com", Bucket: "private-bucket",
|
||
ObjectKey: "ai/users/user-1/image/private.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery("user-1", resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if delivery.RedirectURL == "" || !strings.HasPrefix(delivery.RedirectURL, "https://media.example.com/ai/users/user-1/image/private.png?") || !strings.Contains(delivery.RedirectURL, "e=") || !strings.Contains(delivery.RedirectURL, "token=") {
|
||
t.Fatalf("PrepareResourceDelivery() = %q, want a signed Qiniu URL", delivery.RedirectURL)
|
||
}
|
||
}
|
||
|
||
func TestPrepareResourceDeliveryProxiesQiniuWithoutCDNBaseURL(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: qiniuKodoProvider, Region: "z0", Endpoint: "https://up-z0.qiniup.com",
|
||
Bucket: "private-bucket", AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-qiniu-proxy", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: qiniuKodoProvider, Endpoint: "https://up-z0.qiniup.com", Bucket: "private-bucket",
|
||
ObjectKey: "users/user-1/image/private.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery("user-1", resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if delivery.Resource == nil || delivery.RedirectURL != "" {
|
||
t.Fatalf("PrepareResourceDelivery() = %#v, want backend proxy delivery", delivery)
|
||
}
|
||
}
|
||
|
||
func TestCurrentUserCDNSettingAppliesToHistoricalResourcesInSameStorage(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
svc := newResourceTestService(t)
|
||
actor := &model.User{ID: "user-1"}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: server.URL, Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
historical, _, err := svc.readUserOSSSetting(actor.ID)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-user-cdn", UserID: actor.ID, Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: aliyunOSSProvider, Endpoint: server.URL, Bucket: "private-bucket", StorageSettingID: historical.ID,
|
||
ObjectKey: "users/user-1/image/historical.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: server.URL, CDNBaseURL: server.URL, Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery(actor.ID, resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if delivery.RedirectURL != server.URL+"/users/user-1/image/historical.png" {
|
||
t.Fatalf("PrepareResourceDelivery(historical user resource) = %#v", delivery)
|
||
}
|
||
}
|
||
|
||
func TestBoundHistoricalUserResourceDoesNotFollowCurrentProviderCDN(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
newTestServer := func() *httptest.Server {
|
||
return httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
}
|
||
aliyunEndpoint := newTestServer()
|
||
aliyunCDN := newTestServer()
|
||
qiniuEndpoint := newTestServer()
|
||
qiniuCDN := newTestServer()
|
||
defer aliyunEndpoint.Close()
|
||
defer aliyunCDN.Close()
|
||
defer qiniuEndpoint.Close()
|
||
defer qiniuCDN.Close()
|
||
|
||
svc := newResourceTestService(t)
|
||
actor := &model.User{ID: "user-1"}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: aliyunEndpoint.URL, CDNBaseURL: aliyunCDN.URL, Bucket: "aliyun-bucket",
|
||
AccessKeyID: "aliyun-access", AccessKeySecret: "aliyun-secret",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
historical, _, err := svc.readUserOSSSetting(actor.ID)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: qiniuKodoProvider, Endpoint: qiniuEndpoint.URL, CDNBaseURL: qiniuCDN.URL, Bucket: "qiniu-bucket",
|
||
AccessKeyID: "qiniu-access", AccessKeySecret: "qiniu-secret",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-bound-historical-storage", UserID: actor.ID, Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: aliyunOSSProvider, Endpoint: aliyunEndpoint.URL, Bucket: "aliyun-bucket", StorageSettingID: historical.ID,
|
||
ObjectKey: "ai/users/user-1/image/bound.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery(actor.ID, resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
want := aliyunCDN.URL + "/ai/users/user-1/image/bound.png"
|
||
if delivery.RedirectURL != want || strings.Contains(delivery.RedirectURL, qiniuCDN.URL) {
|
||
t.Fatalf("PrepareResourceDelivery(bound historical resource) = %q, want %q", delivery.RedirectURL, want)
|
||
}
|
||
}
|
||
|
||
func TestHistoricalUserResourceWithoutStorageSettingIDKeepsItsProviderCDN(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
newTestServer := func() *httptest.Server {
|
||
return httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
}
|
||
aliyunEndpoint := newTestServer()
|
||
aliyunCDN := newTestServer()
|
||
qiniuEndpoint := newTestServer()
|
||
qiniuCDN := newTestServer()
|
||
defer aliyunEndpoint.Close()
|
||
defer aliyunCDN.Close()
|
||
defer qiniuEndpoint.Close()
|
||
defer qiniuCDN.Close()
|
||
svc := newResourceTestService(t)
|
||
actor := &model.User{ID: "user-1"}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: aliyunEndpoint.URL, CDNBaseURL: aliyunCDN.URL, Bucket: "aliyun-bucket",
|
||
AccessKeyID: "aliyun-access", AccessKeySecret: "aliyun-secret",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{
|
||
Enabled: true, Provider: qiniuKodoProvider, Endpoint: qiniuEndpoint.URL, CDNBaseURL: qiniuCDN.URL, Bucket: "qiniu-bucket",
|
||
AccessKeyID: "qiniu-access", AccessKeySecret: "qiniu-secret",
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-legacy-user-storage", UserID: actor.ID, Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: aliyunOSSProvider, Endpoint: aliyunEndpoint.URL, Bucket: "aliyun-bucket",
|
||
ObjectKey: "ai/users/user-1/image/legacy.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery(actor.ID, resource.ID, ResourceDeliveryOptions{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
want := aliyunCDN.URL + "/ai/users/user-1/image/legacy.png"
|
||
if delivery.RedirectURL != want || strings.Contains(delivery.RedirectURL, qiniuCDN.URL) {
|
||
t.Fatalf("PrepareResourceDelivery(legacy user resource) = %q, want %q", delivery.RedirectURL, want)
|
||
}
|
||
}
|
||
|
||
func TestPrepareResourceDeliveryKeepsForcedOriginDirectWithoutCDN(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-origin-direct", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: aliyunOSSProvider, Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
ObjectKey: "users/user-1/image/direct.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
delivery, err := svc.PrepareResourceDelivery("user-1", resource.ID, ResourceDeliveryOptions{ForceDirect: true})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if !strings.Contains(delivery.RedirectURL, "private-bucket.oss-cn-test.aliyuncs.com/users/user-1/image/direct.png") || !strings.Contains(delivery.RedirectURL, "Signature=") {
|
||
t.Fatalf("PrepareResourceDelivery(force direct) = %#v", delivery)
|
||
}
|
||
}
|
||
|
||
func TestNormalizeSingleByteRange(t *testing.T) {
|
||
tests := map[string]string{
|
||
"bytes=0-1023": "bytes=0-1023",
|
||
"bytes=1024-": "bytes=1024-",
|
||
"bytes=-2048": "bytes=-2048",
|
||
"bytes=0-1,10-20": "",
|
||
"items=0-10": "",
|
||
"bytes=invalid-1024": "",
|
||
}
|
||
for input, expected := range tests {
|
||
if actual := normalizeSingleByteRange(input); actual != expected {
|
||
t.Fatalf("normalizeSingleByteRange(%q) = %q, want %q", input, actual, expected)
|
||
}
|
||
}
|
||
}
|
||
|
||
func TestHydrateNewAPIChannel1ResourceUsesSignedOSSURL(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{
|
||
Enabled: true, Provider: "aliyun", Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
AccessKeyID: "access-id", AccessKeySecret: "secret-value",
|
||
})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{
|
||
ID: "resource-1", UserID: "user-1", Kind: "image", Status: model.ResourceStatusReady,
|
||
Provider: "aliyun", Endpoint: "https://oss-cn-test.aliyuncs.com", Bucket: "private-bucket",
|
||
ObjectKey: "users/user-1/image/reference.png", MimeType: "image/png",
|
||
}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
media := providerMedia{StorageKey: "resource:resource-1", DataURL: "data:image/png;base64,old"}
|
||
if err := svc.hydrateProviderMedia("user-1", &media, true); err != nil {
|
||
t.Fatalf("hydrateProviderMedia() error = %v", err)
|
||
}
|
||
if !strings.HasPrefix(media.URL, "https://private-bucket.oss-cn-test.aliyuncs.com/") || media.DataURL != "" || !strings.Contains(media.URL, "Signature=") {
|
||
t.Fatalf("media = %#v", media)
|
||
}
|
||
if err := svc.hydrateProviderMedia("other-user", &providerMedia{StorageKey: "resource:resource-1"}, true); err == nil {
|
||
t.Fatal("hydrateProviderMedia() allowed another user's resource")
|
||
}
|
||
}
|
||
|
||
func TestHydrateNewAPIChannel1ResourceUsesSignedLocalURL(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
svc := newResourceTestService(t)
|
||
settingJSON, _ := json.Marshal(ossSettingValue{Provider: "aliyun", PublicBaseURL: server.URL})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(settingJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
resource := model.Resource{ID: "resource-local", UserID: "user-1", Status: model.ResourceStatusReady, Provider: "local", ObjectKey: "local.png"}
|
||
if err := svc.repo.CreateResource(&resource); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
media := providerMedia{StorageKey: "resource:resource-local"}
|
||
if err := svc.hydrateProviderMedia("user-1", &media, true); err != nil {
|
||
t.Fatalf("hydrateProviderMedia() error = %v", err)
|
||
}
|
||
if !strings.HasPrefix(media.URL, server.URL+"/api/public/resources/resource-local/file/resource-local.png?") || !strings.Contains(media.URL, "signature=") || media.DataURL != "" {
|
||
t.Fatalf("media = %#v", media)
|
||
}
|
||
stored, err := svc.repo.Resource("resource-local")
|
||
if err != nil || stored.Provider != "local" {
|
||
t.Fatalf("resource provider changed: %#v, %v", stored, err)
|
||
}
|
||
}
|
||
|
||
func TestPublicResourceSignatureRejectsExpiredAndAlteredLinks(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
expires := strconv.FormatInt(time.Now().Add(time.Minute).Unix(), 10)
|
||
signature, err := svc.signPublicResource("resource-local", expires)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if err := svc.verifyPublicResourceSignature("resource-local", expires, signature); err != nil {
|
||
t.Fatalf("verifyPublicResourceSignature() error = %v", err)
|
||
}
|
||
if err := svc.verifyPublicResourceSignature("resource-other", expires, signature); err == nil {
|
||
t.Fatal("verifyPublicResourceSignature() accepted another resource ID")
|
||
}
|
||
expired := strconv.FormatInt(time.Now().Add(-time.Minute).Unix(), 10)
|
||
expiredSignature, err := svc.signPublicResource("resource-local", expired)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if err := svc.verifyPublicResourceSignature("resource-local", expired, expiredSignature); err == nil {
|
||
t.Fatal("verifyPublicResourceSignature() accepted an expired link")
|
||
}
|
||
}
|
||
|
||
func TestUpdateOSSSettingRequiresLocalServerAddress(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
svc := newResourceTestService(t)
|
||
admin := &model.User{ID: "admin-1", Role: model.UserRoleAdmin}
|
||
if _, err := svc.UpdateOSSSetting(admin, OSSSettingRequest{Provider: "aliyun"}); err == nil || !strings.Contains(err.Error(), "服务器访问地址") {
|
||
t.Fatalf("UpdateOSSSetting() error = %v", err)
|
||
}
|
||
if _, err := svc.UpdateOSSSetting(admin, OSSSettingRequest{Provider: "aliyun", PublicBaseURL: server.URL + "/api"}); err == nil || !strings.Contains(err.Error(), "不要包含 /api") {
|
||
t.Fatalf("UpdateOSSSetting(/api) error = %v", err)
|
||
}
|
||
setting, err := svc.UpdateOSSSetting(admin, OSSSettingRequest{Provider: "aliyun", PublicBaseURL: server.URL})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if setting.Enabled || setting.PublicBaseURL != server.URL {
|
||
t.Fatalf("setting = %#v", setting)
|
||
}
|
||
}
|
||
|
||
func TestActiveResourceOSSSettingPrefersUserVersion(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
svc := newResourceTestService(t)
|
||
systemJSON, _ := json.Marshal(ossSettingValue{Enabled: true, Provider: "aliyun", Endpoint: server.URL, Bucket: "system", AccessKeyID: "system-id", AccessKeySecret: "system-secret"})
|
||
if err := svc.repo.SaveSystemSetting(&model.SystemSetting{Key: ossSettingKey, ValueJSON: string(systemJSON)}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
actor := &model.User{ID: "user-1"}
|
||
created, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{Enabled: true, Provider: "aliyun", Endpoint: server.URL, Bucket: "user", AccessKeyID: "user-id", AccessKeySecret: "user-secret"})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
setting, settingID, useOSS, err := svc.activeResourceOSSSetting(actor.ID)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if !useOSS || settingID == "" || setting.Bucket != "user" || !created.Enabled {
|
||
t.Fatalf("activeResourceOSSSetting() = %#v, %q, %v", setting, settingID, useOSS)
|
||
}
|
||
}
|
||
|
||
func TestUserOSSSettingVersionsKeepHistoricalSecrets(t *testing.T) {
|
||
t.Setenv("CANVAS_ALLOW_PRIVATE_UPSTREAMS", "true")
|
||
server := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
|
||
defer server.Close()
|
||
svc := newResourceTestService(t)
|
||
actor := &model.User{ID: "user-1"}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{Enabled: true, Provider: "aliyun", Endpoint: server.URL, Bucket: "old", AccessKeyID: "old-id", AccessKeySecret: "old-secret"}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
oldSetting, _, err := svc.readUserOSSSetting(actor.ID)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if _, err := svc.UpdateUserOSSSetting(actor, OSSSettingRequest{Enabled: true, Provider: "aliyun", Endpoint: server.URL, Bucket: "new", AccessKeyID: "new-id", AccessKeySecret: "new-secret"}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
_, oldValue, err := svc.readUserOSSSettingByID(actor.ID, oldSetting.ID)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if oldValue.Bucket != "old" || oldValue.AccessKeySecret != "old-secret" {
|
||
t.Fatalf("historical setting = %#v", oldValue)
|
||
}
|
||
}
|
||
|
||
func newResourceTestService(t *testing.T) *Service {
|
||
t.Helper()
|
||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if err := db.AutoMigrate(&model.SystemSetting{}, &model.UserOSSSetting{}, &model.StorageLocation{}, &model.UserDailyUploadUsage{}, &model.Resource{}, &model.SessionFile{}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
return &Service{repo: repository.New(db), dataDir: t.TempDir()}
|
||
}
|
||
|
||
func TestStoreResourceReusesReadyUploadIdentity(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
uploadKey := normalizedResourceUploadKey([]string{"image:user-1:logical-upload"})
|
||
first, stored, err := svc.storeResource("user-1", "image", "first.png", "image/png", 7, 1, 1, 0, bytes.NewReader([]byte("payload")), uploadKey)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if !stored {
|
||
t.Fatal("first upload was not stored")
|
||
}
|
||
second, stored, err := svc.storeResource("user-1", "image", "second.png", "image/png", 7, 1, 1, 0, bytes.NewReader([]byte("payload")), uploadKey)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if stored || second.ID != first.ID || second.ObjectKey != first.ObjectKey {
|
||
t.Fatalf("idempotent upload = %#v, stored=%v; first=%#v", second, stored, first)
|
||
}
|
||
resources, err := svc.repo.Resources("user-1", 10)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if len(resources) != 1 {
|
||
t.Fatalf("resource count = %d, want 1", len(resources))
|
||
}
|
||
}
|
||
|
||
func TestRetryStoredResourceKeepsOriginalObjectKey(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
uploadKey := normalizedResourceUploadKey([]string{"image:user-1:retry-upload"})
|
||
failed := &model.Resource{
|
||
ID: "resource-failed", UserID: "user-1", Kind: "image", Status: model.ResourceStatusFailed,
|
||
Provider: "local", ObjectKey: "users/user-1/image/fixed.png", MimeType: "image/png", Size: 7,
|
||
UploadKey: uploadKey, CreatedAt: time.Now(), UpdatedAt: time.Now(),
|
||
}
|
||
if err := svc.repo.CreateResource(failed); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
retried, err := svc.retryStoredResource("user-1", failed, "image", "image/png", 7, bytes.NewReader([]byte("payload")))
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if retried.ID != failed.ID || retried.ObjectKey != "users/user-1/image/fixed.png" || retried.Status != model.ResourceStatusReady {
|
||
t.Fatalf("retried resource = %#v", retried)
|
||
}
|
||
resources, err := svc.repo.Resources("user-1", 10)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if len(resources) != 1 {
|
||
t.Fatalf("resource count = %d, want 1", len(resources))
|
||
}
|
||
day := time.Now().UTC().Format("2006-01-02")
|
||
usage, err := svc.repo.DailyUploadBytes("user-1", day)
|
||
if err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
if usage != 7 {
|
||
t.Fatalf("daily upload usage = %d, want 7", usage)
|
||
}
|
||
}
|
||
|
||
func TestRetryStoredResourceReleasesDailyQuotaAfterFailure(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
uploadKey := normalizedResourceUploadKey([]string{"image:user-1:failed-retry"})
|
||
failed := &model.Resource{
|
||
ID: "resource-failed-retry", UserID: "user-1", Kind: "image", Status: model.ResourceStatusFailed,
|
||
Provider: "local", ObjectKey: "users/user-1/image/failed.png", MimeType: "image/png", Size: 7,
|
||
UploadKey: uploadKey, CreatedAt: time.Now(), UpdatedAt: time.Now(),
|
||
}
|
||
if err := svc.repo.CreateResource(failed); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
_, err := svc.retryStoredResource("user-1", failed, "image", "image/png", 7, iotest.ErrReader(errors.New("write failed")))
|
||
if err == nil || !strings.Contains(err.Error(), "write failed") {
|
||
t.Fatalf("retryStoredResource() error = %v", err)
|
||
}
|
||
day := time.Now().UTC().Format("2006-01-02")
|
||
usage, usageErr := svc.repo.DailyUploadBytes("user-1", day)
|
||
if usageErr != nil {
|
||
t.Fatal(usageErr)
|
||
}
|
||
if usage != 0 {
|
||
t.Fatalf("daily upload usage = %d, want 0", usage)
|
||
}
|
||
}
|
||
|
||
func TestLegacyMediaMigrationSkipsInvalidDataURL(t *testing.T) {
|
||
svc := &Service{}
|
||
input := map[string]interface{}{
|
||
"history": []interface{}{
|
||
map[string]interface{}{"content": "data:video/mp4;base64,broken"},
|
||
},
|
||
}
|
||
|
||
result, err := svc.persistLegacyGeneratedMediaResult("user-1", input)
|
||
if err != nil {
|
||
t.Fatalf("persistLegacyGeneratedMediaResult() error = %v", err)
|
||
}
|
||
history := result["history"].([]interface{})
|
||
content := history[0].(map[string]interface{})["content"]
|
||
if content != "data:video/mp4;base64,broken" {
|
||
t.Fatalf("invalid legacy content changed to %v", content)
|
||
}
|
||
}
|
||
|
||
func TestGeneratedMediaRejectsInvalidDataURL(t *testing.T) {
|
||
svc := &Service{}
|
||
_, err := svc.persistGeneratedMediaResult("user-1", map[string]interface{}{
|
||
"content": "data:video/mp4;base64,broken",
|
||
})
|
||
if err == nil {
|
||
t.Fatal("persistGeneratedMediaResult() error = nil, want invalid data URL error")
|
||
}
|
||
}
|
||
|
||
func TestPersistGeneratedMediaAppliesStoredFileQuota(t *testing.T) {
|
||
svc := newResourceTestService(t)
|
||
if err := svc.repo.Create(&model.Resource{
|
||
ID: "existing",
|
||
UserID: "user-1",
|
||
Status: model.ResourceStatusReady,
|
||
Size: gigabytes(defaultRuntimePolicy().Resource.StoredFileGB) - 1,
|
||
}); err != nil {
|
||
t.Fatal(err)
|
||
}
|
||
|
||
_, err := svc.persistGeneratedMediaResult("user-1", map[string]interface{}{
|
||
"image": map[string]interface{}{"dataUrl": "data:image/png;base64,YQ=="},
|
||
})
|
||
if err == nil || !strings.Contains(err.Error(), "20GB 上限") {
|
||
t.Fatalf("persistGeneratedMediaResult() error = %v", err)
|
||
}
|
||
}
|