feat(eval): ASR 模型评估框架
横向对比云端 gummy 与本地开源模型(faster-whisper/SenseVoice/Paraformer), 重点覆盖中英混说,产出准确率(CER/WER/MER)/速度(延迟/RTF)/资源(cpu/mem/模型大小) 对比报告。公共集(ASCEND/AISHELL/LibriSpeech)统一走 HF 适配器 + 自定义 JSONL manifest。 gummy 引擎对照 server/internal/asr/gummy.go 协议移植。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,32 @@
|
||||
"""统一样本模型:所有数据集适配器都产出 Sample,下游引擎/指标只认它。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class Sample:
|
||||
"""一条评估样本。
|
||||
|
||||
lang 取值约定(决定 breakdown 归类):
|
||||
- "zh" 纯普通话
|
||||
- "en" 纯英文
|
||||
- "zh-en" 中英混说(code-switching,本框架重点)
|
||||
"""
|
||||
|
||||
id: str
|
||||
audio_path: str # 16k/mono/16bit wav 优先;非此规格引擎侧会重采样
|
||||
ref_text: str # 参考(标注)文本
|
||||
lang: str = "zh"
|
||||
domain: str = "general"
|
||||
dataset: str = "custom"
|
||||
|
||||
@property
|
||||
def category(self) -> str:
|
||||
"""报告里的粗分类。"""
|
||||
if self.lang == "zh-en":
|
||||
return "code-switch"
|
||||
if self.lang == "en":
|
||||
return "pure-en"
|
||||
return "pure-zh"
|
||||
Reference in New Issue
Block a user