Compare commits
2 Commits
7e7206a8d4
...
7892e35d2d
| Author | SHA1 | Date |
|---|---|---|
 liwenhao2
|
7892e35d2d | |
|
liwenhao2
|
236037a0b0 |
|
|
@ -14,5 +14,11 @@
|
||||||
.cxx
|
.cxx
|
||||||
local.properties
|
local.properties
|
||||||
|
|
||||||
|
# 构建产物与日志不入库
|
||||||
|
/build-check/
|
||||||
|
*.log
|
||||||
|
/tools/llama-jni/llama.cpp/
|
||||||
|
/tools/llama-jni/build-*/
|
||||||
|
|
||||||
# 模型文件体积大，不入库；运行时手动放入 app/src/main/assets/
|
# 模型文件体积大，不入库；运行时手动放入 app/src/main/assets/
|
||||||
*.onnx
|
*.onnx
|
||||||
|
|
|
||||||
|
|
@ -52,6 +52,8 @@ dependencies {
|
||||||
implementation 'com.google.android.material:material:1.7.0'
|
implementation 'com.google.android.material:material:1.7.0'
|
||||||
implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
|
implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
|
||||||
testImplementation 'junit:junit:4.13.2'
|
testImplementation 'junit:junit:4.13.2'
|
||||||
|
// 摘要流水线的 JVM 单测需要真实 org.json(android.jar 里的是会抛 Stub! 的桩)
|
||||||
|
testImplementation 'org.json:json:20180813'
|
||||||
androidTestImplementation 'androidx.test.ext:junit:1.1.4'
|
androidTestImplementation 'androidx.test.ext:junit:1.1.4'
|
||||||
androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.0'
|
androidTestImplementation 'androidx.test.espresso:espresso-core:3.5.0'
|
||||||
}
|
}
|
||||||
|
|
@ -0,0 +1,139 @@
|
||||||
|
# 摘要功能三段提示词(里程碑 1)
|
||||||
|
|
||||||
|
> 适用模型:`gemma-3-1b-it`(ggml-org/gemma-3-1b-it-GGUF,Q4_K_M;2026-09-11 由
|
||||||
|
> gemma-4-E2B 换小提速。此前:`google/gemma-4-E2B-it-qat-q4_0-gguf`,架构 gemma4,Apache-2.0)
|
||||||
|
> 本文件是唯一事实源:App 内 `SummarizePipeline.kt` 逐字引用这里的文本。改提示词先改本文件。
|
||||||
|
> PC 调试方法见文末「用 PC llama.cpp 复现」。
|
||||||
|
|
||||||
|
## 0. 运行参数(与 App 实现一致)
|
||||||
|
|
||||||
|
| 步骤 | temperature | maxNewTokens | 备注 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| P1 滚动笔记更新 | 0(Greedy) | 900 | 可复现;笔记要求确定性 |
|
||||||
|
| P2 成稿 | 0.5 / topK 64 / topP 0.95 | 2500 | Gemma 4 官方推荐采样档位,低温折中 |
|
||||||
|
|
||||||
|
思考(thinking)关闭:App 渲染模板时不注入 `<|think|>` 标记;模型若仍输出 `<|channel>thought…` 块,App 侧正则剔除。
|
||||||
|
|
||||||
|
## 1. 聊天模板(App 手工渲染,与 Gemma 4 canonical template 单轮路径等价)
|
||||||
|
|
||||||
|
```
|
||||||
|
<bos>(tokenizer 自动)
|
||||||
|
<|turn>system
|
||||||
|
{SYSTEM}<turn|>
|
||||||
|
<|turn>user
|
||||||
|
{USER}<turn|>
|
||||||
|
<|turn>model
|
||||||
|
```
|
||||||
|
|
||||||
|
特殊 token id:`<bos>`=2,`<eos>`=1,`<|turn>`=105,`<turn|>`=106。生成遇 `<eos>`、`<turn|>`、或 8192 上下文耗尽即停。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P1 — 滚动笔记更新
|
||||||
|
|
||||||
|
### SYSTEM(固定,勿动态改写)
|
||||||
|
|
||||||
|
```
|
||||||
|
あなたは会議の文字起こしを処理する記録係です。与えられた「現在のメモ」に「新しい断片」の内容を統合し、更新後のメモをJSONだけで出力してください。
|
||||||
|
|
||||||
|
厳守事項:
|
||||||
|
- 出力は指定スキーマのJSONオブジェクトのみ。説明文・コードフェンス・前置き禁止。
|
||||||
|
- メモは日本語で書く。ただし人名・組織名・製品名・固有名詞は原文表記をそのまま維持。
|
||||||
|
- 数字、金額、日時、期限、回数、単位は原文どおり正確に転記。ざっくり要約しない。
|
||||||
|
- 否定・保留・反対の意図を消さない。「見送り」「再検討」「反対意見」はそれ自体が重要な情報。
|
||||||
|
- 各項目に根拠となる発話のタイムレンジを「出典」に付ける(形式: "mm:ss-mm:ss"、複数可 ";" 区切り)。
|
||||||
|
- 断片に新情報がない場合はメモをほぼ変更しない(前回末尾は必ず更新)。
|
||||||
|
- 全体で500字を超えそうな場合、重要度の低い過去の項目を統合・削除して圧縮する。削除時は行動項目と已定事項を最後に守る。
|
||||||
|
- 「前回末尾」には、次の断片と文脈が繋がるための直近話題を1文で書く。
|
||||||
|
- JSONの値の中にダブルクォートが必要なら \" にエスケープ。改行は \n を使う。
|
||||||
|
|
||||||
|
スキーマ(キー名は変更禁止):
|
||||||
|
{"已定事項":[{"内容":"","出典":""}],"討論中":[{"内容":"","出典":""}],"行動項目":[{"内容":"","担当者":"","期限":"","出典":""}],"人物":[{"名前":"","立場":""}],"前回末尾":""}
|
||||||
|
```
|
||||||
|
|
||||||
|
### USER(每块渲染;`{}` 为占位符)
|
||||||
|
|
||||||
|
```
|
||||||
|
【現在のメモ】
|
||||||
|
{notes_json}
|
||||||
|
|
||||||
|
【新しい断片】(タイムスタンプは会議開始からの分:秒、[]内は話者ラベル)
|
||||||
|
{chunk_text}
|
||||||
|
|
||||||
|
更新後のメモJSONを出力してください。
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P2 — 最终成稿
|
||||||
|
|
||||||
|
### SYSTEM(固定)
|
||||||
|
|
||||||
|
```
|
||||||
|
あなたは会議の公式議事録作成者です。会議メモ(JSON)と用語集を受け取ったら、指定言語で完成した議事録をMarkdownで出力してください。
|
||||||
|
|
||||||
|
厳守事項:
|
||||||
|
- 出力言語は指定された「成稿言語」を厳守。日本語メモの内容をそのまま指定言語の書き言葉文体で記述する。
|
||||||
|
- 本文は成稿言語の文字のみで書く:中文稿なら仮名を一切出さない、English稿なら漢字・仮名を一切出さない、日本語稿なら製品名以外に英語の文を残さない。
|
||||||
|
- ただし人物・会社・ブランドなどの固有名詞は、メモ・用語集にある原文表記のまま保留してよい(上記の文字制限の例外)。
|
||||||
|
- 見出しは次の4つを必ずこの順で:「議題と経過」「決定事項」「行動項目」「未決事項・持ち帰り」。該当ゼロなら「(なし)」と明記。
|
||||||
|
- 行動項目は表形式(担当者 / 内容 / 期限 / 出典)。担当者が不明なら「未定」と書く。
|
||||||
|
- 各項目の末尾に出典を付ける。形式: `出典: mm:ss-mm:ss`。出典はメモJSONの「出典」フィールドのみを使う。捏造禁止。
|
||||||
|
- メモにない事実・推測・補足説明を追加しない。
|
||||||
|
- 人名・製品名は用語集の正式表記に統一する(用語集にない表記はメモのまま採用)。
|
||||||
|
- 冒頭にメタ情報を置く: 会議日時(指定があれば)、話者数、使用言語、文字起こし総尺。
|
||||||
|
- Markdown本文のみ出力。コードフェンスで全体を囲まない。
|
||||||
|
```
|
||||||
|
|
||||||
|
### USER
|
||||||
|
|
||||||
|
```
|
||||||
|
【成稿言語】{language_name}
|
||||||
|
|
||||||
|
【会議メタ情報】開始: {meeting_start} / 話者数: {n_speakers} / 総尺: {total_duration}
|
||||||
|
|
||||||
|
【用語集(正式表記の一覧。人名・製品名の誤変換をこれに統一)】
|
||||||
|
{glossary_lines}
|
||||||
|
|
||||||
|
【会議メモJSON】
|
||||||
|
{notes_json}
|
||||||
|
|
||||||
|
上記に基づき、議事録Markdownを出力してください。
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 语言名映射(P2 `{language_name}`)
|
||||||
|
|
||||||
|
| 会议语言 | language_name |
|
||||||
|
|---|---|
|
||||||
|
| ja | 日本語 |
|
||||||
|
| zh | 中文(简体书面语) |
|
||||||
|
| en | English (formal written English) |
|
||||||
|
|
||||||
|
## 分块参数(Kotlin 侧,不进提示词)
|
||||||
|
|
||||||
|
- 目标 2500±500 字/块,按句子边界聚合,块间重叠 1 句;同一说话人轮次不拆开。
|
||||||
|
- 行格式:`(mm:ss-mm:ss)[话者名] 文本`,时间戳来自 SessionTranscript。
|
||||||
|
- 首轮 `{notes_json}` = 空 schema(`{"已定事項":[],"討論中":[],"行動項目":[],"人物":[],"前回末尾":""}`)。
|
||||||
|
- 单块+笔记的总输入若超过 ~5000 字符,Kotlin 先把该块对半拆成两次 P1 调用(块长自适应,不截断内容)。
|
||||||
|
|
||||||
|
## 用 PC llama.cpp 复现(提示词调优用)
|
||||||
|
|
||||||
|
1. 取 llama.cpp master(≥2026-09;gemma3 架构早已支持,App 的 `libsummarize-llm-jni.so` 无需重编)。
|
||||||
|
2. 模型:`gemma-3-1b-it-Q4_K_M.gguf`(ggml-org/gemma-3-1b-it-GGUF)。
|
||||||
|
3. 跑 P1 示例(替换 sys/user 两段;`--jinja` 走 GGUF 内置模板,与 App 的
|
||||||
|
`renderGemma3Prompt`——system 并入首个 user turn——等价):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
llama-cli -m gemma-3-1b-it-Q4_K_M.gguf -c 8192 -ctk q8_0 -ctv f16 --jinja \
|
||||||
|
--temp 0 -n 900 \
|
||||||
|
--sys "$(sed -n '/^### SYSTEM/,/^```$/p' prompts.md | sed '1,2d;$d')" \
|
||||||
|
-p "$USER_PROMPT"
|
||||||
|
```
|
||||||
|
|
||||||
|
4. 调优结果(好例/坏例)记录到本文件末尾「调优日志」。
|
||||||
|
|
||||||
|
## 调优日志
|
||||||
|
|
||||||
|
(空——等 PC/真机验证后追加)
|
||||||
|
|
@ -0,0 +1,151 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import android.util.Log
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// LlmEngine — libsummarize-llm-jni.so(llama.cpp + 薄 JNI)的 Kotlin 封装。
|
||||||
|
//
|
||||||
|
// 与工程其它 native wrapper(OnlineRecognizer/OfflineRecognizer/Extractor)
|
||||||
|
// 同范式:ptr + require + finalize()/release()。但注意:
|
||||||
|
// * 本库绝不能进录音/主线程路径 —— generate() 是长阻塞调用,设计为只在
|
||||||
|
// 摘要专用后台线程上使用,一次只能有一个 generate 在跑(JNI 内有锁)。
|
||||||
|
// * 模型文件必须由用户 adb push,App 不下载;调用 nativeNewFromFile 前
|
||||||
|
// 必须在 Kotlin 侧确认文件存在(工程教训:缺文件时有的 native 会 exit(),
|
||||||
|
// 本 .so 已做 access() 双保险并抛异常,但仍按规矩先查)。
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
/** 摘要 LLM 的可用性探测。so 只编了 arm64-v8a,x86_64 模拟器上自动不可用。 */
|
||||||
|
object LlmRuntime {
|
||||||
|
private const val LIB = "summarize-llm-jni"
|
||||||
|
@Volatile private var tried = false
|
||||||
|
@Volatile var nativeAvailable = false
|
||||||
|
private set
|
||||||
|
|
||||||
|
fun ensure(): Boolean {
|
||||||
|
if (!tried) {
|
||||||
|
synchronized(this) {
|
||||||
|
if (!tried) {
|
||||||
|
tried = true
|
||||||
|
nativeAvailable = try {
|
||||||
|
System.loadLibrary(LIB)
|
||||||
|
true
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.w("sherpa-onnx", "summarize lib $LIB not loadable: $t")
|
||||||
|
false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nativeAvailable
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
class LlmEngine(
|
||||||
|
modelPath: String,
|
||||||
|
ctxSize: Int = 8192, // 滚动笔记+单块输入远小于此(§3 资源账)
|
||||||
|
nThreads: Int = 6, // 8 Gen3 大核 4 个,给摘要留余量:6 线程够用
|
||||||
|
kvTypeK: String = "q8_0", // KV 用 q8_0 压显存/内存(§3)
|
||||||
|
kvTypeV: String = "f16", // V 保持 f16:K 量化到 q8_0 是 llama.cpp 推荐档
|
||||||
|
chatTemplate: String = "gemma4", // "gemma4" | "gemma3",须与 GGUF 内嵌模板一致
|
||||||
|
) {
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "sherpa-onnx"
|
||||||
|
|
||||||
|
// ---- Gemma 4 canonical template 的单轮(system+user)渲染 ----
|
||||||
|
// 与 GGUF 内嵌模板(enable_thinking=false, 无 tools)逐字等价:
|
||||||
|
// bos <|turn>system\nSYS<turn|>\n <|turn>user\nUSR<turn|>\n <|turn>model\n
|
||||||
|
// bos 由 tokenizer 的 add_bos_token 自动添加(native 侧 parse_special=true)。
|
||||||
|
fun renderGemma4Prompt(system: String, user: String): String =
|
||||||
|
buildString {
|
||||||
|
append("<|turn>system\n").append(system.trim()).append("<turn|>\n")
|
||||||
|
append("<|turn>user\n").append(user.trim()).append("<turn|>\n")
|
||||||
|
append("<|turn>model\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Gemma 3 官方模板的单轮渲染(ggml-org/gemma-3-1b-it-GGUF)----
|
||||||
|
// 官方 jinja:system 不作为独立 turn,而是拼进第一个 user turn;
|
||||||
|
// <start_of_turn>user\n{system}\n\n{user}<end_of_turn>\n<start_of_turn>model\n
|
||||||
|
// EOG token 是 <end_of_turn>(gemma3 的 eot),依赖 native 按 eog 停止。
|
||||||
|
fun renderGemma3Prompt(system: String, user: String): String =
|
||||||
|
buildString {
|
||||||
|
append("<start_of_turn>user\n")
|
||||||
|
append(system.trim()).append("\n\n")
|
||||||
|
append(user.trim())
|
||||||
|
append("<end_of_turn>\n")
|
||||||
|
append("<start_of_turn>model\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
fun renderChatPrompt(system: String, user: String,
|
||||||
|
template: String = "gemma4"): String =
|
||||||
|
when (template) {
|
||||||
|
"gemma3" -> renderGemma3Prompt(system, user)
|
||||||
|
else -> renderGemma4Prompt(system, user)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private val chatTemplate: String = chatTemplate
|
||||||
|
private var ptr: Long = 0L
|
||||||
|
|
||||||
|
init {
|
||||||
|
require(LlmRuntime.ensure()) { "摘要推理库不可用(设备 ABI 不支持或未打包)" }
|
||||||
|
require(existsFile(modelPath)) { "找不到模型文件: $modelPath" }
|
||||||
|
ptr = nativeNewFromFile(modelPath, ctxSize, nThreads, kvTypeK, kvTypeV)
|
||||||
|
require(ptr != 0L) { "LLM 模型加载失败: $modelPath" }
|
||||||
|
}
|
||||||
|
|
||||||
|
// 与工程教训对齐的存在性检查:加载任何模型文件前必须先确认文件在
|
||||||
|
private fun existsFile(path: String): Boolean =
|
||||||
|
java.io.File(path).let { it.isFile && it.length() > 100L }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 阻塞式生成(仅供摘要后台线程调用)。返回模型输出文本(不含输入)。
|
||||||
|
* temperature=0 为 Greedy。abort() 可从其它线程安全打断。
|
||||||
|
*/
|
||||||
|
fun generate(
|
||||||
|
systemPrompt: String,
|
||||||
|
userPrompt: String,
|
||||||
|
maxNewTokens: Int = 900,
|
||||||
|
temperature: Float = 0f,
|
||||||
|
topK: Int = 64,
|
||||||
|
topP: Float = 0.95f,
|
||||||
|
): String {
|
||||||
|
require(ptr != 0L) { "LlmEngine already released" }
|
||||||
|
return nativeGenerate(
|
||||||
|
ptr, renderChatPrompt(systemPrompt, userPrompt, chatTemplate),
|
||||||
|
maxNewTokens, temperature, topK, topP
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun abort() {
|
||||||
|
if (ptr != 0L) nativeAbort(ptr)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 模型内嵌的 jinja 聊天模板(调试用,验证渲染一致性)。 */
|
||||||
|
fun chatTemplate(): String? {
|
||||||
|
if (ptr == 0L) return null
|
||||||
|
return nativeGetStringMeta(ptr, "tokenizer.chat_template")
|
||||||
|
}
|
||||||
|
|
||||||
|
protected fun finalize() {
|
||||||
|
if (ptr != 0L) {
|
||||||
|
nativeFree(ptr) // JNI 内部等待在途 generate 结束后再释放
|
||||||
|
ptr = 0L
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun release() = finalize()
|
||||||
|
|
||||||
|
// ---- native(实现在 tools/llama-jni/src/summarize_llm_jni.cpp)----
|
||||||
|
private external fun nativeNewFromFile(
|
||||||
|
path: String, ctxSize: Int, nThreads: Int, typeK: String, typeV: String
|
||||||
|
): Long
|
||||||
|
|
||||||
|
private external fun nativeGenerate(
|
||||||
|
ptr: Long, prompt: String, maxNewTokens: Int,
|
||||||
|
temperature: Float, topK: Int, topP: Float
|
||||||
|
): String
|
||||||
|
|
||||||
|
private external fun nativeAbort(ptr: Long)
|
||||||
|
private external fun nativeFree(ptr: Long)
|
||||||
|
private external fun nativeGetStringMeta(ptr: Long, key: String): String?
|
||||||
|
}
|
||||||
|
|
@ -15,13 +15,17 @@ import android.os.Process
|
||||||
import android.os.SystemClock
|
import android.os.SystemClock
|
||||||
import android.text.SpannableStringBuilder
|
import android.text.SpannableStringBuilder
|
||||||
import android.text.Spanned
|
import android.text.Spanned
|
||||||
|
import android.text.TextPaint
|
||||||
|
import android.text.method.LinkMovementMethod
|
||||||
import android.text.method.ScrollingMovementMethod
|
import android.text.method.ScrollingMovementMethod
|
||||||
|
import android.text.style.ClickableSpan
|
||||||
import android.text.style.ForegroundColorSpan
|
import android.text.style.ForegroundColorSpan
|
||||||
import android.text.style.StyleSpan
|
import android.text.style.StyleSpan
|
||||||
import android.util.Log
|
import android.util.Log
|
||||||
import android.view.LayoutInflater
|
import android.view.LayoutInflater
|
||||||
import android.view.View
|
import android.view.View
|
||||||
import android.view.ViewGroup
|
import android.view.ViewGroup
|
||||||
|
import android.view.WindowManager
|
||||||
import android.widget.Button
|
import android.widget.Button
|
||||||
import android.widget.EditText
|
import android.widget.EditText
|
||||||
import android.widget.TextView
|
import android.widget.TextView
|
||||||
|
|
@ -146,11 +150,6 @@ class MainActivity : AppCompatActivity() {
|
||||||
private var activeSpeakerModel: String? = null
|
private var activeSpeakerModel: String? = null
|
||||||
private val speakerThreshold = 0.5f
|
private val speakerThreshold = 0.5f
|
||||||
private var speakersEnabled = false
|
private var speakersEnabled = false
|
||||||
private var lastEmbedding: FloatArray? = null
|
|
||||||
// 同一句话在 fast(轻量)空间的镜像嵌入,命名时双写进 fastRegistry
|
|
||||||
private var lastEmbeddingFast: FloatArray? = null
|
|
||||||
private lateinit var nameSpeakerButton: Button
|
|
||||||
private lateinit var clearSpeakersButton: Button
|
|
||||||
|
|
||||||
// ---- rolling speaker-change detection (需求一.4) -------------------------
|
// ---- rolling speaker-change detection (需求一.4) -------------------------
|
||||||
// Every ~1.5 s the speaker-check thread embeds the last 2 s window and
|
// Every ~1.5 s the speaker-check thread embeds the last 2 s window and
|
||||||
|
|
@ -166,6 +165,11 @@ class MainActivity : AppCompatActivity() {
|
||||||
private val speakerCheckThread = HandlerThread("speaker-window").also { it.start() }
|
private val speakerCheckThread = HandlerThread("speaker-window").also { it.start() }
|
||||||
private val speakerCheckHandler by lazy { Handler(speakerCheckThread.looper) }
|
private val speakerCheckHandler by lazy { Handler(speakerCheckThread.looper) }
|
||||||
private val speakerSwitchDrop = 0.30f // cos(window, reference) below this => switch
|
private val speakerSwitchDrop = 0.30f // cos(window, reference) below this => switch
|
||||||
|
// 最近邻兜底阈值(用户要求:未识别归到最相似的已识别人):
|
||||||
|
// endpoint 精确匹配不中→nearest≥0.35 归入;再不行才注册新人
|
||||||
|
private val verdictFallbackMin = 0.35f
|
||||||
|
// 滚动检测 fast 空间的兜底阈值(仅影响换行标签显示,endpoint 仍权威)
|
||||||
|
private val rollingFallbackMin = 0.30f
|
||||||
// 滚动检测在 fast 空间查"这人是谁"仅用于**显示标签**,阈值放宽到 0.4;
|
// 滚动检测在 fast 空间查"这人是谁"仅用于**显示标签**,阈值放宽到 0.4;
|
||||||
// endpoint 权威判定仍用准确模型 + speakerThreshold(0.5)。
|
// endpoint 权威判定仍用准确模型 + speakerThreshold(0.5)。
|
||||||
private val fastSearchThreshold = 0.4f
|
private val fastSearchThreshold = 0.4f
|
||||||
|
|
@ -177,6 +181,10 @@ class MainActivity : AppCompatActivity() {
|
||||||
@Volatile private var rollingLowHits = 0
|
@Volatile private var rollingLowHits = 0
|
||||||
// C(最新者胜):每次投递滚动检测都刷新序号,队列里被超额的旧任务直接跳过
|
// C(最新者胜):每次投递滚动检测都刷新序号,队列里被超额的旧任务直接跳过
|
||||||
@Volatile private var rollingSeq = 0L
|
@Volatile private var rollingSeq = 0L
|
||||||
|
// 兜底残料:当前句子槽位最近一次滚动窗口声纹。整句判定因碎片算不出
|
||||||
|
// embedding 时复用它做归属(用户要求 2026-09-11)
|
||||||
|
@Volatile private var rollingEmbSid = -1L
|
||||||
|
@Volatile private var rollingEmb: FloatArray? = null
|
||||||
@Volatile private var pendingSwitch: SpeakerSwitch? = null
|
@Volatile private var pendingSwitch: SpeakerSwitch? = null
|
||||||
@Volatile private var windowRefEmb: FloatArray? = null
|
@Volatile private var windowRefEmb: FloatArray? = null
|
||||||
@Volatile private var activeLabelForWindow: String? = null
|
@Volatile private var activeLabelForWindow: String? = null
|
||||||
|
|
@ -208,9 +216,6 @@ class MainActivity : AppCompatActivity() {
|
||||||
@Volatile
|
@Volatile
|
||||||
private var isRecording: Boolean = false
|
private var isRecording: Boolean = false
|
||||||
|
|
||||||
@Volatile
|
|
||||||
private var modelsReady: Boolean = false
|
|
||||||
|
|
||||||
// ---- External model loading (adb push) ---------------------------------
|
// ---- External model loading (adb push) ---------------------------------
|
||||||
// Models live in Android/data/<pkg>/files/models/ instead of assets:
|
// Models live in Android/data/<pkg>/files/models/ instead of assets:
|
||||||
// adb push <model-dir> /sdcard/Android/data/com.k2fsa.sherpa.onnx/files/models/
|
// adb push <model-dir> /sdcard/Android/data/com.k2fsa.sherpa.onnx/files/models/
|
||||||
|
|
@ -226,6 +231,31 @@ class MainActivity : AppCompatActivity() {
|
||||||
private var loadFromAssets = true
|
private var loadFromAssets = true
|
||||||
private var modelPrefix = ""
|
private var modelPrefix = ""
|
||||||
|
|
||||||
|
// ---- Summarize (离线 LLM 会议纪要,里程碑 1) ------------------------------
|
||||||
|
// §7 红线:总开关 false 时不建会话记录、不显示按钮,App 行为与加功能前
|
||||||
|
// 完全一致。摘要是独立世界:专用后台线程 + 独立 .so(llama.cpp),
|
||||||
|
// 不 import 任何 sherpa-onnx ASR native 对象。
|
||||||
|
private val summarizeEnabled = true
|
||||||
|
private val summarizeModelDirName = "gemma4-e2b"
|
||||||
|
private val summarizeGgufName = "gemma-4-E2B_q4_0-it.gguf"
|
||||||
|
private val summarizeChatTemplate = "gemma4"
|
||||||
|
private lateinit var summarizeButton: Button
|
||||||
|
private lateinit var copyMinutesButton: Button
|
||||||
|
private lateinit var minutesStatus: TextView
|
||||||
|
private lateinit var minutesText: TextView
|
||||||
|
private lateinit var minutesHint: TextView
|
||||||
|
private var sessionTranscript: SessionTranscript? = null
|
||||||
|
@Volatile private var isSummarizing = false
|
||||||
|
@Volatile private var summarizeCancelled = false
|
||||||
|
// generate 阻塞在 native,取消靠 abort;引擎只在摘要线程持有
|
||||||
|
@Volatile private var summarizeEngine: LlmEngine? = null
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private val SUMMARY_THREADS = 6 // 空闲时:大核全用
|
||||||
|
private val SUMMARY_THREADS_BUSY = 4 // 录音中成稿:给 ASR 让核(实测零丢帧)
|
||||||
|
private val SUMMARY_CTX = 8192 // §3 资源账:笔记+单块远小于此
|
||||||
|
}
|
||||||
|
|
||||||
override fun onRequestPermissionsResult(
|
override fun onRequestPermissionsResult(
|
||||||
requestCode: Int, permissions: Array<String>, grantResults: IntArray
|
requestCode: Int, permissions: Array<String>, grantResults: IntArray
|
||||||
) {
|
) {
|
||||||
|
|
@ -254,21 +284,24 @@ class MainActivity : AppCompatActivity() {
|
||||||
recordButton.isEnabled = false
|
recordButton.isEnabled = false
|
||||||
recordButton.setOnClickListener { onclick() }
|
recordButton.setOnClickListener { onclick() }
|
||||||
|
|
||||||
nameSpeakerButton = findViewById(R.id.name_speaker_button)
|
summarizeButton = findViewById(R.id.summarize_button)
|
||||||
nameSpeakerButton.isEnabled = false
|
copyMinutesButton = findViewById(R.id.copy_minutes_button)
|
||||||
nameSpeakerButton.setOnClickListener { onNameSpeakerClick() }
|
minutesStatus = findViewById(R.id.minutes_status)
|
||||||
|
minutesText = findViewById(R.id.minutes_text)
|
||||||
clearSpeakersButton = findViewById(R.id.clear_speakers_button)
|
minutesHint = findViewById(R.id.minutes_hint)
|
||||||
clearSpeakersButton.setOnClickListener {
|
if (summarizeEnabled) {
|
||||||
SpeakerRecognition.resetSpeakers()
|
// 生成中再点同一按钮 = 取消(按钮文案随之切换)
|
||||||
lastEmbedding = null
|
summarizeButton.setOnClickListener {
|
||||||
lastEmbeddingFast = null
|
if (isSummarizing) cancelSummarize() else onSummarizeClick()
|
||||||
lastLabel = null
|
|
||||||
lastConfirmedLabel = null
|
|
||||||
activeLabelForWindow = null
|
|
||||||
nameSpeakerButton.isEnabled = false
|
|
||||||
Toast.makeText(this, "已清除所有说话人记录", Toast.LENGTH_SHORT).show()
|
|
||||||
}
|
}
|
||||||
|
copyMinutesButton.isEnabled = false
|
||||||
|
copyMinutesButton.setOnClickListener { onCopyMinutesClick() }
|
||||||
|
} else {
|
||||||
|
summarizeButton.visibility = View.GONE
|
||||||
|
copyMinutesButton.visibility = View.GONE
|
||||||
|
}
|
||||||
|
loadLastMinutes()
|
||||||
|
hintUnfinishedMeeting()
|
||||||
|
|
||||||
textView = findViewById(R.id.my_text)
|
textView = findViewById(R.id.my_text)
|
||||||
textView.movementMethod = ScrollingMovementMethod()
|
textView.movementMethod = ScrollingMovementMethod()
|
||||||
|
|
@ -305,7 +338,6 @@ class MainActivity : AppCompatActivity() {
|
||||||
application.assets.list("").orEmpty()
|
application.assets.list("").orEmpty()
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
modelsReady = true
|
|
||||||
runOnUiThread { recordButton.isEnabled = true }
|
runOnUiThread { recordButton.isEnabled = true }
|
||||||
val speakerNote = if (speakersEnabled) "说话人功能已启用"
|
val speakerNote = if (speakersEnabled) "说话人功能已启用"
|
||||||
else "⚠ 说话人功能未启用:缺少声纹模型 ${speakerModels[0]}" +
|
else "⚠ 说话人功能未启用:缺少声纹模型 ${speakerModels[0]}" +
|
||||||
|
|
@ -360,6 +392,11 @@ class MainActivity : AppCompatActivity() {
|
||||||
override fun onDestroy() {
|
override fun onDestroy() {
|
||||||
super.onDestroy()
|
super.onDestroy()
|
||||||
isRecording = false
|
isRecording = false
|
||||||
|
sessionTranscript?.close()
|
||||||
|
if (isSummarizing) {
|
||||||
|
summarizeCancelled = true
|
||||||
|
summarizeEngine?.abort()
|
||||||
|
}
|
||||||
subtitleThread.quitSafely()
|
subtitleThread.quitSafely()
|
||||||
speakerCheckThread.quitSafely()
|
speakerCheckThread.quitSafely()
|
||||||
}
|
}
|
||||||
|
|
@ -375,6 +412,8 @@ class MainActivity : AppCompatActivity() {
|
||||||
audioRecord!!.startRecording()
|
audioRecord!!.startRecording()
|
||||||
recordButton.setText(R.string.stop)
|
recordButton.setText(R.string.stop)
|
||||||
isRecording = true
|
isRecording = true
|
||||||
|
// 会议记录场景:录音期间保持屏幕常亮,防息屏冻结/MIUI 杀后台
|
||||||
|
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||||
resetBuffer()
|
resetBuffer()
|
||||||
pendingSwitch = null
|
pendingSwitch = null
|
||||||
windowRefEmb = null
|
windowRefEmb = null
|
||||||
|
|
@ -383,6 +422,11 @@ class MainActivity : AppCompatActivity() {
|
||||||
activeLabelForWindow = null
|
activeLabelForWindow = null
|
||||||
lastConfirmedLabel = null
|
lastConfirmedLabel = null
|
||||||
uiHandler.post { resetSubtitles() }
|
uiHandler.post { resetSubtitles() }
|
||||||
|
if (summarizeEnabled) {
|
||||||
|
// 每次录音 = 一个新会话文件(§4.2);主线程操作
|
||||||
|
sessionTranscript?.close()
|
||||||
|
sessionTranscript = SessionTranscript.begin(filesDir)
|
||||||
|
}
|
||||||
|
|
||||||
recordingThread = thread(true) {
|
recordingThread = thread(true) {
|
||||||
processSamples()
|
processSamples()
|
||||||
|
|
@ -390,10 +434,16 @@ class MainActivity : AppCompatActivity() {
|
||||||
Log.i(TAG, "Started recording")
|
Log.i(TAG, "Started recording")
|
||||||
} else {
|
} else {
|
||||||
isRecording = false
|
isRecording = false
|
||||||
|
window.clearFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
|
||||||
audioRecord!!.stop()
|
audioRecord!!.stop()
|
||||||
audioRecord!!.release()
|
audioRecord!!.release()
|
||||||
audioRecord = null
|
audioRecord = null
|
||||||
recordButton.setText(R.string.start)
|
recordButton.setText(R.string.start)
|
||||||
|
if (summarizeEnabled) {
|
||||||
|
// 保留 sessionTranscript 引用直到下次开始录音:
|
||||||
|
// 「刚停止」是点生成纪要最常见的时间窗
|
||||||
|
sessionTranscript?.flush()
|
||||||
|
}
|
||||||
Log.i(TAG, "Stopped recording")
|
Log.i(TAG, "Stopped recording")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -452,6 +502,8 @@ class MainActivity : AppCompatActivity() {
|
||||||
win, sampleRateInHz
|
win, sampleRateInHz
|
||||||
) ?: return@post
|
) ?: return@post
|
||||||
if (rseq != rollingSeq) return@post
|
if (rseq != rollingSeq) return@post
|
||||||
|
rollingEmbSid = owner
|
||||||
|
rollingEmb = emb
|
||||||
// 计算耗时内句子可能已结算:回写前二次核销
|
// 计算耗时内句子可能已结算:回写前二次核销
|
||||||
if (seq != checkSerial) return@post
|
if (seq != checkSerial) return@post
|
||||||
val ref = windowRefEmb
|
val ref = windowRefEmb
|
||||||
|
|
@ -619,35 +671,7 @@ class MainActivity : AppCompatActivity() {
|
||||||
|
|
||||||
// 整句声纹判定移出录音线程(停摆根因):后台算完
|
// 整句声纹判定移出录音线程(停摆根因):后台算完
|
||||||
// 回主线程 confirmSpeaker 修正归属
|
// 回主线程 confirmSpeaker 修正归属
|
||||||
speakerCheckHandler.post {
|
postSpeakerVerdict(sid, snap)
|
||||||
val emb = SpeakerRecognition.computeEmbedding(
|
|
||||||
snap, sampleRateInHz
|
|
||||||
)
|
|
||||||
// 镜像嵌入:② 用句尾 2s 切片(与滚动检测同视角),
|
|
||||||
// 整句 embedding 会拉低窗口匹配度,导致换行时查不到人
|
|
||||||
val fastEmb = if (emb != null &&
|
|
||||||
SpeakerRecognition.fastReady()
|
|
||||||
) {
|
|
||||||
val tailSlice = if (snap.size > 32000)
|
|
||||||
snap.copyOfRange(
|
|
||||||
snap.size - 32000, snap.size
|
|
||||||
) else snap
|
|
||||||
SpeakerRecognition.computeEmbeddingFast(
|
|
||||||
tailSlice, sampleRateInHz
|
|
||||||
)
|
|
||||||
} else null
|
|
||||||
val label = if (emb == null) null else
|
|
||||||
SpeakerRecognition.registry.search(
|
|
||||||
emb, speakerThreshold
|
|
||||||
)
|
|
||||||
?: SpeakerRecognition.nextAutoLabel().also {
|
|
||||||
SpeakerRecognition.registry.addAuto(it, emb)
|
|
||||||
if (fastEmb != null)
|
|
||||||
SpeakerRecognition.fastRegistry
|
|
||||||
.addAuto(it, fastEmb)
|
|
||||||
}
|
|
||||||
uiHandler.post { confirmSpeaker(sid, label, emb, fastEmb) }
|
|
||||||
}
|
|
||||||
// 定格 + 送检立即走主线程队列,不等声纹
|
// 定格 + 送检立即走主线程队列,不等声纹
|
||||||
uiHandler.post {
|
uiHandler.post {
|
||||||
val accepted = submitRefineTask(
|
val accepted = submitRefineTask(
|
||||||
|
|
@ -661,9 +685,67 @@ class MainActivity : AppCompatActivity() {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Stop 兜底(软 endpoint):最后一句未完的转写同样走定格→判定→精修,
|
||||||
|
// 否则丢半句 + 灰"未识别"行滞留(用户反馈 2026-09-11)
|
||||||
|
if (sentenceId != -1L) {
|
||||||
|
val raw = onlineRecognizer.getResult(stream).text
|
||||||
|
val disp =
|
||||||
|
if (raw.isNotBlank() && isInAllowedLanguages(raw)) raw else ""
|
||||||
|
onlineRecognizer.reset(stream)
|
||||||
|
val sid = sentenceId
|
||||||
|
sentenceId = -1L
|
||||||
|
windowRefEmb = null
|
||||||
|
if (disp.isNotBlank() && bufferTotal > 0) {
|
||||||
|
val snap = flattenSamples()
|
||||||
|
val dur = bufferTotal / sampleRateInHz.toFloat()
|
||||||
|
resetBuffer()
|
||||||
|
postSpeakerVerdict(sid, snap)
|
||||||
|
uiHandler.post {
|
||||||
|
val accepted = submitRefineTask(
|
||||||
|
sid, snap, dur, "STOP", null, disp, disp
|
||||||
|
)
|
||||||
|
finalizeSentence(sid, accepted)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
resetBuffer()
|
||||||
|
uiHandler.post { finalizeSentence(sid, false) }
|
||||||
|
}
|
||||||
|
}
|
||||||
stream.release()
|
stream.release()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 整句声纹判定:后台算准确 embedding + fast 镜像(句尾 2s 切片,与滚动
|
||||||
|
* 检测同视角),注册后回主线程 confirmSpeaker 修正归属。
|
||||||
|
* 自然 endpoint / 强制断句 / Stop 软 endpoint 共用。
|
||||||
|
*/
|
||||||
|
private fun postSpeakerVerdict(sid: Long, snap: FloatArray) {
|
||||||
|
speakerCheckHandler.post {
|
||||||
|
var emb = SpeakerRecognition.computeEmbedding(snap, sampleRateInHz)
|
||||||
|
if (emb == null && rollingEmbSid == sid) {
|
||||||
|
// 兜底①:句子太碎、整句 embedding 算不出 → 复用该槽滚动窗口残料
|
||||||
|
emb = rollingEmb
|
||||||
|
}
|
||||||
|
val fastEmb = if (emb != null && SpeakerRecognition.fastReady()) {
|
||||||
|
val tailSlice = if (snap.size > 32000)
|
||||||
|
snap.copyOfRange(snap.size - 32000, snap.size) else snap
|
||||||
|
SpeakerRecognition.computeEmbeddingFast(tailSlice, sampleRateInHz)
|
||||||
|
} else null
|
||||||
|
val label = if (emb == null) null else
|
||||||
|
(SpeakerRecognition.registry.search(emb, speakerThreshold)
|
||||||
|
// 兜底②:精确匹配不中 → 归入最相似的已注册身份
|
||||||
|
?: SpeakerRecognition.registry.nearest(
|
||||||
|
emb, verdictFallbackMin
|
||||||
|
))
|
||||||
|
?: SpeakerRecognition.nextAutoLabel().also {
|
||||||
|
SpeakerRecognition.registry.addAuto(it, emb)
|
||||||
|
if (fastEmb != null)
|
||||||
|
SpeakerRecognition.fastRegistry.addAuto(it, fastEmb)
|
||||||
|
}
|
||||||
|
uiHandler.post { confirmSpeaker(sid, label, emb, fastEmb) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Keep only zh/ja/en output when firstPassLanguage == "auto":
|
// Keep only zh/ja/en output when firstPassLanguage == "auto":
|
||||||
// letters must be ASCII latin (en), kana (ja), or CJK hanzi (zh/ja);
|
// letters must be ASCII latin (en), kana (ja), or CJK hanzi (zh/ja);
|
||||||
// everything else (hangul, cyrillic, thai, arabic, accented latin ...)
|
// everything else (hangul, cyrillic, thai, arabic, accented latin ...)
|
||||||
|
|
@ -752,54 +834,83 @@ class MainActivity : AppCompatActivity() {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun onNameSpeakerClick() {
|
// 点击行首说话人标签(ClickableSpan)→ 命名或改名。
|
||||||
val emb = lastEmbedding
|
// internalKey:该行的内部声纹键("S0"/真名);null=归属未定,提示稍后再命名。
|
||||||
if (emb == null) {
|
private fun showNameSpeakerDialog(internalKey: String?) {
|
||||||
Toast.makeText(this, "先录一句话,再为这位说话人命名", Toast.LENGTH_SHORT).show()
|
if (internalKey == null) {
|
||||||
|
Toast.makeText(this, "该行声纹归属尚未确定,先说一句完整的话再命名",
|
||||||
|
Toast.LENGTH_SHORT).show()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
val input = EditText(this).apply { hint = "说话人名字，如 张三" }
|
val input = EditText(this).apply {
|
||||||
AlertDialog.Builder(this)
|
hint = "「${displayName(internalKey)}」的真名,如 张三"
|
||||||
.setTitle("为说话人「${displayName(lastLabel)}」命名")
|
|
||||||
.setView(input)
|
|
||||||
.setPositiveButton("注册") { _, _ ->
|
|
||||||
val name = input.text.toString().trim()
|
|
||||||
if (name.isNotEmpty()) {
|
|
||||||
SpeakerRecognition.registry.addNamed(name, emb)
|
|
||||||
lastEmbeddingFast?.let {
|
|
||||||
SpeakerRecognition.fastRegistry.addNamed(name, it)
|
|
||||||
}
|
}
|
||||||
relabelSpeakerLines(name)
|
AlertDialog.Builder(this)
|
||||||
Toast.makeText(this, "之后「$name」说话时会显示真名", Toast.LENGTH_SHORT).show()
|
.setTitle(if (autoLabelRegex.matches(internalKey))
|
||||||
|
"命名说话人「${displayName(internalKey)}」"
|
||||||
|
else "将「$internalKey」改名为…")
|
||||||
|
.setView(input)
|
||||||
|
.setPositiveButton("确定") { _, _ ->
|
||||||
|
val name = input.text.toString().trim()
|
||||||
|
if (name.isNotEmpty() && name != internalKey) {
|
||||||
|
renameSpeaker(internalKey, name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
.setNegativeButton("取消", null)
|
.setNegativeButton("取消", null)
|
||||||
.show()
|
.show()
|
||||||
}
|
}
|
||||||
|
|
||||||
// 命名后:当前会话里这位说话人的所有行立即改显真名(颜色跟人走);
|
private fun renameSpeaker(oldKey: String, name: String) {
|
||||||
// 之后的新行也会用真名(声纹 search 命名优先返回真名)。
|
// 真名不得长得像内部键/显示名,避免与 S{n}→已识别人N 的映射冲突
|
||||||
private fun relabelSpeakerLines(name: String) {
|
if (autoLabelRegex.matches(name) ||
|
||||||
val old = lastLabel
|
Regex("^(已识别人?\\d+|未识别\\d*)$").matches(name)) {
|
||||||
if (old == null) {
|
Toast.makeText(this, "名字不能是 S数字/已识别人N/未识别N 形式",
|
||||||
lastLabel = name
|
Toast.LENGTH_SHORT).show()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if (old != name) {
|
// 声纹库升级:S{n} 的嵌入转为真名注册(真名可多组嵌入并存,
|
||||||
val idx = labelColorIdx.remove(old) ?: colorIdxFor(name)
|
// 即"两个误判声纹归为同一人",今后 search 对两组嵌入都返回同名)
|
||||||
labelColorIdx[name] = idx
|
SpeakerRecognition.registry.renameToName(oldKey, name)
|
||||||
|
SpeakerRecognition.fastRegistry.renameToName(oldKey, name)
|
||||||
for (l in subtitleLines) {
|
for (l in subtitleLines) {
|
||||||
if (l.label == old) {
|
if (l.label == oldKey) {
|
||||||
l.label = name
|
l.label = name
|
||||||
l.colorIdx = idx
|
l.colorIdx = colorIdxFor(name)
|
||||||
val pos = subtitleLines.indexOf(l)
|
|
||||||
if (pos >= 0) subtitleAdapter.notifyItemChanged(pos)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
labelColorIdx.remove(oldKey)
|
||||||
|
if (lastConfirmedLabel == oldKey) lastConfirmedLabel = name
|
||||||
|
if (lastLabel == oldKey) lastLabel = name
|
||||||
|
if (activeLabelForWindow == oldKey) activeLabelForWindow = name
|
||||||
|
if (summarizeEnabled) sessionTranscript?.onRename(oldKey, name)
|
||||||
|
mergeAdjacentSameLabel()
|
||||||
|
subtitleAdapter.notifyItemRangeChanged(0, subtitleLines.size)
|
||||||
|
Log.i(TAG, "speaker renamed: $oldKey -> $name")
|
||||||
|
Toast.makeText(this, "已命名「$name」,相邻同名行已合并",
|
||||||
|
Toast.LENGTH_SHORT).show()
|
||||||
|
}
|
||||||
|
|
||||||
|
// 命名归并(用户确认的规则:只并相邻同名行,时间线不回改旧行)。
|
||||||
|
// 含未定格句子的行不并入 —— 保护"in-progress 只存在于最底行"不变量。
|
||||||
|
private fun mergeAdjacentSameLabel() {
|
||||||
|
var i = 0
|
||||||
|
while (i < subtitleLines.size - 1) {
|
||||||
|
val a = subtitleLines[i]
|
||||||
|
val b = subtitleLines[i + 1]
|
||||||
|
if (a.label != null && a.label == b.label &&
|
||||||
|
b.sentences.none { !it.finalized }
|
||||||
|
) {
|
||||||
|
Log.i(TAG, "rename merge: line@${i + 1} -> line@$i (${a.label})")
|
||||||
|
for (s in b.sentences.toList()) {
|
||||||
|
s.line = a
|
||||||
|
a.sentences.add(s)
|
||||||
|
}
|
||||||
|
subtitleLines.removeAt(i + 1)
|
||||||
|
subtitleAdapter.notifyItemRemoved(i + 1)
|
||||||
|
} else {
|
||||||
|
i++
|
||||||
|
}
|
||||||
}
|
}
|
||||||
lastLabel = name
|
|
||||||
lastConfirmedLabel = name
|
|
||||||
activeLabelForWindow = name
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun initMicrophone(): Boolean {
|
private fun initMicrophone(): Boolean {
|
||||||
|
|
@ -1106,10 +1217,15 @@ class MainActivity : AppCompatActivity() {
|
||||||
private inner class SubtitleAdapter : RecyclerView.Adapter<SubtitleAdapter.VH>() {
|
private inner class SubtitleAdapter : RecyclerView.Adapter<SubtitleAdapter.VH>() {
|
||||||
inner class VH(val text: TextView) : RecyclerView.ViewHolder(text)
|
inner class VH(val text: TextView) : RecyclerView.ViewHolder(text)
|
||||||
|
|
||||||
override fun onCreateViewHolder(parent: ViewGroup, type: Int): VH = VH(
|
override fun onCreateViewHolder(parent: ViewGroup, type: Int): VH {
|
||||||
|
val vh = VH(
|
||||||
LayoutInflater.from(parent.context)
|
LayoutInflater.from(parent.context)
|
||||||
.inflate(R.layout.item_subtitle, parent, false) as TextView
|
.inflate(R.layout.item_subtitle, parent, false) as TextView
|
||||||
)
|
)
|
||||||
|
// 行首标签是 ClickableSpan;没有 movementMethod 点击不会生效
|
||||||
|
vh.text.movementMethod = LinkMovementMethod.getInstance()
|
||||||
|
return vh
|
||||||
|
}
|
||||||
|
|
||||||
override fun getItemCount(): Int = subtitleLines.size
|
override fun getItemCount(): Int = subtitleLines.size
|
||||||
|
|
||||||
|
|
@ -1132,6 +1248,18 @@ class MainActivity : AppCompatActivity() {
|
||||||
StyleSpan(Typeface.BOLD), start, start + head.length,
|
StyleSpan(Typeface.BOLD), start, start + head.length,
|
||||||
Spanned.SPAN_EXCLUSIVE_EXCLUSIVE
|
Spanned.SPAN_EXCLUSIVE_EXCLUSIVE
|
||||||
)
|
)
|
||||||
|
// 点击标签 → 命名/改名(绑定时捕获内部键,重命名后整表重绑)
|
||||||
|
val internalKey = line.label
|
||||||
|
sb.setSpan(object : ClickableSpan() {
|
||||||
|
override fun onClick(widget: View) {
|
||||||
|
showNameSpeakerDialog(internalKey)
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun updateDrawState(ds: TextPaint) {
|
||||||
|
ds.color = color
|
||||||
|
ds.isUnderlineText = true
|
||||||
|
}
|
||||||
|
}, start, start + head.length, Spanned.SPAN_EXCLUSIVE_EXCLUSIVE)
|
||||||
for ((i, s) in line.sentences.withIndex()) {
|
for ((i, s) in line.sentences.withIndex()) {
|
||||||
if (s.text.isBlank()) continue
|
if (s.text.isBlank()) continue
|
||||||
if (sb.length > start + head.length) {
|
if (sb.length > start + head.length) {
|
||||||
|
|
@ -1146,16 +1274,7 @@ class MainActivity : AppCompatActivity() {
|
||||||
return sb
|
return sb
|
||||||
}
|
}
|
||||||
|
|
||||||
// 句间衔接:已带标点则空格;CJK/假名结尾用句号;拉丁结尾用英文句号。
|
// 句间衔接已提出为顶层函数 SubtitleFormat.kt#joinerAfter(可 JVM 单测)
|
||||||
private fun joinerAfter(prev: String): String {
|
|
||||||
val last = prev.trimEnd().lastOrNull() ?: return " "
|
|
||||||
return when {
|
|
||||||
last in "。！？；.!?;…\"”’』】》）)" -> " "
|
|
||||||
last.code >= 0x2E80 -> "。"
|
|
||||||
Character.isLetterOrDigit(last) -> ". "
|
|
||||||
else -> " "
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun resetSubtitles() {
|
private fun resetSubtitles() {
|
||||||
val n = subtitleLines.size
|
val n = subtitleLines.size
|
||||||
|
|
@ -1170,14 +1289,14 @@ class MainActivity : AppCompatActivity() {
|
||||||
subtitleRecycler.visibility = View.VISIBLE
|
subtitleRecycler.visibility = View.VISIBLE
|
||||||
}
|
}
|
||||||
|
|
||||||
// 内部标签 S0/S1/…(注册键,可被命名为真名)对外显示为 未识别1/未识别2…;
|
// 内部标签 S0/S1/…(已注册、能区分但尚未起名的身份)对外显示"已识别人1/2…";
|
||||||
// 已命名者直接显示真名。
|
// 已命名者显示真名;label==null(完全无法归属)才显示灰色"未识别"。
|
||||||
private val autoLabelRegex = Regex("^S(\\d+)$")
|
private val autoLabelRegex = Regex("^S(\\d+)$")
|
||||||
|
|
||||||
private fun displayName(label: String?): String {
|
private fun displayName(label: String?): String {
|
||||||
if (label == null) return "未识别"
|
if (label == null) return "未识别"
|
||||||
val m = autoLabelRegex.matchEntire(label)
|
val m = autoLabelRegex.matchEntire(label)
|
||||||
return if (m != null) "未识别${m.groupValues[1].toInt() + 1}" else label
|
return if (m != null) "已识别人${m.groupValues[1].toInt() + 1}" else label
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun colorIdxFor(label: String): Int =
|
private fun colorIdxFor(label: String): Int =
|
||||||
|
|
@ -1242,6 +1361,12 @@ class MainActivity : AppCompatActivity() {
|
||||||
val sentence = sentenceById[sid] ?: return
|
val sentence = sentenceById[sid] ?: return
|
||||||
if (sentence.finalized) return
|
if (sentence.finalized) return
|
||||||
sentence.finalized = true
|
sentence.finalized = true
|
||||||
|
if (summarizeEnabled) {
|
||||||
|
// 会话记录器挂钩①(settle):时长取 slotBook 记账(SKIP_SHORT 时无
|
||||||
|
// 记录,按 0 处理 —— startMs≈endMs,不影响纪要内容)
|
||||||
|
val durMs = ((slotBook[sid]?.durSec ?: 0f) * 1000f).toLong()
|
||||||
|
sessionTranscript?.onSettled(sid, durMs, sentence.text)
|
||||||
|
}
|
||||||
if (!speakersEnabled) {
|
if (!speakersEnabled) {
|
||||||
// 声纹缺失:退化为按 endpoint 分行(显示"未识别")
|
// 声纹缺失:退化为按 endpoint 分行(显示"未识别")
|
||||||
sentence.line?.open = false
|
sentence.line?.open = false
|
||||||
|
|
@ -1277,10 +1402,9 @@ class MainActivity : AppCompatActivity() {
|
||||||
) {
|
) {
|
||||||
Log.i(TAG, "SLOT sid=$sid VERDICT speaker=" +
|
Log.i(TAG, "SLOT sid=$sid VERDICT speaker=" +
|
||||||
(label?.let { displayName(it) } ?: "未匹配/无判定"))
|
(label?.let { displayName(it) } ?: "未匹配/无判定"))
|
||||||
if (emb != null) {
|
if (summarizeEnabled && label != null) {
|
||||||
lastEmbedding = emb
|
// 会话记录器挂钩③(说话人归属;改名走 renameSpeaker 的 RENAME 事件)
|
||||||
lastEmbeddingFast = fastEmb
|
sessionTranscript?.onSpeaker(sid, label)
|
||||||
nameSpeakerButton.isEnabled = true
|
|
||||||
}
|
}
|
||||||
val sentence = findSentence(sid) ?: return
|
val sentence = findSentence(sid) ?: return
|
||||||
var line = sentence.line ?: return
|
var line = sentence.line ?: return
|
||||||
|
|
@ -1424,6 +1548,10 @@ class MainActivity : AppCompatActivity() {
|
||||||
} else {
|
} else {
|
||||||
sentence.refined = false // 未精修标记
|
sentence.refined = false // 未精修标记
|
||||||
}
|
}
|
||||||
|
if (summarizeEnabled) {
|
||||||
|
// 会话记录器挂钩②(终稿文本):与字幕层看到同一最终结果
|
||||||
|
sessionTranscript?.onRefined(sid, sentence.text, sentence.refined)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 换人强制断句后:封掉旧行,之后的 partial 开新行并归属新说话人。
|
// 换人强制断句后:封掉旧行,之后的 partial 开新行并归属新说话人。
|
||||||
|
|
@ -1439,8 +1567,14 @@ class MainActivity : AppCompatActivity() {
|
||||||
private fun rollingSpeakerSearch(emb: FloatArray): String? =
|
private fun rollingSpeakerSearch(emb: FloatArray): String? =
|
||||||
if (SpeakerRecognition.fastReady())
|
if (SpeakerRecognition.fastReady())
|
||||||
SpeakerRecognition.fastRegistry.search(emb, fastSearchThreshold)
|
SpeakerRecognition.fastRegistry.search(emb, fastSearchThreshold)
|
||||||
|
?: SpeakerRecognition.fastRegistry.nearest(
|
||||||
|
emb, rollingFallbackMin
|
||||||
|
)
|
||||||
else
|
else
|
||||||
SpeakerRecognition.registry.search(emb, speakerThreshold)
|
SpeakerRecognition.registry.search(emb, speakerThreshold)
|
||||||
|
?: SpeakerRecognition.registry.nearest(
|
||||||
|
emb, verdictFallbackMin
|
||||||
|
)
|
||||||
|
|
||||||
// ③ 滚动检测查到了已知人、但当前行还没有归属(占位"未识别"):
|
// ③ 滚动检测查到了已知人、但当前行还没有归属(占位"未识别"):
|
||||||
// 直接把活跃归属采纳为他,等 endpoint 权威判定给行冠名,不再多切一行
|
// 直接把活跃归属采纳为他,等 endpoint 权威判定给行冠名,不再多切一行
|
||||||
|
|
@ -1448,6 +1582,15 @@ class MainActivity : AppCompatActivity() {
|
||||||
if (lastConfirmedLabel == null) {
|
if (lastConfirmedLabel == null) {
|
||||||
lastConfirmedLabel = label
|
lastConfirmedLabel = label
|
||||||
activeLabelForWindow = label
|
activeLabelForWindow = label
|
||||||
|
// 修"灰色未识别滞留"来源之一:滚动已匹配到人,顺带给当前灰色
|
||||||
|
// 占位行冠名(能并回上方同人行则并),不必等 endpoint 权威判定
|
||||||
|
subtitleLines.lastOrNull()
|
||||||
|
?.takeIf { it.open && it.label == null }?.let { gray ->
|
||||||
|
gray.label = label
|
||||||
|
gray.colorIdx = colorIdxFor(label)
|
||||||
|
if (!mergeLineIntoPrevIfSame(gray, label)) notifyLine(gray)
|
||||||
|
Log.i(TAG, "adopt named gray line -> ${displayName(label)}")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1474,4 +1617,234 @@ class MainActivity : AppCompatActivity() {
|
||||||
subtitleRecycler.scrollToPosition(last)
|
subtitleRecycler.scrollToPosition(last)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ==================== Summarize(会中/会后一键成稿,§4.2)================
|
||||||
|
// 录音中点击 = 对截至当时已输出的全部定稿字幕生成纪要,录音转写全程不停。
|
||||||
|
// 进度与结果内嵌在下半屏「会议纪要」区(生成中旧纪要不清空);成稿成功
|
||||||
|
// 落 <会话>.notes.json,下次点击只处理新增句子续写笔记,纪要始终覆盖
|
||||||
|
// "开会至今"全量。模型点击才 mmap、完成即 release(§4.2 模型生命周期)。
|
||||||
|
private fun onSummarizeClick() {
|
||||||
|
if (isSummarizing) {
|
||||||
|
Toast.makeText(this, "纪要正在生成中,再点一次可取消", Toast.LENGTH_SHORT).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!LlmRuntime.ensure()) {
|
||||||
|
Toast.makeText(this, "本机 ABI 未包含摘要运行时库(仅 arm64-v8a 真机可用)",
|
||||||
|
Toast.LENGTH_LONG).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
val modelDir = File(externalModelDir, summarizeModelDirName)
|
||||||
|
val gguf = File(modelDir, summarizeGgufName)
|
||||||
|
// 存在性检查必须在任何 native 调用之前(工程教训:缺文件会让 native exit())
|
||||||
|
if (!gguf.isFile) {
|
||||||
|
Toast.makeText(
|
||||||
|
this,
|
||||||
|
"缺少摘要模型。请执行:\nadb push $summarizeGgufName " +
|
||||||
|
"/sdcard/Android/data/com.k2fsa.sherpa.onnx/files/models/$summarizeModelDirName/",
|
||||||
|
Toast.LENGTH_LONG
|
||||||
|
).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// 会话选择(2026-09-11 修订):只要存在当前会话,它就是唯一数据源
|
||||||
|
// (含刚 Stop 未久的会话——供会后整理用)。新会话还没有定格句子时
|
||||||
|
// 如实提示,绝不回退到上一场的 jsonl/notes,避免"新会议按旧笔记摘要"。
|
||||||
|
// 仅当本进程没有会话上下文(如重启 app)时,才取盘上最新一场会议。
|
||||||
|
val entries: List<TranscriptEntry>
|
||||||
|
val wallStart: Long
|
||||||
|
val baseName: String
|
||||||
|
val cur = sessionTranscript
|
||||||
|
if (cur != null) {
|
||||||
|
val snap = cur.snapshot()
|
||||||
|
if (snap.isEmpty()) {
|
||||||
|
Toast.makeText(this, "当前会议还没有定格的转写,先录一段再生成纪要",
|
||||||
|
Toast.LENGTH_SHORT).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
entries = snap; wallStart = cur.startWallMs
|
||||||
|
baseName = cur.file.nameWithoutExtension
|
||||||
|
} else {
|
||||||
|
val f = SessionTranscript.latestFile(filesDir)
|
||||||
|
if (f == null) {
|
||||||
|
Toast.makeText(this, "还没有任何会议转写,请先录一场会议",
|
||||||
|
Toast.LENGTH_SHORT).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
entries = SessionTranscript.load(f)
|
||||||
|
if (entries.isEmpty()) {
|
||||||
|
Toast.makeText(this, "最近一场会议没有定格句子", Toast.LENGTH_SHORT).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
wallStart = SessionTranscript.fileStartWallMs(f)
|
||||||
|
baseName = f.nameWithoutExtension
|
||||||
|
}
|
||||||
|
if (isRecording) {
|
||||||
|
Toast.makeText(this, "纪要生成中,录音转写不会中断", Toast.LENGTH_SHORT).show()
|
||||||
|
}
|
||||||
|
// 增量断点:上次成稿留下的 <会话>.notes.json → 只处理新增句子续写笔记
|
||||||
|
val saved = MeetingNotesIO.load(
|
||||||
|
MeetingNotesIO.pathFor(SessionTranscript.dirOf(filesDir), baseName), baseName)
|
||||||
|
var resumeIndex = 0
|
||||||
|
var resumeNotes: String? = null
|
||||||
|
if (saved != null && saved.entriesP1Done > 0 && saved.notes != SummarizePipeline.EMPTY_NOTES) {
|
||||||
|
resumeIndex = saved.entriesP1Done.coerceAtMost(entries.size)
|
||||||
|
resumeNotes = saved.notes
|
||||||
|
Log.i(TAG, "SUMMARIZE resume from notes at $resumeIndex/${entries.size}")
|
||||||
|
}
|
||||||
|
isSummarizing = true
|
||||||
|
summarizeCancelled = false
|
||||||
|
runOnUiThread { setMinutesBusyUi(true, "正在加载摘要模型(约 3.2 GB)…") }
|
||||||
|
|
||||||
|
val entriesRef = entries // 快照已在主线程拷贝,线程安全
|
||||||
|
thread(true) {
|
||||||
|
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND)
|
||||||
|
var engine: LlmEngine? = null
|
||||||
|
try {
|
||||||
|
val tLoad = System.currentTimeMillis()
|
||||||
|
engine = LlmEngine(gguf.absolutePath, SUMMARY_CTX,
|
||||||
|
if (isRecording) SUMMARY_THREADS_BUSY else SUMMARY_THREADS,
|
||||||
|
chatTemplate = summarizeChatTemplate)
|
||||||
|
Log.i(TAG, "SUMMARIZE engine loaded ms=" +
|
||||||
|
(System.currentTimeMillis() - tLoad))
|
||||||
|
summarizeEngine = engine
|
||||||
|
runOnUiThread {
|
||||||
|
setMinutesBusyUi(true, if (resumeNotes != null)
|
||||||
|
"模型就绪,从上次断点续跑($resumeIndex/${entriesRef.size} 句已消化)…"
|
||||||
|
else "模型就绪,共 ${entriesRef.size} 句,开始滚动笔记…")
|
||||||
|
}
|
||||||
|
val prompts = PromptsLoader.load(application.assets)
|
||||||
|
val glossary = File(modelDir, "glossary.txt").let {
|
||||||
|
if (it.isFile) runCatching { it.readText() }.getOrNull() ?: "" else ""
|
||||||
|
}
|
||||||
|
val pipeline = SummarizePipeline(engine, prompts)
|
||||||
|
val result = pipeline.run(
|
||||||
|
labelOf = { l -> displayName(l) }, // 纯函数,跨线程安全
|
||||||
|
entries = entriesRef,
|
||||||
|
meetingStartWallMs = wallStart,
|
||||||
|
glossary = glossary,
|
||||||
|
resumeNotes = resumeNotes,
|
||||||
|
resumeFromIndex = resumeIndex,
|
||||||
|
listener = object : SummarizeListener {
|
||||||
|
override fun onProgress(stage: String, done: Int, total: Int) {
|
||||||
|
runOnUiThread {
|
||||||
|
minutesStatus.text = "正在$stage: $done/$total"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
override fun isCancelled(): Boolean = summarizeCancelled
|
||||||
|
},
|
||||||
|
)
|
||||||
|
val mdFile = File(SessionTranscript.dirOf(filesDir), "$baseName.md")
|
||||||
|
mdFile.writeText(result.markdown)
|
||||||
|
// 增量断点落盘:本次条目全部进了笔记,下次点击只消化新增句子
|
||||||
|
MeetingNotesIO.save(
|
||||||
|
MeetingNotesIO.pathFor(SessionTranscript.dirOf(filesDir), baseName),
|
||||||
|
MeetingNotes(result.notesJson, entriesRef.size,
|
||||||
|
entriesRef.last().endMs, result.failedChunks, baseName))
|
||||||
|
Log.i(TAG, "SUMMARIZE done: ${result.totalChunks} chunks, " +
|
||||||
|
"failed=${result.failedChunks}, saved=${mdFile.absolutePath}")
|
||||||
|
runOnUiThread {
|
||||||
|
minutesText.text = result.markdown
|
||||||
|
minutesHint.visibility = View.GONE
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
minutesStatus.text = "纪要已保存(${result.language}," +
|
||||||
|
"${result.totalChunks} 块" +
|
||||||
|
(if (result.failedChunks > 0) ",${result.failedChunks} 块解析失败" else "") +
|
||||||
|
"):${mdFile.name}"
|
||||||
|
copyMinutesButton.isEnabled = true
|
||||||
|
}
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
if (t is SummarizeEmptyException) Log.i(TAG, "SUMMARIZE empty: ${t.message}")
|
||||||
|
else Log.e(TAG, "SUMMARIZE failed", t)
|
||||||
|
runOnUiThread {
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
minutesStatus.text = when (t) {
|
||||||
|
is SummarizeCancelledException -> "已取消生成(保留上一次的纪要)"
|
||||||
|
is SummarizeEmptyException ->
|
||||||
|
"已自动取消:${t.message}(上次纪要未覆盖)"
|
||||||
|
else -> "生成失败:${t.message}"
|
||||||
|
}
|
||||||
|
Toast.makeText(this, when (t) {
|
||||||
|
is SummarizeCancelledException -> "已取消生成"
|
||||||
|
is SummarizeEmptyException -> "已自动取消生成"
|
||||||
|
else -> "纪要生成失败:${t.message}"
|
||||||
|
}, Toast.LENGTH_LONG).show()
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
summarizeEngine = null
|
||||||
|
engine?.release() // 用完必须显式释放,勿依赖 finalize()
|
||||||
|
isSummarizing = false
|
||||||
|
runOnUiThread { setMinutesBusyUi(false, null) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 生成中:纪要按钮变「取消生成」+ 状态行显示 msg;结束:复原按钮。 */
|
||||||
|
private fun setMinutesBusyUi(busy: Boolean, msg: String?) {
|
||||||
|
if (busy) {
|
||||||
|
summarizeButton.setText(R.string.cancel_minutes)
|
||||||
|
if (msg != null) {
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
minutesStatus.text = msg
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
summarizeButton.setText(R.string.generate_minutes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun cancelSummarize() {
|
||||||
|
summarizeCancelled = true
|
||||||
|
summarizeEngine?.abort() // 打断在途 generate;finally 里统一收尾
|
||||||
|
minutesStatus.text = "正在取消…"
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 把当前纪要正文复制到剪贴板(方便粘到 PC/邮件)。 */
|
||||||
|
private fun onCopyMinutesClick() {
|
||||||
|
val body = minutesText.text?.toString().orEmpty()
|
||||||
|
if (body.isBlank()) {
|
||||||
|
Toast.makeText(this, "尚无纪要可复制", Toast.LENGTH_SHORT).show()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
val cm = getSystemService(CLIPBOARD_SERVICE) as android.content.ClipboardManager
|
||||||
|
cm.setPrimaryClip(android.content.ClipData.newPlainText("meeting-minutes", body))
|
||||||
|
Toast.makeText(this, "纪要全文已复制(${body.length} 字)", Toast.LENGTH_SHORT).show()
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Toast.makeText(this, "复制失败:$t", Toast.LENGTH_SHORT).show()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 冷启动把上一场最近的纪要填进下方区(.md 在盘上,读文件即可)。 */
|
||||||
|
private fun loadLastMinutes() {
|
||||||
|
try {
|
||||||
|
val latest = SessionTranscript.dirOf(filesDir)
|
||||||
|
.listFiles { f -> f.isFile && f.name.endsWith(".md") }
|
||||||
|
?.maxByOrNull { it.lastModified() } ?: return
|
||||||
|
val body = latest.readText()
|
||||||
|
if (body.isNotBlank()) {
|
||||||
|
minutesText.text = body
|
||||||
|
minutesHint.visibility = View.GONE
|
||||||
|
copyMinutesButton.isEnabled = true
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
minutesStatus.text = "上次纪要:${latest.name}(生成新纪要后会覆盖)"
|
||||||
|
}
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.w(TAG, "loadLastMinutes: $t")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ②息屏生存能力(重启提示部分):本进程没有会话上下文时,若盘上
|
||||||
|
* 最新会议的 .md 缺失或比转写旧(生成中途被杀/没点生成),状态行提示可续写。
|
||||||
|
*/
|
||||||
|
private fun hintUnfinishedMeeting() {
|
||||||
|
if (!summarizeEnabled || sessionTranscript != null) return
|
||||||
|
val f = SessionTranscript.latestFile(filesDir) ?: return
|
||||||
|
val md = File(SessionTranscript.dirOf(filesDir), f.nameWithoutExtension + ".md")
|
||||||
|
if (!md.isFile || md.lastModified() < f.lastModified()) {
|
||||||
|
minutesStatus.visibility = View.VISIBLE
|
||||||
|
minutesStatus.text = "上一场会议「${f.nameWithoutExtension}」尚未出纪要," +
|
||||||
|
"点「生成纪要」可续写(增量,只处理新增句子)"
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,63 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import android.util.Log
|
||||||
|
import org.json.JSONObject
|
||||||
|
import java.io.File
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// MeetingNotes — 会议滚动笔记的盘上断点(meetings/<会话>.notes.json)
|
||||||
|
//
|
||||||
|
// 「会中手动成稿」的增量基础:每次成稿成功后把 P1 笔记 + 已消化句子数写入;
|
||||||
|
// 下次点「生成纪要」只处理新增句子(接同一份笔记续跑),长会第 N 次点击仍是
|
||||||
|
// 分钟级。笔记文件缺失/校验不过 → 自动退回全量重跑(里程碑 1 行为)。
|
||||||
|
// 取代了已取消的"会中自动滚动"(RollingSummarizer,2026-09-11 需求变更)。
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
data class MeetingNotes(
|
||||||
|
val notes: String,
|
||||||
|
val entriesP1Done: Int,
|
||||||
|
val lastEndMs: Long,
|
||||||
|
val failedChunks: Int,
|
||||||
|
val sessionId: String,
|
||||||
|
)
|
||||||
|
|
||||||
|
object MeetingNotesIO {
|
||||||
|
private const val TAG = "sherpa-onnx"
|
||||||
|
|
||||||
|
fun pathFor(meetingDir: File, sessionBaseName: String): File =
|
||||||
|
File(meetingDir, "$sessionBaseName.notes.json")
|
||||||
|
|
||||||
|
/** sessionId 必须与文件一致(防止把别场会议的笔记接错);笔记 JSON 校验必需键。 */
|
||||||
|
fun load(file: File, sessionId: String): MeetingNotes? {
|
||||||
|
if (!file.isFile) return null
|
||||||
|
return try {
|
||||||
|
val o = JSONObject(file.readText())
|
||||||
|
if (o.optString("sessionId") != sessionId) return null
|
||||||
|
val notes = o.optString("notes")
|
||||||
|
if (SummarizePipeline.extractJson(notes) == null) return null
|
||||||
|
MeetingNotes(
|
||||||
|
notes, o.optInt("entriesP1Done"), o.optLong("lastEndMs"),
|
||||||
|
o.optInt("failedChunks"), sessionId,
|
||||||
|
)
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.w(TAG, "notes file unreadable: $t")
|
||||||
|
null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** tmp+rename 原子写:成稿线程调用(会议目录私有 IO,毫秒级)。 */
|
||||||
|
fun save(file: File, p: MeetingNotes) {
|
||||||
|
try {
|
||||||
|
val tmp = File(file.parentFile, file.name + ".tmp")
|
||||||
|
tmp.writeText(JSONObject().apply {
|
||||||
|
put("sessionId", p.sessionId); put("notes", p.notes)
|
||||||
|
put("entriesP1Done", p.entriesP1Done); put("lastEndMs", p.lastEndMs)
|
||||||
|
put("failedChunks", p.failedChunks); put("savedAt", System.currentTimeMillis())
|
||||||
|
}.toString())
|
||||||
|
if (file.exists()) file.delete()
|
||||||
|
if (!tmp.renameTo(file)) tmp.copyTo(file, overwrite = true)
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.w(TAG, "notes persist failed: $t")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,216 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import android.os.SystemClock
|
||||||
|
import android.util.Log
|
||||||
|
import org.json.JSONObject
|
||||||
|
import java.io.BufferedWriter
|
||||||
|
import java.io.File
|
||||||
|
import java.text.SimpleDateFormat
|
||||||
|
import java.util.Date
|
||||||
|
import java.util.Locale
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// SessionTranscript — 全量会议转写记录器(摘要功能·里程碑 1,§4.2)
|
||||||
|
//
|
||||||
|
// 独立于字幕 UI 的追加式事件流:字幕层受 300 行上限并会被迁移/合并/改名,
|
||||||
|
// 这里以 sid 为主键记录每条定格句的 (startMs, endMs, speaker, text, refined),
|
||||||
|
// 落盘 JSONL(filesDir/meetings/<yyyyMMdd-HHmm>.jsonl),回放事件重建全文。
|
||||||
|
//
|
||||||
|
// 线程规矩:所有 on* 方法只在主线程被调用(finalizeSentence / confirmSpeaker /
|
||||||
|
// applyRefined / relabelSpeakerLines 本身就在主线程);JSONL 逐行 append 是
|
||||||
|
// 毫秒级 IO,每句几秒一次的频率不会卡 UI。snapshot()/load() 在摘要后台线程
|
||||||
|
// 读文件,与写入的逐行 flush 兼容(读侧容忍最后一行残缺)。
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
class TranscriptEntry(
|
||||||
|
val sid: Long,
|
||||||
|
val startMs: Long, // 相对会话开始(ms)
|
||||||
|
val endMs: Long, // 相对会话开始(ms)
|
||||||
|
var speaker: String?,// 内部标签 S{n}/真名;null=未识别
|
||||||
|
var text: String,
|
||||||
|
var refined: Boolean,
|
||||||
|
)
|
||||||
|
|
||||||
|
class SessionTranscript private constructor(
|
||||||
|
val file: File,
|
||||||
|
val sessionStartElapsed: Long,
|
||||||
|
) {
|
||||||
|
val startWallMs: Long = System.currentTimeMillis()
|
||||||
|
private val entries = ArrayList<TranscriptEntry>()
|
||||||
|
private val bySid = HashMap<Long, TranscriptEntry>()
|
||||||
|
private var writer: BufferedWriter? = null
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "sherpa-onnx"
|
||||||
|
const val DIR_NAME = "meetings"
|
||||||
|
/** 本地最多保留的会议场数(超出自动清理 jsonl/notes/md) */
|
||||||
|
const val KEEP_SESSIONS = 50
|
||||||
|
|
||||||
|
private fun tsFmt() = SimpleDateFormat("yyyyMMdd-HHmm", Locale.US)
|
||||||
|
|
||||||
|
fun dirOf(filesDir: File): File = File(filesDir, DIR_NAME).apply { mkdirs() }
|
||||||
|
|
||||||
|
fun latestFile(filesDir: File): File? =
|
||||||
|
dirOf(filesDir).listFiles { f -> f.name.endsWith(".jsonl") }
|
||||||
|
?.maxByOrNull { it.lastModified() }
|
||||||
|
|
||||||
|
/** 从文件第一行 H 事件读会议真实开始时间;缺失时退回文件 mtime。 */
|
||||||
|
fun fileStartWallMs(file: File): Long {
|
||||||
|
return try {
|
||||||
|
file.bufferedReader().use { r ->
|
||||||
|
val line = r.readLine() ?: return@use file.lastModified()
|
||||||
|
val o = JSONObject(line)
|
||||||
|
if (o.optString("t") == "H") o.optLong("wall", file.lastModified())
|
||||||
|
else file.lastModified()
|
||||||
|
}
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
file.lastModified()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 录音开始时在主线程调用,新建会话文件。 */
|
||||||
|
fun begin(filesDir: File): SessionTranscript {
|
||||||
|
val dir = dirOf(filesDir)
|
||||||
|
pruneOldSessions(dir, KEEP_SESSIONS)
|
||||||
|
val name = tsFmt().format(Date()) + ".jsonl"
|
||||||
|
var f = File(dir, name)
|
||||||
|
var i = 1
|
||||||
|
while (f.exists()) { f = File(dir, name.removeSuffix(".jsonl") + "-$i.jsonl"); i++ }
|
||||||
|
val t = SessionTranscript(f, SystemClock.elapsedRealtime())
|
||||||
|
t.openWriter()
|
||||||
|
t.appendLine(JSONObject().apply {
|
||||||
|
put("t", "H"); put("wall", System.currentTimeMillis())
|
||||||
|
})
|
||||||
|
return t
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* ③ 保留策略:按会话名分组(jsonl/notes/md 同前缀归一组),
|
||||||
|
* 按组内最新修改时间排序,只留最近 KEEP_SESSIONS 场。
|
||||||
|
* 删除失败静默(文件可能被并发读),不影响新会话创建。
|
||||||
|
*/
|
||||||
|
internal fun pruneOldSessions(dir: File, keep: Int = KEEP_SESSIONS) {
|
||||||
|
runCatching {
|
||||||
|
dir.listFiles { f -> f.isFile }
|
||||||
|
?.groupBy { it.name.substringBeforeLast('.') }
|
||||||
|
?.entries
|
||||||
|
?.sortedByDescending { (_, files) -> files.maxOf { it.lastModified() } }
|
||||||
|
?.drop(keep)
|
||||||
|
?.forEach { (_, files) -> files.forEach { runCatching { it.delete() } } }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 从 JSONL 回放(摘要后台线程用)。容忍最后一行残缺。 */
|
||||||
|
fun load(file: File): List<TranscriptEntry> {
|
||||||
|
val out = ArrayList<TranscriptEntry>()
|
||||||
|
val idx = HashMap<Long, TranscriptEntry>()
|
||||||
|
var renames = ArrayList<JSONObject>()
|
||||||
|
file.forEachLine { line ->
|
||||||
|
val o = try { JSONObject(line) } catch (t: Throwable) { return@forEachLine }
|
||||||
|
when (o.optString("t")) {
|
||||||
|
"S" -> {
|
||||||
|
val e = TranscriptEntry(
|
||||||
|
sid = o.optLong("sid"),
|
||||||
|
startMs = o.optLong("b"), endMs = o.optLong("e"),
|
||||||
|
speaker = o.optString("sp").takeIf { it.isNotEmpty() && it != "null" },
|
||||||
|
text = o.optString("x"), refined = o.optInt("r") == 1,
|
||||||
|
)
|
||||||
|
idx[e.sid] = e; out.add(e)
|
||||||
|
}
|
||||||
|
"R" -> idx[o.optLong("sid")]?.let {
|
||||||
|
it.text = o.optString("x", it.text); it.refined = o.optInt("r") == 1
|
||||||
|
}
|
||||||
|
"P" -> idx[o.optLong("sid")]?.let {
|
||||||
|
val sp = o.optString("sp").takeIf { s -> s.isNotEmpty() && s != "null" }
|
||||||
|
if (sp != null) it.speaker = sp
|
||||||
|
}
|
||||||
|
"N" -> renames.add(o)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 改名事件最后统一回放(顺序与录制时一致)
|
||||||
|
for (rn in renames) {
|
||||||
|
val from = rn.optString("from"); val to = rn.optString("to")
|
||||||
|
for (e in out) if (e.speaker == from) e.speaker = to
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 主线程挂钩点 --------------------------------------------------------
|
||||||
|
|
||||||
|
/** finalizeSentence 处调用:endMs=定格时刻(相对会话开始),startMs=endMs-dur。 */
|
||||||
|
fun onSettled(sid: Long, durMs: Long, text: String) {
|
||||||
|
if (text.isBlank()) return
|
||||||
|
val end = SystemClock.elapsedRealtime() - sessionStartElapsed
|
||||||
|
val e = TranscriptEntry(sid, (end - durMs).coerceAtLeast(0), end, null, text, false)
|
||||||
|
bySid[sid] = e; entries.add(e)
|
||||||
|
appendLine(JSONObject().apply {
|
||||||
|
put("t", "S"); put("sid", sid); put("b", e.startMs); put("e", end)
|
||||||
|
put("x", text)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/** applyRefined 处调用(每槽恰好一次,与字幕层幂等规则一致)。 */
|
||||||
|
fun onRefined(sid: Long, text: String, refined: Boolean) {
|
||||||
|
val e = bySid[sid] ?: return
|
||||||
|
if (text.isBlank()) return
|
||||||
|
e.text = text; e.refined = refined
|
||||||
|
appendLine(JSONObject().apply {
|
||||||
|
put("t", "R"); put("sid", sid); put("x", text); put("r", if (refined) 1 else 0)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/** confirmSpeaker 处调用(含声纹线程回传后的所有修正路径)。 */
|
||||||
|
fun onSpeaker(sid: Long, label: String?) {
|
||||||
|
val e = bySid[sid] ?: return
|
||||||
|
if (label != null && e.speaker != label) {
|
||||||
|
e.speaker = label
|
||||||
|
appendLine(JSONObject().apply {
|
||||||
|
put("t", "P"); put("sid", sid); put("sp", label)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** renameSpeaker(点击标签命名/改名)处调用:S{n}→真名。 */
|
||||||
|
fun onRename(from: String, to: String) {
|
||||||
|
var touched = false
|
||||||
|
for (e in entries) if (e.speaker == from) { e.speaker = to; touched = true }
|
||||||
|
if (touched) appendLine(JSONObject().apply {
|
||||||
|
put("t", "N"); put("from", from); put("to", to)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 读取 / 收尾 ----------------------------------------------------------
|
||||||
|
|
||||||
|
/** 主线程取快照(拷贝,供摘要后台线程安全读)。 */
|
||||||
|
fun snapshot(): List<TranscriptEntry> = synchronized(entries) {
|
||||||
|
entries.map {
|
||||||
|
TranscriptEntry(it.sid, it.startMs, it.endMs, it.speaker, it.text, it.refined)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun flush() {
|
||||||
|
try { writer?.flush() } catch (t: Throwable) { Log.w(TAG, "transcript flush: $t") }
|
||||||
|
}
|
||||||
|
|
||||||
|
fun close() {
|
||||||
|
flush()
|
||||||
|
try { writer?.close() } catch (_: Throwable) {}
|
||||||
|
writer = null
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 内部 -----------------------------------------------------------------
|
||||||
|
|
||||||
|
private fun openWriter() {
|
||||||
|
try { writer = file.bufferedWriter() } catch (t: Throwable) {
|
||||||
|
Log.e(TAG, "SessionTranscript cannot open $file", t)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun appendLine(o: JSONObject) {
|
||||||
|
val w = writer ?: return
|
||||||
|
try {
|
||||||
|
w.write(o.toString()); w.write("\n"); w.flush()
|
||||||
|
} catch (t: Throwable) { Log.w(TAG, "transcript append: $t") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -123,6 +123,49 @@ class SpeakerRegistry {
|
||||||
return best
|
return best
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 最近邻兜底(用户要求 2026-09-11):"未识别"归到与 e 最相似的已注册者
|
||||||
|
* (命名条目优先,其后自动条目;相似度须 ≥ minScore)。全空/不达标返回 null。
|
||||||
|
*/
|
||||||
|
@Synchronized
|
||||||
|
fun nearest(e: FloatArray, minScore: Float): String? {
|
||||||
|
val q = l2normalize(e)
|
||||||
|
var bestName: String? = null
|
||||||
|
var best = -1f
|
||||||
|
for ((name, emb) in named) {
|
||||||
|
val s = dot(q, emb)
|
||||||
|
if (s > best) { best = s; bestName = name }
|
||||||
|
}
|
||||||
|
for ((name, emb) in auto) {
|
||||||
|
val s = dot(q, emb)
|
||||||
|
if (s > best) { best = s; bestName = name }
|
||||||
|
}
|
||||||
|
return if (best >= minScore) bestName else null
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 点击标签命名用:把内部键 S{n}(自动注册、未命名)升级为真名,
|
||||||
|
* 或把已有真名改成另一个真名。同名允许多条嵌入并存 —— 之前误判成
|
||||||
|
* 两个说话人、后被命名为同一名字的两组声纹,自此 search 都返回同一名。
|
||||||
|
*/
|
||||||
|
@Synchronized
|
||||||
|
fun renameToName(oldKey: String, name: String): Boolean {
|
||||||
|
val ai = auto.indexOfFirst { it.first == oldKey }
|
||||||
|
if (ai >= 0) {
|
||||||
|
named.add(Pair(name, auto[ai].second))
|
||||||
|
auto.removeAt(ai)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
var changed = false
|
||||||
|
for (i in named.indices) {
|
||||||
|
if (named[i].first == oldKey) {
|
||||||
|
named[i] = Pair(name, named[i].second)
|
||||||
|
changed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return changed
|
||||||
|
}
|
||||||
|
|
||||||
private fun match(
|
private fun match(
|
||||||
list: List<Pair<String, FloatArray>>,
|
list: List<Pair<String, FloatArray>>,
|
||||||
e: FloatArray,
|
e: FloatArray,
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,16 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// SubtitleFormat — 字幕行内句间衔接的纯函数(从 MainActivity 提出以便 JVM 单测)
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
/** 句间衔接:已带标点则空格;CJK/假名结尾用句号;拉丁结尾用英文句号。 */
|
||||||
|
internal fun joinerAfter(prev: String): String {
|
||||||
|
val last = prev.trimEnd().lastOrNull() ?: return " "
|
||||||
|
return when {
|
||||||
|
last in "。！？；.!?;…\"”’』】》）)" -> " "
|
||||||
|
last.code >= 0x2E80 -> "。"
|
||||||
|
Character.isLetterOrDigit(last) -> ". "
|
||||||
|
else -> " "
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,443 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import android.content.res.AssetManager
|
||||||
|
import android.util.Log
|
||||||
|
import org.json.JSONObject
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// SummarizePipeline — §5 摘要流水线(分块 → 滚动笔记 → 成稿)
|
||||||
|
//
|
||||||
|
// 提示词唯一事实源是 assets/summarize/prompts.md(用户可直接编辑/复制到 PC
|
||||||
|
// llama.cpp 调试)。解析失败时退回本文件内置的同源副本 —— 两份不一致以 md
|
||||||
|
// 为准,改提示词请同步改 md。
|
||||||
|
//
|
||||||
|
// 只在摘要专用后台线程上 new 出来跑完就丢,绝不触碰录音/主线程对象。
|
||||||
|
// ============================================================================
|
||||||
|
|
||||||
|
class SummarizeCancelledException : Exception("summarize cancelled")
|
||||||
|
|
||||||
|
/** 空笔记防线(用户要求 2026-09-11):笔记/成稿为空 → 提示并自动取消,不落盘。 */
|
||||||
|
class SummarizeEmptyException(msg: String) : Exception(msg)
|
||||||
|
|
||||||
|
data class Prompts(
|
||||||
|
val p1System: String, val p1UserTpl: String,
|
||||||
|
val p2System: String, val p2UserTpl: String,
|
||||||
|
)
|
||||||
|
|
||||||
|
data class MeetingSummary(
|
||||||
|
val markdown: String,
|
||||||
|
val language: String, // ja / zh / en
|
||||||
|
val notesJson: String,
|
||||||
|
val totalChunks: Int,
|
||||||
|
val failedChunks: Int,
|
||||||
|
)
|
||||||
|
|
||||||
|
interface SummarizeListener {
|
||||||
|
/** stage 人话描述("正在加载模型…"、"第 3/12 块…")。在摘要线程回调。 */
|
||||||
|
fun onProgress(stage: String, done: Int, total: Int)
|
||||||
|
fun isCancelled(): Boolean
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- prompts.md 解析 --------------------------------------------------------
|
||||||
|
|
||||||
|
object PromptsLoader {
|
||||||
|
private const val TAG = "sherpa-onnx"
|
||||||
|
private const val ASSET = "summarize/prompts.md"
|
||||||
|
|
||||||
|
fun load(assets: AssetManager): Prompts {
|
||||||
|
val md = try {
|
||||||
|
assets.open(ASSET).bufferedReader().use { it.readText() }
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.w(TAG, "prompts.md missing/unreadable, using builtin: $t")
|
||||||
|
return BUILTIN
|
||||||
|
}
|
||||||
|
return try {
|
||||||
|
parse(md)
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
Log.e(TAG, "prompts.md parse failed, using builtin", t)
|
||||||
|
BUILTIN
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// "## P1 …" 小节里前两个代码围栏块依次是 SYSTEM、USER;"## P2" 同理。
|
||||||
|
internal fun parse(md: String): Prompts {
|
||||||
|
fun section(marker: String): Pair<String, String> {
|
||||||
|
val start = md.indexOf(marker)
|
||||||
|
require(start >= 0) { "section $marker not found" }
|
||||||
|
val end = md.indexOf("\n## ", start + marker.length).let {
|
||||||
|
if (it < 0) md.length else it
|
||||||
|
}
|
||||||
|
val body = md.substring(start, end)
|
||||||
|
val fences = ArrayList<String>()
|
||||||
|
var i = 0
|
||||||
|
while (true) {
|
||||||
|
val open = body.indexOf("```", i)
|
||||||
|
if (open < 0) break
|
||||||
|
val close = body.indexOf("```", open + 3)
|
||||||
|
if (close < 0) break
|
||||||
|
fences.add(body.substring(open + 3, close).trim('\n').trim())
|
||||||
|
i = close + 3
|
||||||
|
}
|
||||||
|
require(fences.size >= 2) { "$marker: need 2 fenced blocks, got ${fences.size}" }
|
||||||
|
return fences[0] to fences[1]
|
||||||
|
}
|
||||||
|
val (p1s, p1u) = section("## P1")
|
||||||
|
val (p2s, p2u) = section("## P2")
|
||||||
|
require(p1s.isNotBlank() && p2s.isNotBlank())
|
||||||
|
return Prompts(p1s, p1u, p2s, p2u)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 与 prompts.md 同源的内置副本(md 丢失/被改坏时的兜底)。
|
||||||
|
val BUILTIN = Prompts(
|
||||||
|
p1System = "あなたは会議の文字起こしを処理する記録係です。与えられた「現在のメモ」に「新しい断片」の内容を統合し、更新後のメモをJSONだけで出力してください。\n\n" +
|
||||||
|
"厳守事項:\n" +
|
||||||
|
"- 出力は指定スキーマのJSONオブジェクトのみ。説明文・コードフェンス・前置き禁止。\n" +
|
||||||
|
"- メモは日本語で書く。ただし人名・組織名・製品名・固有名詞は原文表記をそのまま維持。\n" +
|
||||||
|
"- 数字、金額、日時、期限、回数、単位は原文どおり正確に転記。ざっくり要約しない。\n" +
|
||||||
|
"- 否定・保留・反対の意図を消さない。「見送り」「再検討」「反対意見」はそれ自体が重要な情報。\n" +
|
||||||
|
"- 各項目に根拠となる発話のタイムレンジを「出典」に付ける(形式: \"mm:ss-mm:ss\"、複数可 \";\" 区切り)。\n" +
|
||||||
|
"- 断片に新情報がない場合はメモをほぼ変更しない(前回末尾は必ず更新)。\n" +
|
||||||
|
"- 全体で500字を超えそうな場合、重要度の低い過去の項目を統合・削除して圧縮する。削除時は行動項目と已定事項を最後に守る。\n" +
|
||||||
|
"- 「前回末尾」には、次の断片と文脈が繋がるための直近話題を1文で書く。\n" +
|
||||||
|
"- JSONの値の中にダブルクォートが必要なら \\\" にエスケープ。改行は \\n を使う。\n\n" +
|
||||||
|
"スキーマ(キー名は変更禁止):\n" +
|
||||||
|
"{\"已定事項\":[{\"内容\":\"\",\"出典\":\"\"}],\"討論中\":[{\"内容\":\"\",\"出典\":\"\"}],\"行動項目\":[{\"内容\":\"\",\"担当者\":\"\",\"期限\":\"\",\"出典\":\"\"}],\"人物\":[{\"名前\":\"\",\"立場\":\"\"}],\"前回末尾\":\"\"}",
|
||||||
|
p1UserTpl = "【現在のメモ】\n{notes_json}\n\n" +
|
||||||
|
"【新しい断片】(タイムスタンプは会議開始からの分:秒、[]内は話者ラベル)\n{chunk_text}\n\n" +
|
||||||
|
"更新後のメモJSONを出力してください。",
|
||||||
|
p2System = "あなたは会議の公式議事録作成者です。会議メモ(JSON)と用語集を受け取ったら、指定言語で完成した議事録をMarkdownで出力してください。\n\n" +
|
||||||
|
"厳守事項:\n" +
|
||||||
|
"- 出力言語は指定された「成稿言語」を厳守。日本語メモの内容をそのまま指定言語の書き言葉文体で記述する。\n" +
|
||||||
|
"- 本文は成稿言語の文字のみで書く:中文稿なら仮名を一切出さない、English稿なら漢字・仮名を一切出さない、日本語稿なら製品名以外に英語の文を残さない。\n" +
|
||||||
|
"- ただし人物・会社・ブランドなどの固有名詞は、メモ・用語集にある原文表記のまま保留してよい(上記の文字制限の例外)。\n" +
|
||||||
|
"- 見出しは次の4つを必ずこの順で:「議題と経過」「決定事項」「行動項目」「未決事項・持ち帰り」。該当ゼロなら「(なし)」と明記。\n" +
|
||||||
|
"- 行動項目は表形式(担当者 / 内容 / 期限 / 出典)。担当者が不明なら「未定」と書く。\n" +
|
||||||
|
"- 各項目の末尾に出典を付ける。形式: `出典: mm:ss-mm:ss`。出典はメモJSONの「出典」フィールドのみを使う。捏造禁止。\n" +
|
||||||
|
"- メモにない事実・推測・補足説明を追加しない。\n" +
|
||||||
|
"- 人名・製品名は用語集の正式表記に統一する(用語集にない表記はメモのまま採用)。\n" +
|
||||||
|
"- 冒頭にメタ情報を置く: 会議日時(指定があれば)、話者数、使用言語、文字起こし総尺。\n" +
|
||||||
|
"- Markdown本文のみ出力。コードフェンスで全体を囲まない。",
|
||||||
|
p2UserTpl = "【成稿言語】{language_name}\n\n" +
|
||||||
|
"【会議メタ情報】開始: {meeting_start} / 話者数: {n_speakers} / 総尺: {total_duration}\n\n" +
|
||||||
|
"【用語集(正式表記の一覧。人名・製品名の誤変換をこれに統一)】\n{glossary_lines}\n\n" +
|
||||||
|
"【会議メモJSON】\n{notes_json}\n\n" +
|
||||||
|
"上記に基づき、議事録Markdownを出力してください。",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 流水线主体 --------------------------------------------------------------
|
||||||
|
|
||||||
|
class SummarizePipeline(
|
||||||
|
private val engine: LlmEngine,
|
||||||
|
private val prompts: Prompts,
|
||||||
|
) {
|
||||||
|
companion object {
|
||||||
|
private const val TAG = "sherpa-onnx"
|
||||||
|
private const val CHUNK_TARGET = 2500 // §5:2500±500 字/块
|
||||||
|
private const val CHUNK_MIN = 2000
|
||||||
|
private const val CHUNK_MAX = 3000
|
||||||
|
// P1 输入(笔记+块+固定文案)字符预算:超了就对半拆块,
|
||||||
|
// 保证 ≤8192 ctx 永远够用(日语 1 字 ≈ 1 token 多一点)
|
||||||
|
private const val P1_MAX_PROMPT_CHARS = 6500
|
||||||
|
private const val NOTES_RESERVE = 1200 // 笔记+模板文案预留
|
||||||
|
val EMPTY_NOTES =
|
||||||
|
"{\"已定事項\":[],\"討論中\":[],\"行動項目\":[],\"人物\":[],\"前回末尾\":\"\"}"
|
||||||
|
|
||||||
|
// 脚本占比判语种:假名出现即日语信号(日语里汉字占比也高,不能只看汉字);
|
||||||
|
// 无假名时看汉字 vs 拉丁;全兜底 en。
|
||||||
|
fun detectLanguage(text: String): String {
|
||||||
|
var kana = 0
|
||||||
|
var han = 0
|
||||||
|
var latin = 0
|
||||||
|
for (ch in text) {
|
||||||
|
val cp = ch.code
|
||||||
|
when {
|
||||||
|
cp in 0x3040..0x30FF || cp in 0x31F0..0x31FF || cp in 0xFF66..0xFF9F -> kana++
|
||||||
|
cp in 0x3400..0x4DBF || cp in 0x4E00..0x9FFF || cp in 0xF900..0xFAFF -> han++
|
||||||
|
cp in 0x41..0x5A || cp in 0x61..0x7A -> latin++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return when {
|
||||||
|
kana > 0 && kana * 20 >= kana + han -> "ja" // 假名/汉字 ≥ 5%
|
||||||
|
han >= latin && han > 0 -> "zh"
|
||||||
|
else -> "en"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun languageName(lang: String) = when (lang) {
|
||||||
|
"ja" -> "日本語"
|
||||||
|
"zh" -> "中文(简体书面语)"
|
||||||
|
else -> "English (formal written English)"
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 笔记是否无任何实质条目(四类条目全空;"前回末尾"不算)。解析失败按空处理。 */
|
||||||
|
internal fun notesEntriesEmpty(notes: String): Boolean = runCatching {
|
||||||
|
val o = JSONObject(notes)
|
||||||
|
listOf("已定事項", "討論中", "行動項目", "人物").all { k ->
|
||||||
|
o.optJSONArray(k)?.length() == 0
|
||||||
|
}
|
||||||
|
}.getOrDefault(true)
|
||||||
|
|
||||||
|
fun mmss(ms: Long): String {
|
||||||
|
val t = ms / 1000
|
||||||
|
return "%02d:%02d".format(t / 60, t % 60)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun formatDate(wallMs: Long): String =
|
||||||
|
java.text.SimpleDateFormat("yyyy-MM-dd HH:mm", java.util.Locale.US)
|
||||||
|
.format(java.util.Date(wallMs))
|
||||||
|
|
||||||
|
/** P2 成稿 user 提示词构造(滚动成稿与里程碑 1 共用,保证两路输出同构)。 */
|
||||||
|
fun buildDraftUserPrompt(
|
||||||
|
prompts: Prompts, lang: String, meetingStartWallMs: Long,
|
||||||
|
nSpeakers: Int, totalDurMs: Long, glossary: String, notes: String,
|
||||||
|
): String = prompts.p2UserTpl
|
||||||
|
.replace("{language_name}", languageName(lang))
|
||||||
|
.replace("{meeting_start}", formatDate(meetingStartWallMs))
|
||||||
|
.replace("{n_speakers}", nSpeakers.toString())
|
||||||
|
.replace("{total_duration}", mmss(totalDurMs))
|
||||||
|
.replace("{glossary_lines}", glossary.ifBlank { "(なし)" })
|
||||||
|
.replace("{notes_json}", notes)
|
||||||
|
|
||||||
|
// §5 分块铁律:句子边界聚合到 2500±500 字;块间重叠 1 句;
|
||||||
|
// 同一说话人连续段(轮次)不拆开:切点落在同人连续段中间则延到段尾。
|
||||||
|
// 另:超 P1 预算的块按行二分一次,保证 prompt ≤ ctx。
|
||||||
|
internal fun chunk(lines: List<String>): List<String> {
|
||||||
|
val blocks = ArrayList<String>()
|
||||||
|
var i = 0
|
||||||
|
var overlap: String? = null
|
||||||
|
while (i < lines.size) {
|
||||||
|
var j = i
|
||||||
|
var len = 0
|
||||||
|
while (j < lines.size) {
|
||||||
|
val l = lines[j]
|
||||||
|
if (len > 0 && len + l.length > CHUNK_MAX) break
|
||||||
|
if (len >= CHUNK_TARGET) {
|
||||||
|
// 与上一句同一说话人 → 轮次未完,继续收
|
||||||
|
if (j == i || speakerOf(l) != speakerOf(lines[j - 1])) {
|
||||||
|
if (len >= CHUNK_MIN) break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
len += l.length + 1
|
||||||
|
j++
|
||||||
|
}
|
||||||
|
if (j == i) j++ // 单句超长的极端情况:至少吞一句
|
||||||
|
blocks.add(
|
||||||
|
StringBuilder().apply {
|
||||||
|
overlap?.let { append(it).append('\n') }
|
||||||
|
for (k in i until j) append(lines[k]).append('\n')
|
||||||
|
}.toString().trimEnd()
|
||||||
|
)
|
||||||
|
overlap = if (j < lines.size) lines[j - 1] else null
|
||||||
|
i = j
|
||||||
|
}
|
||||||
|
|
||||||
|
val fine = ArrayList<String>()
|
||||||
|
for (c in blocks) {
|
||||||
|
if (c.length + NOTES_RESERVE <= P1_MAX_PROMPT_CHARS) {
|
||||||
|
fine.add(c)
|
||||||
|
} else {
|
||||||
|
val ls = c.split('\n').filter { it.isNotBlank() }
|
||||||
|
val half = (ls.size / 2).coerceAtLeast(1)
|
||||||
|
val a = ls.subList(0, half).joinToString("\n")
|
||||||
|
val b = ls.subList(half, ls.size).joinToString("\n")
|
||||||
|
if (a.isNotBlank()) fine.add(a)
|
||||||
|
if (b.isNotBlank()) fine.add(b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return fine
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun speakerOf(line: String): String {
|
||||||
|
val a = line.indexOf('[')
|
||||||
|
val b = line.indexOf(']', a + 1)
|
||||||
|
return if (a >= 0 && b > a) line.substring(a, b) else line
|
||||||
|
}
|
||||||
|
|
||||||
|
// 笔记输出健壮性:取首个平衡 {...}(前后的客套话/代码围栏自然被忽略)、
|
||||||
|
// 必需键检查
|
||||||
|
internal fun extractJson(raw: String): String? {
|
||||||
|
val start = raw.indexOf('{')
|
||||||
|
if (start < 0) return null
|
||||||
|
var depth = 0
|
||||||
|
var inStr = false
|
||||||
|
var esc = false
|
||||||
|
for (i in start until raw.length) {
|
||||||
|
val c = raw[i]
|
||||||
|
if (esc) { esc = false; continue }
|
||||||
|
when {
|
||||||
|
c == '\\' && inStr -> esc = true
|
||||||
|
c == '"' -> inStr = !inStr
|
||||||
|
!inStr && c == '{' -> depth++
|
||||||
|
!inStr && c == '}' -> {
|
||||||
|
depth--
|
||||||
|
if (depth == 0) {
|
||||||
|
val cand = raw.substring(start, i + 1)
|
||||||
|
return try {
|
||||||
|
val o = JSONObject(cand)
|
||||||
|
if (o.has("已定事項") && o.has("行動項目")) cand else null
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
internal fun cleanDraft(md: String): String {
|
||||||
|
var s = md.trim()
|
||||||
|
if (s.startsWith("```")) {
|
||||||
|
s = s.substringAfter('\n').substringBeforeLast("```")
|
||||||
|
}
|
||||||
|
s = s.replace(
|
||||||
|
Regex("<\\|channel>thought.*?<channel\\|>", RegexOption.DOT_MATCHES_ALL), ""
|
||||||
|
)
|
||||||
|
s = s.replace(Regex("\n{3,}"), "\n\n")
|
||||||
|
return s.trim()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 输出语种纯度统计(2026-09-11 起只用于对账日志,不再触发整篇重写):
|
||||||
|
* zh → 假名 > 8 个判违规;en → 汉字+假名 > 6 个判违规;ja → 不检查。
|
||||||
|
* 返回违规样本描述,null = 通过。
|
||||||
|
*/
|
||||||
|
internal fun scriptViolation(lang: String, text: String): String? {
|
||||||
|
var kana = 0
|
||||||
|
var han = 0
|
||||||
|
var firstBad = -1
|
||||||
|
for (i in text.indices) {
|
||||||
|
val cp = text[i].code
|
||||||
|
when {
|
||||||
|
cp in 0x3040..0x30FF || cp in 0xFF66..0xFF9F -> { kana++; if (firstBad < 0) firstBad = i }
|
||||||
|
cp in 0x3400..0x4DBF || cp in 0x4E00..0x9FFF || cp in 0xF900..0xFAFF -> {
|
||||||
|
han++
|
||||||
|
if (firstBad < 0) firstBad = i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fun sample(): String = text.substring(
|
||||||
|
firstBad, (firstBad + 16).coerceAtMost(text.length)
|
||||||
|
).replace("\n", " ")
|
||||||
|
return when (lang) {
|
||||||
|
"zh" -> if (kana > 8) "假名 $kana 个(如「${sample()}」)" else null
|
||||||
|
"en" -> if (kana + han > 6) "中日文字 $han+$kana 个(around \"${sample()}\")" else null
|
||||||
|
else -> null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param labelOf 内部说话人键(S{n}/真名)→ 显示名(已识别人N/真名/未识别)
|
||||||
|
* @param entries SessionTranscript 快照(定格顺序,不可为空)
|
||||||
|
* @param resumeNotes/resumeFromIndex M2:会中滚动笔记已覆盖前 resumeFromIndex 句,
|
||||||
|
* 仅补跑尾部(进度、成稿复用同一份笔记)
|
||||||
|
*/
|
||||||
|
fun run(
|
||||||
|
labelOf: (String?) -> String,
|
||||||
|
entries: List<TranscriptEntry>,
|
||||||
|
meetingStartWallMs: Long,
|
||||||
|
glossary: String,
|
||||||
|
listener: SummarizeListener,
|
||||||
|
resumeNotes: String? = null,
|
||||||
|
resumeFromIndex: Int = 0,
|
||||||
|
): MeetingSummary {
|
||||||
|
if (entries.isEmpty()) throw IllegalArgumentException("会议转写为空")
|
||||||
|
val lang = detectLanguage(entries.joinToString("") { it.text })
|
||||||
|
|
||||||
|
// ---- 分块(从断点句开始;断点前借 1 句做上下文衔接,同块间重叠规则)----
|
||||||
|
val startIdx = resumeFromIndex.coerceIn(0, entries.size)
|
||||||
|
fun renderLine(e: TranscriptEntry) = "(%s-%s)[%s] %s".format(
|
||||||
|
mmss(e.startMs), mmss(e.endMs), labelOf(e.speaker), e.text.trim()
|
||||||
|
)
|
||||||
|
val lines = ArrayList<String>()
|
||||||
|
if (startIdx > 0 && startIdx < entries.size) lines.add(renderLine(entries[startIdx - 1]))
|
||||||
|
for (e in entries.drop(startIdx)) lines.add(renderLine(e))
|
||||||
|
val chunks = chunk(lines)
|
||||||
|
val total = chunks.size
|
||||||
|
// 用户要求(2026-09-11):没有新句可消化(再点生成纪要,断点已覆盖全部
|
||||||
|
// 转写)时不再空跑"成稿 0/0",直接提示并自动取消
|
||||||
|
if (total == 0) {
|
||||||
|
throw SummarizeEmptyException(if (resumeNotes != null)
|
||||||
|
"本场会议没有新增转写,纪要已是最新"
|
||||||
|
else "没有可分析的转写内容")
|
||||||
|
}
|
||||||
|
Log.i(TAG, "SUMMARIZE: ${entries.size} utterances -> $total chunks, " +
|
||||||
|
"lang=$lang, resumeIdx=$startIdx")
|
||||||
|
|
||||||
|
// ---- 滚动笔记(§5.2)----
|
||||||
|
var notes = resumeNotes ?: EMPTY_NOTES
|
||||||
|
var failed = 0
|
||||||
|
val tP1Start = System.currentTimeMillis()
|
||||||
|
chunks.forEachIndexed { idx, chunkText ->
|
||||||
|
if (listener.isCancelled()) throw SummarizeCancelledException()
|
||||||
|
val user = prompts.p1UserTpl
|
||||||
|
.replace("{notes_json}", notes)
|
||||||
|
.replace("{chunk_text}", chunkText)
|
||||||
|
listener.onProgress("滚动笔记", idx + 1, total)
|
||||||
|
val t0 = System.currentTimeMillis()
|
||||||
|
var out = engine.generate(
|
||||||
|
systemPrompt = prompts.p1System, userPrompt = user,
|
||||||
|
maxNewTokens = 900, temperature = 0f,
|
||||||
|
)
|
||||||
|
var parsed = extractJson(out)
|
||||||
|
if (parsed == null) {
|
||||||
|
// 一次修复重试:把畸形输出递回去(仍失败则保留旧笔记,计 failed)
|
||||||
|
if (listener.isCancelled()) throw SummarizeCancelledException()
|
||||||
|
val fix = user + "\n\n(前回の出力はJSONとして不正でした: " +
|
||||||
|
out.take(200).replace("\n", " ") + " 正しいJSONのみを再出力してください。)"
|
||||||
|
out = engine.generate(
|
||||||
|
systemPrompt = prompts.p1System, userPrompt = fix,
|
||||||
|
maxNewTokens = 900, temperature = 0f,
|
||||||
|
)
|
||||||
|
parsed = extractJson(out)
|
||||||
|
}
|
||||||
|
if (parsed != null) notes = parsed
|
||||||
|
else {
|
||||||
|
failed++
|
||||||
|
// 及时止损:第一块都解析不出来,笔记链条已断,后面块只会白烧 token
|
||||||
|
if (idx == 0) {
|
||||||
|
Log.w(TAG, "P1 first chunk unparsable, abort. raw=${out.take(120)}")
|
||||||
|
throw SummarizeEmptyException("开头转写片段无法生成有效笔记(内容过少或输出不可解析)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// 测量点(baseline,不改行为):raw≫json 说明解码浪费在废话/思考上;
|
||||||
|
// ms 是单块真实耗时,决定 diff 化/大块合并的参数
|
||||||
|
Log.i(TAG, "P1 chunk=${idx + 1}/$total prompt_len=${user.length} " +
|
||||||
|
"raw=${out.length} json=${parsed?.length ?: 0} " +
|
||||||
|
"retry=${if (parsed == null || out.length < 1) 1 else 0} " +
|
||||||
|
"ms=${System.currentTimeMillis() - t0}")
|
||||||
|
}
|
||||||
|
Log.i(TAG, "P1 phase total_ms=${System.currentTimeMillis() - tP1Start} " +
|
||||||
|
"chunks=$total failed=$failed")
|
||||||
|
// 空笔记终检:全部块失败,或笔记仍是空壳(total==0 已在入口取消)
|
||||||
|
if (failed == total || notesEntriesEmpty(notes)) {
|
||||||
|
throw SummarizeEmptyException("整场转写未能形成任何笔记条目")
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 成稿(§5.3)----
|
||||||
|
if (listener.isCancelled()) throw SummarizeCancelledException()
|
||||||
|
listener.onProgress("成稿", total, total)
|
||||||
|
val nSpeakers = entries.map { it.speaker ?: "?" }.distinct().size
|
||||||
|
val user2 = buildDraftUserPrompt(
|
||||||
|
prompts, lang, meetingStartWallMs, nSpeakers,
|
||||||
|
entries.last().endMs, glossary, notes
|
||||||
|
)
|
||||||
|
val tP2 = System.currentTimeMillis()
|
||||||
|
val md = cleanDraft(engine.generate(
|
||||||
|
systemPrompt = prompts.p2System, userPrompt = user2,
|
||||||
|
maxNewTokens = 2500, temperature = 0.5f, topK = 64, topP = 0.95f,
|
||||||
|
))
|
||||||
|
Log.i(TAG, "P2 draft prompt_len=${user2.length} draft_len=${md.length} " +
|
||||||
|
"ms=${System.currentTimeMillis() - tP2}")
|
||||||
|
// 语种"校正"(整篇重写)已按用户要求移除:识别是什么语言,纪要就存
|
||||||
|
// 什么语言,不再为混语倾向多花一次 2500 token 生成。这里只留一行
|
||||||
|
// 纯统计日志(零成本),纪要中的混语留给人工处理。
|
||||||
|
val viol = scriptViolation(lang, md)
|
||||||
|
if (viol != null) Log.i(TAG, "SUMMARIZE langcheck NOTE($lang): $viol (no rewrite)")
|
||||||
|
if (md.isBlank()) throw SummarizeEmptyException("成稿输出为空")
|
||||||
|
Log.i(TAG, "SUMMARIZE wall clock total_ms=${System.currentTimeMillis() - tP1Start}")
|
||||||
|
return MeetingSummary(md, lang, notes, total, failed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -9,7 +9,6 @@
|
||||||
<LinearLayout
|
<LinearLayout
|
||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="match_parent"
|
android:layout_height="match_parent"
|
||||||
android:gravity="center"
|
|
||||||
android:orientation="vertical">
|
android:orientation="vertical">
|
||||||
|
|
||||||
<FrameLayout
|
<FrameLayout
|
||||||
|
|
@ -17,7 +16,8 @@
|
||||||
android:layout_height="0dp"
|
android:layout_height="0dp"
|
||||||
android:layout_weight="2.5">
|
android:layout_weight="2.5">
|
||||||
|
|
||||||
<!-- subtitle dialog stream, one item per speaker line -->
|
<!-- subtitle dialog stream, one item per speaker line;
|
||||||
|
tap the underlined speaker label to (re)name that speaker -->
|
||||||
<androidx.recyclerview.widget.RecyclerView
|
<androidx.recyclerview.widget.RecyclerView
|
||||||
android:id="@+id/subtitle_list"
|
android:id="@+id/subtitle_list"
|
||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
|
|
@ -43,32 +43,82 @@
|
||||||
app:layout_constraintTop_toTopOf="parent" />
|
app:layout_constraintTop_toTopOf="parent" />
|
||||||
</FrameLayout>
|
</FrameLayout>
|
||||||
|
|
||||||
|
<!-- 录音控制紧贴字幕区下方:开始/停止一眼可见 -->
|
||||||
<LinearLayout
|
<LinearLayout
|
||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="wrap_content"
|
android:layout_height="wrap_content"
|
||||||
android:layout_weight="0.5"
|
|
||||||
android:gravity="center"
|
android:gravity="center"
|
||||||
android:orientation="horizontal">
|
android:orientation="horizontal"
|
||||||
|
android:paddingTop="4dp"
|
||||||
|
android:paddingBottom="4dp">
|
||||||
|
|
||||||
<Button
|
<Button
|
||||||
android:id="@+id/record_button"
|
android:id="@+id/record_button"
|
||||||
android:layout_width="wrap_content"
|
android:layout_width="wrap_content"
|
||||||
android:layout_height="wrap_content"
|
android:layout_height="wrap_content"
|
||||||
android:text="@string/start" />
|
android:text="@string/start" />
|
||||||
|
</LinearLayout>
|
||||||
|
|
||||||
|
<!-- 纪要面板:状态一行 + 正文(hint 占位/markdown 正文叠放) -->
|
||||||
|
<TextView
|
||||||
|
android:id="@+id/minutes_status"
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:paddingStart="24dp"
|
||||||
|
android:paddingEnd="24dp"
|
||||||
|
android:textColor="#666666"
|
||||||
|
android:textSize="12sp"
|
||||||
|
android:visibility="gone" />
|
||||||
|
|
||||||
|
<FrameLayout
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="0dp"
|
||||||
|
android:layout_weight="1.2">
|
||||||
|
|
||||||
|
<TextView
|
||||||
|
android:id="@+id/minutes_hint"
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="match_parent"
|
||||||
|
android:gravity="center"
|
||||||
|
android:padding="16dp"
|
||||||
|
android:text="点击「生成纪要」离线总结本场对话;结果与上次纪要显示在这里"
|
||||||
|
android:textColor="#999999"
|
||||||
|
android:textSize="13sp" />
|
||||||
|
|
||||||
|
<ScrollView
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="match_parent"
|
||||||
|
android:padding="12dp">
|
||||||
|
|
||||||
|
<TextView
|
||||||
|
android:id="@+id/minutes_text"
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:textIsSelectable="true"
|
||||||
|
android:textSize="13sp" />
|
||||||
|
</ScrollView>
|
||||||
|
</FrameLayout>
|
||||||
|
|
||||||
|
<!-- 纪要操作按钮与纪要面板放在同一区,贴底部 -->
|
||||||
|
<LinearLayout
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:gravity="center"
|
||||||
|
android:orientation="horizontal"
|
||||||
|
android:paddingBottom="12dp">
|
||||||
|
|
||||||
<Button
|
<Button
|
||||||
android:id="@+id/name_speaker_button"
|
android:id="@+id/summarize_button"
|
||||||
|
android:layout_width="wrap_content"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:text="@string/generate_minutes" />
|
||||||
|
|
||||||
|
<Button
|
||||||
|
android:id="@+id/copy_minutes_button"
|
||||||
android:layout_width="wrap_content"
|
android:layout_width="wrap_content"
|
||||||
android:layout_height="wrap_content"
|
android:layout_height="wrap_content"
|
||||||
android:layout_marginStart="12dp"
|
android:layout_marginStart="12dp"
|
||||||
android:text="@string/name_speaker" />
|
android:text="@string/copy_minutes" />
|
||||||
|
|
||||||
<Button
|
|
||||||
android:id="@+id/clear_speakers_button"
|
|
||||||
android:layout_width="wrap_content"
|
|
||||||
android:layout_height="wrap_content"
|
|
||||||
android:layout_marginStart="12dp"
|
|
||||||
android:text="@string/clear_speakers" />
|
|
||||||
</LinearLayout>
|
</LinearLayout>
|
||||||
</LinearLayout>
|
</LinearLayout>
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,5 +4,4 @@
|
||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="wrap_content"
|
android:layout_height="wrap_content"
|
||||||
android:paddingBottom="8dp"
|
android:paddingBottom="8dp"
|
||||||
android:textIsSelectable="true"
|
|
||||||
android:textSize="16sp" />
|
android:textSize="16sp" />
|
||||||
|
|
|
||||||
|
|
@ -4,12 +4,13 @@
|
||||||
\n
|
\n
|
||||||
\n\n\n
|
\n\n\n
|
||||||
The source code and pre-trained models are publicly available.
|
The source code and pre-trained models are publicly available.
|
||||||
Please see https://github.com/k2-fsa/sherpa-onnx for details.
|
Please see https://github.com/k2fsa/sherpa-onnx for details.
|
||||||
\n\n
|
\n\n
|
||||||
Two-pass speech recognition with Next-gen Kaldi.
|
Two-pass speech recognition with Next-gen Kaldi.
|
||||||
</string>
|
</string>
|
||||||
<string name="start">Start</string>
|
<string name="start">Start</string>
|
||||||
<string name="stop">Stop</string>
|
<string name="stop">Stop</string>
|
||||||
<string name="name_speaker">命名说话人</string>
|
<string name="generate_minutes">生成纪要</string>
|
||||||
<string name="clear_speakers">清除说话人</string>
|
<string name="cancel_minutes">取消生成</string>
|
||||||
|
<string name="copy_minutes">复制纪要</string>
|
||||||
</resources>
|
</resources>
|
||||||
|
|
@ -0,0 +1,45 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import org.junit.Assert.assertEquals
|
||||||
|
import org.junit.Assert.assertNull
|
||||||
|
import org.junit.Assert.assertTrue
|
||||||
|
import org.junit.Test
|
||||||
|
import java.io.File
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// 手动会中成稿的增量断点单测:.notes.json 读写校验 + P2 提示词两路同构。
|
||||||
|
// ============================================================================
|
||||||
|
class MeetingNotesTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun persistRoundTripAndValidation() {
|
||||||
|
val dir = File.createTempFile("notes", "dir").apply { delete(); mkdirs() }
|
||||||
|
val f = MeetingNotesIO.pathFor(dir, "20260911-1300")
|
||||||
|
val notes = """{"已定事項":[],"討論中":[],"行動項目":[],"人物":[],"前回末尾":"x"}"""
|
||||||
|
MeetingNotesIO.save(f, MeetingNotes(notes, 42, 321_000L, 1, "20260911-1300"))
|
||||||
|
val back = MeetingNotesIO.load(f, "20260911-1300")!!
|
||||||
|
assertEquals(42, back.entriesP1Done)
|
||||||
|
assertEquals(321_000L, back.lastEndMs)
|
||||||
|
assertEquals(1, back.failedChunks)
|
||||||
|
assertEquals(notes, back.notes)
|
||||||
|
// sessionId 不匹配 → null(不会把别场会议的笔记接错)
|
||||||
|
assertNull(MeetingNotesIO.load(f, "other-session"))
|
||||||
|
// 笔记被写坏 → null(自动退回全量重跑)
|
||||||
|
f.writeText("""{"sessionId":"20260911-1300","notes":"garbage{"}""")
|
||||||
|
assertNull(MeetingNotesIO.load(f, "20260911-1300"))
|
||||||
|
// 文件不存在 → null
|
||||||
|
assertNull(MeetingNotesIO.load(File(dir, "nope.notes.json"), "nope"))
|
||||||
|
dir.deleteRecursively()
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun draftPromptSameShapeBothPaths() {
|
||||||
|
val p = PromptsLoader.BUILTIN
|
||||||
|
val u = SummarizePipeline.buildDraftUserPrompt(
|
||||||
|
p, "ja", 1_789_000_000_000L, 3, 3_720_000L, "张三→張三", "NOTESJSON")
|
||||||
|
assertTrue(u.contains("日本語"))
|
||||||
|
assertTrue(u.contains("62:00"))
|
||||||
|
assertTrue(u.contains("NOTESJSON"))
|
||||||
|
assertTrue(u.contains("张三→張三"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,78 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import org.junit.Assert.assertEquals
|
||||||
|
import org.junit.Assert.assertFalse
|
||||||
|
import org.junit.Assert.assertNull
|
||||||
|
import org.junit.Assert.assertTrue
|
||||||
|
import org.junit.Test
|
||||||
|
|
||||||
|
class SubtitleFormatTest {
|
||||||
|
@Test
|
||||||
|
fun joinerRules() {
|
||||||
|
// 已有标点 → 空格衔接
|
||||||
|
assertEquals(" ", joinerAfter("今天定了预算。"))
|
||||||
|
assertEquals(" ", joinerAfter("deadline is Friday."))
|
||||||
|
assertEquals(" ", joinerAfter("終わりです!"))
|
||||||
|
// CJK/假名结尾 → 中文句号
|
||||||
|
assertEquals("。", joinerAfter("予算を承認"))
|
||||||
|
assertEquals("。", joinerAfter("会议延期"))
|
||||||
|
// 拉丁结尾 → 英文句点+空格
|
||||||
|
assertEquals(". ", joinerAfter("we agreed"))
|
||||||
|
assertEquals(". ", joinerAfter("Q3"))
|
||||||
|
// 空/符号结尾
|
||||||
|
assertEquals(" ", joinerAfter(""))
|
||||||
|
assertEquals(" ", joinerAfter(" "))
|
||||||
|
assertEquals(" ", joinerAfter("待定:"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
class SpeakerRegistryTest {
|
||||||
|
private fun e(vararg v: Float) = v.map { it.toFloat() }.toFloatArray()
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun autoThenRenameKeepsIdentity() {
|
||||||
|
val r = SpeakerRegistry()
|
||||||
|
val a = e(1f, 0f, 0f)
|
||||||
|
r.addAuto("S0", a)
|
||||||
|
assertEquals("S0", r.search(a, 0.5f))
|
||||||
|
assertTrue(r.renameToName("S0", "张三"))
|
||||||
|
// 改名后同一嵌入直接出真名,不再命中 S0
|
||||||
|
assertEquals("张三", r.search(a, 0.5f))
|
||||||
|
assertNull(r.search(e(0f, 1f, 0f), 0.5f)) // 无关向量不匹配
|
||||||
|
assertFalse(r.renameToName("S9", "李四")) // 不存在的键
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun sameNameCanHoldMultipleEmbeddings() {
|
||||||
|
// 两个被误判分开的人(正交嵌入),命名同名后都归一
|
||||||
|
val r = SpeakerRegistry()
|
||||||
|
val a = e(1f, 0f, 0f)
|
||||||
|
val b = e(0f, 1f, 0f)
|
||||||
|
r.addAuto("S0", a)
|
||||||
|
r.addAuto("S1", b)
|
||||||
|
assertTrue(r.renameToName("S0", "张三"))
|
||||||
|
assertTrue(r.renameToName("S1", "张三"))
|
||||||
|
assertEquals("张三", r.search(a, 0.5f))
|
||||||
|
assertEquals("张三", r.search(b, 0.5f))
|
||||||
|
// 部分匹配:与 a 接近的新嵌入仍判张三
|
||||||
|
assertEquals("张三", r.search(e(0.9f, 0f, 0.1f), 0.5f))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun namedWinsOverAutoOnTie() {
|
||||||
|
val r = SpeakerRegistry()
|
||||||
|
val a = e(1f, 0f, 0f)
|
||||||
|
r.addAuto("S0", a)
|
||||||
|
r.addNamed("张三", a)
|
||||||
|
assertEquals("张三", r.search(a, 0.5f))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun similarityBasics() {
|
||||||
|
val a = e(1f, 0f, 0f)
|
||||||
|
assertEquals(1f, SpeakerRecognition.similarity(a, a), 1e-4f)
|
||||||
|
assertEquals(0f, SpeakerRecognition.similarity(a, e(0f, 1f, 0f)), 1e-4f)
|
||||||
|
assertEquals(-1f, SpeakerRecognition.similarity(a, e(-1f, 0f, 0f)), 1e-4f)
|
||||||
|
assertEquals(0f, SpeakerRecognition.similarity(a, e(0f, 0f, 0f)), 1e-4f)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,176 @@
|
||||||
|
package com.k2fsa.sherpa.onnx
|
||||||
|
|
||||||
|
import org.junit.Assert.assertEquals
|
||||||
|
import org.junit.Assert.assertNotNull
|
||||||
|
import org.junit.Assert.assertNull
|
||||||
|
import org.junit.Assert.assertTrue
|
||||||
|
import org.junit.Test
|
||||||
|
import java.io.File
|
||||||
|
|
||||||
|
// ============================================================================
|
||||||
|
// 摘要流水线纯逻辑 JVM 单测(不依赖设备/native):
|
||||||
|
// ./gradlew :app:testDebugUnitTest
|
||||||
|
// 注意:unit test classpath 里放了真实 org.json(build.gradle testImplementation)。
|
||||||
|
// ============================================================================
|
||||||
|
class SummarizeLogicTest {
|
||||||
|
|
||||||
|
// ---- prompts.md ↔ 内置 BUILTIN 一致性(md 是唯一事实源,漂移即失败) ----
|
||||||
|
@Test
|
||||||
|
fun builtinMatchesPromptsMd() {
|
||||||
|
val md = File("src/main/assets/summarize/prompts.md")
|
||||||
|
assertTrue("prompts.md not found at ${md.absolutePath}", md.isFile)
|
||||||
|
val parsed = PromptsLoader.parse(md.readText())
|
||||||
|
val b = PromptsLoader.BUILTIN
|
||||||
|
assertEquals("P1 SYSTEM 与 md 不一致", b.p1System, parsed.p1System)
|
||||||
|
assertEquals("P1 USER 与 md 不一致", b.p1UserTpl, parsed.p1UserTpl)
|
||||||
|
assertEquals("P2 SYSTEM 与 md 不一致", b.p2System, parsed.p2System)
|
||||||
|
assertEquals("P2 USER 与 md 不一致", b.p2UserTpl, parsed.p2UserTpl)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 分块 ----
|
||||||
|
private fun line(spk: String, i: Int) =
|
||||||
|
"(00:%02d-00:%02d)[%s] %s".format(i % 60, i % 60, spk, "あ".repeat(80) + i)
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun chunkCoversEveryUtteranceOnceIgnoringOverlap() {
|
||||||
|
val lines = (0 until 60).map { line(if (it / 7 % 2 == 0) "S0" else "S1", it) }
|
||||||
|
val blocks = SummarizePipeline.chunk(lines)
|
||||||
|
// 每条原句(去掉重叠复制)合起来必须完整覆盖
|
||||||
|
val originals = blocks.flatMap { it.split('\n') }.filter { it.isNotBlank() }
|
||||||
|
val dedup = LinkedHashSet(originals)
|
||||||
|
assertEquals("有句子丢失", 60, dedup.size)
|
||||||
|
assertTrue("块数不合常理", blocks.size in 2..12)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun chunkNeverSplitsSameSpeakerRun() {
|
||||||
|
// 每 7 句一组同人轮次;切点不得落在组中间
|
||||||
|
val lines = ArrayList<String>()
|
||||||
|
var spk = 0
|
||||||
|
var k = 0
|
||||||
|
repeat(70) {
|
||||||
|
if (it % 7 == 0) { spk++; }
|
||||||
|
lines.add(line("S$spk", k++))
|
||||||
|
}
|
||||||
|
val blocks = SummarizePipeline.chunk(lines)
|
||||||
|
fun spkOf(l: String): String =
|
||||||
|
l.substringAfterLast('[').substringBeforeLast(']')
|
||||||
|
// 相邻块之间:上一块最后一句与下一块第二句(第一句是重叠)必须不同人
|
||||||
|
for (i in 0 until blocks.size - 1) {
|
||||||
|
val lastOfPrev = blocks[i].split('\n').last()
|
||||||
|
val firstRealOfNext = blocks[i + 1].split('\n').drop(1).first()
|
||||||
|
assertTrue(
|
||||||
|
"轮次被拆:prev尾=${lastOfPrev.take(30)} next=${firstRealOfNext.take(30)}",
|
||||||
|
spkOf(lastOfPrev) != spkOf(firstRealOfNext)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun chunkOverlapsOneSentence() {
|
||||||
|
val lines = (0 until 40).map { line("S${it % 3}", it) }
|
||||||
|
val blocks = SummarizePipeline.chunk(lines)
|
||||||
|
for (i in 0 until blocks.size - 1) {
|
||||||
|
val tail = blocks[i].split('\n').last()
|
||||||
|
val nextHead = blocks[i + 1].split('\n').first()
|
||||||
|
assertEquals("块 ${i + 1}->${i + 2} 缺少 1 句重叠", tail, nextHead)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 笔记 JSON 提取 ----
|
||||||
|
private val goodJson =
|
||||||
|
"""{"已定事項":[{"内容":"Xを承認","出典":"01:00-01:20"}],"討論中":[],"行動項目":[{"内容":"Y","担当者":"田中","期限":"9/30","出典":"02:00-02:10"}],"人物":[{"名前":"田中","立場":"PM"}],"前回末尾":"Yの進め方" }"""
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun extractJsonPlain() = assertEquals(goodJson, SummarizePipeline.extractJson(goodJson))
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun extractJsonWithFenceAndProse() {
|
||||||
|
val raw = "はい、更新します。\n```json\n$goodJson\n```\n以上です。"
|
||||||
|
assertEquals(goodJson, SummarizePipeline.extractJson(raw))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun extractJsonHandlesBracesInsideStrings() {
|
||||||
|
val raw = """{"已定事項":[{"内容":"値は{要確認}","出典":""}],"討論中":[],"行動項目":[],"人物":[],"前回末尾":""}"""
|
||||||
|
assertNotNull(SummarizePipeline.extractJson(raw))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun extractJsonRejectsGarbageAndMissingKeys() {
|
||||||
|
assertNull(SummarizePipeline.extractJson("ごめんなさい、分かりません"))
|
||||||
|
assertNull(SummarizePipeline.extractJson("""{"hello":"world"}"""))
|
||||||
|
assertNull(SummarizePipeline.extractJson("""{"已定事項":[],}""")) // 不平衡/畸形
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 成稿清洗 ----
|
||||||
|
@Test
|
||||||
|
fun cleanDraftStripsFenceAndThinking() {
|
||||||
|
val raw = "```markdown\n<|channel>thought\n長い熟考…\n<channel|>\n# 議題\n\n\n\n本文\n```"
|
||||||
|
val out = SummarizePipeline.cleanDraft(raw)
|
||||||
|
assertTrue(out.startsWith("# 議題"))
|
||||||
|
assertTrue(!out.contains("thought"))
|
||||||
|
assertTrue(!out.contains("```"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 语言判定 ----
|
||||||
|
@Test
|
||||||
|
fun detectLanguageBasics() {
|
||||||
|
assertEquals("ja", SummarizePipeline.detectLanguage("会議を始めます。本日はよろしくお願いいたします。"))
|
||||||
|
assertEquals("zh", SummarizePipeline.detectLanguage("今天开会讨论季度预算和市场推广计划"))
|
||||||
|
assertEquals("en", SummarizePipeline.detectLanguage("Let us review the quarterly budget plan together"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- 语种纯度校验(中→全中/英→全英,专名例外;阈值=句子级混语) ----
|
||||||
|
@Test
|
||||||
|
fun scriptViolationEnforcement() {
|
||||||
|
// 中文纪要 + 少量假名专名(田中さくら)→ 通过(专名例外)
|
||||||
|
assertNull(SummarizePipeline.scriptViolation(
|
||||||
|
"zh", "行动项:请 田中さくら 在9月30日前提交预算表,负责人张三。"))
|
||||||
|
// 中文纪要被日语句子污染 → 违规
|
||||||
|
assertNotNull(SummarizePipeline.scriptViolation(
|
||||||
|
"zh", "决定事项:来月の予算を承認しました。本件は保留となります。"))
|
||||||
|
// 英文纪要混入汉字 → 违规
|
||||||
|
assertNotNull(SummarizePipeline.scriptViolation(
|
||||||
|
"en", "Action: 佐藤が担当する。Deliver the draft by Friday and review together."))
|
||||||
|
// 英文纪要单个汉字公司名(专名级)→ 阈值内通过
|
||||||
|
assertNull(SummarizePipeline.scriptViolation(
|
||||||
|
"en", "Decision: approve partnership with 丰田汽车 by Q4."))
|
||||||
|
// 日文纪要不做字符检查(汉字假名天然混排)
|
||||||
|
assertNull(SummarizePipeline.scriptViolation("ja", "意思決定:予算を承認。We agree と結論"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Gemma 4 模板渲染 ----
|
||||||
|
@Test
|
||||||
|
fun chatTemplateRendering() {
|
||||||
|
val p = LlmEngine.renderChatPrompt("SYS ", " USER")
|
||||||
|
assertEquals(
|
||||||
|
"<|turn>system\nSYS<turn|>\n<|turn>user\nUSER<turn|>\n<|turn>model\n", p
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- SessionTranscript JSONL 回放 ----
|
||||||
|
@Test
|
||||||
|
fun transcriptReplayFoldsEvents() {
|
||||||
|
val dir = File.createTempFile("mtg", "dir").apply { delete(); mkdirs() }
|
||||||
|
val f = File(dir, "test.jsonl")
|
||||||
|
f.writeText(
|
||||||
|
"""{"t":"H","wall":1000}
|
||||||
|
{"t":"S","sid":1,"b":0,"e":1000,"x":"第一稿"}
|
||||||
|
{"t":"S","sid":2,"b":1000,"e":2000,"x":"第二句"}
|
||||||
|
{"t":"R","sid":1,"x":"精修后第一稿","r":1}
|
||||||
|
{"t":"P","sid":1,"sp":"S0"}
|
||||||
|
{"t":"P","sid":2,"sp":"S1"}
|
||||||
|
{"t":"N","from":"S1","to":"张三"}
|
||||||
|
""".replace("\n", System.lineSeparator())
|
||||||
|
)
|
||||||
|
val out = SessionTranscript.load(f)
|
||||||
|
assertEquals(2, out.size)
|
||||||
|
assertEquals("精修后第一稿", out[0].text)
|
||||||
|
assertTrue(out[0].refined)
|
||||||
|
assertEquals("S0", out[0].speaker)
|
||||||
|
assertEquals("张三", out[1].speaker)
|
||||||
|
assertEquals(1000L, SessionTranscript.fileStartWallMs(f))
|
||||||
|
dir.deleteRecursively()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,57 @@
|
||||||
|
# 「会中手动成稿」共存压测方案
|
||||||
|
|
||||||
|
> 需求变更后取代原"会中自动滚动"压测(2026-09-11):滚动机制已拆除,
|
||||||
|
> 现行为 = 点「生成纪要」→ 对截至当时已定稿的字幕成稿,录音转写全程不停;
|
||||||
|
> 增量笔记(.notes.json)使第二次及以后的点击只处理新增句子(分钟级)。
|
||||||
|
>
|
||||||
|
> 通过线:录音零丢帧、定稿不超时不劣化、成稿期间 UI 不卡死、多次点击增量正确。
|
||||||
|
|
||||||
|
## 1. 准备
|
||||||
|
|
||||||
|
1. 安装 `build-check/summarize-manual-debug.apk`。
|
||||||
|
2. 测试音频:任意真实会议录音(日语最佳)≥30 分钟,播放器开单曲循环凑 2~3 小时。
|
||||||
|
3. 手机充电、屏幕常亮(锁屏/Doze 会干扰后台线程)。
|
||||||
|
4. 如要卸载旧版重装:`adb pull /sdcard/Android/data/com.k2fsa.sherpa.onnx/files/models/ D:\sherpa-models-backup` 备份模型,装完 push 回去。
|
||||||
|
|
||||||
|
## 2. 日志采集(PowerShell 开三个窗口)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 窗1:App 主日志(SLOT 对账 / SUMMARIZE / ROLLING 续跑标记)
|
||||||
|
adb logcat -s sherpa-onnx SummarizeLlmJni | Tee-Object D:\stress-app.log
|
||||||
|
|
||||||
|
# 窗2:每 5 分钟采样 RSS 与电池温度
|
||||||
|
while ($true) {
|
||||||
|
adb shell dumpsys meminfo com.k2fsa.sherpa.onnx | Select-String "TOTAL RSS"
|
||||||
|
adb shell dumpsys battery | Select-String "temperature|level"
|
||||||
|
Start-Sleep 300
|
||||||
|
} | Tee-Object D:\stress-rss.log
|
||||||
|
|
||||||
|
# 窗3:录音丢帧(看 s: 恒 0、m: 不增长)
|
||||||
|
adb logcat | Select-String "audioRecordData" | Tee-Object D:\stress-audio.log
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3. 跑测步骤
|
||||||
|
|
||||||
|
1. Start 录音,循环播放会议音频。
|
||||||
|
2. 约 1 小时后点「生成纪要」→(第 1 次)全量流水线 → 出稿。**记录耗时 T1**。生成期间字幕应继续实时更新。
|
||||||
|
3. 再过 1 小时点第二次 → 日志应出现 `SUMMARIZE resume from notes at N/M`,只消化新增段,**记录耗时 T2**(应远小于 T1)。
|
||||||
|
4. 会议结束 Stop → 再点第三次出终稿。
|
||||||
|
5. 取回产物:
|
||||||
|
```powershell
|
||||||
|
adb exec-out run-as com.k2fsa.sherpa.onnx tar -C files/meetings -c . > D:\stress-out.tar
|
||||||
|
```
|
||||||
|
应看到 `<会话>.jsonl`(全量转写)、`<会话>.notes.json`(断点笔记)、`<会话>.md`(每次覆盖为最新终稿)。
|
||||||
|
|
||||||
|
## 4. 判读标准(把三个 log + 三份 md 发我)
|
||||||
|
|
||||||
|
| 项 | 通过线 | 依据 |
|
||||||
|
|---|---|---|
|
||||||
|
| 录音零丢帧 | `s:0` 恒 0,`m:` 在成稿窗口内不增长 | stress-audio.log |
|
||||||
|
| 定稿不劣化 | 成稿的 1~2 分钟窗口里 `SLOT … RESULT=TIMEOUT` 频率不高于基线 | stress-app.log |
|
||||||
|
| 增量正确 | 第 2/3 次点击 `resume from notes` 命中,T2 << T1;终稿 .md 覆盖全程且无重复条目 | stress-app.log + .md |
|
||||||
|
| 资源 | 引擎驻留只在生成期间(~3.5GB RSS 峰值,完成即回落);温度 >45°C 时记录掉速 | stress-rss.log |
|
||||||
|
| 质量 | 三次纪要人工评审"无漏关键决定";出典时间戳可回查 .jsonl | .md |
|
||||||
|
|
||||||
|
## 5. 红线回归
|
||||||
|
|
||||||
|
`MainActivity` 顶部 `summarizeEnabled=false` → 无按钮、无会话文件、行为与加摘要功能前一致。
|
||||||
|
|
@ -0,0 +1,22 @@
|
||||||
|
# Builds libsummarize-llm-jni.so = llama.cpp (static) + thin JNI layer.
|
||||||
|
# llama.cpp is expected at ./llama.cpp (pinned by build_arm64.sh), NOT committed here.
|
||||||
|
cmake_minimum_required(VERSION 3.22)
|
||||||
|
project(summarize_llm_jni CXX C)
|
||||||
|
|
||||||
|
set(LLAMA_BUILD_TESTS OFF CACHE BOOL "" FORCE)
|
||||||
|
set(LLAMA_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
|
||||||
|
set(LLAMA_BUILD_SERVER OFF CACHE BOOL "" FORCE)
|
||||||
|
set(LLAMA_BUILD_UI OFF CACHE BOOL "" FORCE)
|
||||||
|
set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
|
||||||
|
set(LLAMA_BUILD_COMMON OFF CACHE BOOL "" FORCE)
|
||||||
|
set(GGML_OPENMP OFF CACHE BOOL "" FORCE)
|
||||||
|
set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
|
||||||
|
|
||||||
|
add_subdirectory(llama.cpp EXCLUDE_FROM_ALL)
|
||||||
|
|
||||||
|
add_library(summarize-llm-jni SHARED src/summarize_llm_jni.cpp)
|
||||||
|
target_link_libraries(summarize-llm-jni PRIVATE llama android log)
|
||||||
|
target_compile_options(summarize-llm-jni PRIVATE -O2 -fvisibility=hidden)
|
||||||
|
target_link_options(summarize-llm-jni PRIVATE
|
||||||
|
"-Wl,--build-id=sha1" "-Wl,--gc-sections" "-Wl,-z,max-page-size=16384"
|
||||||
|
"-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/exports.map")
|
||||||
|
|
@ -0,0 +1,38 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# Cross-compiles libsummarize-llm-jni.so (arm64-v8a) for the summarize feature.
|
||||||
|
#
|
||||||
|
# One-off build on a Linux box with network access (this project was built in
|
||||||
|
# a sandbox with NDK r27c; the Android Studio side does NOT need any NDK).
|
||||||
|
# Run from tools/llama-jni/. Requires: git, cmake >= 3.22, unzip, $ANDROID_NDK
|
||||||
|
# pointing at an extracted NDK (r26+), ANDROID_ABI override optional.
|
||||||
|
#
|
||||||
|
# llama.cpp commit pinned below = the one verified to support arch "gemma4"
|
||||||
|
# (src/llama-arch.cpp LLM_ARCH_GEMMA4). Bump only together with re-running
|
||||||
|
# PC-side prompt validation.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
LLAMA_COMMIT="df03399b885831b2a1603b3abb0d8c156808e363"
|
||||||
|
ABI="${ANDROID_ABI:-arm64-v8a}"
|
||||||
|
PLATFORM="${ANDROID_PLATFORM:-android-24}"
|
||||||
|
OUT_DIR="../../app/src/main/jniLibs/${ABI}"
|
||||||
|
|
||||||
|
if [ ! -d llama.cpp ]; then
|
||||||
|
git clone https://github.com/ggml-org/llama.cpp
|
||||||
|
fi
|
||||||
|
cd llama.cpp
|
||||||
|
git fetch --depth 1 origin "${LLAMA_COMMIT}" 2>/dev/null || true
|
||||||
|
git checkout "${LLAMA_COMMIT}" 2>/dev/null || git pull
|
||||||
|
cd ..
|
||||||
|
|
||||||
|
cmake -B "build-${ABI}" \
|
||||||
|
-DCMAKE_TOOLCHAIN_FILE="${ANDROID_NDK}/build/cmake/android.toolchain.cmake" \
|
||||||
|
-DANDROID_ABI="${ABI}" \
|
||||||
|
-DANDROID_PLATFORM="${PLATFORM}" \
|
||||||
|
-DANDROID_STL=c++_static \
|
||||||
|
-DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON \
|
||||||
|
-DCMAKE_BUILD_TYPE=Release
|
||||||
|
cmake --build "build-${ABI}" -j "$(nproc)"
|
||||||
|
|
||||||
|
mkdir -p "${OUT_DIR}"
|
||||||
|
cp -v "build-${ABI}/libsummarize-llm-jni.so" "${OUT_DIR}/"
|
||||||
|
echo "done -> ${OUT_DIR}/libsummarize-llm-jni.so"
|
||||||
|
|
@ -0,0 +1,6 @@
|
||||||
|
{
|
||||||
|
global:
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_*;
|
||||||
|
local:
|
||||||
|
*;
|
||||||
|
};
|
||||||
|
|
@ -0,0 +1,312 @@
|
||||||
|
// Summarize-LLM thin JNI layer over llama.cpp (milestone 1: post-meeting minutes).
|
||||||
|
//
|
||||||
|
// This is the ONLY native code added for the summarize feature. It is compiled
|
||||||
|
// together with a pinned llama.cpp checkout into a single prebuilt shared
|
||||||
|
// library (libsummarize-llm-jni.so) so that the app's Gradle build keeps using
|
||||||
|
// only prebuilt .so files (no externalNativeBuild, no NDK required on the
|
||||||
|
// developer machine). See tools/llama-jni/build_arm64.sh.
|
||||||
|
//
|
||||||
|
// Safety rules (mirrors the project's "missing asset => native exit()" lesson):
|
||||||
|
// - Every entry point validates inputs and throws a Java exception instead
|
||||||
|
// of crashing; model loading never calls exit().
|
||||||
|
// - The caller (Kotlin) MUST check that the GGUF file exists before
|
||||||
|
// nativeNewFromFile(); we double-check with access() here as well.
|
||||||
|
|
||||||
|
#include <jni.h>
|
||||||
|
#include <android/log.h>
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <mutex>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
#include <algorithm>
|
||||||
|
#include <cmath>
|
||||||
|
#include <random>
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
#include "llama.h"
|
||||||
|
|
||||||
|
#define LOG_TAG "SummarizeLlmJni"
|
||||||
|
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)
|
||||||
|
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, LOG_TAG, __VA_ARGS__)
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
struct Engine {
|
||||||
|
llama_model * model = nullptr;
|
||||||
|
llama_context * ctx = nullptr;
|
||||||
|
const llama_vocab * vocab = nullptr;
|
||||||
|
std::mutex mu; // generate() is single-flight
|
||||||
|
std::atomic<bool> abort{false};
|
||||||
|
};
|
||||||
|
|
||||||
|
void throwJava(JNIEnv * env, const char * msg) {
|
||||||
|
jclass cls = env->FindClass("java/lang/RuntimeException");
|
||||||
|
if (cls != nullptr) {
|
||||||
|
env->ThrowNew(cls, msg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::string jstringToUtf8(JNIEnv * env, jstring s) {
|
||||||
|
if (s == nullptr) return {};
|
||||||
|
const char * c = env->GetStringUTFChars(s, nullptr);
|
||||||
|
std::string out = c != nullptr ? c : "";
|
||||||
|
if (c != nullptr) env->ReleaseStringUTFChars(s, c);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
ggml_type parseKvType(const std::string & s, ggml_type fallback) {
|
||||||
|
if (s == "f16") return GGML_TYPE_F16;
|
||||||
|
if (s == "f32") return GGML_TYPE_F32;
|
||||||
|
if (s == "q8_0") return GGML_TYPE_Q8_0;
|
||||||
|
if (s == "q4_0") return GGML_TYPE_Q4_0;
|
||||||
|
if (s == "q4_1") return GGML_TYPE_Q4_1;
|
||||||
|
return fallback;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Abort callback for llama_set_abort_callback.
|
||||||
|
bool abortCb(void * user) {
|
||||||
|
auto * e = static_cast<Engine *>(user);
|
||||||
|
return e->abort.load(std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
extern "C" {
|
||||||
|
|
||||||
|
JNIEXPORT jlong JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeNewFromFile(
|
||||||
|
JNIEnv * env, jclass /*clazz*/,
|
||||||
|
jstring jPath, jint ctxSize, jint nThreads,
|
||||||
|
jstring jTypeK, jstring jTypeV) {
|
||||||
|
|
||||||
|
const std::string path = jstringToUtf8(env, jPath);
|
||||||
|
if (path.empty()) {
|
||||||
|
throwJava(env, "nativeNewFromFile: empty model path");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
if (access(path.c_str(), R_OK) != 0) {
|
||||||
|
throwJava(env, ("nativeNewFromFile: model file not readable: " + path).c_str());
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
llama_model_params mparams = llama_model_default_params();
|
||||||
|
mparams.n_gpu_layers = 0; // CPU only in milestone 1
|
||||||
|
mparams.load_mode = LLAMA_LOAD_MODE_MMAP; // page cache can be reclaimed under memory pressure
|
||||||
|
|
||||||
|
llama_model * model = llama_model_load_from_file(path.c_str(), mparams);
|
||||||
|
if (model == nullptr) {
|
||||||
|
throwJava(env, ("nativeNewFromFile: llama failed to load GGUF: " + path).c_str());
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
llama_context_params cparams = llama_context_default_params();
|
||||||
|
cparams.n_ctx = ctxSize > 0 ? (uint32_t) ctxSize : 8192;
|
||||||
|
cparams.n_threads = nThreads > 0 ? nThreads : 4;
|
||||||
|
cparams.n_threads_batch = nThreads > 0 ? nThreads : 4;
|
||||||
|
cparams.type_k = parseKvType(jstringToUtf8(env, jTypeK), GGML_TYPE_Q8_0);
|
||||||
|
cparams.type_v = parseKvType(jstringToUtf8(env, jTypeV), GGML_TYPE_F16);
|
||||||
|
|
||||||
|
llama_context * ctx = llama_init_from_model(model, cparams);
|
||||||
|
if (ctx == nullptr) {
|
||||||
|
llama_model_free(model);
|
||||||
|
throwJava(env, "nativeNewFromFile: failed to create llama_context");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
auto * e = new Engine();
|
||||||
|
e->model = model;
|
||||||
|
e->ctx = ctx;
|
||||||
|
e->vocab = llama_model_get_vocab(model);
|
||||||
|
llama_set_abort_callback(ctx, abortCb, e);
|
||||||
|
|
||||||
|
LOGI("model loaded: %s n_ctx=%d type_k=%d type_v=%d", path.c_str(),
|
||||||
|
cparams.n_ctx, (int) cparams.type_k, (int) cparams.type_v);
|
||||||
|
return (jlong) (uintptr_t) e;
|
||||||
|
}
|
||||||
|
|
||||||
|
JNIEXPORT jstring JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeGetStringMeta(
|
||||||
|
JNIEnv * env, jclass /*clazz*/, jlong ptr, jstring jKey) {
|
||||||
|
|
||||||
|
auto * e = (Engine *) (uintptr_t) ptr;
|
||||||
|
if (e == nullptr || e->model == nullptr) return nullptr;
|
||||||
|
const std::string key = jstringToUtf8(env, jKey);
|
||||||
|
char buf[8192];
|
||||||
|
int32_t n = llama_model_meta_val_str(e->model, key.c_str(), buf, sizeof(buf));
|
||||||
|
if (n < 0) return nullptr;
|
||||||
|
return env->NewStringUTF(buf);
|
||||||
|
}
|
||||||
|
|
||||||
|
// M2:录音中降线程(给 ASR 让核),空闲/成稿时拉满。与在途 generate 共用
|
||||||
|
// 一把 mu:generate 正在跑时本调用会排队到它结束,不会半路改线程数。
|
||||||
|
JNIEXPORT void JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeSetThreads(
|
||||||
|
JNIEnv * /*env*/, jclass /*clazz*/, jlong ptr, jint n) {
|
||||||
|
auto * e = (Engine *) (uintptr_t) ptr;
|
||||||
|
if (e == nullptr || e->ctx == nullptr || n <= 0) return;
|
||||||
|
std::lock_guard<std::mutex> lock(e->mu);
|
||||||
|
llama_set_n_threads(e->ctx, n, n);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Blocking generation; call from a background thread. Returns generated text
|
||||||
|
// (without the prompt). Stops on EOS, maxNewTokens or nativeAbort().
|
||||||
|
JNIEXPORT jstring JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeGenerate(
|
||||||
|
JNIEnv * env, jclass /*clazz*/, jlong ptr,
|
||||||
|
jstring jPrompt, jint maxNewTokens, jfloat temperature, jint topK, jfloat topP) {
|
||||||
|
|
||||||
|
auto * e = (Engine *) (uintptr_t) ptr;
|
||||||
|
if (e == nullptr || e->ctx == nullptr) {
|
||||||
|
throwJava(env, "nativeGenerate: engine is released");
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
std::lock_guard<std::mutex> lock(e->mu);
|
||||||
|
e->abort.store(false);
|
||||||
|
|
||||||
|
const std::string prompt = jstringToUtf8(env, jPrompt);
|
||||||
|
if (prompt.empty()) {
|
||||||
|
throwJava(env, "nativeGenerate: empty prompt");
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- tokenize the prompt ---
|
||||||
|
// parse_special=true: the Kotlin side renders the Gemma 4 template with
|
||||||
|
// literal <|turn>/<turn|> markers, which must become control tokens 105/106.
|
||||||
|
std::vector<llama_token> tokens(prompt.size() / 2 + 64);
|
||||||
|
int32_t n;
|
||||||
|
while ((n = llama_tokenize(e->vocab, prompt.data(), (int32_t) prompt.size(),
|
||||||
|
tokens.data(), (int32_t) tokens.size(),
|
||||||
|
/*add_special=*/true, /*parse_special=*/true)) < 0) {
|
||||||
|
tokens.resize((size_t) (-n) + 32);
|
||||||
|
if (tokens.size() > 4 * 1024 * 1024) {
|
||||||
|
throwJava(env, "nativeGenerate: prompt too large to tokenize");
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- fit into the context window (left-truncate keeps the newest content,
|
||||||
|
// which for our rolling-notes design is the part that must survive) ---
|
||||||
|
const uint32_t nCtx = llama_n_ctx(e->ctx);
|
||||||
|
const int32_t keep = (int32_t) (nCtx > 64 ? nCtx - 32 : nCtx);
|
||||||
|
if (n > keep) {
|
||||||
|
LOGE("prompt tokens %d exceed ctx %u, left-truncating", n, nCtx);
|
||||||
|
int32_t drop = n - keep;
|
||||||
|
// never drop into the middle of multi-byte UTF-8: only relevant on the
|
||||||
|
// decode side; tokens are atomic here, so a plain shift is safe.
|
||||||
|
tokens.erase(tokens.begin(), tokens.begin() + drop);
|
||||||
|
n = keep;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- prefill in chunks ---
|
||||||
|
const int32_t chunk = 512;
|
||||||
|
llama_pos pos = 0;
|
||||||
|
for (int32_t i = 0; i < n; i += chunk) {
|
||||||
|
if (e->abort.load()) return env->NewStringUTF("");
|
||||||
|
int32_t len = std::min(chunk, n - i);
|
||||||
|
llama_batch b = llama_batch_get_one(tokens.data() + i, len);
|
||||||
|
if (llama_decode(e->ctx, b) != 0) {
|
||||||
|
throwJava(env, "nativeGenerate: prefill decode failed");
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
pos += len;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- decode loop ---
|
||||||
|
const llama_token eos = llama_vocab_eos(e->vocab);
|
||||||
|
std::mt19937 rng(42); // fixed seed => reproducible minutes across runs
|
||||||
|
std::string out;
|
||||||
|
out.reserve(1024);
|
||||||
|
std::vector<char> piece(64);
|
||||||
|
int32_t cur = tokens.back();
|
||||||
|
const int maxNew = maxNewTokens > 0 ? maxNewTokens : 512;
|
||||||
|
const bool greedy = !(temperature > 0.0f);
|
||||||
|
const int k = topK > 0 ? topK : 64; // Gemma 4 recommended: 64
|
||||||
|
const float p = topP > 0.0f ? topP : 0.95f; // recommended: ~0.95
|
||||||
|
|
||||||
|
for (int i = 0; i < maxNew; ++i) {
|
||||||
|
if (e->abort.load()) break;
|
||||||
|
llama_batch b = llama_batch_get_one(&cur, 1);
|
||||||
|
if (llama_decode(e->ctx, b) != 0) {
|
||||||
|
throwJava(env, "nativeGenerate: decode failed");
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
float * logits = llama_get_logits_ith(e->ctx, 0);
|
||||||
|
const int vsize = llama_vocab_n_tokens(e->vocab);
|
||||||
|
|
||||||
|
// pick next token
|
||||||
|
llama_token next;
|
||||||
|
if (greedy) {
|
||||||
|
next = 0;
|
||||||
|
float best = logits[0];
|
||||||
|
for (int t = 1; t < vsize; ++t) {
|
||||||
|
if (logits[t] > best) { best = logits[t]; next = t; }
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// top-k cut (partial sort), then temperature softmax + top-p cut, then sample
|
||||||
|
std::vector<std::pair<float, int32_t>> cand;
|
||||||
|
cand.reserve(k);
|
||||||
|
for (int t = 0; t < vsize; ++t) {
|
||||||
|
if (cand.size() < (size_t) k) {
|
||||||
|
cand.emplace_back(logits[t], t);
|
||||||
|
std::push_heap(cand.begin(), cand.end());
|
||||||
|
} else if (logits[t] > cand.front().first) {
|
||||||
|
std::pop_heap(cand.begin(), cand.end());
|
||||||
|
cand.back() = {logits[t], t};
|
||||||
|
std::push_heap(cand.begin(), cand.end());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
float maxv = cand.front().first;
|
||||||
|
double sum = 0;
|
||||||
|
for (auto & c : cand) { c.first = std::exp((c.first - maxv) / temperature); sum += c.first; }
|
||||||
|
std::sort(cand.begin(), cand.end(), std::greater<>());
|
||||||
|
std::uniform_real_distribution<double> dist(0.0, sum);
|
||||||
|
double target = dist(rng), acc = 0;
|
||||||
|
next = cand[0].second;
|
||||||
|
for (auto & c : cand) {
|
||||||
|
acc += c.first;
|
||||||
|
next = c.second;
|
||||||
|
if (acc / sum >= p) break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (eos >= 0 && next == eos) break;
|
||||||
|
if (llama_vocab_is_eog(e->vocab, next)) break;
|
||||||
|
|
||||||
|
int32_t len = llama_token_to_piece(e->vocab, next, piece.data(),
|
||||||
|
(int32_t) piece.size(), 0, false);
|
||||||
|
if (len > 0) {
|
||||||
|
out.append(piece.data(), (size_t) len);
|
||||||
|
// Gemma 4 closes its turn with "<turn|>" (id 106). It may not be
|
||||||
|
// registered as eog in GGUF metadata, so stop on the string too
|
||||||
|
// and strip it from the output.
|
||||||
|
if (out.size() >= 7 && out.compare(out.size() - 7, 7, "<turn|>") == 0) {
|
||||||
|
out.erase(out.size() - 7);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cur = next;
|
||||||
|
}
|
||||||
|
|
||||||
|
LOGI("generated %d chars", (int) out.size());
|
||||||
|
return env->NewStringUTF(out.c_str());
|
||||||
|
}
|
||||||
|
|
||||||
|
JNIEXPORT void JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeAbort(JNIEnv * /*env*/, jclass, jlong ptr) {
|
||||||
|
auto * e = (Engine *) (uintptr_t) ptr;
|
||||||
|
if (e != nullptr) e->abort.store(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
JNIEXPORT void JNICALL
|
||||||
|
Java_com_k2fsa_sherpa_onnx_LlmEngine_nativeFree(JNIEnv * /*env*/, jclass, jlong ptr) {
|
||||||
|
auto * e = (Engine *) (uintptr_t) ptr;
|
||||||
|
if (e == nullptr) return;
|
||||||
|
std::lock_guard<std::mutex> lock(e->mu); // wait for an in-flight generate
|
||||||
|
if (e->ctx != nullptr) llama_free(e->ctx);
|
||||||
|
if (e->model != nullptr) llama_model_free(e->model);
|
||||||
|
delete e;
|
||||||
|
LOGI("engine released");
|
||||||
|
}
|
||||||
|
|
||||||
|
} // extern "C"
|
||||||
Loading…
Reference in New Issue