隨著企業會議日益頻繁及遠距協作的普及，會議錄音成為重要的知識資產。然而，這些錄音資料多半為非結構化的口語內容，對於未參與會議者來說，理解全貌極具挑戰性；即便是與會者，也難以迅速掌握討論重點。傳統仰賴人工整理與撰寫摘要的方式，不僅耗費時間，亦容易因疏漏或主觀判斷而影響資訊準確性。因此，建立一套自動化、具備語意理解能力的會議記錄摘要系統，成為提升企業資訊管理效率的重要課題。
本研究提出一套融合語音轉文字與檢索增強生成（Retrieval-Augmented Generation, RAG）架構的多階段處理流程，以自動化生成可讀性與精確度兼具的會議摘要。整體系統共分為四個主要階段：第一階段使用 Whisper 模型執行語音辨識，將會議錄音轉換為初步文字稿；第二階段應用 Pyannote.audio 進行語者分離與句子切分，建立具語者標記的結構化語料；第三階段以對比學習方式建構語義向量表示，並透過 FAISS 建立知識檢索庫，加強語句間的語意對齊能力；第四階段則整合 RAG 模型與 Self-Instruct 技術，進行語境感知式的摘要生成，強化多語者、多主題會議內容的整合表現。
本研究在中文語境下實作語音轉文字與強化型 RAG 的整合流程，針對企業實際應用場景設計資料擴增與測試機制，有效提升會議記錄的摘要品質與應用價值。實驗結果顯示，本系統在自建企業語料上的摘要準確度優於傳統提取式方法，整體提升幅度達 27.45%，展現出高度可行性，並能顯著減輕人工會議整理負擔。

With the increasing frequency of corporate meetings and the widespread adoption of remote collaboration, meeting recordings have become valuable knowledge assets. However, these recordings are typically unstructured and conversational in nature, making it challenging for those who did not attend the meeting to comprehend the full context. Even for participants, quickly identifying key discussion points can be difficult. Traditional manual transcription and summarization approaches are time-consuming and prone to errors or subjective bias, which may compromise the accuracy of the information. Therefore, developing an automated meeting summarization system with semantic understanding capabilities has become a critical task for improving enterprise information management efficiency.
This study proposes a multi-stage processing framework that integrates Speech-to-Text technology with a Retrieval-Augmented Generation (RAG) architecture to automatically generate meeting summaries with high readability and precision. The system consists of four major stages:
Speech Recognition: Utilizing the Whisper model to transcribe meeting audio into preliminary text.
Speaker Diarization and Segmentation: Applying Pyannote.audio to separate speakers and segment sentences, producing structured transcripts with speaker labels.
Semantic Vector Construction and Retrieval: Employing contrastive learning to create sentence embeddings and using FAISS to build a semantic retrieval index that enhances contextual alignment between statements.
Context-Aware Summarization: Integrating the RAG model with Self-Instruct techniques to generate coherent summaries that effectively consolidate multi-speaker and multi-topic meeting content.
The system is implemented for Chinese-language environments, with data augmentation and evaluation mechanisms tailored for real-world enterprise use cases. Experimental results demonstrate that the proposed method significantly outperforms traditional extractive approaches, achieving a 27.45% improvement in summarization accuracy on in-house corporate corpora. The findings highlight the system's high feasibility and its potential to substantially reduce the manual burden of meeting documentation.

目錄
 
誌謝	I
目錄	VI
圖目錄	VIII
表目錄	X
第一章 簡介	1
第二章 相關研究	5
2.1	語音轉文字與語者辨識技術發展	5
2.2	 檢索增強生成模型（RAG）與其應用	7
2.3	多語者摘要與對話總結技術	9
第三章 系統架構與背景知識	12
3.1	系統總體架構說明	12
3.2	Whisper 語音轉文字模型	13
3.3	Pyannote 語者分段原理	16
3.4	向量化技術與 FAISS 检索機制	19
3.5	RAG 模型與 Self-Instruct 調整策略	21
第四章 系統實作	25
4.1	音訊處理與轉錄模組設計	25
4.2	多語者結構化文本生成	27
4.3	 向量資料庫建構與檢索優化	31
4.4	RAG 摘要生成模組與參數調校	34
4.5	結果自動化輸出格式與接口	37
第五章 實驗分析	41
5.1	實驗資料來源與語料特性	41
5.2	實驗環境與評估指標（ROUGE、BLEU）	45
5.3	系統效能測試結果與比較	47
5.4	結果分析與討論	52
第六章 結論	56
參考文獻	57

圖目錄
圖一、會議頻繁與紀錄人力負擔示意圖	1
圖二、RAG 問答系統於會議資料中的應用流程圖	3
圖三、結合多模組 AI 技術之問答流程圖	4
圖四、語音轉文字處理流程示意圖	6
圖五、語者辨識技術示意圖	6
圖六、系統架構流程圖	12
圖七、語音處理與分段模組架構	14
圖八、語者特徵提取與分類流程圖	16
圖九、語者分段結果範例	18
圖十、向量檢索與摘要生成流程	22
圖十一、語者標記流程	30
圖十二、語料分布圖表	44
圖十三、FAISS 檢索結果可視化（向量分布）	45
圖十四、模型效能比較折線圖	51


表目錄
表一、系統模組與對應程式檔案表	25
表二、語意檢索功能與參數設定說明表	33
表三、摘要生成流程主要模組與功能說明表	34
表四、語意摘要生成參數設定與功能說明表	36
表五、系統輸出與任務記錄檔案格式說明表	38
表六、系統 CLI 參數設定與範例對照表	40
表七、使用語料與資料集介紹表	42
表八、語料特性與分佈統計表	43
表九、系統實驗環境與套件版本	46
表十、系統所用模型與函式庫版本對照表	46
表十一、Whisper 模組參數設定表	46
表十二、Pyannote 語者辨識模組參數設定表	47
表十三、語意向量檢索參數設定表	47
表十四、摘要生成與 RAG 設定參數表	47
表十五、語音轉文字準確率分析表	48
表十六、多語者語音轉錄準確率統計表	49
表十七、各組參數設定與摘要品質比較表	50
表十八、不同摘要模型比較（TF-IDF、T5-small、本研究法）	51
表十九、系統效能評估（速度與準確度）	52

參考文獻
[1] A. Radford, et al., “Robust Speech Recognition via Large-Scale Weak Supervision (Whisper),” OpenAI, 2022. [Online]. Available: https://openai.com/research/whisper

[2] H. Bredin, et al., “pyannote.audio: Neural Building Blocks for Speaker Diarization,” in *Proc. Interspeech*, 2020, pp. 712–716.

[3] H. Zhang, et al., “Improving Dialogue Summarization via Joint Learning with Discourse and Semantic Structures,” in *Findings of ACL*, 2021, pp. 474–485.

[4] M. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with GPUs,” *IEEE Trans. Big Data*, vol. 7, no. 3, pp. 535–547, 2019.

[5] P. Lewis, et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in *NeurIPS*, vol. 33, 2020.

[6] OpenAI, “GPT-3.5/4 Language Models and Chat API,” OpenAI Platform, 2023. [Online]. Available: https://platform.openai.com/docs

[7] LangChain Team, “LangChain: Building LLM-powered Applications,” 2023. [Online]. Available: https://docs.langchain.com

[8] T. Wolf, et al., “Transformers: State-of-the-Art Natural Language Processing,” in *Proc. EMNLP: System Demonstrations*, 2020, pp. 38–45.

[9] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in *Proc. EMNLP*, 2019, pp. 3982–3992.

[10] K. Kinoshita, et al., “Streaming End-to-End Speech Recognition for Real-Time Applications,” *IEEE J. Sel. Topics Signal Process.*, vol. 15, no. 4, pp. 1032–1046, 2021.

[11] Beijing Academy of Artificial Intelligence (BAAI), “BGE-Large-ZH: Chinese Embedding Model,” HuggingFace, 2023. [Online]. Available: https://huggingface.co/BAAI/bge-large-zh

[12] LangChain Team, “RetrievalQA Chain Documentation,” 2023. [Online]. Available: https://docs.langchain.com/docs/modules/chains/popular/retrieval-qa