淡江大學覺生紀念圖書館 (TKU Library)
進階搜尋


下載電子全文限經由淡江IP使用) 
系統識別號 U0002-0808201115042400
中文論文名稱 使用即時迴授控制與重複資料刪除機制增進廣域雲端儲存網路效能
英文論文名稱 Improving Accessing Efficiency of Cloud Storage by De-duplication and Feedback Scheme
校院名稱 淡江大學
系所名稱(中) 電機工程學系碩士班
系所名稱(英) Department of Electrical Engineering
學年度 99
學期 2
出版年 100
研究生中文姓名 林家範
研究生英文姓名 Chia Fan Lin
電子信箱 698470357@s98.tku.edu.tw
學號 698470357
學位類別 碩士
語文別 中文
口試日期 2011-07-14
論文頁數 44頁
口試委員 指導教授-吳庭育
委員-丁建文
委員-朱國志
委員-賴槿峰
委員-吳庭育
委員-李維聰
中文關鍵字 雲端儲存  分散式雜湊表  索引名稱伺服器  重複資料刪除 
英文關鍵字 Cloud Storage  DHT  INS  Deduplication 
學科別分類 學科別應用科學電機及電子
中文摘要 在雲端儲存的環境中,檔案的分派與儲存過程是由提供者自行或租用第三方的實體儲存設備,在經由中央管理並虛擬化後整合為可用的儲存資源來提供使用者其相關的存取服務,常見的儲存協定像是有Internet Small Computer System Interface (iSCSI)、Fibre Channel、Common Internet File System (CIFS)等基於區塊形式或是檔案形式來進行資料傳輸與保存。因雲端網路涵蓋了相當大的使用範圍以及網域,有時由不同使用者在儲存設備上所寫入的內容都有著高度的相似性,由於數量眾多,管理者將無法確保每一個儲存節點皆能保持最佳狀態,且當檔案數量大幅增加後,不但會造成硬體資源的浪費也會增加資料中心的控管複雜度,進一步的降低雲端儲存系統的整體效能。
有鑑於此,為了減少重複資料對系統架構所造成的負擔,本論文提出了一使用重複資料刪除以及即時迴授控制的新型資料中心架構:索引名稱伺服器 (Index Name Server, INS),其將整合了重複資料刪除以及節點最佳化等機制來提升整體雲端儲存架構的效能。
藉由INS來進行儲存節點的控管並依照客戶端的傳輸情形作最佳化的動作,INS系統可以控制每個儲存節點保持在最佳狀態下工作,並盡可能地給予客戶端符合其頻寬的節點資源供其進行傳輸的動作,如此一來不但可以有效地提升雲端儲存網路的使用效能且也能夠有效的分配並降低儲存節點的負載。
英文摘要 In a cloud storage environment, file distribution and storage is processed by storage devices providers or physical storage devices rented from the third-party companies. Through centralized management and virtualization, files are integrated into available resources for users to access. Common file storage protocols include ISCSI, Fibre Channel, CIFS and so on, which transmit or store files based on blocks or types. Moreover, because of the wide range and extensive domains of the cloud network, it is very possible that files saved by different users on the same storage device are extremely similar. Also, due to the increasing number of files, the manager cannot guarantee the optimal status of each storage node. The great number of files not only leads to the waste of hardware resources, but also worsens the control complexity of data center, which further degrades the performance of the cloud storage system.
For this reason, to decrease the workload caused by duplicated files, this paper proposes a new data management structure: Index Name Server (INS), which integrates data de-duplication with nodes optimization to enhance the performance of the cloud storage system. INS can manage and optimize the nodes according to the client-side transmission conditions. By INS, each node can be controlled to work in the best status and matched to suitable clients as possible. In such a manner, we can efficiently increase the performance of the cloud storage network and distribute the files reasonably to reduce the load of each node.
論文目次 目錄
第一章 緒論 - 1 -
1.1 前言 - 1 -
1.2 動機與目的 - 1 -
1.3 論文章節架構 - 2 -
第二章 相關背景研究 - 4 -
2.1 雲端運算 - 4 -
2.2 雲端儲存 - 7 -
2.3 相關文獻探討 - 8 -
2.4 分散式雜湊表(Distributed Hash Table, DHT) - 10 -
2.5 端對端技術(Peer-to-peer, P2P) - 12 -
2.5.1 非固定結構P2P (Unstructured P2P) - 14 -
2.5.2 固定結構P2P (Structured P2P) - 16 -
2.6 Bloom Filter - 16 -
第三章 索引名稱伺服器 - 18 -
3.1 INS基本架構 - 19 -
3.2 重複資料刪除( De-duplication ) - 21 -
3.3 INS查詢流程 - 23 -
3.4 儲存節點效能參數 - 25 -
3.5 客戶端存取參數 - 27 -
3.6 檔案分塊的多點傳輸 - 28 -
3.7 INS控制流程 - 29 -
第四章 模擬環境以及結果分析 - 31 -
4.1 模擬環境參數 - 31 -
4.2 效能模擬分析 - 32 -
第五章 結論與未來展望 - 41 -
參考文獻 - 42 -

圖目錄
圖 2. 1 雲端運算/網格運算比較示意圖 - 6 -
圖 2. 2 資源池(Resource Pools)示意圖 - 8 -
圖 2. 3傳統伺服器-客戶架構圖 - 13 -
圖 2. 4 P2P網路架構 - 14 -
圖 2. 5非固定式P2P架構示意圖 - 15 -
圖 2. 6 Bloom Filter搜尋模式示意圖 - 17 -
圖 3. 1 INS階層式架構圖 - 20 -
圖 3. 2 INS控管示意圖 - 21 -
圖 3. 3 INS環境流程圖 - 22 -
圖 3. 3 INS 環境流程圖 - 22 -
圖 3. 4 INS 傳輸流程圖 - 24 -
圖 4. 1多點傳輸與延遲機率的關係分布 - 33 -
圖 4. 2 平均傳輸延遲 (資料重複率0%) - 34 -
圖 4. 3 平均負載率 - 35 -
圖 4. 4 因未依客戶端頻寬進行頻寬調整而造成的頻寬浪費率 - 36 -
圖 4. 5 在不同資料重複率下對儲存節點所造成的平均負載值 - 38 -
圖 4. 6資料重複率40%下所造成的平均延遲時間 - 39 -
圖 4. 7資料重複率70%下所造成的平均延遲時間 - 39 -
圖 4. 8 資料重複率40%下所造成的平均負載 - 40 -
圖 4. 9資料重複率70%下所造成的平均負載 - 40 -

表目錄
表 4. 1 模擬參數表 - 31 -
表 4. 2 檔案重複率與其對應之最大負載客戶端數量 - 37 -


參考文獻 [1] Distinguishing Cloud Computing from Utility Computing, http://www.ebizq.net/blogs/saasweek/2008/03/distinguishing_cloud_computing/, June 2011
[2] Gartner Say's Cloud Computing Will Be As Influential As E-business, http://www.gartner.com/it/page.jsp?id=707508, June 2011
[3] What cloud computing really means,http://www.infoworld.com/d/cloud-computing/what-cloud-computing-really-means-031, June 2011
[4] Show Me the Gateway, http://gigaom.com/2010/06/22/show-me-the-gateway-taking-storage-to-the-cloud/, June 2011
[5] Jiyi Wu; Lingdi Ping; Xiaoping Ge; Ya Wang; Jianqing Fu; "Cloud Storage as the Infrastructure of Cloud Computing",inProc. Intelligent Computing and Cognitive Informatics (ICICCI), 2010, pp. 380 – 383.
[6] Ian Foster Argonne,What is the Grid? A Three Point Checklist, National Laboratory & University of Chicago, 2002.
[7] Jiyi Wu; Lingdi Ping; Xiaoping Ge; Ya Wang; JianqingFu,"Cloud Storage as the Infrastructure of Cloud Computing", in Proc.Intelligent Computing and Cognitive Informatics (ICICCI), Malaysi, 2010, pp. 380-383.
[8] YanmeiHuo; Hongyuan Wang; Liang Hu; Hongji Yang; "A Cloud Storage Architecture Model for Data-Intensive Applications", in Proc.Computer and Management (CAMAN), 2011, pp. 1-4.
[9] Microsoft SMB Protocol and CIFS Protocol Overview, MSDN, http://msdn.microsoft.com/en-us/library/aa365233, June 2011
[10] Direct hosting of SMB over TCP/IP. Microsoft, http://support.microsoft.com/kb/204279, June 2011
[11] Costa, L.B.; Ripeanu, M.; “Towards automating the configuration of a distributed storage system”, in Proc.Grid Computing (GRID), 2010, pp. 201-208.
[12] Ohsaki, H.; Watanabe, S.; Imase, M.;“On dynamic resource management mechanism using control theoretic approach for wide-area grid computing”, in Proc. Control Applications, 2005, pp. 891-897.
[13] Dezhi Han; Fu Feng; “Research on Self-Adaptive Distributed Storage System”, in Proc.Wireless Communications, Networking and Mobile Computing (WiCOM '08.), 2008.
[14] Jianzong Wang; Varman, P.; ChangshengXie; “Avoiding performance fluctuation in cloud storage”, in Proc.High Performance Computing (HiPC),2010.
[15] A Survey of DHT Security Techniques http://www.cs.vu.nl/~steen/papers/2009.acm-cs.pdf, June 2011
[16] Qiang Li; QinfenHao; Limin Xiao; Zhoujun Li; “Adaptive Management of Virtualized Resources in Cloud Computing Using Feedback Control” , in Proc.Information Science and Engineering (ICISE), 2009, pp. 99-102.
[17] A Distributed Hash Table
http://pdos.csail.mit.edu/papers/fdabek-phd-thesis.pdf, June 2011
[18] Nasri, M.; Sharifi, M.; " Load Balancing using Consistent Hashing: a Real Challenge for Large Scale Distributed Web Crawlers",Advanced Information Networking and Applications Workshops (WAINA '09),2009, pp. 715 – 720.
[19] Kan Zhang; Antonopoulos, N.; ZaighamMahmood; “A Review of Incentive Mechanism in Peer-to-Peer Systems”, Advances in P2P Systems(AP2PS '09),2009, pp. 45-50.
[20] Javed I. Khan and Adam Wierzbicki,“Foundation of Peer-to-Peer Computing”, Elsevier Journal of Computer Communication, Volume 31, Issue 2, Feb. 2008.
[21] John F. Buford, Heather Yu, EngKeongLua“P2P Networking and Applications.” ISBN 30-12374-214-5, Morgan Kaufmann, Dec. 2008.
[22] Deke Guo; Jie Wu; Honghui Chen; Ye Yuan; XueshanLuo; “The Dynamic Bloom Filters”, Knowledge and Data Engineering, Volume 22 , Issue 1, 2010, pp. 120-133.
[23] Bruck, J.; JieGao; Anxiao Jiang; “ Weighted Bloom Filter”, Information Theory, 2006
[24] Xin Sun, KanLi,Yushu Liu, "An Efficient Replica Location Method in Hierarchical P2P Networks", in Proc.Computer and Information Science(ICIS 2009), 2009, pp. 769-774.
[25] He Huang; Liqiang Wang; “P&P: A Combined Push-Pull Model for Resource Monitoring in Cloud Computing Environment”, Cloud Computing (CLOUD), 2010, pp. 260 - 267.
[26] Wenzheng Li; Hongyan Shi; "Dynamic Load Balancing Algorithm Based on FCFS", in Proc.Innovative Computing, Information and Control (ICICIC), 2009, pp. 1528 – 1531.
[27] Dinerstein, J.; Dinerstein, S.; Egbert, P.K.; Clyde, S.W.; “Learning-Based Fusion for Data Deduplication”, in Proc.Machine Learning and Applications ICMLA '08, 2008, pp. 66-71.
[28] Bloom Filter,
http://en.wikipedia.org/wiki/Bloom_filter, June 2011
論文使用權限
  • 同意紙本無償授權給館內讀者為學術之目的重製使用,於2013-08-10公開。
  • 同意授權瀏覽/列印電子全文服務,於2013-08-10起公開。


  • 若您有任何疑問,請與我們聯絡!
    圖書館: 請來電 (02)2621-5656 轉 2281 或 來信