本論文『基於SoPC之交通標誌偵測與辨識』之設計方式為利用多主從系統架構設計為基礎，在FPGA晶片內設計多個主控端硬體加速模組搭配Nios II處理器來實現。整個系統架構規劃為三大部份:(1)交通標誌偵測(Traffic Sign Recognition) 、(2)交通標誌辨識(Traffic Sign Recognition)及(3)動態影像擷取。主要功能為利用CMOS影像感測器偵測交通標誌，透過線上訓練類神經網路晶片(On-line Training of Neural Network Chip)將擷取之標誌分類，確定時速條件且判斷車輛是否違反標誌限制，使得修正車輛行駛方向或採取減速保持安全，避免意外事故發生。本系統擬使用軟硬體協同設計的方法，並且以較低成本開發較高級複雜演算法及較高效能的方式來設計實現駕駛輔助系統。類神經網路(Neural Network)在影像分類上有很好的表現，並且數學模型容易用硬體實現，希望突顯硬體加速器的優勢，發展出可線上學習的類神經電路，實現於Altera FPGA 開發版。如此一來，不但可以提升訓練與辨識效能，有效降低學習時間，再搭配即時動態影像，使得系統盡量滿足駕駛輔助系統在影像上的應用必須要即時(Real-Time)的特點。並且從實驗結果可以驗證所提之軟硬體共同設計確實可以用較少的影像處理時間達到即時的訓練與辨識。

In this thesis, an intelligent traffic sign detection and recognition system is designed and implemented on an FPGA platform based on a multiple master-slave system architecture to achieve the goals of traffic sign detection and recognition. The whole system is divided into three parts: (1) dynamic image capture, (2) traffic sign detection, and (3) traffic sign recognition. From the images captured by the CMOS image sensor, the traffic sign in the obtained image is detected through a series of image processing steps. The resultant image is further classified by a multilayer neural network with the backpropagation learning algorithm. The results of the neural network based classifier can be used to regulate the speed of car for safety and command certain actions to avoid accidents. The IDAS is realized by taking the software/hardware co-design approach which not only saves cost but also increases efficiency. To fully take advantages of hardware accelerators and satisfy the need of real-time image recognition, a very large   integrated circuit (VLSI) is designed specifically for the on-line training of neural networks. Our experimental results show that hardware/software co-design approach can effectively reduce the learning time of neural networks.

中文摘要............................................................................................................I
英文摘要...........................................................................................................II 目錄.................................................................................................................IV
圖目錄............................................................................................................VII
表目錄............................................................................................................XII
第一章　緒論............................................................................…...................1
	1.1  研究動機….......................................................................................1
	1.2  文獻回顧...........................................................................................2
1.3  論文章節組織...................................................................................4
第二章  影像偵測與辨識理論…..……..……..….........................…............5
2.1  背景知識...........................................................................................5
2.1.1  影像處理技術......................................................................5
2.1.2  霍夫轉換…..……………………......................................17
2.1.3  類神經網路………………………....................................21 
第三章  交通標誌偵測與辨識…………....................................................41
 3.1  前言……….…................................................................................41
	3.2  交通標誌偵測……….....................................................................44	3.3  交通標誌辨識.................................................................................47
第四章　軟硬體協同設計平台.....................................................................51
	4.1  前言.................................................................................................51
	4.2  CMOS擷取模組介紹…………………………….........................55
4.2.1  CMOS擷取模組特性………..............................................55
4.2.2  CMOS擷取模組內部電路圖…..........................................58
4.2.3  CMOS擷取模組內部功能說明..........................................59
第五章　系統整合之軟硬體協同設計……….............................................67
5.1  前言.................................................................................................67
	5.2  多主從系統架構介紹…….………………………........................71
5.3  Master端與Slave端設計介紹……………….….........................73
5.3.1  Slave端設計……….……...................................................74
5.3.2  Master端設計….………..…...............................................78
    5.4  使用者自訂介面介紹……..……..……….……...........................82
第六章　線上學習類神經硬體加速器………….........................................88
6.1  前言……….…................................................................................88
6.2  離線訓練類神經網路.....................................................................88 
6.3  線上訓練類神經網路........................................................................91 
6.4  實現結果……………......................................................................105
第七章　結論與未來研究方向...................................................................110
參考文獻.......................................................................................................112
得獎經歷.......................................................................................................117

























圖目錄

圖2.1影像處理基本流程.................................................................................6
圖2.2 RGB色彩模型示意圖............................................................................8
圖2.3 Sobel演算法 與 之乘積遮罩...................................................11
圖2.4 Sobel演算法之虛乘積矩陣 .............................................................12
圖2.5 Sobel演算法之影像邊緣偵測.............................................................13
圖2.6連續高斯分佈圖形( =1.4)................................................................14
圖2.7高斯分佈之離散逼近矩陣( =1.4)....................................................15
圖2.8 Roberts 與 交叉乘積遮罩………………………….................15
圖2.9 Canny演算法之影像輪廓邊緣偵測................................................... 16
圖2.10(a)原影像空間上各點…………………….........................................19
圖2.10(b)參數空間各直線.............................................................................19
圖2.11霍夫轉換圓方程式示意圖.................................................................21
圖2.12生物神經細胞......................................................................................23
圖2.13單輸入神經元.....................................................................................25
圖2.14(a)硬極限轉移函數............................................................................ 26
圖2.14(b)單輸入hardlim神經元……….…………………………………..26
圖2.15(a)線性轉移函數…………………………………………………….26
圖2.15(b)單輸入purelin神經元……………………………………………26
圖2.16(a)Log-Sigmoid轉移函數…………………………………………...27
圖2.16(b)單輸入logsig神經元……………………………………………..27
圖2.17多輸入神經元……………………………………………………….28
圖2.18具有 個輸入的神經元簡化符號………………………………….29
圖2.19 個神經元組成的層………………………………………………..30
圖2.20由 個神經元組成的層的簡化表示……………………………….31
圖2.21三層網路的簡化表示……………………………………………….32
圖3.1駕駛輔助系統特色…………………………………………………...41
圖3.2交通標誌辨識系統流程……………………………………………...42
圖3.3速限交通標誌………………………………………………………...43
圖3.4系統規劃圖…………………………………………………………...44
圖3.5交通標誌偵測概念圖………………………………………………...44
圖3.6軟體端模擬流程圖…………………………………………………...45
圖3.7軟體端模擬步驟二示意圖………………………………………...…46
圖3.8軟體端模擬步驟三示意圖………………….………………………..47
圖3.9類神經網路辨識示意圖……………………………………………...48
圖3.10類神經網路訓練收斂圖…………………………………………….50
圖4.1DE2開發板實體圖……………………………………….…………...54
圖4.2DE2開發板搭配DVD播放器與LCM顯示模組平台……………...54
圖4.3DE2開發板搭配數位相機與LCM顯示模組平台……….………….54
圖4.4CMOS擷取模組之實體圖…………….……………………………...55
圖4.5CMOS擷取模組之接腳訊號圖……………….……………………...58
圖4.6 CMOS擷取模組內部功能方塊...........................................................59
圖4.7CMOS擷取模組影像框架...................................................................60
圖4.8CMOS擷取模組有效資料格式............................................................61
圖4.9CMOS擷取模組影像訊號....................................................................62
圖4.10Bayer Pattern像素...............................................................................63
圖4.11CMOS擷取模組像素資料時序圖(1).................................................63
圖4.12CMOS擷取模組像素資料時序圖(2).................................................64
圖4.13 界面16位讀出順序......................................................................65
圖4.14 界面16位寫入順序......................................................................66
圖5.1傳統的軟硬體共同設計流程...............................................................68
圖5.2 SoPC系統的軟硬體共同設計架.........................................................69
圖5.3 Nios II處理器標準硬體週邊功能方塊圖...........................................70
圖5.4傳統匯流排的傳輸方式.......................................................................71
圖5.5 Avalon Bus的傳輸方式........................................................................71
圖5.6 Slave讀取模式時序圖.........................................................................75
圖5.7 Slave端寫入模式時序圖.....................................................................76
圖5.8多個等待週期的讀取模式...................................................................77
圖5.9多個等待週期的寫入模式...................................................................77
圖5.10 Master端讀取模式時序圖.................................................................79
圖5.11 Master端寫入模式時序圖.................................................................79
圖5.12 Maste端有等待模式讀取時序圖......................................................81
圖5.13 Master端有等待模式寫入時序圖.....................................................81
圖5.14整合使用者自定介面至Avalon Bus內示意圖.................................84
圖5.15 SoPC Builder使用者介面..................................................................84
圖5.16新增使用者自訂介面程式.................................................................85
圖5.17連接使用者自訂介面訊號.................................................................85
圖5.18 Component Editor介面......................................................................86
圖5.19 Avalon Bus 連接圖............................................................................87
圖6.1多輸入神經元數學模型…………………………….………..............90
圖6.2多輸入神經元硬體模型.......................................................................90
圖6.3硬體訓練流程圖...................................................................................92
圖6.4XOR問題學習收斂圖...........................................................................94
圖6.5初始化硬體方塊圖...............................................................................95
圖6.6 (a)前向傳遞硬體方塊圖......................................................................97
圖6.6 (b)前向傳遞硬體方塊圖......................................................................97
圖6.7靈敏度倒傳遞硬體方塊圖...................................................................99
圖6.8動量方法硬體方塊圖.........................................................................100
圖6.9(a)驗証步伐硬體方塊圖.....................................................................102
圖6.9(b)驗証步伐硬體方塊圖.....................................................................102
圖6.10更新步驟硬體方塊圖.......................................................................104
圖6.11 Master_initial_Weight模組符號圖..................................................105
圖6.12 Master_initial_Weight實際訊號圖..................................................106
圖6.13為Master_Weight_Reader_First模組符號圖..................................106
圖6.14為Master_Weight_Reader_First實際訊號圖...................................107
圖6.15為Master_Weight_Reader_Second模組符號圖..............................108
圖6.16為Master_Weight_Reader_Second實際訊號圖..............................108



表目錄

表3.1類神經網路訓練參數表.......................................................................49
表5.1 Slave端常用訊號表.............................................................................74
表5.2 Master端常用訊號表...........................................................................78
表6.1資料表示法參數表...............................................................................91
表6.2類神經網路訓練參數表.......................................................................93
表6.3軟體模擬與硬體加速比較表.............................................................109

[1]R. Janssen, W. Ritter, and F. Stein, and S. Ott, “Hybrid approach for traffic sign recognitions,” Proceeding of the Intelligent Vehicles 1993 Symposium, pp. 390-395, 1993.
[2]A. D. L. Escalrea, L. E. Moreno, M. A. Salichs, and J. M. Armingol, “Road traffic sign detection and classification,” IEEE Transactions on Industrial Electronics, vol. 44, No. 6, pp. 848-859,1997.
[3]K. Rangaraian, M. Shah, and D. V. Brackle, “Optimal corner detector,” Computer Vision, Graphics, and Image processing, vol. 48, pp. 230-245, 1989.
[4]K. Hirose, T. Askura, and Y. Aoyagi, “Real-time recognition of road traffic sign in moving scene image using new image filter,” Proceedings of the 26th Annual Conference of the IEEE Industrial Electronics Society, vol. 3, pp. 2207-2212.2000.
[5]R. G. Gonzalez and R. E. Woods, “Digital Image Processing,” 2nd ed., Pearson, 2003.
[6]G. Piccioli, E. D. Michel, P. Parodi, and M. Campani. “Robust method for road sign detection and recognition,” Image and Vision Computing, vol. 14, pp. 209-223, 1996.
[7]S.H. Hsu and C. L. Huang, “Road sign detection and recognition using matching pursuit method,” Image and Vision Computing, vol. 19, pp. 119-129, 2001.
[8]R. P. Palmer and P. A. Rounce, “An architecture for application specific neural network processors,” Computing & Control Engineering Journal, vol. 5, no. 6, pp. 260-264, Dec. 1994.
[9]John F. Canny. “A computational approach to edge detection,” IEEE Transaction on Pattern Analysis and Machine Intelligence, 8(6): pp. 679-698, 1986.
[10]Maldonado-Basćon, S. Lafuente-Arroyo, P. Gil-Jimenez, H. Gomez-Moreno, and F. Lopez-Ferreras,  “Road-sign detection and recognition based on support vector machines,” IEEE Trans. Intelligent Transportation System, vol. 8, no.2, pp. 264-278, June 2007.
[11]Y. J. Zheng, W. Ritter, and R. Jamssen, “An adaptive system for traffic sign recognition,” Proc. Intelligent Vehicles 94 Symposium, Paris, France, Oct. 24-26, 1994, pp. 165-170.
[12]Q. Wang, B. Yi, Y. Xie, and B. Liu, “The hardware structure design of perceptron with FPGA implementation,” Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics, vol. 1, pp. 762-767, Oct. 2003.
[13]M. Porrmann, U. Witkowski, H. Kalte, and U. Ruckert, “Implementation of artificial neural networks on a reconfigurable hardware accelerator,” Proc. 10th Euromicro Workshop on parallel, Distributed and Network-based Processing, pp. 243-250, Jan. 2002.
[14]H. Faiedh, Z. Gafsi, K. Torki, and K. Besbes, “Digital hardware implementation of a neural network used for classification,” Proc. of the 16th IEEE Int. Conf. on Microelectronics, pp. 551–554, Dec. 2004.
[15]H. F. Restrepo, R. Hoffmann, A. Perez-Uribe, C. Teuscher, and E. Sanchez, “A network FPGA-based hardware implementation of a neural network application,” IEEE Symposium on Field-Programmable Custom Computing Machines, pp. 337-338, Apr. 2000. 
[16]R. Vicen-Bueno, R. Gil-Pita, M.P. Jarabo-Amores and F. Lopez-Ferrersa,   “Complexity Reduction in Neural Networks Applied to Traffic Sign Recognition,” Proceedings of the 13th European Signal Processing Conference, Antalya, Turkey, September 4-8, 2005.
[17]R. Vicen-Bueno, R. Gil-Pita, M. Rosa-Zurera. M. Utrilla-Manso, and Lopez-Ferreras, “Multilayer Perceptrons Applied to Traffic Sign Recognition Task,” LNCS 3512, IWANN 2005, J. Cabestany, A. Prieto, and D.F. Sandoval(Eds.), Springer-Verlag, Berlin, Heidelberg, 2005, pp. 865-872.
[18]H. X. Liu, and B. Ran, “Vision-Based Stop Sign Detection and Recognition System for Intelligent Vehicle,” Transportation Research Board (TRB) Annual Meeting 2001, Washington, D.C., USA, January 7-11, 2001.
[19]H. Fleyeh, and M. Dougherty, “Road And Traffic Sign Detection And Recognition,” Proceedings of the 16th Mini-EURO Conference and 10th Meeting of EWGT, pp. 644-653.
[20]D. S. Kang, N. C. Griswold, and N. Kehtarnavaz, “An Invariant Traffic Sign Recognition System Based on Sequential Color Processing and Geometrical Transformation,” Proceedings of the IEEE Southwest Symposium on Image Analysis and Interpretation, pp. 88-93, Apr. 1994.
[21]C. Bahlmann, Y. Zhu, V. Ramesh, M Pellkofer, T. Koehler, “A System for traffic Sign Detection, Tracking, and Recognition Using Color, Shape, and Motion Information,” Proceedings of the 2005 IEEE Intelligent Vehicles Symposium, Las Vegas, USA, Jun. 6-8, 2005.
[22]L. Pacheco, J. Batlle, and X. Cufi, “A new approach to real time traffic sign recognition based on color information,” Proceedings of the Intelligent Vehicles Symposium, Paris, pp. 339-344, 1994. 
[23]Y. Aoyagi, and T. Asakura, “A study on traffic sign recognition in scene image using genetic algorithms and neural networks,” Proceedings of the IEEE IECON Int. Conf. on Industrial Electronics, Control, and Instrumentation, Taipei, Taiwan, vol. 3, pp. 1838-1843.
[24]T. P.  Vogl, J. K. Mangis, A. K. Zigler, W.T. Zink and D. L. Alkon,  “Acceleratong the convergence of the backpropagation method,” Biological Cybernetics, vol. 59, 99. 256-264, Sep. 1988.
[25]S. Kumar, “Neural Network: A Classroom Approach,” Mc Graw Hill 2005.
[26]友晶科技網址，URL:http://www.terasic.com.tw/
[27]J. O. Hamblen, and Toker, “Using and FPGA-base SOC approach for senior design projects,” IEEE International Conference on Microelectronic System Education , pp. 18.
[28]Altera Corporation, NIOS Embedded Processor User’s Guide, http://www.altera.com/products/devices/Nios/nio-index.html SOC.
[29]Altera Corporation, Nios 3.0 CPU Data Sheet, 2003.
[30]Altera Corporation, Nios Embedded Processor Software Development Reference Manual, 2003.
[31]Altera Corporation, Nios Embedded Processor Hardware Development Reference Manual, 2003.
[32]Altera Corporation, SOPC Builder User Guide, 2003.
[33]潘松、黃繼業、曾毓，SOPC設計實務，全華出版社，May 2006。
[34]蕭如宣，SOPC系統設計，儒林出版社，Jun. 2003。
[35]Altera Corporation, Avalon Memory-Mapped Interface Specification, November 2006。
[36]Altera Corporation, Avalon Interface Specifications, March 2008。