在此論文中，對於參考背景的建立與維持提出了可靠且更準確的方法。背景的切割對於影像監視系統與相關的應用是很重要的，因為要辨識出目標﹙前景﹚之前，必須先知道在場景中哪些是前景而哪些是屬於背景。因此第一個步驟是替被偵測的場景建立一個參考背景模型，如此才能根據此參考背景擷取出前景。
  在現有的文獻中，在參考背景中的每一個像素都只有一個真實背景的參考值，而在此論文中所提出的multiple reference value background model (MRV background model)中的每一個像素卻有多個參考值，因此即使是再複雜或是紊亂的場景也能被正確地建立參考背景。
  背景切割中的另一個重要的步驟是背景的更新。因為被偵測的環境會一直在改變，例如移動的雲的影子或建築物的影子。因此參考背景也必須被修改以反映這些變化。否則，錯誤的前景就會因此產生。然而，這些會影響場景的因素卻很少被討論。在此論文中，全域更新與區域更新兩種策略一起被用來處理背景的更新；分別負責全部與部分的參考背景修改。因此MRV背景參考模型比其他的背景模型能正常運作更長的時間。此外，影響正確前景切割的因素也被詳細地探討。
  從實驗得知，MRV背景參考模型比其他的背景模型能得到更精確的參考背景與更詳細的前景細節。除此之外，由於可靠的背景更新策略，使得系統不僅能在白天與晚上正常運作，對於攝影鏡頭與場景中背景物體的晃動也有很好的抑制能力。

In this thesis, we proposed a reliable and precise method for building and maintaining the reference background of a detected environment. Background segmentation (sub-traction) plays an important role in video surveillance systems and related applications. In order to extract the specific targets, applications must recognize what are objects (foreground) and what are not. Therefore, the fist step in such systems is usually to build a reference background model for the detected scene, and then the foreground can be extracted by comparing with the reference background model.
  In the existing literature, each pixel of a reference background model has only one reference value to the real background of the detected scene. However, each pixel in the proposed multiple reference value (MRV) background model may have multiple reference values. Thus, even in complex or disorder scenes, reference backgrounds can also be correctly built.
  Updating reference background is another important step for background segmen-tation. Because the detected scene will be changed, such as moving shadows of clouds or buildings, the reference background model must be modified to reflect these varia-tions. Otherwise, such applications will result in erroneous foreground segmentations. However, the situations of causing erroneous foreground segmentations are seldom discussed.
  In this thesis, a global update and a local update methods are employed as the strategies for a reference background update; they control the entire and partial modi-fication of a reference background model, respectively. Therefore, the reference back-ground model can be used more robust than other proposed models for a long period of surveillance. In addition, the situations of causing erroneous foreground segmentations are also discussed in details.
  By experimental results, the proposed method can obtain a more precise reference background model and preserves more details of a segmented foreground. Moreover, because of the reliable update strategies, the system can operate normally at daytime and nighttime. In addition, the system also can resist camera and object shaking.

List of Figures	III
List of Tables	V
Chapter 1 Introduction
1.1 Motivation	1
1.2 Overview	2
1.3 Organization of the thesis	3
Chapter 2 Literature Reviews	5
2.1 Background Segmentation	5
2.2 Reference Background Models	6
2.2.1 Pure Background Model	8
2.2.2 Average Background Model	9
2.2.3 Gaussian Background Model	9
2.2.4 Mixture Gaussian Background Model	12
2.2.5 Histogram Background Model	15
2.2.6 Other Approaches	17
2.3 Update Strategies	19
2.4 Situation for Erroneous Segmentation	21
2.5 Emphases of Background Segmentation	23
Chapter 3 Preliminaries	26
3.1 Color Model	26
3.1.1 RGB Color Model	27
3.1.2 HSI Color Mode	28
3.2 Image Segmentation	29
3.3 Connected Component	31
3.4 Morphology	32
3.5 Sorting	34
Chapter 4 Background Segmentation	35
4.1 Introduction	35
4.2 Building a Reference Background	38
4.2.1 Sampling	38
4.2.2 Choosing Candidates	41
4.2.3 Deciding a Reference Background	43
4.2.4 Convergence and Non-convergence	44
4-3. Non-convergence Processing	45
4.3.1 Re-sampling	46
4.3.2 Parameters Adjusting	47
4.3.3 Oscillation Problem	49
4.4 Transferring Background and Foreground Segmentation	53
Chapter 5 Background Update	58
5.1 Introduction	58
5.2 Occasions of Update	59
5.2.1 Un-update Situations	59
5.2.2 Priority of Update	61
5.3 Global Update	62
5.4 Local Update	64
5.5 Dealing with Situations of Causing Erroneous Segmentation	68
5.5.1 Many Objects Passing	69
5.5.2 Big Objects Passing	71
5.5.3 Jam of Objects	73
5.5.4 Stopping / Leaving Objects	74
5.5.5 Slowly Moving Objects	79
5.5.6 Illumination Variations	81
Chapter 6 Experimental Results	83
6.1 Test Environment	83
6.2 Results	85
Chapter 7 Conclusions and Future Works	92
7.1 Conclusions	92
7.2 Future Works   	92
Reference	    95


List of Figures

1.1 Foreground segmentation	2
2.1 Flow chart of building of reference background model	5
2.2 Foreground segmentation by a pure reference background	7
2.3 Build a reference background dynamically	8
2.4 A case of the pure background model	8
2.5 The different intensity distributions	10
2.6 Foreground segmentation results by a Gaussian background model	12
2.7 A mixture Gaussian distribution approximation of intensity distribution	12
2.8 Foreground segmentation results by a mixture Gaussian background model	14
2.9 Intensity diagram	15
2.10 A case of modified histogram	16
2.11 Object contours extraction by thermal image	19
2.12 Pixel states migration	21
2.13 Discrimination between background and shadow on the RGB color model	23
3.1 RGB color model	27
3.2 HSI color model	28
3.3 Image segmentation	30
3.4 Connected components	32
3.5 N-connected	32
3.6 Opening operation	33
4.1 Flow chart of complete reference background segmentation	37
4.2 Flow char of building reference background	38
4.3 Flow chart of sampling	39
4.4 The recorded colors of pixel and each color’s count	42
4.5 Flow chart of non-convergence process	45
4.6 The homogeneous situation	49
4.7 Oscillation phenomenon	51
4.8 The part of number of recorded colors	52
4.9 The part of reference number	52
4.10 Three results of foreground segmentation of different video clips	54
4.11 Sketch of transferring a reference background	55
4.12 Three results of noise removal from three different video clips	56
4.13 Three results of opening	57
5.1 The erroneous foreground segmentation	59
5.2 The erroneous update occasion	60
5.3 The erroneous foreground segmentation by un-fixed exposure	60
5.4 Flow chart of global update	63
5.5 The foreground segmentation by unstable background model	65
5.6 The erroneous foreground segmentation by slowly moving object	65
5.7 Erroneous foreground segmentation of stop objects	66
5.8 Erosion situation of foreground	67
5.9 Flow chart of local update	68
5.10 The comparison of histograms	71
5.11 The changing of intensity histogram of big object passing	72
5.12 Un-sufficient exposure case	73
5.13 Erroneous foreground segmentation causes by static object	75
5.14 Erroneous foreground segmentation causes by leaving object	77
5.15 The static mask table	78
5.16 Flow chart of background update uses static pixel	78
5.17 Erosion of slowly moving object	79
5.18 The case of normal moving object	80
5.19 Results of dealing with slowly moving objects	80
5.20 The case of static type of illumination change	81
6.1 Complete process of building background	87
6.2 The results of objects stop and leave	87
6.3 Result of dealing with objects stop and leave	88
6.4 Result of dealing with continuous big objects passing	89
6.5 Comparisons of foreground’s completeness and noise suppression	90
6.6 Result of foreground segmentation at night	91


List of Tables

2.1 Classification table of different background models	7
2.2 Comparison table of literatures	24
5.1 Update situations	62
6.1 Descriptions of test videos	84

[1]	D. Butler, S. Sridharan, V. M. Bove, Jr, “Real-time adaptive background segmentation”, in Proc. of Conf. on Acoustics, Speech, and Signal Processing, pp. 349-352, vol. 3, 2003.
[2]	A. Cavallaro, T. Ebrahimi, “Change detection based on color edges”, IEEE International Symposium on Circuits and Systems, pp. 141-144, vol. 2, 2001.
[3]	C. J. Chen, C. C. Chiu, B. F. Wu, S. P. Lin, C. D. Huang, “The Moving Object Segmentation Approach to Vehicle Extraction”, in Conf. on ICNSC, 2004.
[4]	R. Cucchiara, C. Grana, M. Piccardi, A. Prati, “Detecting moving objects, ghosts, and shadows in video streams”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1337-1342, vol. 25, issue: 10, 2003.
[5]	T. H. Cormen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithm, McGraw-Hill, 1994.
[6]	J. W. Davis, V. Sharma, “Fusion-Based Background-Subtraction using Contour Saliency”, in IEEE Conf. on Computer Vision and Pattern Recognition, pp. 11-16, vol. 3, 2005.
[7]	A. Elgammal, D. Harwood, L. Davis, "Non-parametric Model for Backgroud Subtraction", in European Conf. on Computer Vision, pp. 751-767, vol. 11, 2000.
[8]	A. Elgammal, R. Duraiswami, D. Harwood, L. S. Davis, “Background and foreground modeling using nonparametric kernel density estimation for visual surveillance”, in Proc. on IEEE, pp. 1151-1163, vol. 90, issue: 7, 2002.
[9]	N. Friedman, S. Russell, “Image segmentation in video sequences: A probabilistic approach”, in 13th Proc. of Conf. on Uncertainty in Artificial Intelligence, 1997.
[10]	D. Guo, Y. C. Hwang, Y. C. L. Adrian, C. Laugier, “Traffic monitoring using short-long term background memory”, in Proc. of 5th Conf. on Intelligent Transportation Systems, pp. 124-129, 2002.
[11]	R. C. Gonzales, R. E. Woods, Digital Image Processing, 2nd ed., Prentice-Hall, 2002.
[12]	D. Hong, W. Woo, “A background subtraction for a vision-based user interface”, in conf. on 4th Pacific Rim Conference on Multimedia, and in Joint Conf. on Information, Communications and Signal Processing, pp. 263-267, vol. 1, 2003.
[13]	E. Hayman, J. O. Eklundh, “Statistical background subtraction for a mobile observer”, in Proc. of Conf. on Computer Vision, pp. 67-74, 2003.
[14]	H. Han, Z. Wang, J. Liu, Z. Li, B. Li, Z. Han, "Adaptive background modeling with shadow suppression", in proc. IEEE Intelligent Transportation Systems, pp.720-724, vol. 1, 2003.
[15]	I. Haritaoglu, D. Harwood, L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance”, in 15th Conf. on Pattern Recognition, pp. 179-183, vol. 4, 2000.
[16]	M. Harville, G. Gordon, J. Woodfill, “Adaptive video background modeling using color and depth”, in Proc. of Conf. on Image Processing, pp. 90-93, vol. 3, 2001.
[17]	M. R. Hamind, A. Baloch, A. Bilal, N. Zaffar, “Object Segmentation Using Feature Based Conditional Morphology”, in Proc. of  12th Conf. on Image Analysis and Processing, pp. 548 – 553, 2003.
[18]	M. Heikkila, M. Pietikainen, “A Texture-Based Method for Modeling the Background and Detecting Moving Objects”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 657-662, vol. 28, issue: 4, 2006.
[19]	R. M. Haralick, L. G. Shapiro, Computer and Robot Vision: Volume I, Addison Wesley, 1992.
[20]	R. M. Haralick, L. G. Shapiro, Computer and Robot Vision: Volume II, Addison Wesley, 1992.
[21]	S. Huwer, H. Niemann, “Adaptive change detection for real-time surveillance applications”, in 3th IEEE Workshop on Visual Surveillance, pp. 37-46, 2000.
[22]	B. Jain, R. Kasturi, B. G. Schunck, Machine Vision, McGraw-Hill, 1995.
[23]	O. Javed, K. Shafique, M. Shah, “A hierarchical approach to robust background subtraction using color and gradient information”, in Proc. on Motion and Video Computing, pp. 22-27, 2002.
[24]	S. J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, H. Wechsler,” Tracking Groups of People“, Computer Vision and Image Understanding, pp. 42-56, vol. 80, Issue: 1, 2000.
[25]	S. Kamijo, Y. Matsushita, K. Ikeuchi, M. Sakauchi, “Traffic monitoring and accident detection at intersections”, in Proc. of Conf. on Intelligent Transportation Systems, pp. 703-708, 1999.
[26]	M. Lamarre, J. J. Clark, " Background subtraction using competing models in the block-DCT domain ", in Proc. of Conf. on Pattern Recognition, pp. 299-302, vol. 1, 2002.
[27]	S. J. McKEnna, Y. Raja, S. Gong, “Object Tracking using Adaptive Colour Mixture Models”, in Asian Conf. on Computer Vision, vol. 1, 1998.
[28]	J. R. Parker, Algorithms for Image Processing and Computer Vision, Wiley, 1997.
[29]	R. Pless, J. Larson, S. Siebers, B. Westover, “Evaluation of local models of dynamic backgrounds”, in Proc. of Conf. on Computer Vision and Pattern Recognition, pp. 73-78, vol. 2, 2003.
[30]	W. Q. Huang, Y. M. Wang, Y. Zhao, “Image segmentation using temporal-spatial information in dynamic scenes”, in Conf. on Machine Learning and Cybernetics, pp. 3140-3145, vol. 5, 2003.
[31]	A. Rosenfeld, J. L. Pfaltz, "Sequential Operations in Digital Picture Processing", journal of the association for computing Machinery”, pp. 471-494, Vol. 13, 1966.
[32]	C. R. Wren, A. Azarbayejani, T. Darrell, A. P. Pentland, “Pfinder: real-time tracking of the human body”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 780-785, vol. 19, Issue: 7, 1997.
[33]	C. Stauffer, W. E. L. Grimson, “Learning patterns of activity using real-time tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 747-757, vol. 22, issue: 8, 2000.
[34]	C. Stauffer, W. E. L. Grimson, “Adaptive background mixture models for real-time tracking”, in IEEE Conf. on Computer Vision and Pattern Recognition, pp. 246-252, vol. 2, 1999.
[35]	D. S. Lee, J. J. Hull, B. Erol, “A Bayesian framework for Gaussian mixture background modeling”, in Conf. on Image Processing , pp. 973-976, vol. 2, 2003.
[36]	L. D. Stefano, A. Bulgarelli, "A simple and efficient connected components labeling algorithm", in conf. on Image Analysis and Processing, pp. 322-327, 1999.
[37]	L. D. Stefano, S. Mattoccia, M. Mola, “A change-detection algorithm based on structure and colour”, in Proc. of Conf. on Advanced Video and Signal Based Surveillance, pp. 252-259, 2003.
[38]	M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis, and Machine Vision, 2nd ed., PWS, 1999.
[39]	Y. Sun, B. Yuan, "Hierarchical GMM to handle sharp changes in moving object detection", Electronic Letters, pp. 801-802, vol. 40, issue: 13, 2004.
[40]	D. Toth, T. Aach, “Detection and recognition of moving objects using statistical motion detection and Fourier descriptors”, in Proc. of 12th Conf. on Image Analysis and Processing, pp. 430-435, 2003.
[41]	J. C. Tai, K. T. Song, “Background Segmentation and its Application to Traffic Monitoring Using Modified Histogram”, in Conf. on ICNSC, 2004.
[42]	A. R. Webb, Statistical Pattern Recognition, 2nd ed., Wiley, 2002
[43]	Q. Z. Wu, H. Y. Cheng, K. C. Fan, “Motion Detection Based on Two-Piece Linear Approximation for Cumulative Histogram of Ratio Image in Intelligent Transportation Systems”, in Conf. on ICNSC, pp. 309-311, 2004.
[44]	H. Yinghua, H. Wang, B. Zhang, ” Background updating in illumination-variant scenes”, in Proc. on Intelligent Transportation Systems, pp. 515-19, vol. 1, 2003.
[45]	W. Yiming, X. Yang, K. L. Chan, “Unsupervised color image segmentation based on Gaussian mixture model “, in Proc. on Information, Communications and Signal Processing, and in Conf. on 4th Pacific Rim Multimedia, pp. 541-544, vol. 1, 2003.
[46]	J. B. Zheng; D. D. Feng, Y. Z. Zhang, W. C. Siu, R. C. Zhao, “An algorithm for video monitoring under a slow moving background”, in Proc. of Conf. on Machine Learning and Cybernetics, pp.1626-1629, vol. 3, 2002.