||Real-time Dynamic Background Segmentation Based on Multiple Reference Value Model
||Department of Computer Science and Information Engineering
reference background model
在現有的文獻中，在參考背景中的每一個像素都只有一個真實背景的參考值，而在此論文中所提出的multiple reference value background model (MRV background model)中的每一個像素卻有多個參考值，因此即使是再複雜或是紊亂的場景也能被正確地建立參考背景。
||In this thesis, we proposed a reliable and precise method for building and maintaining the reference background of a detected environment. Background segmentation (sub-traction) plays an important role in video surveillance systems and related applications. In order to extract the specific targets, applications must recognize what are objects (foreground) and what are not. Therefore, the fist step in such systems is usually to build a reference background model for the detected scene, and then the foreground can be extracted by comparing with the reference background model.
In the existing literature, each pixel of a reference background model has only one reference value to the real background of the detected scene. However, each pixel in the proposed multiple reference value (MRV) background model may have multiple reference values. Thus, even in complex or disorder scenes, reference backgrounds can also be correctly built.
Updating reference background is another important step for background segmen-tation. Because the detected scene will be changed, such as moving shadows of clouds or buildings, the reference background model must be modified to reflect these varia-tions. Otherwise, such applications will result in erroneous foreground segmentations. However, the situations of causing erroneous foreground segmentations are seldom discussed.
In this thesis, a global update and a local update methods are employed as the strategies for a reference background update; they control the entire and partial modi-fication of a reference background model, respectively. Therefore, the reference back-ground model can be used more robust than other proposed models for a long period of surveillance. In addition, the situations of causing erroneous foreground segmentations are also discussed in details.
By experimental results, the proposed method can obtain a more precise reference background model and preserves more details of a segmented foreground. Moreover, because of the reliable update strategies, the system can operate normally at daytime and nighttime. In addition, the system also can resist camera and object shaking.
||List of Figures III
List of Tables V
Chapter 1 Introduction
1.1 Motivation 1
1.2 Overview 2
1.3 Organization of the thesis 3
Chapter 2 Literature Reviews 5
2.1 Background Segmentation 5
2.2 Reference Background Models 6
2.2.1 Pure Background Model 8
2.2.2 Average Background Model 9
2.2.3 Gaussian Background Model 9
2.2.4 Mixture Gaussian Background Model 12
2.2.5 Histogram Background Model 15
2.2.6 Other Approaches 17
2.3 Update Strategies 19
2.4 Situation for Erroneous Segmentation 21
2.5 Emphases of Background Segmentation 23
Chapter 3 Preliminaries 26
3.1 Color Model 26
3.1.1 RGB Color Model 27
3.1.2 HSI Color Mode 28
3.2 Image Segmentation 29
3.3 Connected Component 31
3.4 Morphology 32
3.5 Sorting 34
Chapter 4 Background Segmentation 35
4.1 Introduction 35
4.2 Building a Reference Background 38
4.2.1 Sampling 38
4.2.2 Choosing Candidates 41
4.2.3 Deciding a Reference Background 43
4.2.4 Convergence and Non-convergence 44
4-3. Non-convergence Processing 45
4.3.1 Re-sampling 46
4.3.2 Parameters Adjusting 47
4.3.3 Oscillation Problem 49
4.4 Transferring Background and Foreground Segmentation 53
Chapter 5 Background Update 58
5.1 Introduction 58
5.2 Occasions of Update 59
5.2.1 Un-update Situations 59
5.2.2 Priority of Update 61
5.3 Global Update 62
5.4 Local Update 64
5.5 Dealing with Situations of Causing Erroneous Segmentation 68
5.5.1 Many Objects Passing 69
5.5.2 Big Objects Passing 71
5.5.3 Jam of Objects 73
5.5.4 Stopping / Leaving Objects 74
5.5.5 Slowly Moving Objects 79
5.5.6 Illumination Variations 81
Chapter 6 Experimental Results 83
6.1 Test Environment 83
6.2 Results 85
Chapter 7 Conclusions and Future Works 92
7.1 Conclusions 92
7.2 Future Works 92
List of Figures
1.1 Foreground segmentation 2
2.1 Flow chart of building of reference background model 5
2.2 Foreground segmentation by a pure reference background 7
2.3 Build a reference background dynamically 8
2.4 A case of the pure background model 8
2.5 The different intensity distributions 10
2.6 Foreground segmentation results by a Gaussian background model 12
2.7 A mixture Gaussian distribution approximation of intensity distribution 12
2.8 Foreground segmentation results by a mixture Gaussian background model 14
2.9 Intensity diagram 15
2.10 A case of modified histogram 16
2.11 Object contours extraction by thermal image 19
2.12 Pixel states migration 21
2.13 Discrimination between background and shadow on the RGB color model 23
3.1 RGB color model 27
3.2 HSI color model 28
3.3 Image segmentation 30
3.4 Connected components 32
3.5 N-connected 32
3.6 Opening operation 33
4.1 Flow chart of complete reference background segmentation 37
4.2 Flow char of building reference background 38
4.3 Flow chart of sampling 39
4.4 The recorded colors of pixel and each color’s count 42
4.5 Flow chart of non-convergence process 45
4.6 The homogeneous situation 49
4.7 Oscillation phenomenon 51
4.8 The part of number of recorded colors 52
4.9 The part of reference number 52
4.10 Three results of foreground segmentation of different video clips 54
4.11 Sketch of transferring a reference background 55
4.12 Three results of noise removal from three different video clips 56
4.13 Three results of opening 57
5.1 The erroneous foreground segmentation 59
5.2 The erroneous update occasion 60
5.3 The erroneous foreground segmentation by un-fixed exposure 60
5.4 Flow chart of global update 63
5.5 The foreground segmentation by unstable background model 65
5.6 The erroneous foreground segmentation by slowly moving object 65
5.7 Erroneous foreground segmentation of stop objects 66
5.8 Erosion situation of foreground 67
5.9 Flow chart of local update 68
5.10 The comparison of histograms 71
5.11 The changing of intensity histogram of big object passing 72
5.12 Un-sufficient exposure case 73
5.13 Erroneous foreground segmentation causes by static object 75
5.14 Erroneous foreground segmentation causes by leaving object 77
5.15 The static mask table 78
5.16 Flow chart of background update uses static pixel 78
5.17 Erosion of slowly moving object 79
5.18 The case of normal moving object 80
5.19 Results of dealing with slowly moving objects 80
5.20 The case of static type of illumination change 81
6.1 Complete process of building background 87
6.2 The results of objects stop and leave 87
6.3 Result of dealing with objects stop and leave 88
6.4 Result of dealing with continuous big objects passing 89
6.5 Comparisons of foreground’s completeness and noise suppression 90
6.6 Result of foreground segmentation at night 91
List of Tables
2.1 Classification table of different background models 7
2.2 Comparison table of literatures 24
5.1 Update situations 62
6.1 Descriptions of test videos 84
|| D. Butler, S. Sridharan, V. M. Bove, Jr, “Real-time adaptive background segmentation”, in Proc. of Conf. on Acoustics, Speech, and Signal Processing, pp. 349-352, vol. 3, 2003.
 A. Cavallaro, T. Ebrahimi, “Change detection based on color edges”, IEEE International Symposium on Circuits and Systems, pp. 141-144, vol. 2, 2001.
 C. J. Chen, C. C. Chiu, B. F. Wu, S. P. Lin, C. D. Huang, “The Moving Object Segmentation Approach to Vehicle Extraction”, in Conf. on ICNSC, 2004.
 R. Cucchiara, C. Grana, M. Piccardi, A. Prati, “Detecting moving objects, ghosts, and shadows in video streams”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1337-1342, vol. 25, issue: 10, 2003.
 T. H. Cormen, C. E. Leiserson, R. L. Rivest, Introduction to Algorithm, McGraw-Hill, 1994.
 J. W. Davis, V. Sharma, “Fusion-Based Background-Subtraction using Contour Saliency”, in IEEE Conf. on Computer Vision and Pattern Recognition, pp. 11-16, vol. 3, 2005.
 A. Elgammal, D. Harwood, L. Davis, "Non-parametric Model for Backgroud Subtraction", in European Conf. on Computer Vision, pp. 751-767, vol. 11, 2000.
 A. Elgammal, R. Duraiswami, D. Harwood, L. S. Davis, “Background and foreground modeling using nonparametric kernel density estimation for visual surveillance”, in Proc. on IEEE, pp. 1151-1163, vol. 90, issue: 7, 2002.
 N. Friedman, S. Russell, “Image segmentation in video sequences: A probabilistic approach”, in 13th Proc. of Conf. on Uncertainty in Artificial Intelligence, 1997.
 D. Guo, Y. C. Hwang, Y. C. L. Adrian, C. Laugier, “Traffic monitoring using short-long term background memory”, in Proc. of 5th Conf. on Intelligent Transportation Systems, pp. 124-129, 2002.
 R. C. Gonzales, R. E. Woods, Digital Image Processing, 2nd ed., Prentice-Hall, 2002.
 D. Hong, W. Woo, “A background subtraction for a vision-based user interface”, in conf. on 4th Pacific Rim Conference on Multimedia, and in Joint Conf. on Information, Communications and Signal Processing, pp. 263-267, vol. 1, 2003.
 E. Hayman, J. O. Eklundh, “Statistical background subtraction for a mobile observer”, in Proc. of Conf. on Computer Vision, pp. 67-74, 2003.
 H. Han, Z. Wang, J. Liu, Z. Li, B. Li, Z. Han, "Adaptive background modeling with shadow suppression", in proc. IEEE Intelligent Transportation Systems, pp.720-724, vol. 1, 2003.
 I. Haritaoglu, D. Harwood, L. S. Davis, “A fast background scene modeling and maintenance for outdoor surveillance”, in 15th Conf. on Pattern Recognition, pp. 179-183, vol. 4, 2000.
 M. Harville, G. Gordon, J. Woodfill, “Adaptive video background modeling using color and depth”, in Proc. of Conf. on Image Processing, pp. 90-93, vol. 3, 2001.
 M. R. Hamind, A. Baloch, A. Bilal, N. Zaffar, “Object Segmentation Using Feature Based Conditional Morphology”, in Proc. of 12th Conf. on Image Analysis and Processing, pp. 548 – 553, 2003.
 M. Heikkila, M. Pietikainen, “A Texture-Based Method for Modeling the Background and Detecting Moving Objects”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 657-662, vol. 28, issue: 4, 2006.
 R. M. Haralick, L. G. Shapiro, Computer and Robot Vision: Volume I, Addison Wesley, 1992.
 R. M. Haralick, L. G. Shapiro, Computer and Robot Vision: Volume II, Addison Wesley, 1992.
 S. Huwer, H. Niemann, “Adaptive change detection for real-time surveillance applications”, in 3th IEEE Workshop on Visual Surveillance, pp. 37-46, 2000.
 B. Jain, R. Kasturi, B. G. Schunck, Machine Vision, McGraw-Hill, 1995.
 O. Javed, K. Shafique, M. Shah, “A hierarchical approach to robust background subtraction using color and gradient information”, in Proc. on Motion and Video Computing, pp. 22-27, 2002.
 S. J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, H. Wechsler,” Tracking Groups of People“, Computer Vision and Image Understanding, pp. 42-56, vol. 80, Issue: 1, 2000.
 S. Kamijo, Y. Matsushita, K. Ikeuchi, M. Sakauchi, “Traffic monitoring and accident detection at intersections”, in Proc. of Conf. on Intelligent Transportation Systems, pp. 703-708, 1999.
 M. Lamarre, J. J. Clark, " Background subtraction using competing models in the block-DCT domain ", in Proc. of Conf. on Pattern Recognition, pp. 299-302, vol. 1, 2002.
 S. J. McKEnna, Y. Raja, S. Gong, “Object Tracking using Adaptive Colour Mixture Models”, in Asian Conf. on Computer Vision, vol. 1, 1998.
 J. R. Parker, Algorithms for Image Processing and Computer Vision, Wiley, 1997.
 R. Pless, J. Larson, S. Siebers, B. Westover, “Evaluation of local models of dynamic backgrounds”, in Proc. of Conf. on Computer Vision and Pattern Recognition, pp. 73-78, vol. 2, 2003.
 W. Q. Huang, Y. M. Wang, Y. Zhao, “Image segmentation using temporal-spatial information in dynamic scenes”, in Conf. on Machine Learning and Cybernetics, pp. 3140-3145, vol. 5, 2003.
 A. Rosenfeld, J. L. Pfaltz, "Sequential Operations in Digital Picture Processing", journal of the association for computing Machinery”, pp. 471-494, Vol. 13, 1966.
 C. R. Wren, A. Azarbayejani, T. Darrell, A. P. Pentland, “Pfinder: real-time tracking of the human body”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 780-785, vol. 19, Issue: 7, 1997.
 C. Stauffer, W. E. L. Grimson, “Learning patterns of activity using real-time tracking”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 747-757, vol. 22, issue: 8, 2000.
 C. Stauffer, W. E. L. Grimson, “Adaptive background mixture models for real-time tracking”, in IEEE Conf. on Computer Vision and Pattern Recognition, pp. 246-252, vol. 2, 1999.
 D. S. Lee, J. J. Hull, B. Erol, “A Bayesian framework for Gaussian mixture background modeling”, in Conf. on Image Processing , pp. 973-976, vol. 2, 2003.
 L. D. Stefano, A. Bulgarelli, "A simple and efficient connected components labeling algorithm", in conf. on Image Analysis and Processing, pp. 322-327, 1999.
 L. D. Stefano, S. Mattoccia, M. Mola, “A change-detection algorithm based on structure and colour”, in Proc. of Conf. on Advanced Video and Signal Based Surveillance, pp. 252-259, 2003.
 M. Sonka, V. Hlavac, R. Boyle, Image Processing, Analysis, and Machine Vision, 2nd ed., PWS, 1999.
 Y. Sun, B. Yuan, "Hierarchical GMM to handle sharp changes in moving object detection", Electronic Letters, pp. 801-802, vol. 40, issue: 13, 2004.
 D. Toth, T. Aach, “Detection and recognition of moving objects using statistical motion detection and Fourier descriptors”, in Proc. of 12th Conf. on Image Analysis and Processing, pp. 430-435, 2003.
 J. C. Tai, K. T. Song, “Background Segmentation and its Application to Traffic Monitoring Using Modified Histogram”, in Conf. on ICNSC, 2004.
 A. R. Webb, Statistical Pattern Recognition, 2nd ed., Wiley, 2002
 Q. Z. Wu, H. Y. Cheng, K. C. Fan, “Motion Detection Based on Two-Piece Linear Approximation for Cumulative Histogram of Ratio Image in Intelligent Transportation Systems”, in Conf. on ICNSC, pp. 309-311, 2004.
 H. Yinghua, H. Wang, B. Zhang, ” Background updating in illumination-variant scenes”, in Proc. on Intelligent Transportation Systems, pp. 515-19, vol. 1, 2003.
 W. Yiming, X. Yang, K. L. Chan, “Unsupervised color image segmentation based on Gaussian mixture model “, in Proc. on Information, Communications and Signal Processing, and in Conf. on 4th Pacific Rim Multimedia, pp. 541-544, vol. 1, 2003.
 J. B. Zheng; D. D. Feng, Y. Z. Zhang, W. C. Siu, R. C. Zhao, “An algorithm for video monitoring under a slow moving background”, in Proc. of Conf. on Machine Learning and Cybernetics, pp.1626-1629, vol. 3, 2002.