# Motion Tracking/Detection in MATLAB using Kalman Filter

MATAB has great capabilities to process video & pictures. One of these capabilities we have tested is the motion detection in a pre-recorded video, using Kalman Filtering technique.

**Here is an example video of Motion Tracking using Kalman Filter:****Video for illustration Purpose Only**

**Basic background of Kalman Filter:**
The

**Kalman filter**, also known as**linear quadratic estimation (LQE)**, is an algorithm that uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone. More formally, the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state. The filter is named for Rudolf (Rudy) E. Kálmán, one of the primary developers of its theory.
The Kalman filter has numerous applications, e.g, for guidance, navigation & control of vehicles, particularly aircraft and spacecraft.

Kalman Filter Algorithm (courtsey: colorado.edu) |

__:__

**MATLAB CODE****%Code in the Bold, & comments in normal font**

**clear all;**

**close all;**

**clc**

**%% Read video into MATLAB using aviread**

video = aviread('rhinos.AVI');

video = aviread('rhinos.AVI');

%'n' for calculating the number of frames in the video file

**n = length(video);**

**% Calculate the background image by averaging the first 10 images**

temp = zeros(size(video(1).cdata));

[M,N] = size(temp(:,:,1));

for i = 1:10

temp = double(video(i).cdata) + temp;

end

imbkg = temp/10;

temp = zeros(size(video(1).cdata));

[M,N] = size(temp(:,:,1));

for i = 1:10

temp = double(video(i).cdata) + temp;

end

imbkg = temp/10;

**% Initialization step for Kalman Filter**

centroidx = zeros(n,1);

centroidy = zeros(n,1);

predicted = zeros(n,4);

actual = zeros(n,4);

centroidx = zeros(n,1);

centroidy = zeros(n,1);

predicted = zeros(n,4);

actual = zeros(n,4);

**% % Initialize the Kalman filter parameters**

% R - measurement noise,

% H - transform from measure to state

% Q - system noise,

% P - the status covarince matrix

% A - state transform matrix

R=[[0.2845,0.0045]',[0.0045,0.0455]'];

H=[[1,0]',[0,1]',[0,0]',[0,0]'];

Q=0.01*eye(4);

P = 100*eye(4);

dt=1;

A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

R=[[0.2845,0.0045]',[0.0045,0.0455]'];

H=[[1,0]',[0,1]',[0,0]',[0,0]'];

Q=0.01*eye(4);

P = 100*eye(4);

dt=1;

A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

**% loop over all image frames in the video**

**% Calculate the difference image to extract pixels with more than 40(threshold) change**

kfinit = 0;

th = 38;

for i=1:n

imshow(video(i).cdata);

hold on

imcurrent = double(video(i).cdata);

kfinit = 0;

th = 38;

for i=1:n

imshow(video(i).cdata);

hold on

imcurrent = double(video(i).cdata);

diffimg = zeros(M,N);

diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...

| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...

| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

diffimg = zeros(M,N);

diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...

| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...

| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

**% Label the image and mark**

labelimg = bwlabel(diffimg,4);

markimg = regionprops(labelimg,['basic']);

[MM,NN] = size(markimg);

labelimg = bwlabel(diffimg,4);

markimg = regionprops(labelimg,['basic']);

[MM,NN] = size(markimg);

**% Do bubble sort (large to small) on regions in case there are more than 1**

% The largest region is the object (1st one)

for nn = 1:MM

if markimg(nn).Area > markimg(1).Area

tmp = markimg(1);

markimg(1)= markimg(nn);

markimg(nn)= tmp;

end

end

for nn = 1:MM

if markimg(nn).Area > markimg(1).Area

tmp = markimg(1);

markimg(1)= markimg(nn);

markimg(nn)= tmp;

end

end

**% Get the upper-left corner, the measurement centroid and bounding window size**

**bb = markimg(1).BoundingBox;**

xcorner = bb(1);

ycorner = bb(2);

xwidth = bb(3);

ywidth = bb(4);

cc = markimg(1).Centroid;

centroidx(i)= cc(1);

centroidy(i)= cc(2);

xcorner = bb(1);

ycorner = bb(2);

xwidth = bb(3);

ywidth = bb(4);

cc = markimg(1).Centroid;

centroidx(i)= cc(1);

centroidy(i)= cc(2);

**% Plot the rectangle of background subtraction algorithm -- blue**

**hold on**

rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');

hold on

plot(centroidx(i),centroidy(i), 'bx');

rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');

hold on

plot(centroidx(i),centroidy(i), 'bx');

**% Kalman window size**

**kalmanx = centroidx(i)- xcorner;**

kalmany = centroidy(i)- ycorner;

kalmany = centroidy(i)- ycorner;

**% Initialize the predicted centroid and volocity**

if kfinit == 0

if kfinit == 0

**predicted =[centroidx(i),centroidy(i),0,0]' ;**

else

% Use the former state to predict the new centroid and volocity

else

**predicted = A*actual(i-1,:)';**

end

kfinit = 1;

end

kfinit = 1;

Ppre = A*P*A' + Q;

K = Ppre*H'/(H*Ppre*H'+R);

actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';

P = (eye(4)-K*H)*Ppre;

Ppre = A*P*A' + Q;

K = Ppre*H'/(H*Ppre*H'+R);

actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';

P = (eye(4)-K*H)*Ppre;

% Plot the tracking rectangle after Kalman filtering -- red

hold on

% Plot the tracking rectangle after Kalman filtering -- red

hold on

**rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth], 'EdgeColor', 'r','LineWidth',1.5);**

hold on

plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);

drawnow;

end

hold on

plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);

drawnow;

end

**%end of the code**

**%Copyright (c) 2012, Xing**

%All rights reserved.

%All rights reserved.

__NOTE:__
The above code sometimes generate the following error & warning, in the initial step itself (while reading the .avi file using "aviread" function.

**Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead.**

> In aviread at 26

In motionnew at 4

Error using aviread (line 80)

Unable to locate decompressor to decompress video stream

Error in motionnew (line 4)

video = aviread('traffic.avi');

> In aviread at 26

In motionnew at 4

Error using aviread (line 80)

Unable to locate decompressor to decompress video stream

Error in motionnew (line 4)

video = aviread('traffic.avi');

# First warning is due to the fact, that

**aviread**function has became obsolete & MATLAB team decided to replace it with a new function called, VideoReader.
# In order to avoid the second error, you need to install the avi video codec, the best thing available in the internet is the K-Lite Mega Codec Pack.

~Team Digital iVision Lab.

# do comment for any code request, help or queries.

Original Unmodified Source can be downloaded here.

If having problem with aviread or mmread function kindly visit the following post.

How to resolve MATLAB Problem With "aviread" or "VideoReader" command, while reading the avi file?

i have install k-lite codec mega but still not support the aviread

ReplyDeleteTry with "rhinos.AVI" first, if it is working. Then you should find the video file which has the same codec compression done as with rhinos.AVI. It will surely work! Other wise use VideoReader function. Use "getfourcc" command to view all the supported codecs in your MATLAB version

ReplyDeletecannot use the getfourcc.. i use R2013a matlab version... now i am doing final project about "vehicles counting system" but its not working... i have use ur code but still cannot support the aviread... when use mmreader then the mistake at :

Deletetemp = zeros(size(video(1).cdata));

This comment has been removed by the author.

ReplyDeleteTry Converting the file, in MPG or ASF format (use Format Factory software for this). And use VideoReader Function instead, I hope that will surely work. In my system the above code is working fine, without any error. The problem with "aviread" is, that it is now an obsolete function. In the next 2 or 3 releases it will be removed.

Deletefor the error you have mentioned: now the error when use "rhinos.AVI"

ReplyDelete??? Error: File: aviread.m Line: 93 Column: 22

Unexpected MATLAB expression.

You are likely missing a ')', ']' or '}' or similar.

convert to swf and mpg also done... so the error still at temp = zeros(size(video(1).cdata));

ReplyDeleteto get the codecs that can run "aviread" is not easy..it ok then.. maybe you use the old version of codecs or matlab..

One last thing you can still try is, Extracting & Saving frames from a Video file using Matlab (http://www.divilabs.com/2013/11/extracting-saving-of-frames-from-video.html) and then, Create A Video File From A Sequence Of Image Stored in a Folder, Using MATLAB (http://www.divilabs.com/2013/12/create-video-file-from-sequence-of.html). If a video file created by MATLAB, it must also be readable by MATLAB!

Deleteya... can but from the code u given ... the video are stored in folder "snaps" where include frames that have been convert rite.. but to call back that video i used this.. video = VideoReader('\snaps\Video.avi'); it become false...

DeleteChange the current directory to "snaps", or copy the video file thus created from "snaps" to your current working directory! Then by just giving the command "video= VideoReader('Video.avi')" should work!

DeleteI Hope you have created back the video, from frames! Without it, it will not able to read!

DeleteMy MATLAB version is 2012b, use this command, formats = VideoReader.getFileFormats()

ReplyDeleteIn my system it is giving the output as: (so they are supported), try in yours, it could work!

Video File Formats:

.asf - ASF File

.asx - ASX File

.avi - AVI File

.m4v - MPEG-4 Video

.mj2 - Motion JPEG2000

.mov - QuickTime movie

.mp4 - MPEG-4

.mpg - MPEG-1

.wmv - Windows Media Video

Error using VideoReader/init (line 450)

DeleteUnable to determine the codec required.

Error in VideoReader (line 147)

obj.init(fileName);

Error in kalman (line 7)

video = VideoReader('Video.AVI');

With this I can assume (not exactly though), that your matlab is not able to find the codecs of the video that are installed in your system. May be some configuration of MATLAB you have to alter so that it will look for the codec that are installed in your system!

Deletean actually when to convert image to video this error appeared:

DeleteWarning: No video frames were written to this file. The file may be invalid.

> In VideoWriter.VideoWriter>VideoWriter.close at 307

In pojek1 at 15

Its a Codec problem, try in other system or other MATLAB's version, it might work!

Deleteits ok...i already frust... still error

ReplyDeleteNo appropriate method, property, or field cdata for class VideoReader.

Error in kalman (line 12)

temp = zeros(size(video(1).cdata));

This comment has been removed by the author.

Deleteyou can use this code for video :

Deletevideo = VideoReader('rhinos.AVI.AVI');

vidFrames = read(video);

nframes = get(video, 'numberOfFrames');

temp = zeros(size(vidFrames(:,:,:,1)));

[M,N] = size(vidFrames(:,:,1));

for i = 1:10

temp = double(vidFrames(:,:,:,i)) + temp;

end

imbkg = temp/10;

% Initialization for Kalman Filtering

centroidx = zeros(nframes,1);

centroidy = zeros(nframes,1);

predicted = zeros(nframes,4);

actual = zeros(nframes,4);

....

for i=1:nframes

imshow(vidFrames(:,:,:,i));

hold on

imcurrent = double(vidFrames(:,:,:,i));

how to use this code for real time detection using web cam?I have tried but didn't get...

ReplyDeleteThis code was written to detect motion with a Pre Recorded Video.... Try to make a composite code using the above code & the code listed in http://www.divilabs.com/2013/11/recording-video-from-webcam-using.html

DeleteReplace the video opening part in the above code, with the code from the link given, to directly access the webcam!

It should work.....

how do i use my videos?

ReplyDeletehttp://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

DeleteVisit the link...n follow the steps, you will be able to use your videos...!

how can I modify code for multiple object tracking using kalman filter to view effect on each frame of a video due to each function call? referring to the Code which is available in R2013a- computer vision tool box. Pl. help.

ReplyDeleteThis code is actually for, multi moving object tracking..! So you can use it.

Deletethanks sir. But how can I observe/see in different windows each and every variable's and called function's effect on each frame apart from videoPlayer() for mask and frame? I am not very good matlab programmer. Thats why i need the necessary modifications in available code. pl help.

ReplyDeleteWell in order to see, whats going inside a function, you can make use of Breakpoints while in the MATLAB's editor window. What a breakpoint will do is, it will stop the execution at that point of code execution (or say it at that particular line). So that you can see, easily the effect of it.!

DeleteMATLAB has a separate window, known as "Workspace". Here you can see all the values of the variables, while execution, in case you have created the breakpoint!

Deletewe install codec pack.... but we can't get the output for long videos... what we do now..?

ReplyDeleteWhat kind of error you are getting? Can you please elaborate?

Delete??? Error using ==> aviread at 84

ReplyDeleteUnable to locate decompressor to decompress video

stream

Error in ==> dummy at 5

video = aviread('as.avi');

Please Refer this link for this problem to be resolved:

Deletehttp://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

we want explanation for each lines

ReplyDelete% Calculate the background image by averaging the first 10 images

temp = zeros(size(video(1).cdata));

[M,N] = size(temp(:,:,1));

for i = 1:10

temp = double(video(i).cdata) + temp;

end

imbkg = temp/10;

temp = zeros(size(video(1).cdata));--> creating an empty frame

Delete[M,N] = size(temp(:,:,1));--> getting the dimension of the frame & storing it into M & N

for i = 1:10

temp = double(video(i).cdata) + temp;

end

imbkg = temp/10;

the above 4 are for averaging 10 frames...........

Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2ve4uOZvg

can you briefly explain the codings?

ReplyDeleteThere is already Enough explanation in comments! If you want specifically a line of code to be explained then please comment with that!

Deletewe want explanation for each lines &also syntax

Deletediffimg = zeros(M,N);

diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...

| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...

| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

diffimg = zeros(M,N);--> will create a image of all zero pixel value

Deletediffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...

| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...

| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2xwFFBpie

They all will be used to subtract the RGB frames from the background!

Is it default value....?

ReplyDeleteR=[[0.2845,0.0045]',[0.0045,0.0455]'];

H=[[1,0]',[0,1]',[0,0]',[0,0]'];

Q=0.01*eye(4);

P = 100*eye(4);

dt=1;

A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

Yes

DeleteFor more details about this: refer Fundamentals of Kalman Filtering: A Practical Approach by Paul Zarchan

DeleteCHapter4 of whose has all the explanation about these params.....

If one can easily track multiobjects/single object with background subtraction method, whats the need of kalman filter?

ReplyDeleteAs KF fails in many other applications but works well only with tracking, whats that specific thing about KF which makes it so special for tracking?

Kalman filter gives a running covariance as well as the filtered result about moving object. Also I have read somewhere that, Kalman filter finds the most optimum averaging factor for each consequent state. Also somehow remembers a little bit about the past states.

DeleteGoto the link, for an equation based understanding of Kalman..... That will be enough to answer for your question

bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

I am using MATLAB 2013a. I want to publish .m file in ppt and doc format. Output file is in D drive. I want to get images i.e.individual frames (i/p and processed) in video also in output file. But its not working. Its giving error in mdbpublish and open files.

ReplyDeleteHow do I get rid of this?

Please give more detail about the error so that I could help..!

Deletesir..why still cannot function when i already instal the codec... like this Error using aviread (line 80)

ReplyDeleteUnable to locate decompressor to decompress video stream

Error in digitalivision1 (line 5)

video = aviread('pedestrian.AVI');

For this problem resolution, I have written another article. That will help you.

Deletehttp://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

Stil error like this.. can you explain 1st step what should i do...

ReplyDeleteThe closest match is: VideoReader

in C:\Program

Files\MATLAB\R2012a\toolbox\matlab\audiovideo\@VideoReader\VideoReader.m

Error in Untitled4 (line 7)

video = videoreader('pedestrian.AVI');

Well this error is due to the fact that MATLAB uses a case-sensitive syntax, so instead of "videoreader" you have to use "VideoReader"!

Deletestill error...

ReplyDeleteNo appropriate method, property, or field cdata for class VideoReader.

Error in digitalivision2 (line 9)

temp = zeros(size(video(1).cdata));

Please refer to the syntax of VideoReader.... Type "help VideoReader"..u will get idea

DeleteThis comment has been removed by the author.

ReplyDeleteStil error like this.. can you explain 1st step what should i do...

ReplyDelete???Index exceeds matrix dimensions.

Error in ==> tracked at 62

bb = markimg(1).BoundingBox;

As the error message says, you are trying to access a position in 'bb' that does not exist. Do check you have written the code before this line (62) correctly. Also if u can, please tell the value of 'bb' variable till this step, as you can see in "Workspace".

DeleteHello! where can I get free download video dataset for multiple objects tracking?

ReplyDeletewhy .flv videos are not read by videoreader in 2013a computer vision toolbox multiple objects tracking prog.?

is width n height needed for video to be read by videoreader? then why error?

.flv file is not readable by matlab. yes width & height is needed for sure, actually the video object you have created by VideoReader command itself contains its information. You need to convert the . flv file to either mpeg or avi. If still you have a problem please visit--> http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

DeleteThe code runs fine as long as it has 'rhinos.avi'. I converted another video to avi format and replaced it with rhinos.avi. First the code was giving the same error about the decompressor. I downloaded the codec pack now it's giving me the following error

ReplyDelete??? Out of memory. Type HELP MEMORY for your options.

Error in ==> rot90 at 38

B = A(n:-1:1,:);

Error in ==> aviread at 123

f = rot90(reshape(f, paddedWidth,height));

Error in ==> usingfilter3 at 6

video = aviread('Wildlife.AVI');

Can you please help me with it?? I need urgent help! Will be quite thankful

http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

DeleteVisit the link...n follow the steps, you will be able to use your videos...!

I tried that too, but still can't find the solution as it says :

ReplyDeleteD:>'ffmpeg' is not recognized as an internal or external command, operable program or batch file.

Try installing ffmpeg correctly

Deletecan you please explain the following commands:

ReplyDelete% Get the upper-left corner, the measurement centroid and bounding window size

bb = markimg(1).BoundingBox;

xcorner = bb(1);

ycorner = bb(2);

xwidth = bb(3);

ywidth = bb(4);

cc = markimg(1).Centroid;

centroidx(i)= cc(1);

centroidy(i)= cc(2);

bb = markimg(1).BoundingBox; will be having all the coordinates necessary for making the detection rectangle.

Deletexcorner = bb(1);

ycorner = bb(2);

xwidth = bb(3);

ywidth = bb(4);

will be used to draw the rectangle.

cc = markimg(1).Centroid;

centroidx(i)= cc(1);

centroidy(i)= cc(2);

for drawing the centroid in the output image

Read more: http://www.divilabs.com/2014/06/left-shifting-histogram-histogram.html

Please explain the following

ReplyDelete% Kalman window size

kalmanx = centroidx(i)- xcorner;

kalmany = centroidy(i)- ycorner;

if kfinit == 0

% Initialize the predicted centroid and volocity

predicted =[centroidx(i),centroidy(i),0,0]' ;

else

% Use the former state to predict the new centroid and volocity

predicted = A*actual(i-1,:)';

end

kalmanx = centroidx(i)- xcorner;

Deletekalmany = centroidy(i)- ycorner;

-->for the object in motion detection window size. i.e., the red square it is displaying while detection.

if kfinit == 0

predicted =[centroidx(i),centroidy(i),0,0]' ;

else

-->The above 3 line for the initial condition. I.e when the 1st frame of the video is encountered.

predicted = A*actual(i-1,:)';

end

--> The velocity of the moving object & the new centroid will be predicted by these 2 lines.

can you please give a code for multiple non salient objects detection n tracking using particle filter?

ReplyDeleteWhere can I get A to Z info. about particle filter?

You can refer various research paper available on internet (IEEE specifically). various tutorials are available on youtube explaining particle filter. This filter I havn't tried out yet, so can't tell about that much.

DeleteHi, the given code only detects one object. How do you extend it to detect multiple objects as shown in the example vide?

ReplyDeleteIn this code, i have taken the average of frames of full size. You can divide the screen into various sections by fixing the coordinates (e.g. in openCV its done through cv::rect() ) & apply the processing there.

DeleteHi Vibhutesh, please take a close look at the code you have published. It can detect only a single object. The video shown as example is misleading!

ReplyDeleteI know about the code which is published here. The video is for illustration only, it shows how the Kalman filter can be utilized effectively. For more advance version of the code you need to refer various texts for this topics. OpenCV will be the best alternative for this.

DeleteI can't find The Sollution for this

ReplyDeleteError in kalmanmotion (line 61)

bb = markimg(1).BoundingBox;

What's the other line of error message is saying?

Deletehello! I hav some doubts regarding KF.

ReplyDelete1. Why jitter and creep are not identified by KF?

2. why KF is called precisely wiener filter for stationary least squares smoothing?

3. Why KF is called as time varient Wiener filter?

4.What is a local minima, which is a critical problem with wiener filter. How it is overcome in KF?

Well I really want to give answers to these questions, but I also need to do much research on KF before saying anything about it. The above code is simply one of the application of KF!

DeleteI am using 2013a MATLAB. exploring multiobj tracking prog. in computer vision toolbox. How can i get coding for kalmanFilter object which is a library obj. there?

ReplyDeleteI am not able to analyse results of predict() and correct() after each frame processing. how can i get coding of these library functions? pl. help.

To analyze any line of code, what it does & what variable it creates is done through breakpoints. The obj version of the code is actually a compiled version, so a very less chance is there for you to decode it. Instead you should check for matlab central for the original source code for the same. It will surely help!

Deletehow can i modify matlab 2013a MTT code using particle filter instead of kalman filter?

ReplyDeleteI have done some work under KF only, still to start with PF!

DeleteI am trying to implement your code for people detection. However, why only detect a single person not multiple people in a frame? do you know how to detect for multiple people? thank you

ReplyDeleteThe code is for Motion Detection. The code also require the background to be fixed. As this is an average based method, it will fail miserably if whole scene is in motion. For a perfect detection of motion the background need to be constant (non changing).

DeleteOkay, I have a static background. But I still cant track multiple people. only a single person is traced. Any idea how to track multiple people using Kalman Filter? I am a bit new with Kalman Filter.

ReplyDeleteOne thing that you can apply, that divide the frame to multiple blocks & process each block separately. Use the Kalman code as a function to track the motion on each block!

Deletecan you sugggest me how to convert any video format to avi format

ReplyDeleteYou can use any video converter available. But to use them with matlab you need to use that codec for compression, that are supported & installed in your matlab!

Deleteclear all; close all; clc

ReplyDelete%% Read video into MATLAB using aviread

video = aviread('1.AVI');

nframes = length(video);

% Calculate the background image by averaging the first 5 images

temp = zeros(size(video(1).cdata));

[M,N] = size(temp(:,:,1));

for i = 1:10

temp = double(video(i).cdata) + temp;

end

imbkg = temp/10;

% Initialization for Kalman Filtering

centroidx = zeros(nframes,1);

centroidy = zeros(nframes,1);

predicted = zeros(nframes,4);

actual = zeros(nframes,4);

% % Initialize the Kalman filter parameters

% R - measurement noise,

% H - transform from measure to state

% Q - system noise,

% P - the status covarince matrix

% A - state transform matrix

R=[[0.2845,0.0045]',[0.0045,0.0455]'];

H=[[1,0]',[0,1]',[0,0]',[0,0]'];

Q=0.01*eye(4);

P = 100*eye(4);

dt=1;

A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

% loop over all image frames in the video

kfinit = 0;

th = 38;

for i=1:nframes

imshow(video(i).cdata);

hold on

imcurrent = double(video(i).cdata);

% Calculate the difference image to extract pixels with more than 40(threshold) change

diffimg = zeros(M,N);

diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...

| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...

| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

% Label the image and mark

labelimg = bwlabel(diffimg,4);

markimg = regionprops(labelimg,['basic']);

[MM,NN] = size(markimg);

% Do bubble sort (large to small) on regions in case there are more than 1

% The largest region is the object (1st one)

for nn = 1:MM

if markimg(nn).Area > markimg(1).Area

tmp = markimg(1);

markimg(1)= markimg(nn);

markimg(nn)= tmp;

end

end

% Get the upper-left corner, the measurement centroid and bounding window size

bb = markimg(1).BoundingBox;

xcorner = bb(1);

ycorner = bb(2);

xwidth = bb(3);

ywidth = bb(4);

cc = markimg(1).Centroid;

centroidx(i)= cc(1);

centroidy(i)= cc(2);

% Plot the rectangle of background subtraction algorithm -- blue

hold on

rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');

hold on

plot(centroidx(i),centroidy(i), 'bx');

% Kalman window size

kalmanx = centroidx(i)- xcorner;

kalmany = centroidy(i)- ycorner;

if kfinit == 0

% Initialize the predicted centroid and volocity

predicted =[centroidx(i),centroidy(i),0,0]' ;

else

% Use the former state to predict the new centroid and volocity

predicted = A*actual(i-1,:)';

end

kfinit = 1;

Ppre = A*P*A' + Q;

K = Ppre*H'/(H*Ppre*H'+R);

actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';

P = (eye(4)-K*H)*Ppre;

% Plot the tracking rectangle after Kalman filtering -- red

hold on

rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth],'EdgeColor','r','LineWidth',1.5);

hold on

plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);

drawnow;

end

%use .avi file to detect motion.. take a video of disturbed crown etc..

I want to know how to use this code to detect multiple football players like above video.

ReplyDeleteOne of the method of doing this,

DeleteSuppose you are having a frame size of 50x50.

Now you can divide this frames in 4, 25x25 blocks & process them the same way as the above code is doing for a whole frame.

Although Apart from Kalman Filter Motion Detection there are various other algo also available for it. Which will use a slight of Training data on the video to detect the motion of object.