Motion Tracking/Detection in MATLAB using Kalman Filter

MATAB has great capabilities to process video & pictures. One of these capabilities we have tested is the motion detection in a pre-recorded video, using Kalman Filtering technique.

Here is an example video of Motion Tracking using  Kalman Filter:


Video for illustration Purpose Only
Basic background of Kalman Filter:
The Kalman filter, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone. More formally, the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state. The filter is named for Rudolf (Rudy) E. Kálmán, one of the primary developers of its theory.
The Kalman filter has numerous applications, e.g, for guidance, navigation & control of vehicles, particularly aircraft and spacecraft.

Kalman Filter Algorithm
Kalman Filter Algorithm (courtsey: colorado.edu)
 MATLAB CODE:

%See the below code update of 11/11/2014 for implementing the code through VideoReader instead of aviread. Now with this update you can solve many of your problems of using your own video for this code.

%Code in the Bold, & comments in normal font

clear all; 
close all; 
clc
%% Read video into MATLAB using aviread
video = aviread('rhinos.AVI');

%'n' for calculating the number of frames in the video file
n = length(video);
% Calculate the background image by averaging the first 10 images
temp = zeros(size(video(1).cdata));
[M,N] = size(temp(:,:,1));
for i = 1:10
    temp = double(video(i).cdata) + temp;
end
imbkg = temp/10;

% Initialization step for Kalman Filter
centroidx = zeros(n,1);
centroidy = zeros(n,1);
predicted = zeros(n,4);
actual = zeros(n,4);

% % Initialize the Kalman filter parameters
% R - measurement noise,
% H - transform from measure to state
% Q - system noise,
% P - the status covarince matrix
% A - state transform matrix

R=[[0.2845,0.0045]',[0.0045,0.0455]'];
H=[[1,0]',[0,1]',[0,0]',[0,0]'];
Q=0.01*eye(4);
P = 100*eye(4);
dt=1;
A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

% loop over all image frames in the video
kfinit = 0;
th = 38;
for i=1:n
  imshow(video(i).cdata);
  hold on
  imcurrent = double(video(i).cdata);
 
 
% Calculate the difference image to extract pixels with more than 40(threshold) change
  diffimg = zeros(M,N);
  diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

 
% Label the image and mark
  labelimg = bwlabel(diffimg,4);
  markimg = regionprops(labelimg,['basic']);
  [MM,NN] = size(markimg);

 
% Do bubble sort (large to small) on regions in case there are more than 1
  % The largest region is the object (1st one)
  for nn = 1:MM
      if markimg(nn).Area > markimg(1).Area
          tmp = markimg(1);
          markimg(1)= markimg(nn);
          markimg(nn)= tmp;
      end
  end

 
% Get the upper-left corner, the measurement centroid and bounding window size
  bb = markimg(1).BoundingBox;
  xcorner = bb(1);
  ycorner = bb(2);
  xwidth = bb(3);
  ywidth = bb(4);
  cc = markimg(1).Centroid;
  centroidx(i)= cc(1);
  centroidy(i)= cc(2);

 
% Plot the rectangle of background subtraction algorithm -- blue
  hold on
  rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
  hold on
  plot(centroidx(i),centroidy(i), 'bx');

 
% Kalman window size
  kalmanx = centroidx(i)- xcorner;
  kalmany = centroidy(i)- ycorner;

  if kfinit == 0
     
% Initialize the predicted centroid and volocity
      predicted =[centroidx(i),centroidy(i),0,0]' ;
  else
     
% Use the former state to predict the new centroid and volocity
      predicted = A*actual(i-1,:)';
  end
  kfinit = 1;

  Ppre = A*P*A' + Q;
  K = Ppre*H'/(H*Ppre*H'+R);
  actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
  P = (eye(4)-K*H)*Ppre;

  % Plot the tracking rectangle after Kalman filtering -- red
  hold on
rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth], 'EdgeColor', 'r','LineWidth',1.5);
  hold on
  plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
  drawnow;
end
%end of the code
%Copyright (c) 2012, Xing
%All rights reserved.

NOTE:
 The above code sometimes generate the following error & warning, in the initial step itself (while reading the .avi file using "aviread" function.

Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead.
> In aviread at 26
  In motionnew at 4
Error using aviread (line 80)
Unable to locate decompressor to decompress video stream

Error in motionnew (line 4)
video = aviread('traffic.avi');

# First warning is due to the fact, that aviread  function has became obsolete & MATLAB team decided to replace it with a new function called, VideoReader.
# In order to avoid the second error, you need to install the avi video codec, the best thing available in the internet is the K-Lite Mega Codec Pack.

~Team Digital iVision Lab.
# do comment for any code request, help or queries.


Original Unmodified Source can be downloaded here.

If having problem with aviread or mmread function kindly visit the following post.
How to resolve MATLAB Problem With "aviread" or "VideoReader" command, while reading the avi file?



UPDATE : 20/10/2014

Some frequent errors Faced by users, & Solutions.

(1)Cannot find an exact (case-sensitive) match for 'videoReader'
The closest match is: VideoReader in
C:\ProgramFiles\MATLAB\R2013a\toolbox\matlab\audiovideo\@VideoReader\VideoReader.m

Solution: You need to use VideoReader instead of videoReader or VideoReader.

(2)No appropriate method, property, or field cdata for class VideoReader
Solution: Since the above code is implemented for aviread function, it will not be same as for VideoReader class implementation.
(3) Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead. 
> In aviread at 26 
Solution: The warning is due to the fact that aviread command has become obsolete & it will be removed from further version.
(4)Error using aviread (line 140)
The AVI file must be 8-bit Indexed or grayscale images, 16-bit grayscale, or 24-bit TrueColor images
Solution: This error is occuring due after implementing the FFMPEG decompression. This is due to the fact that aviread supports only those files only which are 8-bit indexed image or grayscale image. In modern world generally the video are much enhanced & their frames are having more color depth, so this error occurs while using them, even after decompressing step using ffmpeg.


UPDATE : 11/11/2014 (Major Code Update)
Many user are facing problems with the above code, that have been written using "aviread" funciton. So I am re-editing this code to be used using VideoReader class, so that you can easily use it with your own videos.

%Code starts here
clc;
close all;
clear all;
video = VideoReader('rhinos.avi');
%in place of aviread
%nframes = length(video);
nframes=video.NumberOfFrames;
for i=1:nframes
mov(i).cdata=read(video,i) 
%creating '.cdata' field to avoid much changes to previous code
end
 
temp = zeros(size(mov(1).cdata));
[M,N] = size(temp(:,:,1));
for i = 1:10
temp = double(mov(i).cdata) + temp;
end
imbkg = temp/10;
centroidx = zeros(nframes,1);
centroidy = zeros(nframes,1);
predicted = zeros(nframes,4);
actual = zeros(nframes,4);
R=[[0.2845,0.0045]',[0.0045,0.0455]'];
H=[[1,0]',[0,1]',[0,0]',[0,0]'];
Q=0.01*eye(4);
P = 100*eye(4);
dt=1;
A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];
kfinit = 0;
th = 38;
for i=1:nframes
imshow(mov(i).cdata);
hold on
imcurrent = double(mov(i).cdata);
diffimg = zeros(M,N);
diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);
labelimg = bwlabel(diffimg,4);
markimg = regionprops(labelimg,['basic']);
[MM,NN] = size(markimg);
for nn = 1:MM
if markimg(nn).Area > markimg(1).Area
tmp = markimg(1);
markimg(1)= markimg(nn);
markimg(nn)= tmp;
end
end
bb = markimg(1).BoundingBox;
xcorner = bb(1);
ycorner = bb(2);
xwidth = bb(3);
ywidth = bb(4);
cc = markimg(1).Centroid;
centroidx(i)= cc(1);
centroidy(i)= cc(2);
hold on
rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
hold on
plot(centroidx(i),centroidy(i), 'bx');
kalmanx = centroidx(i)- xcorner;
kalmany = centroidy(i)- ycorner;

if kfinit == 0
predicted =[centroidx(i),centroidy(i),0,0]' ;
else
predicted = A*actual(i-1,:)';
end
kfinit = 1;
Ppre = A*P*A' + Q;
K = Ppre*H'/(H*Ppre*H'+R);
actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
P = (eye(4)-K*H)*Ppre;
hold on

 rectangle('Position',[(actual(i,1)-kalmanx)...
(actual(i,2)-kalmany) xwidth ywidth],'EdgeColor','r','LineWidth',1.5);
hold on
plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
drawnow;
end




130 comments:

  1. i have install k-lite codec mega but still not support the aviread

    ReplyDelete
  2. Try with "rhinos.AVI" first, if it is working. Then you should find the video file which has the same codec compression done as with rhinos.AVI. It will surely work! Other wise use VideoReader function. Use "getfourcc" command to view all the supported codecs in your MATLAB version

    ReplyDelete
    Replies
    1. cannot use the getfourcc.. i use R2013a matlab version... now i am doing final project about "vehicles counting system" but its not working... i have use ur code but still cannot support the aviread... when use mmreader then the mistake at :
      temp = zeros(size(video(1).cdata));

      Delete
  3. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Try Converting the file, in MPG or ASF format (use Format Factory software for this). And use VideoReader Function instead, I hope that will surely work. In my system the above code is working fine, without any error. The problem with "aviread" is, that it is now an obsolete function. In the next 2 or 3 releases it will be removed.

      Delete
  4. for the error you have mentioned: now the error when use "rhinos.AVI"
    ??? Error: File: aviread.m Line: 93 Column: 22
    Unexpected MATLAB expression.
    You are likely missing a ')', ']' or '}' or similar.

    ReplyDelete
  5. convert to swf and mpg also done... so the error still at temp = zeros(size(video(1).cdata));
    to get the codecs that can run "aviread" is not easy..it ok then.. maybe you use the old version of codecs or matlab..

    ReplyDelete
    Replies
    1. One last thing you can still try is, Extracting & Saving frames from a Video file using Matlab (http://www.divilabs.com/2013/11/extracting-saving-of-frames-from-video.html) and then, Create A Video File From A Sequence Of Image Stored in a Folder, Using MATLAB (http://www.divilabs.com/2013/12/create-video-file-from-sequence-of.html). If a video file created by MATLAB, it must also be readable by MATLAB!


      Delete
    2. ya... can but from the code u given ... the video are stored in folder "snaps" where include frames that have been convert rite.. but to call back that video i used this.. video = VideoReader('\snaps\Video.avi'); it become false...

      Delete
    3. Change the current directory to "snaps", or copy the video file thus created from "snaps" to your current working directory! Then by just giving the command "video= VideoReader('Video.avi')" should work!

      Delete
    4. I Hope you have created back the video, from frames! Without it, it will not able to read!

      Delete
  6. My MATLAB version is 2012b, use this command, formats = VideoReader.getFileFormats()
    In my system it is giving the output as: (so they are supported), try in yours, it could work!
    Video File Formats:
    .asf - ASF File
    .asx - ASX File
    .avi - AVI File
    .m4v - MPEG-4 Video
    .mj2 - Motion JPEG2000
    .mov - QuickTime movie
    .mp4 - MPEG-4
    .mpg - MPEG-1
    .wmv - Windows Media Video

    ReplyDelete
    Replies
    1. Error using VideoReader/init (line 450)
      Unable to determine the codec required.

      Error in VideoReader (line 147)
      obj.init(fileName);

      Error in kalman (line 7)
      video = VideoReader('Video.AVI');

      Delete
    2. With this I can assume (not exactly though), that your matlab is not able to find the codecs of the video that are installed in your system. May be some configuration of MATLAB you have to alter so that it will look for the codec that are installed in your system!

      Delete
    3. an actually when to convert image to video this error appeared:

      Warning: No video frames were written to this file. The file may be invalid.
      > In VideoWriter.VideoWriter>VideoWriter.close at 307
      In pojek1 at 15

      Delete
    4. Its a Codec problem, try in other system or other MATLAB's version, it might work!

      Delete
  7. its ok...i already frust... still error
    No appropriate method, property, or field cdata for class VideoReader.

    Error in kalman (line 12)
    temp = zeros(size(video(1).cdata));

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. you can use this code for video :
      video = VideoReader('rhinos.AVI.AVI');
      vidFrames = read(video);
      nframes = get(video, 'numberOfFrames');
      temp = zeros(size(vidFrames(:,:,:,1)));
      [M,N] = size(vidFrames(:,:,1));
      for i = 1:10
      temp = double(vidFrames(:,:,:,i)) + temp;
      end
      imbkg = temp/10;

      % Initialization for Kalman Filtering
      centroidx = zeros(nframes,1);
      centroidy = zeros(nframes,1);
      predicted = zeros(nframes,4);
      actual = zeros(nframes,4);
      ....
      for i=1:nframes
      imshow(vidFrames(:,:,:,i));
      hold on
      imcurrent = double(vidFrames(:,:,:,i));

      Delete
  8. how to use this code for real time detection using web cam?I have tried but didn't get...

    ReplyDelete
    Replies
    1. This code was written to detect motion with a Pre Recorded Video.... Try to make a composite code using the above code & the code listed in http://www.divilabs.com/2013/11/recording-video-from-webcam-using.html
      Replace the video opening part in the above code, with the code from the link given, to directly access the webcam!
      It should work.....

      Delete
    2. Could you just help us out with this problem by making this compact code for motion detection using webcam video

      Delete
  9. Replies
    1. http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html
      Visit the link...n follow the steps, you will be able to use your videos...!

      Delete
  10. how can I modify code for multiple object tracking using kalman filter to view effect on each frame of a video due to each function call? referring to the Code which is available in R2013a- computer vision tool box. Pl. help.

    ReplyDelete
    Replies
    1. This code is actually for, multi moving object tracking..! So you can use it.

      Delete
  11. thanks sir. But how can I observe/see in different windows each and every variable's and called function's effect on each frame apart from videoPlayer() for mask and frame? I am not very good matlab programmer. Thats why i need the necessary modifications in available code. pl help.

    ReplyDelete
    Replies
    1. Well in order to see, whats going inside a function, you can make use of Breakpoints while in the MATLAB's editor window. What a breakpoint will do is, it will stop the execution at that point of code execution (or say it at that particular line). So that you can see, easily the effect of it.!

      Delete
    2. MATLAB has a separate window, known as "Workspace". Here you can see all the values of the variables, while execution, in case you have created the breakpoint!

      Delete
  12. we install codec pack.... but we can't get the output for long videos... what we do now..?

    ReplyDelete
    Replies
    1. What kind of error you are getting? Can you please elaborate?

      Delete
  13. ??? Error using ==> aviread at 84
    Unable to locate decompressor to decompress video
    stream

    Error in ==> dummy at 5
    video = aviread('as.avi');

    ReplyDelete
    Replies
    1. Please Refer this link for this problem to be resolved:
      http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

      Delete
  14. we want explanation for each lines

    % Calculate the background image by averaging the first 10 images
    temp = zeros(size(video(1).cdata));
    [M,N] = size(temp(:,:,1));
    for i = 1:10
    temp = double(video(i).cdata) + temp;
    end
    imbkg = temp/10;

    ReplyDelete
    Replies
    1. temp = zeros(size(video(1).cdata));--> creating an empty frame
      [M,N] = size(temp(:,:,1));--> getting the dimension of the frame & storing it into M & N

      for i = 1:10
      temp = double(video(i).cdata) + temp;
      end
      imbkg = temp/10;
      the above 4 are for averaging 10 frames...........

      Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2ve4uOZvg

      Delete
  15. can you briefly explain the codings?

    ReplyDelete
    Replies
    1. There is already Enough explanation in comments! If you want specifically a line of code to be explained then please comment with that!

      Delete
    2. we want explanation for each lines &also syntax
      diffimg = zeros(M,N);
      diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

      Delete
    3. diffimg = zeros(M,N);--> will create a image of all zero pixel value
      diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

      Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2xwFFBpie

      They all will be used to subtract the RGB frames from the background!

      Delete
  16. Is it default value....?
    R=[[0.2845,0.0045]',[0.0045,0.0455]'];
    H=[[1,0]',[0,1]',[0,0]',[0,0]'];
    Q=0.01*eye(4);
    P = 100*eye(4);
    dt=1;
    A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

    ReplyDelete
    Replies
    1. For more details about this: refer Fundamentals of Kalman Filtering: A Practical Approach by Paul Zarchan

      CHapter4 of whose has all the explanation about these params.....

      Delete
  17. If one can easily track multiobjects/single object with background subtraction method, whats the need of kalman filter?
    As KF fails in many other applications but works well only with tracking, whats that specific thing about KF which makes it so special for tracking?

    ReplyDelete
    Replies
    1. Kalman filter gives a running covariance as well as the filtered result about moving object. Also I have read somewhere that, Kalman filter finds the most optimum averaging factor for each consequent state. Also somehow remembers a little bit about the past states.
      Goto the link, for an equation based understanding of Kalman..... That will be enough to answer for your question
      bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

      Delete
  18. I am using MATLAB 2013a. I want to publish .m file in ppt and doc format. Output file is in D drive. I want to get images i.e.individual frames (i/p and processed) in video also in output file. But its not working. Its giving error in mdbpublish and open files.
    How do I get rid of this?

    ReplyDelete
    Replies
    1. Please give more detail about the error so that I could help..!

      Delete
  19. sir..why still cannot function when i already instal the codec... like this Error using aviread (line 80)
    Unable to locate decompressor to decompress video stream

    Error in digitalivision1 (line 5)
    video = aviread('pedestrian.AVI');

    ReplyDelete
    Replies
    1. For this problem resolution, I have written another article. That will help you.
      http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

      Delete
  20. Stil error like this.. can you explain 1st step what should i do...

    The closest match is: VideoReader
    in C:\Program
    Files\MATLAB\R2012a\toolbox\matlab\audiovideo\@VideoReader\VideoReader.m




    Error in Untitled4 (line 7)
    video = videoreader('pedestrian.AVI');

    ReplyDelete
    Replies
    1. Well this error is due to the fact that MATLAB uses a case-sensitive syntax, so instead of "videoreader" you have to use "VideoReader"!

      Delete
  21. still error...

    No appropriate method, property, or field cdata for class VideoReader.

    Error in digitalivision2 (line 9)
    temp = zeros(size(video(1).cdata));

    ReplyDelete
    Replies
    1. Please refer to the syntax of VideoReader.... Type "help VideoReader"..u will get idea

      Delete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Stil error like this.. can you explain 1st step what should i do...

    ???Index exceeds matrix dimensions.
    Error in ==> tracked at 62
    bb = markimg(1).BoundingBox;

    ReplyDelete
    Replies
    1. As the error message says, you are trying to access a position in 'bb' that does not exist. Do check you have written the code before this line (62) correctly. Also if u can, please tell the value of 'bb' variable till this step, as you can see in "Workspace".

      Delete
  24. Hello! where can I get free download video dataset for multiple objects tracking?
    why .flv videos are not read by videoreader in 2013a computer vision toolbox multiple objects tracking prog.?
    is width n height needed for video to be read by videoreader? then why error?

    ReplyDelete
    Replies
    1. .flv file is not readable by matlab. yes width & height is needed for sure, actually the video object you have created by VideoReader command itself contains its information. You need to convert the . flv file to either mpeg or avi. If still you have a problem please visit--> http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

      Delete
  25. The code runs fine as long as it has 'rhinos.avi'. I converted another video to avi format and replaced it with rhinos.avi. First the code was giving the same error about the decompressor. I downloaded the codec pack now it's giving me the following error
    ??? Out of memory. Type HELP MEMORY for your options.

    Error in ==> rot90 at 38
    B = A(n:-1:1,:);

    Error in ==> aviread at 123
    f = rot90(reshape(f, paddedWidth,height));

    Error in ==> usingfilter3 at 6
    video = aviread('Wildlife.AVI');
    Can you please help me with it?? I need urgent help! Will be quite thankful

    ReplyDelete
    Replies
    1. http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html
      Visit the link...n follow the steps, you will be able to use your videos...!

      Delete
  26. I tried that too, but still can't find the solution as it says :
    D:>'ffmpeg' is not recognized as an internal or external command, operable program or batch file.

    ReplyDelete
  27. can you please explain the following commands:
    % Get the upper-left corner, the measurement centroid and bounding window size
    bb = markimg(1).BoundingBox;
    xcorner = bb(1);
    ycorner = bb(2);
    xwidth = bb(3);
    ywidth = bb(4);
    cc = markimg(1).Centroid;
    centroidx(i)= cc(1);
    centroidy(i)= cc(2);

    ReplyDelete
    Replies
    1. bb = markimg(1).BoundingBox; will be having all the coordinates necessary for making the detection rectangle.
      xcorner = bb(1);
      ycorner = bb(2);
      xwidth = bb(3);
      ywidth = bb(4);
      will be used to draw the rectangle.
      cc = markimg(1).Centroid;
      centroidx(i)= cc(1);
      centroidy(i)= cc(2);
      for drawing the centroid in the output image
      Read more: http://www.divilabs.com/2014/06/left-shifting-histogram-histogram.html

      Delete
  28. Please explain the following
    % Kalman window size
    kalmanx = centroidx(i)- xcorner;
    kalmany = centroidy(i)- ycorner;
    if kfinit == 0
    % Initialize the predicted centroid and volocity
    predicted =[centroidx(i),centroidy(i),0,0]' ;
    else
    % Use the former state to predict the new centroid and volocity
    predicted = A*actual(i-1,:)';
    end

    ReplyDelete
    Replies
    1. kalmanx = centroidx(i)- xcorner;
      kalmany = centroidy(i)- ycorner;
      -->for the object in motion detection window size. i.e., the red square it is displaying while detection.

      if kfinit == 0
      predicted =[centroidx(i),centroidy(i),0,0]' ;
      else
      -->The above 3 line for the initial condition. I.e when the 1st frame of the video is encountered.

      predicted = A*actual(i-1,:)';
      end
      --> The velocity of the moving object & the new centroid will be predicted by these 2 lines.

      Delete
  29. can you please give a code for multiple non salient objects detection n tracking using particle filter?
    Where can I get A to Z info. about particle filter?

    ReplyDelete
    Replies
    1. You can refer various research paper available on internet (IEEE specifically). various tutorials are available on youtube explaining particle filter. This filter I havn't tried out yet, so can't tell about that much.

      Delete
  30. Hi, the given code only detects one object. How do you extend it to detect multiple objects as shown in the example vide?

    ReplyDelete
    Replies
    1. In this code, i have taken the average of frames of full size. You can divide the screen into various sections by fixing the coordinates (e.g. in openCV its done through cv::rect() ) & apply the processing there.

      Delete
  31. Hi Vibhutesh, please take a close look at the code you have published. It can detect only a single object. The video shown as example is misleading!

    ReplyDelete
    Replies
    1. I know about the code which is published here. The video is for illustration only, it shows how the Kalman filter can be utilized effectively. For more advance version of the code you need to refer various texts for this topics. OpenCV will be the best alternative for this.

      Delete
  32. I can't find The Sollution for this

    Error in kalmanmotion (line 61)
    bb = markimg(1).BoundingBox;

    ReplyDelete
    Replies
    1. What's the other line of error message is saying?

      Delete
  33. hello! I hav some doubts regarding KF.
    1. Why jitter and creep are not identified by KF?
    2. why KF is called precisely wiener filter for stationary least squares smoothing?
    3. Why KF is called as time varient Wiener filter?
    4.What is a local minima, which is a critical problem with wiener filter. How it is overcome in KF?

    ReplyDelete
    Replies
    1. Well I really want to give answers to these questions, but I also need to do much research on KF before saying anything about it. The above code is simply one of the application of KF!

      Delete
  34. I am using 2013a MATLAB. exploring multiobj tracking prog. in computer vision toolbox. How can i get coding for kalmanFilter object which is a library obj. there?
    I am not able to analyse results of predict() and correct() after each frame processing. how can i get coding of these library functions? pl. help.

    ReplyDelete
    Replies
    1. To analyze any line of code, what it does & what variable it creates is done through breakpoints. The obj version of the code is actually a compiled version, so a very less chance is there for you to decode it. Instead you should check for matlab central for the original source code for the same. It will surely help!

      Delete
  35. how can i modify matlab 2013a MTT code using particle filter instead of kalman filter?

    ReplyDelete
    Replies
    1. I have done some work under KF only, still to start with PF!

      Delete
  36. I am trying to implement your code for people detection. However, why only detect a single person not multiple people in a frame? do you know how to detect for multiple people? thank you

    ReplyDelete
    Replies
    1. The code is for Motion Detection. The code also require the background to be fixed. As this is an average based method, it will fail miserably if whole scene is in motion. For a perfect detection of motion the background need to be constant (non changing).

      Delete
  37. Okay, I have a static background. But I still cant track multiple people. only a single person is traced. Any idea how to track multiple people using Kalman Filter? I am a bit new with Kalman Filter.

    ReplyDelete
    Replies
    1. One thing that you can apply, that divide the frame to multiple blocks & process each block separately. Use the Kalman code as a function to track the motion on each block!

      Delete
    2. hi indratno same problem here can u tell me how u have solve this for multiple vehicle detection

      Delete
  38. can you sugggest me how to convert any video format to avi format

    ReplyDelete
    Replies
    1. You can use any video converter available. But to use them with matlab you need to use that codec for compression, that are supported & installed in your matlab!

      Delete
  39. clear all; close all; clc

    %% Read video into MATLAB using aviread
    video = aviread('1.AVI');
    nframes = length(video);

    % Calculate the background image by averaging the first 5 images
    temp = zeros(size(video(1).cdata));
    [M,N] = size(temp(:,:,1));
    for i = 1:10
    temp = double(video(i).cdata) + temp;
    end
    imbkg = temp/10;

    % Initialization for Kalman Filtering
    centroidx = zeros(nframes,1);
    centroidy = zeros(nframes,1);
    predicted = zeros(nframes,4);
    actual = zeros(nframes,4);

    % % Initialize the Kalman filter parameters
    % R - measurement noise,
    % H - transform from measure to state
    % Q - system noise,
    % P - the status covarince matrix
    % A - state transform matrix

    R=[[0.2845,0.0045]',[0.0045,0.0455]'];
    H=[[1,0]',[0,1]',[0,0]',[0,0]'];
    Q=0.01*eye(4);
    P = 100*eye(4);
    dt=1;
    A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

    % loop over all image frames in the video
    kfinit = 0;
    th = 38;
    for i=1:nframes
    imshow(video(i).cdata);
    hold on
    imcurrent = double(video(i).cdata);

    % Calculate the difference image to extract pixels with more than 40(threshold) change
    diffimg = zeros(M,N);
    diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
    | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
    | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

    % Label the image and mark
    labelimg = bwlabel(diffimg,4);
    markimg = regionprops(labelimg,['basic']);
    [MM,NN] = size(markimg);

    % Do bubble sort (large to small) on regions in case there are more than 1
    % The largest region is the object (1st one)
    for nn = 1:MM
    if markimg(nn).Area > markimg(1).Area
    tmp = markimg(1);
    markimg(1)= markimg(nn);
    markimg(nn)= tmp;
    end
    end

    % Get the upper-left corner, the measurement centroid and bounding window size
    bb = markimg(1).BoundingBox;
    xcorner = bb(1);
    ycorner = bb(2);
    xwidth = bb(3);
    ywidth = bb(4);
    cc = markimg(1).Centroid;
    centroidx(i)= cc(1);
    centroidy(i)= cc(2);

    % Plot the rectangle of background subtraction algorithm -- blue
    hold on
    rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
    hold on
    plot(centroidx(i),centroidy(i), 'bx');

    % Kalman window size
    kalmanx = centroidx(i)- xcorner;
    kalmany = centroidy(i)- ycorner;

    if kfinit == 0
    % Initialize the predicted centroid and volocity
    predicted =[centroidx(i),centroidy(i),0,0]' ;
    else
    % Use the former state to predict the new centroid and volocity
    predicted = A*actual(i-1,:)';
    end
    kfinit = 1;

    Ppre = A*P*A' + Q;
    K = Ppre*H'/(H*Ppre*H'+R);
    actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
    P = (eye(4)-K*H)*Ppre;

    % Plot the tracking rectangle after Kalman filtering -- red
    hold on
    rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth],'EdgeColor','r','LineWidth',1.5);
    hold on
    plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
    drawnow;
    end
    %use .avi file to detect motion.. take a video of disturbed crown etc..

    ReplyDelete
  40. I want to know how to use this code to detect multiple football players like above video.

    ReplyDelete
    Replies
    1. One of the method of doing this,
      Suppose you are having a frame size of 50x50.
      Now you can divide this frames in 4, 25x25 blocks & process them the same way as the above code is doing for a whole frame.
      Although Apart from Kalman Filter Motion Detection there are various other algo also available for it. Which will use a slight of Training data on the video to detect the motion of object.

      Delete
    2. In which part that must be divide as to get the multiple detection..please help me sir

      Delete
  41. Sir. Can u give a code for object recognition and can we use a object detection in survillance?

    ReplyDelete
    Replies
    1. For the object detection, use any available.
      (1)DPM by Felzenszwalb (download at www[.]cs[.]berkeley.edu/~rbg/latent/) or
      (2)Pistor. Dollar's Toolbox are both good. (download at vision.ucsd.edu/~pdollar/toolbox/doc/)
      The second one is much faster.

      Delete
    2. They all are open source, You just need to give citation to their work if publishing somewhere.

      Delete
  42. warning off;
    clear all;
    close all;
    clc;



    Xa=0;
    Possibility=20;
    while Xa~=Possibility
    Xa=menu('VIDEO TRACKING','CROWD OBJECT TRACKING','OBJECT TRACKING','CLOSE');

    if Xa==1

    ReadVideo = mmreader('1.flv', 'tag', 'myreader1');

    % Read in all video frames
    ReadObject = read(ReadVideo);

    % Get the number of frames.
    TotalFrames = get(ReadVideo, 'NumberOfFrames');
    Frames=15;
    for k = 1 :Frames
    mov(k).cdata = ReadObject(:,:,:,k);
    mov(k).colormap = [];
    end

    hf = figure;
    set(hf, 'position', [150 150 ReadVideo.Width ReadVideo.Height])
    movie(hf, mov, 1, ReadVideo.FrameRate);
    figure,
    for kk=1:Frames
    X=mov(kk).cdata;
    Stc='.jpg';
    Img=strcat(num2str(kk),Stc);
    imwrite(X,Img);
    imshow(X);
    pause(.1);
    end

    pause(2);
    figure,
    for ab=1:10
    subplot(1,3,1);

    Imge=strcat(num2str(ab),Stc);

    image=double(imread(Imge));
    Xe=imread(Imge);
    imshow(Xe);
    title('Input Frame');
    pause(.1);
    [maps,images]=MovingRegion(image);
    subplot(1,3,2);
    Edge=Segmentation(images,maps);
    title('Moving Pixel Compensation');
    pause(.1);
    subplot(1,3,3);
    imshow(Edge)
    title('Otsu Threshold Frame');
    pause(.1);
    end
    close all;
    cd src;
    addpath(pwd);
    cd lap;
    addpath(pwd);
    cd ..;
    cd suptitle;
    addpath(pwd);
    cd ../..;
    Fr = 'E:\M.E\PROJECt\Code_16_11_14\FRAME';
    x=imread('1.jpg');
    y=imread('2.jpg');
    % Load the data
    data = load_data(Fr,'jpg');
    mixparam = mixture_parameters();

    % Load parameters for Kalman tracking
    kalparam = KalmanFilterTracking();

    clear pixel_mixture_stat.m;
    clear add_new_hypotheses.m;
    [frames, object_hist] = track(data, mixparam, kalparam);
    [psn] = Psnr(x,y);
    disp('The PSNR Value ...');
    disp(psn);
    end

    if Xa==2

    cd('E:\M.E\PROJECT\Code_16_11_14\DATA');
    figure,
    ST='.jpg';
    for k=1:60;
    Bcx=num2str(k);
    Bcc=strcat(Bcx,ST);
    Abc=imread(Bcc);
    imshow(Abc);
    title('Frames');

    pause(.25);
    end
    cd('E:\M.E\PROJECT\Code_16_11_14');

    x=imread('59.jpg');
    y=imread('60.jpg');
    [psn] = Psnr(x,y);
    disp('The PSNR Value ...');
    disp(psn);

    % compute the background image
    Imzero = zeros(240,320,3);
    for i = 1:5
    Im{i} = double(imread(['DATA/',int2str(i),'.jpg']));
    Imzero = Im{i}+Imzero;
    end
    Imback = Imzero/5;
    [MR,MC,Dim] = size(Imback);

    % loop over all images
    for i = 1 : 601
    % load image
    Im = (imread(['DATA/',int2str(i), '.jpg']));
    imshow(Im)
    Imwork = double(Im);

    [cc(i),cr(i),radius,flag] = Extraction(Imwork,Imback,i);
    if flag==0
    continue
    end
    hold on
    for c = -radius: radius/20 : radius
    r = sqrt(radius^2-c^2);
    plot(cc(i)+c,cr(i)+r,'w')
    plot(cc(i)+c,cr(i)-r,'w')
    end
    pause(0.2)
    title('Moving Object Tracking');
    end
    end

    if Xa==3
    return;
    end
    end

    above is whether correct or not for video tracking.if not means plz put correct coing.i want accurate tracking in my pg project sir.

    ReplyDelete
    Replies
    1. Right Now I am busy with my exams, I will get to you back in 1st week of December.

      Delete
    2. If you want me to review, share the project folder with a dropbox invite to vibhutesh [at] gmail.com. As it is very difficult to visualize the code without the sample data & image/video.

      Delete
  43. With what video did you try the code? Can you share the video? The file name is rhinos.avi and the sample video shown is football players tracking.

    ReplyDelete
    Replies
    1. The above football players video is for illustration only, to show the power of Kalman Filter.
      I have used "rhinos.avi" video file which comes with any MATLAB installation. Just try to open it with VideoReader command, if some error, try reinstalling your MATLAB with the full installing option.

      Delete
  44. Thanks Vibhutesh. I am trying to run this program with the following video
    https://www.youtube.com/watch?v=8XtX5Z5vyMs

    But I am getting the following error

    Error using VideoReader/read (line 145)
    The frame range requested is beyond the end of the file.

    Error in Kalman_Obj_Track (line 8)
    vidFrames = read(video);

    ReplyDelete
    Replies
    1. So its, VideoReader function is throwing an error. What Simply meant from the error is that you are trying to access the frame which is not even present. Kindly have a look at the looping part.

      Delete
    2. I understand it. But how to avoid it? I get nframes = 4964 and in the loop it is not even working for i = 1,
      I mean
      mov(1).cdata = read(video,1) ; throws an error.

      Can it be because of the video?

      Delete
  45. HI
    I am following your source, when i run this file i got this error
    ??? Error using ==> aviread at 76
    Unable to locate decompressor to decompress video stream

    Error in ==> final at 5
    video = aviread(a); %in place of aviread

    but on top your are provide link to solve this error from that links i downloaded getfourcc.m file but am not understand how can i interface this file to the main file, can you plz support me

    ReplyDelete
    Replies
    1. Just use the updated version of the code. Aviread has lot of complications in terms of codecs so the updated code has replaced (given below the older one), aviread to VideoReader.

      Delete
  46. This comment has been removed by the author.

    ReplyDelete
  47. Hi, I am implementing the algorithm and code in FPGA. Do you have C code for the MATLAB code?
    Incase you have share it with me. Will be really useful

    ReplyDelete
    Replies
    1. Sorry I don't have. But the thing you are doing is not new, so you need to search internet or especially github for any source code related to it. I am sure you will find it.

      Delete
  48. thank u for ur code and i have one request for u.could u plz refer the code for extracting the moving object from the video

    ReplyDelete
    Replies
    1. When you got a positive detection, you build a bounding box. That coordinate's bounding box can be used to extract the moving object. Refer ROI bounding box in matlab image processing toolbox, for more help.

      Delete
  49. sir, can you provide me the matlab code for human motion detection in real time using webcam?
    i really need it. reply soon sir

    ReplyDelete
    Replies
    1. The above code will detect anything moving in a video. But the condition is, there should be only one object moving. For human detection you will find multitude of examples in matlabcentral or if you are comfortable with C++ you should find while googling, various exact codes that you will need.

      Delete
  50. Sir, i have another code in matlab for multiple human motion detection which i found on matlab central(suggested by you). But i am having problem in linking up multiple files. So i request you to provide me the solution for this. I have mailed the link of code. Reply soon sir, Thank you

    ReplyDelete
  51. For the code you sent via the links, they will require "computer vision system toolbox". So install the matlab with this tool box, then you will be able to do that.

    ReplyDelete
  52. Thank you sir. I will reply after doing this.

    ReplyDelete
  53. Video above, whosoever have created this, was having the GT Data (ground truth) with him. He havn't applied any of the detection scheme & build a bounding box across the target with the help of GT.
    This code is just for a single target detection.

    ReplyDelete
  54. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. I may able to guide & suggest. But exclusive coding implementation, I may not. Since the schedule is very tight. Email me at vibhutesh [at] gmail.com

      Delete
  55. I am trying to develop code for specific person detection in a video. Previously i have implemented multiple object tracking. What should I do for this?
    Whether Probability Density Function (PDF) will help me?

    ReplyDelete
    Replies
    1. Detection based tracker will do the work in your case. First train the detector with your sample & then apply detection based tracking frame by frame.

      Delete
  56. Sir I'm getting error like this...will u give any solution for it?

    Index exceeds matrix dimensions.

    Error in ==> tracked at 62
    bb = markimg(1).BoundingBox;

    ReplyDelete
    Replies
    1. @Unknown....Read the above commenting thread, you will surely get an idea about your error.

      Delete
  57. thank you so much for the code. it's so help me to finish my project. i will write down your name in my report for my special thanks to you sir.

    ReplyDelete
  58. Im getting out of memory error while running the self developed video

    ReplyDelete
    Replies
    1. It looks codec problem. You can re-encode the video in MATLAB's supported video formats. Then it might not show this error.

      Delete
  59. can you please change this code for tracking multiple objects using kalman filter

    ReplyDelete
  60. Can you please change this code for tracking multiple objects using kalman filter???

    ReplyDelete
  61. how to do it in real time, i mean tack bject from usb camera

    ReplyDelete
  62. Hello Sir,how to change your code to tracking object from camera(real time) by kalman

    ReplyDelete