Motion Tracking/Detection in MATLAB using Kalman Filter

MATAB has great capabilities to process video & pictures. One of these capabilities we have tested is the motion detection in a pre-recorded video, using Kalman Filtering technique.

Here is an example video of Motion Tracking using  Kalman Filter:

Basic background of Kalman Filter:
The Kalman filter, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone. More formally, the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state. The filter is named for Rudolf (Rudy) E. Kálmán, one of the primary developers of its theory.
The Kalman filter has numerous applications, e.g, for guidance, navigation & control of vehicles, particularly aircraft and spacecraft.

Kalman Filter Algorithm
Kalman Filter Algorithm (courtsey: colorado.edu)
 MATLAB CODE:

%Code in the Bold, & comments in normal font

clear all; 
close all; 
clc
%% Read video into MATLAB using aviread
video = aviread('rhinos.AVI');

%'n' for calculating the number of frames in the video file
n = length(video);
% Calculate the background image by averaging the first 10 images
temp = zeros(size(video(1).cdata));
[M,N] = size(temp(:,:,1));
for i = 1:10
    temp = double(video(i).cdata) + temp;
end
imbkg = temp/10;

% Initialization step for Kalman Filter
centroidx = zeros(n,1);
centroidy = zeros(n,1);
predicted = zeros(n,4);
actual = zeros(n,4);

% % Initialize the Kalman filter parameters
% R - measurement noise,
% H - transform from measure to state
% Q - system noise,
% P - the status covarince matrix
% A - state transform matrix

R=[[0.2845,0.0045]',[0.0045,0.0455]'];
H=[[1,0]',[0,1]',[0,0]',[0,0]'];
Q=0.01*eye(4);
P = 100*eye(4);
dt=1;
A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

% loop over all image frames in the video
kfinit = 0;
th = 38;
for i=1:n
  imshow(video(i).cdata);
  hold on
  imcurrent = double(video(i).cdata);
 
 
% Calculate the difference image to extract pixels with more than 40(threshold) change
  diffimg = zeros(M,N);
  diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

 
% Label the image and mark
  labelimg = bwlabel(diffimg,4);
  markimg = regionprops(labelimg,['basic']);
  [MM,NN] = size(markimg);

 
% Do bubble sort (large to small) on regions in case there are more than 1
  % The largest region is the object (1st one)
  for nn = 1:MM
      if markimg(nn).Area > markimg(1).Area
          tmp = markimg(1);
          markimg(1)= markimg(nn);
          markimg(nn)= tmp;
      end
  end

 
% Get the upper-left corner, the measurement centroid and bounding window size
  bb = markimg(1).BoundingBox;
  xcorner = bb(1);
  ycorner = bb(2);
  xwidth = bb(3);
  ywidth = bb(4);
  cc = markimg(1).Centroid;
  centroidx(i)= cc(1);
  centroidy(i)= cc(2);

 
% Plot the rectangle of background subtraction algorithm -- blue
  hold on
  rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
  hold on
  plot(centroidx(i),centroidy(i), 'bx');

 
% Kalman window size
  kalmanx = centroidx(i)- xcorner;
  kalmany = centroidy(i)- ycorner;

  if kfinit == 0
     
% Initialize the predicted centroid and volocity
      predicted =[centroidx(i),centroidy(i),0,0]' ;
  else
     
% Use the former state to predict the new centroid and volocity
      predicted = A*actual(i-1,:)';
  end
  kfinit = 1;

  Ppre = A*P*A' + Q;
  K = Ppre*H'/(H*Ppre*H'+R);
  actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
  P = (eye(4)-K*H)*Ppre;

  % Plot the tracking rectangle after Kalman filtering -- red
  hold on
rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth], 'EdgeColor', 'r','LineWidth',1.5);
  hold on
  plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
  drawnow;
end
%end of the code

NOTE:
 The above code sometimes generate the following error & warning, in the initial step itself (while reading the .avi file using "aviread" function.

Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead.
> In aviread at 26
  In motionnew at 4
Error using aviread (line 80)
Unable to locate decompressor to decompress video stream

Error in motionnew (line 4)
video = aviread('traffic.avi');

# First warning is due to the fact, that aviread  function has became obsolete & MATLAB team decided to replace it with a new function called, VideoReader.
# In order to avoid the second error, you need to install the avi video codec, the best thing available in the internet is the K-Lite Mega Codec Pack.

~Team Digital iVision Lab.
# do comment for any code request, help or queries.

52 comments:

  1. i have install k-lite codec mega but still not support the aviread

    ReplyDelete
  2. Try with "rhinos.AVI" first, if it is working. Then you should find the video file which has the same codec compression done as with rhinos.AVI. It will surely work! Other wise use VideoReader function. Use "getfourcc" command to view all the supported codecs in your MATLAB version

    ReplyDelete
    Replies
    1. cannot use the getfourcc.. i use R2013a matlab version... now i am doing final project about "vehicles counting system" but its not working... i have use ur code but still cannot support the aviread... when use mmreader then the mistake at :
      temp = zeros(size(video(1).cdata));

      Delete
  3. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Try Converting the file, in MPG or ASF format (use Format Factory software for this). And use VideoReader Function instead, I hope that will surely work. In my system the above code is working fine, without any error. The problem with "aviread" is, that it is now an obsolete function. In the next 2 or 3 releases it will be removed.

      Delete
  4. for the error you have mentioned: now the error when use "rhinos.AVI"
    ??? Error: File: aviread.m Line: 93 Column: 22
    Unexpected MATLAB expression.
    You are likely missing a ')', ']' or '}' or similar.

    ReplyDelete
  5. convert to swf and mpg also done... so the error still at temp = zeros(size(video(1).cdata));
    to get the codecs that can run "aviread" is not easy..it ok then.. maybe you use the old version of codecs or matlab..

    ReplyDelete
    Replies
    1. One last thing you can still try is, Extracting & Saving frames from a Video file using Matlab (http://www.divilabs.com/2013/11/extracting-saving-of-frames-from-video.html) and then, Create A Video File From A Sequence Of Image Stored in a Folder, Using MATLAB (http://www.divilabs.com/2013/12/create-video-file-from-sequence-of.html). If a video file created by MATLAB, it must also be readable by MATLAB!


      Delete
    2. ya... can but from the code u given ... the video are stored in folder "snaps" where include frames that have been convert rite.. but to call back that video i used this.. video = VideoReader('\snaps\Video.avi'); it become false...

      Delete
    3. Change the current directory to "snaps", or copy the video file thus created from "snaps" to your current working directory! Then by just giving the command "video= VideoReader('Video.avi')" should work!

      Delete
    4. I Hope you have created back the video, from frames! Without it, it will not able to read!

      Delete
  6. My MATLAB version is 2012b, use this command, formats = VideoReader.getFileFormats()
    In my system it is giving the output as: (so they are supported), try in yours, it could work!
    Video File Formats:
    .asf - ASF File
    .asx - ASX File
    .avi - AVI File
    .m4v - MPEG-4 Video
    .mj2 - Motion JPEG2000
    .mov - QuickTime movie
    .mp4 - MPEG-4
    .mpg - MPEG-1
    .wmv - Windows Media Video

    ReplyDelete
    Replies
    1. Error using VideoReader/init (line 450)
      Unable to determine the codec required.

      Error in VideoReader (line 147)
      obj.init(fileName);

      Error in kalman (line 7)
      video = VideoReader('Video.AVI');

      Delete
    2. With this I can assume (not exactly though), that your matlab is not able to find the codecs of the video that are installed in your system. May be some configuration of MATLAB you have to alter so that it will look for the codec that are installed in your system!

      Delete
    3. an actually when to convert image to video this error appeared:

      Warning: No video frames were written to this file. The file may be invalid.
      > In VideoWriter.VideoWriter>VideoWriter.close at 307
      In pojek1 at 15

      Delete
    4. Its a Codec problem, try in other system or other MATLAB's version, it might work!

      Delete
  7. its ok...i already frust... still error
    No appropriate method, property, or field cdata for class VideoReader.

    Error in kalman (line 12)
    temp = zeros(size(video(1).cdata));

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. you can use this code for video :
      video = VideoReader('rhinos.AVI.AVI');
      vidFrames = read(video);
      nframes = get(video, 'numberOfFrames');
      temp = zeros(size(vidFrames(:,:,:,1)));
      [M,N] = size(vidFrames(:,:,1));
      for i = 1:10
      temp = double(vidFrames(:,:,:,i)) + temp;
      end
      imbkg = temp/10;

      % Initialization for Kalman Filtering
      centroidx = zeros(nframes,1);
      centroidy = zeros(nframes,1);
      predicted = zeros(nframes,4);
      actual = zeros(nframes,4);
      ....
      for i=1:nframes
      imshow(vidFrames(:,:,:,i));
      hold on
      imcurrent = double(vidFrames(:,:,:,i));

      Delete
  8. how to use this code for real time detection using web cam?I have tried but didn't get...

    ReplyDelete
    Replies
    1. This code was written to detect motion with a Pre Recorded Video.... Try to make a composite code using the above code & the code listed in http://www.divilabs.com/2013/11/recording-video-from-webcam-using.html
      Replace the video opening part in the above code, with the code from the link given, to directly access the webcam!
      It should work.....

      Delete
  9. Replies
    1. http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html
      Visit the link...n follow the steps, you will be able to use your videos...!

      Delete
  10. how can I modify code for multiple object tracking using kalman filter to view effect on each frame of a video due to each function call? referring to the Code which is available in R2013a- computer vision tool box. Pl. help.

    ReplyDelete
    Replies
    1. This code is actually for, multi moving object tracking..! So you can use it.

      Delete
  11. thanks sir. But how can I observe/see in different windows each and every variable's and called function's effect on each frame apart from videoPlayer() for mask and frame? I am not very good matlab programmer. Thats why i need the necessary modifications in available code. pl help.

    ReplyDelete
    Replies
    1. Well in order to see, whats going inside a function, you can make use of Breakpoints while in the MATLAB's editor window. What a breakpoint will do is, it will stop the execution at that point of code execution (or say it at that particular line). So that you can see, easily the effect of it.!

      Delete
    2. MATLAB has a separate window, known as "Workspace". Here you can see all the values of the variables, while execution, in case you have created the breakpoint!

      Delete
  12. we install codec pack.... but we can't get the output for long videos... what we do now..?

    ReplyDelete
    Replies
    1. What kind of error you are getting? Can you please elaborate?

      Delete
  13. ??? Error using ==> aviread at 84
    Unable to locate decompressor to decompress video
    stream

    Error in ==> dummy at 5
    video = aviread('as.avi');

    ReplyDelete
    Replies
    1. Please Refer this link for this problem to be resolved:
      http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

      Delete
  14. we want explanation for each lines

    % Calculate the background image by averaging the first 10 images
    temp = zeros(size(video(1).cdata));
    [M,N] = size(temp(:,:,1));
    for i = 1:10
    temp = double(video(i).cdata) + temp;
    end
    imbkg = temp/10;

    ReplyDelete
    Replies
    1. temp = zeros(size(video(1).cdata));--> creating an empty frame
      [M,N] = size(temp(:,:,1));--> getting the dimension of the frame & storing it into M & N

      for i = 1:10
      temp = double(video(i).cdata) + temp;
      end
      imbkg = temp/10;
      the above 4 are for averaging 10 frames...........

      Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2ve4uOZvg

      Delete
  15. can you briefly explain the codings?

    ReplyDelete
    Replies
    1. There is already Enough explanation in comments! If you want specifically a line of code to be explained then please comment with that!

      Delete
    2. we want explanation for each lines &also syntax
      diffimg = zeros(M,N);
      diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

      Delete
    3. diffimg = zeros(M,N);--> will create a image of all zero pixel value
      diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

      Read more: http://www.divilabs.com/2013/11/motion-trackingdetection-in-matlab.html#ixzz2xwFFBpie

      They all will be used to subtract the RGB frames from the background!

      Delete
  16. Is it default value....?
    R=[[0.2845,0.0045]',[0.0045,0.0455]'];
    H=[[1,0]',[0,1]',[0,0]',[0,0]'];
    Q=0.01*eye(4);
    P = 100*eye(4);
    dt=1;
    A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

    ReplyDelete
  17. If one can easily track multiobjects/single object with background subtraction method, whats the need of kalman filter?
    As KF fails in many other applications but works well only with tracking, whats that specific thing about KF which makes it so special for tracking?

    ReplyDelete
    Replies
    1. Kalman filter gives a running covariance as well as the filtered result about moving object. Also I have read somewhere that, Kalman filter finds the most optimum averaging factor for each consequent state. Also somehow remembers a little bit about the past states.
      Goto the link, for an equation based understanding of Kalman..... That will be enough to answer for your question
      bilgin.esme.org/BitsBytes/KalmanFilterforDummies.aspx

      Delete
  18. I am using MATLAB 2013a. I want to publish .m file in ppt and doc format. Output file is in D drive. I want to get images i.e.individual frames (i/p and processed) in video also in output file. But its not working. Its giving error in mdbpublish and open files.
    How do I get rid of this?

    ReplyDelete
    Replies
    1. Please give more detail about the error so that I could help..!

      Delete
  19. sir..why still cannot function when i already instal the codec... like this Error using aviread (line 80)
    Unable to locate decompressor to decompress video stream

    Error in digitalivision1 (line 5)
    video = aviread('pedestrian.AVI');

    ReplyDelete
    Replies
    1. For this problem resolution, I have written another article. That will help you.
      http://www.divilabs.com/2014/01/how-to-resolve-matlab-problem-with.html

      Delete
  20. Stil error like this.. can you explain 1st step what should i do...

    The closest match is: VideoReader
    in C:\Program
    Files\MATLAB\R2012a\toolbox\matlab\audiovideo\@VideoReader\VideoReader.m




    Error in Untitled4 (line 7)
    video = videoreader('pedestrian.AVI');

    ReplyDelete
    Replies
    1. Well this error is due to the fact that MATLAB uses a case-sensitive syntax, so instead of "videoreader" you have to use "VideoReader"!

      Delete
  21. still error...

    No appropriate method, property, or field cdata for class VideoReader.

    Error in digitalivision2 (line 9)
    temp = zeros(size(video(1).cdata));

    ReplyDelete
    Replies
    1. Please refer to the syntax of VideoReader.... Type "help VideoReader"..u will get idea

      Delete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Stil error like this.. can you explain 1st step what should i do...

    ???Index exceeds matrix dimensions.
    Error in ==> tracked at 62
    bb = markimg(1).BoundingBox;

    ReplyDelete