Extracting & Saving frames from a Video file using Matlab Code

A video file consists of frames. These frames when appear before us in a rate more than our perception of vision, gives a sensation of an object moving before us, by looking just at the screen on which frames are appearing at high rate.
Thus one can say that frames are the fundamental entity of a video file.

In this project we are opening one video file, & grabbing each video frame. At last we are saving them frame by frame as an image file, in the folder "snaps" in the working directory.

OUTPUT AT MATLAB COMMAND PROMPT while executing the code for Extracting & Saving of frames from a Video file through Matlab.
Extracting & Saving of frames from a Video file through Matlab Code, MATLAB Command Window Output
Extracting & Saving of frames from a Video file through Matlab Code, MATLAB Command Window Output
Screen Shot of the frames OUTPUT AT The Folder while executing the code for Extracting & Saving of frames from a Video file through Matlab.
The Folder At Which The Frames Are Saved

Above Screen Shot of the frames OUTPUT AT The Folder while executing the code for Extracting & Saving of frames from a Video file through Matlab.

Code:

%%Extracting & Saving of frames from a Video file through Matlab Code%%
clc;
close all;
clear all;


% assigning the name of sample avi file to a variable
filename = 'rhinos.avi';

%reading a video file
mov = VideoReader(filename);

% Defining Output folder as 'snaps'
opFolder = fullfile(cd, 'snaps');
%if  not existing
if ~exist(opFolder, 'dir')
%make directory & execute as indicated in opfolder variable
mkdir(opFolder);
end


%getting no of frames
numFrames = mov.NumberOfFrames;

%setting current status of number of frames written to zero
numFramesWritten = 0;

%for loop to traverse & process from frame '1' to 'last' frames
for t = 1 : numFrames
currFrame = read(mov, t);   
%reading individual frames
opBaseFileName = sprintf('%3.3d.png', t);
opFullFileName = fullfile(opFolder, opBaseFileName);
imwrite(currFrame, opFullFileName, 'png');  
%saving as 'png' file
%indicating the current progress of the file/frame written
progIndication = sprintf('Wrote frame %4d of %d.', t, numFrames);
disp(progIndication);
numFramesWritten = numFramesWritten + 1;
end     
%end of 'for' loop
progIndication = sprintf('Wrote %d frames to folder "%s"',numFramesWritten, opFolder);
disp(progIndication);

%End of the code

Additional MATLAB related Posts That You May Like!

# Implementation of RSA Cryptography Algorithm Using MATLAB Code
# Recording a Video From Webcam Using MATLAB Code 
# Accessing the Serial Port using MATLAB Code: Serial communication through MATLAB
# Accessing Webcam Through MATLAB Code
# Motion Tracking/Detection in MATLAB using Kalman Filter
# Optimizing MATLAB Code
# Controlling Mouse Pointer/Cursor Using MATLAB Code
# Controlling Keyboard Keys (Pressing & Releasing) with Matlab code 
# Starting MATLAB without GUI through Command Prompt/LxTERMINAL
# Some MATLAB Unusual Commands, That You Must Know ( Just For Fun! ) Part 2
# Some MATLAB Unusual Commands, That You Must Know ( Just For Fun! ) Part 1 

6 comments:

Arduino Board Yún

Arduino Yún Front
Arduino Yún Front
Arduino Yún Back
Arduino Yún Back

Overview

The Arduino Yún is a microcontroller board based on the ATmega32u4 (datasheet) and the Atheros AR9331. The Atheros processor supports a Linux distribution based on OpenWRT named Linino. The board has built-in Ethernet and WiFi support, a USB-A port, micro-SD card slot, 20 digital input/output pins (of which 7 can be used as PWM outputs and 12 as analog inputs), a 16 MHz crystal oscillator, a micro USB connection, an ICSP header, and a 3 reset buttons.
Note : In some countries, it is prohibited to sell WiFi enabled devices without government approval. While waiting for proper certification, some local distributors are disabling WiFi functionality. Check with your dealer before purchasing a Yún if you believe you may live in such a country. If you wish to disable WiFi, run this sketch. For more information, refer to this forum post. 

Arduino Yún Different Parts Indicated
Arduino Yún Different Parts Indicated
The Yún distinguishes itself from other Arduino boards in that it can communicate with the Linux distribution onboard, offering a powerful networked computer with the ease of Arduino. In addition to Linux commands like cURL, you can write your own shell and python scripts for robust interactions.
The Yún is similar to the Leonardo in that the ATmega32u4 has built-in USB communication, eliminating the need for a secondary processor. This allows the Yún to appear to a connected computer as a mouse and keyboard, in addition to a virtual (CDC) serial / COM port.

Arduino & Linux Environment Combined In Arduino Board Yún
Arduino & Linux Environment Combined In Arduino Board Yún

The Bridge library facilitates communication between the two processors, giving Arduino sketches the ability to run shell scripts, communicate with network interfaces, and receive information from the AR9331 processor. The USB host, network interfaces and SD card are not connected to the 32U4, but the AR9331, and the Bridge library also enables the Arduino to interface with those peripherals.

Summary

Because the Yún has two processors, the summary section shows the characteristics of each one in two separate tables.

AVR Arduino microcontroller
MicrocontrollerATmega32u4
Operating Voltage5V
Input Voltage5V
Digital I/O Pins20
PWM Channels7
Analog Input Channels12
DC Current per I/O Pin40 mA
DC Current for 3.3V Pin50 mA
Flash Memory32 KB (of which 4 KB used by bootloader)
SRAM2.5 KB
EEPROM1 KB
Clock Speed16 MHz
Linux microprocessor
ProcessorAtheros AR9331
ArchitectureMIPS @400MHz
Operating Voltage3.3V
EthernetIEEE 802.3 10/100Mbit/s
WiFiIEEE 802.11b/g/n
USB Type-A2.0 Host/Device
Card ReaderMicro-SD only
RAM64 MB DDR2
Flash Memory16 MB
PoE compatible 802.3af card support

Schematic & Reference Design

Schematic: arduino-Yun-schematic.pdf

Power

It is recommended to power the board via the micro-USB connection with 5VDC.
If you are powering the board though the Vin pin, you must supply a regulated 5VDC. There is no on-board voltage regulator for higher voltages, which will damage the board.
The Yún is also compatible with PoE power supply but in order to use this feature you need to mount a PoE module on the board or buy a preassembled one.
The power pins are as follows:
  • VIN. The input voltage to the Arduino board. Unlike other Arduino boards, if you are going to provide power to the board through this pin, you must provide a regulated 5V.
  • 5V. The power supply used to power the microcontrollers and other components on the board. This can come either from VIN or be supplied by USB.
  • 3V3. A 3.3 volt supply generated by the on-board regulator. Maximum current draw is 50 mA.
  • GND. Ground pins.
  • IOREF. The voltage at which the i/o pins of the board are operating (i.e. VCC for the board). This is 5V on the Yún.

Memory

The ATmega32u4 has 32 KB (with 4 KB used for the bootloader). It also has 2.5 KB of SRAM and 1 KB of EEPROM (which can be read and written with the EEPROM library).
The memory on the AR9331 is not embedded inside the processor. The RAM and the storage memory are externally connected. The Yún has 64 MB of DDR2 RAM and 16 MB of flash memory. The flash memory is preloaded in factory with a Linux distribution based on OpenWRT called Linino. You can change the content of the factory image, such as when you install a program or when you change a configuration file. You can return to the factory configuration by pressing the "WLAN RST" button for 30 seconds.

Input and Output

It is not possible to access the I/O pins of the Atheros AR9331. All I/O lines are tied to the 32U4.
Each of the 20 digital i/o pins on the Yún can be used as an input or output, using pinMode(), digitalWrite(), and digitalRead() functions. They operate at 5 volts. Each pin can provide or receive a maximum of 40 mA and has an internal pull-up resistor (disconnected by default) of 20-50 kOhms. In addition, some pins have specialized functions:
  • Serial: 0 (RX) and 1 (TX). Used to receive (RX) and transmit (TX) TTL serial data using the ATmega32U4 hardware serial capability. Note that on the Yún, the Serial class refers to USB (CDC) communication; for TTL serial on pins 0 and 1, use the Serial1 class. The hardware serials of the ATmega32U4 and the AR9331 on the Yún are connected together and are used to communicate between the two processors. As is common in Linux systems, on the serial port of the AR9331 is exposed the console for access to the system, this means that you can access to the programs and tools offered by Linux from your sketch.
  • TWI: 2 (SDA) and 3 (SCL). Support TWI communication using the Wire library.
  • External Interrupts: 3 (interrupt 0), 2 (interrupt 1), 0 (interrupt 2), 1 (interrupt 3) and 7 (interrupt 4). These pins can be configured to trigger an interrupt on a low value, a rising or falling edge, or a change in value. See the attachInterrupt() function for details. Is not recommended to use pins 0 and 1 as interrupts because they are the also the hardware serial port used to talk with the Linux processor. Pin 7 is connected to the AR9331 processor and it may be used as handshake signal in future. Is recommended to be careful of possible conflicts if you intend to use it as interrupt.
  • PWM: 3, 5, 6, 9, 10, 11, and 13. Provide 8-bit PWM output with the analogWrite() function.
  • SPI: on the ICSP header. These pins support SPI communication using the SPI library. Note that the SPI pins are not connected to any of the digital I/O pins as they are on the Uno, They are only available on the ICSP connector. This means that if you have a shield that uses SPI, but does NOT have a 6-pin ICSP connector that connects to the Yún's 6-pin ICSP header, the shield will not work.
The SPI pins are also connected to the AR9331 gpio pins, where it has been implemented in software the SPI interface. This means that the ATMega32u4 and the AR9331 can also communicate using the SPI protocol.
  • LED: 13. There is a built-in LED connected to digital pin 13. When the pin is HIGH value, the LED is on, when the pin is LOW, it's off.
  • There are several other status LEDs on the Yún, indicating power, WLAN connection, WAN connection and USB. 
Arduino  Yún Status LEDs
Arduino  Yún Status LEDs
  • Analog Inputs: A0 - A5, A6 - A11 (on digital pins 4, 6, 8, 9, 10, and 12). The Yún has 12 analog inputs, labeled A0 through A11, all of which can also be used as digital i/o. Pins A0-A5 appear in the same locations as on the Uno; inputs A6-A11 are on digital i/o pins 4, 6, 8, 9, 10, and 12 respectively. Each analog input provide 10 bits of resolution (i.e. 1024 different values). By default the analog inputs measure from ground to 5 volts, though is it possible to change the upper end of their range using the AREF pin and the analogReference() function.
  • AREF. Reference voltage for the analog inputs. Used with analogReference().
There are 3 reset buttons with different functions on the board:
Arduino Yún Reset Button
Arduino Yún Reset Button

  • Yún RST. Bring this line LOW to reset the AR9331 microprocessor. Resetting the AR9331 will cause the reboot of the linux system. All the data stored in RAM will be lost and all the programs that are running will be terminated.
  • 32U4 RST. Bring this line LOW to reset the ATmega32U4 microcontroller. Typically used to add a reset button to shields which block the one on the board.
  • WLAN RST. This button has a double feature. Primarly serves to restore the WiFi to the factory configuration. The factory configuration consist to put the WiFi of the Yún in access point mode (AP) and assign to it the default IP address that is 192.168.240.1, in this condition you can connect with your computer to the a WiFi network that appear with the SSID name "Arduino Yun-XXXXXXXXXXXX", where the twelve 'X' are the MAC address of your Yún. Once connected you can reach the web panel of the Yún with a browser at the 192.168.240.1 or "http://arduino.local" address. Note that restoring the WiFi configuration will cause the reboot of the linux environment. To restore your WiFi configuration you have to press and hold the WLAN RST button for 5 seconds. When you press the button the WLAN blue LED will start to blink and will keep still blinking when you release the button after 5 seconds indicating that the WiFi restore procedure has been recorded. The second function of the WLAN RST button is to restore the linux image to the default factory image. To restore the linux environment you must press the button for 30 seconds. Note that restoring the factory image make you lose all the files saved and softwares installed on the on-board flash memory connected to the AR9331.
See also the mapping between Arduino pins and ATmega32u4 ports.

Communication

The Yún has a number of facilities for communicating with a computer, another Arduino, or other microcontrollers. The ATmega32U4 provides a dedicated UART TTL (5V) serial communication. The 32U4 also allows for serial (CDC) communication over USB and appears as a virtual com port to software on the computer. The chip also acts as a full speed USB 2.0 device, using standard USB COM drivers. The Arduino software includes a serial monitor which allows simple textual data to be sent to and from the Arduino board. The RX and TX LEDs on the board will flash when data is being transmitted via the USB connection to the computer.
Digital pins 0 and 1 are used for serial communication between the 32U4 and the AR9331. Communication between the processors is handled by the Bridge library.
A SoftwareSerial library allows for serial communication on any of the Yún's digital pins except for pins 0 and 1.
The ATmega32U4 also supports I2C (TWI) and SPI communication. The Arduino software includes a Wire library to simplify use of the I2C bus; see the documentation for details. For SPI communication, use the SPI library.
The Yún appears as a generic keyboard and mouse, and can be programmed to control these input devices using the Keyboard and Mouse classes.
The onboard Ethernet and WiFi interfaces are exposed directly to the AR9331 processor. To send and receive data through them, use the Bridge library. To configure the interfaces, you can access the network control panel as described in the getting started page.
The Yún also has USB host capabilities through Linino. You can connect peripherals like USB flash devices for additional storage, keyboards, or webcams. You may need to download and install additional software for these devices to work. For information on adding software to the AR9331, refer to the notes on using the package manager.

Programming

The Yún can be programmed with the Arduino software (download). Select "Arduino Yún from the Tools > Board menu (according to the microcontroller on your board).
The ATmega32U4 on the Arduino Yún comes preburned with a bootloader that allows you to upload new code to it without the use of an external hardware programmer. It communicates using the AVR109 protocol.
You can also bypass the bootloader and program the microcontroller through the ICSP (In-Circuit Serial Programming) header; see these instructions for details.

Automatic (Software) Reset and Bootloader Initiation

Rather than requiring a physical press of the reset button before an upload, the Yún is designed in a way that allows it to be reset by software running on a connected computer. The reset is triggered when the Yún's virtual (CDC) serial / COM port is opened at 1200 baud and then closed. When this happens, the processor will reset, breaking the USB connection to the computer (meaning that the virtual serial / COM port will disappear). After the processor resets, the bootloader starts, remaining active for about 8 seconds. The bootloader can also be initiated by pressing the reset button on the Yún. Note that when the board first powers up, it will jump straight to the user sketch, if present, rather than initiating the bootloader.
Because of the way the Yún handles reset it's best to let the Arduino software try to initiate the reset before uploading, especially if you are in the habit of pressing the reset button before uploading on other boards. If the software can't reset the board you can always start the bootloader by pressing the reset button on the board.

USB Overcurrent Protection

The Yún has a resettable polyfuse that protects your computer's USB ports from shorts and overcurrent. Although most computers provide their own internal protection, the fuse provides an extra layer of protection. If more than 500 mA is applied to the USB port, the fuse will automatically break the connection until the short or overload is removed.

Physical Characteristics

The maximum length and width of the Yún PCB are 2.7 and 2.1 inches respectively, with the USB connector extending beyond the former dimension. Four screw holes allow the board to be attached to a surface or case. Note that the distance between digital pins 7 and 8 is 160 mil (0.16"), not an even multiple of the 100 mil spacing of the other pins.
Weight of the board is 0.0406 Kg.

Using your Yun

There is an extensive Getting Started Guide and a number of tutorials found on the library reference page.

Information Source for this Article: Arduino.cc

0 comments:

Recording a Video From Webcam Using MATLAB Code

Recording a Video From Webcam Using MATLAB Code is a very simple task if, you know how to access your webcam or other imaging device using MATLAB & snapping from them.
Click Here to know more about accessing Webcam using MATLAB code.
Click Here to know more about snapping an image from Webcam using MATLAB code.

Code:

clear all;
close all;
clc;

% Constructing a video input interface 
vid = videoinput('winvideo',1,'YUY2_640x480');

% setting the frame rate as 12 (it will depend on your webcam device also)
set(vid,'FramesPerTrigger',12);
set(vid,'FrameGrabInterval',10); 
start(vid);
wait(vid,Inf);

% Retrieve the frames and timestamps for each frame.
[frames,time] = getdata(vid, get(vid,'FramesAvailable'));

% Calculate frame rate by averaging difference between each frame's timestamp
framerate = mean(1./diff(time));
capturetime = 30;

%The FrameGrabInterval property specifies how often frames are stored from 
%the video stream. For instance, if we set it to 5, then only 1 in 5 frames
%is kept -- the other 4 frames will be discarded. Using the framerate, 
%determine how often you want to get frames
interval = get(vid,'FrameGrabInterval');

%To determine how many frames to acquire in total, calculate the total 
%number of frames that would be acquired at the device's frame rate, 
%and then divide by the FrameGrabInterval.
numframes = floor(capturetime * framerate / interval);

%If a large number of frames will be acquired, it is more practical to log 
%the images to disk rather than to memory. 
%Using the Image Acquisition Toolbox, you can log images directly %
%to an AVI file. We configure this using the LoggingMode property.
set(vid,'LoggingMode','disk');

%Create an AVI file object to log to, using the avifile command. 
%We must specify the filename to use, and the frame rate that 
%the AVI file should be played back at. Then, set the DiskLogger property 
%of the video input object to the AVI file.
avi = avifile('timelapsevideo.avi','fps',framerate);
set(vid,'DiskLogger',avi);
start(vid);

wait(vid,Inf); % Wait for the capture to complete before continuing.

%Once the capture is completed, retrieve the AVI file object, and use the 
%close function to release the resources associated with it.

wait(vid,Inf); % Wait for the capture to complete before continuing.

avi = get(vid,'DiskLogger');
avi = close(avi);

%When you are done with the video input object, you should use 
%the delete function to free the hardware resources associated with it, 
%and remove it from the workspace using the clear function.
delete(vid);
clear vid;
 
NOTE: 
# After the code execution you will see a .avi video file in your 
current working Directory as timelapsevideo.avi.
# If the video file is completely black, your camera is slow & thus need some
warm up time to start off. So give delay of 1-2 seconds during the camera capturing
step.
# The following error/warning can also arise, the 1st one is due to the fact that 
"avifile" function is obsolete now & MATLAB has decided to remove in further 
releases.
The 2nd one is due to the fact that the following codec the MATLAB is trying to
use is not installed in your system, so either install it or change it or MATLAB
will self use the 'none' attribute in place of the codec.
Warning: AVIFILE will be removed in a future release. Use VIDEOWRITER instead. 
> In avifile.avifile at 81
  In img at 33 
Warning: Cannot locate Indeo5 compressor, using 'None' as the compression type.
See Mathworks Technical Solution 1-4G50RI for more information.  

59 comments:

Acquiring An Image From Webcam, Using MATLAB Code

Acquiring An Image From Webcam, Using MATLAB Code is rather simple if you know how to access your webcam or other imaging device using MATLAB.
Click Here to know more about accessing Webcam using MATLAB code.

MATLAB Code For Acquiring An Image From Webcam:
 
CODE1: For fast camera in which no warm up delay is required
%creating a "videoinput" abject 'v'
v=videoinput('winvideo', 1, 'YUY2_640x480');
%storing the image in 'im' variable
im=getsnapshot(v);
%displaying the captured image
imshow(im);
%writing/storing the captured image in the hard-disk
imwrite(im,'c:\image.jpg');
%deleting the "videoinput" object to avoid filling up of memory
delete(v);
 
or
 
CODE2: For slow camera in which some warm up delay is required(1-5 seconds)
 
%creating a "videoinput" abject 'v'
v=videoinput('winvideo', 1, 'YUY2_640x480');
%Setting the "FramesPerTrigger" value to '1'
v.FramesPerTrigger= 1;
%for previewing window of the "videoinput" object 'v'
preview(v);
%start capturing
start(v);
%for warming up of webcam
pause(2.0);% Get image data from webcam
im=getdata(v);
%displaying the captured image
imshow(im);
%writing/storing the captured image in the hard-disk
imwrite(im,'c:\image.jpg');
%deleting the "videoinput" object to avoid filling up of memory
delete(v);
 
 
NOTE: 
If your acquired image is completely black, it may be that your camera needs to warm up. To allow for this, try using manual triggering & GETSNAPSHOT.
The exposure time may show up as a device specific property on the source:
More more help regarding this problem do read the below link:

3 comments:

Accessing the Serial Port using MATLAB Code: Serial communication through MATLAB

Through MATLAB, we can access various computer peripherals. One of them are Serial Ports. These serial port supports a maximum baud rate of 115200bps. This program is set for the most commonly used baud rate i.e., 9600bps. If the error rate of the medium is very high, or you are using it to communicate a slower device, then you have to use a lower baud rate like, 4800bps, 2400bps etc.
It is particularly useful when you are communicating through a Micro-controller Controlled Application, it that you can actually give command in you MATLAB's command window, and the MCU will react to that command. 

#  In the program below we have used the serial port COM25, which is fed as a string in the function "serial".
#  In the subsequent lines we are using "set" function to set various other serial communication parameters.
# it will display all the data it got from the external device, communicating through that COM port in nearly 100 iterations
# when the port is being successfully setup, it will display a success message & then you can transmit & receive data from that serial port.

SAMPLE RUN OF THE CODE BELOW:
Accessing the Serial Port using MATLAB Code: Serial communication through MATLAB
Accessing the Serial Port using MATLAB Code: Serial communication through MATLAB

Code:
clc
clear all
close all
sp=serial('COM25');  %assigns the object s the serial port object
set(sp, 'InputBufferSize', 256); 
%number of bytes in inout buffer
set(sp, 'FlowControl', 'hardware');
set(sp, 'BaudRate', 9600);
set(sp, 'Parity', 'none');
set(sp, 'DataBits', 8);
set(sp, 'StopBit', 1);
set(sp, 'Timeout',10);
disp(get(s,'Name'));
prop(1)=(get(sp,'BaudRate'));
prop(2)=(get(sp,'DataBits'));
prop(3)=(get(sp, 'StopBit'));
prop(4)=(get(sp, 'InputBufferSize'));

disp(['Port Setup Finished Successfully!',num2str(prop)]);

fopen(sp);          
%opens the serial port
t=1;
disp('Running');
x=0;
while t<100
   a=fread(sp);
   a=max(a);
   x =[x a];
   plot(x);
   axis auto;
   grid on;
   disp([num2str(t),'th iteration max= ',num2str(a)]);
   hold on;
   t=t+1;
   a=0;
   drawnow;
end

fclose(sp);
%close the serial port

0 comments:

Accessing Webcam Through MATLAB Code

It is actually possible to access various hardware, with MATLAB codes. One of the hardware is your Webcam, in order to take a video or a Picture as an input.

The functions we are going to use for it is: "videoinput" & "preview"
This "videoinput" function creates a "video input object" which can be used for previewing the video through "preview" function.

So code will be like: 
%start of code
clc;
vid = videoinput('winvideo', 1, '
YUY2_640x480');
preview(vid);

 %end of code

Preview of how output will be:

Accessing Webcam Through MATLAB Code: videoinput & preview functions
Accessing Webcam Through MATLAB Code


Click here to see How to Acquire An Image From Webcam, Using MATLAB Code.
Click here to see How to Record a Video From Webcam Using MATLAB Code.

Details of videoinput & preview function:

"videoinput" function:

v = videoinput(Adapter_name)    or
v = videoinput(Adapter_name, Device_ID)    or
v = videoinput(Adapter_name, Device_ID, Format)


# here 'v' is the video object created after executing the "videoinput" function.
# here "Adapter_name" is a string that specifies the name of the adapter used to communicate with that particular device.

NOTE: Use the "imaqhwinfo" function to determine the adaptors available on your system. Its output will be like: 
>> imaqhwinfo
ans = InstalledAdaptors: {'gentl'  'gige'  'matrox'  'ni'  'winvideo'}
         MATLABVersion: '8.0 (R2012b)'
         ToolboxName: 'Image Acquisition Toolbox'
         ToolboxVersion: '4.4 (R2012b)'

With these info., we can use 'gentl' or 'gige' or 'matrox' or 'ni' or 'winvideo' as Adapter_name.

# here Device_ID is a numeric scalar value that identifies a particular device available through the specified adaptor, Adapter_name.

NOTE: Use the "imaqhwinfo(Adapter_name)" syntax to determine the devices available through the specified adaptor. If Device_ID is not specified, the first available device ID is used. For convenience, a device's name can be used in place of the Device_ID. If multiple devices have the same name, the first available device is used.

Its output will be like:
>> imaqhwinfo('ni')
ans = AdaptorDllName: [1x68 char]
          AdaptorDllVersion: '4.4 (R2012b)'
          AdaptorName: 'ni'
          DeviceIDs: {1x0 cell}
          DeviceInfo: [1x0 struct]

# In this case the AdapterName is 'ni', but its device id is not clear, so we can use its name instead of Device_ID.

>> imaqhwinfo('winvideo')
ans = AdaptorDllName: [1x74 char]
          AdaptorDllVersion: '4.4 (R2012b)'
          AdaptorName: 'winvideo'
          DeviceIDs: {[1]}
          DeviceInfo: [1x1 struct]

# In this case the AdapterName is 'winvideo', & its device id is '1', so we can use this value as Device_ID.

# FORMAT is a text string that specifies a particular video format supported by the device
NOTE:
To get a list of the formats supported by a particular device, view the DeviceInfo structure for the device that is returned by the imaqhwinfo  function. Each DeviceInfo structure contains a SupportedFormats field. If FORMAT is not specified, the device's default format is used.

SAMPLE OUTPUT:
>> a=imaqhwinfo('winvideo');
>> a.DeviceInfo.SupportedFormats

ans =

  Columns 1 through 4

    'YUY2_1280x1024'    'YUY2_1280x800'    'YUY2_160x120'    'YUY2_176x144'

  Columns 5 through 7

    'YUY2_320x240'    'YUY2_352x288'    'YUY2_640x480'


Here 'YUY2_1280x1024' , 'YUY2_1280x800'  ,  'YUY2_160x120'  ,  'YUY2_176x144' ,
'YUY2_320x240'  ,  'YUY2_352x288'  &  'YUY2_640x480' are the supported video format of the device_name 'winvideo' use that.


"preview" function:

It Display preview of live video data. preview(v) creates a Video Preview window that displays live video data for video input object 'v'. The window also displays the timestamp & video resolution of each frame, and the current status of 'v'. The Video Preview window displays the video data at 100% magnification (1:1).

2 comments:

Motion Tracking/Detection in MATLAB using Kalman Filter

MATAB has great capabilities to process video & pictures. One of these capabilities we have tested is the motion detection in a pre-recorded video, using Kalman Filtering technique.

Here is an example video of Motion Tracking using  Kalman Filter:


Video for illustration Purpose Only
Basic background of Kalman Filter:
The Kalman filter, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing noise (random variations) and other inaccuracies, and produces estimates of unknown variables that tend to be more precise than those based on a single measurement alone. More formally, the Kalman filter operates recursively on streams of noisy input data to produce a statistically optimal estimate of the underlying system state. The filter is named for Rudolf (Rudy) E. Kálmán, one of the primary developers of its theory.
The Kalman filter has numerous applications, e.g, for guidance, navigation & control of vehicles, particularly aircraft and spacecraft.

Kalman Filter Algorithm
Kalman Filter Algorithm (courtsey: colorado.edu)
 MATLAB CODE:

%See the below code update of 11/11/2014 for implementing the code through VideoReader instead of aviread. Now with this update you can solve many of your problems of using your own video for this code.

%Code in the Bold, & comments in normal font

clear all; 
close all; 
clc
%% Read video into MATLAB using aviread
video = aviread('rhinos.AVI');

%'n' for calculating the number of frames in the video file
n = length(video);
% Calculate the background image by averaging the first 10 images
temp = zeros(size(video(1).cdata));
[M,N] = size(temp(:,:,1));
for i = 1:10
    temp = double(video(i).cdata) + temp;
end
imbkg = temp/10;

% Initialization step for Kalman Filter
centroidx = zeros(n,1);
centroidy = zeros(n,1);
predicted = zeros(n,4);
actual = zeros(n,4);

% % Initialize the Kalman filter parameters
% R - measurement noise,
% H - transform from measure to state
% Q - system noise,
% P - the status covarince matrix
% A - state transform matrix

R=[[0.2845,0.0045]',[0.0045,0.0455]'];
H=[[1,0]',[0,1]',[0,0]',[0,0]'];
Q=0.01*eye(4);
P = 100*eye(4);
dt=1;
A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];

% loop over all image frames in the video
kfinit = 0;
th = 38;
for i=1:n
  imshow(video(i).cdata);
  hold on
  imcurrent = double(video(i).cdata);
 
 
% Calculate the difference image to extract pixels with more than 40(threshold) change
  diffimg = zeros(M,N);
  diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
      | (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
      | (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);

 
% Label the image and mark
  labelimg = bwlabel(diffimg,4);
  markimg = regionprops(labelimg,['basic']);
  [MM,NN] = size(markimg);

 
% Do bubble sort (large to small) on regions in case there are more than 1
  % The largest region is the object (1st one)
  for nn = 1:MM
      if markimg(nn).Area > markimg(1).Area
          tmp = markimg(1);
          markimg(1)= markimg(nn);
          markimg(nn)= tmp;
      end
  end

 
% Get the upper-left corner, the measurement centroid and bounding window size
  bb = markimg(1).BoundingBox;
  xcorner = bb(1);
  ycorner = bb(2);
  xwidth = bb(3);
  ywidth = bb(4);
  cc = markimg(1).Centroid;
  centroidx(i)= cc(1);
  centroidy(i)= cc(2);

 
% Plot the rectangle of background subtraction algorithm -- blue
  hold on
  rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
  hold on
  plot(centroidx(i),centroidy(i), 'bx');

 
% Kalman window size
  kalmanx = centroidx(i)- xcorner;
  kalmany = centroidy(i)- ycorner;

  if kfinit == 0
     
% Initialize the predicted centroid and volocity
      predicted =[centroidx(i),centroidy(i),0,0]' ;
  else
     
% Use the former state to predict the new centroid and volocity
      predicted = A*actual(i-1,:)';
  end
  kfinit = 1;

  Ppre = A*P*A' + Q;
  K = Ppre*H'/(H*Ppre*H'+R);
  actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
  P = (eye(4)-K*H)*Ppre;

  % Plot the tracking rectangle after Kalman filtering -- red
  hold on
rectangle('Position',[(actual(i,1)-kalmanx) (actual(i,2)-kalmany) xwidth ywidth], 'EdgeColor', 'r','LineWidth',1.5);
  hold on
  plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
  drawnow;
end
%end of the code
%Copyright (c) 2012, Xing
%All rights reserved.

NOTE:
 The above code sometimes generate the following error & warning, in the initial step itself (while reading the .avi file using "aviread" function.

Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead.
> In aviread at 26
  In motionnew at 4
Error using aviread (line 80)
Unable to locate decompressor to decompress video stream

Error in motionnew (line 4)
video = aviread('traffic.avi');

# First warning is due to the fact, that aviread  function has became obsolete & MATLAB team decided to replace it with a new function called, VideoReader.
# In order to avoid the second error, you need to install the avi video codec, the best thing available in the internet is the K-Lite Mega Codec Pack.

~Team Digital iVision Lab.
# do comment for any code request, help or queries.


Original Unmodified Source can be downloaded here.

If having problem with aviread or mmread function kindly visit the following post.
How to resolve MATLAB Problem With "aviread" or "VideoReader" command, while reading the avi file?



UPDATE : 20/10/2014

Some frequent errors Faced by users, & Solutions.

(1)Cannot find an exact (case-sensitive) match for 'videoReader'
The closest match is: VideoReader in
C:\ProgramFiles\MATLAB\R2013a\toolbox\matlab\audiovideo\@VideoReader\VideoReader.m

Solution: You need to use VideoReader instead of videoReader or VideoReader.

(2)No appropriate method, property, or field cdata for class VideoReader
Solution: Since the above code is implemented for aviread function, it will not be same as for VideoReader class implementation.
(3) Warning: AVIREAD will be removed in a future release. Use VIDEOREADER instead. 
> In aviread at 26 
Solution: The warning is due to the fact that aviread command has become obsolete & it will be removed from further version.
(4)Error using aviread (line 140)
The AVI file must be 8-bit Indexed or grayscale images, 16-bit grayscale, or 24-bit TrueColor images
Solution: This error is occuring due after implementing the FFMPEG decompression. This is due to the fact that aviread supports only those files only which are 8-bit indexed image or grayscale image. In modern world generally the video are much enhanced & their frames are having more color depth, so this error occurs while using them, even after decompressing step using ffmpeg.


UPDATE : 11/11/2014 (Major Code Update)
Many user are facing problems with the above code, that have been written using "aviread" funciton. So I am re-editing this code to be used using VideoReader class, so that you can easily use it with your own videos.

%Code starts here
clc;
close all;
clear all;
video = VideoReader('rhinos.avi');
%in place of aviread
%nframes = length(video);
nframes=video.NumberOfFrames;
for i=1:nframes
mov(i).cdata=read(video,i) 
%creating '.cdata' field to avoid much changes to previous code
end
 
temp = zeros(size(mov(1).cdata));
[M,N] = size(temp(:,:,1));
for i = 1:10
temp = double(mov(i).cdata) + temp;
end
imbkg = temp/10;
centroidx = zeros(nframes,1);
centroidy = zeros(nframes,1);
predicted = zeros(nframes,4);
actual = zeros(nframes,4);
R=[[0.2845,0.0045]',[0.0045,0.0455]'];
H=[[1,0]',[0,1]',[0,0]',[0,0]'];
Q=0.01*eye(4);
P = 100*eye(4);
dt=1;
A=[[1,0,0,0]',[0,1,0,0]',[dt,0,1,0]',[0,dt,0,1]'];
kfinit = 0;
th = 38;
for i=1:nframes
imshow(mov(i).cdata);
hold on
imcurrent = double(mov(i).cdata);
diffimg = zeros(M,N);
diffimg = (abs(imcurrent(:,:,1)-imbkg(:,:,1))>th) ...
| (abs(imcurrent(:,:,2)-imbkg(:,:,2))>th) ...
| (abs(imcurrent(:,:,3)-imbkg(:,:,3))>th);
labelimg = bwlabel(diffimg,4);
markimg = regionprops(labelimg,['basic']);
[MM,NN] = size(markimg);
for nn = 1:MM
if markimg(nn).Area > markimg(1).Area
tmp = markimg(1);
markimg(1)= markimg(nn);
markimg(nn)= tmp;
end
end
bb = markimg(1).BoundingBox;
xcorner = bb(1);
ycorner = bb(2);
xwidth = bb(3);
ywidth = bb(4);
cc = markimg(1).Centroid;
centroidx(i)= cc(1);
centroidy(i)= cc(2);
hold on
rectangle('Position',[xcorner ycorner xwidth ywidth],'EdgeColor','b');
hold on
plot(centroidx(i),centroidy(i), 'bx');
kalmanx = centroidx(i)- xcorner;
kalmany = centroidy(i)- ycorner;

if kfinit == 0
predicted =[centroidx(i),centroidy(i),0,0]' ;
else
predicted = A*actual(i-1,:)';
end
kfinit = 1;
Ppre = A*P*A' + Q;
K = Ppre*H'/(H*Ppre*H'+R);
actual(i,:) = (predicted + K*([centroidx(i),centroidy(i)]' - H*predicted))';
P = (eye(4)-K*H)*Ppre;
hold on

 rectangle('Position',[(actual(i,1)-kalmanx)...
(actual(i,2)-kalmany) xwidth ywidth],'EdgeColor','r','LineWidth',1.5);
hold on
plot(actual(i,1),actual(i,2), 'rx','LineWidth',1.5);
drawnow;
end




131 comments:

Optimizing MATLAB Code

This post at Stackoverflow by Samuel O'Malley is unique, since there are very few occasion a MATLAB programmer think of doing an Optimization of their code. But they don't know what the benefit they will get if they do so. Here Samuel O'Malley discussed the effect of pre-allocation of arrays in the execution time & the effect of "fread" function. He used tic & toc block to find out the execution time of various codes. We don't know the configuration of the computer system he used(both hardware & software). The result can significantly vary system to system or event the MATLAB version you are using. But still this article is providing in insight to MATLAB CODE OPTIMIZATION, which we generally avoid.

Guide to Optimizing MATLAB Code

I have noticed many individual questions on SO but no one good guide to MATLAB optimization.
Common Questions:
  • Optimize this code for me
  • How do I vectorize this?
I don't think that these questions will stop, but I'm hoping that the ideas presented here will them something centralized to refer to.
Optimizing Matlab code is kind of a black-art, there is always a better way to do it. And sometimes it is straight-up impossible to vectorize your code.
So my question is: when vectorization is impossible or extremely complicated, what are some of your tips and tricks to optimize MATLAB code? Also if you have any common vectorization tricks I wouldn't mind seeing them either.

Preface

All of these tests are performed on a machine that is shared with others, so it is not a perfectly clean environment. Between each test I clear the workspace to free up memory.
Please don't pay attention to the individual numbers, just look at the differences between the before and after optimisation times.
Note: The tic and toc calls I have placed in the code are to show where I am measuring the time taken.

Pre-allocation

The simple act of pre-allocating arrays in Matlab can give a huge speed advantage.
 
tic;
for i = 1:100000
    my_array(i) = 5 * i;
end
toc;
 
This takes 47 seconds

tic;
length = 100000;
my_array = zeros(1, length);
for i = 1:length
    my_array(i) = 5 * i;
end
toc;
 
This takes 0.1018 seconds

47 seconds to 0.1 seconds for a single line of code added is an amazing improvement. Obviously in this simple example you could vectorize it to my_array = 5 * 1:100000 (which took 0.000423 seconds) but I am trying to represent the more complicated times when vectorization isn't an option.
I recently found that the zeros function (and others of the same nature) are not as fast at pre-allocating as simply setting the last value to 0:


tic;
length = 100000;
my_array(length) = 0;
for i = 1:length
    my_array(i) = 5 * i;
end
toc;
 
This takes 0.0991 seconds

Now obviously this tiny difference doesn't prove much but you'll have to believe me over a large file with many of these optimisations the difference becomes a lot more apparent.

Why does this work?
The pre-allocation methods allocate a chunk of memory for you to work with. This memory is contiguous and can be pre-fetched, just like an Array in C++ or Java. However if you do not pre-allocate then MATLAB will have to dynamically find more and more memory for you to use. As I understand it, this behaves differently to a Java ArrayList and is more like a LinkedList where different chunks of the array are split all over the place in memory.
Not only is this slower when you write data to it (47 seconds!) but it is also slower every time you access it from then on. In fact, if you absolutely CAN'T pre-allocate then it is still useful to copy your matrix to a new pre-allocated one before you start using it.

What if I don't know how much space to allocate?
This is a common question and there are a few different solutions:
  1. Overestimation - It is better to grossly overestimate the size of your matrix and allocate too much space, than it is to under-allocate space.
  2. Deal with it and fix later - I have seen this a lot where the developer has put up with the slow population time, and then copied the matrix into a new pre-allocated space. Usually this is saved as a .mat file or similar so that it could be read quickly at a later date.
How do I pre-allocate a complicated structure?
Pre-allocating space for simple data-types is easy, as we have already seen, but what if it is a very complex data type such as a struct of structs?
I could never work out to explicitly pre-allocate these (I am hoping someone can suggest a better method) so I came up with this simple hack:
 
tic;
length = 100000;
% Reverse the for-loop to start from the last element
for i = 1:length
    complicated_structure = read_from_file(i);
end
toc;
 
This takes 1.5 minutes

tic;
length = 100000;
% Reverse the for-loop to start from the last element
for i = length:-1:1
    complicated_structure = read_from_file(i);
end
% Flip the array back to the right way
complicated_structure = fliplr(complicated_structure);
toc;
 
This takes 6 seconds

This is obviously not perfect pre-allocation, and it takes a little while to flip the array afterwards, but the time improvements speak for themselves. I'm hoping someone has a better way to do this, but this is a pretty good hack in the mean time.

Data Structures

In terms of memory usage, an Array of Structs is orders of magnitude worse than a Struct of Arrays:

% Array of Structs
a(1).a = 1;
a(1).b = 2;
a(2).a = 3;
a(2).b = 4;
 
Uses 624 Bytes

% Struct of Arrays
a.a(1) = 1;
a.b(1) = 2;
a.a(2) = 3;
a.b(2) = 4;
 
Uses 384 Bytes

As you can see, even in this simple/small example the Array of Structs uses a lot more memory than the Struct of Arrays. Also the Struct of Arrays is in a more useful format if you want to plot the data.
Each Struct has a large header, and as you can see an array of structs repeats this header multiple times where the struct of arrays only has the one header and therefore uses less space. This difference is more obvious with larger arrays.

File Reads

The less number of freads (or any system call for that matter) you have in your code, the better.

tic;    
for i = 1:100
    fread(fid, 1, '*int32');
end
toc;
 
The previous code is a lot slower than the following:

tic;
fread(fid, 100, '*int32');
toc;
 
You might think that's obvious, but the same principle can be applied to more complicated cases:
 
tic;
for i = 1:100
    val1(i) = fread(fid, 1, '*float32');
    val2(i) = fread(fid, 1, '*float32');
end
toc;
 
This problem is no longer simple because in memory the floats are represented like this:
val1 val2 val1 val2 etc.
 
However you can use the skip value of fread to achieve the same optimizations as before:
 
tic;
% Get the current position in the file
initial_position = ftell(fid);
% Read 100 float32 values, and skip 4 bytes after each one
val1 = fread(fid, 100, '*float32', 4);
% Set the file position back to the start (plus the size of the initial float32)
fseek(fid, position + 4, 'bof');
% Read 100 float32 values, and skip 4 bytes after each one
val2 = fread(fid, 100, '*float32', 4);
toc;
 
So this file read was accomplished using two freads instead of 200, a massive improvement.

Function Calls

I recently worked on some code that used many function calls, all of which were located in separate files. So lets say there were 100 separate files, all calling each other. By "inlining" this code into one function I saw a 20% improvement in execution speed from 9 seconds.
Obviously you would not do this at the expense of re-usability, but in my case the functions were automatically generated and not reused at all. But we can still learn from this and avoid excessive function calls where they are not really needed.

This article obeys cc-wiki licensing, which does require attribution (stackoverflow & Samuel O'Malley).
 

0 comments: