Video Files
Capture: Video Files
Namespace
Emgu.CV
Emgu.CV.Capture
References
EMGU Capture Reference
EMGU VideWriter Reference
OpenCV Use Reference
Downloads
Example
The following example shows the use of the Capture function within EMGU. The function of this library is to allow video streaming for web camera type devices and video files. This example will shows an example of reading and viewing video files, and recoding video files from a capture device using the VideWriter class. A video capture example is available here for the acquisition from devices.
Software
Pre-Requisites
The code provided should run straight out of the Emgu.Example folder (V2.4.2), extract it to this location. If the code fails to execute re-reference the EMGU libraries and include the required opencv dlls in the bin directory. Note that the project is set to build to the output path "..\..\..\bin\" you may wish to change this if you don't extract to the EMGU.Example folder.
The video record function will require a webcam or alternative capture device. It will acquire from the Os default device. Some additional video codecs may be required if they are not already installed on your system.
EMGU Coding Level: While the coding is not advanced the rated level for this example is Intermediate. This is not designed as a full on tutorial and general knowledge of the EMGU is expected. While the coding is basic the are several methods involved that may be of putting the newcomers of EMGU.
The Code
The code provided in this sample is basic there is little or no error checking. Support is available through the Forums but please try and examine the code before saying it doesn't work for you. The code is not optimised instead is better formatted to provided an understanding of the stages involved.
As the example demonstrates two methods, both viewing and recording video, some of the methods have two operations within them. They are controlled by setting VideoMethod enum called CurrentState. The methods are discussed individually in the order they appear in the code bellow.
The Code: Variables
An Open and Save file dialogue are globally declared, this is just so the file name for either saving or reading of the video can be accessed by other threads if needed.
OpenFileDialog OF = new OpenFileDialog();
SaveFileDialog SF = new SaveFileDialog();
The current method of viewing/recording videos is checked using the following variables. The two boolean variables playstate and recordstate are used for changing the button image and action depending on the program operation they record the play/pause/run state of the recording and viewing process. The enumerator CurrentState is used to classify the current state either 'Viewing' or 'Recording', this allows the program to operate according to reading or writing video files.
//current video mode and state
bool playstate = false;
bool recordstate = false;
VideoMethod CurrentState = VideoMethod.Viewing; //default state
public enum VideoMethod
{
Viewing,
Recording
};
The following variables and Class initialiser variables (VW,SW, and _Capture) deal with the capture device/video, storing its variables, and for writing video files. The _Capture variable is used in two ways depending on the program operation. If viewing videos it is initialised using the Capture(string 'video file name') method. If recording a video it is initialised using the default constructor Capture() setting up the variable to capture from the default video acquisition device installed on the system. To enable selection of device see the Capture: Camera example for reference.
The VideoWriter variable VW is used to set up a video writer when recording a video from the acquisition device. Caution must be used when using the class as even when correctly disposing of the object video index buffers can still become corrupt. Using uncompressed video formats and the re-encoding the video later is the most stable method of use. Other variables are used for storing and displaying of video information.
//Capture Setting and variables
Capture _Capture;
double FrameRate = 0;
double TotalFrames = 0;
VideoWriter VW;
Stopwatch SW;
int Frame_width;
int Frame_Height;
int FrameCount;
The Code: Methods
The Form1() method initialises the forms controls as default and no extra code is included here. TheProcessFrame() method has two sections, separated with an two if statements looking at the CurrentState of the program, either viewing or recording videos.
Viewing videos (VideoMethod.Viewing) is a fairly simple in it's application firstly it shows the frame and then updates the forms controls, labels and trackbar information. This is all done using thread safe calls to methods that include delegate operations if the controls need invoking. As the process frame is called from an independent thread set up in the Capture variable an control invoke is required to prevent cross thread interference. Then the ProcessFrame() thread is told to sleep for a specific period of time. This allows the video to be displayed at a correct frame rate. If this is removed the video will play as fast as the computer can process each frame. In this example delay is dependant on the frame rate that the video was encoded with, at least what the codec says it was. This is not always truthful and sometimes the video will play faster than what is expected. Additional check can be introduced by checking the total time of the video and the total number of frames and setting a delay according to that. These variables are available through the GetCaptureProperty() method call. Although to get the final time stamp you will have to set the Capture variable to the final frame before reading the time stamp and then reset it to the starting frame. More information on the GetCaptureProperty() method call is available here.
Finally a check is done to see if the video has played all the way through, this is a relatively easy comparison of checking the current frame and seeing if it's the same number as the total number of frames. If the video has played all the way through an automated button method call is made to stop the video and the video is rewound to the beginning by setting the frame number using the SetCaptureProperty() method call.
if (CurrentState == VideoMethod.Viewing)
{
try
{
//Show image
DisplayImage(_Capture.RetrieveBgrFrame().ToBitmap());
//Show time stamp
double time_index = _Capture.GetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_MSEC);
UpdateTextBox("Time: " + TimeSpan.FromMilliseconds(time_index).ToString(), Time_Label);
//show frame number
double framenumber = _Capture.GetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_FRAMES);
UpdateTextBox("Frame: " + framenumber.ToString(), Frame_lbl);
//update trackbar
UpdateVideo_CNTRL(framenumber);
/*Note: We can increase or decrease this delay to fastforward of slow down the display rate
if we want a re-wind function we would have to use _Capture.SetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_FRAMES, FrameNumber*);
//and call the process frame to update the picturebox ProcessFrame(null, null);. This is more complicated.*/
//Wait to display correct framerate
Thread.Sleep((int)(1000.0 / FrameRate)); //This may result in fast playback if the codec does not tell the truth
//Lets check to see if we have reached the end of the video
//If we have lets stop the capture and video as in pause button was pressed
//and reset the video back to start
if (framenumber == TotalFrames)
{
//pause button update
play_pause_BTN_MouseUp(null, null);
framenumber = 0;
UpdateVideo_CNTRL(framenumber);
_Capture.SetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_FRAMES, framenumber);
//call the process frame to update the picturebox
ProcessFrame(null, null);
}
}
catch
{
}
}
Recording videos (VideoMethod.Recording) is a much smaller section of code as less information is displayed and VideoWriter is managed according to the press of the play_pause_BTN button. The captured frame is always displayed to the user, upon pressing the play_pause_BTN (now set up as a record stop button) the frames are passed to the VideoWriter. The video writer is disposed when the play_pause_BTN is pressed again this tries to provided a successful closing event for the video files so it is written correctly. This is not essential at this stage and the VideoWriter can be written to intermittently say every 3rd frame etc. but it must be disposed correctly using another event call such as the form closing event. Further video compression can be obtained by re-sizing the frame before passing it to the VideoWriter for recording to the video file.
if (CurrentState == VideoMethod.Recording)
{
Image<Bgr, Byte> frame = _Capture.RetrieveBgrFrame(); //capture to a Image variable so we can use it for writing to the VideoWriter
DisplayImage(_Capture.RetrieveBgrFrame().ToBitmap()); //Show the image
//if we wanted to compresse the image to a smaller size to save space on our video we could use
//frame.Resize(100,100, Emgu.CV.CvEnum.INTER.CV_INTER_LINEAR)
//But the VideoWriter must be set up with the correct size
if (recordstate && VW.Ptr != IntPtr.Zero)
{
VW.WriteFrame(frame); //If we are recording and videowriter is avaliable add the image to the videowriter
//Update frame number
FrameCount++;
UpdateTextBox("Frame: " + FrameCount.ToString(), Frame_lbl);
//Show time stamp or there abouts
UpdateTextBox("Time: " + TimeSpan.FromMilliseconds(SW.ElapsedMilliseconds).ToString(), Time_Label);
}
}
The Play/Pause Record/Stop button is made from a panel for ease. As with the ProcessFrame() method there are two sections, one for viewing videos and another for recording them. All of this is only done if a the _Capture is set up. For VideoMethod.Viewing (viewing video) the actions are simple, change the boolean flag operator so any other method now the state of the viewing process (Play/Pause). Start of Pause the capture variable and update the forms controls, in this case thread safe enabling/dis-enabling of the trackbar and the changing of the buttons image. The trackbar is disabled during playing of the video as the program is much more buggy when moving it while a capture event is running. While it should be possible achieving such operation is beyond the scope of this simple application.
if (CurrentState == VideoMethod.Viewing)
{
playstate = !playstate; //change playstate to the opposite
/*Update Play panel image*/
if (playstate)
{
play_pause_BTN.BackgroundImage = VideoCapture.Properties.Resources.Pause;
UpdateVideo_CNTRL(false); //disable this as it's not safe when running
//this may work in legacy call method and be cause by a cross threading issue
_Capture.Start();
}
else
{
play_pause_BTN.BackgroundImage = VideoCapture.Properties.Resources.Play;
_Capture.Pause();
UpdateVideo_CNTRL(true);
}
}
For VideoMethod.Recording (recording video) mode things are a little more involved. As when video viewing a boolean operator is used to update other methods on the current state in recording of the acquired video. If the record process is starting the frame count is reset to 0 and a stopwatch started, the stopwatch won't give us exact timestamps of the video but it will allow the user to see roughly how long they've been recording. This can be important since video files are big and they can happily populate your hard drive until you run out of space. The program also checks to see if the VideoWriter has been disposed, as the VideoWriter is disposed when a video recording has stopped to allow proper writing of the video file. If it has been disposed the user is informed and asked to select a new video file name. This is done by re-calling the rocord video menu selection method recordVideoToolStripMenuItem_Click() and passing null parameters.
If the recording is being stopped the VideoWriter is disposed in an attempt to finalise the building of the frame index and write the video file correctly. This is buggy and sometimes the frame index may become corrupt. Programs like VLC media player will cope with an issue like this where windows media player will usually spit its dummy out. Other than this the buttons image is changed and the stopwatch is stopped. The _Capture is allowed to continue in order to allows the user to see the video frames before recording again.
else if (CurrentState == VideoMethod.Recording)
{
recordstate = !recordstate; //change playstate to the opposite
/*Update Play panel image*/
if (recordstate)
{
//Set up image/varibales
play_pause_BTN.BackgroundImage = VideoCapture.Properties.Resources.Stop;
FrameCount = 0;
SW = new Stopwatch();
//check to see if we have disposed of the video before
if (VW.Ptr == IntPtr.Zero)
{
//explain to the user what's happening
MessageBox.Show("VideoWriter has been finilised, please re-initalise a video file");
//lets re-call the recordVideoToolStripMenuItem_Click to save on programing
recordVideoToolStripMenuItem_Click(null, null);
}
SW.Start();
}
else
{
//Stop recording and dispose of the VideoWriter this will finialise the video
//Some codecs don't dispose correctley use uncompressed to stop this error
//VLC video player will play videos where the index has been corrupted. http://www.videolan.org/vlc/index.html
VW.Dispose();
//set image/variable
play_pause_BTN.BackgroundImage = VideoCapture.Properties.Resources.Record;
SW.Stop();
}
}
The Video_CNTRL_MouseCaptureChanged() event is called when the user has moved the trackbar on the form. This is prevented when recording or playing videos by disabling the trackbar. Only if the user is viewing videos and the video is not playing then this control is enable for the user. The event call does a few checks to see if the _Capture variable has been set up and see if it's running. Despite the checking to see if the _Capture is running this is never the case if it is allowed then the an attempt to pause the capture and adjust the frame position is made. This is buggy since there is cross threading issues. In the programs current state only the else statement will be actuated. This sets the position of the frame in the video and calls the ProcessFrame() method in order to show the current frame to the user.
private void Video_CNTRL_MouseCaptureChanged(object sender, EventArgs e)
{
if (_Capture != null)
{
//we don't use this when running since it has an unstable call and wil cause a crash
if (_Capture.GrabProcessState == System.Threading.ThreadState.Running)
{
_Capture.Pause();
while (_Capture.GrabProcessState == System.Threading.ThreadState.Running) ;//do nothing wait for stop
_Capture.SetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_FRAMES, Video_CNTRL.Value);
_Capture.Start();
}
else
{
_Capture.SetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_POS_FRAMES, Video_CNTRL.Value);
//call the process frame to update the picturebox
ProcessFrame(null, null);
}
}
}
openVideoToolStripMenuItem_Click...
recordVideoToolStripMenuItem_Click...
ThreadSafe Operations...
Methods Available
Used
- name()
Unused
- name()
Bugs
- Numbered list If any