A Tutorial On Principal Component Analysis with the Accord.NET Framework

Principal Component Analysis (PCA) is a technique for exploratory data analysis with many success applications in several research fields. It is often used in image processing, data analysis, data pre-processing, visualization and is often used as one of the most basic building steps in many complex algorithms.

One of the most popular resources for learning about PCA is the excellent tutorial due to Lindsay I Smith. On her tutorial, Lindsay gives an example application for PCA, presenting and discussing the steps involved in the analysis. 

Souza, C. R. "A Tutorial on Principal Component Analysis with the Accord.NET Framework".

Department of Computing, Federal University of São Carlos, Technical Report, 2012.

This said, the above technical report aims to show, discuss and otherwise present the reader to the Principal Component Analysis while also reproducing all Lindsay's example calculations using the Accord.NET Framework. The report comes with complete source C# code listings and also has a companion Visual Studio solution file containing all sample source codes ready to be tinkered inside a debug application. 

While the text leaves out the more detailed discussions about the exposed concepts to Lindsay and does not addresses them in full details, it presents some practical examples to reproduce most of the calculations given  by Lindsay on her tutorial using solely Accord.NET.

If you like a practical example on how to perform matrix operations in C#, this tutorial may help getting you started.

Sequence Classifiers in C#: Hidden Conditional Random Fields

After a preliminary article on hidden Markov models, some months ago I had finally posted the article on Hidden Conditional Random Fields (HCRF) on CodeProject. The HCRF is a discriminative model, forming the generative-discriminative pair with the hidden Markov model classifers.

This CodeProject article is a second on a series of articles about sequence classification, the first being about Hidden Markov Models. I've used this opportunity to write a little about generative versus discriminative models, and also provide a brief discussion on how Vapnik's ideas apply to these learning paradigms.



All the code available on those articles are also available within the Accord.NET Framework. Those articles provide good examples on how to use the framework and can be regarded as a practical implementation on how to use those models with the framework.



Complete framework documentation can be found live at Google Code. The framework has now been referred on 30+ publications over the years, and several more are already in the works, by me and users around the world.

Academical publications

Talking about publications, the framework has been used within my own research on Computer Vision. If you need help in understanding the inner workings of the HCRF, a more visual explanation on the HCRF derivation can also be found at the presentation I gave on Iberamia 2012 about Fingerspelling Recognition with Support Vector Machines and Hidden Conditional Random Fields [pdf].



An application to a more interesting problem, namely natural words drawn from Sign Languages using a Microsoft Kinect, has also been accepted for publication at the 9th International Conference on Machine Learning and Data Mining, MLDM 2013, and will be available publicly shortly.

As usual, hope you find it interesting!

Point and Call and the Windows Phone Ecosystem

So a few days ago I bought a Windows Phone device. So far, I am impressed with the Windows Phone ecosystem.



The nicest thing is that I was finally able to test an app I've been eager to try for months: Point and Call.

Point-and-call in everyday life.

Now, the most interesting part, is that this app uses part of the Accord.NET Framework to do its magic :-)



The app author, Antti Savolainen, was kind enough to share some details about his app. It uses part of the SVMs framelet from Accord.NET to do the digit recognition, mostly based on one of the earlier CodeProject articles I've posted in the past. Needless to say, Antti did an awesome job, as the SVM part was surely just a tiny fraction of all the work in preprocessing, adjusting, locating, and doing the right things at the right times that I would never be able to figure out alone :-)



Surely, he and his company, Sadiga, deserves all the credits for this neat app!



If you would like find more interesting uses of the Accord.NET Framework, don't forget to check the framework's Showcase page for details!

Sequence Classifiers in C#: Hidden Markov Models

Few days ago I published a new article on CodeProject, about classifiers based on banks of hidden Markov models to accomplish sequence classification.

While I have written about this subject in the past on this very blog, this time the article deals with Gaussian, continuous-density hidden Markov models rather than discrete ones. Plus, at this time the Accord.NET Framework has evolved much since that 2010 post, and the new article reflects most of the improvements and additions in those last two years.

In the meantime, this article is also serving as a hook to a future article, an article about Hidden Conditional Random Fields (HCRFs). The HCRF models can serve the same purpose as the HMMs but can be generalized to arbitrary graph structures and be trained discriminatively, which could be an advantage on classification tasks.



As always, I hope readers can find it a good read :-)

Direct3DX9NotFoundException Creating a Direct3D Device With SlimDX

Yet another note for future reference. If you are using SlimDX, and you get the exception:

        SlimDX.Direct3D9.Direct3DX9NotFoundException was unhandled
            Message=Direct3DX 9 was not found. Please install the latest
                DirectX end-user redistributable package from Microsoft.
            Source=SlimDX
            StackTrace:
                at SlimDX.Direct3D9.Direct3D..ctor()
            InnerException: System.Runtime.InteropServices.SEHException
                Message=External component has thrown an exception.
                Source=SlimDX
                ErrorCode=-2147467259
                StackTrace:
                   at D3DXCheckVersion(UInt32 , UInt32 )
                   at SlimDX.Direct3D9.Direct3D..ctor()
                InnerException:

when trying to create a SlimDX.Direct3D9.Direct3D object, most likely you will be able to solve this by placing D3DX9_43.dll into your output folder. Please note that the proper solution would be to install the full DirectX 9 runtime into your system, but it may be too much for some quick testing.

A similar error may occur when using SharpDX, but in SharpDX's case, the error message will be much more helpful and actually tell you what is missing.

The command "sn -Ra ... :VCEnd" exited with code 1

For today, a quick note for future reference. If you have a C++/CLI project and you are getting the error

  MSB3073: The command "sn -Ra "YourProject.dll" "YourKey.snk"
  :VCEnd" exited with code 1. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets

Then try adding the following two lines to your AssemblyInfo.cpp

  [assembly:AssemblyKeyFileAttribute("application_key.snk")];
  [assembly:AssemblyDelaySignAttribute(true)];

and check if it solves the issue. Hope this can be useful for others facing this same situation I encountered some days earlier.

Screencast Capture

I've recently started to record videos to demonstrate some capabilities of the Accord.NET Framework. Surprisingly, there were only a few, free, open source applications to achieve this goal - and none of them had all the features I needed.

It is, until I decided to roll my own.

Screencast Capture Lite is a tool for recording the desktop screen and saving it to a video file, preserving quality as much as possible. However, this does not mean it produces gigantic files which take a long time to be uploaded to the web. The application encodes everything using solely H624 in an almost lossless setting.

As a demonstration, please take a look on the Youtube video sample shown below. However, note that Youtube actually reduced the quality of the video, even if you watch it in HD. The local copy produced by Screencast Capture has an even higher quality than what is being shown, while the generated video file occupied less than 2 megabytes on disk.

And by the way what would be a better approach to demonstrate the capabilities of the AForge.NET and Accord.NET frameworks other than writing this application using them?

Well, actually this application has been created specifically for two things:

• to aid in the recording of instructional videos for the Accord.NET Framework, and;


• to serve itself as a demonstration of the use and capabilities of the AForge and Accord Frameworks. 

This means the application is written entirely in C# making extensive use of both aforementioned frameworks. The application is completely open source and free, distributed under the terms of the GPL, and a suitable project page is already being served on Google Code.

Hope you will find it interesting!

Deep Learning Artificial Neural Networks: Speech Recognition and Universal Translators

Happy new year everyone!



With the beginning of this year, I would like to share a video I wish I had found earlier. It is about the recent breakthrough given by Deep Neural Networks in the field of speech recognition - which, despite I had known was a breakthrough, I didn't know it was already leading to such surprising great results.

Deep neural networks are also available in the Accord.NET Framework. However, they've been a very recent addition - if you find any issues, bugs, or just wish to collaborate on development, please let me know!