Tuesday, 15 January 2013

The command "sn -Ra ... :VCEnd" exited with code 1

For today, a quick note for future reference. If you have a C++/CLI project and you are getting the error




MSB3073: The command "sn -Ra "YourProject.dll" "YourKey.snk"
:VCEnd" exited with code 1. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets




Then try adding the following two lines to your AssemblyInfo.cpp




[assembly:AssemblyKeyFileAttribute("application_key.snk")];
[assembly:AssemblyDelaySignAttribute(true)];



and check if it solves the issue. Hope this can be useful for others facing this same situation I encountered some days earlier.

Monday, 7 January 2013

Screencast Capture

I've recently started to record videos to demonstrate some capabilities of the Accord.NET Framework. Surprisingly, there were only a few, free, open source applications to achieve this goal - and none of them had all the features I needed.




It is, until I decided to roll my own.








Screencast Capture Lite is a tool for recording the desktop screen and saving it to a video file, preserving quality as much as possible. However, this does not mean it produces gigantic files which take a long time to be uploaded to the web. The application encodes everything using solely H624 in an almost lossless setting.




As a demonstration, please take a look on the Youtube video sample shown below. However, note that Youtube actually reduced the quality of the video, even if you watch it in HD. The local copy produced by Screencast Capture has an even higher quality than what is being shown, while the generated video file occupied less than 2 megabytes on disk.








And by the way what would be a better approach to demonstrate the capabilities of the AForge.NET and Accord.NET frameworks other than writing this application using them?




Well, actually this application has been created specifically for two things:




  • to aid in the recording of instructional videos for the Accord.NET Framework, and;

  • to serve itself as a demonstration of the use and capabilities of the AForge and Accord Frameworks. 




This means the application is written entirely in C# making extensive use of both aforementioned frameworks. The application is completely open source and free, distributed under the terms of the GPL, and a suitable project page is already being served on Google Code.




Hope you will find it interesting!

Tuesday, 1 January 2013

Deep Learning Artificial Neural Networks: Speech Recognition and Universal Translators

Happy new year everyone!



With the beginning of this year, I would like to share a video I wish I had found earlier. It is about the recent breakthrough given by Deep Neural Networks in the field of speech recognition - which, despite I had known was a breakthrough, I didn't know it was already leading to such surprising great results.









Deep neural networks are also available in the Accord.NET Framework. However, they've been a very recent addition - if you find any issues, bugs, or just wish to collaborate on development, please let me know!

Wednesday, 26 December 2012

Deep Neural Networks and Restricted Boltzmann Machines

The new version of the Accord.NET brings a nice addition for those working with machine learning and pattern recognition: Deep Neural Networks and Restricted Boltzmann Machines.









Class diagram for Deep Neural Networks in the Accord.Neuro namespace.




Deep neural networks have been listed as a recent breakthrough in signal and image processing applications, such as in speech recognition and visual object detection. However, is not the neural networks which are the new things here; but rather, the learning algorithms. Neural Networks have existed for decades, but previous learning algorithms were unsuitable to learn networks with more than one or two hidden layers.


But why more layers?



The Universal Approximation Theorem (Cybenko 1989; Hornik 1991) states that a standard multi-layer activation neural network with a single hidden layer is already capable of approximating any arbitrary real function with arbitrary precision. Why then create networks with more than one layer?





To reduce complexity. Networks with a single hidden layer may arbitrarily approximate any function, but they may require an exponential number of neurons to do so. We can borrow a more tactile example from the electronics field. Any boolean function can be expressed using only a single layer of AND, OR and NOT gates (or even only NAND gates). However, one would hardly use only this to fully design, let's say, a computer processor. Rather, specific behaviors would be modeled in logic blocks, and those blocks would then be combined to form more complex blocks until we create a all-compassing block implementing the entire CPU.





The use of several hidden layers is no different. By allowing more layers we allow the network to model more complex behavior with less activation neurons; futhermore the first layers of the network may specialize on detecting more specific structures to help in the later classification. Dimensionality reduction and feature extraction could have been performed directly inside the network on its first layers rather than using specific separate algorithms. 



Do computers dream of electric sheep?



The key insight in learning deep networks was to apply a pre-training algorithm which could be used to tune individual hidden layers separately. Each layer is learned separately without supervision. This means the layers are able to learn features without knowing their corresponding output label. This is known as a pre-training algorithm because, after all layers have been learned unsupervised, a final supervised algorithm is used to fine-tune the network to perform the specific classification task at hand.







As shown in the class diagram on top of this post, Deep Networks are simply cascades of Restricted Boltzmann Machines (RBMs). Each layer of the final network is created by connecting the hidden layers of each RBM as if they were hidden layers of a single activation neural network.



Now, the most interesting part about this approach will given now. It is about one specific detail on how the RBMs are learned, which in turn allows a very interesting use of the final networks. As each layer is a RBM learned using an unsupervised algorithm, they can be seen as standard generative models. And if they are generative, they can be used to reconstruct what they have learned. And by sequentially alternating computation and reconstruction steps initialized with a random observation vector, the networks may produce patterns which have been created using solely they inner knowledge about the concepts it has learned. This may be seen fantastically close to the concept of a dream.



--



At this point I would also like to invite you to watch the video linked above. And if you like what you see, I also invite you to download the latest version of the Accord.NET Framework and experiment with those newly added features.



The new release also includes k-dimensional trees, also known as kd-trees, which can be use to speed up nearest neighbor lookups in algorithms which need it. They are particularly useful in algorithms such as the mean shift algorithm for data clustering, which has been included as well; and in instance classification algorithms such as the k-nearest neighbors.

Tuesday, 2 October 2012

Accord.NET and .NET 3.5

Quite some time ago, Accord.NET had been upgraded to .NET 4.0. This upgrade was made to advantage of some of the unique platform features. However, this was a drawback for those users who were still bound to .NET 3.5 and would like to use (or continue using) the framework.



So, some weeks ago I did finish creating a compatibility layer providing .NET3.5 support for the newest version of the framework. While some features had to be left aside, most of the framework is currently functional. A few features, however, had to be removed. Those include the Accord.Audio modules,  support for CancellationTokens on processor-intensive learning algorithms, some parallel processing capabilities and some of the Expression Tree manipulation in some numeric optimization algorithms and in Decision Trees.



With time, some of those could be re-implemented to work correctly under 3.5 as needed. You can leave a request, if you would like. Stay tuned!

Friday, 21 September 2012

The Mistery of Failing Tests - Or how to implement IDisposable correctly

Some days ago my Visual Studio tests started failing randomly. Curiously, the same tests which failed in a previous run would succeed if ran again. There seemed to be nothing wrong with the tests, except for the message "The agent process was stopped while the test was running" appearing the Test Results window.









It turns out this message tends to appear whenever an object throws an exception inside its destructor.



Luckly, only about a dozen classes in Accord.NET do need to implement finalizers. After a few searches, it turned down the symptoms were being caused by a NullReferenceException being thrown in one of those classes.



The solution is to always implement the Disposable pattern correctly.




Disposing managed code correctly


To do so, first, implement IDisposable. Consider, for example, the Metronome class, which makes use of timers:







    /// <summary>
/// Virtual Metronome.
/// </summary>
///
/// <remarks>
/// Objects from this class acts as virtual metronomes. If connected
/// to a beat detector, it can be used to determine the tempo (in
/// beats per minute) of a signal. It can also be used in manual mode
/// by calling <see cref="Tap"/> method. For more details, see the
/// Beat detection sample application which comes together with the
/// framework.
/// </remarks>
///
public class Metronome : IDisposable
{
// ...

private Timer timeUp;
private Timer metronome;

// ...
}



Then, implement the required Dispose method, but do not write the dispose code right away. Instead, write:






        /// <summary>
/// Performs application-defined tasks associated with
/// freeing, releasing, or resetting unmanaged resources.
/// </summary>
///
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}

/// <summary>
/// Releases unmanaged and - optionally - managed resources
/// </summary>
///
/// <param name="disposing"><c>true</c> to release both managed
/// and unmanaged resources; <c>false</c> to release only unmanaged
/// resources.</param>
///
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
// free managed resources
if (timeUp != null)
{
timeUp.Dispose();
timeUp = null;
}
if (metronome != null)
{
metronome.Dispose();
metronome = null;
}
}
}



Always check if a member is not null before disposing it. Alternatively, after disposing, always set it to null so you can't accidentaly dispose it twice. The GC.SuppressFinalize instructs the Garbage Collector that the object has already been disposed, so it won't be required to call its finalizer. The GC will be trusting you have already cleaned the room; so don't fail on it!



Finally, write the finalizer as:







        /// <summary>
/// Releases unmanaged resources and performs other cleanup operations before the
/// <see cref="Metronome"/> is reclaimed by garbage collection.
/// </summary>
///
~Metronome()
{
Dispose(false);
}






Disposing unmanaged code



The disposal of unmanaged code follows exactly the same pattern as above, but some care is required when disposing unmanaged resources. Consider the Signal class, which represents an audio signal in PCM format:








    /// <summary>
/// Represents a PCM sound discrete signal (measured in time).
/// A real discrete-time signal is defined as any real-valued
/// function of the integers.
/// </summary>
///
/// <remarks>
/// <para>
/// In signal processing, sampling is the reduction of a continuous
/// signal to a discrete signal. A common example is the conversion
/// of a sound wave (a continuous-time signal) to a sequence of samples
/// (a discrete-time signal).</para>
///
/// <para>
/// A sample refers to a value or set of values at a point in time
/// and/or space.</para>
///
/// </remarks>
///
public class Signal : IDisposable
{
// ...

private byte[] rawData;
private IntPtr ptrData;
private GCHandle handle;

private SampleFormat format;

private int channels;
private int sampleRate;
private int length;

// ...
}





Note that this class uses an unmanaged memory block to store the audio signal. To dispose it properly, the Disposable pattern has to be implemented as it follows:







        /// <summary>
/// Performs application-defined tasks associated with freeing,
/// releasing, or resetting unmanaged resources.
/// </summary>
///
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}

/// <summary>
/// Releases unmanaged resources and performs other cleanup operations
/// before the <see cref="Signal"/> is reclaimed by garbage collection.
/// </summary>
///
~Signal()
{
Dispose(false);
}

/// <summary>
/// Releases unmanaged and - optionally - managed resources
/// </summary>
///
/// <param name="disposing"><c>true</c> to release both managed
/// and unmanaged resources; <c>false</c> to release only unmanaged
/// resources.</param>
///
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
// free managed resources
}

// free native resources
if (handle.IsAllocated)
{
handle.Free();
ptrData = IntPtr.Zero;
}
}





Here we have almost the same pattern. However, note that unmanaged code should always be freed, even if Dispose has not been called explictly. So the code for releasing unmanaged resources is executed unconditionally in the Dispose(bool) method.

Sunday, 9 September 2012

Haar Feature Face Detection in C#

A new article has been published in CodeProject! This article details the Viola-Jones face detection algorithm available in the Accord.NET Framework. The article page also provides a standalone package for face detection which can be reused without instantiating the entire framework.




 










The article is also participating in CodeProject's monthly competition under the categories Best C# Article and Best Overall Article of the month! Feel free to cast your vote if you liked and otherwise found them useful!  





This month we have other great articles participating in the competition: Marcelo de Oliveira's Social News excels in the Web category; Roy's Inline MSIL in C# also presents a very interesting reading in the C# category. The later may eventually be extremely useful to leverage performance in managed applications. 





For those who don't know, CodeProject is a amazingly useful site which publishes user-created articles and news. Every month, the best articles among all submissions are selected to win prizes and gain popularity as well!