Andreu Sancho Homepage

Today’s mid- and high-end computers come with a tremendous hardware, mostly used in video games and other media software, that can be exploited for advanced computation, that is: High Performance Computing (HPC). This is a hot topic in Deep Learning as modern graphic cards come with huge streaming process power and large and quick memory. The most successful example is in Nvidia’s CUDA platform. In summary, CUDA significantly speeds up the fitting of large neural nets (for instance: from several hours to just a few minutes!). However, the drawbacks come when setting up the scenario: it is non-trivial to install the requirements and set it running, and personally I had a little trouble the first time as many packages need to be manually compiled and installed in a specific order. The purpose of this entry is to reflect what I did for setting up Theano and Keras with HPC using an Nvidia’s graphic card (in my case a GT730) using GNU/Linux. To do so, I will start assuming a clean Debi...

I am glad of sharing some very motivating news: after a little bit more than year, Kaggle hosts a new CERN/LHC competition ! This time it comes from the tau disintegration into three muons. Again, the challenge consists of identifying the pattern from both real and simulated data to identify this rare phenomenon. The training data set consists of 49 distinct real-valued features plus the signal (i.e., background noise and the proper event). This time I am using Python with Scikit learn, Pandas and XGBoost (this latter one by far my favorite ensemble package; also available for R!) and I have to confess that is much easier when you do not have to program everything from scratch (as I usually do in C/C++ or Java), and that I have more time to think in new ways of solving the problem from a pure data scientist view. However, I really adore making use of use my own programs (I ended up 24th out of 1691 in the Forest Cover Type challenge using my own Extra Trees/Random Forest progra...

We live in a dynamic world where new technologies arise, dominate for a brief blink of an eye and then fall to oblivion rapidly. A part from the academia domain, it is interesting to check what are the current trends in the industry. Or even better: see the whole picture without discriminating. In this regard, a nice analytics tool is found in Google Trends, which allow us to track our terms of interest. The following chart shows the search interest for Machine Learning and Data Analytics . In a similar scale I depicted the search interest for Data Analytics and Big Data . As it is clearly noticeable, knowing both disciplines can open many doors in the industry. Finally, as top and also rising queries, Hadoop is the most popular term.

As in many real-world situations, union makes algorithms stronger. With this philosophy in mind, ensemble methods combine several weak classifiers into a massive one---in terms of accuracy. In the last post we learnt a primer with Random Forest . Therefore, the next cornerstone is gradient boosting. I mentioned Gradient Boosting many times in this blog, but I only commented the fundamental ideas, without discussing further the details. In this entry I will share my two cents. Let me introduce a little bit of history, first: recall the Kaggle-Higgs competition . The top scores in the leaderboard have been obtained by using distinct forms of gradient boosting, and XGBoost is the direct responsible of many of these. The question is, hence, how does this algorithm work ? Figure 1. A high-level description of the Gradient Boosting method I programmed. Click to enlarge. Informally, Gradient Boosting generates a sequence of classifiers in the form of an additive expansion, that i...

Several machine learning techniques foster the cooperation between subsolutions for obtaining an accurate outcome by combining many of them. Michigan-style LCSs are one of these families. However, the most well known are those that implement Boosting, and AdaBoost is the most successful of them (or, at least, the most studied one and the first to implement the ideas of Boosting). AdaBoost generates accurate predictions by combining several weak classifiers (also referred to as “the wisdom of the crowd”). The most outstanding fact about AdaBoost is that it is a deadly simple algorithm. The key of its success lies in the combination of many weak classifiers: these are very limited and their error rate is just slightly better than a random choice (hence their name). The typical implementation uses decision stumps (i.e., binary trees) that minimize the error between the prediction and the ground truth (that is, the desired outcome), so the classifiers have the form " if variable_...

An important challenge is learning from domains that do not have the same proportion of classes, that is, learning from problems that contain class imbalances (Orriols-Puig, 2008). Figure 1 shows a toy example of this issue. Notwithstanding, it is challenging because (1) in many real-world problems we cannot assume a balanced distribution of classes and (2) traditional machine learning algorithms cannot induce accurate models in such domains. Oftentimes it happens that the key knowledge to solve a problem that previously eluded solution is hidden in patterns that are rare . To tackle this issue, practitioners rely on re-sampling techniques, that is, algorithms that pre-process the data sets and either (1) add synthetic instances of the minority pattern to the original data or (2) eliminate instances from the majority class. The first type is called over-sampling and the later, under-sampling .  Figure 1. Our unbalanced data set. Black dots are the majority class (0) and ...

It is time to announce the Learning Classifier System Open Repository (LCSOR), a meeting point to foster the popularization of these exciting evolutionary computation techniques. In this repository I will upload distinct implementations of different Learning Classifier Systems. To start with, I uploaded a first version of the sUpervised Classifier System (UCS). Notice I created a page for this contend under the label LCSOR . I hope practitioners will enjoy this experience.

The discipline of machine learning (ML) can be understood as a particular study of data. Data is everything , and it contains patterns and a certain degree of uncertainty. The aim of the field is to identify those and exploit them to solve complex problems. Often algorithms for obtaining such patterns are not easy to implement, involving abstract mathematical concepts and vague descriptions of the internal processes that guide the learning mechanisms. In this particularly discouraging environment there are several useful tools to help with the understanding of the ML techniques. Octave is such a tool. Octave allows the extensive use of vectorization to exploit parallelism and modeling complex algorithms in a very fourth-generation programming language fashion. I started with Octave due to Standord's ml-class course and I have to say that the experience has been very positive. In one of the exercises we had to implement a handwritten character recognizer, that is, a type of percep...

" Machine learning is the science of getting computers to act without being explicitly programmed. " This is a very good definition of what ML is. But let's see what topics are included in the course: (1) supervised learning, (2) unsupervised learning, (3) best practices in machine learning, and (4) reinforcement learning (the course syllabus and the scheduling is found here .) A quite complete course; I only miss evolutive computation, anyway I think this course is a good way to enter to this wonderful discipline. You can sign up freely here (I did and I will post the experience here.) Stanford also offers another two free online courses: introduction to artificial intelligence and introduction to data bases .