Andreu Sancho Homepage

As in many real-world situations, union makes algorithms stronger. With this philosophy in mind, ensemble methods combine several weak classifiers into a massive one---in terms of accuracy. In the last post we learnt a primer with Random Forest . Therefore, the next cornerstone is gradient boosting. I mentioned Gradient Boosting many times in this blog, but I only commented the fundamental ideas, without discussing further the details. In this entry I will share my two cents. Let me introduce a little bit of history, first: recall the Kaggle-Higgs competition . The top scores in the leaderboard have been obtained by using distinct forms of gradient boosting, and XGBoost is the direct responsible of many of these. The question is, hence, how does this algorithm work ? Figure 1. A high-level description of the Gradient Boosting method I programmed. Click to enlarge. Informally, Gradient Boosting generates a sequence of classifiers in the form of an additive expansion, that i...

The ever-increasing integration of technology in the different areas of science and industry has fostered practitioners the design of applications that generate stupendous amounts of data on-line (e.g., smart sensors or network monitoring just to mention two real cases). Extracting information from these data is key, in order to gain a better understanding of the processes that the data are describing. However, learning from these data poses new challenges to traditional data mining techniques, which are not designed to deal with data streams: data in which concepts and noise vary over time. In this regard, I am proud to present the supervised neural constructivist system ( SNCS ), a neural Michigan-style Learning Classifier System  that has been designed to provide a fast reaction capacity and adaptability to the distinct possible changes in concept (concept drifts) with varying noisy inputs.  Inheriting the intrinsically on-line fashion of Michigan-style learning clas...

One can define machine learning (ML) as programs that learn and improve with the use of the experience at some task using a measure of the performance. We can classify ML techniques basing on the desired outcome of the algorithm. There are, in the classic taxonomy, three main types of learning: (1) supervised learning , where an expert provides feedback in the learning process, (2) unsupervised learning , where there is no teacher or expert when the learning process is running, and (3) reinforcement learning , where the program learns interacting with the environment. Of all these, we will talk about the first one. Supervised learning is a ML method for extracting a model from training data. These data consist of a set (called examples ) of input attributes (sometimes called features ) and the desired output. As commented before, the main characteristic of supervised learning is that the program needs an expert or teacher that provides feedback in the learning process. Typically, the...