Azure Machine Learning – Qualitative Activity Recognition of Weight Lifting Exercises

Qualitative Activity Recognition of Weight Lifting ExercisesThis blog is about building a classifier on the Azure Machine Learning platform for qualitative activity recognition of weight lifting exercises. This classifier predicts if an exercise has been done correctly (class A).

We will use data from accelerometers on the belt, forearm, arm, and dumbell of 6 participants. They were asked to perform barbell lifts correctly and incorrectly in 5 different ways:

– Class A: exactly according to the specification
– Class B: throwing the elbows to the front
– Class C: lifting the dumbbell only halfway
– Class D: lowering the dumbbell only halfway
– Class E: throwing the hips to the front

The goal of this experiment is to predict the manner in which people did the exercise. This is the “classe” variable in the data set.

The data and more information is available from the website here (see the section on the Weight Lifting Exercise Dataset).

We started to build this classifier as an exercise for the Practical Machine Learning course on Coursera and the results of this prediction can be found here on Github.

However, as Microsoft recently launched the Azure Machine Learning (Azure ML) Studio, we were wondering if we could obtain the same results, building a model on Azure ML.

So we did! We trained 3 models and selected the Multiclass Decision Forest model, with an overall accuracy of 0.9961, which was actually -very slightly- better than our initial results.

The experiment can be downloaded here from the Azure Machine Learning Gallery.

Step 1: Getting the data

We first use the Reader module to obtain the data. The data is available in a .csv format:

Reader AzureML

Step 2: Clean missing data

We will only use those columns that have no missing values. All the other columns will removed in the Clean missing module.

Clean Missing data Azure ML

Step 3: Select required columns

We only need information about the sensors from the belt, forearm, arm, and dumbell sensors. The first columns are about the test person and the time frames, which we won’t need, so we exclude them with the Project columns module.

Select columns Azure ML

Step 4: Remove highy correlated columns (optional)

In this experiment, we will test several models. For one of them, we will remove the highly correlated columns. We use the findCorrelation() function from the caret package. This package is not installed by default. In order to use this package, please download and zip the package, upload it to Azure, drag it on the canvas, and then you can install it (see code) using the Execute R module.

# Map 1-based optional input ports to variables
wle <- maml.mapInputPort(1) # class: data.frame

# install caret package
library(caret, lib.loc=”.”,verbose=TRUE)

last <- as.numeric(ncol(wle))
prior <- last – 1

# set variables to numerics, except the “classe”
for (i in 1:prior) {
wle[,i] <- as.numeric(wle[,i])

# find the highly correlated variables
highly.cor <- findCorrelation(cor(wle[, -c(last)]), cutoff=0.9)

# remove highly correlated variables
wle <- wle[, -highly.cor]

# Select data.frame to be sent to the output Dataset port

Step 5: Split the dataset into a training and a test set

We split the dataset into 70% for training, and 30% for testing. We set the seed to make this experiment reproducible.  In this case, we didn’t use the stratefied split, as it didn’t improve the results. However, be aware that sometimes a normal split is not good enough.

Split Azure ML

Step 7: Train the models

Based on the nature of the data, we decided to test 3 models: Multiclass Decision Forest, Multiclass Decision Jungle, and Multiclass Decision Forest on the dataset where we removed the highly correlated columns. If you don’t know which parameters to use for specific models, you can use the Sweep Parameters module instead of the Train model module. The Sweep Parameters will test several settings and will come back with the best model settings.

Multiclass Decision Forest Azure ML

Multiclass Decision Jungle Azure ML

Step 8: Score the test data

With the Score model module, the 30% testing data will be scored with the trained model.

Step 9: Evaluate the models

With the Evaluate model module, we are able to evaluate the model. First, we get the overall results.

Evaluate model results Azure ML

Secondly, we get the confusion matrix, showing the results per class.

Confusion matrix Azure ML

Step 10: Conclusion

Comparing the results of the 3 models, we come to the conclusion that Model 1 (Multiclass Decision Forest without removing the highly correlated columns) has the best result. This model can also be published to a webservice, which we will cover in another blog.

classes precision model 1 precision model 2 precision model 3
A 0.9982 0.9988 0.9991
B 0.9965 0.9956 0.9956
C 0.9907 0.9830 0.9892
D 0.9933 0.9937 0.9938
E 1.0000 0.9991 0.9995
Average precision 0.9957 0.9940 0.9955


We also checked the final outcomes, using the test set as provided by Coursera. We therefore saved our best performing model as “Trained Model” and build a new experiment, using the test data as input set.

The final outcome was equal to the -correct- outcome of the Random Forest model, built in R.

Velloso, E.; Bulling, A.; Gellersen, H.; Ugulino, W.; Fuks, H. Qualitative Activity Recognition of Weight Lifting Exercises. Proceedings of 4th International Conference in Cooperation with SIGCHI (Augmented Human ’13) . Stuttgart, Germany: ACM SIGCHI, 2013.

Read more:

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.