Would you like to take a course on Keras and deep learning in Python? Consider taking DataCamp’s Deep Learning in Python course!

Also, don’t miss our Keras cheat sheet, which shows you the six steps that you need to go through to build neural networks in Python with code examples!

Introducing Artificial Neural Networks

Before going deeper into Keras and how you can use it to get started with deep learning in Python, you should probably know a thing or two about neural networks. As you briefly read in the previous section, neural networks found their inspiration and biology, where the term “neural network” can also be used for neurons. The human brain is then an example of such a neural network, which is composed of a number of neurons.

And, as you all know, the brain is capable of performing quite complex computations and this is where the inspiration for Artificial Neural Networks comes from. The network a whole is a powerful modeling tool.

Perceptrons

The most simple neural network is the “perceptron”, which, in its simplest form, consists of a single neuron. Much like biological neurons, which have dendrites and axons, the single artificial neuron is a simple tree structure which has input nodes and a single output node, which is connected to each input node. Here’s a visual comparison of the two:

As you can see from the picture, there are six components to artificial neurons. From left to right, these are:

1. Input nodes. As it so happens, each input node is associated with a numerical value, which can be any real number. Remember that real numbers make up the full spectrum of numbers: they can be positive or negative, whole or decimal numbers.
2. Connections. Similarly, each connection that departs from the input node has a weight associated with it and this can also be any real number.
3. Next, all the values of the input nodes and weights of the connections are brought together: they are used as inputs for a weighted sum: y=f(∑Di=1wi∗xi)y=f(∑i=1Dwi∗xi), or, stated differently, y=f(w1∗x1+w2∗x2+...wD∗xD)y=f(w1∗x1+w2∗x2+...wD∗xD).
4. This result will be the input for a transfer or activation function. In the simplest but trivial case, this transfer function would be an identity function, f(x)=xf(x)=x or y=xy=x. In this case, xx is the weighted sum of the input nodes and the connections. However, just like a biological neuron only fires when a certain treshold is exceeded, the artificial neuron will also only fire when the sum of the inputs exceeds a treshold, let’s say for example 0. This is something that you can’t find back in an identity function! The most intuitive way that one can think of is by devising a system like the following:

   f(x)=0f(x)=0 if x<0x<0
   f(x)=0.5f(x)=0.5 if x=0x=0
   f(x)=1f(x)=1 if x>0x>0

5. Of course, you can already imagine that the output is not going to be a smooth line: it will be a discontinuous function. Because this can cause problems in the mathematical processing, a continuous variant, the sigmoid function, is often used. An example of a sigmoid function that you might already know is the logistic function. Using this function results in a much smoother result!
6. As a result, you have the output node, which is associated with the function (such as the sigmoid function) of the weighted sum of the input nodes. Note that the sigmoid function is a mathematical function that results in an “S” shaped curve; You’ll read more about this later.
7. Lastly, the perceptron may be an additional parameter, called a bias, which you can actually consider as the weight associated with an additional input node that is permanently set to 1. The bias value is important because it allows you to shift the activation function to the left or right, which can make a determine the success of your learning.

Note that the logical consequence of this model is that perceptrons only work with numerical data. This implies that you should convert any nominal data into a numerical format.

Now that you know that perceptrons work with tresholds, the step to using them for classification purproses isn’t that far off: the perceptron can agree that any output above a certain treshold indicates that an instance belongs to one class, while an output below the treshold might result in the input being a member of the other class. The straight line where the output equals the treshold is then the boundary between the two classes.

Multi-Layer Perceptrons

Networks of perceptrons are multi-layer perceptrons, and this is what this tutorial will implement in Python with the help of Keras! Multi-layer perceptrons are also known as “feed-forward neural networks”. As you sort of guessed it by now, these are more complex networks than the perceptron, as they consist of multiple neurons that are organized in layers. The number of layers is usually limited to two or three, but theoretically, there is no limit!

The layers act very much like the biological neurons that you have read about above: the outputs of one layer serve as the inputs for the next layer.

Among the layers, you can distinguish an input layer, hidden layers and an output layer. Multi-layer perceptrons are often fully connected. This means that there’s a connection from each perceptron in a certain layer to each perceptron in the next layer. Even though the connectedness is no requirement, this is typically the case.

Note that while the perceptron could only represent linear separations between classes, the multi-layer perceptron overcomes that limitation and can also represent more complex decision boundaries.

Predicting Wine Types: Red or White?

For this tutorial, you’ll use the wine quality data set that you can find in the wine quality data set from the UCI Machine Learning Repository. Ideally, you perform deep learning on bigger data sets but for the purpose of this tutorial, you will make use of a smaller one. This is mainly because the goal is to get you started with the library and to familiarize yourself with how neural networks work.

You might already know this data set, as it’s one of the most popular data sets to get started on learning how to work out machine learning problems. In this case, it will serve for you to get started with deep learning in Python with Keras.

Let’s get started now!

Understanding The Data

However, before you start loading in the data, it might be a good idea to check how much you really know about wine (in relation with the dataset, of course). Most of you will know that there are, in general, two very popular types of wine: red and white.

(I’m sure that there are many others, but for simplicity and because of my limited knowledge of wines, I’ll keep it at this. I’m sorry if I’m disappointing the true connoisseurs among you :)).

Knowing this is already one thing but if you want to analyze this data, you will need to know just a little bit more.

First, check out the data description folder to see which variables have been included. This is usually the first step to understanding your data. Go to this page to check out the description or keep on reading to get to know your data a little bit better.

The data consists of two datasets that are related to red and white variants of the Portuguese “Vinho Verde” wine. As stated in the description, you’ll only find physicochecmical and sensory variables included in this data set. The data description file just list the 12 variables that are included in the data, but for those who, like me, aren’t really chemistry experts either, here’s a short description of each variable:

1. Fixed acidity: acids are major wine properties and contribute greatly to the wine’s taste. Usually, the total acidity is divided into two groups: the volatile acids and the nonvolatile or fixed acids. Among the fixed acids that you can find in wines are the following: tartaric, malic, citric, and succinic. This variable is expressed in g(tartaricacidtartaricacid)/dm3dm3 in the data sets.
2. Volatile acidity: the volatile acidity is basically the process of wine turning into vinegar. In the U.S, the legal limits of Volatile Acidity are 1.2 g/L for red table wine and 1.1 g/L for white table wine. In these data sets, the volatile acidity is expressed in g(aceticacidaceticacid)/dm3dm3.
3. Citric acid is one of the fixed acids that you’ll find in wines. It’s expressed in g/dm3dm3 in the two data sets.
4. Residual sugar typically refers to the sugar remaining after fermentation stops, or is stopped. It’s expressed in g/dm3dm3 in the red and white data.
5. Chlorides can be a major contributor to saltiness in wine. Here, you’ll see that it’s expressed in g(sodiumchloridesodiumchloride)/dm3dm3.
6. Free sulfur dioxide: the part of the sulphur dioxide that is added to a wine and that is lost into it is said to be bound, while the active part is said to be free. Winemaker will always try to get the highest proportion of free sulphur to bind. This variables is expressed in mg/dm3dm3 in the data.
7. Total sulfur dioxide is the sum of the bound and the free sulfur dioxide (SO2). Here, it’s expressed in mg/dm3dm3. There are legal limits for sulfur levels in wines: in the EU, red wines can only have 160mg/L, while white and rose wines can have about 210mg/L. Sweet wines are allowed to have 400mg/L. For the US, the legal limits are set at 350mg/L and for Australia, this is 250mg/L.
8. Density is generally used as a measure of the conversion of sugar to alcohol. Here, it’s expressed in g/cm3cm3.
9. pH or the potential of hydrogen is a numeric scale to specify the acidity or basicity the wine. As you might know, solutions with a pH less than 7 are acidic, while solutions with a pH greater than 7 are basic. With a pH of 7, pure water is neutral. Most wines have a pH between 2.9 and 3.9 and are therefore acidic.
10. Sulphates are to wine as gluten is to food. You might already know sulphites from the headaches that they can cause. They are a regular part of the winemaking around the world and are considered necessary. In this case, they are expressed in g(potassiumsulphatepotassiumsulphate)/dm3dm3.
11. Alcohol: wine is an alcoholic beverage and as you know, the percentage of alcohol can vary from wine to wine. It shouldn’t surprised that this variable is inclued in the data sets, where it’s expressed in % vol.
12. Quality: wine experts graded the wine quality between 0 (very bad) and 10 (very excellent). The eventual number is the median of at least three evaluations made by those same wine experts.

This all, of course, is some very basic information that you might need to know to get started. If you’re a true wine connoisseur, you probably know all of this and more!

Now, it’s time to get your data!

Loading In The Data

This can be easily done with the Python data manipulation library Pandas. You follow the import convention and import the package under it’s alias, pd.

Next, you make use of the read_csv() function to read in the CSV files in which the data is stored. Additionally, use the sep argument to specify that the separator in this case is a semicolon and not a regular comma.

Try it out in the DataCamp Light chunk below:

Awesome! That wasn’t a piece of cake, wasn’t it?

You have probably done this a million times by now, but it’s always an essential step to get started. Now you’re completely set to start exploring, manipulating and modeling your data!

Data Exploration

With the data at hand, it’s easy for you to learn more about these wines! One of the first things that you’ll probably want to do is to start off with getting a quick view on both of your DataFrames:

Now is the time to check whether your import was successful: double check whether the data contains all the variables that the data description file of the UCI Machine Learning Repository promised you. Besides the number of variables, also check the quality of the import: are the data types correct? Did all the rows come through? Are there any null values that you should take into account when you’re cleaning up the data?

You might also want to check out your data with more than just info():

A brief recap of all these pandas functions: you see that head(), tail() and sample() are awesome because they provide you with a quick way of inspecting your data without any hassle.

Next, describe() offers some summary statistics about your data that can help you to assess your data quality. You see that some of the variables really have a lot of difference in their min and max values. This is something that you’ll deal with later, but at this point, it’s just very important to be aware of this.

Lastly, you have double checked the presence of null values in red with the help of isnull(). This is a function that always can come in handy when you’re still in doubt after having read the results of info().

Tip: also check out whether the wine data contains null values. You can do this by using the IPython shell of the DataCamp Light chunk which you see right above.

Now that you have already inspected your data to see if the import was successful and correct, it’s time to dig a little bit deeper.

Visualizing The Data

One way to do this is by looking at the distribution of some of the dataset’s variables and make scatter plots to see possible correlations. Of course, you can take this all to a much higher level if you would use this data for your own project.

Alcohol

One variable that you could find interesting at first sight is alcohol. It’s probably one of the first things that catches your attention when you’re inspecting a wine data set. You can visualize the distributions with any data visualization library, but in this case, the tutorial makes use of matplotlib to quickly plot the distributions:

As you can see in the image below, you see that the alcohol levels between the red and white wine are mostly the same: they have around 9% of alcohol. Of course, there are also a considerable amount of observations that have 10% or 11% of alcohol percentage.

Note that you can double check this if you use the histogram() function from the numpy package to compute the histogram of the white and red data, just like this

If you’re interested in matplotlib tutorials, make sure to check out DataCamp’s Matplotlib tutorial for beginners and Viewing 3D Volumetric Data tutorial, which shows you how to make use of Matplotlib’s event handler API.

Sulphates

Next, one thing that interests me is the relation between the sulphates and the quality of the wine. As you have read above, sulphates can cause people to have headaches and I’m wondering if this infuences the quality of the wine. What’s more, I often hear that women especially don’t want to drink wine exactly because it causes headaches. Maybe this affects the ratings for the red wine?

Let’s take a look.

As you can see in the image below, the red wine seems to contain more sulphates than the white wine, which has less sulphates above 1 g/dm3dm3. For the white wine, there only seem to be a couple of exceptions that fall just above 1 g/dm3dm3, while this is definitely more for the red wines. This could maybe explain the general saying that red wine causes headaches, but what about the quality?

You can clearly see that there is white wine with a relatively low amount of sulphates that gets a score of 9, but for the rest it’s difficult to interpret the data correctly at this point.

Of course, you need to take into account that the difference in observations could also affect the graphs and how you might interpret them.

Acidity

Apart from the sulphates, the acidity is one of the major and important wine characteristics that is necessary to achieve quality wines. Great wines often balance out acidity, tannin, alcohol and sweetness. Some more research taught me that in quantities of 0.2 to 0.4 g/L, volatile acidity doesn’t affect a wine’s quality. At higher levels, however, volatile acidity can give wine a sharp, vinegary tactile sensation. Extreme volatile acidity signifies a seriously flawed wine.

Let’s put the data to the test and make a scatter plot that plots the alcohol versus the volatile acidity. The data points should be colored according to their rating or quality label:

Note that the colors in this image are randomly chosen with the help of the NumPy random module. You can always change this by passing a list to the redcolors or whitecolors variables. Make sure that they are the same (except for 1 because the white wine data has one unique quality value more than the red wine data), though, otherwise your legends are not going to match!

Check out the full graph here:

In the image above, you see that the levels that you have read about above especially hold for the white wine: most wines with label 8 have volatile acidity levels of 0.5 or below, but whether or not it has an effect on the quality is too difficult to say, since all the data points are very densely packed towards one side of the graph.

This is just a quick data exploration. If you would be interested in elaborating this step in your own projects, consider DataCamp’s data exploration posts, such as Python Exploratory Data Analysis and Python Data Profiling tutorials, which will guide you through the basics of EDA.

Wrapping Up The Exploratory Data Analysis

This maybe was a lot to digest, so it’s never too late for a small recap of what you have seen during your EDA that could be important for the further course of this tutorial:

• Some of the variables of your data sets have values that are considerably far apart. You can and will deal with this in the next section of the tutorial.
• You have an ideal scenario: there are no null values in the data sets.
• Most wines that were included in the data set have around 9% of alcohol.
• Red wine seems to contain more sulphates than the white wine, which has less sulphates above 1 g/dm3dm3.
• You saw that most wines had a volatile acidity of 0.5 and below. At the moment, there is no direct relation to the quality of the wine.

Up until now, you have looked at the white wine and red wine data separately. The two seem to differ somewhat when you look at some of the variables from close up, and in other cases, the two seem to be very similar. Do you think that there could there be a way to classify entries based on their variables into white or red wine?

There is only one way to find out: preprocess the data and model it in such a way so that you can see what happens!

Preprocess Data

Now that you have explored your data, it’s time to act upon the insights that you have gained! Let’s preprocess the data so that you can start building your own neural network!

You set ignore_index to True in this case because you don’t want to keep the index labels of white when you’re appending the data to red: you want the labels to continue from where they left off in red, not duplicate index labels from joining both data sets together.

# Import `StandardScaler` from `sklearn.preprocessing`
from sklearn.preprocessing import StandardScaler# Define the scaler
scaler = StandardScaler().fit(X_train)# Scale the train set
X_train = scaler.transform(X_train)# Scale the test set
X_test = scaler.transform(X_test)

Compile and Fit

Next, it’s time to compile your model and fit the model to the data: once again, make use of compile() and fit() to get this done.

model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])model.fit(X_train, y_train,epochs=20, batch_size=1, verbose=1)

In compiling, you configure the model with the adam optimizer and the binary_crossentropy loss function. Additionally, you can also monitor the accuracy during the training by passing ['accuracy'] to the metricsargument.

The optimizer and the loss are two arguments that are required if you want to compile the model. Some of the most popular optimization algorithms used are the Stochastic Gradient Descent (SGD), ADAM and RMSprop. Depending on whichever algorithm you choose, you’ll need to tune certain parameters, such as learning rate or momentum. The choice for a loss function depends on the task that you have at hand: for example, for a regression problem, you’ll usually use the Mean Squared Error (MSE). As you see in this example, you used binary_crossentropy for the binary classification problem of determining whether a wine is red or white. Lastly, with multi-class classification, you’ll make use of categorical_crossentropy.

After, you can train the model for 20 epochs or iterations over all the samples in X_train and y_train, in batches of 1 sample. You can also specify the verbose argument. By setting it to 1, you indicate that you want to see progress bar logging.

In other words, you have to train the model for a specified number of epochs or exposures to the training dataset. An epoch is a single pass through the entire training set, followed by testing of the verification set. The batch size that you specify in the code above defines the number of samples that going to be propagated through the network. Also, by doing this, you optimize the efficiency because you make sure that you don’t load too many input patterns into memory at the same time.

Predict Values

Let’s put your model to use! You can make predictions for the labels of the test set with it. Just use predict() and pass the test set to it to predict the labels for the data. In this case, the result is stored in y_pred:

y_pred = model.predict(X_test)

Before you go and evaluate your model, you can already get a quick idea of the accuracy by checking how y_pred and y_test compare:

y_pred[:5]

array([[0],[1],[0],[0],[0]], dtype=int32)

y_test[:5]

array([0, 1, 0, 0, 0])

You see that these values seem to add up, but what is all of this without some hard numbers?

Evaluate Model

Now that you have built your model and used it to make predictions on data that your model hadn’t seen yet, it’s time to evaluate it’s performance. You can visually compare the predictions with the actual test labels (y_test), or you can use all types of metrics to determine the actual performance. In this case, you’ll use evaluate() to do this. Pass in the test data and test labels and if you want, put the verbose argument to 1. You’ll see more logs appearing when you do this.

score = model.evaluate(X_test, y_test,verbose=1)print(score)

[0.025217213829228164, 0.99487179487179489]

The score is a list that holds the combination of the loss and the accuracy. In this case, you see that both seem very great, but in this case it’s good to remember that your data was somewhat imbalanced: you had more white wine than red wine observations. The accuracy might just be reflecting the class distribution of your data because it’ll just predict white because those observations are abundantly present!

Before you start re-arranging the data and putting it together in a different way, it’s always a good idea to try out different evaluation metrics. For this, you can rely on scikit-learn (which you import as sklearn, just like before when you were making the train and test sets) for this.

In this case, you will test out some basic classification evaluation techniques, such as:

• The confusion matrix, which is a breakdown of predictions into a table showing correct predictions and the types of incorrect predictions made. Ideally, you will only see numbers in the diagonal, which means that all your predictions were correct!
• Precision is a measure of a classifier’s exactness. The higher the precision, the more accurate the classifier.
• Recall is a measure of a classifier’s completeness. The higher the recall, the more cases the classifier covers.
• The F1 Score or F-score is a weighted average of precision and recall.
• The Kappa or Cohen’s kappa is the classification accuracy normalized by the imbalance of the classes in the data.

# Import the modules from `sklearn.metrics`
from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score, cohen_kappa_score# Confusion matrix
confusion_matrix(y_test, y_pred)

array([[1585,    3],[   8,  549]])

# Precision
precision_score(y_test, y_pred)

0.994565217391

# Recall
recall_score(y_test, y_pred)

0.98563734290843807

# F1 score
f1_score(y_test,y_pred)

0.99008115419296661

# Cohen's kappa
cohen_kappa_score(y_test, y_pred)

0.98662321692498967

All these scores are very good! You have made a pretty accurate model despite the fact that you have considerably more rows that are of the white wine type.

Good job!

Some More Experiments

You’ve successfully built your first model, but you can go even further with this one. Why not try out the following things and see what their effect is? Like you read above, the two key architectural decisions that you need to make involve the layers and the hidden nodes. These are great starting points:

• You used 1 hidden layers. Try to use 2 or 3 hidden layers;
• Use layers with more hidden units or less hidden units;
• Take the quality column as the target labels and the rest of the data (including the encoded type column!) as your data. You now have a multi-class classification problem!

But why also not try out changing the activation function? Instead of relu, try using the tanh activation function and see what the result is!

Predicting Wine Quality

Your classification model performed perfectly for a first run!

But there is so much more that you can do besides going a level higher and trying out more complex structures than the multi-layer perceptron. Why not try to make a neural network to predict the wine quality?

In this case, the tutorial assumes that quality is a continuous variable: the task is then not a binary classification task but an ordinal regression task. It’s a type of regression that is used for predicting an ordinal variable: the quality value exists on an arbitrary scale where the relative ordering between the different quality values is significant. In this scale, the quality scale 0-10 for “very bad” to “very good” is such an example.

Note that you could also view this type of problem as a classification problem and consider the quality labels as fixed class labels.

In any case, this situation setup would mean that your target labels are going to be the quality column in your red and white DataFrames for the second part of this tutorial. This will require some additional preprocessing.

import numpy as np
from sklearn.model_selection import StratifiedKFoldseed = 7
np.random.seed(seed)kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=seed)
for train, test in kfold.split(X, Y):model = Sequential()model.add(Dense(64, input_dim=12, activation='relu'))model.add(Dense(1))model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])model.fit(X[train], Y[train], epochs=10, verbose=1)

mse_value, mae_value = model.evaluate(X[test], Y[test], verbose=0)print(mse_value)

0.522478731072

print(mae_value)

0.561965950103

from sklearn.metrics import r2_scorer2_score(Y[test], y_pred)

0.3125092543

model = Sequential()
model.add(Dense(64, input_dim=12, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1))

model = Sequential()
model.add(Dense(128, input_dim=12, activation='relu'))
model.add(Dense(1))

from keras.optimizers import RMSprop
rmsprop = RMSprop(lr=0.0001)
model.compile(optimizer=rmsprop, loss='mse', metrics=['mae'])

from keras.optimizers import SGD, RMSprop
sgd=SGD(lr=0.1)
model.compile(optimizer=sgd, loss='mse', metrics=['mae'])