DAY 14-100 DAYS MLCODE: SVM

This blog is continuation of our previous blog about SVM.  In today’s blog, we are going to focus other classes of the SciKit Learn to implement the SVM for linear and none linear classification.

In last blog we used the LinearSVC class of SciKit-Learn to implement the the SVM .  Let’s start creating the linear classification using the SVC class of the SciKit-Learn. we can pass the value of kernel as “linear” to implement linear classification.

Test the SVM Model

Just to remind that this is continuation of our previous mode to detect whether the digit is 7 or not.

# Predict for One Observation (image)
import random
i = random.randint(0, (len(X_test)-1))
print(f"Prediction is {svm_clf.predict(X_test[i].reshape(1,-1))} and the actual value is {y_test[i]}")

Output: Prediction is [1] and the actual value is 1

Let’s print the digit to verify whether Digit is 7 or not.

plt.figure(figsize=(20,4))
plt.imshow(np.reshape(X_test[i], (8,8)) ,cmap=plt.cm.gray)
plt.title('Training: %i\n' % y_test[i], fontsize = 20)

Evaluate the Model:

We are going to use the accuracy class of SciKit-Learn to measure performance

from sklearn.metrics import accuracy_score

y_pred = svm_clf.predict(X_test)
accuracy_score(y_test, y_pred)

Output: 0.9944444444444445

This result is almost same as previous model of day’s 13. I think this accuracy is not perfect but in this blog our aim is not measure the performance of model so I’ll move to next topic.

None Linear SVM Classification:

Linear model works very well if the data is linearly separable but in real life, data are very rarely separated by straight line. In our previous blog, we have seen that how the pipe line with polynomial features helps to fit the complex data and work perfect. But there is an issue with performance. Like if polynomial degree is low but if data is complex, the above technique suffer the performance issue. On other hand, if the Polynomial degree is hight to fit the data, its suffer issues because it creates huge numbers of features.

There is another way to handle the data which is complex in nature and can not separated by linear models.  Its called Kernel trick, we shall use the kernel value as “poly” of SVC class without adding any new polynomial feature in pipe line.

Let’s create a None Linear SVM classification using the SVC class on the Iris Dataset

Downlod the Iris data and split into Train and Test data ( 80%, 20%)

iris_data = datasets.load_iris()
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(iris_data.data, iris_data.target, test_size=0.20, random_state=42)

Now, lets training the model with degree 2  :

poly_clf = Pipeline([
        ("scaler", StandardScaler()),
        ("svm_clf", SVC(kernel="poly", degree=2, coef0=1, C=5))
    ])
poly_clf.fit(X, y)egree=3, coef0=1, C=5))
    ])
poly_clf.fit(X, y)

Output:Pipeline(memory=None, steps=[(‘scaler’, StandardScaler(copy=True, with_mean=True, with_std=True)), (‘svm_clf’, SVC(C=5, cache_size=200, class_weight=None, coef0=1, decision_function_shape=’ovr’, degree=2, gamma=’auto’, kernel=’poly’, max_iter=-1, probability=False, random_state=None, shrinking=True, tol=0.001, verbose=False))])

Now let’s plot the graph to see how our model has drawn the decision boundary suing kernel “Poly”:

plt.figure(1)
plt.clf()
plt.scatter(X[:, 0], X[:, 1], c=y, zorder=10, cmap=plt.cm.Paired,
            edgecolor='k', s=20)

# Circle out the test data
plt.scatter(X_test[:, 0], X_test[:, 1], s=80, facecolors='none',
            zorder=10, edgecolor='k')

plt.axis('tight')
x_min = X[:, 0].min()
x_max = X[:, 0].max()
y_min = X[:, 1].min()
y_max = X[:, 1].max()

XX, YY = np.mgrid[x_min:x_max:200j, y_min:y_max:200j]
Z = poly_clf.decision_function(np.c_[a.ravel(), YY.ravel()])

# Put the result into a color plot
Z = Z.reshape(XX.shape)
plt.pcolormesh(XX, YY, Z > 0, cmap=plt.cm.Paired)
plt.contour(XX, YY, Z, colors=['k', 'k', 'k'],
            linestyles=['--', '-', '--'], levels=[-.5, 0, .5])

plt.title("None Linear Classification using SVC Class")

From the above two plots, it’s clear that polynomial with 10 degree of features try to fit the data . This may bring the overfitting problem.

In conclusion, two we developed the both Linear and None Linear machine learning models with SVC class of SciKit-Learns. Although SVC class is not recommended for the Linear classification if you have large data, but this call is useful for the None Linear Classification. You can find the enitre code here.