Support Vector Machine

Support Vector Machine is a classification algorithm that distributes all the test patients (separated in two classes just like we did with the Linear Discriminant Analysis) in a space depending on their characteristics, and finds a threshold (border) between the classes. Once this border is established, the algorithm will be able to add new patients to this space, and see in which side of the border they fall on. This way, when we add a new patient, we will have a fairly accurate knowledge of which class they belong to.

How does it work?

With Support Vector Machine, the first thing we have to notice is that, when it comes to set the threshold, the key patients from each class are the ones closest to the other class. This is because the most important thing about the border is where a patient stops being class A and starts being class B. If a patient is far away from the threshold, we instantly know in what class it belongs. However, patients who are very close to the border could either be class A or class B, and this is why setting the threshold properly (taking into consideration only the closest points) is important.

The difficult thing this algorithm has to solve is: "What is the optimal threshold?". If we look at any dataset, chances are that there's more than one possible border we can draw.

To find the best border possible, the algorithm will try to maximize the distance between the border and the closest point from each dataset. This means that we will try to have the border as separated from the least remote point of each dataset as possible. To measure it we draw the only line that is both perpendicular to the border and coincident with the point we're interested in. In some cases, there will be more than one point at the same distance from the border, but it doesn't change the functioning of the algorithm.