Machine Learning

Machine learning is a branch of computer science algorithms that allow the computer to display behavior learned from past experience, rather than human instruction. Machine learning is essential to the development of artificial intelligence, but it is also applicable to many everyday computing tasks. Common examples of programs that employ machine learning include e-mail spam filters, optical character recognition, and news feeds on social networking sites that alter their displays based on previous user activity and preferences.

One of the earliest attempts to enable machine learning was the perceptron algorithm, developed by Frank Rosenblatt in 1957. The algorithm was intended to teach pattern recognition and was based on the structure of a neural network, which is a computing model designed to imitate an animal's central nervous system. While the perceptron model showed early promise, Marvin Minsky and Seymour Papert demonstrated in their 1969 book Perceptrons: An Introduction to Computational Geometry that it had significant limitations, as there were certain classes of problems the model was unable to learn. Consequently, researchers did not pursue this area of study for some time.

In the 1980s, after other avenues for developing artificial intelligence had resulted in only limited success, scientists began to revisit the perceptron model. Multilayer perceptrons, or neural networks composed of multiple layered computational units, proved to have the processing power to express problems that Rosenblatt's single-layer, or linear, perceptrons could not. Around the same time, John Ross Quinlan introduced decision-tree algorithms, which use predictive models to determine a variable's value based on available data.

Since then, numerous machine-learning algorithms have been developed. Among those most commonly used are support vector machines (SVMs) and naive Bayes classifiers. SVMs, introduced by Vladimir N. Vapnik and Corinna Cortes in 1995 and based on an algorithm previously created by Vapnik, are used to recognize patterns in data and classify the various data points. Naive Bayes classifiers are applications of Bayes's theorem, named for the eighteenth-century mathematician and reverend Thomas Bayes, which deals with conditional probabilities. This algorithm was used in one of the earliest e-mail spam filters, iFile, released by Jason Rennie in 1996. Many e-mail clients still employ Bayesian spam filtering, which works by determining the probability that an e-mail containing certain keywords is spam.

Machine-learning algorithms can be divided into categories based on how they train the machine. These categories include supervised learning, in which the machine learns from inputs that are mapped to desired outputs; unsupervised learning, in which the machine analyzes input without knowledge of the desired output; semi-supervised learning, in which some of the input is paired with a desired output and some is not; transduction, in which the machine tries to predict new outputs based on training with previous inputs and outputs; reinforcement learning, in which the machine must form a policy on how to act based on observing how certain actions affect its environment; and learning to learn, which teaches inductive bias based on previous experience. SVMs, multilayer perceptrons, decision trees, and naive Bayes classifiers all fall into the category of supervised learning.

It is important to distinguish between machine learning and data mining. Although the two concepts are related and use similar, often overlapping, methods, data mining is focused on discovering information about given data, while machine learning focuses more on learning from the given data in order to make predictions about other data in the future. Many consider data mining to be a subset of machine learning.

—Randa Tantawi, PhD