Classification

In statistical learning, classification problems and approaches predict qualitative responses (think eye colour, for example).

In general, there’s no easy way to use quantitative analysis (like linear regression) on qualitative problems. For example:

• If we consider an encoding for qualitative variables (i.e., 1 is cholesterol level, 2 is age, 3 is blood type, and we try to predict cancer probability), that implies a certain ranking or importance in the relationship.
• If they did take a natural ranking of importance, then this would be fine, but many problems don’t have clear relationships in importance.
• If this was a binary problem (two encodings), it’s much easier to fit a linear regression model to this problem, because if we changed the encoding the regression will produce the same final predictions.

Generative models (discriminant analysis, naïve Bayes) are used to estimate the terms in the Bayes classifier.

Deep learning

The number of classes in the problem

Sub-pages

• Classifier models
  • Logistic regression
  • Linear discriminant analysis
  • Quadratic discriminant analysis
  • Naïve Bayes
  • K-nearest neighbours
  • SVC
  • Random forests
• Classification metrics
  • Accuracy, precision, recall, F1-score
• Error function
  • Cross entropy (and binary cross entropy)
  • KL divergence
  • Hinge loss

See also

• Regression