The input parameters for which the output is about to be predicted, are called as features, while the output(s) to be predicted, are called as labels.
Each feature vector (row) consists of features such as BP, Insulin Amount, BMI, etc. and the Outcome (Whether the person has diabetes or not) is the only label.
Bias: the error between the actual value of a label and the value predicted by the model.
Variance: The measure of spread of the features given in the train split
When a model is overfit (accuracy tending to 100%), The training error will tend to zero, but it will fail to predict any other data with accuracy, such as the test set. To avoid getting overfit due to high variances, the bias may be allowed to increase.
As mentioned as “degree of randomness” in chemistry, similarly in machine learning, Entropy denotes the probability of detecting an output other than the expected output (impurity)
Confusion Matrix is a matrix of different possible situations on training a model.
Example : For a binary classification as shown in figure
TP : True Positive (The output is predicted true, and it is actually true)
TN : True Negative (The output is predicted false, and it is actually false)
FP : False Positive (The output is predicted true, but it is actually false)
(Type 1 Error)
FN : False Negative (The output is predicted false, and it is actually true)
(Type 2 Error)
Precision
$$ Precision = \frac{TP}{TP+FP} $$
Recall
$$ Recall = \frac{TP}{TP+FN} $$
Fig : A binary confusion matrix of single boolean outcome.
It is the harmonic mean of precision and recall
$$ F1 = \frac{2}{{\frac{1}{precision}}+{\frac{1}{recall}}} $$
Entropy is a measure of randomness / impurity in a sample. A Model with a minimum entropy in classification will have more accuracy.
