In statistical learning, bias is the error introduced by approximating a problem by a much simpler model. For example, when we do linear regression, we assume a linear relationship for variables that may not have that relationship.