In statistical learning, inference focuses on understanding the association between an output and a data set of inputs. This is contrasted with prediction. Think of it as interpretability.
Inference draws broad conclusions about the data, like the underlying distribution, better understanding relationships between variables, or parameters of interest.
More complicated learning methods tends to decrease the interpretability of the output. In particular, least squares, with a linear relationship between variables may be fairly good for inference. Neural networks may not be.