Features and Labels

A spreadsheet is the easiest way to picture it. Each row is one example, such as one house that sold. The columns describing the house (bedrooms, square footage, neighborhood, year built) are features. The column you want to predict, the sale price, is the label. A model in supervised learning studies many rows where both are known, then predicts the label for new rows where only the features are known. The list of a row’s feature values, fed to the model as numbers, is its feature vector.

Labels can be numbers, as in regression, or categories such as “spam” and “not spam,” as in classification. Getting them usually takes people (data labeling): someone has to mark which emails were spam or which scans show a fracture. Labels collected this way also form the answer key used to score the model. Unsupervised methods work without labels and look for structure in the features alone (unsupervised learning).

Why it matters: most of the practical work in classic machine learning is choosing and cleaning features and getting trustworthy labels, not picking the algorithm. Deciding which raw data becomes which features is feature engineering. A feature that secretly contains the answer, such as a “refund issued” flag in a model predicting refunds, is data leakage. In large language models the vocabulary shifts: the input is a sequence of tokens, and during pre-training the “label” is simply the next token in the text. In interpretability research, “feature” has a narrower meaning, a concept a network represents internally, so check which sense a writer intends.