Encoding Categorical Data

The one idea: Models only understand numbers, so every category has to become one — but how you turn "red/green/blue" or "small/medium/large" into numbers tells the model a story about order and distance that may or may not be true. Get that story wrong and the model learns a relationship that doesn't exist.

⚠️ The trap: Running LabelEncoder on a column like color turns blue→0, green→1, red→2 — and now every distance-based or coefficient-based algorithm reads that as "red is greater than green is greater than blue," and worse, "the gap from blue to green equals the gap from green to red." There was never an order in color. You encoded a lie that runs cleanly and returns plausible-looking predictions.

Ordinal vs. nominal: the distinction that decides everything

Ordinal categories have a real, agreed-upon order: T-shirt size (S < M < L < XL), education level (high school < bachelor's < master's), a satisfaction rating (poor < fair < good < excellent). The order carries information a model should be allowed to use.

Nominal categories have no order: color, city, payment method, product category. red is not "more than" blue — they're just different. There is no true encoding-as-a-number for a nominal category that doesn't invent a fake ordering, which is exactly why nominal features need a different treatment than ordinal ones.

Get this classification right first — it decides which encoding technique below is correct, and picking the wrong one is the single most common categorical-encoding mistake.

Ordinal features: map them by hand

There's no automatic way for code to know that XL outranks L — you have to supply the order:

import pandas…