AI Engineer Exhaustive
EasyFeature Engineering

Ordinal Encoding

Feature Engineering

Easy

Problem

Ordinal encoding maps ordered categories to integer positions. The ordering list defines the rank explicitly, with its first category mapped to 0, its second category mapped to 1, and so on.

Every input value appears in ordering. Return one integer for each value, preserving the original sequence.

Theory

Ordinal encoding is a technique for converting categorical features into numerical values by assigning integers to categories based on their inherent order or rank. Unlike one-hot encoding, it produces a single column with ordered integer values.

For categories with natural order (like education level or size), ordinal encoding preserves this meaningful ranking in the numerical representation.

When to Use Ordinal Encoding

Ordinal encoding is appropriate when:

Examples of ordinal features:

The Basic Process

Step 1: Identify the natural order of categories

Step 2: Assign integers starting from 0 (or 1) based on the order

Step 3: Replace each category with its assigned integer

Step 4: Store the mapping for consistent encoding of new data

Worked Example: Education Level

Categories (ordered):

  1. High School
  2. Bachelor's
  3. Master's
  4. PhD

Mapping:

Original data:

[Bachelor's, PhD, High School, Master's, Bachelor's]

Encoded data:

[1, 3, 0, 2, 1]

Worked Example: T-Shirt Size

Categories (ordered):

Mapping:

Original: [M, L, S, XL, M, XXL]

Encoded: [2, 3, 1, 4, 2, 5]

Ordinal vs Label Encoding

Ordinal encoding:

Label encoding:

Example:

Ordinal vs One-Hot Encoding

Ordinal encoding:

One-hot encoding:

Trade-off: Ordinal is more compact but imposes an ordering that may or may not be appropriate.

Mathematical Representation

Given categories C = \{c_1, c_2, ..., c_k\} with order c_1 < c_2 < ... < c_k:

f(c_i) = i - 1

This maps:

Alternatively, starting from 1:

f(c_i) = i

Custom Starting Points

You can start from any integer:

Starting from 1:

Starting from 0 (more common in programming):

Choose based on your model's requirements or conventions.

Non-Uniform Spacing

Sometimes the gap between categories is not equal:

Example: Pain scale

This reflects that the jump from Mild to Moderate is larger than from No pain to Mild.

Use with caution: Only if you have domain knowledge supporting the spacing.

Handling Unknown Categories

When new data contains unseen categories:

Option 1: Assign special value

f(c_{unknown}) = -1 \text{ or } k

Option 2: Raise an error

Force explicit handling of unexpected categories.

Option 3: Map to most frequent category

Use the mode of the training data.

Option 4: Map to middle category

Neutral assumption when order is unknown.

Handling Missing Values

Option 1: Separate encoding

f(\text{NaN}) = -1 \text{ or } k

Option 2: Impute before encoding

Fill with mode, median category, or domain-specific default.

Option 3: Preserve as NaN

Let the model handle missing values directly.

Ordinal Encoding for Tree-Based Models

Decision trees and random forests can naturally handle ordinal features:

Benefit: Trees can find optimal split points within the ordered values.

Example:

If Education encoded as [0, 1, 2, 3], tree might split at Education < 2 (separating High School/Bachelor's from Master's/PhD).

Ordinal Encoding for Linear Models

Linear models treat ordinal encoding as numerical:

y = \beta_0 + \beta_1 \cdot \text{Education}

Assumption: Equal spacing is meaningful.

If Education increases by 1 (e.g., from Bachelor's to Master's), y changes by \beta_1.

Problem: Is the effect of Bachelor's to Master's the same as High School to Bachelor's?

If not, consider one-hot encoding or polynomial terms.

Common Ordinal Features

Customer satisfaction:

Risk level:

Age groups:

Ordinal Encoding in Surveys

Survey responses often use Likert scales:

Agreement scale:

Frequency scale:

Verifying Order Correctness

Before applying ordinal encoding, verify the order makes sense:

Questions to ask:

  1. Is there a clear greater than / less than relationship?
  2. Would the domain expert agree on the ordering?
  3. Does the order have predictive meaning for the target?

If no clear order exists: Use one-hot encoding instead.

Ordinal Encoding with Pandas Example Concept

Conceptual process:

  1. Define the category order explicitly
  2. Create a mapping dictionary
  3. Apply the mapping to the data

Important: Always explicitly define the order rather than relying on alphabetical or appearance order.

Multi-Level Ordinal Features

Some ordinal features have hierarchical levels:

Example: Military rank

Enlisted ranks (E1-E9) < Warrant officers (W1-W5) < Commissioned officers (O1-O10)

Approach 1: Single ordinal encoding across all levels

Approach 2: Separate ordinal encodings for each level plus a level indicator

Ordinal Encoding and Feature Interactions

Ordinal features can be used in interactions:

Example:

Interaction: Education_level * Experience

This allows the effect of experience to vary by education level.

Testing the Ordinality Assumption

Check if ordinality helps:

  1. Fit model with ordinal encoding
  2. Fit model with one-hot encoding
  3. Compare performance on validation data

If one-hot significantly outperforms: The ordinality assumption may not hold.

Ordinal Encoding in Ensemble Methods

Random Forests:

Ordinal encoding works well because trees can split at any threshold.

Gradient Boosting (XGBoost, LightGBM):

Also handles ordinal features well. LightGBM has native support for categorical features.

Neural Networks:

Can use ordinal encoding, but embeddings may capture richer relationships.

Potential Problems

1. Implied equidistance:

Ordinal encoding implies equal spacing between categories, which may not be true.

2. Numerical operations:

Some models might compute means or other operations that are meaningless for ordinal data. Mean of [Master's, Bachelor's] is not meaningful as (2 + 1) / 2 = 1.5.

3. Wrong order:

Incorrect ordering can hurt model performance significantly.

Ordinal vs Numeric

Sometimes ordinal data appears numeric:

Example: Star ratings 1-5

This is ordinal, not truly numeric, because:

However, treating it as numeric often works well in practice.

Monotonic Relationships

Ordinal encoding assumes monotonic relationship with target:

Monotonically increasing:

Higher category value corresponds to higher (or lower) target value consistently.

Non-monotonic:

Relationship is not consistent. Middle categories might have different behavior.

If relationship is non-monotonic, one-hot encoding may be better.

Best Practices

1. Verify natural ordering:

Ensure the categories have a genuine ordinal relationship.

2. Document the mapping:

Keep clear records of which integer maps to which category.

3. Be consistent:

Use the same mapping for training and inference.

4. Consider alternatives:

If performance is poor, try one-hot encoding.

5. Handle unknowns:

Plan for how to handle new categories in production.

Common Mistakes

1. Assuming all categorical features are ordinal:

Colors, names, and IDs are not ordinal.

2. Using wrong order:

Assigning integers without considering the true ranking.

3. Relying on default alphabetical order:

Alphabetical order rarely matches meaningful order.

4. Forgetting to store the mapping:

Unable to encode new data consistently.

5. Treating as continuous:

Computing statistics like mean and standard deviation on ordinal data.

Examples

Example 1

Input
values = ["low", "medium", "high", "medium"], ordering = ["low", "medium", "high"]
Output
[0, 1, 2, 1]
Explanation
The categories occupy positions 0, 1, and 2 in the supplied ordering.

Example 2

Input
values = ["S", "M", "L", "XL", "S"], ordering = ["S", "M", "L", "XL"]
Output
[0, 1, 2, 3, 0]

Hints

  1. Create a dictionary from each ordered category to its index.
  2. Look up each input value in that dictionary.

Requirements

Constraints

Starter Code

def ordinal_encoding(values: list, ordering: list) -> list:
    """
    Returns the ordinal index of every input value.
    """
    # Write code here
    pass

Test Cases

CaseMatches
basic—public
sizes—public