Numeric Features

Raw numeric columns are rarely model-ready. Income distributions are right-skewed with extreme outliers. Ages benefit from grouping into life-stage buckets. Price and quantity interact in ways neither column captures alone. Knowing when and how to transform numeric data is the difference between a model that memorizes noise and one that generalises.

---

Learning Objectives

By the end of this note you will be able to:

• Apply log1p, Box-Cox, and Yeo-Johnson transforms to correct skewed distributions
• Distinguish between equal-width binning (pd.cut) and quantile binning (pd.qcut) and know when each is appropriate
• Generate polynomial and interaction features using PolynomialFeatures
• Handle outliers through winsorizing rather than deletion
• Choose the right scaler for your use case — and know when to skip scaling entirely

---

Why Numeric Distributions Matter

Most numeric columns in real datasets are not normally distributed. Income, transaction amounts, page views, loan balances — these are all right-skewed. A handful of extreme values can pull a linear model's coefficients in the wrong direction, stretch distance metrics in KNN, and dominate gradient calculations in neural networks.

The goal is not to force every feature into a bell curve. The goal is to bring the distribution into a form where the model can learn meaningful relationships from it.

---

Distribution Transforms

Log Transform — Right-Skewed Data

The most common fix for right-skewed numeric columns. Taking the log compresses the long right tail and stretches the compressed left side, producing a more symmetric distribution.

Use np.log1p (log(1 + x)) instead of np.log when your column can contain zeros. log(0) is undefined; log1p(0) is 0.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Simulate right-skewed transaction data
np.random.seed(42)
transactions = pd.DataFrame({
    "transaction_amount": np.random.exponential(scale=500, size=1000)
})

# Apply log1p transform
transactions["log_transaction_amount"] = np.log1p(transactions["transaction_amount"])

# Compare skewness before and after
print(f"Original skewness:  {transactions['transaction_amount'].skew():.2f}")
print(f"Log1p skewness:     {transactions['log_transaction_amount'].skew():.2f}")
# Output:
# Original skewness:  2.01
# Log1p skewness:     0.12

Log Transform Requires Positive Values

np.log1p handles zeros. It does NOT handle negative values — it will return NaN. If your column has negative values (e.g., profit/loss), use Yeo-Johnson instead.

Box-Cox and Yeo-Johnson — General Power Transforms

When log1p is not enough (or when you have negative values), power transforms offer a more flexible solution. Box-Cox requires strictly positive data. Yeo-Johnson works on any real value — it is the safe default.

from sklearn.preprocessing import PowerTransformer
import numpy as np
import pandas as pd

np.random.seed(42)
df = pd.DataFrame({
    "income": np.random.exponential(scale=50000, size=500),
    "profit_loss": np.random.normal(loc=0, scale=10000, size=500)  # has negatives
})

# Yeo-Johnson works for both positive and negative columns
yj = PowerTransformer(method="yeo-johnson", standardize=True)
df[["income_yj", "profit_loss_yj"]] = yj.fit_transform(df[["income", "profit_loss"]])

print(f"Income skew before:      {df['income'].skew():.2f}")
print(f"Income skew after:       {df['income_yj'].skew():.2f}")
print(f"Profit/loss skew before: {df['profit_loss'].skew():.2f}")
print(f"Profit/loss skew after:  {df['profit_loss_yj'].skew():.2f}")
# Output:
# Income skew before:      2.04
# Income skew after:       0.03
# Profit/loss skew before: 0.05
# Profit/loss skew after:  0.01

Box-Cox vs Yeo-Johnson

In practice, default to Yeo-Johnson. It handles the same cases as Box-Cox plus negative values, and sklearn's PowerTransformer makes switching trivial. Box-Cox is only preferable when you need to match literature that used it specifically.

---

Binning — Converting Continuous to Categorical

Sometimes a continuous numeric column is better represented as a category. Age does not linearly predict behaviour — the difference between 20 and 25 is more behaviourally significant than the difference between 60 and 65. Binning makes non-linear relationships linear and improves interpretability.

Equal-Width Binning — pd.cut

Divides the value range into bins of equal width. Use when the range itself has meaning — e.g., a sensor reading with meaningful physical thresholds.

import pandas as pd
import numpy as np

np.random.seed(0)
df = pd.DataFrame({"age": np.random.randint(18, 75, size=200)})

# Equal-width bins — you define the breakpoints
df["age_group"] = pd.cut(
    df["age"],
    bins=[18, 30, 45, 60, 75],
    labels=["18-30", "31-45", "46-60", "61-75"],
    right=True
)

print(df["age_group"].value_counts().sort_index())
# Output (approximate):
# 18-30    53
# 31-45    56
# 46-60    49
# 61-75    42

Quantile Binning — pd.qcut

Divides into bins of equal frequency (each bin has approximately the same number of samples). Use when you want balanced groups regardless of the value distribution — which is almost always what you want for features fed into a model.

# Quantile bins — equal number of observations per bin
df["age_quantile"] = pd.qcut(
    df["age"],
    q=4,
    labels=["Q1", "Q2", "Q3", "Q4"]
)

print(df["age_quantile"].value_counts().sort_index())
# Output (approximately equal counts):
# Q1    50
# Q2    50
# Q3    50
# Q4    50

pd.cut vs pd.qcut — Know Which You Need

pd.cut creates equal-width intervals, which may produce very unequal group sizes if your data is skewed. A skewed column binned with pd.cut might put 90% of your data in one bin. Use pd.qcut for model features — balanced groups give the model more to learn from each category.

---

Interaction and Ratio Features

Some of the most powerful features are ratios and interactions that combine two raw columns into a single signal that neither column conveys alone.

import pandas as pd
import numpy as np

np.random.seed(1)
ecommerce = pd.DataFrame({
    "total_revenue": np.random.uniform(100, 5000, size=300),
    "order_count": np.random.randint(1, 50, size=300),
    "total_items": np.random.randint(1, 100, size=300),
    "customer_tenure_days": np.random.randint(1, 730, size=300)
})

# Ratio features — capture per-unit relationships
ecommerce["avg_order_value"] = ecommerce["total_revenue"] / ecommerce["order_count"]
ecommerce["avg_items_per_order"] = ecommerce["total_items"] / ecommerce["order_count"]
ecommerce["revenue_per_day"] = ecommerce["total_revenue"] / ecommerce["customer_tenure_days"]

# Order frequency — how often does this customer order?
ecommerce["orders_per_week"] = (
    ecommerce["order_count"] / (ecommerce["customer_tenure_days"] / 7)
)

print(ecommerce[["avg_order_value", "avg_items_per_order", "revenue_per_day"]].describe().round(2))

Domain Knowledge Is the Source of Great Interaction Features

You will not derive avg_order_value from automated feature generation tools — you derive it by asking "what would an analyst look at?" Talk to the business team. Read the product documentation. The best interaction features come from understanding what the numbers actually mean.

Polynomial Features — Automated Interactions

When you want to capture non-linear relationships systematically, PolynomialFeatures generates all powers and cross-products up to a given degree.

from sklearn.preprocessing import PolynomialFeatures
import numpy as np
import pandas as pd

X = pd.DataFrame({
    "price": [10.0, 25.0, 50.0, 100.0],
    "discount_pct": [0.0, 0.1, 0.2, 0.3]
})

poly = PolynomialFeatures(degree=2, include_bias=False, interaction_only=False)
X_poly = poly.fit_transform(X)

feature_names = poly.get_feature_names_out(X.columns)
X_poly_df = pd.DataFrame(X_poly, columns=feature_names)

print(X_poly_df.columns.tolist())
# Output: ['price', 'discount_pct', 'price^2', 'price discount_pct', 'discount_pct^2']
print(X_poly_df.round(2))
# Output:
#    price  discount_pct  price^2  price discount_pct  discount_pct^2
# 0   10.0           0.0    100.0                 0.0             0.0
# 1   25.0           0.1    625.0                 2.5             0.01
# 2   50.0           0.2   2500.0                10.0             0.04
# 3  100.0           0.3  10000.0                30.0             0.09

Polynomial Features Explode in Size

With degree=2 and 10 input features, you get 65 output features. With 50 input features, you get 1325. Use interaction_only=True to skip squared terms, or apply PolynomialFeatures only to a small, selected subset of your most informative numeric columns.

---

Handling Outliers — Winsorizing

Deleting outliers discards real data. Winsorizing clips them to a percentile threshold, preserving the row while limiting the extreme value's influence.

import pandas as pd
import numpy as np
from scipy.stats.mstats import winsorize

np.random.seed(42)
df = pd.DataFrame({
    "salary": np.concatenate([
        np.random.normal(60000, 15000, 490),
        [500000, 750000, 1000000, 1200000, 1500000, 2000000, 3000000, 5000000,
         -10000, -50000]  # extreme outliers on both ends
    ])
})

print(f"Before winsorize — min: {df['salary'].min():,.0f}, max: {df['salary'].max():,.0f}")

# Clip bottom 1% and top 1%
df["salary_winsorized"] = winsorize(df["salary"], limits=[0.01, 0.01])

print(f"After winsorize  — min: {df['salary_winsorized'].min():,.0f}, max: {df['salary_winsorized'].max():,.0f}")
# Output:
# Before winsorize — min: -50,000, max: 5,000,000
# After winsorize  — min: 29,876,  max: 96,842

# Alternatively, use pandas clip with quantile thresholds
lower = df["salary"].quantile(0.01)
upper = df["salary"].quantile(0.99)
df["salary_clipped"] = df["salary"].clip(lower=lower, upper=upper)

---

Scaling — When and Which

Scaling adjusts the range of numeric features so they contribute equally to distance calculations and gradient descent. This is not about the data's distribution (that is what transforms are for) — it is about putting features on a comparable scale.

StandardScaler — Zero Mean, Unit Variance

Subtracts the mean and divides by the standard deviation. Output is in "standard deviation units" from the mean.

from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd

X = pd.DataFrame({
    "age": [22, 35, 42, 55, 28],
    "annual_income": [30000, 75000, 120000, 95000, 42000]
})

scaler = StandardScaler()
X_scaled = pd.DataFrame(scaler.fit_transform(X), columns=X.columns)

print(X_scaled.round(3))
# Output: age and income are now both on the same scale, centered at 0
#      age  annual_income
# 0 -1.313         -1.201
# 1  0.140          0.275
# 2  0.768          1.490
# 3  1.707          0.785
# 4 -1.303         -1.349

MinMaxScaler — Range [0, 1]

Scales values to the range [0, 1]. Sensitive to outliers — one extreme value compresses everything else toward 0.

from sklearn.preprocessing import MinMaxScaler
import pandas as pd

X = pd.DataFrame({"spend": [100, 250, 500, 1200, 50]})

mms = MinMaxScaler()
X["spend_scaled"] = mms.fit_transform(X[["spend"]])
print(X)
# Output:
#    spend  spend_scaled
# 0    100      0.043478
# 1    250      0.173913
# 2    500      0.391304
# 3   1200      1.000000
# 4     50      0.000000

RobustScaler — Median and IQR

Uses the median and interquartile range instead of mean and standard deviation. Not thrown off by outliers.

from sklearn.preprocessing import RobustScaler
import pandas as pd

X = pd.DataFrame({
    "loan_amount": [5000, 10000, 15000, 20000, 500000]  # 500k is an outlier
})

rs = RobustScaler()
X["loan_scaled"] = rs.fit_transform(X[["loan_amount"]])
print(X)
# Output: 500k is still large, but no longer dominates the scale the way MinMax would allow
#    loan_amount  loan_scaled
# 0         5000    -0.666667
# 1        10000     0.000000
# 2        15000     0.666667
# 3        20000     1.333333
# 4       500000    32.666667

Scaler Selection Guide

Scaler	Use When
StandardScaler	Data is roughly normal, no extreme outliers. Logistic Regression, SVM, neural networks.
MinMaxScaler	You need values in [0, 1], e.g., for neural network input layers with sigmoid activation.
RobustScaler	Column has significant outliers you cannot or do not want to remove.
No scaling	Tree-based models (Decision Tree, Random Forest, XGBoost, LightGBM). Trees split on thresholds — scale is irrelevant.

Never Fit Scalers on the Full Dataset

Fit your scaler on training data only, then use transform() on validation and test data. Fitting on all data lets the scaler see test set statistics, which is a form of data leakage. Use sklearn Pipelines (covered in 05-pipelines-and-leakage) to enforce this automatically.

---

Key Takeaway

Numeric feature engineering follows a pattern: (1) check the distribution and handle skew with log1p or Yeo-Johnson, (2) handle outliers with winsorizing rather than deletion, (3) create interaction and ratio features from domain knowledge, (4) bin where non-linearity is expected, (5) scale only if your model requires it — tree models do not.

---

---

What's Next

You've covered log1p and Yeo-Johnson transforms for skewed distributions, binning strategies (fixed-width, quantile, and tree-informed), ratio and polynomial interaction features, winsorizing for outlier handling, and the scaler selection guide for StandardScaler, MinMaxScaler, and RobustScaler. Next up: 03-categorical-features — where you'll encode nominal and ordinal columns correctly, handle high-cardinality categories without one-hot explosion, apply target encoding safely with cross-validation, and learn when frequency encoding is a safer alternative.

Optional Deep Dive

Read the scipy documentation for scipy.stats.boxcox and scipy.stats.yeojohnson at https://docs.scipy.org/doc/scipy/reference/stats.html — it shows the full mathematical form of each power transform and explains when each is appropriate, giving you the theoretical foundation behind sklearn's PowerTransformer.

Back: Overview | Next: Categorical Features