Skip to content
Multigrid

Machine learning on tabular data

Encoding, leakage, imbalance and validation on the data format most real problems arrive in, where gradient boosting still wins.

Most machine learning problems that anyone is paid to solve arrive as a table: rows that are customers or transactions or sensors, columns that are a mixture of numbers, categories, dates and free text, and a target somewhere on the right. Almost none of the last decade’s headline progress applies to it. The models that win on this format are gradient-boosted decision trees, and they have been winning since well before the current wave of attention, for reasons that are structural rather than historical.

These pages are about the part of the work that decides whether a tabular model is any good, which is almost never the choice of algorithm. It is how a high-cardinality category is turned into numbers without leaking the target through it, what a missing value means before you replace it, whether the validation split respects the way the data was actually generated, and which of the features you are proud of are just the label wearing a hat. Where a figure comes from a library default or a published benchmark, the source is named in the sentence.

Categorical Encoding Methods, Compared by Mechanism

One-hot, ordinal, target and learned-embedding encoding worked on the same five-category column, so the dimensionality and leakage differences are visible rather than described.

10 min read

Why Neural Networks Still Underperform Gradient Boosting on Tabular Data

The gap is an inductive-bias mismatch with three named components, not a leaderboard accident — and the published evidence says it is real but narrower than the folk version.

11 min read

Inferring a Table's Schema From a CSV With an LLM

A script that samples column values, asks a model for a typed schema, and validates the answer against the file before you trust it — including flagging a mixed-type column.

10 min read

What an LLM Can and Cannot Do With a Spreadsheet

The line is drawn by two things a model has no mechanism for: row count against context, and the dependency graph a formula creates.

10 min read

Detecting a Column's Data Type Automatically

The heuristic cascade every ingestion tool runs — numeric, boolean, date, categorical, free text — in the order that matters, and the real values that break each stage.

10 min read

Tabular Data Embeddings for Row Similarity Search

How numeric and categorical columns are combined into one vector space, worked through a distance calculation that shows why the scaling decision dominates the result.

10 min read

Handling Missing Values Before Training a Tabular Model

Mean imputation, chained-equation imputation and a missingness indicator worked on the same column, with the mechanism that decides which one is defensible.

11 min read

Imbalanced Classification With Gradient Boosting: Weighting or Resampling

Class weighting and SMOTE worked on the same 1%-positive dataset, with the published evidence on when balancing stops helping a strong model at all.

11 min read

Automated Feature Selection Methods for Tabular Models

Filter, wrapper and embedded selection applied to the same eight-feature set, with the correlation matrix that shows why each one keeps a different subset.

10 min read

Turning a Question Into a Pandas Query With an LLM

A function that turns a question into a pandas filter-and-groupby chain, validates the generated code against an allowlist before executing it, and returns the code alongside the answer.

11 min read

Training a Model on Spreadsheet Data Without Writing Code

A public dataset taken end to end through a no-code AutoML path, with the specific points past a few thousand rows where the approach stops being adequate.

10 min read

Numeric Feature Scaling: Standardisation, Normalisation and When Each Distorts a Distribution

Both transforms applied to the same skewed column, with the arithmetic that shows what each one does to an outlier and to the rest of the distribution.

10 min read

Cross-Validation Strategies for Tabular Models

How to pick a splitter when rows are grouped, ordered or imbalanced, and why plain k-fold quietly reports a score you will never see in production.

10 min read

Gradient Boosting Hyperparameter Tuning: Where to Start

The three parameters that move a boosted-tree score, the order to tune them in, and a randomised search you can run on your own table.

11 min read

Target Leakage: Finding a Feature That Already Knows the Answer

What target leakage does to a model, the four places it enters a tabular pipeline, and a concrete check that separates a leaked column from a merely strong one.

11 min read

Detecting Duplicate Columns and Redundant Features in a Wide Table

How to find exact duplicates, renamed copies, constant columns and near-duplicates in a table with hundreds of features, and which of them are safe to drop.

10 min read

Binning Continuous Variables: When It Helps a Model and When It Hides Signal

The measurable cost of discretising a continuous column, worked arithmetically, and the three situations where paying it is still the right call.

10 min read

Ordinal and Nominal Categorical Features: Why the Distinction Changes the Encoding

What a model can do with an ordered category that it cannot do with an unordered one, and what each wrong choice costs in splits, coefficients and data efficiency.

10 min read

Detecting Multicollinearity Before It Wrecks a Model

What correlated predictors actually do to a fitted model, how to compute and read VIF, and why the standard advice to drop one of every correlated pair is often wrong.

11 min read

Synthetic Tabular Data Generation for Testing a Pipeline

A Gaussian-copula generator you can run today that reproduces a real table's marginals and rank correlations, plus what it will not reproduce.

11 min read

Ranking Features by Predictive Power Without Training a Model

Correlation, mutual information and the F test as pre-model screens, what each one can and cannot see, and the interaction they all miss.

10 min read

What Running Gradient Boosting Predictions at Batch Scale Costs

A cost model for scoring millions of rows with a boosted tree ensemble, derived from tree count, row count and a stated CPU-hour price.

10 min read

Stacking Gradient Boosting Models: When the Extra Complexity Pays For Itself

How a stacked ensemble is built without leaking, where its gain actually comes from, and the latency and operational cost it adds in exchange.

10 min read

Detecting a Mislabeled Row in a Training Set

Using out-of-fold confidence and fold disagreement to surface rows whose label is probably wrong, and how to tell label noise from a genuinely hard example.

10 min read

Other topics