Data Equivalence Modeling › Modeling Type Coercion Across Engines

# Modeling Type Coercion Across Engines

This page answers a foundational equivalence question: when the same logical value is stored as an int32 in one engine and an int64 in another, or a DECIMAL(18,2) versus a DOUBLE, are the two equal — and in which direction is the coercion safe? It sits under the data equivalence modeling reference and builds the type lattice that the whole comparison stage depends on, because a reconciler that does not model coercion either floods the manifest with benign type differences or, far worse, silently accepts a narrowing that truncated data.

# Problem Framing

You reconcile a table migrated from PostgreSQL to a system that widened every integer to 64-bit and stored decimals as doubles. Most of these coercions are safe — an int32 fits in an int64 with no loss — and should reconcile as equal. But some are not: a DECIMAL(18,4) rendered as an IEEE-754 double lost exact cents, and a DOUBLE narrowed back to a FLOAT truncated precision. The reconciler must encode coercion as a directed relation: widening is equivalence-preserving, narrowing is not, and the lattice must know which is which. A symmetric “these types differ” check is useless here; only a directionality-aware model can accept the safe widening while catching the lossy narrowing.

# Implementation

The coercion model is a directed graph of allowed widenings. Two typed values are equivalent if their types are equal or if the source type widens to the target along a safe edge; any other type difference is a real mismatch.

import logging
from dataclasses import dataclass
from typing import Dict, FrozenSet, Set

logging.basicConfig(
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s", level=logging.INFO
)
logger = logging.getLogger("recon.arch.coercion")

# Directed safe-widening edges: source -> set of types it may widen to without loss.
SAFE_WIDENINGS: Dict[str, FrozenSet[str]] = {
    "int8": frozenset({"int16", "int32", "int64"}),
    "int16": frozenset({"int32", "int64"}),
    "int32": frozenset({"int64"}),
    "float32": frozenset({"float64"}),
    "decimal(18,2)": frozenset({"decimal(19,2)", "decimal(38,2)"}),
    "date": frozenset({"timestamp"}),
}


def _closure(src: str, seen: Set[str]) -> Set[str]:
    """Transitive closure of safe widenings from a source type."""
    for tgt in SAFE_WIDENINGS.get(src, ()):  # e.g. int8 -> int16 -> int32 -> int64
        if tgt not in seen:
            seen.add(tgt)
            _closure(tgt, seen)
    return seen


def types_equivalent(src_type: str, tgt_type: str) -> bool:
    """Equal, or source widens to target along a loss-free path. Directionality matters."""
    if src_type == tgt_type:
        return True
    if tgt_type in _closure(src_type, set()):
        return True                          # safe widening: equivalent
    logger.warning("non-equivalent coercion %s -> %s (possible narrowing)", src_type, tgt_type)
    return False

# Coercion Directionality Table

The same type pair is safe in one direction and lossy in the other; the model must encode both.

# Key Implementation Notes

• Coercion is directed, never symmetric. The single most important modeling decision: a widening edge from int32 to int64 does not imply the reverse. Encoding only safe widenings means a narrowing can never be silently accepted, because it simply is not in the graph.
• Compute the transitive closure once. int8 is equivalent to int64 through a chain of widenings; the closure captures that without listing every pair. Cache the closure per source type so equivalence checks stay O(1) after warm-up.
• Money and timezones are special-cased to strict. A decimal to double reinterpretation and a timezone-dropping timestamp coercion are never safe, regardless of nominal width; keep them out of the widening graph so they always read as mismatches, echoing the floating-point tolerance rule that money is never a float.
• The lattice is the contract the manifest records. Which coercions were accepted as equivalent is part of the reconciliation verdict; version the widening graph and record its version so an audit can see exactly which coercions a run permitted.

# Verification

Assert widenings are equivalent, narrowings and lossy reinterpretations are not, and transitivity holds.

assert types_equivalent("int32", "int64")               # safe widen
assert types_equivalent("int8", "int64")                # transitive widen
assert not types_equivalent("int64", "int32")           # narrowing -> mismatch
assert not types_equivalent("decimal(18,2)", "double")  # money to float -> mismatch
assert types_equivalent("date", "timestamp")            # safe widen
logger.info("type coercion model verified")

# Operational Considerations

Derive the safe-widening graph from the actual engine pairs in your migration, since two engines may disagree on what a nominal type means, and keep it in version control with a fingerprint attached to every reconciliation run so a verdict is reproducible. Special-case regulated numeric and temporal types to strict equivalence by default and require an explicit, reviewed edge to relax them, so the safe behavior is the one you get without asking. Expose coercions_accepted_total by type pair so an operator can see which widenings the reconciler is absorbing, and route any non-equivalent coercion to structural mismatch detection rather than the value comparison, because a type mismatch is a shape problem, not a value one. Feed the accepted-coercion set into the equivalence models for heterogeneous databases so the whole parity contract stays consistent.

# Related

• Data equivalence modeling — the parent reference whose parity rules this coercion lattice implements.
• Building equivalence models for heterogeneous databases — the broader model this feeds.
• Setting floating-point tolerance for numeric parity — the numeric rule that money is never coerced to float.
• Detecting structural mismatches in Parquet files — where a non-equivalent coercion is caught as a shape mismatch.