GPU kernels serve as the computational core of modern AI. A single bug inside one kernel can silently compromise the entire model's correctness and put both training and inference at risk[1A Fix for Softmax and Cross-EntropyarXiv:2610.31642, 2026, 2The Llama 3 Herd of ModelsarXiv:2407.21783, 2024, 3OpGuard: Bitwise Alignment for Precise and General Debugging of Production LLM TrainingOSDI 2026]. Formal verification promises correctness over any configuration and GPU schedule, but it has never reached production kernels for modern AI. We present TensorLean, the first framework to formally verify the correctness of production GPU kernels. With TensorLean, we verified 149 real-world open-source kernel instances on their PTX assembly, including 102 from FlashAttention and those from DeepSeek, Liger Kernel, etc., and proved 83 of them correct. In verification, we also discovered 38 bugs of various kinds in frontier kernels that had eluded testing for months and years. We expect that, with TensorLean, formal verification will scale and become an essential step in developing correct and efficient kernels.
Introduction
In February 2024, ChatGPT began producing text that made no sense [4Unexpected responses from ChatGPTOpenAI incident report, February 2024]. The following year, a miscompilation disrupted Claude's token selection [5A postmortem of three recent issuesAnthropic Engineering, September 2025]. Now, in October 2026, Google DeepMind showed that even standard softmax and cross-entropy in PyTorch and JAX contain numerical bugs that can corrupt gradients and collapse a model's accuracy during training [1A Fix for Softmax and Cross-EntropyarXiv:2610.31642, 2026]. All of them were caused by silent correctness bugs in kernels, which are programs that implement the model's modules and algorithms, such as matrix multiplication and attention.
As AI models become infrastructure, we need to detect and fix these bugs, and prove their absence before they waste millions of GPU hours and dollars. Formal verification is the only way to provide the exact guarantee. In formal verification, developers translate their code into a mathematical model, then try to prove that the model satisfies a set of mathematical properties, known as specifications. Once the proof is ,machine-checkedThe proof is written as a program in a formal language, e.g., Lean. A proof assistant then checks the proof's validity by type checking it. the code meets its specification on all allowed inputs and in every possible execution.
However, formal verification is often expensive: we must state precisely what the code should do, in a specification, and why it does so, in a proof. Both can cost orders of magnitude more than the development per se. For example, the seL4 microkernel took 2.5 person-years and 8,700 lines of C to build, but 20 person-years and 200,000 lines of proof to verify [6seL4: Formal Verification of an Operating-System KernelCommunications of the ACM, 2010]. Even with state-of-the-art verifiers, specifications and proofs are still several times larger than code [7Verus: A Practical Foundation for Systems VerificationSOSP 2024]. Moreover, GPU kernels are known to be complicated for their massive concurrency, distributed and memory layouts, and floating-point arithmetic, all challenging the scalability of formal verification.
To achieve formal guarantees on kernels at an affordable cost, we introduce TensorLean, the first to formally verify the correctness of production kernels. In TensorLean, the efforts in specification and proofs are both reduced to a minimum. For specification, TensorLean decouples mathematical computation from memory layouts, so specifications are as easy to write and audit as an ordinary tensor program and scale to production kernels at about a 15:1 code-to-spec ratio. For verification, TensorLean proves that a kernel's compiled PTX meets its tensor-level specification on every input and thread schedule, within floating-point tolerance. With TensorLean, we verified 149 kernel instances on their PTX, covering all 37 kernels of FlashAttention-2 to -4 and more from DeepGEMM and Liger Kernel: 83 are proved correct, and the other 66 contain bugs we found. Along the way, we found 38 bugs that silently eluded the kernels' own tests.
O = softmax(Q · K⊤/√d) · V
TensorLean
theorem flash_attention_correct :
∀ inputs executions,
ptx ≈ attention := by
tensorleanFlashAttention-12022
mma.sync.aligned.m16n8k16.row.col.f32.f16.f16.f32 …
FlashAttention-22023
cp.async.cg.shared.global.L2::128B [%r93], [%rd324], 16;
FlashAttention-32024
wgmma.mma_async.sync.aligned.m64n224k32.f32.e4m3.e4m3 …
FlashAttention-42026
tcgen05.mma.cta_group::2.kind::f16 [tmem_acc], …
⋮
We will soon release our paper and open-source TensorLean, including its PTX semantics and the specifications and proofs of the verified kernels, so that anyone can reuse them and verify other frontier kernels. We expect that, with TensorLean, we can pace the formal verification of GPU kernels in step with their evolving development, with both experts and agents involved [8CAKE: Compiler-Agent Co-Design for Frontier Kernel EvolutionarXiv:2608.12629, 2026].
In the rest of this article, we cover TensorLean in more detail.
Tensor-Level Specification
To prove that a kernel is correct, we must first state what a "correct behavior" is, known as its specification (spec).
A kernel is correct if, on every input it accepts and in every execution, it writes the same valuesDue to floating-point rounding, "the same" means equal within a bound, known as tolerance. as the operation it implements, and changes nothing else in GPU memory.
For example, an attention kernel with a KV cache is correct if, for every
The definition of kernel correctness is :extensionalProperty
Kernel Specification at Scale. The correctness specifications are notoriously costly to write, as they demand expertise in both verification and the target domain, and can be several times larger than code [7Verus: A Practical Foundation for Systems VerificationSOSP 2024]. The root cause is that a specification couples the mathematics of the computation with the representation of its data in the machine model's memory.
TensorLean makes the specification of GPU kernels scale: we achieve an average 15:1 code-to-spec ratio. For example, it only needs 2,291 lines of specification to define the correctness of 12 attention (forward) kernel variants, including the FlashAttention family and more, totaling 35,852 lines of source:
Code vs. Spec Size Comparison: Attention (Forward)
Kernel SourceTensorLean SpecificationShared Spec Library
Decoupling Mathematics from Memory. The key observation behind TensorLean's scalability is that the math and memory can be decoupled because a GPU kernel's correctness is extensional: the specification refers to memory only through the memory layout, which maps each tensor element to the bytes that hold it, while the mathematics refers to tensors alone.
spec_mathSpecification
spec_layoutspec_layoutImplementation
SpecificationImplementation
With math and memory decoupled, a kernel’s specification can be written at the tensor level instead of the PTX level: the mathematics is stated over tensors, and the memory layout of each tensor is given separately. TensorLean then automatically lowers this tensor-level specification to a PTX-level one, in which each tensor element is read from the bytes that hold it, as given by the memory layout:
over tensors
Q[i, e]over PTX memory
fp16(mem[q_ptr + 2*(i*D + e)])Concretely, a TensorLean specification consists of five parts, over the tensors that the kernel only reads (input), writes without reading their old values (output), and reads and overwrites in place (state), such as a KV cache:
spec_math: The mathematical computation the kernel implements, as a tensor function fromstateandinputtooutputtensors of real numbers.spec_mut: The GPU memory update the kernel makes in place, as a function fromstateandinputtostate', thestateafter execution.spec_layout: The memory layouts of the tensors, as a function from each tensor's indices to GPU memory addresses.spec_tol: The toleranceof each outputandstate'tensor, by which its floating-point values in memory may differ from its exact real values.requires: The precondition onstateandinput.
requires C has room for K, V
q_ptr k_ptr v_ptrc_ptro_ptrPTX memory
Writing Specifications in Lean. TensorLean provides a library of tensor operators to define the tensor-level specifications, built on Lean's Mathlib.
It lets developers declare tensors, with operators that follow PyTorch's names and semantics, e.g., @ for torch.matmul, so a kernel's tensor-level specification takes no more effort to write, or to audit, than a PyTorch reference.
For example, an attention kernel's spec_math can be defined as below in TensorLean:
-- ℝ: real number, ℕ: natural number
-- L: sequence length, D: head dimension
structure Attention (L D : ℕ) where
Q : Tensor ℝ [L, D]
K : Tensor ℝ [L, D]
V : Tensor ℝ [L, D]
scale : ℝ
def attention : KernelSpec (Attention L D) (Tensor ℝ [L, D]) where
spec_math input := softmax((input.Q @ input.Kᵀ) * input.scale) @ input.V
spec_layout := with (q_ptr, ...) -- the kernel's arguments
{{ tensor := input.Q[i, e] -- Q : Tensor ℝ [L, D], indexed by i, e
var := q_ptr -- the source variable that holds Q
layout := i * D + e },...} -- the offset of Q[i, e] from q_ptr
On Production Kernels. By decoupling mathematical computation (spec_math) from memory layouts (spec_layout), the TensorLean specifications across various same-type kernels share a significant number of the same tensor operations, thereby reducing the complexity in both writing and auditing the specifications. In our study, we formalized 72 open-source kernels from 9 operations with 151,285 lines of source code and only 10,278 lines of specifications, achieving a nearly 15:1 code-spec ratio.
PTX-Level Verification
With the kernel's (1) PTX code and (2) specification provided, TensorLean automatically constructs the proof goals and verifies the kernel's functional correctness with Lean and SMT solvers. The workflow can be illustrated as below:
From UserAuto-gen by TensorLeanProvenTrusted
1.Source CodeUnmodified
copy(gK, sK); __syncthreads(); gemm(S, sQ, sK); softmax(S); gemm(O, S, sV);
2.SpecificationTensor-Level
spec_math input := softmax(Q @ Kᵀ*s) @ Vspec_layout := { Q[i,e] ↦ q_ptr+i*D+e, O[i,e] ↦ o_ptr+i*D+e }
3.PTX AssemblyFrom CUDA/
cp.async sK, gK; bar.sync 0; mma.sync s, q, k; ex2.approx p, s;
PTX Semantics
4.Happens-Before GraphProgram & Sync Order
✓ Race-Free✓ Terminates
5.Post-Execution MemorySame Across Schedules
pre mem post mem′[o_ptr+…] = Σj pij·mem[v_ptr+…]-- pij: online softmax
6.Lowered SpecPTX-Memory-Level
lowered := pre ∀ i e, mem[q_ptr+i*D+e] = Q[i,e] post ∀ i e, mem′[o_ptr+i*D+e] ≈ spec_math[i,e]
7.ProofMachine-Checked
theorem fwd : mem′ ⊨ lowered := by apply post_of_goals · post_vc -- ℝ · bound_sound -- FP
✓ Correctness Proved
Proof CheckersLean's Kernel, z3, cvc5
Verifying the correctness is hard: TensorLean needs to reason out exactly what GPU memory holds after the kernel's execution and prove that it meets the specification. For production kernels, the challenges for verification are:
- Fine-grained concurrency across hundreds of threads with asynchronous operations and relaxed PTX memory.
- Dynamic input shapes and work partitioning: the batch size and sequence lengths, known only at launch, determine the kernel's loop trip counts and the tiles each iteration indexes, and a scheduler assigns the work tiles to thread blocks at runtime.
- Floating-point rounding that makes the kernel's result not exactly the same as the mathematical specification, but within a tolerance.
- Bit-level address arithmetic from hardware-specific optimizations, such as swizzled memory layouts and warp shuffles, whose XORs, shifts, and masks bring nonlinear integer arithmetic into the proof goals.
Next, we discuss how TensorLean solves these challenges in formal verification.
Reducing Concurrency to Sequential Execution. GPU kernels run hundreds of threads that synchronize through barriers and asynchronous copies, and a bug may appear only in a rare interleaving of them: a data race we found in FlashAttention-4 [9[Bug][FA4] compute_block_sparsity: data race on reduction_buffer can drop blocks with unmasked keysFlashAttention GitHub issue #3195] never showed up in 17,760 runs of its tests. Instead of enumerating interleavings, TensorLean reasons on the kernel's happens-before graph [10Time, Clocks, and the Ordering of Events in a Distributed SystemCACM 1978], the order in which its operations take effect in every execution. A node is one execution of an operation by one thread, and an edge is an ordering that the PTX ISA guarantees, such as program order within a thread, or an arrive before the wait it completes. TensorLean builds the graph symbolically, over every CTA, thread, and loop iteration of every launch, and proves in Lean that it is sound.
On this graph, TensorLean first proves that the kernel is free of data races: any two accesses to the same address, at least one a write, are ordered by the graph. Every read then returns the last write that happens before it. It then proves that the kernel terminates: every loop is bounded, and every wait receives the arrivals it waits for, with no cycle among them; a kernel that hangs would otherwise satisfy any specification vacuously.
Together, the two properties reduce the kernel's concurrency to sequential execution. The final memory is the initial memory updated by the last writes, the write to each address that no other write to it follows in the graph. To compute these writes, TensorLean symbolically runs each group of threads that play the same role, such as the producer warp or the consumer warps, once in program order, with its CTA, thread, and iteration as variables. The groups exchange writes only at synchronizations: at an arrive, a group pushes the writes before it, and at a wait, a group pulls in the writes pushed by the arrives it waits for. Each read thus resolves to the last write before it, and TensorLean repeats this on that write's value until the value depends only on the kernel's inputs.
Inferring Loop Invariants over Semirings. A kernel's loops run as many iterations as its tensors' shapes require, and the shapes are known only at launch. All recent equivalence checkers for kernels [11Equivalence Checking of ML GPU KernelsOOPSLA 2026, 12SentryWard: Formal Verification of Warp-Specialized GPU Kernels2026] check one concrete configuration at a time, unrolling the loops, so their proofs can only cover the configurations they enumerate and miss subtle bugs in the combinations of shapes and features that developers overlook. Indeed, 19 of the 23 bugs we found in FlashAttention appear only in such configurations. To prove a kernel correct under every configuration at once, TensorLean takes the configuration as symbolic parameters and reasons about loops with symbolic bounds, which requires a loop invariantA property that holds at the start of every iteration, and therefore after the loop, whatever its trip count. for each value a loop carries. Synthesizing loop invariants is undecidable in general, so verifiers usually ask developers to write them by hand [7Verus: A Practical Foundation for Systems VerificationSOSP 2024]. For example, FlashAttention-3's split-KV combine kernel alone carries 86 values, and every optimization of the kernel changes them, so writing invariants by hand keeps verification from scaling to production kernels.
We observe that most kernel loops compute over semirings, and TensorLean synthesizes their invariants automatically. This is because most kernel loops share three common properties:
- Tiled iteration: each iteration processes one tile of a tensor, so the trip count is fixed by the configuration before the loop starts.
- Accumulative updates: each iteration only combines its own contribution into each value the loop carries.
- Data-independent control flow: a loop's branches and synchronization depend on the iteration and the configuration, not on the data.
Together, these properties make each value
For example, the online softmax at the core of FlashAttention reads one block
| Carried value |
|||
|---|---|---|---|
| Running maximum |
|||
| Running sum |
|||
| Output |
By distributivity, the rescalings to each new maximum multiply out, so every key seen so far ends up measured against the current maximum, and the closed forms are
TensorLean finds these closed forms automatically: it reads spec_math's
Correctness under Tolerance.
For generality, a TensorLean specification is defined over tensors of real numbers, so that one specification covers an operation's kernels in every precision, such as FP32, BF16, and FP8.
A kernel, however, computes in floating point, so its arithmetic results inevitably involve ,roundingFor example,
These rounding errors propagate through the kernel's computation and can accumulate, so an output may deviate from its mathematical value beyond an acceptable tolerance. In the worst case, the error is passed to, and cascades through, the model's later computations until it corrupts training or inference, as with the recent bugs found in standard softmax [1A Fix for Softmax and Cross-EntropyarXiv:2610.31642, 2026].
In practice, to catch such rounding deviations, kernel tests use a reference implementation, such as PyTorch’s, as a stand-in for the exact values. They require the kernel’s outputs on sampled inputs to match the reference’s within a tolerance ε, which bounds the allowed error:
To formally verify a kernel's correctness despite floating-point rounding, the proof goal must bound the kernel's deviation from spec_math by the same ε:
To scale automated verification, TensorLean never compares
Thus, by the triangle inequality, TensorLean proves the kernel's correctness within tolerance:
Assertions as Proof Hints.
A proof goal contains the terms of the values it reads, each value written out as the operations that computed it from the kernel's inputs, and in production kernels these terms grow too large for SMT solvers to decide.
In TensorLean, developers can guide the proofs by writing assertions in the kernel's source code: a statement tensorlean_assert at a line states a property
Case 1: Assertions for proving functional correctness. The largest terms come from the values that tensor cores accumulate. FlashAttention-4's backward kernel on Hopper, for example, accumulates the gradient softmax_scale, copies it from registers to shared memory, sdK, and, after a barrier, stores it to global memory. The tile's values at the barrier can be asserted in the source code as below:
flash_attn/cute/flash_bwd_sm90.py (simplified)
for m_block in range(m_min, m_max): # each block of queries
mma_one_m_block(m_block) # dK += dS.T @ Q (WGMMA)
acc_dK *= softmax_scale
copy_dK_r2s(acc_dK) # registers to shared memory, sdK
fence_view_async_shared()
epi_barrier.arrive_and_wait()
tensorlean_assert(sdK[r, c] == softmax_scale * dK[j, c])
if warp_idx == 4:
store_dK() # shared to global memory (TMA)The assertion at the barrier states that sdK then holds, element by element, the values of spec_math, row r of the tile being key j = blockIdx.x * tile_n + r. TensorLean proves it once, through the accumulation loop by the invariant it infers for that loop, a sum over the semiring sdK as the values of spec_math, so the goal of each store no longer contains the accumulation's term. Three such assertions, for
Case 2: Assertions as loop invariants. Some kernel loops do not compute over semirings, because they fall outside the three properties above, such as the scheduling loops of persistent kernels, which balance work across SMs by the lengths of the sequences. Developers state their invariants as assertions at the loop's head, and TensorLean proves each by induction: it holds at the first iteration, and each iteration keeps it. DeepGEMM's paged MQA kernel, for example, balances its work by the queries' context lengths lens: before the launch, its tasks, each a query and one split of its KV blocks, are divided into equal, consecutive ranges, one per CTA, and each CTA walks its range in a scheduler loop (Figure 5). The invariant of its scheduler loop can be asserted in the source code as below:
deep_gemm/impls/sm90_fp8_paged_mqa_logits.cuh (simplified, its scheduler inlined)
q = meta[sm].x, kv = meta[sm].y * SPLIT; // this CTA's first task
end_q = meta[sm + 1].x, end_kv = meta[sm + 1].y * SPLIT;
num_kv = ceil_div(lens[q], BLOCK_KV);
while (q != end_q || kv != end_kv) {
// the task comes before this CTA's end
tensorlean_assert(q < end_q || (q == end_q && kv < end_kv));
// num_kv is query q's number of KV blocks
tensorlean_assert(num_kv == ceil_div(lens[q], BLOCK_KV));
// kv starts a split of those blocks, or is 0 if there are none
tensorlean_assert(kv % SPLIT == 0 && (kv == 0 || kv < num_kv));
compute(q, kv); // logits of query q, KV blocks kv .. kv + SPLIT - 1
kv += SPLIT;
if (kv >= num_kv) { // the next query
kv = 0, q += 1;
if (q < end_q || end_kv > 0)
num_kv = ceil_div(lens[q], BLOCK_KV);
}
}The loop's trip count, and the iterations at which it moves to the next query, depend on meta and lens, the kernel's inputs, and kv restarts at 0 for every query: the loop falls outside all three properties. Every proof goal inside the loop assumes the invariant, by the cut rule: from it, TensorLean proves that each task's reads of the block table and the KV cache, and its writes of the logits, stay within their tensors.
(q, kv): the task of query[q] from KV block kvlens[q]: tokens of query[q]num_kv: KV blocks of query[q]meta[k]: first task of CTA k, set before the launch
Tasks of each query
one KV block, 64 tokensone task, 4 blocks
Tasks of each CTA
CTA 2 at (2, 12)
Loop Invariant 1(q, kv) < meta[3]✓(2, 12) < (3, 4)
Loop Invariant 2num_kv == ceil_div(lens[q], 64)✓15 == ceil_div(930, 64)
Loop Invariant 3kv % 4 == 0 && (kv == 0 || kv < num_kv)✓12 % 4 == 0 && 12 < 15
In total, we wrote 298 assertions to verify the kernels of FlashAttention-2 to -4 and DeepGEMM.
Our paper will present the full technical details and usage of TensorLean.
FlashAttention and Beyond
To demonstrate end-to-end verification capability, we use TensorLean to define the formal specifications of the entire FlashAttention-2 to -4 family and verify its kernels at the PTX level. In four years, FlashAttention has gone through four generations, from Ampere to Blackwell, and each has set the state of the art in attention kernels. As the de facto attention implementation, it has been widely deployed in production systems for both training and serving large language models, including PyTorch, vLLM, and SGLang.
This central role makes FlashAttention's correctness critical: a single bug in it could put the training and serving of every model on these systems at risk. With TensorLean, we formally specified and verified all 37 kernels of FlashAttention-2 to -4. We verified each kernel on the PTX of at least one compiled instance, 102 instances in total.
Specification: From Math to Machine. With TensorLean, we first defined the specifications of 37 FlashAttention kernels, from each kernel's mathematics to its memory layout.
For example, FlashAttention-2's spec_math reads as plainly as the PyTorch reference in its tests (simplified):
TensorLean Spec for FlashAttention-2
structure Attn (H G L D : ℕ) where
q : Tensor ℝ [H * G, L, D]
k v : Tensor ℝ [H, L, D]
scale cap : ℝ
mask : Tensor Bool [H * G, L, L]
spec_math input :=
let k := repeat_interleave(k, G)
let v := repeat_interleave(v, G)
let S := scale * (q @ kᵀ)
let S := cap * tanh(S / cap)
let S := masked_fill(S, ~mask, -∞)
softmax(S) @ vPyTorch Reference in FlashAttention-2's Tests
def attention_ref(
q, # [H * G, L, D]
k, v, # [H, L, D]
G, scale, cap,
mask, # [H * G, L, L]
):
k = k.repeat_interleave(G, dim=0)
v = v.repeat_interleave(G, dim=0)
S = scale * (q @ k.mT)
S = cap * torch.tanh(S / cap)
S[~mask] = -torch.inf
return S.softmax(dim=-1) @ vThe specification then defines tensors' memory layouts, e.g., for a paged KV cache:
{ tensor := state.k[b, p, g, d] -- the key cache, indexed by b, p, g, d
var := k_ptr -- the source variable that holds it
layout := block_table[b, p / page_size] -- its offset from k_ptr:
* k_batch_stride -- the page that holds key p,
+ p % page_size * k_row_stride -- key p's row in that page,
+ g * k_head_stride + d } -- and its head and dimension
In total, it takes 6,242 lines of specification to define the correctness of 37 FlashAttention kernels, totaling 55,910 lines of source:
Verification: In Minutes. We verified these instances against the four properties on their PTX: data-race freedom, termination, functional correctness, and outputs within a floating-point tolerance. The three generations span two languages: FlashAttention-2 and -3 are written in CUTLASS's CuTe, and FlashAttention-4 in the CuTe DSL. Each exercises the asynchronous hardware of its GPU: FlashAttention-2 computes with mma on Ampere, overlapping it with cp.async copies; FlashAttention-3 specializes its warps into a producer that loads tiles by TMA and consumers that multiply them with WGMMA on Hopper, pipelined through mbarriers; and FlashAttention-4 adds Blackwell's tensor memory and tcgen05 matrix multiplications. TensorLean verifies an instance in about 10 minutes at the median on a 16-core workstation. The largest, FlashAttention-3's forward kernel on Hopper with 31,402 proof goals, takes 50 minutes. Of the 102 instances, 56 are proved to satisfy all four properties. In each of the other 46, a proof goal fails on a bug in the kernel's source.
Finding and Fixing Bugs. In FlashAttention-2 to -4, TensorLean found 23 previously unknown bugs, and we have reported all of them to the developers. Beyond FlashAttention, we applied TensorLean to the kernels of DeepSeek's DeepGEMM, Triton's kernel library, Liger Kernel, and TileOPs, and verified 47 more kernel instances, from DeepGEMM and Liger Kernel. With end-to-end formal verification from math to machine, TensorLean can and has captured a wide range of correctness bugs in production kernels:
- Incorrect Algorithm Implementation caused by kernel code that departs from its intended algorithm, such as an incorrect formula or condition, or an unhandled special case. The kernel is incorrect even without any of the bugs below.
- Numerical Bugs caused by insufficient precision or range, or by numerical instability, leading to rounding error beyond the tolerance, overflow, NaN, or integer wraparound. The kernel is correct over the real arithmetic, but not in finite-precision arithmetic.
- Uninitialized Memory, where the kernel reads shared memory before initializing it, or leaves outputs unwritten. These lead to incorrect results or nondeterministic control flow.
- Memory Access Errors caused by an incorrect tensor memory or distributed layout, or a missing bounds check, leading to out-of-bounds accesses and memory corruption.
- Data Races between two accesses to the same global/shared memory, at least one of them a write, when no barrier or fence orders them. Such a race can silently corrupt output tensors.
- Deadlocks, where threads wait on each other in a cycle, or a wait is never matched by the arrivals it waits for, e.g., due to barrier divergence. Such a deadlock hangs the kernel.
- Undefined Behaviors, such as a division by zero or a signed integer overflow in the kernel's source, or initializing an mbarrier that is already initialized, which the language or the PTX ISA leaves undefined, so the compiler or the GPU may do anything with it.
- Miscompilations, where a kernel compiler such as NVCC, Triton, or CuTe DSL compiles a correct source into incorrect executable. Such a miscompilation can introduce any of the bugs above. TensorLean captures all of them by verifying the PTX.
Figure 8 shows all 38 bugs by category and library. Memory access errors (19) and numerical bugs (13) are the most common, and every category has at least one bug. Hovering over or tapping a bar lists its bugs, each linked to its report.
Limitations
Trusted Computing Base. First, we trust NVIDIA's PTX ISA [16Parallel Thread Execution ISA Version 9.4NVIDIA CUDA Documentation] and ptxas, and assume that TensorLean's PTX semantics is consistent with them. We also trust TensorLean's model of how tensor cores round, which the PTX ISA does not specify, for bounding the round-off errors of matrix multiplications. Second, we trust each kernel's specification and TensorLean's 1.8K lines-of-code PTX parser. Third, we trust the results of Lean's kernel [17The Lean 4 Theorem Prover and Programming LanguageCADE 2021] and the cvc5 [18cvc5: A Versatile and Industrial-Strength SMT SolverTACAS 2022] and Z3 [19Z3: An Efficient SMT SolverTACAS 2008] SMT solvers.
Verification Scope. A TensorLean specification defines a kernel's outputs and in-place updates as mathematical functions of its inputs (spec_math and spec_mut), so it cannot yet express kernels that sample at random, such as dropout and top-p sampling [20FlashInfer: Efficient and Customizable Attention Engine for LLM Inference ServingMLSys 2025]. TensorLean derives the happens-before order of the relaxed PTX memory model [21A Formal Analysis of the NVIDIA PTX Memory Consistency ModelASPLOS 2019] from barriers, mbarriers, and asynchronous copies, but does not yet model the orders that acquire and release operations establish; kernels that synchronize through atomic flags, including multi-GPU communication kernels such as NCCL's [22NVIDIA Collective Communications Library (NCCL)NVIDIA Developer] and DeepEP's [23DeepEP: A High-Performance Communication Library for Machine Learning Training and InferenceGitHub repository], are thus out of its scope.
Incompleteness. TensorLean is sound but can fail to verify a correct kernel because of missing input constraints in its specification, overly conservative round-off bounds, or SMT solver timeouts.
Conclusion
We presented TensorLean, the first framework for formally verifying the correctness of production GPU kernels. By decoupling mathematics from memory, TensorLean lets developers specify a kernel at the tensor level, with no more effort than writing a PyTorch reference. Given the kernel's specification and PTX, TensorLean verifies the correctness automatically: it reduces concurrency to sequential execution, infers loop invariants over semirings, and proves correctness under tolerance. With TensorLean, we verified 149 kernel instances on their PTX, covering all 37 kernels of FlashAttention-2 to -4 and more from DeepGEMM and Liger Kernel, and proved 83 of them correct. During verification, we found 38 bugs that the kernels' own tests never caught. From math to machine, TensorLean leaves no step unproven.
References
- Jonathan T. Barron et al. A Fix for Softmax and Cross-Entropy. arXiv:2610.31642, 2026.
- Aaron Grattafiori et al. The Llama 3 Herd of Models. arXiv:2407.21783, 2024.
- Ziming Zhou et al. OpGuard: Bitwise Alignment for Precise and General Debugging of Production LLM Training. OSDI 2026.
- OpenAI. Unexpected responses from ChatGPT. OpenAI incident report, February 2024.
- Anthropic. A postmortem of three recent issues. Anthropic Engineering, September 2025.
- Gerwin Klein et al. seL4: Formal Verification of an Operating-System Kernel. Communications of the ACM, 2010.
- Andrea Lattuada et al. Verus: A Practical Foundation for Systems Verification. SOSP 2024.
- Zihao Ye et al. CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution. arXiv:2608.12629, 2026.
- [Bug][FA4] compute_block_sparsity: data race on reduction_buffer can drop blocks with unmasked keys. FlashAttention GitHub issue #3195.
- Leslie Lamport. Time, Clocks, and the Ordering of Events in a Distributed System. CACM 1978.
- Kshitij Dubey et al. Equivalence Checking of ML GPU Kernels. OOPSLA 2026.
- Mengdi Wu et al. SentryWard: Formal Verification of Warp-Specialized GPU Kernels. 2026.
- Martin Brain et al. Building Better Bit-Blasting for Floating-Point Problems. TACAS 2019.
- Alexey Solovyev et al. Rigorous Estimation of Floating-Point Round-Off Errors with Symbolic Taylor Expansions. TOPLAS 2018.
- K. Rustan M. Leino. Dafny: An Automatic Program Verifier for Functional Correctness. LPAR 2010.
- NVIDIA. Parallel Thread Execution ISA Version 9.4. NVIDIA CUDA Documentation.
- Leonardo de Moura and Sebastian Ullrich. The Lean 4 Theorem Prover and Programming Language. CADE 2021.
- Haniel Barbosa et al. cvc5: A Versatile and Industrial-Strength SMT Solver. TACAS 2022.
- Leonardo de Moura and Nikolaj Bjørner. Z3: An Efficient SMT Solver. TACAS 2008.
- Zihao Ye et al. FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving. MLSys 2025.
- Daniel Lustig et al. A Formal Analysis of the NVIDIA PTX Memory Consistency Model. ASPLOS 2019.
- NVIDIA. NVIDIA Collective Communications Library (NCCL). NVIDIA Developer.
- DeepSeek-AI. DeepEP: A High-Performance Communication Library for Machine Learning Training and Inference. GitHub repository.