TAG: Tangential Amplifying Guidance for Hallucination-Resistant Sampling

We amplify the tangential component of each diffusion update, steering sampling trajectories toward the data manifold without retraining or extra denoiser calls.

Hyunmin Cho1*, Donghoon Ahn2*, Susung Hong3*, Jee Eun Kim1, Seungryong Kim4†, and Kyong Hwan Jin1†

1Korea University 2UC Berkeley 3University of Washington 4KAIST AI

*Equal contribution †Corresponding authors

Paper OpenReview Code Hugging Face Demo BibTeX

Visual summary

Overview of TAG: trajectory geometry, tangential decomposition, quantitative evaluations, and qualitative comparisons across diffusion backbones and guidance methods.
TAG at a glance. An intermediate sample provides a local projection basis; amplifying only the tangential component sharpens motion along the data manifold while preserving the sampler’s radial schedule.

Overview

Abstract

Diffusion models achieve state-of-the-art image generation but often produce semantic inconsistencies, or hallucinations. Existing inference-time guidance methods rely on external signals or architectural modifications, adding computational overhead. We propose Tangential Amplifying Guidance (TAG), a training-free, architecture-agnostic, plug-and-play guidance method that operates purely on trajectory signals.

TAG uses an intermediate sample as a projection basis and amplifies the tangential components of the estimated score to correct the sampling trajectory. A first-order Taylor analysis shows that this steers the state toward higher-probability regions of the data manifold, reducing inconsistencies and improving fidelity while adding negligible overhead to existing samplers.

Why hallucinations emerge

Diffusion trajectories can drift through low-density valleys between modes of the data distribution, yielding mixed-up objects, implausible structures, and mismatched attributes. Standard classifier-free guidance applies a single scalar magnification to its residual and ignores this local directional geometry.

Most guidance is geometry-unaware: it does not distinguish updates that move along the data manifold from those that distort the prescribed noise schedule.

TAG makes that distinction explicit. The radial component follows the schedule; the tangential component carries rich structural information along the manifold. Strengthening the latter improves semantics without breaking the former.

Method

Amplify the data-aligned direction

Without guidance, a trajectory drifts away from the data manifold.
(a) No guidance
TAG decomposes and amplifies the tangential part of a diffusion update.
(b) TAG update
The TAG trajectory follows the geometry of the data manifold.
(c) Manifold trajectory
On the latent sphere, TAG separates the base increment into a radial component $P_{\mathcal M}\Delta$ and a tangential component $P^{\perp}_{\mathcal M}\Delta$, then strengthens only the tangential motion toward higher-density regions.

01

A base update from any sampler

At noise level $t_{k+1}$, a base solver such as DDIM or EDM produces a provisional next state $\tilde{x}_k$. Its increment is

$\Delta_{k+1} := \tilde{x}_k - x_{k+1}.$

02

Decompose on the latent sphere

With $\widehat{x}_{k+1}=x_{k+1}/\lVert x_{k+1}\rVert_2$, the projectors $P_{\mathcal M}=\widehat{x}\widehat{x}^{\top}$ and $P^{\perp}_{\mathcal M}=I-P_{\mathcal M}$ separate schedule-following radial motion from structure-carrying tangential motion.

03

Amplify only the tangential part

A single hyperparameter $\eta \ge 1$ controls the intervention:

$x_k \leftarrow x_{k+1} + P_{\mathcal M_{k+1}}\Delta_{k+1} + \eta\,P^{\perp}_{\mathcal M_{k+1}}\Delta_{k+1}.$

No extra forward pass, auxiliary model, or schedule change is required.

Theory

Higher likelihood, intact schedule

We cast inference-time guidance as locally maximizing log-likelihood under a bounded-step constraint. A first-order Taylor analysis measures the gain of a TAG update as

$G(\eta) := \big(\Delta^{\mathrm{TAG}}_{k+1}\big)^\top \nabla_x \log p(x\mid t_{k+1})\big|_{x=x_{k+1}}.$

Increasing $eta$ monotonically raises $G(\eta)$. TAG therefore steers the sampler toward higher-density regions of the data manifold. At the same time, the radial first-order term is preserved exactly:

$\langle \widehat{x}_{k+1},\Delta^{\mathrm{TAG}}_{k+1}\rangle = \langle \widehat{x}_{k+1},\Delta_{k+1}\rangle.$

This is why the intervention sharpens data alignment without disturbing the noise schedule the sampler was trained to follow.

Evidence

Consistent gains across backbones

TAG improves unconditional and text-conditional generation across Stable Diffusion backbones with no training, no architecture change, and zero extra denoiser calls. Orange rows apply TAG; bold values mark the better result within each backbone pair.

Table 1

Unconditional generation

SD-series · COCO 2014

MethodFID ↓IS ↑AES ↑CMMD ↓
SD v1.558.4115.595.0031.069
TAG v1.546.2016.775.0640.778
SD v2.178.5412.525.2991.395
TAG v2.159.9413.365.3201.122
SDXL119.149.085.6452.474
TAG XL90.718.915.5772.201
SD384.2611.535.2611.671
TAG 379.1111.735.3651.564

Table 2

Conditional generation

Text-to-image · COCO 2014

MethodFID ↓ImageReward ↑CLIP ↑
SD v1.533.49−0.34225.00
TAG v1.526.61−0.33925.09
SD v2.126.120.14325.35
TAG v2.121.590.42426.16
SDXL29.280.27425.41
TAG XL28.530.29225.49
SD329.021.03026.39
TAG 327.541.04326.56

TAG consistently improves fidelity in both settings. Results for PAG and SEG stacks, T2I-CompBench, geometry-aware CFG variants, Qwen-Image, Wan2.2 video, and efficiency overhead are available in the paper.

Qualitative

Hallucination-resistant sampling

The top row shows each baseline; the bottom row uses the same prompt with TAG. Across unconditional, conditional, PAG, and SEG settings, tangential amplification resolves structural artifacts while preserving the backbone and sampling schedule.

Baseline

SD3 unconditional baseline sample oneSD3 unconditional baseline sample two
SDXL conditional baseline sample oneSDXL conditional baseline sample two
PAG and SDXL baseline sample onePAG and SDXL baseline sample two
SEG and SDXL baseline sample oneSEG and SDXL baseline sample two

+ TAG

SD3 unconditional sample with TAG oneSD3 unconditional sample with TAG two
SDXL conditional sample with TAG oneSDXL conditional sample with TAG two
PAG and SDXL sample with TAG onePAG and SDXL sample with TAG two
SEG and SDXL sample with TAG oneSEG and SDXL sample with TAG two

Same prompt, same backbone. TAG sharpens object identity and binding in standalone sampling, and composes with existing PAG and SEG guidance without additional model evaluations.

Citation

BibTeX

@inproceedings{
cho2026tag,
title={{TAG}: Tangential Amplifying Guidance for Hallucination-Resistant Sampling},
author={Hyunmin Cho and Donghoon Ahn and Susung Hong and Jee Eun Kim and Seungryong Kim and Kyong Hwan Jin},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=1XJqPy2LhA}
}