CityTrajBench Finally Puts AI Traffic Generators in a Fair Race
CityTrajBench standardizes how we compare vehicle trajectory generators, revealing that no single model wins across all quality dimensions
- CityTrajBench is the first unified benchmark for city-scale vehicle trajectory generation, standardizing datasets, preprocessing, and evaluation across model families.
- No single model wins on all metrics: DiffTraj leads on geometric path fidelity, DiffRNTraj on road-network realism, and TrajFlow on overall balance and efficiency.
- A simple Markov baseline remains competitive on coarse-grained trip statistics, challenging the assumption that complex deep models always win.
- The benchmark evaluates five quality dimensions: global spatial realism, trip-level distribution, geometric similarity, conditional consistency, and efficiency.
- Three real-world datasets are used: Porto, Chengdu, and Shanghai, covering taxi GPS trajectories at city scale.
- Practical use cases include privacy-preserving synthetic data for traffic simulation, urban planning, and training downstream ML models.
Synthetic vehicle trajectories are quietly becoming critical infrastructure for smart cities. Trajectories serve as a cornerstone of intelligent transportation systems, playing an important role in many applications such as traffic flow prediction, route planning, and urban management. But there is a catch: the availability of such data is limited due to privacy issues, ethical concerns, and the high cost associated with infrastructure deployment. That gap has pushed researchers toward generative models that can synthesize realistic GPS trajectories from scratch.
Rapidly developing generative models such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Diffusion Models have demonstrated strong capabilities in learning complex data distributions and generating synthetic data, thereby alleviating the data accessibility issue. The problem is that every team building these models has been doing it on different datasets, with different preprocessing steps and different metrics. That makes comparing them nearly impossible.
A new paper introduces CityTrajBench, a unified benchmark framework designed to finally put all these models on the same playing field. The result is the first apples-to-apples comparison of the major trajectory generation paradigms, and the findings are more nuanced than the field expected.
Why the Comparison Problem Is Worse Than It Sounds
Trajectory models are often implemented in different pipelines, making direct comparisons challenging and fairness hard to ensure. This is not just an academic inconvenience. When a paper claims its diffusion model beats a GAN, you genuinely cannot tell whether that is because the model is better or because it was trained on a cleaner dataset with a more favorable evaluation metric. The field has been building on sand.
The use of synthetic data generated by generative models offers significant advantages for privacy protection in transportation, with the ability to preserve the statistical or distributional properties and patterns of real-world data while excluding personally identifiable information. The stakes are high: urban planners can use synthetic data to simulate the impact of new infrastructure projects on traffic flow, while data scientists can train machine learning models to predict congestion or optimize public transport schedules. Getting the benchmark right matters.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
