About Me
I am a CS PhD student at Duke University, advised by Prof. Anru Zhang, interning at ByteDance, doing autoresearch on post-training.
I also run Nonlinearity, an independent research effort on self-influencing systems.
Action required
Problem: The current root path of this site is "baseurl ("_config.yml.
Solution: Please set the
baseurl in _config.yml to "Education
-
Duke UniversityPh.D. in Computer ScienceAug. 2025 – Present -
University of PennsylvaniaM.S.E. in Scientific ComputingAug. 2023 - May. 2025 -
Wuhan UniversityB.E. in Engineering MechanicsSep. 2019 - Jul. 2023
Experience
-
ByteDance, San Jose, CAResearch Scientist InternJun. 2026 - Present
News
Selected Publications (view all )
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
Yufa Zhou
arXiv preprint 2026
We show that LLMs reach an in-context fact by its order of mention, not by the name in the question: a shared, steerable, low-rank subspace of fact addresses, found across 14 models from 1.5B to 32B parameters and formed early in pretraining, lets a question point to the fact it asks about while the context supplies what that fact says.
The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention
Yufa Zhou
arXiv preprint 2026
We show that LLMs reach an in-context fact by its order of mention, not by the name in the question: a shared, steerable, low-rank subspace of fact addresses, found across 14 models from 1.5B to 32B parameters and formed early in pretraining, lets a question point to the fact it asks about while the context supplies what that fact says.
The Geometry of Reasoning: Flowing Logics in Representation Space
Yufa Zhou*, Yixiao Wang*, Xunjian Yin*, Shuyan Zhou, Anru R. Zhang(* equal contribution)
ICLR 2026
We study how LLMs “think” through their embeddings by introducing a geometric framework of reasoning flows, where reasoning emerges as smooth trajectories in representation space whose velocity and curvature are governed by logical structure rather than surface semantics, validated through cross-topic and cross-language experiments, opening a new lens for interpretability.
The Geometry of Reasoning: Flowing Logics in Representation Space
Yufa Zhou*, Yixiao Wang*, Xunjian Yin*, Shuyan Zhou, Anru R. Zhang(* equal contribution)
ICLR 2026
We study how LLMs “think” through their embeddings by introducing a geometric framework of reasoning flows, where reasoning emerges as smooth trajectories in representation space whose velocity and curvature are governed by logical structure rather than surface semantics, validated through cross-topic and cross-language experiments, opening a new lens for interpretability.
Why Do Transformers Fail to Forecast Time Series In-Context?
Yufa Zhou*, Yixiao Wang*, Surbhi Goel, Anru R. Zhang(* equal contribution)
NeurIPS 2025 Workshop: What Can('t) Transformers Do? Oral (3/68 ≈ 4.4%)
We analyze why Transformers fail in time-series forecasting through in-context learning theory, proving that, under AR($p$) data, linear self-attention cannot outperform classical linear predictors and suffers a strict $O(1/n)$ excess-risk gap, while chain-of-thought inference compounds errors exponentially—revealing fundamental representational limits of attention and offering principled insights.
Why Do Transformers Fail to Forecast Time Series In-Context?
Yufa Zhou*, Yixiao Wang*, Surbhi Goel, Anru R. Zhang(* equal contribution)
NeurIPS 2025 Workshop: What Can('t) Transformers Do? Oral (3/68 ≈ 4.4%)
We analyze why Transformers fail in time-series forecasting through in-context learning theory, proving that, under AR($p$) data, linear self-attention cannot outperform classical linear predictors and suffers a strict $O(1/n)$ excess-risk gap, while chain-of-thought inference compounds errors exponentially—revealing fundamental representational limits of attention and offering principled insights.
Mentees
-
Yixiao Wang — MS in Statistical Science @ Duke
Teaching
-
COMPSCI 590.04: Principles of Deep Learning (Spring 2026, Duke) — Instructor: Prof. Rong Ge
Academic Services
-
Conference Reviewer: ICLR (2025, 2026), NeurIPS 2026, ICML 2026, AAAI (2026, 2027).