About
I'm Shiyan Xu. I work on distributed systems and data infrastructure. At Applied Intuition (opens in a new tab), I focus on AI data infrastructure: the platform that curates massive multimodal datasets for training AI models, built on a multimodal data lakehouse in Lance (opens in a new tab).
Much of my work has been in the open-source lakehouse ecosystem. I'm a member of the project management committee for Apache Hudi (opens in a new tab) and created Hudi-rs (opens in a new tab), its native Rust implementation. I also wrote Apache Hudi: The Definitive Guide (opens in a new tab) (O'Reilly) as lead author, and the ebook Apache Hudi: Zero to One (opens in a new tab).
Outside work, I mentor data engineers through the Data Engineer Things mentorship program (opens in a new tab), and I spend most weekends hiking around the Bay Area.
Follow
Find me on X and LinkedIn.
