Hongseung Yu
M.S. student at Seoul National University
I am Hongseung Yu, a M.S. candidate in Electrical and Computer Engineering (ECE) at Seoul National University, advised by Prof. Kyunghan Lee.
My research centers on efficient computing systems for machine learning, with a focus on LLM inference. I am particularly interested in designing inference systems that are tightly coupled with the underlying environment—hardware and operating systems. My interests also extend to hardware-software co-design and resource-constrained edge computing.
Publications
-
mzCache: On-Device LLM Memory Management under Multitasking
Hongseung Yu, Minsung Kim, Jongseok Park, Kyunghan Lee
ACM MobiCom 2026, Austin, TX, USA (Acceptance rate: 14.0%, 88/628) | ACM DL | Demo
On mobile SoCs with unified memory, the OS evicts an on-device LLM's weights and KV cache under multitasking, and restoring them significantly delays the response. mzCache is the first LLM inference system to manage LLM memory itself rather than leaving it to the OS, reducing Time-to-First-Token by 2.1–5.5× over storage-backed offloading.
-
Accelerating Offloading-Based MoE Inference by Interleaving Concurrent Requests
Hongseung Yu, In Gim, Kyunghan Lee, Lin Zhong
Under Review | arXiv
Expert offloading lets large MoE models run locally, but the GPU stalls on expert fetches, and prefetching wastes bandwidth on mispredicted experts. Bevy schedules concurrent requests instead of predicting experts, letting requests whose experts are ready run ahead to reveal what to fetch next. It delivers up to 3.46× the decode throughput of llama.cpp and 1.55× that of the fastest prefetching baseline.