Virtual AI testbed lets developers verify massive LLM servers before construction - Tech Xplore

Operating large language model (LLM) services like ChatGPT requires a server infrastructure on the scale of tens of thousands of units. However, constructing actual equipment every time a new AI semiconductor or system architecture needs to be verified incurs massive costs and time.

A research team at our university has developed a "virtual testbed" that can pre-verify performance and efficiency inside a computer before building an actual large-scale AI server.

The research on a large language model (LLM) serving infrastructure simulator (virtual testing software) developed by Professor Jongse Park's research team in the School of Computing won the Best Paper Award at ISPASS 2026 (IEEE International Symposium on Performance Analysis of Systems and Software).

"LLMServingSim 2.0," developed by the research team, is a simulation platform capable of virtually analyzing various hardware and software combinations in complex AI service environments. Researchers and developers can freely experiment with various design options and verify performance without having to directly build expensive, large-scale server infrastructures.

In particular, this technology is drawing attention because it goes beyond the existing Graphics Processing Unit (GPU)-centric environment to support diverse hardware environments, including Neural Processing Units (NPUs), which are rising as next-generation AI semiconductors, and Processing-In-Memory (PIM, a semiconductor technology that performs operations inside the memory).

In other words, it is a technology that allows future-oriented AI semiconductors that have not yet been commercialized to be tested in advance within a virtual datacenter environment.

Through this, it is possible to replicate and analyze inside a computer how much the service speed improves, how much power consumption is reduced, and whether it operates stably even in a server environment scaled to tens of thousands of units when a specific semiconductor is applied.

In addition, it reproduces complex operations that occur during actual AI service operations—such as data processing, request distribution, and memory utilization—at the system level, enabling performance evaluations that are close to reality.

Notably, it can even analyze disaggregated infrastructure environments where multiple server resources are separated and connected for use, showing great potential for utilization in next-generation AI datacenter research.

This simulator is expected to be widely utilized not only by researchers but also by LLM service companies and AI semiconductor startups to design and optimize next-generation AI infrastructures. This is because it can rapidly verify new AI semiconductors or service architectures prior to actual construction, thereby significantly reducing the cost and time of AI infrastructure development.

Professor Jongse Park said, "The competitiveness of AI services is determined not only by the model itself but also by the infrastructure technology that operates it stably and efficiently."

He added, "We hope this simulator will serve as an important foundation for researchers and the industry to develop next-generation AI infrastructures faster and more efficiently."