From conversational chatbots to humanoid robots, artificial intelligence (AI) technology is expanding beyond daily life into the industrial realm. The emergence of frontier models with capabilities exceeding human experts and agentic AI that plans and acts on its own has created new threats of unpredictability alongside innovation. The scope of risks is all-encompassing: unfair discrimination in hiring and lending, fake news generation, cyberattacks, and the leaking of chemical and biological weapons manufacturing methods. These risks have a "low-probability, high-impact" characteristic — even if the likelihood of occurrence appears low, recovery becomes impossible once a critical threshold is crossed. For this reason, experts predict that if these risks materialize, they will spread at a pace that overwhelms human response capabilities.
AI-leading nations have already shifted their AI safety policy direction. The United Kingdom, which launched the world's first AI Safety Institute (AISI) in 2023, recently renamed the agency the "AI Security Institute," elevating its purpose from safety to national security. The United States, with the launch of Donald Trump's second administration, has also withdrawn the regulation-centered executive orders of the Joe Biden administration and reorganized its safety institute into the "Center for AI Standards and Innovation (CAISI)," moving to defend its leadership as an AI-leading nation.
While each country competes for AI leadership, they are nonetheless joining hands on AI safety. Because AI risks do not respect borders and trigger global chain reactions, no single country's independent regulation can address them. Under this shared understanding, the "International Network of AI Safety Institutes," launched in November 2024, recently renamed itself the "Network for AI Measurements, Evaluations and Science (NAAIMES)" and changed its approach to addressing AI risks.
The renaming of major countries' AISIs reflects a shift in the international community's attitude toward AI safety. The global trend is moving away from the name of "safety" toward newly establishing standards centered on "evidence-based evaluation science" that quantitatively measures AI performance and risks. This reflects a consensus among nations to minimize the uncertainty of AI risks through the standardization of evaluation criteria.
Korea successfully hosted the second AI Safety Summit in Seoul in May 2024, and has recently contributed to establishing a global AI safety net through independent evaluation science research, including the construction of self-evaluation indicators for multilingual models led by industry and academia. Institutional changes are also following the global trend. The AI Safety Institute was launched in November 2024, and the AI Framework Act, which took effect in January this year, established the legal basis for the agency.
What remains is overcoming the structural limitations of the current response system. Korea's current framework operates as a "silo structure" in which the fields of security, safety, ethics, and policy function separately. The absence of a common definition of risk and shared measurement indicators limits comprehensive risk analysis and response. Defending against AI risks that simultaneously cross the boundaries of technology, society, and institutions requires an integrated response system that encompasses all of these dimensions.
The National AI Strategy Committee — which includes 12 ministers, the Chairperson of the Personal Information Protection Commission, and the Third Deputy Director of the National Security Office as ex officio members — is well-suited for this role. The enforcement decree of the AI Framework Act must be revised so that the committee can exercise governance leadership as the national control tower for AI safety. Furthermore, the National Assembly and the government must work together to increase the budget so that the AI Safety Institute can operate with independent AI evaluation authority, like the UK's Security Institute or the US Center for Standards and Innovation.