Reliability Analysis of the PLAYTOPIK Question Generation System

PLAYTOPIK Question System Governance

Four Core Elements of Question Creation

Systematic Design

- A multi-layered and sophisticated difficulty control system is established across four dimensions: vocabulary, grammar, text, and culture.

Cultural Expertise

- Understanding the high-context characteristics of Korean culture, systematizing them by level, and reflecting them in assessment.

- Adjusting the depth of cultural understanding to include uniquely Korean emotional concepts such as “nunchi,” “jeong,” “che-myeon,” and “han.”

Quality Assurance

- Ensuring validity and reliability of all questions through a 3-step verification system and four quality control standards.

- Final quality assurance through review by evaluators (human + AI) and final validation by the question committee.

Relevance to the Times

- While maintaining traditional assessment principles, modern materials reflecting learners’ real-life digital language environments are actively utilized.

- Incorporating modern contexts such as messenger conversations, online shopping, and K-POP into the questions.

Beyond simply generating questions, PLAYTOPIK analyzes past TOPIK test questions to
structure and systematize test creation and evaluation criteria, providing a consistent and fair Korean proficiency assessment platform.

Systematic and Professional Framework for Question Creation and Evaluation

Clear Difficulty-Level Classification System

PLAYTOPIK’s Strength : Multi-layered difficulty control across four dimensions

- Vocabulary, grammar, text, and culture

- More than three detailed levels within each dimension

- Precise difficulty settings with a 10-level sentence structure classification

Vocabulary

Over three levels of vocabulary difficulty classification

Grammar

Over three levels based on syntactic complexity

Text

Classification based on sentence length and complexity

Culture

Three levels: Universal → Korean Context 1 → Korean Context 2

Evaluation of Practical Language Proficiency

PLAYTOPIK’s Approach

- Reflecting diverse real-life situations, from daily topics to modern digital media

- Actively utilizing modern materials such as messenger chats, online shopping, and K-POP

- Assessing comprehension of practical texts such as charts, notices, and advertisements

Emphasis on Practicality

Real Conversations Messenger chats, SNS posts, etc.

Modern Services Online shopping, banking services, etc.

Public Information Notices, instructions, signs, etc.

Advertising & Promotion Charts, ads, promotions, etc.

Sophisticated Understanding of Cultural Contexts

Systematic Classification of Korean Contexts

PLAYTOPIK’s Distinctive Approach

- Three-tier structure: Universal Situations → Korean Context 1 (TOPIK I) → Korean Context 2 (TOPIK II, Levels 5–6)

- Assessing indirect communication reflecting the characteristics of high-context culture

- Including unique emotional concepts of Korean culture such as “nunchi,” “jeong,” “che-myeon,” and “han.”

Universal Situations

Friends, visits, invitations

Korean Context 1 (TOPIK I)

First birthday parties, kimchi festivals, K-POP

Korean Context 2 (TOPIK II, Levels 5–6)

Generational values, the dilemma of “jeong”

PLAYTOPIK’s Design

- TOPIK I: Balanced between traditional culture (first birthday parties, kimchi festivals) and modern culture (K-POP classes)

- TOPIK II Advanced: Complex social issues such as generational value conflicts and dilemmas between “jeong” and public responsibility

- Reflecting high-context characteristics of Korean culture through cultural context

Interpersonal Relations (friends, visits)

Cultural Expression (traditional, modern)

Cultural Dilemmas (jeong, responsibility)

Indirect Communication

Comprehensive Evaluation Domains and Quality Management System

Diverse Question Types

Core Assessment Elements of PLAYTOPIK

- Measuring “main idea grasping,” “detail comprehension,” and “inference” from multiple perspectives

Topic Identification Type

Understanding the main idea

Fill-in-the-Blank Type

Integrated vocabulary and grammar

Chart/Notice Type

Information processing

Short Passage Consistency Type

Fact checking

Main Idea Identification Type

Understanding implied meaning

Long Passage Comprehension Type

Integrated understanding

Sentence Ordering Type

Discourse structure

Sentence Insertion Type

Information flow

Connector Selection Type

Logical relationship

Purpose Identification Type

Pragmatics

Long Passage Inference Type

Contextual inference

Multi-layered Quality Verification

PLAYTOPIK’s Verification Process

- 3-step verification system and four quality control standards

Step 1: AI + Human Question Creation

Collaboration between AI and experts to secure diverse question sets

Step 2: Review by Evaluators

Validation of content validity, linguistic naturalness, and cultural appropriateness by experts and AI

Step 3: Final Validation

Final quality assurance and direction setting by the question committee

Vocabulary

Building and adhering to a vocabulary database by curriculum level

Grammar

Utilizing practical grammar elements centered on communicative functions

Text

Providing realistic questions through natural and authentic discourse composition

Culture

Assessing cultural understanding by harmoniously reflecting universal situations and Korean contexts