JunYoung Lee
B.S. in Materials Science & Engineering, researching multimodal learning, agent systems, and domain-specialized foundation models.
Research Statement
As an undergraduate in Materials Science and Engineering at Yonsei University, my research is driven by a central question: what does it truly take for AI and its users to become genuinely integrated? My interest lies at the intersection of multimodal learning and agent systems: building models that unify text, images, and structured data within a single representation space to produce outputs precisely aligned with user intent.
My ultimate goal is to train and develop domain-specialized foundation models from the ground up, and to build systems capable of serving them in real-world settings. I aim to close the gap between general-purpose AI and domain-specialized, genuinely personalized assistance.
Rather than merely adopting existing AI tools, I place equal weight on redesigning model architectures, training pipelines, and serving infrastructure from scratch to meet the demands of each domain, pursuing research that allows AI to integrate efficiently into human life.
News
Research
Unified Multimodal Parsing System for Automated Domain Data Construction
Training domain-specialized models requires clean, structured datasets, yet real-world data is fragmented across heterogeneous modalities (PDFs, tables, figures, charts). I propose a single-system agentic pipeline that integrates multiple multimodal parsers (VLM, OCR, table-extraction) and fuses their outputs via an LLM into coherent structured records: automated crawling → multimodal parsing → LLM fusion → self-verification → quality assessment. Constructed datasets are validated for real task-utility through a downstream AutoML pipeline, forming a closed-loop methodology that goes beyond collection to verify the data's usefulness.
VLM-based Clinical Decision Support for Pressure Ulcer Care
A domain-specialized vision-language model for automated stage classification of pressure ulcers from clinical images and automated nursing-record generation. The backbone is Qwen2.5-VL adapted via LoRA parameter-efficient fine-tuning, then aligned with Direct Preference Optimization (DPO) on preference data collected from licensed clinical nurses, yielding gains in both reasoning quality and clinical adequacy, beyond classification accuracy alone.
BioMatAI: Biocompatible Materials Predictive Modeling
An end-to-end pipeline for predicting properties of biocompatible materials. An automated crawling and curation pipeline builds a literature-derived database of ~1,100 materials; protein–surface docking runs through an automated CIF → slab → PDB pipeline on the Rosetta SurfRosetta protocol, with custom metal parameter files for TiO₂, TiAlV, and calcite surfaces. Docking outputs are structured into tabular form to serve as training data for downstream ML models.
Experience
Yonsei tAI LAB
- Domain-specialized VLM clinical research: pressure-ulcer stage classification and automated nursing-record generation (LoRA fine-tuning of Qwen2.5-VL, DPO alignment).
- Led BioMatAI: data acquisition, Rosetta-based protein–surface docking, and tabular ML for biocompatible-material property prediction.
AI Agent System
- Backend architecture (primary): custom LLM-agent backend on Hono + Bun with the Vercel AI SDK; a multi-stage tool-call pipeline: invocation → sandboxed execution → result verification.
- Built a per-user isolated code-execution sandbox and a memory-context module for cross-session knowledge retention and retrieval.
- Developing a domain-specialized materials-science lab agent integrating TabPFN foundation models and SHAP-based interpretability; Next.js frontend.
Namdong Senior Club
- Built and maintain the official website: board system, banner/popup management, and admin interfaces.
Honors & Awards
KoMaP Digital Data Platform: AI Model Development Competition
- Ceramic track: a Transformer-based regression model predicting polarization resistance (Rp), ohmic resistance (Ro), and maximum power density of SOFC electrode materials from multi-layered compositional and multi-stage process variables, achieving top performance via domain-informed feature engineering and multi-task learning.
- Chemical track: extended the same backbone to property prediction of lightweight composite materials for future mobility, applying domain adaptation across material families.
Samsung AI Challenge: Visually-rich Document Understanding
- A two-stage VLM pipeline that decouples layout detection from content recognition, with module fusion dynamically reweighted by LLM-driven confidence scores.
Yonsei × Upstage AGI Agent Application Hackathon
- An agent framework that normalizes free-text radiology reports into SNOMED CT standard concepts.
Talks
“Current Trends in 3D Vision Research: Integrating Foundations with Transformers”
Education
Yonsei University, Seoul
Incheon Science High School
Technical Skills
Selected Projects & Service
Deepfake Detection (Team D.K)
- CLIP ViT-L/14 backbone with a D2ST temporal adapter; a multi-stage training pipeline: classification → GradCAM-based pseudo-mask → joint detection.
NFC Team, Yonsei Social Innovation Town Program
- Led a project on eco-friendly thermal-insulation wallpaper; responsible for research and promotional planning.