Paper |  Leaderboard

LingLan is a large-scale, expert-curated benchmark for evaluating large language models (LLMs) in Traditional Chinese Medicine (TCM). It spans five domains (licensing exam, fundamental knowledge, Chinese patent medicine, information extraction, diagnostic–therapeutic reasoning) with 13 subtasks and 25,624 instances. Each dataset includes a 400-item Hard subset to probe robustness.

• What’s in LingLan
• Data Statistics
• Evaluation Metrics
• Download
• Quick Start
• Leaderboards
• Cite LingLan
• Related Benchmarks
• License & Ethics
• Acknowledgements

LingLan covers five domains with diverse formats (single-choice, multiple-choice, cloze, multi-label lists, dosage vectors) under a unified metric system:

• single-choice: Accuracy
• multi-choice: instance-level Accuracy; option-level Precision/Recall/F1
• cloze: character-level F1
• extraction & multi-label: list-level Precision/Recall/F1
• dosage: cosine similarity (primary), MAE when prescription overlap is adequate

LingLan further introduces decision-recognition versions of clinical tasks (single-choice) for clean, reproducible comparisons.

• TCM licensing exam: 1,832
• Fundamental TCM knowledge (single/multi-choice, cloze): 5,844
• Chinese patent medicine (single/multi-choice, cloze): 5,948
• Decision recognition (syndrome / treatment / prescription): 2,000 + 2,000 + 2,000
• Diagnostic–therapeutic reasoning (syndrome / treatment / prescription / dosage): 2,000
• Information extraction — classical: 2,000
• Information extraction — EMR: 2,000
• Total: 9 files, 25,624 items (each dataset includes a 400-item Hard subset)

Classification: Accuracy (single-choice; decision recognition)
Multi-choice: Accuracy; option-level Precision/Recall/F1
Cloze: character-level F1
Extraction & multi-label clinical tasks: list-level Precision/Recall/F1
Dosage: cosine similarity (primary); MAE optionally reported when prescription overlap is reasonable

data: LingLan/data

Refer to simple_evals for more details.

Evaluation codes: tcm_evals

The annotations will not be released until the paper is accepted; therefore, the evaluation code is not currently executable.

Overall averages (%) on Full / Hard sets. (Hard reveals a marked gap between current LLMs and expert-level TCM reasoning.)

@misc{hua2026linglanmidiansystematicevaluationllms,
      title={LingLanMiDian: Systematic Evaluation of LLMs on TCM Knowledge and Clinical Reasoning}, 
      author={Rui Hua and Yu Wei and Zixin Shu and Kai Chang and Dengying Yan and Jianan Xia and Zeyu Liu and Hui Zhu and Shujie Song and Mingzhong Xiao and Xiaodong Li and Dongmei Jia and Zhuye Gao and Yanyan Meng and Naixuan Zhao and Yu Fu and Haibin Yu and Benman Yu and Yuanyuan Chen and Fei Dong and Zhizhou Meng and Pengcheng Yang and Songxue Zhao and Lijuan Pei and Yunhui Hu and Kan Ding and Jiayuan Duan and Wenmao Yin and Yang Gu and Runshun Zhang and Qiang Zhu and Jian Yu and Jiansheng Li and Baoyan Liu and Wenjia Wang and Xuezhong Zhou},
      year={2026},
      eprint={2602.01779},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2602.01779}, 
}


@misc{LingLan2026,
  title={LingLanMiDian: Systematic Evaluation of LLMs on TCM Knowledge and Clinical Reasoning},
  author={Ruihua},
  year={2026},
  note={https://github.com/TCMAI-BJTU/LingLan}
}

• Data are curated from de-identified clinical records, classical texts, official exam sources, and structured knowledge resources.

We thank the curators, annotators, and domain experts who contributed to dataset construction and verification.