GPT Performance on the Bar Exam in Taiwan (GPT在台灣專門職業及技術人員高等考試律師考試中的表現)

11 Pages Posted: 31 Mar 2023 Last revised: 5 Apr 2023

See all articles by Mark Shope

Mark Shope

National Yang Ming Chiao Tung University; Indiana University Robert H. McKinney School of Law

Date Written: March 21, 2023

Abstract

English abstract: This paper reports the performance of the GPT-4 Model of ChatGPT Plus (“ChatGPT4”) on the multiple-choice section of the 2022 Lawyer’s Bar Exam in Taiwan. ChatGPT4 outperforms approximately half of human test-takers on the multiple-choice section with a score of 342. This score, however, would not advance a test taker to the second and final essay portion of the exam. Therefore, this paper will not include an evaluation of ChatGPT4’s performance on the essay portion of the exam.

Mandarin abstract: 本文報告了ChatGPT Plus的GPT-4模型(以下簡稱“ChatGPT4”)在2022年台灣專門職業及技術人員高等考試律師考試選擇題部分的表現。ChatGPT4在選擇題部分得分為342分,其表現超過了約一半的人類考生。然而,這個分數無法使考生晉級至該考試的第二部分即申論題的部分。因此,本文不包括對ChatGPT4在申論題部分表現的評估。

Keywords: GPT-4, ChatGPT, bar exam, law, Taiwan

JEL Classification: K00, K10

Suggested Citation

Shope, Mark, GPT Performance on the Bar Exam in Taiwan (GPT在台灣專門職業及技術人員高等考試律師考試中的表現) (March 21, 2023). Available at SSRN: https://ssrn.com/abstract=4394826 or http://dx.doi.org/10.2139/ssrn.4394826

Mark Shope (Contact Author)

National Yang Ming Chiao Tung University ( email )

No. 1001, Daxue Rd, East District
Hsinchu, 300
Taiwan

Indiana University Robert H. McKinney School of Law ( email )

530 West New York Street
Indianapolis, IN 46202
United States

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
736
Abstract Views
2,767
Rank
63,957
PlumX Metrics