Performance Evaluation of the Generative Pre-trained Transformer (GPT-4) on the Family Medicine In-Training Examination

References

1. 1.
   1. Arora A,
   2. Arora A

   . The promise of large language models in health care. Lancet 2023;401:641.

   OpenUrlCrossRef
2. 2.
   1. Sallam M

   . The utility of ChatGPT as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. medRxiv 2023. Published online 2023–02.
3. 3.
   1. Abbasian M,
   2. Azimi I,
   3. Rahmani AM,
   4. Jain R

   . Conversational health agents: a personalized LLM-powered agent framework. Published online January 23, 2024. Accessed January 30, 2024. Available at: http://arxiv.org/abs/2310.02374.
4. 4.
   1. Zhu L,
   2. Mou W,
   3. Chen R

   . Can the ChatGPT and other large language models with internet-connected database solve the questions and concerns of patient with prostate cancer and help democratize medical knowledge? J Transl Med 2023;21:269.

   OpenUrlCrossRef
5. 5.
   1. Rao AS,
   2. Pang M,
   3. Kim J,
   4. et al

   . Assessing the utility of ChatGPT throughout the entire clinical workflow. medRxiv 2023. Published online 2023–02.
6. 6.
   1. Shaheen MY

   . Applications of artificial intelligence (AI) in healthcare: A review. ScienceOpen Preprints. Published online 2021. Accessed January 30, 2024. Available at: https://www.scienceopen.com/hosted-document?doi=10.14293/S2199-1006.1.SOR-.PPVRY8K.v1.
7. 7.
   1. Patil S,
   2. Shankar H

   . Transforming healthcare: harnessing the power of AI in the modern era. International Journal of Multidisciplinary Sciences and Arts 2023;2:60–70.

   OpenUrl
8. 8.
   1. Hanna K

   . Exploring the applications of ChatGPT in family medicine education: five innovative ways for faculty integration. PRiMER 2023;7:26. Accessed January 30, 2024. Available at: https://journals.stfm.org/primer/2023/lte-hanna-2023-0053/.

   OpenUrl
9. 9.
   1. Han ER,
   2. Yeo S,
   3. Kim MJ,
   4. Lee YH,
   5. Park KH,
   6. Roh H

   . Medical education trends for future physicians in the era of advanced technology and artificial intelligence: an integrative review. BMC Med Educ 2019;19:1–15.

   OpenUrl
10. 10.
    1. Nori H,
    2. King N,
    3. McKinney SM,
    4. Carignan D,
    5. Horvitz E

    . Capabilities of gpt-4 on medical challenge problems. arXiv preprint 2023; arXiv:230313375. Published online.
11. 11.
    1. Brin D,
    2. Sorin V,
    3. Vaid A,
    4. et al

    . Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep 2023;13:16492.

    OpenUrl
12. 12.
    1. Ali R,
    2. Tang OY,
    3. Connolly ID,
    4. et al

    . Performance of ChatGPT and GPT-4 on neurosurgery written board examinations. Neurosurgery 2022. Published online10–1227.
13. 13.
    1. Hofmann HL,
    2. Guerra GA,
    3. Le JL,
    4. et al

    . The rapid development of artificial intelligence: GPT-4’s performance on orthopedic surgery board questions. Orthopedics 2023;47:1–5. Published online September 27.

    OpenUrl
14. 14.
    GPT-4. Available at: https://openai.com/research/gpt-4.
15. 15.
    1. Lee P,
    2. Bubeck S,
    3. Petro J

    . Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. Drazen JM, Kohane IS, Leong TY, eds. N Engl J Med 2023;388:1233–9.

    OpenUrlCrossRefPubMed
16. 16.
    1. Ito N,
    2. Kadomatsu S,
    3. Fujisawa M,
    4. et al

    . The accuracy and potential racial and ethnic biases of GPT-4 in the diagnosis and triage of health conditions: evaluation study. JMIR Med Educ 2023;9:e47532.

    OpenUrl
17. 17.
    1. Gertz RJ,
    2. Bunck AC,
    3. Lennartz S,
    4. et al

    . GPT-4 for automated determination of radiologic study and protocol based on radiology request forms: a feasibility study. Radiology 2023;307:e230877.

    OpenUrlCrossRef
18. 18.
    1. Fink MA,
    2. Bischoff A,
    3. Fink CA,
    4. et al

    . Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology 2023;308:e231362.

    OpenUrlCrossRefPubMed
19. 19.
    1. Zhou J,
    2. He X,
    3. Sun L,
    4. et al

    . SkinGPT-4: an interactive dermatology diagnostic system with visual large language model. Published online 2023. Accessed January 30, 2024. Available at: https://repository.kaust.edu.sa/handle/10754/692649.
20. 20.
    1. Han C,
    2. Kim DW,
    3. Kim S,
    4. et al

    . Evaluation of GPT-4 for 10-year cardiovascular risk prediction: insights from the UK Biobank and KoGES data. iScience 2023. Accessed January 30, 2024. Available at: https://www.cell.com/iscience/pdf/S2589-0042(24)00243-8.pdf.
21. 21.
    1. Cheng S,
    2. Chang C,
    3. Chang W,
    4. et al

    . The now and future of ChatGPT and GPT in psychiatry. Psychiatry Clin Neurosci 2023;77:592–6.

    OpenUrl
22. 22.
    1. Jee SH,
    2. Cabana MD

    . Indices for continuity of care: a systematic review of the literature. Med Care Res Rev 2006;63:158–88.

    OpenUrlCrossRefPubMedWeb of Science
23. 23.
    1. Fernández-Ruiz I

    . Low-dose colchicine shows promise in chronic coronary disease. Nat Rev Cardiol 2020;17:680–1.

    OpenUrl