Appendix Table 5.

Estimated AUROCC for All Candidate Models

	Derivation group		Validation group
	Clinician	Patient	Clinician	Patient
A priori symptom score	0.77	0.79	0.75	0.56
Cough/fever heuristic	0.71	0.75	0.67	0.55
Cough/fever symptom score	0.71	0.76	0.67	0.57
Cough/fever/acute onset heuristic	0.64	0.62	0.61	0.57
Cough/fever/acute onset symptom score	0.73	0.77	0.67	0.55
Cough/fever/myalgia heuristic	0.72	0.72	0.75	0.58
Cough/fever/myalgia symptom score	0.75	0.76	0.74	0.55
Re-fit FluScore model (Ebell 2012)	0.77	0.79	0.74	0.57
LASSO score	0.86	0.78	0.71	0.60
LASSO heuristic	0.50	0.74		0.57
CART (manual)	0.81	0.82	0.67	0.55
FFT	0.77	0.73	0.70	0.53
C5.0 tree (manual)	0.73	0.79	0.65	0.55
Conditional inference tree (manual)	0.79	0.80	0.63	0.57
Bayesian Additive Regression Trees (BART)	0.86	0.81	0.70	0.64
C5.0 tree (tuned)	0.85	0.75	0.60	0.57
CART (tuned)	0.81	0.82	0.67	0.55
Conditional inference tree (tuned)	0.79	0.79	0.63	0.55
Elastic net logistic regression	0.87	0.83	0.70	0.65
Unpenalized logistic regression	0.88	0.84	0.67	0.62
k-Nearest Neighbors classifier	0.89	0.92	0.66	0.65
LASSO logistic regression	0.87	0.83	0.70	0.65
Naive Bayes classifier	0.83	0.79	0.74	0.68
Random forest	0.89	0.87	0.73	0.59
SVM (linear kernel)	0.85	0.82	0.75	0.65
SVM (polynomial kernel)	0.83	0.79	0.74	0.67
SVM (RBF kernel)	0.83	0.82	0.74	0.69
Gradient-boosted tree	0.85	0.87	0.71	0.61

Abbreviations: AUROCC, Area Under the Receiver Operating Characteristic Curve; LASSO, Least Absolute Shrinkage and Selection Operator; SVM, Support vector machines; CART, Classification and Regression Tree Algorithm.
Notes: The AUROCC was not estimable for the LASSO heuristic model on the validation set of clinician-reported symptom data, as all patients were assigned the same score in this set.