Investigating a predictive certainty measure for ensemble based HIV classification systems

This paper investigates whether there is a correlation between the predictive certainty measure for ensemble based classifiers and the prediction accuracy. The predictive certainty measure is the percentage of most dominant outcome from all the possible outcomes for the ensemble of classifiers. Three neural network ensemble classifiers were created using Bagging, Boosting and Bayesian Methods. All three ensembles are used to classify a patients HIV status using demographic variables obtained from an antenatal seroprevalence survey. All three ensembles perform equally well for the HIV classification but the ensemble obtained using Bayesian training method is most suited for giving a relevant predictive certainty measure. The predictive certainty measures obtained for the Bagging and Boosting ensembles are not suitable to use as a confidence measure because the prediction accuracy is low for cases that have high predictive certainty. The Bayesian ensemble is more suitable for making decisions.