Missing data imputation through the use of the random forest algorithm

Adam Pantanowitz; Tshilidzi Marwala

doi:10.1007/978-3-642-03156-4_6

Missing data imputation through the use of the random forest algorithm

Adam Pantanowitz
, Tshilidzi Marwala

Faculty of Engineering and the Built Environment

University of the Witwatersrand

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

79 Citations (Scopus)

Abstract

This paper presents a comparison of different paradigms used for missing data imputation. The data set used is HIV seroprevalence data from an antenatal clinic study survey performed in 2001. Data imputation is performed through fivemethods:RandomForests; auto-associative neural networks with genetic algorithms; auto-associative neuro-fuzzy configurations; and two random forest and neural network based hybrids. Results indicate that Random Forests are superior in imputing missing data for the given data set in terms of accuracy and in terms of computation time, with accuracy increases of up to 32 % on average for certain variables when compared with auto-associative networks. While the concept of hybrid systems has promise, the presented systems appear to be hindered by their auto-associative neural network components.

Original language	English
Title of host publication	Advances in Computational Intelligence
Publisher	Springer Verlag
Pages	53-62
Number of pages	10
ISBN (Print)	9783642031557
DOIs	https://doi.org/10.1007/978-3-642-03156-4_6
Publication status	Published - 2009
Event	2nd International Workshop on Advanced Computational Intelligence, IWACI 2009 - Mexico City, Mexico Duration: 22 Jun 2009 → 23 Jun 2009

Publication series

Name	Advances in Intelligent and Soft Computing
Volume	61 AISC
ISSN (Print)	1867-5662

Conference

Conference	2nd International Workshop on Advanced Computational Intelligence, IWACI 2009
Country/Territory	Mexico
City	Mexico City
Period	22/06/09 → 23/06/09

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Auto-associative
Imputation
Missing data
Neural network
Random forest

ASJC Scopus subject areas

General Computer Science

Access to Document

10.1007/978-3-642-03156-4_6

Cite this

@inproceedings{a3651da517e34ef0abaf2cfb4e7e7230,

title = "Missing data imputation through the use of the random forest algorithm",

abstract = "This paper presents a comparison of different paradigms used for missing data imputation. The data set used is HIV seroprevalence data from an antenatal clinic study survey performed in 2001. Data imputation is performed through fivemethods:RandomForests; auto-associative neural networks with genetic algorithms; auto-associative neuro-fuzzy configurations; and two random forest and neural network based hybrids. Results indicate that Random Forests are superior in imputing missing data for the given data set in terms of accuracy and in terms of computation time, with accuracy increases of up to 32 \% on average for certain variables when compared with auto-associative networks. While the concept of hybrid systems has promise, the presented systems appear to be hindered by their auto-associative neural network components.",

keywords = "Auto-associative, Imputation, Missing data, Neural network, Random forest",

author = "Adam Pantanowitz and Tshilidzi Marwala",

year = "2009",

doi = "10.1007/978-3-642-03156-4\_6",

language = "English",

isbn = "9783642031557",

series = "Advances in Intelligent and Soft Computing",

publisher = "Springer Verlag",

pages = "53--62",

booktitle = "Advances in Computational Intelligence",

address = "Germany",

note = "2nd International Workshop on Advanced Computational Intelligence, IWACI 2009 ; Conference date: 22-06-2009 Through 23-06-2009",

}

Pantanowitz, A & Marwala, T 2009, Missing data imputation through the use of the random forest algorithm. in Advances in Computational Intelligence. Advances in Intelligent and Soft Computing, vol. 61 AISC, Springer Verlag, pp. 53-62, 2nd International Workshop on Advanced Computational Intelligence, IWACI 2009, Mexico City, Mexico, 22/06/09. https://doi.org/10.1007/978-3-642-03156-4_6

TY - GEN

T1 - Missing data imputation through the use of the random forest algorithm

AU - Pantanowitz, Adam

AU - Marwala, Tshilidzi

PY - 2009

Y1 - 2009

N2 - This paper presents a comparison of different paradigms used for missing data imputation. The data set used is HIV seroprevalence data from an antenatal clinic study survey performed in 2001. Data imputation is performed through fivemethods:RandomForests; auto-associative neural networks with genetic algorithms; auto-associative neuro-fuzzy configurations; and two random forest and neural network based hybrids. Results indicate that Random Forests are superior in imputing missing data for the given data set in terms of accuracy and in terms of computation time, with accuracy increases of up to 32 % on average for certain variables when compared with auto-associative networks. While the concept of hybrid systems has promise, the presented systems appear to be hindered by their auto-associative neural network components.

AB - This paper presents a comparison of different paradigms used for missing data imputation. The data set used is HIV seroprevalence data from an antenatal clinic study survey performed in 2001. Data imputation is performed through fivemethods:RandomForests; auto-associative neural networks with genetic algorithms; auto-associative neuro-fuzzy configurations; and two random forest and neural network based hybrids. Results indicate that Random Forests are superior in imputing missing data for the given data set in terms of accuracy and in terms of computation time, with accuracy increases of up to 32 % on average for certain variables when compared with auto-associative networks. While the concept of hybrid systems has promise, the presented systems appear to be hindered by their auto-associative neural network components.

KW - Auto-associative

KW - Imputation

KW - Missing data

KW - Neural network

KW - Random forest

UR - https://www.scopus.com/pages/publications/84894310847

U2 - 10.1007/978-3-642-03156-4_6

DO - 10.1007/978-3-642-03156-4_6

M3 - Conference contribution

AN - SCOPUS:84894310847

SN - 9783642031557

T3 - Advances in Intelligent and Soft Computing

SP - 53

EP - 62

BT - Advances in Computational Intelligence

PB - Springer Verlag

T2 - 2nd International Workshop on Advanced Computational Intelligence, IWACI 2009

Y2 - 22 June 2009 through 23 June 2009

ER -

Missing data imputation through the use of the random forest algorithm

Abstract

Publication series

Conference

UN SDGs

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this