Ronald K. Pearson
"An accessible presentation of statistical methods and analysis to deal with imperfect data in real data mining applications."
--Joydeep Ghosh, University of Texas at Austin.
"An appealing feature of this book is the use of fresh datasets that are much larger than those currently found in standard books on outliers and statistical diagnostics."
--Anthony Atkinson, London School of Economics.
Data mining is concerned with the analysis of databases large enough that various anomalies, including outliers, incomplete data records, and more subtle phenomena such as misalignment errors, are virtually certain to be present. Mining Imperfect Data describes in detail a number of these problems, as well as their sources, their consequences, their detection, and their treatment. Specific strategies for data pretreatment and analytical validation that are broadly applicable are described, making them useful in conjunction with most data mining analysis methods. Examples are presented to illustrate the performance of the pretreatment and validation methods in a variety of situations, both simulation based, where "correct" results are known unambiguously, and real data examples that illustrate typical cases met in practice.
Mining Imperfect Data, which deals with a wider range of data anomalies than are usually treated in one book, includes a discussion of detecting anomalies through generalized sensitivity analysis (GSA), a process of identifying inconsistencies using of systematic and extensive comparisons of results obtained by analysis of exchangeable datasets. The book makes extensive use of real data, both in the form of a detailed analysis of a few real datasets and various published examples. Also included is a succinct introduction to functional equations that illustrates their utility in describing various forms of qualitative behavior for useful data characterizations.
Industrial and academic researchers will be interested in this book to learn how to develop strategies and tactics for dealing with a number of critically important data imperfections that must be addressed before obtaining useful analysis results from large databases.
Preface; Chapter 1: Introduction; Chapter 2: Imperfect Datasets: Characters, Consequences, and Causes; Chapter 3: Univariate Outlier Detection; Chapter 4: Data Pretreatment; Chapter 5: What Is a "Good" Data Characterization?; Chapter 6: Generalized Sensitivity Analysis; Chapter 7: Sampling Schemes for a Fixed Dataset; Chapter 8: Concluding Remarks and Open Questions; Bibliography; Index
2005 / vi + 312 pages / Softcover / ISBN-13: 978-0-898715-82-8 / ISBN-10: 0-89871-582-2 /
List Price $78.00 / Member Price $54.60 / Order Code OT93