ACM Journal of

Data and Information Quality (JDIQ)

Latest Articles

January, 2018 - Call for papers:
Special Issue on Combating Digital Misinformation and Disinformation

Initial submission deadline:
- April 1st, 2018

NEW Non-CS Initial submission:
- May 1st, 2018

Jan. 2016 -- Book Announcement
Carlo Batini and Monica Scannapieco have a new book:

"Data and Information Quality: Dimensions, Principles and Techniques" 

Springer Series: Data-Centric Systems and Applications, soon available from the Springer shop

The Springer flyer is available here

Experience and Challenge papers:  JDIQ now accepts two new types of papers. Experience papers describe real-world applications, datasets and other experiences in handling poor quality data. Challenges papers briefly describe a novel problem or challenge for the IQ community. See Author Guidelines for details.

Forthcoming Articles
Machine Reading of Biomedical Data Dictionaries

This paper describes an approach for automated ingestion of biomedical data dictionaries. Automated ingestion or reading is the process of extracting element details for each of the data elements from a data dictionary in a document format (such as PDF) to a completely structured format. The structured format is essential if the data dictionary metadata is to be used in applications such as data integration, and also in evaluating the quality of the associated data. We present a machine-learning classification solution to the problem using conditional random field (CRF) classifiers and leveraging multiple text and character based features of text rows in the document. We present an evaluation using several actual data dictionary documents demonstrating the effectiveness of our approach.

InfoClean: Protecting Sensitive Information in Data Cleaning

Data quality has become a pervasive challenge for organizations as they wrangle with large, heterogeneous datasets to extract value. Given the proliferation of sensitive and confidential information, it is crucial to consider data privacy concerns during the data cleaning process. For example, in medical database applications, varying levels of privacy are enforced across the attribute values. Attributes such as a patient's country or city of residence may be less sensitive than the patient's prescribed medication. Traditional data cleaning techniques assume the data is openly accessible, without considering the differing levels of information sensitivity. In this work, we take the first steps towards a data cleaning model that integrates privacy as part of the data cleaning process. We present a privacy-aware, constraint based data cleaning framework that differentiates the information content among the attribute values during the data cleaning process to resolve data inconsistencies while minimizing the amount of information disclosed. Our data repair algorithm includes a set of data disclosure operations that considers the information content of the underlying attribute values, while maximizing data utility. Our evaluation using real datasets show that our algorithm scales well, and achieves improved performance and repair accuracy over existing differentially private data cleaning solutions.

Adaptive and Cost-Effective Collection of High-Quality Data for Critical Infrastructure and Emergency Management in Smart Cities -- Framework and Challenges

The paper discusses challenges related to design of a framework for real-time, adaptive, cost-effective collection of high-quality data for critical infrastructure and emergence management. A key objective of the framework is to provide the ability to adaptively collect data based on: capabilities of available data collection technologies; communication capabilities; temporal deadlines; required classification/prediction accuracy; relevant data quality requirements.

Challenge Paper: Data Quality Issues in Queue Mining

Queue mining is a novel research area of data mining that learns queueing models from data logs. These models are then used for performance prediction in queueing-oriented systems. Queue mining combines techniques from process mining, queueing theory, statistics, and optimization. This paper reviews challenges that stem from data quality issues in queue mining, as well as some existing solutions to these challenges.

Addressing Selection Bias in Event Studies with General Purpose Social Media Panels

Data from Twitter have been increasingly employed to study the impact of events. Conventionally, researchers have relied on keywords to create a panel of Twitter users who mention event-related keywords during and after an event. There are limitations to the keyword-based approach. First, the technique suffers from selection bias since users who discuss an event are already more interested in event-related topics beforehand; it is thus unclear whether observed impacts are merely driven by a set of users who are intrinsically more interested in an event. Second, there are no viable groups for comparison to a keyword-based sample of Twitter users. We propose an alternative sampling approachgeolocated panels defined by users geolocation to studying response to events on Twitter. Geolocated panels are exogenous to the keywords in users tweets, resulting in less selection bias than the keyword-based approach. Geolocated panels allow us to follow within-person changes over time and enable the creation of comparison groups. We evaluate our panel selection approach in two real-world settings: response to mass shootings and response to TV advertising. We first empirically show that geolocated panels are subject to selection biases, while geolocated panels reduce selection biases. Then we show how geolocated panels can provide qualitatively different results. We believe that we are the first to provide a clear empirical example of how a better panel-selection design, based on an exogenous variable such as geography, both reduces selection bias compared to the current state-of-the-art and increases the value of Twitter research for studying events.

Experience: Open Fiscal Datasets, Common Issues, and Recommendations

Public administrations are increasingly publishing Open data to make their governance more transparent. These publicly available Open data include fiscal data, e.g., budget and spending data. The publication of Open fiscal datasets is an important part of transparent and accountable governance. Another critical part of governance transparency and accountability is that published datasets should meet open data publication guidelines. When requirements in data guidelines are not met, effective data analysis over published datasets cannot be done. In this paper, we present an extensive assessment in published real-world Open fiscal data, common data quality issues, and guidelines for publishing Open fiscal data. The reported work has been done by studying works related to Open fiscal data publication, as well as collecting important factors that should be present in Open fiscal datasets. Moreover, collected factors have been scored according to the results of a survey. As a result, we have come up with an Open Fiscal Data Publication (OFDP) framework to assess the quality of Open fiscal datasets which is also described in the paper. . We gather and comprehensively analyze a representative set of more than 75 fiscal datasets from several public administrations across different regions at different levels (e.g., supranational, national, municipality). We characterize quality issues commonly arising in these datasets. Our evaluation shows that there are many factors in fiscal data publication that still need to be taken care of so that the data can be effectively analyzed. In the end, we provide a set of specific guidelines for publishing open fiscal data.

Visual Interactive Creation, Customization, and Analysis of Data Quality Metrics

During data pre-processing, analysts spend a significant part of their time and effort profiling the quality of the data along with cleansing and transforming the data for further analysis. While quality metricsranging from general to domain-specific measuresfacilitate assessment of the quality of a dataset, there are hardly any approaches to visually support the analyst in customizing and applying such metrics. Yet, visual approaches could facilitate user involvement for data quality assessment. We present MetricDoc, an interactive environment for assessing data quality that provides customizable, reusable quality metrics in combination with immediate visual feedback. Moreover, we provide an overview visualization of these quality metrics along with error visualizations that facilitate interactive navigation of the data to determine the causes of quality issues present in the data. In this paper we describe the architecture, design, and evaluation of MetricDoc which underwent several design cycles, including heuristic evaluation and expert reviews as well as a focus group with data quality, human-computer interaction, and visual analytics experts.

Information Quality Awareness and Information Quality Practice

Healthcare organizations increasingly rely on electronic information to optimize their operations. Information of high diversity from various sources accentuate the relevance and importance of information quality (IQ). The quality of information needs to be improved to support a more efficient and reliable utilization of healthcare information systems (IS). This can only be achieved through the implementation of initiatives followed by most users across an organization. The purpose of this study is to examine how awareness of IS users about IQ issues would affect their actual practices toward IQ initiatives. Influenced by the awareness on beneficial and problematic situations generated by IQ practices, users motivation is found to influence their IQ-related behavior. In addition, social influences and facilitating conditions moderate the relationship between user intention and actual practice. The theoretical and practical implications of findings are discussed, especially IQ best practices in the healthcare settings.

SPMDL: Software Product Metrics Definition Language

Software metrics are becoming more acceptable measures for software quality assessment. However, there is no standard form for representing metric definitions, which would be useful for metrics exchange and customization. In this paper, we propose the Software Product Metrics Definition Language (SPMDL). We developed an XML-based description language, for defining software metrics in a precise and reusable form. Metric definitions in SPMDL are based on meta-models extracted from either source code or design artifacts, such as the Dagstuhl Middle Meta-model, with support for various abstraction levels. The language defines several flexible computation mechanisms such as extended OCL queries and predefined graph operations on the meta-model. SPMDL provides unambiguous description of the metric definition; it is also easy to use and extensible.


Publication Years 2009-2018
Publication Count 140
Citation Count 261
Available for Download 140
Downloads (6 weeks) 1359
Downloads (12 Months) 11891
Downloads (cumulative) 86408
Average downloads per article 617
Average citations per article 2
First Name Last Name Award
Peter Aiken ACM Senior Member (2011)
Mikhail Atallah ACM Fellows (2006)
Ahmed Elmagarmid ACM Fellows (2012)
ACM Distinguished Member (2009)
Wenfei Fan ACM Fellows (2012)
Matthias Jarke ACM Fellows (2013)
Daniel S Katz ACM Senior Member (2011)
Beth A. Plale ACM Senior Member (2006)
Clifford A Shaffer ACM Distinguished Member (2015)
ACM Senior Member (2007)
Clifford A Shaffer ACM Distinguished Member (2015)
ACM Senior Member (2007)

First Name Last Name Paper Counts
Yang Lee 4
Stuart Madnick 3
John Talburt 3
Nan Tang 3
G Shankaranarayanan 3
Roman Lukyanenko 3
Peter Edwards 3
Peter Christen 3
Mathias Klier 2
Kewei Sha 2
Felix Naumann 2
Carolyn Matheus 2
Xiaobai Li 2
Ali Sunyaev 2
Vassilios Verykios 2
Wenfei Fan 2
Roger Blake 2
Monica Tremblay 2
Bernd Heinrich 2
Daisyzhe Wang 2
Eitel LauríA 2
Dinusha Vatsalan 2
Ross Gayler 2
Christan Grant 2
Arnon Rosenthal 2
Sherali Zeadally 2
Wolfgang Lehner 2
Mario Mezzanzanica 1
Roberto Boselli 1
Sandra Geisler 1
Luvai Motiwalla 1
Daniel Katz 1
Douglas Hodson 1
Dov Biran 1
Edward Anderson 1
Aseel Basheer 1
Ralf Tönjes 1
Pierpaolo Vittorini 1
Karthikeyan Ramamurthy 1
Laurent Lecornu 1
Shelly Sachdeva 1
Stuart Madnick 1
Debra VanderMeer 1
Foster Provost 1
Nicola Ferro 1
Christian Becker 1
Chintan Amrit 1
Hossameldin Shahin 1
Christoph Lange 1
Sören Auer 1
Sharad Mehrotra 1
Sandra Sampaio 1
Therese Williams 1
Dustin Lange 1
Chris Baillie 1
Leopoldo Bertossi 1
Banda Ramadan 1
Jianyong Wang 1
Beth Plale 1
John Krogstie 1
John O’Donoghue 1
Wenjun Li 1
Davide Ceolin 1
Khoi Tran 1
Lan Cao 1
Diego Marcheggiani 1
Nour El Mawass 1
Payam Barnaghi 1
Jean Caillec 1
Arputharaj Kannan 1
Anupkumar Sen 1
Rashid Ansari 1
Fahima Nader 1
Shuai Ma 1
Philip Woodall 1
Nigel Martin 1
Axel Polleres 1
Venkata Meduri 1
Suzanne Embury 1
Hubert Österle 1
Erhard Rahm 1
Jeffrey Vaughan 1
Huizhi Liang 1
Paolo Coletti 1
Michalis Mountantonakis 1
Jens Lehmann 1
Lizhu Zhou 1
Melanie Herschel 1
Mirko Cesarini 1
Hongjiang Xu 1
Vincenzo Maltese 1
Xiaoping Liu 1
Fred Morstatter 1
Valentina Maccatrozzo 1
Paul Groth 1
A Borthick 1
Mohamed Yakout 1
Fabrizio Sebastiani 1
Peter Arbuckle 1
Rahul Basole 1
Jimeng Sun 1
Sara Tonelli 1
Kush Varshney 1
Dmitry Chornyi 1
Danilo Montesi 1
Omar Alonso 1
Ashfaq Khokhar 1
Alan Labouseur 1
C Fratto 1
Alexandra Poulovassilis 1
Honglinh Truong 1
Yuheng Hu 1
Yi Chen 1
Robert Meusel 1
Maurice Van Keulen 1
Stephen Chong 1
Aniketh Reddy 1
Irit Askira Gelman 1
Edoardo Pignotti 1
Eric Medvet 1
Fabiano Tarlao 1
John Herbert 1
Juan Augusto 1
Maurice Mulvenna 1
Paul Mccullagh 1
Fabio Mercorio 1
Laure Berti-Équille 1
Fei Chiang 1
Siddharth Sitaramachandran 1
J Jha 1
Sven Weber 1
Richard Briotta 1
Saad Alaboodi 1
Johann Freytag 1
María Bermúdez-Edo 1
Maria Alvarez 1
Panagiotis Ipeirotis 1
Justin St-Maurice 1
Milan Markovic 1
Wenyuan Yu 1
Cinzia Cappiello 1
Diana Hristova 1
Alexander Schiller 1
Clifford Shaffer 1
Jürgen Umbrich 1
Fabian Panse 1
Fumiko Kobayashi 1
Paolo Missier 1
Kristin Weber 1
Paul Glowalla 1
Wenyuan Yu 1
Yang Lei 1
Benjamin Ngugi 1
Beverly Kahn 1
Xu Pu 1
Fausto Giunchiglia 1
Christoph Quix 1
Matthias Jarke 1
Wan Fokkink 1
Jeffrey Fisher 1
Jeremy Millar 1
Adriane Chapman 1
Hilko Donker 1
Heiko Müller 1
Terry Clark 1
H Nehemiah 1
Steven Brown 1
Matthew Jensen 1
Jay Nunamaker, 1
Rachid Chalal 1
Adir Even 1
Fons Wijnhoven 1
Jeremy Debattista 1
Sushovan De 1
Dominique Ritze 1
Heiko Paulheim 1
Dezhao Song 1
Rabia Nuray-Turan 1
Dmitri Kalashnikov 1
Yinle Zhou 1
Daniel Dalip 1
Pável Calado 1
Tobias Vogel 1
Arvid Heise 1
Uwe Draisbach 1
Justin Zobel 1
Mostafa Milani 1
Youwei Cheah 1
Olivier Curé 1
Claire Collins 1
Ioannis Anagnostopoulos 1
Patricia Franklin 1
Huan Liu 1
Willem Van Hage 1
Gilbert Peterson 1
Martin Hahmann 1
Peter Aiken 1
Len Seligman 1
Robert Ulbricht 1
Michael Zack 1
Nitin Joglekar 1
Hongwei Zhu 1
Mikhail Atallah 1
Yanjuan Yang 1
Paul Bowen 1
Ulf Leser 1
Irit Gelman 1
Min Chen 1
Dennis Wei 1
Aleksandra Mojsilović 1
Ion Todoran 1
Ali Khenchaf 1
D Elizabeth 1
Trent Rosenbloom 1
Shawn Hardenbrook 1
Subhash Bhalla 1
Kaushik Dutta 1
Jeffrey Parsons 1
Valerie Sessions 1
Kresimir Duretec 1
Leena Al-Hussaini 1
Pim Dietz 1
Barbara Pernici 1
Michael Szubartowicz 1
Aitor Murguzur 1
Kyuhan Koh 1
Eric Fouh 1
Eric Nelson 1
Manoranjan Dash 1
M Kaiser 1
Thomas Redman 1
David Becker 1
Floris Geerts 1
Xiuzhen Zhang 1
Diego Esteves 1
Xiaoming Fan 1
Giannis Haralabopoulos 1
Kyle Niemeyer 1
Arfon Smith 1
Archana Nottamkandath 1
Darryl Ahner 1
Claudio Hartmann 1
Hongwei Zhu 1
Cihan Varol 1
Coşkun Bayrak 1
David Robb 1
Ezra Kahn 1
Adam Kriesberg 1
Mark Braunstein 1
Rosella Gennari 1
Marta Zárraga-Rodríguez 1
Peter Elkin 1
C Raj 1
Amitava Bagchi 1
Hema Meda 1
Matteo Magnani 1
Craig Fisher 1
Sufyan Ababneh 1
Jiannan Wang 1
C Cerletti 1
Erica Yang 1
Jianing Wang 1
Sebastian Neumaier 1
Norbert Ritter 1
R Greenwood 1
Ayush Singhania 1
George Moustakides 1
Marcos Gonçalves 1
Hongwei Zhu 1
Dirk Ahlers 1
Yu Wan 1
Bing Lv 1
Paul Mangiameli 1
Alberto Bartoli 1
James McNaull 1
Kelly Janssens 1
Mouhamadoulamine Ba 1
Ciro D'Urso 1
Judith Gelernter 1
Hua Zheng 1
Ahmed Elmagarmid 1
Michael Mannino 1
Fiona Rohde 1
David Corsar 1
Elliot Fielstein 1
Theodore Speroff 1
Yang Lee 1
Judee Burgoon 1
Josh Attenberg 1
Sean Goldberg 1
Marco Valtorta 1
Andreas Rauber 1
Sabrina Abdellaoui 1
Catherine Burns 1
Mohammed Farghally 1
Subbarao Kambhampati 1
Jeff Heflin 1
Alun Preece 1
Anja Klein 1
Boris Otto 1
Alan March 1
Marco Cristo 1
Richard Wang 1
Christian Skalka 1
Maurizio Murgia 1
Yannis Tzitzikas 1
Qingyu Chen 1
Karin Verspoor 1
Anisa Rula 1
Marilyn Tremaine 1
Andrea Lorenzo 1

Affiliation Paper Counts
Birla Institute of Technology and Science Pilani 1
University of Padua 1
Assiut University 1
University of Illinois at Urbana-Champaign 1
Federal University of Amazonas 1
Florida State University 1
Virginia Commonwealth University 1
University of Amsterdam 1
Vanderbilt University 1
Instituto Superior Tecnico 1
University of Houston 1
Google Inc. 1
University of Leipzig 1
Hospital Universitario Austral 1
Harvard University 1
University of Colorado at Denver 1
University of Ulm 1
Oklahoma City University 1
University of Rhode Island 1
RMIT University 1
State University of New York at Albany 1
Georgia State University 1
University of Antwerp 1
University of Texas at Austin 1
Oregon State University 1
Beihang University 1
University of Massachusetts System 1
Indian Institute of Science, Bangalore 1
University of Saskatchewan 1
Elsevier 1
University of Augsburg 1
The College of William and Mary 1
Carleton University 1
University of South Carolina 1
Simon Fraser University 1
Memorial University of Newfoundland 1
Boston University 1
Technical University of Munich 1
Butler University 1
University of Maryland 1
Italian National Research Council 1
New Jersey Institute of Technology 1
National Institute of Standards and Technology 1
Rutherford Appleton Laboratory 1
Cardiff University 1
Sam Houston State University 1
University College Cork 1
Ben-Gurion University of the Negev 1
Charleston Southern University 1
Commonwealth Scientific and Industrial Research Organization 1
Rutgers, The State University of New Jersey 1
University of Cambridge 1
University of Patras 1
California State University 1
Hellenic Open University 1
University of Baghdad 1
Universite Paris-Est 1
Facebook, Inc. 1
USDA ARS Beltsville Agricultural Research Center 2
Humboldt University of Berlin 2
University of Crete 2
Fraunhofer Institute for Applied Information Technology 2
Nanyang Technological University 2
Old Dominion University 2
Suffolk University 2
Free University of Bozen-Bolzano 2
University of Innsbruck 2
University of Arizona 2
Norwegian University of Science and Technology 2
King Saud University 2
University of Waterloo 2
University of Kentucky 2
University of Trento 2
RWTH Aachen University 2
University of Toronto 2
Vienna University of Technology 2
University of Surrey 2
Indiana University 2
New York University 2
Massachusetts Institute of Technology 2
University of Massachusetts Boston 2
Microsoft Corporation 2
Virginia Tech 2
University of Bologna 2
University of Hamburg 2
Federal University of Minas Gerais 2
University of Oklahoma 2
University of Queensland 2
University of Aizu 2
Universidad de Navarra 2
Indian Institute of Management Calcutta 2
Lehigh University 3
Vienna University of Economics and Business Administration 3
University of Massachusetts Medical School 3
University of Mannheim 3
University of California, Irvine 3
Birkbeck University of London 3
Purdue University 3
Telecom Bretagne 3
Georgia Institute of Technology 3
University of Cologne 3
Babson College 3
University of Thessaly 3
University of St. Gallen 3
Northeastern University 3
McMaster University 3
Ecole nationale superieure d'Informatique 3
University of Manchester 4
Qatar Computing Research institute 4
University of Regensburg 4
University of Illinois at Chicago 4
University of Edinburgh 4
United States Department of Veterans Affairs 4
Politecnico di Milano 4
Anna University 4
University of Ulster 4
University of Twente 4
United States Air Force Institute of Technology 4
University of Trieste 4
Vrije Universiteit Amsterdam 4
University of Milan - Bicocca 4
University of Florida 4
IBM Thomas J. Watson Research Center 4
Technical University of Dresden 4
University of Massachusetts Lowell 5
Marist College 5
Arizona State University 5
Tsinghua University 5
MITRE Corporation 5
University of Melbourne 5
University of Bonn 6
Hasso-Plattner-Institut fur Softwaresystemtechnik GmbH 6
Florida International University 6
University of Aberdeen 7
University of Arkansas at Little Rock 8
Australian National University 9

Journal of Data and Information Quality (JDIQ) - Special Issue on Improving the Veracity and Value of Big Data

Volume 9 Issue 3, March 2018 Special Issue on Improving the Veracity and Value of Big Data
Volume 9 Issue 2, January 2018 Challenge Paper, Experience Paper and Research Paper

Volume 9 Issue 1, October 2017 Research Papers and Challenge Papers
Volume 8 Issue 3-4, July 2017 Challenge Papers, Experience Paper and Research Papers
Volume 8 Issue 2, February 2017 Challenge Papers and Research Papers

Volume 8 Issue 1, November 2016 Special Issue on Web Data Quality
Volume 7 Issue 4, October 2016 Challenge Papers and Regular Papers
Volume 7 Issue 3, September 2016 Research Paper, Challenge Papers and Experience Paper
Volume 7 Issue 1-2, June 2016 Challenge Papers, Regular Papers and Experience Paper

Volume 6 Issue 4, October 2015 Challenge Papers and Regular Papers
Volume 6 Issue 2-3, July 2015
Volume 6 Issue 1, March 2015
Volume 5 Issue 4, February 2015
Volume 5 Issue 3, February 2015 Special Issue on Provenance, Data and Information Quality

Volume 5 Issue 1-2, August 2014
Volume 4 Issue 4, May 2014

Volume 4 Issue 3, May 2013
Volume 4 Issue 2, March 2013 Special Issue on Entity Resolution

Volume 4 Issue 1, October 2012
Volume 3 Issue 4, September 2012
Volume 3 Issue 3, August 2012
Volume 3 Issue 2, May 2012
Volume 3 Issue 1, April 2012
Volume 2 Issue 4, February 2012

Volume 2 Issue 3, December 2011
Volume 2 Issue 2, February 2011

Volume 2 Issue 1, July 2010

Volume 1 Issue 3, December 2009
Volume 1 Issue 2, September 2009
Volume 1 Issue 1, June 2009
All ACM Journals | See Full Journal Index

Search JDIQ
enter search term and/or author name