LearnPADS ++: Incremental inference of ad hoc data formats

Kenny Q. Zhu; Kathleen Fisher; David Walker

doi:10.1007/978-3-642-27694-1_13

LearnPADS ⁺⁺: Incremental inference of ad hoc data formats

Kenny Q. Zhu, Kathleen Fisher, David Walker

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

6 Scopus citations

Abstract

An ad hoc data source is any semi-structured, non-standard data source. The format of such data sources is often evolving and frequently lacking documentation. Consequently, off-the-shelf tools for processing such data often do not exist, forcing analysts to develop their own tools, a costly and time-consuming process. In this paper, we present an incremental algorithm that automatically infers the format of large-scale data sources. From the resulting format descriptions, we can generate a suite of data processing tools automatically. The system can handle large-scale or streaming data sources whose formats evolve over time. Furthermore, it allows analysts to modify inferred descriptions as desired and incorporates those changes in future revisions.

Original language	English (US)
Title of host publication	Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings
Pages	168-182
Number of pages	15
DOIs	https://doi.org/10.1007/978-3-642-27694-1_13
State	Published - 2012
Event	14th International Symposium on Practical Aspects of Declarative Languages, PADL 2012 - Philadelphia, PA, United States Duration: Jan 23 2012 → Jan 24 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	7149 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	14th International Symposium on Practical Aspects of Declarative Languages, PADL 2012
Country/Territory	United States
City	Philadelphia, PA
Period	1/23/12 → 1/24/12

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-642-27694-1_13

Cite this

Zhu, K. Q., Fisher, K., & Walker, D. (2012). LearnPADS ⁺⁺: Incremental inference of ad hoc data formats. In Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings (pp. 168-182). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7149 LNCS). https://doi.org/10.1007/978-3-642-27694-1_13

@inproceedings{4cb4bc0f8572479391c396a77e32f347,

title = "LearnPADS ++: Incremental inference of ad hoc data formats",

abstract = "An ad hoc data source is any semi-structured, non-standard data source. The format of such data sources is often evolving and frequently lacking documentation. Consequently, off-the-shelf tools for processing such data often do not exist, forcing analysts to develop their own tools, a costly and time-consuming process. In this paper, we present an incremental algorithm that automatically infers the format of large-scale data sources. From the resulting format descriptions, we can generate a suite of data processing tools automatically. The system can handle large-scale or streaming data sources whose formats evolve over time. Furthermore, it allows analysts to modify inferred descriptions as desired and incorporates those changes in future revisions.",

author = "Zhu, {Kenny Q.} and Kathleen Fisher and David Walker",

year = "2012",

doi = "10.1007/978-3-642-27694-1_13",

language = "English (US)",

isbn = "9783642276934",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "168--182",

booktitle = "Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings",

note = "14th International Symposium on Practical Aspects of Declarative Languages, PADL 2012 ; Conference date: 23-01-2012 Through 24-01-2012",

}

Zhu, KQ, Fisher, K & Walker, D 2012, LearnPADS ⁺⁺: Incremental inference of ad hoc data formats. in Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7149 LNCS, pp. 168-182, 14th International Symposium on Practical Aspects of Declarative Languages, PADL 2012, Philadelphia, PA, United States, 1/23/12. https://doi.org/10.1007/978-3-642-27694-1_13

LearnPADS ⁺⁺: Incremental inference of ad hoc data formats. / Zhu, Kenny Q.; Fisher, Kathleen; Walker, David.
Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings. 2012. p. 168-182 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7149 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - LearnPADS ++

T2 - 14th International Symposium on Practical Aspects of Declarative Languages, PADL 2012

AU - Zhu, Kenny Q.

AU - Fisher, Kathleen

AU - Walker, David

PY - 2012

Y1 - 2012

N2 - An ad hoc data source is any semi-structured, non-standard data source. The format of such data sources is often evolving and frequently lacking documentation. Consequently, off-the-shelf tools for processing such data often do not exist, forcing analysts to develop their own tools, a costly and time-consuming process. In this paper, we present an incremental algorithm that automatically infers the format of large-scale data sources. From the resulting format descriptions, we can generate a suite of data processing tools automatically. The system can handle large-scale or streaming data sources whose formats evolve over time. Furthermore, it allows analysts to modify inferred descriptions as desired and incorporates those changes in future revisions.

AB - An ad hoc data source is any semi-structured, non-standard data source. The format of such data sources is often evolving and frequently lacking documentation. Consequently, off-the-shelf tools for processing such data often do not exist, forcing analysts to develop their own tools, a costly and time-consuming process. In this paper, we present an incremental algorithm that automatically infers the format of large-scale data sources. From the resulting format descriptions, we can generate a suite of data processing tools automatically. The system can handle large-scale or streaming data sources whose formats evolve over time. Furthermore, it allows analysts to modify inferred descriptions as desired and incorporates those changes in future revisions.

UR - http://www.scopus.com/inward/record.url?scp=84857076712&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84857076712&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-27694-1_13

DO - 10.1007/978-3-642-27694-1_13

M3 - Conference contribution

AN - SCOPUS:84857076712

SN - 9783642276934

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 168

EP - 182

BT - Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings

Y2 - 23 January 2012 through 24 January 2012

ER -

Zhu KQ, Fisher K, Walker D. LearnPADS ⁺⁺: Incremental inference of ad hoc data formats. In Practical Aspects of Declarative Languages - 14th International Symposium, PADL 2012, Proceedings. 2012. p. 168-182. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-27694-1_13