A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus

Ben Miller; Jennifer Olive; Shakthidhar Gopavaram; Yanjun Zhao; Ayush Shrestha; Cynthia Berger

doi:10.1109/BigData.2015.7363938

A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus

Ben Miller, Jennifer Olive, Shakthidhar Gopavaram, Yanjun Zhao, Ayush Shrestha, Cynthia Berger

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Identifying similar narrative sections across longer documents would help identify key events within a corpus, enrich understanding of those events, provide a mechanism for organizing corpora according to their event content, and allow for bottom-up testing of theories of narrative. This paper proposes an automated method for narrative alignment across large textual corpora using techniques from natural language processing and similarity-based image segmentation. This method proceeds by segmenting each document into a series of events, constructs sequences of abstracted representations of those events, compares pairs of sequences to generate image matrices, segments the images, identifies similar segments to discover commonly occurring narrative units, and, finally, returns the source sentences to make the clusters of narrative similarity readable. Preliminary tests of elements of this method were conducted on a small heterogeneous corpus (< 100 documents) and a moderate heterogeneous corpus (10k documents). Further implementation as described in this position paper is necessary to scale to the full 251k document corpus from which the moderate corpus was drawn.

Original language	English
Title of host publication	Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015
Editors	Feng Luo, Kemafor Ogan, Mohammed J. Zaki, Laura Haas, Beng Chin Ooi, Vipin Kumar, Sudarsan Rachuri, Saumyadipta Pyne, Howard Ho, Xiaohua Hu, Shipeng Yu, Morris Hui-I Hsiao, Jian Li
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1673-1677
Number of pages	5
ISBN (Electronic)	9781479999255
DOIs	https://doi.org/10.1109/BigData.2015.7363938
State	Published - 22 Dec 2015
Externally published	Yes
Event	3rd IEEE International Conference on Big Data, IEEE Big Data 2015 - Santa Clara, United States Duration: 29 Oct 2015 → 1 Nov 2015

Publication series

Name	Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

Conference

Conference	3rd IEEE International Conference on Big Data, IEEE Big Data 2015
Country/Territory	United States
City	Santa Clara
Period	29/10/15 → 1/11/15

Bibliographical note

Publisher Copyright:
© 2015 IEEE.

Keywords

Computational models of narrative
big data
computational linguistics
text mining

ASJC Scopus subject areas

Computer Networks and Communications
Computer Science Applications
Information Systems
Software

Access to Document

10.1109/BigData.2015.7363938

Cite this

Miller, B., Olive, J., Gopavaram, S., Zhao, Y., Shrestha, A., & Berger, C. (2015). A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus. In F. Luo, K. Ogan, M. J. Zaki, L. Haas, B. C. Ooi, V. Kumar, S. Rachuri, S. Pyne, H. Ho, X. Hu, S. Yu, M. H-I. Hsiao, & J. Li (Eds.), Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015 (pp. 1673-1677). [7363938] (Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2015.7363938

Miller, Ben ; Olive, Jennifer ; Gopavaram, Shakthidhar et al. / A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus. Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. editor / Feng Luo ; Kemafor Ogan ; Mohammed J. Zaki ; Laura Haas ; Beng Chin Ooi ; Vipin Kumar ; Sudarsan Rachuri ; Saumyadipta Pyne ; Howard Ho ; Xiaohua Hu ; Shipeng Yu ; Morris Hui-I Hsiao ; Jian Li. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 1673-1677 (Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015).

@inproceedings{5d2322a4017e49d68991c2d44dbd4adc,

title = "A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus",

abstract = "Identifying similar narrative sections across longer documents would help identify key events within a corpus, enrich understanding of those events, provide a mechanism for organizing corpora according to their event content, and allow for bottom-up testing of theories of narrative. This paper proposes an automated method for narrative alignment across large textual corpora using techniques from natural language processing and similarity-based image segmentation. This method proceeds by segmenting each document into a series of events, constructs sequences of abstracted representations of those events, compares pairs of sequences to generate image matrices, segments the images, identifies similar segments to discover commonly occurring narrative units, and, finally, returns the source sentences to make the clusters of narrative similarity readable. Preliminary tests of elements of this method were conducted on a small heterogeneous corpus (< 100 documents) and a moderate heterogeneous corpus (10k documents). Further implementation as described in this position paper is necessary to scale to the full 251k document corpus from which the moderate corpus was drawn.",

keywords = "Computational models of narrative, big data, computational linguistics, text mining",

author = "Ben Miller and Jennifer Olive and Shakthidhar Gopavaram and Yanjun Zhao and Ayush Shrestha and Cynthia Berger",

note = "Publisher Copyright: {\textcopyright} 2015 IEEE.; 3rd IEEE International Conference on Big Data, IEEE Big Data 2015 ; Conference date: 29-10-2015 Through 01-11-2015",

year = "2015",

month = dec,

day = "22",

doi = "10.1109/BigData.2015.7363938",

language = "English",

series = "Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1673--1677",

editor = "Feng Luo and Kemafor Ogan and Zaki, {Mohammed J.} and Laura Haas and Ooi, {Beng Chin} and Vipin Kumar and Sudarsan Rachuri and Saumyadipta Pyne and Howard Ho and Xiaohua Hu and Shipeng Yu and Hsiao, {Morris Hui-I} and Jian Li",

booktitle = "Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015",

address = "United States",

}

Miller, B, Olive, J, Gopavaram, S, Zhao, Y, Shrestha, A & Berger, C 2015, A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus. in F Luo, K Ogan, MJ Zaki, L Haas, BC Ooi, V Kumar, S Rachuri, S Pyne, H Ho, X Hu, S Yu, MH-I Hsiao & J Li (eds), Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015., 7363938, Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015, Institute of Electrical and Electronics Engineers Inc., pp. 1673-1677, 3rd IEEE International Conference on Big Data, IEEE Big Data 2015, Santa Clara, United States, 29/10/15. https://doi.org/10.1109/BigData.2015.7363938

A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus. / Miller, Ben; Olive, Jennifer; Gopavaram, Shakthidhar et al.

Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. ed. / Feng Luo; Kemafor Ogan; Mohammed J. Zaki; Laura Haas; Beng Chin Ooi; Vipin Kumar; Sudarsan Rachuri; Saumyadipta Pyne; Howard Ho; Xiaohua Hu; Shipeng Yu; Morris Hui-I Hsiao; Jian Li. Institute of Electrical and Electronics Engineers Inc., 2015. p. 1673-1677 7363938 (Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus

AU - Miller, Ben

AU - Olive, Jennifer

AU - Gopavaram, Shakthidhar

AU - Zhao, Yanjun

AU - Shrestha, Ayush

AU - Berger, Cynthia

PY - 2015/12/22

Y1 - 2015/12/22

N2 - Identifying similar narrative sections across longer documents would help identify key events within a corpus, enrich understanding of those events, provide a mechanism for organizing corpora according to their event content, and allow for bottom-up testing of theories of narrative. This paper proposes an automated method for narrative alignment across large textual corpora using techniques from natural language processing and similarity-based image segmentation. This method proceeds by segmenting each document into a series of events, constructs sequences of abstracted representations of those events, compares pairs of sequences to generate image matrices, segments the images, identifies similar segments to discover commonly occurring narrative units, and, finally, returns the source sentences to make the clusters of narrative similarity readable. Preliminary tests of elements of this method were conducted on a small heterogeneous corpus (< 100 documents) and a moderate heterogeneous corpus (10k documents). Further implementation as described in this position paper is necessary to scale to the full 251k document corpus from which the moderate corpus was drawn.

AB - Identifying similar narrative sections across longer documents would help identify key events within a corpus, enrich understanding of those events, provide a mechanism for organizing corpora according to their event content, and allow for bottom-up testing of theories of narrative. This paper proposes an automated method for narrative alignment across large textual corpora using techniques from natural language processing and similarity-based image segmentation. This method proceeds by segmenting each document into a series of events, constructs sequences of abstracted representations of those events, compares pairs of sequences to generate image matrices, segments the images, identifies similar segments to discover commonly occurring narrative units, and, finally, returns the source sentences to make the clusters of narrative similarity readable. Preliminary tests of elements of this method were conducted on a small heterogeneous corpus (< 100 documents) and a moderate heterogeneous corpus (10k documents). Further implementation as described in this position paper is necessary to scale to the full 251k document corpus from which the moderate corpus was drawn.

KW - Computational models of narrative

KW - big data

KW - computational linguistics

KW - text mining

UR - http://www.scopus.com/inward/record.url?scp=84963764679&partnerID=8YFLogxK

U2 - 10.1109/BigData.2015.7363938

DO - 10.1109/BigData.2015.7363938

M3 - Conference contribution

AN - SCOPUS:84963764679

T3 - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

SP - 1673

EP - 1677

BT - Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015

A2 - Luo, Feng

A2 - Ogan, Kemafor

A2 - Zaki, Mohammed J.

A2 - Haas, Laura

A2 - Ooi, Beng Chin

A2 - Kumar, Vipin

A2 - Rachuri, Sudarsan

A2 - Pyne, Saumyadipta

A2 - Ho, Howard

A2 - Hu, Xiaohua

A2 - Yu, Shipeng

A2 - Hsiao, Morris Hui-I

A2 - Li, Jian

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 3rd IEEE International Conference on Big Data, IEEE Big Data 2015

Y2 - 29 October 2015 through 1 November 2015

ER -

Miller B, Olive J, Gopavaram S, Zhao Y, Shrestha A, Berger C. A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus. In Luo F, Ogan K, Zaki MJ, Haas L, Ooi BC, Kumar V, Rachuri S, Pyne S, Ho H, Hu X, Yu S, Hsiao MH-I, Li J, editors, Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 1673-1677. 7363938. (Proceedings - 2015 IEEE International Conference on Big Data, IEEE Big Data 2015). https://doi.org/10.1109/BigData.2015.7363938

A method for cross-document narrative alignment of a two-hundred-sixty-million word corpus

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this