Feasibility of using attention mechanism in abstractive summarization

AlMazrouei, RZ ORCID: https://orcid.org/0000-0003-2846-2322, Nelci, J ORCID: https://orcid.org/0000-0001-5467-0593, Salloum, S ORCID: https://orcid.org/0000-0002-6073-3981 and Shaalan, K ORCID: https://orcid.org/0000-0003-0823-8390 2022, Feasibility of using attention mechanism in abstractive summarization , in: International Conference on Emerging Technologies and Intelligent Systems (ICETIS), 25th-26th June 2021, Al Buraimi, Oman.

Full text not available from this repository. (Request a copy)


The Prevalence of information and its magnitude mandates a short description of the core of a document, an article, or legal documents. Abstractive summarization helps to concur with this problem utilizing the evolutions in machine learning and deep neural network. Attention-mechanism has extensively applied in the challenging issue of abstraction a text, in shorter length yet informative. We noticed in [13] after removing the attention layer from their proposed model, the performance only experience soft drawback, even can be ignored. Thus, motivates us to survey the latest models using attention-mechanism and its achievements, and the second objective is to run an experiment to compare standard stacked 3- Long Short-Term Memory (LSTM) layers incorporated with attention layer only (without any other hand-crafted algorithm) to explore how efficient this technique can generate better summarization, then a stand-alone model. The standard proposed model incorporated with attention-mechanism suffered from drawback performance and scored less than a stand-alone model by at least 6 point scores on ROUGE-1&2.

Item Type: Conference or Workshop Item (Paper)
Schools: Schools > School of Computing, Science and Engineering
Journal or Publication Title: Lecture Notes in Networks and Systems
Publisher: Springer
Series Name: Lecture Notes in Networks and Systems
ISBN: 9783030826154 (paperback); 9783030826161 (ebook)
ISSN: 2367-3370
Related URLs:
Depositing User: USIR Admin
Date Deposited: 30 Nov 2021 10:14
Last Modified: 15 Feb 2022 14:47
URI: http://usir.salford.ac.uk/id/eprint/62445

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)