Performance Analysis of Modified Shuffled Frog Leaping Algorithm for Multi-document Summarization Problem

Due to massive growth of Web information, handling useful information has become a challenging issue in now-a-days. In the past few decades, text summarization is considered as one of the solution to obtained relevant information from extensive collection of information. In this paper, a novel approach using modified shuffled frog leaping algorithm (MSFLA) to extract the important sentence from multiple documents is presented. The effectiveness of MSFLA algorithm for summarization model is evaluated by comparing the ROUGE score and statistical analysis of the model with respect to results of other summarization models. The models are demonstrated by the simulation results over DUC datasets. In the present work, it elucidates that MSFLA based model improves the results and find advisable solution for summary extraction.


Introduction
Present days, growing of information exponentially in Web initiates information overload problem. As a result automatic text summarization (TS) has increasing value to various real-world applications. TS summarize information from single or multiple documents which share an explicit or implicit main topic. It facilitates users to quickly catch the most relevant and important information through large text data collections. Shortening of a large text document or documents into a concise form is called as single or multi document summarization respectively.
As multi-document summarization processes multiple documents, therefore search space of multi-document summarization is large and considered as an enlargement of single document summarization. Hence it makes more challenging for selection of important sentences. In that context, summarization of multiple documents can be recognized as an optimization problem with the objective of producing optimal summary containing relevant and informative sentences of the original input documents [1].
As a challenging issue for text mining, automatic document summarization had been well-studied during half a century years [2]. The great majority of numerous approaches developed are extraction-based, which produce a summary using only existing sentences (or text fragments) extracted from the original text, so they are conceptually simple and more practicable compared to abstractive methods, which attempt to reproduce sentence by using complicated natural language generation techniques such as sentence compression [3] information fusion [4] and reformulation [5].
In literature, many evolutionary computation based contributions have been suggested for sentence selection from huge collection of information. The domain is already rich with the proposal of many evolutionary based summarization models development such as (GA),differentia evolution(DE), particle swarm optimization(PSO), harmony search (HS), cat swarm optimization(CSO), cuckoo search (CS) etc. In few cases these techniques suffers from premature convergence and gets trapped into its local optima. The focuses have to be exploitation and exploration abilities of these evolutionary algorithms.
Further to improve the performance motivations have to be made on population diversity in the progressive procedure and a sophisticated approach for information sharing among each participant in the distribution. To overcome these issues an evolutionary approach called Shuffled frog leaping algorithm (SFLA) is proposed.
There are many applications in which shuffle frog leaping algorithm is applied. A detail description is highlighted as follows.
[6] applied opposition based leaning to improve the global search of SFL algorithm. It not only improves the local search but also improves its diversity. This technique is experimented on 10 benchmark optimization function, 10 shifted function (from CEC2005) and on cost management problem in cellular network.
Dalavi et.al [7] proposed a modified SFL algorithm for hole making operation in plate of ejector mould. Many kinds of machining operations are required for hole making operation. Implementing this technique the optimal sequence of operation is identified minimizing cost for hole making operation.
An improved SFL algorithm (ISFL) is proposed by Dash et.al. [8] for currency rate prediction problem. In this technique the author has tried to improve the local and global search process. This technique introduces a new frog leaping rule with the acceleration factor and leaping inertia components. The experimental result is compared with simple SFL algorithm and based PSO and the result shows the superiority of ISFL algorithm in terms of both convergence rate and predictor accuracy.
In [9], Kaur et.al. applied an augmented shuffle frog leaping (ASFLA) approach for resources provisioning work flow scheduling infrastructure as service cloud computing environment. For task scheduling ASFLA tries to minimize the execution time and transfer time among dependent task. This approach is compared with simple SFL and PSO and a significant improvement is observed using ASFLA.
A simulated shuffle frog leaping algorithm (SSFLA) is suggested by Amirian et.al [10] for grey scale project selection scheduling in tri-objective grey environment. Implementing time limit, budget constraint and multiple objectives, a modified grey shuffle frog leaping algorithm is proposed. This technique is compared with nondominated sorting genetic algorithm (NSGA-II) and multi objective PSO to solve this NP hard problem.
In [11], the author has stated that SFLA is a novel meta-heuristic approach, applied in many combination problem. However in continuous optimization problem the algorithm falls in local optima. Thus author has proposed a variant of SFL called levy flight based shuffle frog leaping algorithm. The effectiveness of this technique is explored using 30 benchmark function and six continuous optimization functions. As a stochastic search based learning technique, Sharma et.al. [12], has suggested a varient of shuffle frog leaping algorithm called as centroid mutated SFLA for both discrete and continuous optimization problem.
Bhattacharjee et.al. [13] proposed a modified discrete shuffle frog leaping algorithm for 01 knapsack problem. This technique is extensively investigated taking considering large number of experimental studies. Due to its discretization property, performance shows a remarkable growth for small as well as medium sized knapsack problem and as an alternative solution for large knapsack problem.
Inspired from the successful implementation of SFLA in many application areas as an optimization approach, in this study a novel Modified Shuffled frog leaping algorithm based multi document summarizer is presented. MSFLA aims to maximize content coverage criteria while reducing redundancy and preserving length of the summary. The effectiveness of the proposed model has been evaluated with respect to particle swarm optimization (PSO), cuckoo search (CS) and standard shuffled frog leaping algorithm over DUC datasets. From the experimental results, it is clearly observed that MSFLA based multi document summarizer outperforms than conventional PSO, CS and SFLA based summarizer.
The rest of the paper is organized as follows. The detail of Shuffled frog leaping algorithm is presented in Section 2. In Section 3, Modified Shuffled frog leaping algorithm has been discussed. A detail framework for multi-document extractive summarization using MSFLA is presented in Section 4. The summary evaluation criteria and empirical study for performance analysis are discussed in Section 5 and 6 respectively. Finally conclusion is highlighted in Section 6.

Shuffled frog leaping algorithm
The Shuffled Frog Leaping Algorithm (SFLA) is a recent population based meta-heuristic algorithm. It incorporates both the benefits of memetic algorithm and social behavior of particle swarm optimization (PSO) algorithm. In SFLA (Figure 1), the population is a group of frogs, which are seeking for best available foods using search guidelines related to PSO algorithm. With the aim to search for food, the entire search process can be carried out by alternating communication of frogs in intra-cluster and inter-cluster. The intra-cluster communication performs within a memeplex for local invention and the inter-cluster communication performs between the frogs belongs to different memeplexes for global exploration. In traditional SFLA, assume that the initial population is generated randomly of P solutions (frogs). For each individual frog, evaluate fitness value. Afterward, sort the P in descending order of their fitness value. Then entire frogs are distributed into M number of memeplexes and each memeplex contains N frogs. In this process, the distribution of frogs is done in such a way that the first frog goes to first memeplex, second goes to first memeplex, frog M goes to M th memeplex, and frog M+1 goes to first memeplex, and so on till the last frog [14,15].
Within each memeplex, the best and the worst frog according to their fitness are represented as Xb and Xw. The frog having global fitness is denoted as Xg. The location of worst frog is updated either based on location of local best frog or global best frog or randomly to a position, so that the frogs can move towards the optimal solution. The updating measure is as follows: produced a better solution, it replaces w X . Otherwise, the calculation in Equation (1) and (2) are repeated by replacing Xb with Xg. If there is no improvement in such situation, then a randomly generated new solution is replaces to w X . Thereafter all the memeplexes are shuffled together to exchange information and generate new population for next search space. searching process only it updates worst solution without updating the best solution. Therefore it has a negative influence on convergence speed of algorithm and solving precision. Hence to improve the performance of SFLA, a modified version of SFLA (MSFLA) is suggested in this work [8,[16][17][18]. In MSFLA, the frog leaping step size is controlled by inserting search learning coefficient S with inertia component to change present movement status of frog during local search. The leaping rule discussed in equation 1 and 2 is modified and presented in equation 3 and 4. ( Where, S is search learning coefficient (i.e. S is any constant but greater than one) to scale the frog's step size during local search process. If S contains a big value, leads premature convergence or results in missing of the local search producing only random search with little improvement. In equation 3 and 4, the leaping step size Stp at any instance t depends on frog's leaping step size with worst position in previous iteration. The objective of introducing inertia weight  is to balance in the search process. It assumes that the greater inertia weight offers exploration while a smaller one raises the local exploration. Instead of considering a fixed inertia weight value, it is decreased repeatedly from a greater to smaller specified value [19,20].

Overview of proposed summarization approach
The proposed framework of document summarization is illustrated in Figure 2. Primary objective of this approach is to generate a document in summarized form, from a set of input documents. First of all, for each document, sentences are segmented and each term of that sentence is tokenized followed by removal of stop word and stemming. Next, modified term frequency and inverse document frequency ) (MTfIdf is applied to compute weight of each sentence (Wtsen). For evaluation of inter sentence similarity, the most widely used cosine similarity metric is used. Once the similarity matrix is created, a similarity threshold is employed to select significant sentences from each document. Thereafter all significant sentences are merged into a document and their respective sentence weights are used to prepare input matrix for optimization algorithm. Finally, by comparing with summary threshold the top weighted sentences are selected to generate summary.

Detailed steps of proposed approach using MSFLA
Step In this process, the distribution of frogs is done in such a way that the first frog goes to first memeplex, second goes to first memeplex, frog M goes to M th memeplex, and frog M+1 goes to first memeplex, and so on.
Step 12: Local search (until iterative steps for each memeplex is not reached): Step 12.1: Within each memeplex, determine the frog with best and worst position such as Xb and Xw. determine the frog with global best position Xg with respect to entire frog population.
Step 15: Finally, the frog with best fitness value is considered as candidate summary sentences.
Step 16: Select summary sentences chronologically from the document set based on their threshold by comparing with candidate summary sentences.

Summary evaluation criteria
The objective function f of proposed model is prepared in such a way that it maximizes coverage criteria while reducing redundancy and preserving length of the summary. Therefore, the authors have tried to form a summary from a set of documents with the objective of content coverage, non-redundancy and length. The summarization problem can be formalized as follows: In equation (10), the first two term guarantees that relevance of the summary and it covers main content of the document set. Whereas, the third term avoid multiple textual units, that convey the same information. The length of the summary is bounded by cardinality constraint, is discussed in equation (11).

Empirical study
This section introduces: 1) detail of benchmark dataset for evaluation of summary; 2) setting of controlling parameters during simulation; 3) describe the metrics to assess performance of the summary; 4) performance analysis.

Evaluation setup on the benchmark dataset
For the evaluation of proposed framework, the datasets from Document Understanding Conference (DUC) is used [22]. It is a benchmark data corpus for text summarization problem. This contains input documents along with few reference summaries. The DUC datasets i.e., DUC2006 and DUC2007 are distributed through ACQUINT, and used for this experimental study. DUC2006 and DUC2007 contains 50 and 45 different document clusters, each cluster contains 25 independent documents respectively. Each document cluster contains reference summaries of 250 words, which answers the question(s) in the topic statement.

Parameter setup
The setup of control parameters of any optimization algorithm is application oriented and no fixed value is assign to these parameters. Therefore derivation of parameters is obtained through number of simulations. In this study, the experimentation has been conducted taking the population size 50 to 200 and fixing the number of memeplex 10. Initially the memeplex are upgraded with 10 number of iteration. However, after 8 iterations no such remarkable upgradation has been observed. Here the author has added the optimal environment setup for considered application. The controlling parameters of MSFLA algorithm such as: size of population (SP) is 160, memeplex size (m) is 10, iterative steps for memeplex (im) are 8, number of iteration (it) is 50,search learning coefficient and inertia weight are decreasing value of range [1,1+rand()] and [0,1] respectively.

Performance evaluation metrics
The comprehensive performance study of MSFLA based summarizer has been conducted over DUC dataset and evaluated by ROUGE [23,24]. ROUGE stands for Recall-Oriented Understudy for Gisting Evaluation. It is used as the official evaluation metric for text summarization. A set of metrics such as ROUGE-L, ROUGE-N, ROUGE-S, ROUGE-W and ROUGE-SU are available in ROUGE to evaluate system generated summary against a set of reference summaries. In this study, ROUGE-N metric is used to match between system generated summaries and reference summaries.
Where N is contiguous sequence of terms, count match is the highest number of N-grams co-occurring in (10) (8) (9) (11) (12) system summary and reference-summaries. Count is the number of contiguous sequence of N-terms in the reference summaries.

Result analysis
The summary result has been evaluated by ROUGE-N score with N is 1 and 2 i.e. ROUGE-1 (R1) and ROUGE-2 (R2) metrics. ROUGE-1 and ROUGE-2 refers to word wise comparison and two words comparison at a time between the system summary and the reference summaries. Based on content coverage, anti-redundancy and length of summary ROUGE-N is calculated. A model producing higher ROUGE score reveals higher similarity of the system generated summary with respect to the original input document sets. Table 2 shows the ROUGE score obtained by proposed summarization model. ROUGE-1 score of all summarizer are falling within the range 0.41 to 0.44 and with respect to ROUGE-2 it is within the range 0.07 to 0.16 for DUC 2006 and DUC2007 dataset respectively. The statistical analysis in term of min (Min), average (Avg) and standard deviation (Std) are observed in Table 2 and Table 3     Furthermore, the validity of MSFLA based summarizer is verified by obtaining relative improvement of MSFLA based summarizer over other methods in terms of ROUGE score. The relative improvement comparison is calculated using the following formula.
Where ℎ includes PSO, CS or SFLA based summarizer.
From the analysis of results it is clearly observed that, the performance of proposed method based on MSFLA based summarizer is providing consistent result for the given scenario.
Thereafter a statistical significance test has been conducted at the 5% significance level of the summarization results [25].The average values and 95% confidence interval (CI) of ROUGE-1 and ROUGE-2 scores of each method for DUC 2006 and DUC 2007 data sets are shown in Tables 5 and 6 respectively.
From Table 5 and 6, it is observed that the average values of ROUGE-1 and ROUGE 2 for MSFLA method on all data sets are better than that for the other methods.

Conclusion
This paper attempts to present a MSFLA based multidocument summarizer. It highlights the implementation steps of MSFLA algorithm to optimize the sentence weight to generate summary from multiple documents. It is modified variation of standard SFLA. The application of PSO, CS and SFLA to extract sentences from multiple documents is also explored. Proposed summarizers are validated over DUC dataset. From the experimental analysis of ROUGE metrics and confidential interval of statistical significance test, clearly refer that the MSFLA based multi-document summarizer outperforms than other summarizer models discussed in this experimental study.
The future study of this research work will be extended for the abstractive summarization problem. The performance of summarizer will also be compared with other competent nature inspired algorithms.