A new unsupervised feature selection method for text clustering based on genetic algorithms

Shamsinejadbabki, P and Saraee, MH ORCID: https://orcid.org/0000-0002-3283-1912 2012, 'A new unsupervised feature selection method for text clustering based on genetic algorithms' , Journal of Intelligent Information Systems, 38 (3) , pp. 669-684.

PDF - Accepted Version
Download (1MB) | Preview


Nowadays a vast amount of textual information is collected and stored in various databases around the world, including the Internet as the largest database of all. This rapidly increasing growth of published text means that even the most avid reader cannot hope to keep up with all the reading in a field and consequently the nuggets of insight or new knowledge are at risk of languishing undiscovered in the literature. Text mining offers a solution to this problem by replacing or supplementing the human reader with automatic systems undeterred by the text explosion. It involves analyzing a large collection of documents to discover previously unknown information. Text clustering is one of the most important areas in text mining, which includes text preprocessing, dimension reduction by selecting some terms (features) and finally clustering using selected terms. Feature selection appears to be the most important step in the process. Conventional unsupervised feature selection methods define a measure of the discriminating power of terms to select proper terms from corpus. However up to now the valuation of terms in groups has not been investigated in reported works. In this paper a new and robust unsupervised feature selection approach is proposed that evaluates terms in groups. In addition a new Modified Term Variance measuring method is proposed for evaluating groups of terms. Furthermore a genetic based algorithm is designed and implemented for finding the most valuable groups of terms based on the new measure. These terms then will be utilized to generate the final feature vector for the clustering process . In order to evaluate and justify our approach the proposed method and also a conventional term variance method are implemented and tested using corpus collection Reuters-21578. For a more accurate comparison, methods have been tested on three corpuses and for each corpus clustering task has been done ten times and results are averaged. Results of comparing these two methods are very promising and show that our method produces better average accuracy and F1-measure than the conventional term variance method.

Item Type: Article
Themes: Media, Digital Technology and the Creative Economy
Memory, Text and Place
Schools: Schools > School of Computing, Science and Engineering > Salford Innovation Research Centre
Journal or Publication Title: Journal of Intelligent Information Systems
Publisher: Kluwer
Refereed: Yes
ISSN: 0925-9902
Related URLs:
Depositing User: Prof. Mo Saraee
Date Deposited: 19 Oct 2011 10:34
Last Modified: 15 Feb 2022 16:08
URI: http://usir.salford.ac.uk/id/eprint/18510

Actions (login required)

Edit record (repository staff only) Edit record (repository staff only)


Downloads per month over past year