Fuzzy Approach Topic Discovery in Health and Medical Corpora

TL;DR

This paper introduces Fuzzy Latent Semantic Analysis (FLSA), a novel fuzzy-based topic modeling method that improves health and medical document retrieval by addressing redundancy and estimating the number of topics more effectively than LDA.

Contribution

The paper presents FLSA, a new fuzzy approach to topic modeling that enhances medical text analysis and outperforms traditional LDA in accuracy and feature extraction.

Findings

FLSA outperforms LDA in quantitative evaluations.

FLSA effectively handles redundancy in medical corpora.

FLSA provides a new method for estimating the number of topics.

Abstract

The majority of medical documents and electronic health records (EHRs) are in text format that poses a challenge for data processing and finding relevant documents. Looking for ways to automatically retrieve the enormous amount of health and medical knowledge has always been an intriguing topic. Powerful methods have been developed in recent years to make the text processing automatic. One of the popular approaches to retrieve information based on discovering the themes in health & medical corpora is topic modeling, however, this approach still needs new perspectives. In this research we describe fuzzy latent semantic analysis (FLSA), a novel approach in topic modeling using fuzzy perspective. FLSA can handle health & medical corpora redundancy issue and provides a new method to estimate the number of topics. The quantitative evaluations show that FLSA produces superior performance and features to latent Dirichlet allocation (LDA), the most popular topic model.

Tables

Table 1. Table 1: GTW Methods

Name	Formula
Entropy	$1+\frac{{\sum }_j{p}_{ij}{\log }_2\left(p_{ij}\right)}{{\log }_{2}n}$
IDF	${\log }_2\frac{n}{{\sum }_{j}t{f}_{ij}}$
Normal	$\frac{1}{\sqrt{{\sum }_{j}t{f}_{ij}^2}}$
ProbIDF	${\log }_2\frac{n-{\sum }_{j}b\left(t{f}_{ij}\right)}{{\sum }_{j}b\left(t{f}_{ij}\right)}$

Table 2. Table 2: Basic Statistics for Datasets

Dataset Name	#Documents	#Term Tokens	#Unique Terms	Avg Term Per Document	Description
M-Dataset	1,527	245,931	14,411	96.3	Medical Papers
N-Dataset	1,607	299,449	11,059	124.8	Nursing Notes
O-Dataset	2,092	198,998	15,768	95.1	Medical Papers
T-Dataset	58,927	395,635	25,310	6.7	Tweets
R-Dataset-1	2,288	754,801	3,265	231.2	News
R-Dataset-2	3,310	785,467	23,408	236.4	News
R-Dataset-3	3,254	757,603	22,982	232.8	News
R-Dataset-4	3,162	705,100	22,264	222.9	News
R-Dataset-5	3,211	751,496	23,046	234.1	News
R-Dataset-6	3,124	727,687	22,944	232.9	News
R-Dataset-7	3,251	747,067	22,609	229.8	News
R-Dataset-8	3,212	747,956	22,961	232.6	News
R-Dataset-9	3,257	755,261	22,929	231.9	News
R-Dataset-10	3,258	739,041	22,913	226.8	News
R-Dataset-11	3,284	764,658	23,134	232.8	News

Table 3. Table 3: M-Datatset Classification - 50 and 100 Topics

Method	Acc %	F-Measure	MCC	ROC	#Topics
LDA	90.05	0.9	0.799	0.969	50
FLSA(Entropy)	97.66	0.977	0.953	0.99	50
FLSA(IDF)	95.90	0.959	0.917	0.982	50
FLSA(Normal)	91.22	0.912	0.824	0.971	50
FLSA(ProbIDF)	97.66	0.977	0.953	0.987	50

Table 4. Table 7: Confusion Matrix

		Predicted
		Negative	Positive
Actual	Negative	TN	FP
	Positive	FN	TP

Table 5. Table 8: A Sample of Topics

$T_{83}$	$T_{42}$	$T_{33}$	$T_{101}$	$T_{85}$	$T_{107}$	$T_{71}$	$T_{25}$
teen	sick	teens	deadly	diabetes	rt	rt	rt
nursing	pregnant	put	government	week	health	health	health
dementia	billion	popular	nurses	find	ebola	ebola	ebola
benefit	clinical	save	affordable	recipes	lightning	pms	study
ready	dying	alzheimer	clinic	work	study	poached	cancer
babies	stories	leave	premiums	low	welcomes	upset	care
race	rare	run	ban	kids	vape	study	amp
black	access	docs	city	rt	cyberbullying	tension	sneeze
worst	role	quality	warns	women	visually	psychiatrists	poo
meet	hope	tied	deadline	life	tremors	embryo	risk

Equations

$$A=\hbox{}\vbox{\kern 0.86108pt\hbox{$\kern 0.0pt\kern 2.5pt\kern-5.0pt\left[\kern 0.0pt\kern-2.5pt\kern-5.55557pt\vbox{\kern-0.86108pt\vbox{\vbox{ \halign{\kern\arraycolsep\hfil\@arstrut$\kbcolstyle#$\hfil\kern\arraycolsep& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep&& \kern\arraycolsep\hfil$\@kbrowstyle#$\ifkbalignright\relax\else\hfil\fi\kern\arraycolsep\cr 5.0pt\hfil\@arstrut$\scriptstyle$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle Word_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle Word_{2}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle Word_{3}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle Word_{4}\\Document_{1}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 3$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 1$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 4$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0\\Document_{2}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 1$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0\\Document_{3}$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 3$\hfil\kern 5.0pt&5.0pt\hfil$\scriptstyle 0\\$\hfil\kern 5.0pt\crcr}}}}\right]$}}$$

$$b(tf_{ij})=\left\{\begin{array}[]{c l}1&tf_{ij}>0\\ 0&tf_{ij}=0\end{array}\right.$$

$$p_{ij}=\frac{t{f}_{ij}}{{\sum }_{j}t{f}_{ij}}$$

$$Min{J}_q(\mu ,V,X)=\sum _{k=1}^c\sum _{j=1}^n({\mu }_{kj}{)}^{q}DI{S}_{kj}^2$$

$$0\le {\mu }_{kj}\le 1;$$

$$\sum _{k=1}^c{\mu }_{kj}=1$$

$$0<\sum _{j=1}^n{\mu }_{kj}<n;$$

$${\mu }_{ij}=\frac{1}{{\sum }_{l=1}^c(\frac{DI{S}_{kj}}{DI{S}_{lj}}{)}^{\frac{2}{q-1}}}$$

$$v_i=\frac{{\sum }_{j=1}^n({\mu }_{kj}{)}^q{x}_j}{{\sum }_{j=1}^n({\mu }_{kj}{)}^q}$$

$$P(D_j)=\frac{{\sum }_{i=1}^m(W_i,D_j)}{{\sum }_{i=1}^m{\sum }_{j=1}^n(W_i,D_j)}$$

$$P(D_j,T_k)=P(T_k|D_j)\times P(D_j)$$

$$P(D_j|T_k)=\frac{P(D_j,T_k)}{{\sum }_{j=1}^{n}P(D_j,T_k)}$$

$$P(W_i|D_j)=\frac{P(W_i,D_j)}{{\sum }_{i=1}^{m}P(W_i,D_j)}$$

$$P(W_i|T_k)=\sum _{j=1}^{n}P(W_i|D_j)\times P(D_j|T_k)$$

$$Precision(P)=\frac{TP}{TP+FP}$$

$$Recall(R)=\frac{TP}{TP+FN}$$

$$Accuracy(Acc)=\frac{TP+TN}{TP+TN+FP+FN}$$

$$F-measure=2\times \frac{P\times R}{P+R}$$

$$\frac{(TP\times TN)-(FN\times FP)}{\sqrt{(TP+FP)\times (TP+FN)\times (TN+FP)\times (TN+FN)}}$$

Full text

1 Introduction

There is a growing need to analyze large collections of electronic documents. Moreover, very large-scale scientific data management and analysis is one of the data-intensive challenges identified by National Science Foundation (NSF) as an area for future study Council [2016]. Large collections of electronic documents abound as our collective knowledge continues to be digitized and stored, requiring new tools for organization, search, indexing, and browsing. As a consequence, finding relevant documents has become more difficult for experts. In particular, large scale health and medical text data historically has been generated and stored. For example, the total number of papers published on PubMed website is more than 6 million papers in 2015111http://www.ncbi.nlm.nih.gov/pubmed and the annual average number of US hospital discharges is more than 30 million records Karami et al. [2015a, b]. This huge amount of text data and EHRs is a great motivation for companies to save $450 billion a year using advanced data analytical approaches222http://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/the-big-data-revolution-in-us-health-care.

Developing efficient techniques for discovering the hidden structure in large complicated health and medical data sets, and using that structure to answer questions about those data, is at the core of big health and medical data science research. Substantial resources were allocated in developing new data analytic methods and tools. However, retrieving big health and medical text data is a major current challenge.

One of the popular methods in medical text data representation is bag-of-words (BOW). This technique represents documents based on the frequency of words with a matrix like $A$.

[TABLE]

For example, matrix $A$ shows that word 3 appeared 3 times in document 3. However, this matrix is a sparse matrix for large number of documents Karami and Gangopadhyay [2014]. Sparsity means that there are a lot of words in a corpus; however, one document covers a small percentage of all words. Therefore, most elements are zero in BOW matrix Aggarwal and Zhai [2012].

Topic modeling is a popular method to address sparsity and high dimensionality issues. This method was originally introduced as a text analysis technique that the objects are documents and the features are the frequency of terms. The output of topic modeling is two matrices. The first one is the probability of words for each topic or $P(W|T)$ and the second one is the probability of topics for each document or $P(T|D)$ (Figure 1).

The words with higher probability in $Words\times Topic$ matrix discloses semantic structure. $Topics\times Documents$ matrix reduces the number of dimension from the number of words in BOW approach to the number of topics. For example, suppose that there are 100 topics in a corpus with 5000 documents and 10,000 words. Topic modeling converts $W\times D$ matrix to two matrices: Words $\times$ Topics matrix with 10,000 rows and 100 columns, and Topics $\times$ Documents matrix with 100 rows and 5000 columns. It is worth mentioning that the second non-sparse matrix is used for document classification and clustering with 100 topics as the number of features instead of 10,000 words as the number of features in the BOW approach. Document classification and clustering problems categorized labeled and unlabeled documents based on extracted features (topics) from documents using topic modeling.

Topic modeling is an effective method for health and medical text mining; however, due to the intensive amount of available data, there is still the need to improve the performance of this approach. In addition, copy and paste (redundancy) has a negative impact on topic modeling Cohen et al. [2013] and previous work has shown that most of medical notes are redundant Wrenn et al. [2010].

In this research, we propose F**uzzy Latent Semantic Analysis (FLSA) model for health and medical text mining. This model shows better performance in both redundant and non-redundant document and can help topic models estimating number of topics in corpus. The remainder of this paper is organized as follows. In the related work section, we review the related research. In the methodology section, we provide more details for FLSA. An empirical study was conducted to verify the effectiveness of FLSA. Finally, we provide an illustrative example for FLSA, and present a summary and future directions in the last two sections.

2 Related Work

Text mining can be defined as the methods of machine learning and statistics with the goal of recognizing patterns and disclosing the hidden information in text data Hotho et al. [2005]. In this section, we review key concepts, and health and medical applications of topic modeling and fuzzy clustering (FC).

2.1 Topic Modeling in Health and Medical

There are two main approaches in text mining: supervised and unsupervised. The goal of supervised approach is to disclose hidden structure in labeled datasets and the goal of unsupervised approach is to discover patterns in unlabeled datasets. The most popular techniques in supervised and unsupervised approaches are classification and clustering, correspondingly. The purpose of classification is to train a corpus with predefined labels and assign a label to a new document Mitchell [1997]. Clustering assigns a cluster to each document in a corpus based on similarity in a cluster and dissimilarity between clusters. Among text mining techniques, topic modeling is one of popular unsupervised methods with a wide range of applications from SMS spam detection Karami and Zhou [2014a] to image tagging Xu et al. [2009].

Topic modeling defines each topic as probability distribution over words and a document as probability distribution over topics. In health and medical text mining, latent Dirichlet allocation (LDA) shows better performance than other topic models Sarioglu et al. [2012].

2.2 Health and Medical Applications of LDA

LDA is a generative probabilistic model based on a three-level hierarchical Bayesian model. LDA assumes that documents contain latent topics and each topic can be represented by a distribution across words Blei et al. [2003].

LDA has a wide range of health and medical applications such as predicting protein-protein relationships based on the literature knowledge Asou and Eguchi [2008], discovering relevant clinical concepts and structures in patients’ health records Arnold et al. [2010], identifying patterns of clinical events in a cohort of brain cancer patients Arnold and Speier [2012], and analyzing time-to-event outcomes Dawson and Kendziorski [2012]. The discovery of clinical pathway (CP) patterns is a method for revealing the structure, semantics, and dynamics of CPs to provide clinicians with explicit knowledge used to guide treatment activities of individual patients. LDA has used for CPs to find treatment behaviors of patients Huang et al. [2014], to predict clinical order patterns, and to model various treatment activities Chen et al. [2016] and their occurring time stamps in CPs Defossez et al. [2014]. LDA has also customized to determine patient mortality Ghassemi et al. [2014], and to discover knowledge from modeling disease and patient characteristics Pivovarov et al. [2015]. Redundancy-aware LDA (Red-LDA) is one of the versions of LDA for handling redundancy issue in medical documents and has shown better performance than LDA Cohen et al. [2014].

2.3 Health and Medical Applications of FC

There are two major clustering approaches: hard and fuzzy (soft). In hard clustering, every object may belong to exactly one cluster but, in fuzzy clustering (FC), the membership is fuzzy and objects may belong to several clusters Karami [2015]. Among fuzzy clustering techniques, fuzzy C-means (FCM) is the most popular model Bezdek [1981]. FCM is based on minimizing the overall distance from a cluster prototype to each datum.

Fuzzy clustering has used in predicting the response to treatment with citalopram in alcohol dependence Naranjo et al. [1997], analyzing diabetic neuropathy Di Lascio et al. [2002], detecting early diabetic retinopathy Zahlmann et al. [2000], characterizing stroke subtypes and coexisting causes of ischemic stroke Helgason and Jobe [1998], Helgason et al. [2001], Helgason and Jobe [1999], improving decision-making in radiation therapy Papageorgiou et al. [2003], and detecting cancer such as breast cancer Hassanien [2003]. In addition, fuzzy clustering was used to improve ultrasound imaging technique Moon et al. [2011] and analyze microarray data Gasch and Eisen [2002].

Although there are a lot of fuzzy clustering applications in health and medical domains especially in image processing, this approach has not been considered for topic modeling yet. This paper proposes a new approach to provide a bridge between fuzzy clustering and topic modeling to analyze big health and medical corpora.

3 Methodology

In this part, we describe our method, fuzzy latent semantic analysis (FLSA), for uncovering latent semantic features from text documents. FLSA treats fuzzy view as a new approach in topic modeling and will be validated through a series of experiments, conducted on health and medical text data.

Although LDA has shown a better performance than other topic models, redundancy has negative effect on LDA performance Cohen et al. [2013], Downey et al. [2006]. The reason is that, for example, words like $w_{1}w_{2}w_{3}$ in a document such as $d_{k}=\{w_{1}w_{2}w_{3}w_{6}w_{9}\}$ are copied to a document like $d_{p}=\{w_{7}w_{8}\}$ to be $d_{p}=\{w_{1}w_{2}w_{3}w_{7}w_{8}\}$. $w_{1}w_{2}w_{3}$ should be assigned to the same topic by LDA but it is possible to be assigned by LDA to different topics. FLSA has a potential to handle the redundancy issue, estimate the optimum number of topics, and provide better performance than its competitors.

3.1 Fuzzy Logic and Fuzzy Clustering

The traditional reasoning has precise character that is yes-or-no (true-or-false) rather than more-or-less Zimmermann [2010]. Fuzzy logic added a new extension to move from the classical logic, 0 or 1, to the truth values between zero and one, [0,1]) Zadeh [1973] (Figure 2).

Fuzzy logic assumes that if $X$ is a collection of data points represented by $x$, then a fuzzy set $A$ in $X$ is a set of order pairs, $A=\{(x,\mu_{A}(x)|x\in X)\}$.

$\mu_{A}(x)$ is the membership function which maps $X$ to the membership space $M$ which is between 0 and 1 Karami and Guo [2012]. The goal of most clustering algorithms is to minimize the objective function $J$ that measures the quality of clusters to find the optimum $J$ which is the sum of the squared distances between each cluster center and each data point.

The main goal of fuzzy models is to formulate uncertainty for applications such as decision-making Karami et al. [2010], Karami and Guo [2012]. For example, a voter decides to select some candidates among a set of candidates in an election. The voter has different preferences in terms of economic, foreign policy, health, etc. Based on the preferences, the distance between each candidate’s plans and the voter’s preferences can be changed. These preferences can be formulated and measured in fuzzy clustering with $\mu$, degree of membership.

3.2 FLSA

FLSA assumes that documents and words can be fuzzy clustered and each cluster is a topic. For example, given a corpus FLSA discovers topic 1 including dna, genome, and human words with “Genetics” theme, topic 2 including infectious, bacteria, and disease words with “Disease” theme, and topic 3 including organisms, species, and evolution words with “Evolution” theme (Figure 3). In this process, words are assigned a fuzzy degree of membership with respect to each cluster (topic). The color of circles shows the magnitude of membership from low (light Grey) to high (dim Grey).

The main goal of FLSA is to find two matrices: $P(T|D)$ and $P(W|T)$ mentioned in Figure 1. FLSA has seven steps using Local Term Weighting (LTW), Global Term Weighting (GTM), and Fuzzy Clustering (FC):

Step 1. The first step is to calculate LTW. Among different LTW methods, we use term frequency (TF) as it is the most popular method.

Step 2. The next step is to calculate GTW. We explore four GTW methods in this research including Entropy, Inverse Document Frequency (IDF), Probabilistic Inverse Document Frequency (ProbIDF), and Normal (Table 1):

• •

Entropy gives higher weight to the terms with less frequency in few documents Dumais [1992].

• •

IDF assigns higher weights to rare terms and lower weights to common terms Papineni [2001].

• •

Normal is used to correct discrepancies in document lengths and also normalize the document vectors Kolda [1998].

• •

ProbIDF is similar to IDF and assigns very low negative weight for the terms occurring in every document Kolda [1998]

Symbol $tf_{ij}$ defines the number of times word $i$ occurs in document $j$. With $m$ words and $n$ documents, we need to find $b(tf_{ij})$ and $p_{ij}$ for calculating the four mentioned GTW methods:

[TABLE]

[TABLE]

The outputs of this step are the document term matrices with applied TF-Entropy, TF-IDF, TF-Normal, and TF-ProbIDF methods.\

Step 3. We use Fuzzy C-means (FCM) in this research to fuzzy cluster the documented represented by the four mentioned GTW methods. FCM minimizes an objective function by considering constraints:

[TABLE]

subject to:

[TABLE]

[TABLE]

[TABLE]

Where: \

$n$= number of data\ $c$= number of clusters (topics)\ $\mu_{kj}$= membership value\ $q$= fuzzifier, $1<q\leq\infty$ \ $V$= cluster center vector\ $DIS_{kj}=d(x_{j},v_{k})$= distance between $x_{j}$ and $v_{k}$ \

By optimizing eq.3:

[TABLE]

[TABLE]

The iterations in the clustering algorithm continue till the maximum changes in $\mu_{kj}$ becomes less than or equal to a pre-specified threshold with $O(n)$ computational time complexity.

We use $\mu_{kj}$ as the membership degrees for each document (D) with respect to each of topics (clusters). The value of $\mu_{kj}$ is between 0 and 1 that can be interpreted as $\mathbf{\underline{P(T_{k}|D_{j})}}$ or probability of topic k in document j. This step finds $Topics\times Documents$ matrix and uses it along with the following steps to find $Words\times Topics$ matrix in Figure 1. It is worth mentioning that FLAS’s steps are dependent and integrated. For example, the documents (D) and the topics (T) in step 3 are the same topics and documents in steps 5,6 and 7.

To avoid the negative impact of high dimensionality of the four mentioned matrices in step 2, we use singular value decomposition (SVD), which is a popular method Fodor [2002] to reduce the data dimension before using fuzzy clustering. We select two dimensions, as the minimum number of dimensions, for SVD to have a fast process. \

Step 4. We use document-term matrices with the GTW methods in step 2 ($Words\times Documents$ matrix) to find $\mathbf{\underline{P(D_{j}})}$ or probability of document j:

[TABLE]

Step 5. The next step has two parts. The first part is to find $\mathbf{\underline{P(D_{j}|T_{k})}}$ or probability of document j in topic k using $P(T_{k}|D_{j})$ in step 3 and $P(D_{j})$ in step 4:

[TABLE]

Then normalizing $P(D,T)$ in each topic:

[TABLE]

Step 6: We use document-term matrices with the GTW methods in step 2 to find $\mathbf{\underline{P(W_{i}|D_{j})}}$ or probability of word i in document j:

[TABLE]

Step 7: The final step is to find $\mathbf{P(W_{i}|T_{k})}$ or probability of word i in topic k ($Topics\times Documents$ matrix in Figure 1) using $P(D_{j}|T_{k})$ in step 5 and $P(W_{i}|D_{j})$ in step 6:

[TABLE]

FLSA is flexible to work with all dimensionality reduction techniques Fodor [2002] such as principal component analysis (PCA) Jolliffe [2002] and all fuzzy clustering techniques such as self-organizing map (SOM) Kohonen [1990].

Fuzzy c-means, as it is the core of FLSA, can be applied on both discrete and continuous data. This feature enables FLSA to use a wide range of LTW and GTW methods and to be used for other machine learning and data science applications such as image processing Keller et al. [2005]. Moreover, optimization nature of fuzzy clustering provides a solution to estimate the optimum number of topics.

4 Experiments

In this section, we evaluate FLSA against LDA by document classification using Random Forest, document clustering using k-means, document modeling using log-likelihood, and execution time test. We also evaluate FLSA against RedLDA by document modeling on redundant documents.

We use five datasets, the Matlab package for Chib-style estimation of log-likelihood333http://www.cs.umass.edu/~wallach/code/etm/, the FCM Matlab package444http://www.mathworks.com/help/fuzzy/fcm.html with its default settings including 100 iterations and 1e-5 as the minimum improvement in objective function between two consecutive iterations, the Weka tool555http://www.cs.waikato.ac.nz/ml/weka/ for classification evaluation, the MALLET package666http://mallet.cs.umass.edu/ with its default settings for implementing LDA, and the Python package for implementing RedLDA777https://sourceforge.net/projects/redlda/. The source code for FLSA will be available in the first author’s website888https://sites.google.com/site/karamihomepage/*,*999https://github.com/amir-karami in R and Matlab platforms.

4.1 Datasets

We leverage five available health and medical datasets in this research (Table 2):

• •

The first dataset101010http://muchmore.dfki.de/resources1.htm is MuchMore Springer Bilingual Corpus (M-Dataset) which is a labeled corpus of English scientific medical abstracts from the Springer website. In this research, we use the first 2 journals including: Arthroscopy and Federal Health Standard Sheet.

• •

The second dataset111111http://physionet.org/ is an unlabeled corpus of 2,434 nursing notes (N-Dataset).

• •

The third dataset121212http://disi.unitn.it/moschitti/corpora/ohsumed-first-20000-docs.tar.gz is Ohsumed Collection (O-Dataset) that is a labeled corpus of medical abstracts from the MeSH categories including Bacterial Infections and Mycoses, and Virus Diseases.

• •

The fourth dataset (T-Dataset) is health news from Twitter131313www.twitter.com. We collected the tweets from the health related Twitter accounts including cbchealth, everydayhealth, foxnewshealth, goodhealth, kaiserhealthnews, latimeshealth, msnhealthnews, NBChealth, nprhealth, usnewshealth, bbchealth, cnnhealth, gdnhealthcare, nytimeshealth, reuters_health, and wsjhealth from August 2011 to December 2014141414https://github.com/amir-karami/Health-News-Tweets-Data.

• •

The fifth dataset is R-Datasets151515https://sourceforge.net/projects/corpusredundanc/files/?source=navbar are the synthesis text documents to track the negative effect of redundancy in documents Cohen et al. [2013]. These datasets are subsets of a larger dataset called WSJ which has a collection of the abstracts of Wall Street Journal. In this dataset, 1300 abstracts were sampled between 1 and 5 times in a uniform manner for 11 times to eliminate bias from random sampling.

4.2 Document Classification

The first evaluation measure is document classification using two labeled datasets, M-Dataset and O-Dataset. Both datasets have two classes (labels), M-Dataset with Arthroscopy and Federal Health Standard Sheet classes, and O-Dataset with Bacterial Infections and Mycoses, and Virus Diseases classes.

Document classification problem assigns a document to a class and this problem needs to extract features from text data. To avoid high dimensionality of BOW approach for document classification, topic modeling reduces the number of features by clustering the meaningful related words as a topic. To avoid any possible bias, we track the performance of FLSA against LDA with the 10-fold cross validation method that the data is broken into 10 subsets for 10 iterations. Each of the subsets is selected for testing and the rest of sets are selected for training. We use 50, 100, 150, and 200 topics as the input features of documents for Random Forest method as it is one of the popular and high performance classification methods Qi et al. [2006]. The output of Random Forest is presented as a confusion matrix (Table 7) with the following definitions:

• •

True Negative (TN) is the number of correct predictions that an instance is negative.

• •

False Negative (FN) is the number of incorrect of predictions that an instance negative.

• •

False Positive (FP) is the number of incorrect predictions that an instance is positive.

• •

True Positive (TP) is the number of correct predictions that an instance is positive.

For evaluating the performance of the classification algorithm, we use accuracy (ACC), F-measure, Matthews Correlation Coefficient (MCC), and the area under ROC (AUC). The evaluation metrics are defined based on the confusion matrix, as shown in equations 14 - 18:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

ROC curves plot FP on the X axis vs. TP on the Y axis to find the trade off between them; therefore, the ROC is closer to the upper left indicating better performance (Figure 6).

MCC is used to determine the quality of classification methods, ranging between -1 (the worst performance) and +1 (the best performance). \

MCC=

[TABLE]

This experiment shows that FLSA with Entropy, IDF, Normal, and ProbIDF show better performance than LDA with different numbers of topics (Tables 6 - 6). The highest performance in each table is shown in bold format.

4.3 Document Clustering

The second evaluation is document clustering using unlabeled N-Dataset. Internal and external validation are two major methods for clustering validation; however, comparison between these two major methods shows that internal validation is more precise Rendón et al. [2011]. We evaluate different numbers of topics and clusters with Calinski-Harabasz (CH) index, as one of the popular internal validation methods, using K-means with 500 iterations. CH index evaluates the cluster validity based on the average of the sum of squared error cluster between and within clusters. Higher CH index indicates “better” clustering. \ We track the performance of FLSAs and LDA using different numbers of clusters ranging from 2 to 8 with different numbers of topics including 50, 100, 150, and 200. CH index shows that FLSAs have better performance than LDA with the different ranges of features and clusters (Figures 5 & 5). The gap between FLSAs and LDA does not change significantly with different numbers of topics and clusters.

4.4 Redundancy Issue

The next experiment explores the effect of redundancy issue using the fifth dataset. R-Datasets are not a medical corpus; however, they were created as a synthetic redundant corpus without having privacy issue to measure the effect of redundancy issue Cohen et al. [2013]. We select publicly available unlabeled R-Datasets to make the evaluation process easier for possible future research. We compare FLSAs with not only LDA but also RedLDA, as it was developed to handle redundancy issue in medical text data Cohen et al. [2014].

We train LDA, RedLDA and FLSA models on R-Datasets to compare the generalization performance of the models. We compute the log-likelihood on a held-out test set to evaluate the models. A higher log-likelihood score indicates better generalization performance. Figure 7 shows the average log-likelihood of the R-Datasets with different numbers of topics from 25 to 350. This experiment indicates that FLSAs performs better than RedLDA and LDA on redundant documents.

4.5 Execution Time

In this section, we compare the speed of FLSA in comparison with LDA using T-Dataset, the biggest dataset in this paper. The major process in topic modeling is based on a joint probability distribution over hidden topics and the observed words to infer the word with higher probability in each topic by using the posterior distribution. The most popular approximate method for LDA is collapsed Gibbs sampling applied in the experiments. All the inference algorithms require multiple iterations which increase the computational cost linearly with the number of documents, topics, words, and iterations Zeng et al. [2012]. Figure 8 shows that the time performance of FLSAs is stable with an increase in the numbers of topics and considerably better than LDA.

5 Example

Documents and papers in health and medical domains contain terms and words that need expertise to understand them. Therefore, we show an illustrative example with T-Dataset containing tweets with more understandable words in this section. First, we run FLSA on different numbers of topics to estimate optimum number of topics using the objective function ,$J_{q}$ (Figure 9). This estimation feature of fuzzy clustering is on of advantageous of FLSA. The elbow or the knee part of the figure shows $\sim$125 as the number of topics161616https://www.cs.princeton.edu/courses/archive/spring07/cos424/scribe_notes/0306.pdf.

Table 8 shows a sample of topics that are about family, cancer, Ebola, Alzheimer, nursing, and teenagers:

• •

$T_{83}$ is about the benefit of nursing black teens with dementia.

• •

$T_{42}$ is about the huge costs of teens pregnancy.

• •

$T_{33}$ is reporting a teen’s invention to save Alzheimer’s patients and improve the quality of their lives.

• •

$T_{101}$ is about the government’s need to hire more nurses in different cities.

• •

$T_{85}$ is about the recipes for family having diabetes specially for women and kids.

• •

$T_{107}$ is about using Ebola for cyberbullying.

• •

$T_{71}$ is about the role of psychiatrists against Ebola.

• •

$T_{25}$ is about sneezing as a risk for spreading Ebola and the relation between Ebola and lung cancer.

6 Conclusion

The vast array of health and medical text data represents a valuable resource that can be analyzed to advance state-of-the-art medicine and health. Large electronic health and medical archives such as PubMed provide an extremely useful service to the scholarly community. However, the needs of readers go beyond a simple keyword search.

Topic modeling is one of the popular unsupervised methods to automatically discover a hidden thematic structure in a large collection of unstructured health and medical documents. This discovered structure facilitates browsing, searching, and summarizing the collection.

Fuzzy perspective is a machine learning approach that has been used more in medical image processing than text processing. Existing techniques of topic modeling are based on two main approaches: linear algebra and statistical distributions; however, this paper proposes FLSA to utilize fuzzy perspective for disclosing latent semantic features of health and medical text data.

FLSA is a new competitor to the established topic models such as LDA and has the flexibility to work with a wide range of dimension reduction and fuzzy clustering techniques. FLSA also works with both discrete and continuous data, estimates the optimum number of topics, and avoids the negative effect of the redundancy issue in health and medical corpora.

In our future work, we will develop dynamic and hierarchal topic models using fuzzy perspective. In addition, FLSA will be applied on social media data to track public opinions and will be used for online review and SMS spam detection Karami and Zhou [2015, 2014b].