Causal relationship between eWOM topics and profit of rural tourism at Japanese Roadside Stations "MICHINOEKI"

TL;DR

This study investigates how eWOM topics on Twitter causally influence sales at Japanese roadside stations, revealing that mentions of desserts, access issues, and biker groups positively impact revenue.

Contribution

It applies LiNGAM causal analysis to Twitter data and sales, providing new insights into eWOM's impact on rural tourism economic revitalization.

Findings

Mentions of desserts positively influence sales.

Accessibility and traffic issues affect customer engagement.

Motorcycle biker groups represent an untapped customer segment.

Abstract

Affected by urbanization, centralization and the decrease of overall population, Japan has been making efforts to revitalize the rural areas across the country. One particular effort is to increase tourism to these rural areas via regional branding, using local farm products as tourist attractions across Japan. Particularly, a program subsidized by the government called Michinoeki, which stands for 'roadside station', was created 20 years ago and it strives to provide a safe and comfortable space for cultural interaction between road travelers and the local community, as well as offering refreshment, and relevant information to travelers. However, despite its importance in the revitalization of the Japanese economy, studies with newer technologies and methodologies are lacking. Using sales data from establishments in the Kyushu area of Japan, we used Support Vector to classify content from Twitter into relevant topics and studied their causal relationship to the sales for each establishment using LiNGAM, a linear non-gaussian acyclic model built for causal structure analysis, to perform an improved market analysis considering more than just correlation. Under the hypotheses stated by the LiNGAM model, we discovered a positive causal relationship between the number of tweets mentioning those establishments, specially mentioning deserts, a need for better access and traf^ic options, and a potentially untapped customer base in motorcycle biker groups.

Tables

Table 1. Table 1 : Topic Classification Content.

Topic ID	Topic content	Example keywords
Topic 1	Products and Services	shop; set meal; popular; chocolate; ice cream; soft serve; café;
Topic 2	Special Events	exhibition; event; illumination lighting ceremony; Christmas tree
Topic 3	Promotional	N/A(Determined by related service company twitter accounts list)
Topic 4	Traffic and Weather	national highway; attention; road traffic information
Topic 5	Check-in	I’m at; in; Location
Topic 6	Positive Reviews	delicious; Instagram-able; cute; cheap; ate; happy; (*´\textomega`*)
Topic 7	Motorcycles	refueling; bike; meeting place; yaeya sticker
Topic 8	Unrelated and Others	JR; train; wagon; rotary; west exit; ride

Table 2. Table 2 : Results of the F 1 subscript 𝐹 1 F_{1} -Score in each topic.

Topic ID	Topic content	$F_1$-Score
Topic 1	Products and Services	0.85
Topic 2	Special Events	0.81
Topic 4	Traffic and Weather	0.88
Topic 5	Check-in	0.78
Topic 6	Positive Reviews	0.79
Topic 7	Motorcycles	0.87
Topic 8	Unrelated and Others	0.78

Table 3. Table 3 : Results of the LiNGAM causality analysis.

Topic ID	Topic content	Connection Strength	Causality Direction
Topic 1	Products and Services	11095.95	$x_1\to y$
Topic 2	Special Events	515.17	$x_2\to y$
Topic 3	Promotional	6738.08	$x_3\to y$
Topic 4	Traffic and Weather	-231845.95	$x_4\to y$
Topic 5	Check-in	1724.36	$x_5\to y$
Topic 6	Positive Reviews	15387.70	$x_6\to y$
Topic 7	Motorcycles	-7770.03	$x_7\to y$

Equations

$$P_{ijP}=\frac{N_{ijP}}{{\sum }_{i=1}^M{N}_{ijP}}$$

$$P_{ijN}=\frac{N_{ijN}}{{\sum }_{i=1}^M{N}_{ijN}}$$

$$H_{Pj}=-\sum _{i=1}^M[P_{ijP}{\log }_2{P}_{ijP}]$$

$$H_{Nj}=-\sum _{i=1}^M[P_{ijN}{\log }_2{P}_{ijN}]$$

$$H_{Pj}>\alpha H_{Nj}$$

$$x_i=\sum _{k(j)<k(i)}{b}_{ij}{x}_j+e$$

$$\mathbf{x}=\mathbf{Bx}+\mathbf{e}$$

$$\mathbf{x}=(\mathbf{I}-\mathbf{B}{)}^{-1}\mathbf{e}$$

$$\mathbf{x}={\mathbf{A}}_{ICA}\mathbf{s}$$

$$\mathbf{A}=(\mathbf{I}-\mathbf{B}{)}^{-1}$$

$${\mathbf{A}}_{ICA}=(\mathbf{I}-\mathbf{B}{)}^{-1}\mathbf{PD}=\mathbf{APD}$$

$$\mathbf{W}={\mathbf{A}}^{-1}=\mathbf{I}-\mathbf{B}$$

$${\mathbf{W}}_{ICA}=\mathbf{PDW}=\mathbf{PD}{\mathbf{A}}^{-1}$$

Full text

††journal: The 10th International Conference on Management of Emergent Digital EcoSystems (MEDES’18)

1 Introduction

Within the last decades, Japan’s economic growth has been greatly influenced by inbound tourism, creating new potential customers to local businesses across the country and touristic attractions. However, both natural disasters such as earthquakes, as well as the diminishing population in rural areas and its concentration in urban areas have led peripheral areas across the country to attempt regional revitalization via tourism [1]. One of these regional revitalization projects, subsidized by the government after a successful social experiment in the early 1990’s, was created as a way to establish strong links between road users and local communities. This was called ‘Michinoeki’, which stands for ‘Roadside Station’ in Japanese [2].

Michinoeki strives to act as a safe and comfortable space in which road travelers can refresh themselves (parking and restrooms); interact with local community and culture, tourist attractions and recreational activities and facilities; as well as provide travelers with relevant information such as maps, emergency care, et cetera. There is currently a network of $1145$ facilities of Michinoeki across different areas of Japan [3]. It is important then, in order to revitalize these areas that these facilities match the needs of travelers and tourists.

One notorious example, based in the Kumamoto prefecture in the Kyushu area is ‘Michinoeki Shichijo Melon Dome’, pictured in Figure 2. The local product is, of course, melon, and they have used it to make melon taste ‘Melon Pan’ bread (which is named after its shape and not the taste), ice cream and also sold on its own. The place also offers other local souvenirs and products, aside from melon-based ones.

Studies pertaining to Michinoeki in Japan are scarce, but within the literary review for this study we found that previous studies, such as one involving the classification of Michinoeki in Hokkaido [4], are mostly based on small surveys. There are additionally studies focusing on the implementation of the idea of Michinoeki in different parts of Asia, such as one in Korea [5], and another in Vietnam and China [6]. In addition to studies on Michinoeki, there has been focus on the use of regional branding for the revitalization of rural areas by using the local brand farm products as a tourist attraction across Japan [1, 7, 8, 9].

However, in recent years, electronic word-of-mouth (eWOM) has become an important resource for analysis of marketing research and has increasingly been approached by researchers for many products and services [10, 11, 12]. In addition to this new source of information, Machine Learning approaches, extracting vital information via text mining, among other methodologies of the information age are widely available and increasing in use as well [e.g. 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]. These methodologies, when applied to larger databases, provide more trustworthy results than those of statistically small questionnaire samples which can be influenced by the inflexibility of previously posed questions for the customer base. Furthermore, many studies regardless of topic study correlation between the sampled values, leaving to consideration if there is also causation involved. Recently, a Non-Gaussian methodology was presented by Shimizu in 2014, called LiNGAM [25], which allows for a clearer view at the causal relationship between our data.

Because of the importance of the subject of rural tourism to the revitalization of the economy across Japan, and the lack in use of better methodology in this field, we propose to use an Entropy-based Support Vector Machine Learning approach to classify and recognize different topics in eWOM related to Michinoeki extracted from Twitter, and then study the causal relationship between the amount of mentions in those topics and the sales of Michinoeki establishments in the Kyushu area of Japan.

2 Methodology

2.1 Word Segmentation

For an analysis to be made possible for each word, we segmented the collected Japanese texts without spaces into words using a Japanese morphological analyzer tool called MeCab [26]. After segmenting the words, we extracted only self-sufficient words.

2.2 Entropy Based Keyword Extraction

Feature selection of our method is based on the Shannon’s entropy (hereinafter referred as entropy) value [27] of each word. According to information theory, entropy is the expected value of the information content in a signal.

Applying this knowledge to the study of words allows us to observe the probability distribution of any given word inside the corpus. For example, a word that keeps reappearing in many different documents will have a high entropy, while a word that only was used in a single text and not in any other documents in the corpus will bear an entropy of zero. This concept is shown in Figure 2.

Having previously tagged a sample of texts positive and negative by pertinence to each category, if a word has higher entropy in positive documents than in negative documents by a factor of alpha greater than 1 ($\alpha>1$), then it means its probability distribution is more spread in positive texts, meaning that it is commonly used in positive tagged documents compared to negative ones.

To calculate the entropy in a set of documents, for each word $j$ that appears in each document $i$, we counted the number of times a word appears in positive comments as $N_{ijP}$, and the number of times a word appears in negative comments as $N_{ijN}$. Then, as shown in the formulas below, we calculated the probability of each word appearing in each document shown below as $P_{ijP}$ (1) and $P_{ijN}$ (2).

[TABLE]

[TABLE]

We then substitute these values in the formula that defines Shannon’s Entropy. We calculated the entropy for each word $j$ in relation to positive documents as $H_{Pj}$ (3), and the entropy for each word $j$ in relation to negative texts as $H_{Nj}$ (4). That is, all instances of the summation when the probabilities $P_{ijP}$ or $P_{ijN}$ are zero and the logarithm of these becomes undefined are substituted as zero into (3) and (4).

[TABLE]

[TABLE]

After calculating the positive and negative entropies for each word, we measured their proportion using the mutually independent coefficients $\alpha$ for positive keywords and $\alpha^{\prime}$ for negative keywords, for which we applied several values experimentally. A positive keyword is determined when (5) is true.

[TABLE]

2.3 Topic Classification Using SVM

In machine learning, Support Vector Machines are supervised learning models commonly used for statistical classification or regression [28]. We implemented this theory in Python using the Support Vector Classifier (SVC) included in the library scikit-learn. We also used the mathematics library numpy. To evaluate each of our trained machines, we used the K-fold Cross Validation method, which has been proven to provide good results. To evaluate each of our trained machines, we calculated then the Precision, Recall and $F_{1}$-Score values for our predictions.

2.4 Causal Relationship Analysis with LiNGAM

Most previous studies used a linear correlation model to analyze tourism data [e.g. 29, 30], and other traditional methods like the multiple variable regression analysis [31], while others have used improved regression models such as the quantile regression to attempt to overcome deficiencies of simple linear correlation models [32]. Correlation suggests an association between two variables. On the other hand, causality shows that one variable directly effects a change in the other. For analysis of tourism industry, it is important to detect cause of sales. Causal structure models have been studied for some time, and there are examples of tourism analysis analyzing predefined causal structures with SEM (Structural Equation Models) [33]. LiNGAM is currently a widely used model to discover causal structures of continuous-valued data without the need of defining them, under the assumptions that “the data generating process is linear”, “there are no unobserved confounders”, and “disturbance variables have non-Gaussian distributions of non-zero variances”.

The LiNGAM model is as follows:

[TABLE]

Where $k$ represents the causal order of each variable, and in which we are only considering causal orders in the desired direction. This model can also be expressed as:

[TABLE]

Where the matrix $\mathbf{x}$ is comprised of all the measured variables, including in our case the number of tweets and the sales profits; $e_{i}$ are continuous latent variables that are exogenous contained in the matrix $\mathbf{e}$, and $b_{ij}$ are the connection strengths from $x_{j}$ to $x_{i}$. If $b_{ij}$ is not equal to 0, $j$ is cause of $i$. Contrary to this, if $b_{ij}$ is equal to 0, there is no causality between $i$ and $j$, since $x_{i}$ would only be comprised of exogenous values (i.e. $x_{i}=e_{i}$). The matrix $\mathbf{B}$ contains all the strength constants and therefore, we must identify it to detect causal structure. Formulation (8) can be modified,

[TABLE]

In order to calculate the matrix $\mathbf{B}$, $\mathbf{x}$ is expressed in the form of the Independent Component Analysis, or ICA [34, 35] as follows:

[TABLE]

Where the ICA matrix $\mathbf{A}_{ICA}$ collects the coefficients $a_{ij}$, and $\mathbf{s}$ collects the independent components $s_{j}$ respectively. However, the output ICA matrix might result in different permutations at the time of calculation. Based on the ICA matrix $\mathbf{A}_{ICA}$, in order to identify a mixing matrix $\mathbf{A}$ such that

[TABLE]

The matrix $\mathbf{A}_{ICA}$ is expressed as

[TABLE]

Where $\mathbf{P}$ is an unknown permutation matrix and $\mathbf{D}$ is an unknown diagonal matrix with no zeros on the diagonal. The separating matrix $\mathbf{W}$ is defined as

[TABLE]

Following this, the separating matrix $\mathbf{W}$ is estimated up to the permutation $\mathbf{P}$, and scaling and sign $\mathbf{D}$ of the rows.

[TABLE]

However, in LiNGAM, the correct permutation matrix $\mathbf{P}$ can be found [36]: the correct $\mathbf{P}$ is the only one that contains no zeros in the diagonal of $\mathbf{D}\mathbf{W}$, since $\mathbf{B}$ should be a matrix that can be permuted to become lower triangular with all zeros on the diagonal and $\mathbf{W}=\mathbf{I}-\mathbf{B}$. Furthermore, the correct scaling and signs of the independent components can be determined by using the unity on the diagonal of $\mathbf{W}=\mathbf{I}-\mathbf{B}$. To obtain $\mathbf{W}$ it is only necessary to divide the rows of $\mathbf{D}\mathbf{W}$ by its corresponding diagonal elements. Finally, the connection strength matrix $\mathbf{B}=\mathbf{I}-\mathbf{W}$ may be computed.

3 Experiment Results

3.1 Multi-label Topic Classification

Before the analysis using Support Vector Machines, we defined eight topics by manually. In the sample of tweets related to Michinoeki, we noticed most of the tweets could be classified in these eight topics, with multiple labels in some cases, which are shown in Table 1. We made the training data by sampling 1000 posts and classifying them manually into eight topics. We then calculated the entropy value for each word in each category. With an alpha value of 2, ($\alpha=2$) we assigned a word as a keyword for a category only if the entropy value for that category was more than twice the entropy values of all other categories for that word. An exception was made for Topic 5: Check-in, for which the keyword extraction process was done heuristically, choosing keywords such as “I’m at”, indicating only that the user had gone to that particular Michinoeki, which had sufficiently good results.

The topic content and example keywords for each topic obtained from cross comparison of their Entropy values across categories are shown in Table 1.

Because all of the tweets related to Topic 3: Promotional are from official accounts, we categorized these automatically using the usernames without the need to train an SVM or extract entropy-based keywords. Regarding the Topic 9: Unrelated and Others, because of the way we extracted the tweets related to ‘Michinoeki’ which translates to ‘Roadside Stations’, there were cases where actual railroad stations were mentioned instead. We filtered these cases using entropy based SVM as well. Topic 6 describes positive reviews, which we had found differ from other topics; however, little to no negative reviews were found and thus not enough data was available to train a classifier for negative reviews.

To evaluate each of our trained machines, we calculated the $F_{1}$-Score for each using a K-fold Cross Validation methodology, calculating the $F_{1}$-Score from the values of Precision and Recall calculations. The $F_{1}$-Score values calculated for each SVM are shown in Table 2.

By searching by the names of Michinoeki establishments plus the keyword “michinoeki” in Japanese, we collected a total of $111,142$ tweets related to Michinoeki, of which $9264$ were related to 94 different Michinoeki establishments across the Kyushu area of Japan for which we had sales data. We then automatically classified posts into one of the eight topics shown on above by using Support Vector Machines in a hierarchical manner. We first classified the tweets belonging to official accounts automatically to Topic 3: Promotional; then classified tweets from Topic 5: Check-In for their particular structure; followed by classification of Topics 4, 7 and 8, which are mostly about external factors and topics unrelated directly with Michinoeki content; finally proceeding to determine positive pertinence to each remaining topic by their respective SVM in order of their $F_{1}$-Score. This heuristic hierarchical binary classification method is shown in Figure 3.

3.2 LiNGAM Analysis Results

To evaluate the causal relationship between the number of tweets for each of the topics related to Michinoeki as $x_{i}$ and the sales of Michinoeki establishments in the Kyushu area of Japan as y, we applied the LiNGAM causal structure analysis to the classified data from the $9264$ tweets that were matched to those establishments. Results are shown in Table 3.

4 Discussion

According to the LiNGAM causal structure analysis results, the number of tweets mentioning products and services of Michinoeki establishments, positive reviews of the establishment and their products, as well as promotional tweets published by official accounts and mentions of special events all show a causal relationship with the sales of those establishments. This positive causal relationship is thought to be the influence that twitter mentions have to attract more new customers. As for the tweets from Topic 5: Check-in, there is a clear direct influence, since those users are not only stating that they actually visited and purchased from those establishments, but they are also giving free promotion to the users that follow them.

Tweets mentioning weather and traffic conditions were observed in our sample to be of mostly expressing inconvenience, which, as is shown by the negative causal constant, has a negative effect on the number of customers that venture to the establishment during those hours, limiting its profit. Tweets in this category also included many complaints about access to the establishments. This shows an opportunity of investment to make clearer signs or routes of access for the affected establishments, as well as marketing campaigns (perhaps using Twitter as well) making their location and access routes more known.

One thing to note is that under inspection of our data, many of the places where there are positive reviews of ice cream and soft serve products are leaders in profitability compared to other establishments. Local ingredients, as well as unique recipes are the main focus of Michinoeki establishments in general, but there could be differences in influence for different kinds of products and specialties.

Lastly, there is the particular case of the tweets by motorcycle drivers. While Michinoeki establishments strive to be a connection point for road travelers and tourists alike, many bikers use the stations as meeting points only, not contributing to the sales of those establishments. It is necessary in these cases to revise the services provided, such as gasoline stations, for example, so that this untapped customer base can be turned to a positive influence in the future. Another possible strategy could be group campaigns. Most examples in our data of motorcycle drivers were using the place as a meeting point with other motorcycle drivers. Group campaigns or discounts could very well increase their patronage as well as attract more customers in general.

5 Conclusion and Future Work

We found that the tweets related to Michinoeki could be classified into eight topics with a well performing hierarchical and heuristic approach for multi-class classification using binary SVM classifiers; for which the feature vectors were extracted heuristically in one case, and mathematically in all the other cases by using an entropy-based keyword extraction method. All of the SVM based classifications performed with an $F_{1}$-Score above 0.78, and the highest performing classifier was that of the Topic 4: Traffic and Weather.

Under the assumptions posed by the LiNGAM model (“the data generating process is linear”, “there are no unobserved confounders”, and “disturbance variables have non-Gaussian distributions of non-zero variances”), we found a causal relationship for all topics and found that most tweets, especially ones praising products, or promoting special events, have a positive influence in sales, with the exception of traffic and weather, and motorcycle travelers, which might be an unexplored market by Michinoeki establishments. However, in future work we will further investigate the assumptions posed by the LiNGAM model in regard to the structure of the data.

In future work we will study the influence of specific products, their different marketing strategies across different establishments and their relation to their profits in order to discover potential strategies to increase profit in all Michinoeki establishments.

ACKNOWLEDGMENTS

This research was supported by Japan Construction Information Center Foundation (JACIC).