A Framework for Capturing and Analyzing Unstructured and Semi-structured Data for a Knowledge Management System

TL;DR

This paper presents a comprehensive framework that captures, processes, and analyzes various types of unstructured and semi-structured data using machine learning, enhancing knowledge management systems for organizational decision-making.

Contribution

The framework uniquely integrates multiple data types and employs machine learning techniques, providing a versatile tool for knowledge extraction across diverse unstructured data sources.

Findings

Successful development of a textual data API using Python and Beautiful Soup.

Effective sentiment analysis performed on student review data.

Framework demonstrates versatility across different unstructured data types.

Abstract

Mainstream knowledge management researchers generally agree that knowledge extracted from unstructured data and semi-structured data have become imperative for organizational strategic decision making. In this research, we develop a framework that captures and analyses unstructured data using machine learning techniques and integrates knowledge and insight gained from the data into traditional knowledge management systems. Unlike most frameworks published in the literature that focuses on a specific type of unstructured data, our frameworks cut across the varieties of unstructured data ranging from textual data from social network sites, online forums, discussion boards, reviews to audio data, image data and video data. We highlight some pre-processing and processing techniques for these data and also highlight some standard output. We evaluate the framework by developing a textual data application programming interface (API) using python and beautiful soup and we perform sentiment analysis on the students review data collected through the API.