An Approach to Log Management: Prototyping a Design of Agent for Log Harvesting

TL;DR

This paper presents a prototype system for harvesting, normalizing, and transporting logs from network devices in an e-science environment, using agents and a custom protocol to improve traditional log management.

Contribution

It introduces a novel agent-based architecture with a transportation protocol for efficient log harvesting and management in scientific networks.

Findings

Prototype successfully captures and transmits logs

Agents normalize and catalog logs effectively

System addresses deficiencies of traditional log protocols

Abstract

This paper describes a work in progress implementing a solution for harvesting and transporting information logs from network devices in a e-science environment. The system is composed for servers, agents, active devices and a transporting protocol. This document describes the state of development of agents. Agents capture logs from devices, normalize, reduce and cataloged them by using metadata. Once all these processes are done, they transmit the cataloged data by using Transportation Protocol to a warehouse server. Also an agent use orchestration parameters to transmit modified logs to a data warehouse server. These parameters can be received from orchestration applications such as Taverna. The operation of the agents and the communication protocol solve some of the deficiencies of traditional logs management protocols. Finally, we show some test realized over the new prototype.