Data Processing

Introduction

The AcumenSolution system aim to be cloud based application (even though it can be hosted in on-promise type of contract) and to gather information about the assets it has to have data collection channel prepared. Local data reading units send all necessary data to the REST-based services.


Architecture

To support continuous asset condition evaluation for multiple yet unknown datasources the enhanced data processing framework had to be designed. The dynamic set of rules, dynamic parameters and even dynamic querying was created to allow such functionality.

Data processing archtecure overview

The principal of this architecture is dynamic set of services that contain number of data processing algorithms that can be registered in the system and then launched by the user, scheduled job or run continuously as it was used for the dashboard functionality.

Technologies

The whole new light-weight job management framework was created, where communication was based on Bus Service (here Azure Bus Service). Depending on the mode the results are stored in the database or cached in Redis (specifically for the Continuous mode).

All trademarks are the property of their respective owners


Communucation channel
Data processing services framework
Result chasing