ES

SNGULAR Data Acceleration Architecture

Challenges

Access to mainframe systems remains a handicap in large multinational organizations. Being able to use host information to create rich visualizations in user interfaces or access data efficiently and inexpensively are two of the most common challenges.

SNGULAR Data Acceleration Architecture is a distributed platform for managing and propagating historical and real-time data. This cache provides a layer of services for digital channels and reporting tools through a series of pipelines that fetch and refresh data. 

Elastic infrastructure

Combined with containerized deployments and cloud environments, it can create an elastic infrastructure that can respond to peaks in interface consumption without significant economic impacts on mainframe access.

 

The platform provides a solution to the traditional constraints of backend systems and meets the needs that end-users demand from digital channels and following the CQRS pattern.

 

By design, it stores all data (historical and current), processes information in real-time, and offers more effective search capabilities and better response times. It is a read-only replica of the data stored in the system, to which certain tags are added to manage and maintain the context.

Our data processing architecture can collect data from many different sources such as:

All of this allows the information to be collected and transmitted to a centralized cluster. It also allows the temporary information storage to coordinate this data through the different processing stages of our solution.

 

In this centralized cluster, each data stream is serialized in Avro according to a schema. That helps to validate that the data format is correct.

 

The information is distributed through topics, including messages representing a specific category or type, and is replicated through several brokers to prevent data loss.

These brokers are nothing more than automatically interconnected servers, which allows for rapid scaling in the case of increased processing needs.

 

On the other hand, a streaming API is enabled. This API allows simple operations to be performed on the data in real-time, such as filtering, aggregating, restructuring, or merging messages before they are consumed or stored. This "preprocessing" helps optimize the time and computational cost of subsequent analytics or visualization solutions. 

 

Finally, the information is deposited in a single data warehouse, being available in near-real-time. It is organized in a structured and easily consumable form. BI systems can exploit that data to create functional, intuitive, and eye-catching dashboards or reports that can be exported and shared.

Likewise, that same information can be used by machine learning systems to generate predictive models or infer analytics. The output of these machine learning systems can, in turn, lead to the creation of new tables or collections, which can also be used by the same BI and data visualization tools.

The main benefits of the SNGULAR Data Acceleration platform

  • Support for all major search models

     The engine allows searching transactions based on open text fields, by amounts, by concepts, categories, and any combination. These search capabilities apply to the entire transaction history because they can be extended to all datasets.

  • Continuity of operations

    In case of mainframe maintenance or mainframe downtime, the platform will allow end-users to at least access the information and will work in read-only mode. Operations persisting data modifications will not be possible, but the system will be accessible.

  • Independence of deployment models, on-premise, in a public or private cloud

    The use of containers allows the platform to be deployed in the infrastructure model chosen by the customer.

  • Near real-time updates

    The use of data streaming techniques means that the information remains updated, if not in real-time, then in near real-time (NRT).

     

  • Cost savings

    Reduced consumption of reading operations against the host, making it more economically efficient. 

     

  • Built on market standards and open products

    That avoid Vendor Lock-in.

  • Prepared for testing automation

    Integration, and continuous deployment circuits, which accelerate development cycles.

Finally, the SNGULAR Data Acceleration Architecture can be a layer to build value-added services for end customers (such as personal finance management platforms) or even give rise to monetization models based on access to queries.

© Copyright - Sngular 2021

facebook youtube instagram twitter linkedin