473,287 Members | 1,899 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes and contribute your articles to a community of 473,287 developers and data experts.

Hyperconverged All-in-One Streaming Engine, ushering in new age for distributed database

26 16bit
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine
In the overall architecture, a new "hyper-convergence" concept was proposed, which integrated multiple engines and capabilities to meet the increasingly complex mixed load scenarios and business needs of mixed data type for enterprises.
At the mean time, to support users’ increasingly demanding requirements, through the cloud-native "stream processing engine", stream computing and database are integrated and innovated from the kernel level, meeting the needs of real-time business analysis, real-time reporting and other types of asynchronous transaction scenarios such as Internet+.

Figure 1: Hyper-convergence architecture of AntDB
In our point of view, HTAP, lake/warehouse integration and stream/batch integration are only transitional products on the evolution of database. The future database will put “data” at its core and integrate with various business data to meet IT systems and industrial data in China.

Ⅱ. Integration of five application requirements
Traditional transaction and analysis, streaming processing, time-series and memory computing

The current application demands of users for data mainly lie in five aspects: traditional transaction, analysis, big data mining, high-performance memory computing and real-time streaming data processing. These five aspects are carried separately by different technology stacks. HTAP is attempting to integrate transactions and analyses while AntDB aims to bring the five types of data services under a unified technical framework, making one product a "one-stop service" for users.

Figure 2: Integration of five data application requirements
The hyper-convergence framework of database proposed by AntDB V7.2 can make full use of the architectural advantages of distributed database engines and further expand on the concept of HTAP to encapsulate multiple engines such as time-series storage, stream processing execution and vectorized analysis in a unified architecture. Supporting multiple business models in the same database cluster greatly reduces the complexity of supporting diverse data requirements for business systems and brings convenience to application developers as well as DBAs and architects.
Ⅲ. Born to change
AntDB’s new-generation streaming processing engine
If the hyper-convergence framework is the basic architecture of database and does not solve a specific type of scenario problem, then AntDB-S streaming data engine of AntDB V7.2 completely overturns the design and development model of real-time computing applications.

The problems of difficult development and high cost of streaming business affect the rapid promotion and implementation of streaming computing in actual production. For developers, whether it is Apache Storm, Spark Streaming, or Flink and other mainstream stream processing framework design, all focus on the "processing" itself. Since they do not have database capabilities, they require complex data extraction when they need to interact with other data for correlation and temporary storage,and manual processing operations is required inside the stream processing engine via Java/Scala program code.

After more than ten years of practical experience in core business scenarios of operators, AntDB found that some business scenarios cannot be realized by traditional technology and need real-time processing capabilities such as streaming computing to support.

Therefore, AntDB has done a lot of innovative explorations and researches from scratch to integrate stream computing into the database kernel. Users can freely define the structure of data and real-time processing logic through standard SQL under the framework of one database engine; meanwhile, data can flow freely between stream objects and table objects inside the database; users can conduct performance optimization, data processing, cluster monitoring and business logic customization of data by establishing indexes, stream-table association, triggers, materialized views, etc. at any time.

Figure3: AntDB’s streaming processing scenario
Under AntDB's all-in-one streaming engine mechanism, developers can get rid of the complexity of real-time business development and do not need to use Java/Scala code to define data processing logic; and for operation and maintenance personnel, they can achieve the goal of "one product to meet multiple data processing types", which greatly reduces the complexity of the overall technical framework, and the security stability and development efficiency of the system are improved.
Ⅳ. Forged for the future
Stepping into the new age of evolutional distributed database
AntDB database is a product born in a special scenario, which has experienced 15 years of polishing in the core scenario of operators, and in the process of continuous iteration and upgrading of China's communication technology and Internet technology, it has been cultivating its skills in the core transaction scenario of ultra-high frequency and high density in the communication industry, and solving the demand problems that international brand databases can hardly cope with.

As a representative of domestic database, AntDB is also shouldering the mission of information technology innovation, from behind the scenes to the front stage, through its inherent security, stability and foresight to future application scenarios and cutting-edge technologies, we can see that domestic database will march to a very promising new era and blaze a database development trail of its own.
May 24 '23 #1
0 8777

Sign in to post your reply or Sign up for a free account.

Similar topics

1
by: channa_s | last post by:
Hi, I would be very grateful if anyone could help me with the following. I want to create a SIMPLE distributed database. It is as follows: There is a institution which has branches all...
7
by: J Goldman | last post by:
I'm looking for documentation pointers to learn what I need to put together a distributed database system. I've read through "Oracle 9i Database Administrator's Guide: Distributed Database...
1
by: Evan Smith | last post by:
Years ago I worked on a system which I believe was one of the first instances of EEE being used in an OLTP environment. It was fraught with lots of gotchas, but overall was made to work. This was...
2
by: thunder | last post by:
Hey! Suggest me. What are needed to creat a distributed database library. Protocol,design,and implementation,etc... I am a master thesis student.
1
by: sakinam | last post by:
i want to create a distributed database for a property selling agent. i need some feedback on how to go about it n which is d best database platform to use. d firm has three branches in different...
1
by: nicky123 | last post by:
A distributed database is a specialization of distributed computing. A distributed database allows certain data specific operations to be distributed to one or more different machines. This means...
3
by: lucoin | last post by:
I have a project to build a distributed database for an organization which run to branches located in Sydney and Melbourne. Sydney office is considered as the head office and Sydney branch office as...
0
by: antdb | last post by:
On December 28, the special session for databases of “Full-Stack Empowerment Through Digital Intelligence" - AsiaInfo 2022 Series Product Launch Event was held online. It is worth noting that...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.