26
In the overall architecture, a new "hyper-convergence" concept was proposed, which integrated multiple engines and capabilities to meet the increasingly complex mixed load scenarios and business needs of mixed data type for enterprises.
At the mean time, to support users’ increasingly demanding requirements, through the cloud-native "stream processing engine", stream computing and database are integrated and innovated from the kernel level, meeting the needs of real-time business analysis, real-time reporting and other types of asynchronous transaction scenarios such as Internet+.
Figure 1: Hyper-convergence architecture of AntDB
In our point of view, HTAP, lake/warehouse integration and stream/batch integration are only transitional products on the evolution of database. The future database will put “data” at its core and integrate with various business data to meet IT systems and industrial data in China.
Ⅱ. Integration of five application requirements
Traditional transaction and analysis, streaming processing, time-series and memory computing
The current application demands of users for data mainly lie in five aspects: traditional transaction, analysis, big data mining, high-performance memory computing and real-time streaming data processing. These five aspects are carried separately by different technology stacks. HTAP is attempting to integrate transactions and analyses while AntDB aims to bring the five types of data services under a unified technical framework, making one product a "one-stop service" for users.
Figure 2: Integration of five data application requirements
The hyper-convergence framework of database proposed by AntDB V7.2 can make full use of the architectural advantages of distributed database engines and further expand on the concept of HTAP to encapsulate multiple engines such as time-series storage, stream processing execution and vectorized analysis in a unified architecture. Supporting multiple business models in the same database cluster greatly reduces the complexity of supporting diverse data requirements for business systems and brings convenience to application developers as well as DBAs and architects.Ⅲ. Born to change
AntDB’s new-generation streaming processing engine
If the hyper-convergence framework is the basic architecture of database and does not solve a specific type of scenario problem, then AntDB-S streaming data engine of AntDB V7.2 completely overturns the design and development model of real-time computing applications.
The problems of difficult development and high cost of streaming business affect the rapid promotion and implementation of streaming computing in actual production. For developers, whether it is Apache Storm, Spark Streaming, or Flink and other mainstream stream processing framework design, all focus on the "processing" itself. Since they do not have database capabilities, they require complex data extraction when they need to interact with other data for correlation and temporary storage,and manual processing operations is required inside the stream processing engine via Java/Scala program code.
After more than ten years of practical experience in core business scenarios of operators, AntDB found that some business scenarios cannot be realized by traditional technology and need real-time processing capabilities such as streaming computing to support.
Therefore, AntDB has done a lot of innovative explorations and researches from scratch to integrate stream computing into the database kernel. Users can freely define the structure of data and real-time processing logic through standard SQL under the framework of one database engine; meanwhile, data can flow freely between stream objects and table objects inside the database; users can conduct performance optimization, data processing, cluster monitoring and business logic customization of data by establishing indexes, stream-table association, triggers, materialized views, etc. at any time.
Figure3: AntDB’s streaming processing scenario
Under AntDB's all-in-one streaming engine mechanism, developers can get rid of the complexity of real-time business development and do not need to use Java/Scala code to define data processing logic; and for operation and maintenance personnel, they can achieve the goal of "one product to meet multiple data processing types", which greatly reduces the complexity of the overall technical framework, and the security stability and development efficiency of the system are improved.Ⅳ. Forged for the future
Stepping into the new age of evolutional distributed database
AntDB database is a product born in a special scenario, which has experienced 15 years of polishing in the core scenario of operators, and in the process of continuous iteration and upgrading of China's communication technology and Internet technology, it has been cultivating its skills in the core transaction scenario of ultra-high frequency and high density in the communication industry, and solving the demand problems that international brand databases can hardly cope with.
As a representative of domestic database, AntDB is also shouldering the mission of information technology innovation, from behind the scenes to the front stage, through its inherent security, stability and foresight to future application scenarios and cutting-edge technologies, we can see that domestic database will march to a very promising new era and blaze a database development trail of its own.
