473,289 Members | 1,884 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes and contribute your articles to a community of 473,289 developers and data experts.

Your Detailed Guide to Apache ShardingSphere ’s Operating Modes

In Apache ShardingSphere 5.0.0 GA version, we added the new concept Operating Mode and provided three configuration methods: Memory, Standalone, and Cluster. Why does ShardingSphere provide these operating modes? What are the differences between them in actual development scenarios?

This article is a guide for you to better understand ShardingSphere’s new operating modes.

Background: Distributed Governance
Distributed governance is the foundation of cluster deployment in ShardingSphere. In previous versions, users needed to configure the governance tag in the configuration file to enable distributed governance:
Expand|Select|Wrap|Line Numbers
  1. governance:
  2. name: # Governance Name
  3.  registryCenter: # Configuration Center 
  4.  type: # Governance Persistence Governance Type such as Zookeeper             and etcd
  5. serverLists: # Governance Service Lists, including IP address and   port number (comma-separated), such as host1:2181,host2:2181 
  6. overwrite: #Decide whether the local configuration overwrites the configuration of the configuration center. If yes, after every startup, the local configuration still works.
The most important features of distributed governance include persistent user configuration and metadata.

They are also the basic capabilities supporting Distribured SQL (DistSQL). In the previous ShardingSphere 5.0.0 stories, the core developers of DistSQL have shared the concept of DistSQL , its syntax, and its usage in detail, and they have showcased how you can develop your own DistSQL.

To recap, DistSQL gives ShardingSphere’s users a database-like experience: users can use DistSQL to build and manage the entire ShardingSphere distributed database ecosystem.

Like other standard SQLs, DistSQL, known as the operating language of the distributed database ecosystem, needs to ensure that any configuration and operations metadata can be persisted to keep data consistency when the system is restored.

In previous versions, only when you enabled distributed governance could the feature be implemented. That’s the reason why DistSQL was only available in the distributed governance scenario in its early development stage.



Why We Created Operating Modes

Based on the cluster deployment capability given by the current distributed governance function, ShardingSphere now redefines its distribution capability as Cluster Mode.

The cluster mode supports ShardingSphere as a stateless compute node for multi-instance deployment and with a register center, it can synchronize metadata of all instances in the cluster in real-time.

The mode naturally supports DistSQL: under the mode, you can use DistSQL to perform operations on computing/storage nodes such as node online/offline or disabled.

In the past, DistSQL was restricted to distributed scenarios. To fix the issue, ShardingSphere first needs to figure out how metadata can be stored in a non-distributed environment. The simplest solution is to write metadata to local files and therefore when a service restarts, metadata can be loaded from local files according to different configurations.

Unlike the cluster mode used in the distributed scenario, local files cannot share configurations among multiple ShardingSphere instances in real-time. In Standalone Mode, all configuration updates only work in respective instances.

ShardingSphere 5.0.0 not only provides users with better features but also builds stable and user-friendly APIs to optimize user experiences.

In addition to Cluster Mode and Standalone Mode, another useful mode is called Memory Mode. Why did we design it? Because some users need to quickly start the integration of ShardingSphere but don’t need persistent configuration. For example, some may use ShardingSphere to quickly verify some functions, or just want to test integration. Given such a scenario requirement, we created Memory Mode.

So far, ShardingSphere has three modes, i.e. Memory, Standalone, and Cluster. The operating modes are not difficult to understand in terms of our API design, and they are perfectly suitable in the actual use case scenarios of ShardingSphere. Additionally, the three operating modes can support DistSQL to quickly build and manage distributed database services.

The governance configuration method is removed from the 5.0.0 version, and instead, we start to use the different operating modes.
Expand|Select|Wrap|Line Numbers
  1. mode:
  2.  type: # Mode Type Standalone/Cluster
  3.  repository:
  4.  type: # Persistence Type, Different modes have different implementations: Standalone-File and Cluster-ZooKeeper/Etcd
  5.  props: # Different persistence types have different user-defined configurations. 
  6.  …
  7.  overwrite: false # Decide whether to use local configuration to overwrite local/remote configuration
Next, I’d like to explain the basic concepts of the three operating modes in detail, and show you how to choose the right operating mode when you use ShardingSphere for development.

Concepts and Application Scenarios

Memory Mode
Memory is the default operating mode, so you do not need to configure mode. With this mode, users do not need to configure any persistence components or strategies because any metadata change caused by local initialization configuration or SQL/DistSQL operation, only works in the current thread, and the configurations are restored after the service restarts.

The Memory mode is perfect for integration testing: it’s convenient because developers don’t have to clean running traces after they integrate ShardingSphere and have integration testing.

Standalone Mode

ShardingSphere’s Standalone Mode provides local files with a persistence method by default. It can persist metadata information (e.g.data sources and rules) to local files so even when the service restarts, configurations can still be read from the local files to ensure metadata consistency.

The Standalone mode makes it convenient for development engineers to quickly build a local development environment for ShardingSphere, test integration and verify features.
The mode’s configuration is shown as follows:

Expand|Select|Wrap|Line Numbers
  1. mode:
  2.  type: Standalone
  3.  repository:
  4.  type: File
  5.  props:
  6.  path: …
  7.  overwrite: false
The Standalone mode can persist local files by default. Configurations are persisted in the user directory .shardingsphere by default, but you can also customize your storage path by configuring path.

Cluster Mode

We recommend you apply Cluster Mode in a real deployment and production environment. Moreover, if you adopt hybrid deployment architecture with both JDBC and Proxy, you must use the Cluster mode.

The mode can provide distributed governance capability. By integrating an independently-deployed third-party register center, the mode can realize metadata persistence, share data between multiple instances, and implement state coordination in a distributed scenario. Cluster mode is also the reason why ShardingSphere’s horizontal scaling can greatly enhance computing capabilities and lay the foundation of core features like high availability.

We take Zookeeper as the example, to demonstrate mode configuration:
Expand|Select|Wrap|Line Numbers
  1. mode:
  2.  type: Cluster
  3.  repository:
  4.  type: ZooKeeper
  5.  props:
  6.  namespace: governance_ds
  7.  server-lists: localhost:2181
  8.  retryIntervalMilliseconds: 500
  9.  timeToLiveSeconds: 60
  10.  maxRetries: 3
  11.  operationTimeoutMilliseconds: 500
  12.  overwrite: false
We also compare the differences between the three modes (shown in the table below). Our suggestion is that you consider your needs first and then choose the right mode.



Summary
ShardingSphere’s three operating modes can meet virtually all user needs in various environments from testing, to development, to deployment.

Combined with ShardingSphere’s remarkable pluggable architecture, developers can also flexibly customize the persistence methods of each mode and create their own operating modes to make operating modes more suitable for their development and business needs. If you are interested in distributed governance, feel free to reach out to the ShardingSphere community.

Apache ShardingSphere Open Source Project Links:
ShardingSphere Github
ShardingSphere Twitter
ShardingSphere Slack Channel
Contributor Guide
Sep 6 '22 #1
0 9118

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: dave | last post by:
Hello, Not sure if this is an apache configuration or a php configuration problem, i'm hoping someone can help. I'm running a webserver on FreeBSD consisting of apache2 and php4 among other items....
5
by: Phil Powell | last post by:
print_r(is_file("$logPath/$logFileName")); // RETURNS 1 unlink("$logPath/$logFileName"); // RETURNS WARNING: PERMISSION DENIED This code should tell me that the file located at...
3
by: New to PHP | last post by:
I have 3 computers with Ethernet connection to a local router box(SMC7008ABR) and on the wan side to Verizion DSL. I was able to install Apache and PHP on one of the PC with XP Home edition. How...
48
by: Foobarius Frobinium | last post by:
http://thelinuxlink.net/~fingolfin/pointer-guide Tell me what you think...
5
by: raisins | last post by:
This is kind of an obscure question, and I hope someone has a clue what's going on. I've got PHP4 running on Apache 1.3 on win32, and whenever I request a URL like so: http://localhost/tag/ It...
5
by: m.banaouas | last post by:
Hi, bonjour, witch versions are suitable to use for apache & mod_python ? Can i install and use "Apache 2.2.3" & "mod_python 3.2.10" (most recent versions) without facing any known major...
6
by: Liming | last post by:
Hi, In a typical 3 tier model (view layer, busines layer and data access layer) where do you handle your exceptions? do you let it buble up all the way to the .aspx pages or do you handle it in...
0
by: tochisan | last post by:
Hello all, I'm writing a c# console program that's going to be run as a service but I want it to have the ability to handle arguments and run in a certain way. The program is supposed to monitor...
4
by: Mark Space | last post by:
Hi all, I'm looking for the docs on the php module. Ol' Apache says that the docs for LoadModule come from that module (PHP in this case) but unfortuanlely I can't find it. Is php5apache2_2.dll...
0
by: Yacine Si Tayeb | last post by:
This post analyzes ops & maintenance solutions with experiences taken from real production scenarios for data sharding and other functions provided by ShardingSphere-Proxy Version 5.1.0. Unless...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.