473,289 Members | 1,952 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes and contribute your articles to a community of 473,289 developers and data experts.

ElasticJob 3.0.2 is released including failover optimization, scheduling stability, and Java 19 compatibility

1. Introduction

ElasticJob, one of the sub-projects of the Apache ShardingSphere community, is a distributed scheduling solution oriented towards Internet applications and massive tasks.

Since ElasticJob 3.0.1 was released, we’ve received a lot of feedback from users on GitHub. After merging the code committed by the contributors, ElasticJob has been significantly optimized.

The resulting ElasticJob 3.0.2 has been improved in terms of Failover, scheduling stability, job API, and Java 19 compatibility.

2. Release 3.0.2 Key Features

2.1 Failover Optimization

Failover is an important ElasticJob feature.

In ElasticJob 3.0.1, the Failover feature had some limitations and disadvantages:
  • Failover could take effect with at least two job instances. Assume that there’s only one job instance executing tasks and a fault occurs, Failover cannot take effect even if the instance recovers or a new job instance is started after the fault occurs. In this case, the old Failover feature required redundant resources. Particularly, if sharding was not needed for the job, the backup instance would be idle for a long time and continuously occupy resources.
  • Failover was sensitive. Even if the job was not in execution, as long as one instance goes offline, failover of another instance would still be triggered to immediately execute the sharding owned by the offline instance. In other words, failover was triggered by the offline instance whether it is a normal offline or a fault occurs, which is disturbing for users.

ElasticJob 3.0.2 optimized the Failover feature.
  • Suppose a single job instance is executing the tasks and it exits due to a fault in the execution process. After a fault occurs, a new job instance is started, which can detect the failed sharding and trigger Failover immediately. For example, we deploy a job instance with one copy in the Kubernetes environment. If the job instance exits unexpectedly due to a fault, Kubernetes automatically starts a new job instance. By this time, Failover will be triggered and continue to execute the previously failed job.
  • The optimized Failover takes effect only for ongoing jobs. If no job is being executed when a fault occurs, Failover is not triggered. In this case, Failover will not be triggered when the job instance goes offline normally.

2.2 Separating the event notification threads of different jobs to avoid an endless loop.

ElasticJob achieves distributed coordination through ZooKeeper. In practical scenarios, users may start multiple jobs in the same project simultaneously, all of which use the same Apache Curator client. There are certain risks due to the nature of ZooKeeper and the callback method of Curator in a single event thread.
  • Callbacks will increase accordingly if a huge number of jobs are triggered frequently. The processing capability of Curator event threads may reach an upper limit, resulting in delayed triggering or failure to trigger.
  • If one job blocks the event thread accidentally, other jobs will also be affected, leading to delayed triggering or failure to trigger.

In ElasticJob 3.0.2, the callback method of each job will be executed on the thread to which the job belongs, preventing jobs from affecting each other.

Moreover, ElasticJob 3.0.2 modified the code which may lead to an endless loop. When a job instance is online and the server nodes in ZooKeeper change (such as being deleted), ElasticJob is stuck in determining whether the server is enabled. This problem has been tackled.

2.3 Java 19 Support
Java 19 was released on September 20, 2022. ElasticJob’s code closely follows Java versions and currently can be built and used through Java 19. Elasticjob now supports Java 8 through 19.

3. Release Notes

3.1 Bug Fixes
  • Fix itemErrorMessages not cleared after the job finished.
  • Fix Curator notify thread may be blocked and avoid probably endless loop in ServerService.
  • Fix the problem that NPE may occur in the deserialization of job instance ID and job configuration.
  • Fix failover to sensitive.

3.2 Enhancements
  • Script Job exception’s stack was ignored.
  • Support using different event trace data sources when using Spring Boot.
  • Supports building projects with Java 19.

Please refer to GitHub ElasticJob Milestone for details.

Useful Links
🔗 ElasticJob Milestone

🔗 Release Notes

🔗 ShardingSphere ElasticJob GitHub Address

🔗 ShardingSphere Project Address

🔗 ShardingSphere ElasticJob Official Website

Author

Wu Weijie, is an infrastructure R&D engineer at SphereEx, and an Apache ShardingSphere PMC.

He now focuses on the R&D of the Apache ShardingSphere access port and the sub-project ElasticJob.
Nov 2 '22 #1
0 8539

Sign in to post your reply or Sign up for a free account.

Similar topics

133
by: Gaurav | last post by:
http://www.sys-con.com/story/print.cfm?storyid=45250 Any comments? Thanks Gaurav
5
by: mayamorning123 | last post by:
A comparison among six VSS remote tools including SourceOffSite , SourceAnyWhere, VSS Connect, SourceXT, VSS Remoting, VSS.NET To view the full article, please visit...
0
by: bettervssremoting | last post by:
To view the full article, please visit http://www.BetterVssRemoting.com Better VSS Remote Access Tool including SourceOffSite, SourceAnyWhere and VSS Remoting This article makes a detailed...
29
by: keredil | last post by:
Hi, Will the memory allocated by malloc get released when program exits? I guess it will since when the program exits, the OS will free all the memory (global, stack, heap) used by this...
2
by: LandWolf | last post by:
Hi, I have a Web services write in Java and I want to write a client in C#. My web services work with Weather prevision. Hence, I set a date and my web services return the prevision for that date....
2
by: Jobs | last post by:
Download the JAVA , .NET and SQL Server interview with answers Download the JAVA , .NET and SQL Server interview sheet and rate yourself. This will help you judge yourself are you really worth of...
0
by: Yacine Si Tayeb | last post by:
Our new 5.2.0 release enhances features such as SQL audit, elastic migration, SQL execution process management, and data governance on the cloud. Introduction Since Apache ShardingSphere...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.