Background Jobs Done Right with Quarkus
Learn how to make your Java apps resilient, efficient, and production-ready with smart scheduling strategies.
Enterprise applications frequently need to perform tasks automatically in the background or at specific times. Common examples include generating daily reports, sending periodic email notifications, synchronizing data with external systems, or performing regular cleanup operations. These scheduled and background jobs are critical components, and they must be reliable, efficient, and resilient to failures.
Quarkus provides robust mechanisms for handling these requirements. You primarily have two choices, each suited for different scenarios:
Quarkus Scheduler (
@Scheduled
): A lightweight, annotation-driven scheduler built into Quarkus core. It's simple to use, requires minimal configuration, and is ideal for straightforward, in-memory scheduling needs within a single application instance.Quarkus Quartz Extension (
quarkus-quartz
): An integration with the mature and feature-rich Quartz Scheduler library. It offers advanced capabilities like persistent job storage (e.g., in a database), clustered job execution across multiple application nodes, and complex, programmatic scheduling logic.
This article looks at how to effectively implement scheduled and background tasks in Quarkus using both approaches. I'll cover common use cases, look into handling failures with retries and fallbacks, discuss strategies for offloading long-running tasks, and touch upon monitoring essentials.
Choosing the Right Tool: Quarkus Scheduler vs. Quartz
Selecting the appropriate scheduling mechanism depends heavily on your application's specific requirements. Here’s a breakdown to guide your decision:
Decision Guidelines:
Use Quarkus Scheduler (
@Scheduled
) when:Your scheduling needs are simple (e.g., fixed rate, delay, standard cron).
Jobs don't need to survive application restarts (in-memory is acceptable).
You are running a single instance, or it's acceptable for the job to run concurrently on all instances in a cluster (e.g., cleanup tasks that are idempotent).
You prefer the simplicity of annotations and integration with MicroProfile Fault Tolerance.
Use Quarkus Quartz Extension when:
You need persistence: Jobs and their schedules must survive application restarts or crashes.
You need clustering: Only one node in a cluster should execute a specific job instance at a time (high availability, avoiding duplicate work).
You require complex scheduling logic that goes beyond simple cron or fixed rates, potentially determined dynamically at runtime.
You need fine-grained control over job execution, interruption, and state management.
Implementing Jobs with Quarkus Scheduler (@Scheduled
)
The built-in scheduler is the easiest way to get started. Simply annotate a method within a CDI bean (e.g., @ApplicationScoped
) with @Scheduled
.
Key Scheduling Attributes:
cron
: Defines a Unix-like cron expression for precise scheduling (e.g.,"0 15 10 * * ?"
for 10:15 AM daily).every
: Specifies a frequency or interval (e.g.,"10s"
for every 10 seconds,"5m"
for every 5 minutes,"2h"
for every 2 hours). Supports ISO 8601 duration format.delayed
: Specifies an initial delay before the first execution (e.g.,"30s"
). Can be combined withevery
.identity
: Assigns a unique identifier to the scheduled job, useful for managing jobs programmatically if needed (though programmatic management is less common with@Scheduled
).
Use Case 1: Periodic Reminder Notifications (Every 15 Minutes)
Let's implement a job that checks for upcoming reminders every 15 minutes and sends notifications.
import io.quarkus.scheduler.Scheduled;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.eclipse.microprofile.faulttolerance.Retry;
import org.jboss.logging.Logger;
@ApplicationScoped
public class ReminderJob {
private static final Logger log = Logger.getLogger(ReminderJob.class);
@Inject
ReminderService reminderService; // Your service to fetch reminders
@Inject
NotificationService notifier; // Your service to send notifications
// Runs every 15 minutes
@Scheduled(every = "15m")
// Use @Blocking if the task performs blocking I/O (database, network calls)
// to avoid blocking the event loop or scheduler threads.
@Blocking
void sendReminders() {
log.info("Starting reminder check...");
var pendingReminders = reminderService.findUpcomingReminders();
if (pendingReminders.isEmpty()) {
log.info("No pending reminders found.");
return;
}
log.infof("Found %d reminders to process.", pendingReminders.size());
for (Reminder reminder : pendingReminders) {
try {
log.debugf("Attempting to send reminder %s", reminder.getId());
notifier.send(reminder);
reminderService.markAsSent(reminder); // Mark success
log.infof("Successfully sent reminder %s", reminder.getId());
} catch (TransientFailureException ex) {
// Specific, potentially recoverable error (e.g., temporary network issue)
log.warnf("Transient failure sending reminder %s: %s. Marking for retry.",
reminder.getId(), ex.getMessage());
reminderService.markForRetry(reminder); // Implement logic to retry later
} catch (Exception fatal) {
// Unexpected or non-recoverable error
log.errorf(fatal, "Fatal error sending reminder %s. Skipping.", reminder.getId());
// Optionally: Alert operations team, move to a dead-letter queue
reminderService.markAsFailed(reminder); // Mark as failed to prevent reprocessing
}
}
log.info("Finished reminder check.");
}
}
// Placeholder classes/interfaces for context
interface ReminderService {
List<Reminder> findUpcomingReminders();
void markAsSent(Reminder r);
void markForRetry(Reminder r);
void markAsFailed(Reminder r);
}
interface NotificationService {
void send(Reminder r) throws TransientFailureException;
}
class Reminder { String getId() { return "some-id"; } }
class TransientFailureException extends RuntimeException {
public TransientFailureException(String message) { super(message); }
}
Threading Considerations:
By default, scheduled methods run on Vert.x worker threads.
If your task involves blocking I/O (database calls, network requests, file system access), annotate the method with
@io.quarkus.scheduler.Scheduled.Blocking
(or@io.smallrye.common.annotation.Blocking
which is equivalent). This ensures the task runs on a dedicated worker thread pool, preventing it from blocking the main scheduler threads or the Vert.x event loop.If your task is purely CPU-bound and very fast, or non-blocking, you might omit
@Blocking
, but most real-world scheduled jobs involving external interactions benefit from it.
Enhancing Reliability with MicroProfile Fault Tolerance
Quarkus integrates seamlessly with MicroProfile Fault Tolerance, allowing you to easily add resilience to your scheduled jobs using annotations.
@Retry
: Automatically retries the method if it throws a specified exception.maxRetries
: Maximum number of retry attempts.delay
: Wait time between retries.jitter
: Random variation added to the delay.retryOn
: Specify exceptions that trigger a retry.abortOn
: Specify exceptions that should not trigger a retry and cause immediate failure.
@Timeout
: Aborts the method execution if it exceeds a specified duration.@Fallback
: Provides an alternative execution path (a fallback method) if the original method fails (after exhausting retries, if any).
Use Case 2: External System Sync with Retry and Fallback
Let's schedule a job to sync data from an external CRM every 10 minutes, with retries on failure and a fallback notification.
import io.quarkus.scheduler.Scheduled;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.eclipse.microprofile.faulttolerance.Fallback;
import org.eclipse.microprofile.faulttolerance.Retry;
import org.eclipse.microprofile.faulttolerance.Timeout;
import org.jboss.logging.Logger;
@ApplicationScoped
public class CrmSyncJob {
private static final Logger log = Logger.getLogger(CrmSyncJob.class);
@Inject
CrmService crmService; // Service interacting with the CRM API
@Inject
NotificationService notificationService; // Service to alert operations
// Run every 10 minutes
@Scheduled(every = "10m", identity = "crm-sync-job")
// Retry up to 3 times on CrmApiException, wait 5s between retries
@Retry(maxRetries = 3, delay = 5000, retryOn = {CrmApiException.class})
// Abort if execution takes longer than 45 seconds
@Timeout(45000)
// If it ultimately fails, call the handleSyncFailure method
@Fallback(fallbackMethod = "handleSyncFailure")
// This task involves network I/O
@Blocking
void syncFromCrm() throws CrmApiException {
log.info("Starting CRM data synchronization...");
crmService.syncAllData(); // This method might throw CrmApiException
log.info("CRM data synchronization completed successfully.");
}
// Fallback method: Must have the same method signature (or accept the triggering Throwable)
void handleSyncFailure() {
log.error("CRM sync failed after retries or timed out. Triggering alert.");
// Send an alert to the operations team
notificationService.alertOpsTeam("Critical Failure: CRM sync failed repeatedly.");
// Optionally, add metrics to track fallback occurrences
}
}
// Placeholder classes/interfaces for context
interface CrmService {
void syncAllData() throws CrmApiException;
}
interface NotificationService {
void alertOpsTeam(String message);
}
class CrmApiException extends Exception {
public CrmApiException(String message) { super(message); }
public CrmApiException(String message, Throwable cause) { super(message, cause); }
}
This setup provides significant resilience: temporary CRM glitches are handled by @Retry
, prolonged hangs are stopped by @Timeout
, and persistent failures trigger the @Fallback
for alerting.
Advanced Scheduling with the Quarkus Quartz Extension
When you need persistence, clustering, or more complex scheduling logic, the quarkus-quartz
extension is the way to go.
Setup:
Add the dependency:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-quartz</artifactId>
</dependency>
Configure the Job Store (Optional, for persistence/clustering):
To enable persistence, configure a JDBC job store in application.properties. Quarkus needs a datasource configured.
# Configure Quartz to use a JDBC Job Store
quarkus.quartz.store-type=jdbc-tx
# quarkus.quartz.clustered=true # Uncomment if running in a cluster
# Ensure you have a datasource configured (default or named)
# quarkus.datasource.db-kind=postgresql
# quarkus.datasource.username=...
# quarkus.datasource.password=...
# quarkus.datasource.jdbc.url=jdbc:postgresql://localhost:5432/mydatabase
# Optional: Initialize the Quartz schema if needed
# quarkus.quartz.jdbc-store.initialize-schema=if-not-exists
Quarkus will automatically create the necessary Quartz tables if
initialize-schema
is set appropriately.
Core Quartz Concepts:
Job
: An interface representing the task to be executed. You implement theexecute(JobExecutionContext context)
method containing your business logic.JobDetail
: Defines an instance of aJob
. It includes identity (name, group), durability, and associated data (JobDataMap
).Trigger
: Defines the schedule upon which aJob
executes. Can be based on cron expressions, simple intervals, or calendars.JobDataMap
: A map to pass parameters or state to aJob
instance.Scheduler
: The main component responsible for managing and executing Jobs based on their Triggers. You inject theorg.quartz.Scheduler
instance into your beans.
Use Case 3: Daily Report Generation (Persistent & Clustered)
Let's implement a daily report generation job at 6 AM using Quartz, ensuring it runs only once across a cluster and survives restarts.
1. Implement the Quartz Job
:
import jakarta.inject.Inject;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.jboss.logging.Logger;
// Note: Quartz Jobs are instantiated by Quartz itself,
// so use programmatic lookup or pass dependencies via JobDataMap if needed.
// Alternatively, use CDI integration for simpler injection (requires careful setup).
public class ReportGenerationJob implements Job {
private static final Logger log = Logger.getLogger(ReportGenerationJob.class);
// Injecting directly into Quartz Jobs requires CDI integration setup.
// A common pattern is to look up beans programmatically or pass service references.
// For simplicity here, assume ReportService can be accessed statically or via context.
// ReportService reportService = CDI.current().select(ReportService.class).get();
@Override
public void execute(JobExecutionContext context) throws JobExecutionException {
// Accessing CDI beans within a Quartz Job:
ReportService reportService = CDI BeanProvider needed here or passed via JobDataMap
CustomerRepository customerRepository = CDI BeanProvider needed here
String reportType = context.getMergedJobDataMap().getString("reportType");
log.infof("Executing ReportGenerationJob for type: %s", reportType);
try {
List<Customer> customers = customerRepository.findAllActive();
log.infof("Generating reports for %d customers.", customers.size());
for (Customer customer : customers) {
try {
reportService.generateCustomerReport(customer.getId(), reportType);
} catch (Exception e) {
log.errorf(e, "Failed to generate report for customer %s", customer.getId());
// Decide on error handling: continue, fail job, etc.
// Could re-throw JobExecutionException to indicate failure
// throw new JobExecutionException("Failed on customer " + customer.getId(), e, false); // 'false' = don't refire immediately
}
}
log.info("Finished generating reports.");
} catch (Exception e) {
log.error("Critical error during report generation setup.", e);
// Indicate job failure to Quartz
throw new JobExecutionException("Failed to fetch customers or critical setup error", e, false);
}
}
}
// Placeholder interfaces/classes
interface ReportService { void generateCustomerReport(String customerId, String reportType); }
interface CustomerRepository { List<Customer> findAllActive(); }
class Customer { String getId() { return "cust-id"; } }
Note on CDI Injection: Injecting CDI beans directly into Job
classes requires careful handling as Quartz instantiates these objects. Quarkus offers integration (quarkus-quartz
leverages quarkus-scheduler
's CDI support under the hood), but simpler approaches often involve programmatic lookup (CDI.current().select(...)
) or passing necessary service references/data via the JobDataMap
.
2. Schedule the Job (e.g., during application startup):
import io.quarkus.runtime.StartupEvent;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.event.Observes;
import jakarta.inject.Inject;
import org.quartz.*;
import org.jboss.logging.Logger;
@ApplicationScoped
public class ReportScheduler {
private static final Logger log = Logger.getLogger(ReportScheduler.class);
@Inject
Scheduler quartzScheduler; // Inject the Quartz Scheduler managed by Quarkus
void onStart(@Observes StartupEvent ev) {
log.info("Scheduling the daily report generation job.");
scheduleDailyReportJob("DailySummary");
}
private void scheduleDailyReportJob(String reportType) {
String jobIdentity = "reportJob_" + reportType;
String triggerIdentity = "reportTrigger_" + reportType;
try {
// Check if the job already exists to prevent duplicates on restart if using persistence
JobKey jobKey = new JobKey(jobIdentity, "reports");
if (quartzScheduler.checkExists(jobKey)) {
log.infof("Job %s already scheduled. Skipping.", jobKey);
return;
}
JobDetail jobDetail = JobBuilder.newJob(ReportGenerationJob.class)
.withIdentity(jobKey)
.usingJobData("reportType", reportType) // Pass parameters
.storeDurably() // Keep the job definition even if no triggers are associated
.build();
Trigger trigger = TriggerBuilder.newTrigger()
.withIdentity(triggerIdentity, "reports")
.forJob(jobDetail)
// Schedule to run every day at 6:00 AM
.withSchedule(CronScheduleBuilder.cronSchedule("0 0 6 * * ?"))
// .startNow() // Use startNow() for immediate execution + schedule, or just rely on cron
.build();
quartzScheduler.scheduleJob(jobDetail, trigger);
log.infof("Scheduled job %s with trigger %s.", jobKey, trigger.getKey());
} catch (SchedulerException e) {
log.error("Failed to schedule report job.", e);
// Handle scheduling failure (e.g., alert, log critical error)
}
}
}
With the JDBC store configured and quarkus.quartz.clustered=true
, Quartz ensures that only one node in the cluster will execute the ReportGenerationJob
at 6 AM. If that node fails mid-execution (depending on configuration and transaction usage), another node might pick it up.
Handling Long-Running or CPU-Intensive Tasks
Whether using @Scheduled
or Quartz, if a background task is expected to run for a long time or consume significant CPU resources, it's crucial to prevent it from starving other tasks or blocking essential threads (like scheduler threads or the Vert.x event loop).
The best practice is to offload the intensive work to a dedicated ExecutorService
.
1. Define a Managed Executor:
You can use Quarkus's built-in managed executor service or define your own.
Using Quarkus Managed Executor: Configure it in
application.properties
:
# Configure a custom thread pool for long tasks
quarkus.thread-pool.executor-service=true # Make it available for injection
quarkus.thread-pool.name=long-task-pool
quarkus.thread-pool.core-threads=5
quarkus.thread-pool.max-threads=10
quarkus.thread-pool.queue-size=100
Inject it using
@Inject jakarta.enterprise.concurrent.ManagedExecutorService managedExecutorService;
Creating a Custom Executor (Alternative):
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.inject.Produces;
import jakarta.inject.Named;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
@ApplicationScoped
public class ExecutorConfig {
@Produces
@ApplicationScoped
@Named("longTaskExecutor")
public ExecutorService longTaskExecutorService() {
// Consider using a bounded queue and rejection policy for production
return Executors.newFixedThreadPool(10);
}
// Remember to shut down the executor gracefully on application stop
// void onStop(@Observes ShutdownEvent ev, @Named("longTaskExecutor") ExecutorService service) {
// service.shutdown();
// }
}
Inject it using
@Inject @Named("longTaskExecutor") ExecutorService longTaskExecutor;
2. Submit Tasks from the Scheduled Method:
Modify your scheduled method (or Quartz Job
) to submit the actual work to the executor.
import io.quarkus.scheduler.Scheduled;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import jakarta.inject.Named; // Use if using @Named producer
import jakarta.enterprise.concurrent.ManagedExecutorService; // Use if using managed executor
import java.util.concurrent.ExecutorService;
import org.jboss.logging.Logger;
@ApplicationScoped
public class LongTaskTriggerJob {
private static final Logger log = Logger.getLogger(LongTaskTriggerJob.class);
// Inject the appropriate executor
// @Inject @Named("longTaskExecutor")
// ExecutorService executor;
@Inject
ManagedExecutorService executor; // Using managed executor example
@Inject
LongRunningService longTaskService; // The service containing the heavy logic
// This scheduled method is lightweight; it just submits the real work.
@Scheduled(every = "30m")
void triggerLongRunningTask() {
log.info("Triggering long-running task execution.");
executor.submit(() -> {
try {
log.info("Long-running task started on dedicated executor.");
longTaskService.executeComplexProcess();
log.info("Long-running task finished successfully.");
} catch (Exception e) {
log.error("Long-running task failed.", e);
// Handle failure within the async task
}
});
log.info("Long-running task submitted to executor.");
}
}
// Placeholder service
interface LongRunningService { void executeComplexProcess(); }
This pattern ensures your scheduler threads remain available and responsive, only responsible for the lightweight act of triggering the work on the appropriate thread pool.
Monitoring and Observability
Background jobs are often critical, yet run "silently." Proper monitoring is essential to ensure they are running correctly and efficiently. Quarkus recommends using the OpenTelemetry standard in it’s monitoring and observability guidelines. I’ll use Micrometer here to make it simple.
Logging: Implement detailed logging within your jobs. Log start/end times, key steps, records processed, and especially any errors. Use structured logging where possible for easier parsing. Centralized logging (e.g., ELK stack, Loki) is vital in distributed systems.
Metrics: Use the
quarkus-micrometer
extension to expose metrics.Track job execution counts (
Counter
).Measure job duration (
Timer
).Count failures (
Counter
).Monitor queue sizes if using custom executors.
// Example using Micrometer (inject MeterRegistry)
// Counter successCounter = registry.counter("myapp.job.executions", "jobName", "myJob", "status", "success");
// Counter failureCounter = registry.counter("myapp.job.executions", "jobName", "myJob", "status", "failure");
// Timer durationTimer = registry.timer("myapp.job.duration", "jobName", "myJob");
// In your job:
// Timer.Sample sample = Timer.start(registry);
// try {
// // job logic ...
// successCounter.increment();
// } catch (Exception e) {
// failureCounter.increment();
// } finally {
// sample.stop(durationTimer);
// }
Tracing: Integrate with distributed tracing using
quarkus-opentelemetry
. Traces help visualize the flow of execution, especially if a scheduled job interacts with multiple services.Health Checks: Use
quarkus-smallrye-health
to create health checks that verify job status, perhaps by checking the last successful run timestamp stored in a database.
Summary and Best Practices
Quarkus provides excellent tools for managing scheduled and background tasks, from simple annotations to sophisticated clustered execution.
Key Takeaways:
Choose Wisely: Start with
@Scheduled
for simplicity. Move to Quartz for persistence, clustering, and complex scheduling needs.Embrace Fault Tolerance: Use
@Retry
,@Timeout
, and@Fallback
with@Scheduled
jobs for resilience. Implement robust error handling within Quartz jobs.Manage Threads: Use
@Blocking
for I/O-bound@Scheduled
tasks. Offload long-running or CPU-intensive work to dedicated executors.Design for Failure: Assume jobs can fail. Make them idempotent where possible so retries are safe.
Monitor Actively: Implement logging, metrics, and potentially tracing to ensure visibility into job execution and health.
Mastering background and scheduled tasks is essential for robust enterprise applications, and Quarkus provides flexible and powerful tools to meet these needs effectively. Whether you require simple periodic execution or complex, clustered, persistent jobs, the framework offers well-integrated solutions that prioritize developer productivity and application performance.
For more detailed configuration options, advanced features, and specific API usage, always refer to the official Quarkus documentation.
Further Reading:
Quarkus Datasource Guide (Relevant for Quartz JDBC Store)
Quarkus Micrometer Guide (For Metrics)
Quarkus OpenTelemetry Guide (For Tracing)