Module 8

Parallel Processing in Java

Learn how Java executes independent tasks simultaneously using multiple CPU cores.

⚡

1. What is Parallel Processing?

Parallel processing means performing multiple independent computations at the same time, usually by using multiple CPU cores.

Parallelism can improve performance for CPU-intensive workloads such as:

Large mathematical calculations
Image and video processing
Data analysis
Large collection processing
Scientific calculations
Machine learning workloads

Important: Concurrency means dealing with multiple tasks. Parallelism means actually executing multiple computations at the same time.

2. Concurrency vs Parallelism

Concurrency	Parallelism
Multiple tasks make progress during overlapping periods.	Multiple tasks execute simultaneously.
Can work on a single CPU core.	Benefits from multiple CPU cores.
Focuses on task coordination.	Focuses on simultaneous computation.
Useful for I/O-bound applications.	Often useful for CPU-bound applications.

3. Parallel Streams

Java Streams can process collection elements in parallel using parallelStream().

Java

import java.util.List;

public class Main {

    public static void main(String[] args) {

        List<Integer> numbers =
            List.of(1, 2, 3, 4, 5, 6, 7, 8);

        numbers.parallelStream()
               .forEach(number ->
                   System.out.println(
                       Thread.currentThread().getName()
                       + " : "
                       + number
                   )
               );
    }
}

Note: Parallel stream execution order is not guaranteed when using forEach().

4. Sequential vs Parallel Stream

Java

List<Integer> numbers =
    List.of(1, 2, 3, 4, 5, 6);

numbers.stream()
       .forEach(System.out::println);

numbers.parallelStream()
       .forEach(System.out::println);

A sequential stream processes elements through a sequential pipeline, while a parallel stream may split the work among multiple worker threads.

5. Fork/Join Framework

The Fork/Join Framework is designed for parallel divide-and-conquer algorithms.

A large task is divided into smaller subtasks. The subtasks are processed independently and their results are combined.

Java

import java.util.concurrent.RecursiveTask;

class SumTask extends RecursiveTask<Integer> {

    private final int[] numbers;
    private final int start;
    private final int end;

    SumTask(int[] numbers, int start, int end) {
        this.numbers = numbers;
        this.start = start;
        this.end = end;
    }

    @Override
    protected Integer compute() {

        if (end - start <= 2) {

            int sum = 0;

            for (int i = start; i < end; i++) {
                sum += numbers[i];
            }

            return sum;
        }

        int middle = (start + end) / 2;

        SumTask left =
            new SumTask(numbers, start, middle);

        SumTask right =
            new SumTask(numbers, middle, end);

        left.fork();

        int rightResult = right.compute();

        int leftResult = left.join();

        return leftResult + rightResult;
    }
}

6. ForkJoinPool

ForkJoinPool manages worker threads for Fork/Join tasks.

Java

import java.util.concurrent.ForkJoinPool;

ForkJoinPool pool =
    new ForkJoinPool();

int result =
    pool.invoke(task);

System.out.println(result);

pool.shutdown();

7. RecursiveAction

Use RecursiveAction when a parallel task does not return a result.

Java

import java.util.concurrent.RecursiveAction;

class PrintTask extends RecursiveAction {

    private final int start;
    private final int end;

    PrintTask(int start, int end) {
        this.start = start;
        this.end = end;
    }

    @Override
    protected void compute() {

        if (end - start <= 2) {

            for (int i = start; i < end; i++) {
                System.out.println(i);
            }

            return;
        }

        int middle = (start + end) / 2;

        PrintTask left =
            new PrintTask(start, middle);

        PrintTask right =
            new PrintTask(middle, end);

        invokeAll(left, right);
    }
}

8. Work-Stealing

The Fork/Join framework uses a work-stealing strategy. When one worker finishes its own tasks, it can take available work from another worker's queue.

Benefit: Work-stealing helps keep worker threads busy and can improve utilization.

9. CompletableFuture

CompletableFuture provides an API for asynchronous computation and composition of dependent tasks.

Java

import java.util.concurrent.CompletableFuture;

CompletableFuture<String> future =
    CompletableFuture.supplyAsync(() -> {

        return "Java";
    });

future.thenAccept(result -> {

    System.out.println(
        "Result: " + result
    );
});

10. Combining Parallel Tasks

Multiple asynchronous operations can be combined using thenCombine().

Java

CompletableFuture<Integer> first =
    CompletableFuture.supplyAsync(() -> 10);

CompletableFuture<Integer> second =
    CompletableFuture.supplyAsync(() -> 20);

CompletableFuture<Integer> total =
    first.thenCombine(
        second,
        (a, b) -> a + b
    );

System.out.println(total.join());

11. Running Multiple Tasks with allOf()

CompletableFuture.allOf() can be used when multiple asynchronous operations need to complete before continuing.

Java

CompletableFuture<Void> task1 =
    CompletableFuture.runAsync(() -> {
        System.out.println("Task 1");
    });

CompletableFuture<Void> task2 =
    CompletableFuture.runAsync(() -> {
        System.out.println("Task 2");
    });

CompletableFuture<Void> task3 =
    CompletableFuture.runAsync(() -> {
        System.out.println("Task 3");
    });

CompletableFuture.allOf(
    task1,
    task2,
    task3
).join();

System.out.println("All tasks completed");

12. Custom Executor

Asynchronous tasks can use a custom executor when application-specific thread management is required.

Java

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

ExecutorService executor =
    Executors.newFixedThreadPool(4);

CompletableFuture.runAsync(
    () -> {
        System.out.println(
            "Running in custom executor"
        );
    },
    executor
);

executor.shutdown();

13. CPU-Bound vs I/O-Bound Tasks

CPU-Bound	I/O-Bound
Uses significant CPU computation.	Spends significant time waiting for I/O.
Examples: calculations and image processing.	Examples: database and network requests.
Parallelism can improve throughput.	Concurrency can improve resource utilization.

14. Parallel Array Operations

The Arrays utility class provides methods such as parallelSort() for parallel array processing.

Java

import java.util.Arrays;

int[] numbers = {
    9, 5, 2, 8, 1, 7, 3
};

Arrays.parallelSort(numbers);

System.out.println(
    Arrays.toString(numbers)
);

15. Parallel Prefix

Java arrays also provide parallel prefix operations for suitable workloads.

Java

import java.util.Arrays;

int[] numbers = {
    1, 2, 3, 4
};

Arrays.parallelPrefix(
    numbers,
    (a, b) -> a + b
);

System.out.println(
    Arrays.toString(numbers)
);

16. Thread Safety in Parallel Processing

Parallel code must be designed carefully when multiple tasks access shared mutable state.

Java

List<Integer> results =
    Collections.synchronizedList(
        new ArrayList<>()
    );

numbers.parallelStream()
       .forEach(number -> {
           results.add(number * 2);
       });

Better approach: Prefer stream operations that avoid shared mutable state whenever possible.

17. Parallel Reduction

Reduction combines multiple elements into a single result.

Java

List<Integer> numbers =
    List.of(1, 2, 3, 4, 5);

int sum =
    numbers.parallelStream()
           .reduce(
               0,
               Integer::sum
           );

System.out.println(sum);

18. Ordering in Parallel Streams

Parallel streams do not automatically guarantee encounter order for every terminal operation.

Java

List<Integer> numbers =
    List.of(1, 2, 3, 4, 5);

numbers.parallelStream()
       .forEach(number ->
           System.out.println(number)
       );

numbers.parallelStream()
       .forEachOrdered(number ->
           System.out.println(number)
       );

Use forEachOrdered() when encounter order needs to be preserved, understanding that ordering can reduce some parallel performance benefits.

19. Parallel Processing and Performance

Parallel processing is not automatically faster. Creating tasks, scheduling work, synchronization, communication, and combining results all introduce overhead.

Rule of thumb: Parallelism is most useful when tasks are sufficiently independent and large enough to justify the overhead.

20. Common Problems

Race conditions
Data corruption
Excessive synchronization
Thread contention
Deadlocks
Too many tasks
Unexpected ordering
Parallel overhead

21. Parallel Processing Best Practices

Use parallelism only when it provides a real performance benefit.
Prefer immutable data where possible.
Avoid shared mutable state.
Use Fork/Join for divide-and-conquer workloads.
Use parallel streams for suitable collection operations.
Use CompletableFuture for asynchronous task composition.
Measure performance instead of assuming parallel code is faster.
Use appropriate executors for application workloads.
Keep tasks reasonably independent.
Handle exceptions from asynchronous tasks carefully.

22. Example 🌍❤️

Consider an application that needs to process thousands of images. Each image can be processed independently.

Java

List<String> images =
    List.of(
        "image1.jpg",
        "image2.jpg",
        "image3.jpg",
        "image4.jpg"
    );

images.parallelStream()
      .forEach(image -> {

          System.out.println(
              "Processing " + image
              + " on "
              + Thread.currentThread().getName()
          );
      });

Since each image can be processed independently, the work is a good candidate for parallel execution.

23. Interview Questions

Parallel processing executes independent computations simultaneously, typically using multiple CPU cores.

A parallel stream allows stream operations to be divided among multiple worker threads.

Fork/Join is a Java framework for divide-and-conquer parallel algorithms using worker threads and work-stealing.

No. Task management, scheduling, synchronization, and communication introduce overhead, so small tasks may be faster sequentially.

Work-stealing allows an idle worker to take available tasks from another worker's queue.

Avoid them for very small workloads, operations with significant shared mutable state, or workloads where parallel overhead exceeds the performance benefit.

Summary

In this lesson, you learned:

Parallel processing and parallelism
Concurrency vs parallelism
Parallel streams
Fork/Join Framework
RecursiveTask and RecursiveAction
ForkJoinPool and work-stealing
CompletableFuture
Combining asynchronous tasks
Parallel array operations
CPU-bound vs I/O-bound workloads
Thread safety and shared state
Performance considerations
Parallel processing best practices

← Previous: Multithreading Next: Modern Java →

Parallel Processing in Java

1. What is Parallel Processing?

2. Concurrency vs Parallelism

3. Parallel Streams

4. Sequential vs Parallel Stream

5. Fork/Join Framework

6. ForkJoinPool

7. RecursiveAction

8. Work-Stealing

9. CompletableFuture

10. Combining Parallel Tasks

11. Running Multiple Tasks with allOf()

12. Custom Executor

13. CPU-Bound vs I/O-Bound Tasks

14. Parallel Array Operations

15. Parallel Prefix

16. Thread Safety in Parallel Processing

17. Parallel Reduction

18. Ordering in Parallel Streams

19. Parallel Processing and Performance

20. Common Problems

21. Parallel Processing Best Practices

22. Example 🌍❤️

23. Interview Questions

What is parallel processing?

What is a parallel stream?

What is Fork/Join?

Is parallel processing always faster?

What is work-stealing?

When should parallel streams be avoided?

Summary