How to remove duplicates from Stream in Java 8 - Stream.distinct() Example

You can use the Stream.distinct() method to remove duplicates from a Stream in Java 8 and beyond. The distinct() method behaves like distinct clause of SQL which removes duplicate rows from the result set. The distinct() is also an intermediate method which means it will return a new Stream without duplicates, which can be used for further processing. Like other methods of Stream class e.g. map() or filter(), distinct() is also lazy and it will not remove duplicate elements until you call a terminal method on Stream.  The actual processing of Stream pipeline starts only after calling terminal methods like collect() or forEach().


If you are interested in learning more about how Stream processing works or internal details of Stream, I will encourage you to go through  The Complete Java MasterClass, which covers them nicely, along with other Java 8 concept.


Java 8 Stream.distinct() Example


Here is an example of Stream.distinct() method to remove duplicate elements from Stream in Java 8:


package tool;

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

/**
* 
* A simple Java Program to to remove duplicates from Stream in Java 8
* This example uses Stream.distinct() method to remove
* duplicates. 
*/
public class Hello {

public static void main(String args[]) {

List<Integer> withDupes = Arrays.asList(10, 10, 20, 20, 30, 30, 40, 50);

System.out.println("List with duplicates: " + withDupes);

List<Integer> withoutDupes = withDupes.stream()
                                      .distinct()
                                      .collect(Collectors.toList());

System.out.println("List without duplicates: " + withoutDupes);

}
}


Output
List with duplicates: [10, 10, 20, 20, 30, 30, 40, 50]
List without duplicates: [10, 20, 30, 40, 50]


You can see that the resulting List doesn't have duplicate because we have collected it from the Stream after calling the distinct method which removes the duplicate elements 10, 20, and 30. In other words, this is a new way to remove duplicate elements from a List in Java. 

If you are interested about learning new features of Java 8 then I encourage you to check out  What's New in Java 8, which covers essential Java 8 changes in fine details, including Stream and Lambda expression.

Java 8 - Stream.distinct() example




Important points

1. The distinct() method belongs to java.util.stream.Stream class.

2. The distinct() method represent an intermediate Stream operation which means it returns a new Stream without duplicates.

3. Like other stream methods, distinct is also lazy and will not start working unless you call a terminal method. If you are interested more about learning internal details of Stream, I will encourage you to read Core Java Volume 1 and 2 by Cay S Horstman.

How to remove duplicates from Stream in Java 8


4. You can also use the distinct method to remove duplicates from List or ArrayList as well, by first converting them to Stream, removing duplicates and then collecting result back to Stream.


That's all about how to remove duplicate values form Stream in Java 8. As seen in above example, you can use the distinct() method of Stream class to remove duplicate elements from given Stream. Whenever you want to work with unique values of Stream you can use distinct, it works with all kind of Stream e.g. Stream of String or Integer.

Further Reading
What's New in Java 8
The Ultimate Java 8 Tutorial
The Complete Java MasterClass

Thanks for reading this article so far. If you like this article, then please share with your friends and colleagues. If you have any questions or feedback then please drop a note.

No comments:

Post a Comment