How to remove duplicates from Stream in Java 8 - Stream.distinct() Example

You can use the Stream.distinct() method to remove duplicates from a Stream in Java 8 and beyond. The distinct() method behaves like a distinct clause of SQL, which eliminates duplicate rows from the result set. The distinct() is also a standard method, which means it will return a new Stream without duplicates, which can be used for further processing. Like other methods of Stream class, like map() or filter(), distinct() is also lazy, and it will not remove duplicate elements until you call a terminal method on Stream.  The actual processing of Stream pipeline starts only after calling terminal methods like collect() or forEach().

If you are interested in learning more about how Stream processing works or internal details of Stream, I will encourage you to go through  The Complete Java MasterClass, which covers them nicely, along with other Java 8 concept.

How to remove duplicates from Stream in Java 8 - Stream.distinct() Example



Java 8 Stream.distinct() Example

Here is an example of Stream.distinct() method to remove duplicate elements from Stream in Java 8:

package tool;

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

/**
* 
* A simple Java Program to to remove duplicates from Stream in Java 8
* This example uses Stream.distinct() method to remove
* duplicates. 
*/
public class Hello {

public static void main(String args[]) {

List<Integer> withDupes = Arrays.asList(10, 10, 20, 20, 30, 30, 40, 50);

System.out.println("List with duplicates: " + withDupes);

List<Integer> withoutDupes = withDupes.stream()
                                      .distinct()
                                      .collect(Collectors.toList());

System.out.println("List without duplicates: " + withoutDupes);

}
}


Output
List with duplicates: [10, 10, 20, 20, 30, 30, 40, 50]
List without duplicates: [10, 20, 30, 40, 50]

You can see that the resulting List doesn't have duplicate because we have collected it from the Stream after calling the distinct method, which removes the duplicate elements 10, 20, and 30. In other words, this is a new way to remove duplicate elements from a List in Java.

If you are interested in learning new features of Java 8, then I encourage you to check out What's New in Java 8 course on Pluralsight which covers essential Java 8 changes in exquisite details, including Stream and Lambda expression.

Java 8 - Stream.distinct() example




Important points

1. The distinct() method belongs to java.util.stream.Stream class.

2. The distinct() method represents an intermediate Stream operation, which means it returns a new Stream without duplicates.

3. Like other stream methods, distinct is also lazy and will not start working unless you call a terminal method. If you are interested more in learning internal details of Stream, I will encourage you to read Core Java Volume 1 and 2 by Cay S Horstman.

How to remove duplicates from Stream in Java 8


4. You can also use the distinct method to remove duplicates from List or ArrayList as well by first converting them to Stream, removing duplicates, and then collecting results back to Stream.

That's all about how to remove duplicate values from Stream in Java 8. As seen in the above example, you can use the distinct() method of Stream class to remove duplicate elements from given Stream. Whenever you want to work with unique values of Stream, you can use distinct; it works with all kinds of Stream, like Stream of String or Integer.

Further Reading
What's New in Java 8
The Complete Java MasterClass
Java 8 New Features in Simple Way

Other Java 8 tutorials and Resources for further learning
  • Top 5 Courses to Learn Java 8 Programming (courses)
  • 5 Books to Learn Java 8 from Scratch (books)
  • How to use Stream class in Java 8 (tutorial)
  • How to use forEach() method in Java 8 (example)
  • 10 Java Date, Time, and Calendar based Questions from Interviews (questions)
  • How to change the date format of String in Java 8? (tutorial)
  • How to format/parse the date with LocalDateTime in Java 8? (tutorial)
  • How to join String in Java 8 (example)
  • How to use filter() method in Java 8 (tutorial)
  • Java 8 map + filter + stream example (tutorial)
  • 10 Examples to format and parse Date in Java 8? (tutorial)
  • How to convert Timestamp to Date in Java? (example)
  • 20 Examples to learn new Date and Time API in Java 8 (example)
  • How to compare two Dates in Java 8? (example)
  • 5 Free Courses to learn Java 8 and 9 (courses)

Thanks for reading this article so far. If you like this article, then please share it with your friends and colleagues. If you have any questions or feedback, then please drop a note.

No comments:

Post a Comment