How to Remove Duplicates from ArrayList in Java [Example]

ArrayList is the most popular implementation of the List interface from Java's Collection framework, but it allows duplicates. Though there is another collection called Set which is primarily designed to store unique elements, there are situations when you receive a List like ArrayList in your code and you need to ensure that it doesn't contain any duplicate before processing. Since with ArrayList you cannot guarantee uniqueness, there is no other choice but to remove repeated elements from ArrayList. 

There are multiple ways to do this, you can follow the approach we used for removing duplicates from array in Java, where we loop through array and inserting each element in a Set, which ensures that we discard duplicate because Set doesn't allow them to insert, or you can also use remove method of ArrayList to get rid of them, once you found that those are duplicates.

Btw, the simplest approach to remove repeated objects from ArrayList is to copy them to a Set e.g. HashSet and then copy it back to ArrayList. This will remove all duplicates without writing any more code.

One thing to noted is that, if original order of elements in ArrayList is important for you, as List maintains insertion order, you should use LinkedHashSet because HashSet doesn't provide any ordering guarantee.

If you are using deleting duplicates while iterating, make sure you use Iterator's remove() method and not the ArrayList one to avoid ConcurrentModificationException.  In this tutorial we will see this approach to remove duplicates.




Java Program to Remove duplicates from ArrayList

Here is our sample program to learn how to remove duplicates from ArrayList. The steps followed in the below example are:
  • Copying all the elements of ArrayList to LinkedHashSet. Why we choose LinkedHashSet? Because it removes duplicates and maintains the insertion order.
  • Emptying the ArrayList, you can use clear() method to remove all elements of ArrayList and start fresh. 
  • Copying all the elements of LinkedHashSet (non-duplicate elements) to the ArrayList. 

Please find below the complete code :

import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;


/**
 * Java Program to remove repeated elements from ArrayList in Java.
 *
 * @author WINDOWS 8
 */

public class ArrayListDuplicateDemo{
 
   
    public static void main(String args[]){
   
        // creating ArrayList with duplicate elements
        List<Integer> primes = new ArrayList<Integer>();
       
        primes.add(2);
        primes.add(3);
        primes.add(5);
        primes.add(7);  //duplicate
        primes.add(7);
        primes.add(11);
       
        // let's print arraylist with duplicate
        System.out.println("list of prime numbers : " + primes);
       
        // Now let's remove duplicate element without affecting order
        // LinkedHashSet will guaranteed the order and since it's set
        // it will not allow us to insert duplicates.
        // repeated elements will automatically filtered.
       
        Set<Integer> primesWithoutDuplicates
                   = new LinkedHashSet<Integer>(primes);
       
        // now let's clear the ArrayList so that we can 
        // copy all elements from LinkedHashSet
        primes.clear();
       
        // copying elements but without any duplicates
        primes.addAll(primesWithoutDuplicates);
       
        System.out.println("list of primes without duplicates : " + primes);
       
    }
 
}

Output
list of prime numbers : [2, 3, 5, 7, 7, 11]
list of primes without duplicates : [2, 3, 5, 7, 11]


In this example, you can see we have created an ArrayList and added numbers into it, all prime numbers. We added '7' twice, so that it become duplicate. Now we print the ArrayList and you can see that it contains number 7 twice.

How to remove duplicates from ArrayList in Java


After that we created a LinkedHashSet from our ArrayList, clear our original ArrayList and then added all elements from set to the list. This time we should not have any duplicates because Set doesn't allow them and they should have filtered when elements copied from ArrayList to HashSet by Java. This is proved by printing the ArrayList again, now it doesn't contain 7 twice, but only once.

That's all about how to remove duplicates from ArrayList in Java. Though there are multiple ways to do this, I think using LinkedHashSet is the simplest one because its simple and also preserve the order of elements.

If you are interested in learning ArrayList, you should check out my following tutorials :
  • What is difference between two remove() methods of ArrayList class? (answer)
  • What is the correct way to remove objects from ArrayList while Iterating? (answer)
  • How to get rid of repeated elements from ArrayList? (solution)
  • How to reverse an ArrayList in Java? (solution)
  • How to synchronize ArrayList in Java? (answer)
  • Difference between Array and ArrayList in Java? (answer)
  • What is the difference between ArrayList and HashSet in Java? (answer)
  • What is the difference between HashMap and ArrayList? (answer)
  • How to convert String ArrayList to String Array in Java? (answer)
  • Beginners Guide to ArrayList in Java (guide)
  • How to get a sublist from ArrayList in Java? (program)
  • When to use ArrayList over LinkedList in Java? (answer)
  • How to create and initialize ArrayList in one line? (trick)
  • How to sort ArrayList of Integers in ascending order? (solution)
  • What is difference between Vector and ArrayList in Java? (answer)
  • How to loop ArrayList in Java? (solution)
  • How to convert an ArrayList to String in Java? (solution)
  • Array's length() vs ArrayList size() method (read here)
  • What is CopyOnWriteArrayList in Java? When do you use it? (answer)
  • How and when to use ArrayList in Java? (answer)
  • How to make read-only ArrayList in Java? (trick)
  • 3 ways to traverse List in Java? (examples)
  • How to convert List to Set in Java? (example)
Thanks for reading this ArrayList tutorial. You have seen how easy it is to remove duplicate elements from ArrayList in Java, you just need to change the data structure. In fact, if you need unique elements to prefer a Set vs List but sometimes you also need benefits of ArrayList like fast search in constant time and that's where this technique can help you. 

14 comments:

  1. How about having a condition like

    if(!set.contains(object))
    {
    // adding to arraylist
    }

    ReplyDelete
  2. You can also use the new Java 8 Streams API, and the "distinct" method like this:

    List primesWithoutDuplicates = primes.stream()
    .distinct()
    .collect(Collectors.toList());

    ReplyDelete
  3. good program to remove duplicate values from an ArrayList in Java. Thanks

    ReplyDelete
  4. How do you remove duplicate values form ArrayList without using HashSet?

    ReplyDelete
    Replies
    1. @Anonymous, use the algorithm given in this article to remove duplicate values form ArrayList without using HashSet.

      Delete
  5. This approach is only working for array lists of primitive Java types. If you put complex objects in Java set you will still have duplicates. You must override hashCode method of object type to get this approach to work.

    ReplyDelete
  6. We can use java stream api for this.

    List names= Arrays.asList("a","v","d","c","d","v","v","a");
    System.out.print(names.stream().distinct().collect(Collectors.toList()));

    Output will be [a, v, d, c]

    ReplyDelete
    Replies
    1. But you didn't think it's not easy method?

      Delete
  7. hey , it was said that "your solution should be independent of the type of array" , but you just care about integer for other we should have different LinkedHashSet yes? thx

    ReplyDelete
  8. i thought you said: Write a program to remove duplicates from an array in Java without using the Java Collection API.

    ReplyDelete
  9. If we try to stick somehow to the "Top 50 Java Programs from Coding Interviews" exercise 9, which wasn't very explicit on what exactly should we do, and try to build some logic of our own not using Sets or API methods, i think this solution for suite it better:

    static List removeDuplicates(List list) {
    List noDupes = new ArrayList<>();

    for(int numberFromList : list) {
    boolean isContaining = false;
    for(int numberFromNoDupes : noDupes) {
    if(numberFromList == numberFromNoDupes) {
    isContaining = true;
    break;
    }
    }
    if(!isContaining)
    noDupes.add(numberFromList);
    }

    return noDupes;
    }

    ReplyDelete
  10. function getUnique(array){
    var uniqueArray = [];

    // Loop through array values
    for(i=0; i < array.length; i++){
    if(uniqueArray.indexOf(array[i]) === -1) {
    uniqueArray.push(array[i]);
    }
    }
    return uniqueArray;
    }

    var names = ["John", "Peter", "Clark", "Harry", "John", "Alice"];
    var uniqueNames = getUnique(names);
    console.log(uniqueNames);

    ReplyDelete

Feel free to comment, ask questions if you have any doubt.