> For the complete documentation index, see [llms.txt](https://docs.zerowork.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zerowork.io/building-blocks/data/remove-duplicates.md).

# Remove Duplicates

<figure><picture><source srcset="https://3220183989-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fg9LcR8XM7TGOzOCDGsCS%2Fuploads%2Fgit-blob-8857e348e837b2970d35d70c86fb2c5a908fd34c%2Fremove-duplicates-tile-dark.png?alt=media" media="(prefers-color-scheme: dark)"><img src="https://3220183989-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fg9LcR8XM7TGOzOCDGsCS%2Fuploads%2Fgit-blob-20beeef6072514e9a337472f8735ccd760573587%2Fremove-duplicates-tile.png?alt=media" alt="Remove Duplicates block" width="279"></picture><figcaption></figcaption></figure>

*This page is due to be updated. Some details may differ from the current version.*

This action removes all the rows that are duplicate.

{% hint style="warning" %}
It is best practice to always specify a column, unless you have a very special case. Specifying a column will make your TaskBot run faster.
{% endhint %}

Notes: Files in *file* columns will be ignored when searching for duplicates.

The TaskBot will remove **the latest (the newest) rows** and preserve previous rows per default. If you want to reverse this behavior, then check the checkbox *Preserve newest rows* (note that this option is only available in native tables and not available for Google Sheets).

<figure><picture><source srcset="https://3220183989-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fg9LcR8XM7TGOzOCDGsCS%2Fuploads%2Fgit-blob-c67fc2dc7709ed7e31463ac2dcc47590632086c0%2Fremove-duplicates-settings-dark.png?alt=media" media="(prefers-color-scheme: dark)"><img src="https://3220183989-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fg9LcR8XM7TGOzOCDGsCS%2Fuploads%2Fgit-blob-e3f72d410a3b504919a66e93551508d4267c8b86%2Fremove-duplicates-settings.png?alt=media" alt="Remove Duplicates settings" width="555"></picture><figcaption></figcaption></figure>

### Do Not Place Inside Loops

This is a batch operation. This means that all table rows - no matter how many there are - are fetched and the duplicate removal is applied to the whole table. Hence, it does not make sense to use this on every loop iteration.

{% hint style="warning" %}
If you place this building block inside a loop, duplicates will only be removed once.
{% endhint %}

### Video Tutorial

{% embed url="<https://youtu.be/O9nZUwabzBw?si=A_ILoIOKLtUsf6Gu>" %}
