{"id":64582,"date":"2025-05-12T10:34:10","date_gmt":"2025-05-12T07:34:10","guid":{"rendered":"https:\/\/wpcalc.com\/en\/?p=64582"},"modified":"2025-05-12T10:34:10","modified_gmt":"2025-05-12T07:34:10","slug":"duplicate-lines-remover","status":"publish","type":"post","link":"https:\/\/wpcalc.com\/en\/web-tools\/duplicate-lines-remover\/","title":{"rendered":"Duplicate Lines Remover"},"content":{"rendered":"<section class=\"not-prose calculator-box\" aria-labelledby=\"calculator-title\"><div class=\"flex items-baseline sm:items-center justify-between gap-2 sm:gap-3\"><div class=\"flex flex-col sm:flex-row sm:items-center gap-2\"><span class=\"icon icon-calculator text-primary-700! text-base! dark:text-primary-300!\" aria-hidden=\"true\"><\/span><h2 class=\"text-lg font-display font-bold\">Remove Repeated Lines from Text Tool<\/h2><\/div><div class=\"relative group inline-block\">\n  <button class=\"favorite\" id=\"favorite\" data-favorite-id=\"64582\" data-favorite-title=\"Duplicate Lines Remover\" data-favorite-url=\"https:\/\/wpcalc.com\/en\/web-tools\/duplicate-lines-remover\/\" data-favorite-excerpt=\"Remove duplicate lines from any block of text instantly with this free Duplicate Lines Remover. Useful for cleaning up lists, log files, CSV data, or any content where uniqueness matters.\" aria-label=\"Add to Favorites\" data-favorite-icon=\"icon icon-d-edit\">\n    <span class=\"icon icon-shape-star-empty\"><\/span>\n  <\/button>\n  <div class=\"absolute right-full -translate-y-1\/2 top-1\/2 mr-2 w-max max-w-xs px-3 py-2 bg-gray-800 text-white text-xs rounded shadow-lg opacity-0 group-hover:opacity-100 transition-opacity duration-200 z-10 pointer-events-none\">\n   <span class=\"favorite-tooltip\" id=\"favorite-tooltip\"><\/span>\n  <\/div>\n<\/div><\/div><form action=\"https:\/\/wpcalc.com\/en\/web-tools\/duplicate-lines-remover\/\" method=\"POST\" class=\"calculator\" id=\"calculator-64582\" data-post=\"64582\"><fieldset class=\"fieldset-input\"><legend class=\"sr-only\">Input Fields<\/legend><div class=\"field is-single\" id=\"input-1\"><label for=\"field-1\">Text Input<\/label><textarea id=\"field-1\" name=\"text_input\">Apple\nBanana\nApple\nOrange\nBanana<\/textarea><\/div><\/fieldset><div class=\"buttons\"><button type=\"submit\" data-text=\"Re-Calculate\" id=\"calculate-button\" data-post=\"64582\">Calculate<\/button><button type=\"reset\">Reset<\/button><\/div><div class=\"field is-checkbox hidden!\" id=\"field-auto-calc\"><input type=\"checkbox\" id=\"auto-calc\"><label for=\"auto-calc\">Calculate automatically<\/label><small>If enabled, the result will update automatically when you change any value.<\/small><\/div><div class=\"fieldset-result is-hidden\" aria-labelledby=\"result-title\" aria-live=\"polite\" role=\"region\"> <h3 class=\"result-title bg-gradient-to-r from-primary-50 to-gray-50 dark:from-primary-900 dark:to-gray-800\" id=\"result-title\"><span class=\"icon icon-s-pulse\" aria-hidden=\"true\"><\/span> Your Results<\/h3><div class=\"result-box\"><div class=\"field-result is-column\" id=\"output-1\"><span class=\"field-title\"><span>Unique Lines<\/span><\/span><span class=\"field-value\" id=\"result-1\"><\/span><button class=\"copy-result\" data-tooltip=\"Copy Result\"><span class=\"copy-icon icon icon-document-copy\"><\/span><\/button><\/div><\/div><\/div><a href=\"#respond\" class=\"hidden transition-opacity duration-300 opacity-0 w-50 text-sm justify-center items-center gap-2 px-4 py-2 rounded bg-gray-200 text-gray-700 hover:bg-gray-300\" id=\"leave-comment\"><span class=\"icon icon-comments\" aria-hidden=\"true\"><\/span>Leave a Comment<\/a><\/form><\/section><section id=\"calc-reactions\" class=\"not-prose hidden my-12 bg-gradient-to-r from-primary-50 to-gray-50 border border-indigo-100 rounded-xl px-6 py-4 shadow-sm\" data-post=\"64582\" aria-live=\"polite\"><h2 class=\"text-sm text-gray-500 text-center mb-2\">How did this result make you feel?<\/h2><div class=\"grid grid-cols-3 sm:flex sm:flex-row sm:justify-around gap-4 sm:items-center sm:flex-wrap\"><button class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"like\"><span class=\"reaction-count\" data-reaction-count=\"like\">0<\/span><span class=\"reaction text-3xl\">\ud83d\ude00 <\/span><span class=\"reaction-description font-medium\">Like <\/span><\/button><button class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"helpful\"><span class=\"reaction-count\" data-reaction-count=\"helpful\">0<\/span><span class=\"reaction text-3xl\">\ud83d\udca1 <\/span><span class=\"reaction-description font-medium\">Helpful <\/span><\/button><button class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"confused\"><span class=\"reaction-count\" data-reaction-count=\"confused\">0<\/span><span class=\"reaction text-3xl\">\ud83d\ude15 <\/span><span class=\"reaction-description font-medium\">Confused <\/span><\/button><button class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"disappointed\"><span class=\"reaction-count\" data-reaction-count=\"disappointed\">0<\/span><span class=\"reaction text-3xl\">\ud83d\ude1e <\/span><span class=\"reaction-description font-medium\">Disappointed <\/span><\/button><button class=\"reaction-btn flex flex-col items-center gap-1 text-sm text-gray-700 hover:text-primary-600 cursor-pointer transition-transform hover:scale-105\" data-reaction=\"inaccurate\"><span class=\"reaction-count\" data-reaction-count=\"inaccurate\">0<\/span><span class=\"reaction text-3xl\">\u274c <\/span><span class=\"reaction-description font-medium\">Inaccurate <\/span><\/button><\/div><div id=\"reaction-message\" class=\"hidden mt-8 rounded-md border border-primary-100 bg-white\/50 backdrop-blur-sm px-4 py-2 text-sm text-center shadow-sm\"><\/div><\/section><ins class=\"adsbygoogle\"\n     style=\"display:block; text-align:center; margin: 32px 0;\"\n     data-ad-layout=\"in-article\"\n     data-ad-format=\"fluid\"\n     data-ad-client=\"ca-pub-1721569815777345\"\n     data-ad-slot=\"6317458308\"><\/ins>\n<script>\n     (adsbygoogle = window.adsbygoogle || []).push({});\n<\/script>\n<p><section class=\"formula-box\" aria-labelledby=\"formula-title\">\r\n\t<div id=\"formula-title\" class=\"not-prose font-display mb-4\"><span class=\"icon icon-formula\" aria-hidden=\"true\"><\/span><h2>Duplicate Removal Logic<\/h2><\/div>\r\n\t<figure class=\"not-prose formula\">\r\n\t\t<figcaption class=\"formula-title\">Formula<\/figcaption>\r\n\t\t<div class=\"text-base text-gray-800\" id=\"formula\">\n$$\\text{Unique Lines} = \\text{Set}(\\text{split}(\\text{input},\\ \\texttt{\\textbackslash n}))$$\n<\/div>\r\n\t<\/figure>\r\n\t<br \/>\nThe input text is split into lines, then converted to a set (to eliminate duplicates), and rejoined into a clean, unique list.<br \/>\n<strong>Example:<\/strong><\/p>\n<pre>\r\nApple  \r\nBanana  \r\nApple  \r\n\u2192  \r\nApple  \r\nBanana\r\n<\/pre>\n<p>\r\n<\/section><br \/>\nThe Duplicate Lines Remover tool helps eliminate redundancy in large or small text blocks. Whether you&#8217;re processing tags, lists, email addresses, logs, or code snippets \u2014 this tool ensures every line appears only once. It trims leading\/trailing whitespace and works with line-by-line data. You can optionally preserve order or sort the final result alphabetically.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Remove duplicate lines from any block of text instantly with this free Duplicate Lines Remover. Useful for cleaning up lists, log files, CSV data, or any content where uniqueness matters.<\/p>\n","protected":false},"author":70,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[63],"tags":[65],"class_list":["post-64582","post","type-post","status-publish","format-standard","hentry","category-web-tools","tag-text-tools"],"acf":[],"_links":{"self":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/64582","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/users\/70"}],"replies":[{"embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/comments?post=64582"}],"version-history":[{"count":2,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/64582\/revisions"}],"predecessor-version":[{"id":64584,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/posts\/64582\/revisions\/64584"}],"wp:attachment":[{"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/media?parent=64582"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/categories?post=64582"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wpcalc.com\/en\/wp-json\/wp\/v2\/tags?post=64582"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}