« Selenium presentation @ Neon Guild
I got a great time slot for speaking@CodeMash »

Better iTunes song deduping

Posted on Saturday, November 11th, 2006 by Eric Pugh

I’m happy to share my hacked version of Doug Adams Corral iTunes Dupes AppleScript. You can grab my version from here.

His version uses some wicked fast PERL to find all the duplicate songs based on info in the Library, and then puts them all in a Dupes playlist. However, I wanted to filter by Name, Artist, and Album, and have a single playlist where I could delete in one fell swoop all the duplicates. So I tweaked his AppleScript to have another prompt that asks if you want all dupes to go in the playlist, or just copies 2…X to go in.

I run it once with “All” to make sure I have what I want, and then rerun it with “Just Dupes” and build the playlist of files to delete. Works great, and I’m mailing him a copy, maybe it’ll be in the 1.1 version!

I was also very happy with the application Seek and Destroy Music Duplicates. It is written in Java using WebStart, and gave some good results as well. It is better in that you can pick which copy of the duplications you want to keep, and color codes each set of duplicates.

The tool iDupe really didn’t do that much for me, and the demo is limited to just doing 3 deletes. Seek and Destroy is a better choice.

Categories: Uncategorized.

You can leave a response, or trackback from your own site.
  • tommo
    dude. you just saved me about 6 hours of mind numbing manual labour. thanks a million!
  • nick
    where can I get your de-duping script? the link is broken. sounds like it's exactly what everyone needs
  • I've updated the links in this blog post. Sorry about the confusion, this was caused by a revamp of our blog system and web site.
  • kate
    The script wants to run in Classic. Did I do something wrong?
    Also, the mac os x folder is empty ... ?
  • kate
    Aha -- tried something other than Stuffit -- alles gut/
  • Jim dropped me a note with this question:

    _After running your program, it creates the
    "dupes" playlist with all the dupes. Now that I have
    these in a playlist, is there anyway that I can
    actually have the dupes removed from the itunes
    library? When I select all in the dupes playlist and
    hit delete, it just deletes this playlist and i still
    have the dupes in my library._

    To actually delete the underlying files, you need to hold down the option (or is it control??) key. ITunes will ask you if you want to delete the actual files, and you click "Yes". Drove me crazy for a while as well.
  • ERIC
    Excellent! Works perfectly, I had around 1600 duplicate files due to a transfer to a new computer, m3u files, and general forgetfulness. THE most important script I have on my computer.
  • Dave
    Do you have anything that will run on a pc?
  • Kyle
    Late to the party here but this is EXACTLY what I needed! Thank you.
  • Ok so I've got the dupes playlist. How do I delete the FILES? not just the entires in the Library?
    Cheers
    c
  • vin
    To answer Craig's: the best way i think is to add a comment to all songs in the dupe playlist (select all -> show info), then sort by that column and delete all the songs with that particular comment. Example: ' * ' as a comment.
  • got 6000 duplicates, so this script is heaven 4 me!!! THANKS
    (ps: i just renamed all the artist names in the dupes list to 'jsdhdgfsdjkjgvks' and then deleted those in the library :)
  • John
    Nice contribution. I'm puzzled though, when you've expanded on someone else's work (and indeed contributed a patch to them), why did you not allow access to your script inside the app? I would much rather have just downloaded the script than an app, or an app where I can get at the script. I want to see the code before I run it, and possibly tweak it for my own needs. I think I'll just get Doug's script.
  • John
    Ah, I see that Doug packaged the script in the same way, and that I can open these apps in the Script Editor. I'm new to Applescript - my bad, my apologies.
  • shivers
    Totally amazing script, I found the best results where returned when I filtered the Dupes for 'Size' rather than name or albumn. That way you can make sure the ones that are the same size i.e. same song (regarless of albumn or name) are filtered into the Dupes playlist. Thanks to Jim for telling us how to delete tunes directly from a playlist.
  • Keith
    So, im currently running your script, its been going for 2 days now, i wish it didnt take so long, but i do have 330 gigs of music it is going through... I'm hoping it ends soon, cause i cant use itunes to put any new music on my iphone or ipods.... Is there a way to end the script and remove the duplicates its found so far... im afraid to force quit itunes and I dont see any other way to quit the script... thanks and if anybody knows how to do that, post a replyon here... also was wondering if anybody tried phelix, and how that compares to removing dupes versus this script... btw thaks a ton for putting in your fine work on this script...
  • Clayton
    Yeah - I believe the Size is the best attribute to sort by if you have an extensive collection (which I'm sure everyone does) and have 3 or 4 artists that play the same song name. Excellent Script!
  • Chris Weiss
    I've run into a rather frustrating problem that maybe this script can help fix. I'm re-ripping my library with Apple Lossless and have discovered that iTunes isn't actually replacing the songs, but making duplicates. Would there be a way to tweak the script to automatically choose the lower bitrate (or at least smaller file) file for the duplicate? I'm sure I could figure it out with a week or two of studying, but if you're a Perl wiz (and really, the mod should stay in your excellent script), you could probably knock it out in an hour, ya?
  • christian_rennt
    Months of looking for a solution have come to an end. Eric, this is the best thing that has happened to me in my MacLife so far. Cheers for that.
  • Stijn
    Thank you very much! This is exactly what i needed!
  • Mike
    Thanks!
    Saving me LOTS of time!
    In line with Keith's comment, it would be great to have a cancel option while the script is running (versus force quit the script)
  • Jerm
    This looks compiled. iTunes does not recognize it as a script, nor does the script editor.
    unzip corral-itunes-dupes-eric.zip

    open in Script Editor:
    unzip corral-itunes-dupes-eric.zip
    ˛Ì˙Œ
  • prk
    i have never used the scripts, so I have no clue how to use this... and i have million duplicates... please advise =)
  • I just got a Macbook and imported 18000+ songs onto it with iTunes. In my pc i had never found a good way to get rid of all the dupes. Now this is amazing. Thanks a million!!
  • Andrew
    Hi,

    This is a great tool! Not sure that I am using it correctly though? I have followed your instructions and have the dedupe folder populated, trouble is, it seems to have selected all of the songs. For example, where I have three songs, it moves all three to the dedupe folder, not just the duplicates? thus if I mass delete the folder I delete my entire itunes directory?


    can you please help?
  • Fang
    Hi,

    when I run this, it puts all songs into the dedupe list, not just the 3 or 3 dupes?
  • mariko
    can someone pleeease tell us challenged folk how to actually use it? i downloaded the script but don't know what im doing after that.
  • Peter
    Eric
    As an owner of a new (second) ipod and a new PC, I am shocked by how CRAPPY iTunes is when it comes to migrating and synching music. Like everyone here, I now have an iPod with 60 GB - half of which are duplicated (more than 8,000 files). I Googled and found your killer script. Unfortunately I am enslaved on Windows Vista. Is there any Win version of your most valuable utility?

    It would be awesome if you gifted your script to Apple to make their core iTunes EXE more useful for serious music fans like us all. Any tips appreciated! Cheers, Peter
  • Iwana Anderson
    I am just another of those "your script rocks" posts. Maybe you could mention that one can delete the files from any playlist by pressing "Command(Alt?)" + "Delete".

    I had to scroogle this first, but it works like a charm.


    Thank you for your contribution!
  • ty
    man, what an amazing tool. found 722 duplicates (3+GB on my laptop) very nice.

    for deleting the files, i just went into the "Dupes" playlist and Edit/Select All...Then held down CTRL-Option-Applekey and hit delete. I'm not sure which one does the trick, but using all 3 does it!
  • Ruben
    Hi,

    Make sure that the XML file path is without caracters like ' and > and < and such... Otherwise! Great! Idiots like me usually use paths like those.... hehehe so a warning would be great here!

    Ruben
  • Chris
    I need one that works for windows. Do you know of one.
  • Shawn Lesky
    Need a windows alternative. I'm a Mac man but my music library is twice the size of the hard drive in my macbook pro. I have 11,000+ duplicates to get rid of (22,000+ when I show duplicates)
  • amy
    amazing. thank you a thousand times over for this script.
  • Mim
    Any version for PC, and Lovely Vista Sh...?
    thanks
  • Mike
    Hello All,

    Can someone describe how to install the files?

    Thanks a bunch!
  • ben
    i got phelix, ran it, have a folder with all the files now...so to create room do i just delete the phelix folder? sorry....comp ill.

    so now do i delete all the songs in itunes and make a new folder or what? help!
  • Thank you very much for this script! I've just bought a new mac recently and what I've found so far - the best community that makes and shares stuff! Keep up the good work. I think the 3-years lasting comments on this post just prove how good your script is!
  • RD
    Thanks! It worked perfectly!
  • OmerCan
    Great script enhancement. Congrats.

    Works flawlessly!!! :-)
  • PunkyD
    You rock!!!
    This is amazing!
    Thanks so much!
  • destructordt
    AWSOME!!!!
    You saved me 15$ for Dupin!! I had soooo many duplicates.
  • Zach
    Great script
    thank you soooo much. Cleaned my library up!
  • Thanks man! This rock! I luv u! =D
  • Mike
    Excellent app! Appreciate your skils.
    MAC users rock!
  • aa
    this is FANTASTIC! works so well.
    thank you so much!!! i cleared out almost 3,000 songs in my library in just a few minutes :]
  • Anonymous Incognito
    Awesome Eric!
    It removed all my 1210 duplicate files. It didn't remove the files with same name and different albums, that was really what I wanted.
    Thanks a lot.
  • cmoslim
    Ran it, deleted all the files, deleted all my music, fortunately I have it backed up in like 4 different places, must not have followed directions :/
  • guideaux
    hi there, thnx for the script. question: the script runs, but says " no duplicates found" whilst my " show doubles" in itunes shows about 2000 songs??

    i just installed snow leopard and have all the last updates.

    somebody knows why the script cant find anything?
  • HELP!
    Okay, so I've run the script, it puts the dupes in the playlist, and I click remove.

    But every time I re-run the script, it finds the same dupes, and every time I delete them, it only deletes them from the playlist, even if I hit CTRL.

    What am I doing wrong? Help!
  • Jay
    Thanks dude. that was really helpful. Was looking for something like this for a while... Thanks for the tip on option-deleting too!
  • Mikitukka
    Hay i know im abit late for the party but can you tell me, when you select dupes only, how dose the script determine which dupe to put in the playlist? I have a few dupes in my list, about 1000, but some are broken mp3's that stop abruptly.
    I think ill just have to go through th "all" list slowly and manually! :-(
blog comments powered by Disqus