The Molecular Universe

I felt inspired recently to make an attempt to get my mp3 collection onto a single device that can be easily backed up. For some reason my attention turned to an old 15GB 3rd generation iPod that was given to me a while back. I like this device - but its battery was old and its ancient disk drive was only able to hold 15GB of files. Now 15GB of music is a lot of music. But what I really want to do is consolidate music and audio files in one place - so that I streamline their storage - storage which is currently distributed across various computers in a most incovenient way. Plenty of storage space, I reason, should allow me to collect everything in one backed up location, and I can then remove the duplication that has occurred over the years...That is the theory at least...

So I wanted to replace the battery in the 3rd generation iPod. I have done that before and I spent some time investigating where I could buy such a battery. Last time (about 2008!), Fry's had these replacement batteries hanging on their racks. Now Fry's seem to have given up having a large stock, so I investigated online sources.

Amazon was full of scare stories about replacement batteries not being the correct size to fit into the case. Apparently some of the newer replacement batteries are a bit too long or a bit too thick. This results in lengthy investigation of the reviews and questions. Eventually, I decided to buy this battery, and it fitted without problems: http://www.amazon.com/gp/product/B00RGJ48N8.

While investigating the battery replacement I found various sites that discussed increasing the storage in these machines using a CF card and an adapter. Well I was tempted by this - and I bought the necessary pieces to have a 60GB 3rd generation iPod...

Here at the parts I bought:

A CF to 50 Pin 1.8 IDE Adapter, http://www.amazon.com/gp/product/B004FCY7WQ. This connects the CF card to the connector formerly used by the small disk drive in the iPod. There is a mismatch between the number of sockets in the connector and the number of pins in the iPod's connector - but the converter is clearly labeled as to where you line up the first (left hand) pin into the connector's socket.

I also bought a 60GB CF card, http://www.amazon.com/gp/product/B009QXFSLI. CF memory seems to be fairly expensive - relative to SD cards and USB sticks - but this one is relatively inexpensive for this type of media and it is working well for me so far.

All you do to put these parts into the iPod is open up the case, remove the old hard drive, and make the necessary connections. I used some of the foam rubber that was used in the packing for the parts to insure that the new CF drive does not move around in the case.

And if I ever decide that 60GB is too small - I can alway buy a larger card.

There is a lot of discussion on how to carry out this conversion here: http://www.head-fi.org/t/337215/ipod-3g-compact-flash-mod. Two of the problems that people have are that the jumper plug on the converter board protrudes a little and formatting the drive is difficult.

For the jumper (which sets whether the CF drive will be slave drive or not) I simply took off the connector, removed the base of the socket, clipped the brass pins back a little, and slid the connector back on. This effectively brings the height of the connector back in line with the rest of the converter assembly.

To format the drive, I formatted the CF card initially using a USB adapter and an a MacPro - I formated the drive to what the Mac said was 'msdos' format - which I presume means FAT32 as this drive is 60GB. I tested the resulting drive on Linux and it was working fine, so I figured that this would be satisfactory. When the iPod was reassembled, I initially connected the iPod to a Windows version of iTunes, and this offered to reinitialize the iPod, which I accepted - and the iPod has been working fine since then - with its 60GB pleasingly reported under Settings/About.

There is talk about reboots when disconnecting from iTunes on www.head-fi.org, but my device is behaving totally normally since being converted. It connects and disconnects as usual and behaves as it used to except it now has 4 times more memory, is lighter, and probably less fragile.

Unfortunately I failed to get any photographs of the drive conversion when the iPod was opened up (and it is a pain to open these devices). I will try to remember to grab a photograph when it is next opened - and this may come soon - because I'm using it a great deal at present and that battery may need to be replaced again soon...

Occasionally, I have duplicated directories of mp3 file - for example when loading a second mp3 player with music. Then, later when trying to free up disk space, I have been unsure as to which directory to keep. This is complicated because mp3 files can have their headers updated which makes conventional 'diff' useless in determining whether files are identical. However, as noted yesterday, avconv can report the CRC signature of just the music, and you can use this to determine if two directories are really identical so that one can deleted.

I used two scripts to achive this. The first uses avconv to report the CRC of the supplied file. This is called, mp3crc.sh. The second called mp3rdiff.sh uses the first script and simply collects the CRC values of the mp3 files below a supplied pair of directories. After collection the CRCs are diffed, and the user informed of the results.

Here are the scripts. First mp3crc.sh:

 
#!/bin/sh

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

avconv -i "$1" -f crc - 2>/dev/null | grep CRC | awk '{print substr($1,5)}'

And mp3rdiff.sh:

#!/bin/sh 

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

#provide a simple recursive mp3 diff based on checksums

echo "Directory 1: $1"
echo "Directory 2: $2"

find "$1" -exec ~/bin/mp3crc.sh {} \;  > /tmp/dir1.txt

find "$2" -exec ~/bin/mp3crc.sh {} \;  > /tmp/dir2.txt

diff /tmp/dir1.txt /tmp/dir2.txt
ret=$?
if [ $ret -eq 0 ]
then
  echo "The directories are identical"
else
  echo "The directories are different"
  diff /tmp/dir1.txt /tmp/dir2.txt
fi

Installing avconv on OS X involved following the instructions here: http://superuser.com/questions/568464/how-to-install-libav-avconv-on-osx

How do you check if two collections of MP3 files are identical when their normal checksums differ because different programs have done different things to their headers?

I had this problem recently when I was trying to save a bit of diskspace and I found myself wondering if two groups of MP3 files were in fact identical. The solution seems to be to use the avconv command, like this:

avconv -i temp.mp3 -f crc - 2>/dev/null | grep CRC

This yields a CRC value for just the music portion of the file temp.mp3, ignoring the ID3 and other forms of data which are added to the file. If the CRCs are the same for two files, the two files are the same, from a sound point of view at least.