Wed 19 Aug 2009
Digg This Entry , No Comments
There are times when you want to create an archive of many files or even one large file but need the archive file to be smaller than some file size limitation. In my case, I needed to move 90GB of data literal over 1 million files over FTP. ( There is a story behind why it was done this way.) You do not want to move a million little files over FTP. You can get better performance by creating one large file or in my case about 26 TAR archives that were 3.5GB in size. The files were to be moved to a FAT-32 external hard drive prior to sending them via FTP. The Max file size for FAT-32 is 4GB. At least when I tried to create TAR files larger than that I would get an error. The 3.5GB files size worked well.
This works on Linux and Mac OS. Open a terminal window. Then run the following command:
# tar -cf – [source] | split -b 3584m – archive.tar
This will produce a series of TAR files from your [source] file/folder with the suffix of:
taraa, tarab, tarac … taraz, tarba, tarbb …
and so on. It will append the alpha series to the suffix of the archive. Then on the destination terminal run:
#cat archive.tara* | (tar x)
That will assemble the split files and extract the archive in to their source files.