Box.com bulk transfer: Difference between revisions
Jump to navigation
Jump to search
| Line 10: | Line 10: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
| split \<br /> | | # chop file into 4G pieces <br /> | ||
split \<br /> | |||
--bytes=4000m \<br /> | --bytes=4000m \<br /> | ||
big_file.fastq.gz \<br /> | big_file.fastq.gz \<br /> | ||
big_file.fastq.gz.split4g. | big_file.fastq.gz.split4g.<br /> | ||
|- | |||
| # record checksums of original and chunks<br /> | |||
md5sum \<br /> | |||
big_file.fastq.gz \<br /> | |||
big_file.fastq.gz.split4g.* \<br /> | |||
> big_file.fastq.gz.md5 | |||
|} | |} | ||
Revision as of 18:52, 15 December 2015
UAB has an Enterprise contract with Box.com, which is currently in BETA.
This page describes what we have learned about doing bulk-transfers of data.
Warning: size limitations
Box.com claims to have a 5G max filesize limit, which will be removed in 2016. If you need to work around this, you can use the Linux "split" utility
| # chop file into 4G pieces split \ |
| # record checksums of original and chunks md5sum \ |
Warning: time stamps
When using an FTP client to transfer data up, it is easy to lose both modification and creation timestamps. In particular, many clients will (optionally) preserve modification time, but few will (optionally) preserve creation date.
| FTP client | platform | modification | creation |
|---|---|---|---|
| SmartFTP | GUI/Win Only/$$ | yes | can be enabled |
| lftp | cmd_line/linux/free | yes | no |
| fileZilla | GUI/linux+win/free | can be enabled | no |
| ftp_ssl | cmd_line/linux/free | yes | no |