Box.com bulk transfer

From UABgrid Documentation
(Difference between revisions)
Jump to: navigation, search
(start list of FTP programs to push data to box.com)
 
(Warning: size limitations)
Line 8: Line 8:
 
If you need to work around this, you can use the Linux "[http://ss64.com/bash/split.html split]" utility
 
If you need to work around this, you can use the Linux "[http://ss64.com/bash/split.html split]" utility
  
<nowiki>:Indented line
+
{| class="wikitable"
split --bytes=4000m big_file.fastq.gz big_file.fastq.gz.split4g.
+
|-
</nowiki>
+
| split \<br />
 +
--bytes=4000m \<br />
 +
big_file.fastq.gz \<br />
 +
big_file.fastq.gz.split4g.
 +
|}
  
 
== Warning: time stamps ==
 
== Warning: time stamps ==

Revision as of 13:47, 15 December 2015

UAB has an Enterprise contract with Box.com, which is currently in BETA.

This page describes what we have learned about doing bulk-transfers of data.

Warning: size limitations

Box.com claims to have a 5G max filesize limit, which will be removed in 2016. If you need to work around this, you can use the Linux "split" utility

split \

--bytes=4000m \
big_file.fastq.gz \
big_file.fastq.gz.split4g.

Warning: time stamps

When using an FTP client to transfer data up, it is easy to lose both modification and creation timestamps. In particular, many clients will (optionally) preserve modification time, but few will (optionally) preserve creation date.

FTP client platform modification creation
SmartFTP GUI/Win Only/$$ yes can be enabled
lftp cmd_line/linux/free yes no
fileZilla GUI/linux+win/free can be enabled no
ftp_ssl cmd_line/linux/free yes no
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox