I always enjoy reading the posts by Francis Smart on his Econometrics by Simulation blog. A couple of days ago he wrote a nice piece titled, "It is Time for RData Files to Become the Standard for Data Transfer".
Francis made some very good points about the handling of large amounts of data, and he provided some convincing examples regarding the compression rates and opening times for RData files as compared with other options. The comments to his post are also very relevant.
If you're using or exchanging large data files, you'll find Francis's post most helpful.
The link to the blog is broken, should be:
ReplyDeletehttp://www.econometricsbysimulation.com/2014/03/it-is-time-for-rdata-files-to-become.html
Thanks very much! Fixed now.
ReplyDeleteHi Dave, Thanks for the promo!
ReplyDeleteFrancis - my pleasure. I always enjoy what you have to say.
DeleteThe biggest issue for me is not the size (that, of course, matters for Big Data science) but formatting the data itself. For example, importing csv files into R from Excel (in automatic mode) makes some data unpredictably changed, especially for the dates format. So, I use "as.is=TRUE" argument and change to correct format manually. You proposal may help with this.
ReplyDelete