存储输出的pickle文件作为数据源

这个东西的好处是你可以把输出的pickle文件作为数据源，而且不会占用disk的空间

####################################下面是官方教程(注意已经过时)##################################

下面是原文转载：

Our data science and engineering teams have some big news to share… you can now use any public kernel’s output files as a data source!

Plug and Play

This new functionality enables code that is more flexible, reusable, and easier to troubleshoot. With kernels as data sources, you can neatly plug together a data polishing script in R, a visualization script, and a model fitting script in Python without messy dependencies.

Follow Along

Cleaner and more compartmentalized code makes kernels an even better learning resource. Now your code can follow better practices, making it easier for new data scientists to follow along and for collaborators to pull out the pieces that will help them iterate effectively.

Adding a Kernel Data Source

Only kernels with data files as output can be used as a data source.

There are two ways to add a kernel as a data source:

1.Click Add a Data Source from within a kernel you are editing.

You’ll see that Kernels is now listed alongside Datasets and Competitions in the pop up. You can search for specific kernels using the search box.

Go to any usable kernel’s Output tab and click “New Kernel Using This Data”.

Aurelio, the lead engineer on this feature, would love to hear what you think!

#########################下面是自己的教程###################################

1.在一个名叫IEEE Simple XGBoost的Notebook中写一大堆代码(代码中必须包含to_pickle之类的输出函数),然后commit

2.重新打开

https://www.kaggle.com/appleyuchi/ieee-simple-xgboost/output

选择output一栏:

3.点击上面的Nwe Dataset,会弹出对话框

我们把新的数据集(同时也是一个文件夹)取名为useNewData,然后点击Create,会出现下面的进度条:

5.然后点击New Notebook,此时就会新建New Notebook来使用自己刚刚生成的pickle数据.

6.等待一段时间后,我们会看到新生成NoteBook中的右侧是:

7.最后,当你重新打开kaggle的时候,就能看到这个:

上面的useNewData就是你新建的数据集的名字.

小结:

pickle数据的读取速度远远快于csv文件

Reference:

https://www.kaggle.com/product-feedback/45472