运行make_datafiles的过程
2024-06-09 17:48:38
1。 第一个bug
运行
echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer 后显示。提示：
- -bash: java: command not found。
那我就觉得可能是java没安装。然后，我就去官网
下载的是放到了/data 目录下，然后解压，
解压完成后，vim ./bashrc,打开，然后输入如下的内容。
保存退出，source ~/.bashrc 一下。
这时候再 echo "Please tokenize this text." | java edu.stanford.nlp.process.PTBTokenizer就可以了。
2. 第二个bug UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128) 这个bug
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# ls
README.md make_datafiles.py url_lists
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail#
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl_abs_other/data/cnn/stories /data/rl_abs_other/data/dailymail/stories
Preparing to tokenize /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized...
Making list of files to tokenize...
Tokenizing 92579 files in /data/rl_abs_other/data/cnn/stories and saving in cnn_stories_tokenized...
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+20A9, decimal: 8361)
Untokenizable: ? (U+F06E, decimal: 61550)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+F022, decimal: 61474)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
PTBTokenizer tokenized 80043350 tokens at 42671.94 tokens per second.
Stanford CoreNLP Tokenizer has finished.
Successfully finished tokenizing /data/rl_abs_other/data/cnn/stories to cnn_stories_tokenized.Preparing to tokenize /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized...
Making list of files to tokenize...
Tokenizing 219506 files in /data/rl_abs_other/data/dailymail/stories and saving in dm_stories_tokenized...
Untokenizable: ? (U+FFFC, decimal: 65532)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202D, decimal: 8237)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+2012, decimal: 8210)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202D, decimal: 8237)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202B, decimal: 8235)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202D, decimal: 8237)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+F001, decimal: 61441)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+F001, decimal: 61441)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+70E, decimal: 1806)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202F, decimal: 8239)
Untokenizable: ? (U+2010, decimal: 8208)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+206E, decimal: 8302)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+200D, decimal: 8205)
Untokenizable: ? (U+202A, decimal: 8234)
Untokenizable: ? (U+FFFD, decimal: 65533)
Untokenizable: ? (U+202C, decimal: 8236)
PTBTokenizer tokenized 203118231 tokens at 32507.27 tokens per second.
Stanford CoreNLP Tokenizer has finished.
Successfully finished tokenizing /data/rl_abs_other/data/dailymail/stories to dm_stories_tokenized.Making bin file for URLs listed in url_lists/all_test.txt...
Writing story 0 of 11490; 0.00 percent done
Traceback (most recent call last):File "make_datafiles.py", line 253, in <module>write_to_tar(all_test_urls, os.path.join(finished_files_dir, "test.tar"))File "make_datafiles.py", line 182, in write_to_tararticle_sents, abstract_sents = get_art_abs(story_file)File "make_datafiles.py", line 106, in get_art_abslines = read_story_file(story_file)File "make_datafiles.py", line 78, in read_story_filelines = f.read().split('\n\n')File "/root/anaconda3/envs/jjenv_pytorch/lib/python3.6/encodings/ascii.py", line 26, in decodereturn codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 858: ordinal not in range(128)
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# 然后我以为是编码问题，就去 make_datafiles.py 的文件开头加上 # coding: utf-8 ,但是没有解决问题，后来参考了一篇帖子https://blog.csdn.net/qq_36847641/article/details/78414718
所以就把我自己的代码，做如下更改，就可以了。
但是，
然后我就继续运行make_datafiles.py文件，然后一路都顺利直到完成。
(jjenv_pytorch) root@032ba38f2b6e:/data/rl_abs_other/cnn-dailymail# python make_datafiles.py /data/rl/rl_abs_other/data/dailymail/stories Making bin file for URLs listed in url_lists/all_test.txt... Writing story 0 of 11490; 0.00 percent done Writing story 1000 of 11490; 8.70 percent done Writing story 2000 of 11490; 17.41 percent done Writing story 3000 of 11490; 26.11 percent done Writing story 4000 of 11490; 34.81 percent done Writing story 5000 of 11490; 43.52 percent done Writing story 6000 of 11490; 52.22 percent done Writing story 7000 of 11490; 60.92 percent done Writing story 8000 of 11490; 69.63 percent done Writing story 9000 of 11490; 78.33 percent done Writing story 10000 of 11490; 87.03 percent done Writing story 11000 of 11490; 95.74 percent done Finished writing file finished_files/test.tarMaking bin file for URLs listed in url_lists/all_val.txt... Writing story 0 of 13368; 0.00 percent done Writing story 1000 of 13368; 7.48 percent done Writing story 2000 of 13368; 14.96 percent done Writing story 3000 of 13368; 22.44 percent done Writing story 4000 of 13368; 29.92 percent done Writing story 5000 of 13368; 37.40 percent done Writing story 6000 of 13368; 44.88 percent done Writing story 7000 of 13368; 52.36 percent done Writing story 8000 of 13368; 59.84 percent done Writing story 9000 of 13368; 67.32 percent done Writing story 10000 of 13368; 74.81 percent done Writing story 11000 of 13368; 82.29 percent done Writing story 12000 of 13368; 89.77 percent done Writing story 13000 of 13368; 97.25 percent done Finished writing file finished_files/val.tarMaking bin file for URLs listed in url_lists/all_train.txt... Writing story 0 of 287227; 0.00 percent done Writing story 1000 of 287227; 0.35 percent done Writing story 2000 of 287227; 0.70 percent done Writing story 3000 of 287227; 1.04 percent done Writing story 4000 of 287227; 1.39 percent done Writing story 5000 of 287227; 1.74 percent done Writing story 6000 of 287227; 2.09 percent done Writing story 7000 of 287227; 2.44 percent done Writing story 8000 of 287227; 2.79 percent done Writing story 9000 of 287227; 3.13 percent done Writing story 10000 of 287227; 3.48 percent done Writing story 11000 of 287227; 3.83 percent done Writing story 12000 of 287227; 4.18 percent done Writing story 13000 of 287227; 4.53 percent done Writing story 14000 of 287227; 4.87 percent done Writing story 15000 of 287227; 5.22 percent done Writing story 16000 of 287227; 5.57 percent done Writing story 17000 of 287227; 5.92 percent done Writing story 18000 of 287227; 6.27 percent done Writing story 19000 of 287227; 6.61 percent done Writing story 20000 of 287227; 6.96 percent done Writing story 21000 of 287227; 7.31 percent done Writing story 22000 of 287227; 7.66 percent done Writing story 23000 of 287227; 8.01 percent done Writing story 24000 of 287227; 8.36 percent done Writing story 25000 of 287227; 8.70 percent done Writing story 26000 of 287227; 9.05 percent done Writing story 27000 of 287227; 9.40 percent done Writing story 28000 of 287227; 9.75 percent done Writing story 29000 of 287227; 10.10 percent done Writing story 30000 of 287227; 10.44 percent done Writing story 31000 of 287227; 10.79 percent done Writing story 32000 of 287227; 11.14 percent done Writing story 33000 of 287227; 11.49 percent done Writing story 34000 of 287227; 11.84 percent done Writing story 35000 of 287227; 12.19 percent done Writing story 36000 of 287227; 12.53 percent done Writing story 37000 of 287227; 12.88 percent done Writing story 38000 of 287227; 13.23 percent done Writing story 39000 of 287227; 13.58 percent done Writing story 40000 of 287227; 13.93 percent done Writing story 41000 of 287227; 14.27 percent done Writing story 42000 of 287227; 14.62 percent done Writing story 43000 of 287227; 14.97 percent done Writing story 44000 of 287227; 15.32 percent done Writing story 45000 of 287227; 15.67 percent done Writing story 46000 of 287227; 16.02 percent done Writing story 47000 of 287227; 16.36 percent done Writing story 48000 of 287227; 16.71 percent done Writing story 49000 of 287227; 17.06 percent done Writing story 50000 of 287227; 17.41 percent done Writing story 51000 of 287227; 17.76 percent done Writing story 52000 of 287227; 18.10 percent done Writing story 53000 of 287227; 18.45 percent done Writing story 54000 of 287227; 18.80 percent done Writing story 55000 of 287227; 19.15 percent done Writing story 56000 of 287227; 19.50 percent done Writing story 57000 of 287227; 19.84 percent done Writing story 58000 of 287227; 20.19 percent done Writing story 59000 of 287227; 20.54 percent done Writing story 60000 of 287227; 20.89 percent done Writing story 61000 of 287227; 21.24 percent done Writing story 62000 of 287227; 21.59 percent done Writing story 63000 of 287227; 21.93 percent done Writing story 64000 of 287227; 22.28 percent done Writing story 65000 of 287227; 22.63 percent done Writing story 66000 of 287227; 22.98 percent done Writing story 67000 of 287227; 23.33 percent done Writing story 68000 of 287227; 23.67 percent done Writing story 69000 of 287227; 24.02 percent done Writing story 70000 of 287227; 24.37 percent done Writing story 71000 of 287227; 24.72 percent done Writing story 72000 of 287227; 25.07 percent done Writing story 73000 of 287227; 25.42 percent done Writing story 74000 of 287227; 25.76 percent done Writing story 75000 of 287227; 26.11 percent done Writing story 76000 of 287227; 26.46 percent done Writing story 77000 of 287227; 26.81 percent done Writing story 78000 of 287227; 27.16 percent done Writing story 79000 of 287227; 27.50 percent done Writing story 80000 of 287227; 27.85 percent done Writing story 81000 of 287227; 28.20 percent done Writing story 82000 of 287227; 28.55 percent done Writing story 83000 of 287227; 28.90 percent done Writing story 84000 of 287227; 29.25 percent done Writing story 85000 of 287227; 29.59 percent done Writing story 86000 of 287227; 29.94 percent done Writing story 87000 of 287227; 30.29 percent done Writing story 88000 of 287227; 30.64 percent done Writing story 89000 of 287227; 30.99 percent done Writing story 90000 of 287227; 31.33 percent done Writing story 91000 of 287227; 31.68 percent done Writing story 92000 of 287227; 32.03 percent done Writing story 93000 of 287227; 32.38 percent done Writing story 94000 of 287227; 32.73 percent done Writing story 95000 of 287227; 33.07 percent done Writing story 96000 of 287227; 33.42 percent done Writing story 97000 of 287227; 33.77 percent done Writing story 98000 of 287227; 34.12 percent done Writing story 99000 of 287227; 34.47 percent done Writing story 100000 of 287227; 34.82 percent done Writing story 101000 of 287227; 35.16 percent done Writing story 102000 of 287227; 35.51 percent done Writing story 103000 of 287227; 35.86 percent done Writing story 104000 of 287227; 36.21 percent done Writing story 105000 of 287227; 36.56 percent done Writing story 106000 of 287227; 36.90 percent done Writing story 107000 of 287227; 37.25 percent done Writing story 108000 of 287227; 37.60 percent done Writing story 109000 of 287227; 37.95 percent done Writing story 110000 of 287227; 38.30 percent done Writing story 111000 of 287227; 38.65 percent done Writing story 112000 of 287227; 38.99 percent done Writing story 113000 of 287227; 39.34 percent done Writing story 114000 of 287227; 39.69 percent done Writing story 115000 of 287227; 40.04 percent done Writing story 116000 of 287227; 40.39 percent done Writing story 117000 of 287227; 40.73 percent done Writing story 118000 of 287227; 41.08 percent done Writing story 119000 of 287227; 41.43 percent done Writing story 120000 of 287227; 41.78 percent done Writing story 121000 of 287227; 42.13 percent done Writing story 122000 of 287227; 42.48 percent done Writing story 123000 of 287227; 42.82 percent done Writing story 124000 of 287227; 43.17 percent done Writing story 125000 of 287227; 43.52 percent done Writing story 126000 of 287227; 43.87 percent done Writing story 127000 of 287227; 44.22 percent done Writing story 128000 of 287227; 44.56 percent done Writing story 129000 of 287227; 44.91 percent done Writing story 130000 of 287227; 45.26 percent done Writing story 131000 of 287227; 45.61 percent done Writing story 132000 of 287227; 45.96 percent done Writing story 133000 of 287227; 46.30 percent done Writing story 134000 of 287227; 46.65 percent done Writing story 135000 of 287227; 47.00 percent done Writing story 136000 of 287227; 47.35 percent done Writing story 137000 of 287227; 47.70 percent done Writing story 138000 of 287227; 48.05 percent done Writing story 139000 of 287227; 48.39 percent done Writing story 140000 of 287227; 48.74 percent done Writing story 141000 of 287227; 49.09 percent done Writing story 142000 of 287227; 49.44 percent done Writing story 143000 of 287227; 49.79 percent done Writing story 144000 of 287227; 50.13 percent done Writing story 145000 of 287227; 50.48 percent done Writing story 146000 of 287227; 50.83 percent done Writing story 147000 of 287227; 51.18 percent done Writing story 148000 of 287227; 51.53 percent done Writing story 149000 of 287227; 51.88 percent done Writing story 150000 of 287227; 52.22 percent done Writing story 151000 of 287227; 52.57 percent done Writing story 152000 of 287227; 52.92 percent done Writing story 153000 of 287227; 53.27 percent done Writing story 154000 of 287227; 53.62 percent done Writing story 155000 of 287227; 53.96 percent done Writing story 156000 of 287227; 54.31 percent done Writing story 157000 of 287227; 54.66 percent done Writing story 158000 of 287227; 55.01 percent done Writing story 159000 of 287227; 55.36 percent done Writing story 160000 of 287227; 55.71 percent done Writing story 161000 of 287227; 56.05 percent done Writing story 162000 of 287227; 56.40 percent done Writing story 163000 of 287227; 56.75 percent done Writing story 164000 of 287227; 57.10 percent done Writing story 165000 of 287227; 57.45 percent done Writing story 166000 of 287227; 57.79 percent done Writing story 167000 of 287227; 58.14 percent done Writing story 168000 of 287227; 58.49 percent done Writing story 169000 of 287227; 58.84 percent done Writing story 170000 of 287227; 59.19 percent done Writing story 171000 of 287227; 59.53 percent done Writing story 172000 of 287227; 59.88 percent done Writing story 173000 of 287227; 60.23 percent done Writing story 174000 of 287227; 60.58 percent done Writing story 175000 of 287227; 60.93 percent done Writing story 176000 of 287227; 61.28 percent done Writing story 177000 of 287227; 61.62 percent done Writing story 178000 of 287227; 61.97 percent done Writing story 179000 of 287227; 62.32 percent done Writing story 180000 of 287227; 62.67 percent done Writing story 181000 of 287227; 63.02 percent done Writing story 182000 of 287227; 63.36 percent done Writing story 183000 of 287227; 63.71 percent done Writing story 184000 of 287227; 64.06 percent done Writing story 185000 of 287227; 64.41 percent done Writing story 186000 of 287227; 64.76 percent done Writing story 187000 of 287227; 65.11 percent done Writing story 188000 of 287227; 65.45 percent done Writing story 189000 of 287227; 65.80 percent done Writing story 190000 of 287227; 66.15 percent done Writing story 191000 of 287227; 66.50 percent done Writing story 192000 of 287227; 66.85 percent done Writing story 193000 of 287227; 67.19 percent done Writing story 194000 of 287227; 67.54 percent done Writing story 195000 of 287227; 67.89 percent done Writing story 196000 of 287227; 68.24 percent done Writing story 197000 of 287227; 68.59 percent done Writing story 198000 of 287227; 68.94 percent done Writing story 199000 of 287227; 69.28 percent done Writing story 200000 of 287227; 69.63 percent done Writing story 201000 of 287227; 69.98 percent done Writing story 202000 of 287227; 70.33 percent done Writing story 203000 of 287227; 70.68 percent done Writing story 204000 of 287227; 71.02 percent done Writing story 205000 of 287227; 71.37 percent done Writing story 206000 of 287227; 71.72 percent done Writing story 207000 of 287227; 72.07 percent done Writing story 208000 of 287227; 72.42 percent done Writing story 209000 of 287227; 72.76 percent done Writing story 210000 of 287227; 73.11 percent done Writing story 211000 of 287227; 73.46 percent done Writing story 212000 of 287227; 73.81 percent done Writing story 213000 of 287227; 74.16 percent done Writing story 214000 of 287227; 74.51 percent done Writing story 215000 of 287227; 74.85 percent done Writing story 216000 of 287227; 75.20 percent done Writing story 217000 of 287227; 75.55 percent done Writing story 218000 of 287227; 75.90 percent done Writing story 219000 of 287227; 76.25 percent done Writing story 220000 of 287227; 76.59 percent done Writing story 221000 of 287227; 76.94 percent done Writing story 222000 of 287227; 77.29 percent done Writing story 223000 of 287227; 77.64 percent done Writing story 224000 of 287227; 77.99 percent done Writing story 225000 of 287227; 78.34 percent done Writing story 226000 of 287227; 78.68 percent done Writing story 227000 of 287227; 79.03 percent done Writing story 228000 of 287227; 79.38 percent done Writing story 229000 of 287227; 79.73 percent done Writing story 230000 of 287227; 80.08 percent done Writing story 231000 of 287227; 80.42 percent done Writing story 232000 of 287227; 80.77 percent done Writing story 233000 of 287227; 81.12 percent done Writing story 234000 of 287227; 81.47 percent done Writing story 235000 of 287227; 81.82 percent done Writing story 236000 of 287227; 82.16 percent done Writing story 237000 of 287227; 82.51 percent done Writing story 238000 of 287227; 82.86 percent done Writing story 239000 of 287227; 83.21 percent done Writing story 240000 of 287227; 83.56 percent done Writing story 241000 of 287227; 83.91 percent done Writing story 242000 of 287227; 84.25 percent done Writing story 243000 of 287227; 84.60 percent done Writing story 244000 of 287227; 84.95 percent done Writing story 245000 of 287227; 85.30 percent done Writing story 246000 of 287227; 85.65 percent done Writing story 247000 of 287227; 85.99 percent done Writing story 248000 of 287227; 86.34 percent done Writing story 249000 of 287227; 86.69 percent done Writing story 250000 of 287227; 87.04 percent done Writing story 251000 of 287227; 87.39 percent done Writing story 252000 of 287227; 87.74 percent done Writing story 253000 of 287227; 88.08 percent done Writing story 254000 of 287227; 88.43 percent done Writing story 255000 of 287227; 88.78 percent done Writing story 256000 of 287227; 89.13 percent done Writing story 257000 of 287227; 89.48 percent done Writing story 258000 of 287227; 89.82 percent done Writing story 259000 of 287227; 90.17 percent done Writing story 260000 of 287227; 90.52 percent done Writing story 261000 of 287227; 90.87 percent done Writing story 262000 of 287227; 91.22 percent done Writing story 263000 of 287227; 91.57 percent done Writing story 264000 of 287227; 91.91 percent done Writing story 265000 of 287227; 92.26 percent done Writing story 266000 of 287227; 92.61 percent done Writing story 267000 of 287227; 92.96 percent done Writing story 268000 of 287227; 93.31 percent done Writing story 269000 of 287227; 93.65 percent done Writing story 270000 of 287227; 94.00 percent done Writing story 271000 of 287227; 94.35 percent done Writing story 272000 of 287227; 94.70 percent done Writing story 273000 of 287227; 95.05 percent done Writing story 274000 of 287227; 95.39 percent done Writing story 275000 of 287227; 95.74 percent done Writing story 276000 of 287227; 96.09 percent done Writing story 277000 of 287227; 96.44 percent done Writing story 278000 of 287227; 96.79 percent done Writing story 279000 of 287227; 97.14 percent done Writing story 280000 of 287227; 97.48 percent done Writing story 281000 of 287227; 97.83 percent done Writing story 282000 of 287227; 98.18 percent done Writing story 283000 of 287227; 98.53 percent done Writing story 284000 of 287227; 98.88 percent done Writing story 285000 of 287227; 99.22 percent done Writing story 286000 of 287227; 99.57 percent done Writing story 287000 of 287227; 99.92 percent done Finished writing file finished_files/train.tarWriting vocab file... Finished writing vocab file
转载于:https://www.cnblogs.com/www-caiyin-com/p/10187985.html
运行make_datafiles的过程相关推荐
- CTF--PWN必备技能--理解c程序从编译开始到运行结束的过程
重温c语言 我们在linux平台下建立一个a.c文件,程序很简单,显示输出Please input your name:,然后让我们输入名字,最后调用了一个子函数输出hello,我们的名字 #incl ...
- 记一次rc.local中python脚本无法运行的解决过程
记一次rc.local中python脚本无法运行的解决过程 问题记录: 解决过程: 1. 检查/etc/rc.local的权限 2. 看运行出错日志 3. 修改文件不重启啊(用户切换到root了,我再 ...
- MapReduce运行原理和过程
一．Map的原理和运行流程 Map的输入数据源是多种多样的,我们使用hdfs作为数据源.文件在hdfs上是以block(块,Hdfs上的存储单元)为单位进行存储的. 1.分片 我们将这一个个block ...
- java win7 jdk_WIN7下配置JDK并运行JAVA的过程
WIN7下配置JDK,成功编辑运行JAVA程序的过程: 1. 我安装的是jdk-6u22-windows-i586,安装路径为D:\Java\; 2. 环境变量的配置: (1)JAVA_HOME=D: ...
- docker-machine create -d generic 运行的波折过程及遇见的问题
这是一个愚蠢的学习过程,但是因为觉得过程还是值得记录的,还是写了下来 2>driver = generic 1)在这个过程中使用的都是本地的mac系统,然后尝试在mac本地create -d g ...
- MR详细运行原理及过程
文章目录 MR的原理和运行流程Map的运行过程Reduce处理过程Shuffle过程MR运行过程Yarn && Job MR的原理和运行流程 Map的运行过程 以HDFS上的文件作为默 ...
- matlab运行函数的过程,关于matlab的一些作业一、要求写出窗口运行过程及结果1．利用Matlab求函数f(x)=-x2+ex+lnx的导数、...
共回答了20个问题采纳率:90% 代码 function main() % 一.要求写出窗口运行过程及结果 % % 1．利用Matlab求函数f(x)=-x2+ex+lnx的导数.不定积分和1到10区 ...
- C语言编译运行代码的过程
源程序是指未经编译的,按照一定的程序设计语言规范书写的,人类可读的文本文件,源程序就是所写好的代码.可执行程序,即常说的.exe程序,可以执行程序,完成计算机功能.在C语言中,.c文件就是所谓的源文件 ...
- C语言程序的运行与调试过程
目录 前言 一.编辑 二.编译 三.连接与运行 前言 C语言源程序需要经过编译.连接等一系列步骤才能够生成真正可运行程序. 一.编辑 编辑是指将已经编写好的源程序录入计算机并生成磁盘文件的过程.在编辑 ...
最新文章
- 大龄屌丝自学笔记--Java零基础到菜鸟--028
- 【Linux 内核 内存管理】虚拟地址空间布局架构 ① ( 虚拟地址空间布局架构 | 用户虚拟地址空间划分 )
- html中name和id的区别 [ZT]
- Apache Web Server - httpd的HTTP的多路处理模块MPM
- EMLOG复制网站文字提醒弹窗源码美化版
- 简单易用的网络调试工具——NetAssist
- 5分钟教小白通过ipv6远程访问白群晖
- python 读写西门子PLC 包含S7协议和Fetch/Write协议，s7支持200smart，300PLC，1200PLC，1500PLC...
- C#实现检测打印机状态(包括打印机是否缺纸、打印队列任务数)
- 【百页AI报告】2017人工智能现状、创业图景与未来（98PPT）
- 短信中心号码iphone_如何在iPhone上阻止来自特定号码的呼叫
- codevs 1296
- HTTP状态码分类（常用HTTP状态码和HTTP状态码大全）
- 科普一下IP路由基础
- java设计一个user类_关于JAVA设计一个用户类
- 《目标检测蓝皮书》第5篇 目标检测基础
- 一周信创舆情观察(1.18~1.24)
- HQL17 计算男生人数以及平均GPA
- python猪脸识别_没想到，这是一家AI公司
- C#中无法将文件“obj\x86\Debug\xxx.exe”复制到“bin\Debug\xxx.exe”。文件“bin\Debug\xxx.exe”正由另一进程使用，因此该进程无法访问此文件.
热门文章
- java数组原理_Java数组排序原理
- php psd图层重命名,ps批量修改图层名字的脚本(附批量替换方法)
- centos7 怎么封装自己的镜像_在Centos7系统上制作一个7系的Docker镜像
- keep it SMPL: Automatic estimation of 3d human pose and shape from a single image
- Dropout抑制过拟合
- java实现EXcel的RC地址变成常规地址
- PyTorch深度学习快速实战入门《pytorch-handbook》
- mvc html安全检测,Spring MVC和HtmlUnit测试
- java上帝模块常见的情况_JVM上帝视角看JVM内存模型，分而治之论各模块详情详解...
- node访问oracledb的环境搭建