基因数据处理8之BWA_MEM小数据集处理（成功）
2024-06-16 21:17:19
基因数据处理8之BWA_MEM小数据集处理
环境：ubuntu14.04 6G内存
参考基因：GRCH38 来源请参考【1】
1.fastq数据：SRR003161.fastq 的头20行，即5条reads
操作记录：
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161.fastq |head -20 >SRR003161h20.fastq
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ wc -l SRR003161*0 SRR003161a.sam350 SRR003161b.sam
^C
hadoop@Master:~/cloud/adam/xubo/data/test20160310$
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ wc -l SRR003161h20.fastq
20 SRR003161h20.fastq
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h20.fastq >SRR003161h20.sam
[M::bwa_idx_load_from_disk] read 261 ALT contigs
[M::process] read 5 sequences (2691 bp)...
[M::mem_process_seqs] Processed 5 reads in 0.283 CPU sec, 30.784 real sec
[main] Version: 0.7.13-r1126
[main] CMD: bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h20.fastq
[main] Real time: 312.605 sec; CPU: 8.826 sec
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ ls
GCA_000001405.15_GRCh38 SRR003161a.sam SRR003161b.sam SRR003161.fastq SRR003161.fastq.gz SRR003161h20.fastq SRR003161h20.sam SRR003161.sai
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161h20.sam
@SQ SN:chr1 LN:248956422
@SQ SN:chr2 LN:242193529
@SQ SN:chr3 LN:198295559
@SQ SN:chr4 LN:190214555
@SQ SN:chr5 LN:181538259
@SQ SN:chr6 LN:170805979
@SQ SN:chr7 LN:159345973
@SQ SN:chr8 LN:145138636
@SQ SN:chr9 LN:138394717
@SQ SN:chr10 LN:133797422
@SQ SN:chr11 LN:135086622
@SQ SN:chr12 LN:133275309
@SQ SN:chr13 LN:114364328
@SQ SN:chr14 LN:107043718
@SQ SN:chr15 LN:101991189
@SQ SN:chr16 LN:90338345
@SQ SN:chr17 LN:83257441
@SQ SN:chr18 LN:80373285
@SQ SN:chr19 LN:58617616
@SQ SN:chr20 LN:64444167
@SQ SN:chr21 LN:46709983
@SQ SN:chr22 LN:50818468
@SQ SN:chrX LN:156040895
@SQ SN:chrY LN:57227415
@SQ SN:chrM LN:16569
@SQ SN:chr1_KI270706v1_random LN:175055
@SQ SN:chr1_KI270707v1_random LN:32032
@SQ SN:chr1_KI270708v1_random LN:127682
@SQ SN:chr1_KI270709v1_random LN:66860
@SQ SN:chr1_KI270710v1_random LN:40176
@SQ SN:chr1_KI270711v1_random LN:42210
@SQ SN:chr1_KI270712v1_random LN:176043
@SQ SN:chr1_KI270713v1_random LN:40745
@SQ SN:chr1_KI270714v1_random LN:41717
@SQ SN:chr2_KI270715v1_random LN:161471
@SQ SN:chr2_KI270716v1_random LN:153799
@SQ SN:chr3_GL000221v1_random LN:155397
@SQ SN:chr4_GL000008v2_random LN:209709
@SQ SN:chr5_GL000208v1_random LN:92689
@SQ SN:chr9_KI270717v1_random LN:40062
@SQ SN:chr9_KI270718v1_random LN:38054
@SQ SN:chr9_KI270719v1_random LN:176845
@SQ SN:chr9_KI270720v1_random LN:39050
@SQ SN:chr11_KI270721v1_random LN:100316
@SQ SN:chr14_GL000009v2_random LN:201709
@SQ SN:chr14_GL000225v1_random LN:211173
@SQ SN:chr14_KI270722v1_random LN:194050
@SQ SN:chr14_GL000194v1_random LN:191469
@SQ SN:chr14_KI270723v1_random LN:38115
@SQ SN:chr14_KI270724v1_random LN:39555
@SQ SN:chr14_KI270725v1_random LN:172810
@SQ SN:chr14_KI270726v1_random LN:43739
@SQ SN:chr15_KI270727v1_random LN:448248
@SQ SN:chr16_KI270728v1_random LN:1872759
@SQ SN:chr17_GL000205v2_random LN:185591
@SQ SN:chr17_KI270729v1_random LN:280839
@SQ SN:chr17_KI270730v1_random LN:112551
@SQ SN:chr22_KI270731v1_random LN:150754
@SQ SN:chr22_KI270732v1_random LN:41543
@SQ SN:chr22_KI270733v1_random LN:179772
@SQ SN:chr22_KI270734v1_random LN:165050
@SQ SN:chr22_KI270735v1_random LN:42811
@SQ SN:chr22_KI270736v1_random LN:181920
@SQ SN:chr22_KI270737v1_random LN:103838
@SQ SN:chr22_KI270738v1_random LN:99375
@SQ SN:chr22_KI270739v1_random LN:73985
@SQ SN:chrY_KI270740v1_random LN:37240
@SQ SN:chrUn_KI270302v1 LN:2274
@SQ SN:chrUn_KI270304v1 LN:2165
@SQ SN:chrUn_KI270303v1 LN:1942
@SQ SN:chrUn_KI270305v1 LN:1472
@SQ SN:chrUn_KI270322v1 LN:21476
@SQ SN:chrUn_KI270320v1 LN:4416
@SQ SN:chrUn_KI270310v1 LN:1201
@SQ SN:chrUn_KI270316v1 LN:1444
@SQ SN:chrUn_KI270315v1 LN:2276
@SQ SN:chrUn_KI270312v1 LN:998
@SQ SN:chrUn_KI270311v1 LN:12399
@SQ SN:chrUn_KI270317v1 LN:37690
@SQ SN:chrUn_KI270412v1 LN:1179
@SQ SN:chrUn_KI270411v1 LN:2646
@SQ SN:chrUn_KI270414v1 LN:2489
@SQ SN:chrUn_KI270419v1 LN:1029
@SQ SN:chrUn_KI270418v1 LN:2145
@SQ SN:chrUn_KI270420v1 LN:2321
@SQ SN:chrUn_KI270424v1 LN:2140
@SQ SN:chrUn_KI270417v1 LN:2043
@SQ SN:chrUn_KI270422v1 LN:1445
@SQ SN:chrUn_KI270423v1 LN:981
@SQ SN:chrUn_KI270425v1 LN:1884
@SQ SN:chrUn_KI270429v1 LN:1361
@SQ SN:chrUn_KI270442v1 LN:392061
@SQ SN:chrUn_KI270466v1 LN:1233
@SQ SN:chrUn_KI270465v1 LN:1774
@SQ SN:chrUn_KI270467v1 LN:3920
@SQ SN:chrUn_KI270435v1 LN:92983
@SQ SN:chrUn_KI270438v1 LN:112505
@SQ SN:chrUn_KI270468v1 LN:4055
@SQ SN:chrUn_KI270510v1 LN:2415
@SQ SN:chrUn_KI270509v1 LN:2318
@SQ SN:chrUn_KI270518v1 LN:2186
@SQ SN:chrUn_KI270508v1 LN:1951
@SQ SN:chrUn_KI270516v1 LN:1300
@SQ SN:chrUn_KI270512v1 LN:22689
@SQ SN:chrUn_KI270519v1 LN:138126
@SQ SN:chrUn_KI270522v1 LN:5674
@SQ SN:chrUn_KI270511v1 LN:8127
@SQ SN:chrUn_KI270515v1 LN:6361
@SQ SN:chrUn_KI270507v1 LN:5353
@SQ SN:chrUn_KI270517v1 LN:3253
@SQ SN:chrUn_KI270529v1 LN:1899
@SQ SN:chrUn_KI270528v1 LN:2983
@SQ SN:chrUn_KI270530v1 LN:2168
@SQ SN:chrUn_KI270539v1 LN:993
@SQ SN:chrUn_KI270538v1 LN:91309
@SQ SN:chrUn_KI270544v1 LN:1202
@SQ SN:chrUn_KI270548v1 LN:1599
@SQ SN:chrUn_KI270583v1 LN:1400
@SQ SN:chrUn_KI270587v1 LN:2969
@SQ SN:chrUn_KI270580v1 LN:1553
@SQ SN:chrUn_KI270581v1 LN:7046
@SQ SN:chrUn_KI270579v1 LN:31033
@SQ SN:chrUn_KI270589v1 LN:44474
@SQ SN:chrUn_KI270590v1 LN:4685
@SQ SN:chrUn_KI270584v1 LN:4513
@SQ SN:chrUn_KI270582v1 LN:6504
@SQ SN:chrUn_KI270588v1 LN:6158
@SQ SN:chrUn_KI270593v1 LN:3041
@SQ SN:chrUn_KI270591v1 LN:5796
@SQ SN:chrUn_KI270330v1 LN:1652
@SQ SN:chrUn_KI270329v1 LN:1040
@SQ SN:chrUn_KI270334v1 LN:1368
@SQ SN:chrUn_KI270333v1 LN:2699
@SQ SN:chrUn_KI270335v1 LN:1048
@SQ SN:chrUn_KI270338v1 LN:1428
@SQ SN:chrUn_KI270340v1 LN:1428
@SQ SN:chrUn_KI270336v1 LN:1026
@SQ SN:chrUn_KI270337v1 LN:1121
@SQ SN:chrUn_KI270363v1 LN:1803
@SQ SN:chrUn_KI270364v1 LN:2855
@SQ SN:chrUn_KI270362v1 LN:3530
@SQ SN:chrUn_KI270366v1 LN:8320
@SQ SN:chrUn_KI270378v1 LN:1048
@SQ SN:chrUn_KI270379v1 LN:1045
@SQ SN:chrUn_KI270389v1 LN:1298
@SQ SN:chrUn_KI270390v1 LN:2387
@SQ SN:chrUn_KI270387v1 LN:1537
@SQ SN:chrUn_KI270395v1 LN:1143
@SQ SN:chrUn_KI270396v1 LN:1880
@SQ SN:chrUn_KI270388v1 LN:1216
@SQ SN:chrUn_KI270394v1 LN:970
@SQ SN:chrUn_KI270386v1 LN:1788
@SQ SN:chrUn_KI270391v1 LN:1484
@SQ SN:chrUn_KI270383v1 LN:1750
@SQ SN:chrUn_KI270393v1 LN:1308
@SQ SN:chrUn_KI270384v1 LN:1658
@SQ SN:chrUn_KI270392v1 LN:971
@SQ SN:chrUn_KI270381v1 LN:1930
@SQ SN:chrUn_KI270385v1 LN:990
@SQ SN:chrUn_KI270382v1 LN:4215
@SQ SN:chrUn_KI270376v1 LN:1136
@SQ SN:chrUn_KI270374v1 LN:2656
@SQ SN:chrUn_KI270372v1 LN:1650
@SQ SN:chrUn_KI270373v1 LN:1451
@SQ SN:chrUn_KI270375v1 LN:2378
@SQ SN:chrUn_KI270371v1 LN:2805
@SQ SN:chrUn_KI270448v1 LN:7992
@SQ SN:chrUn_KI270521v1 LN:7642
@SQ SN:chrUn_GL000195v1 LN:182896
@SQ SN:chrUn_GL000219v1 LN:179198
@SQ SN:chrUn_GL000220v1 LN:161802
@SQ SN:chrUn_GL000224v1 LN:179693
@SQ SN:chrUn_KI270741v1 LN:157432
@SQ SN:chrUn_GL000226v1 LN:15008
@SQ SN:chrUn_GL000213v1 LN:164239
@SQ SN:chrUn_KI270743v1 LN:210658
@SQ SN:chrUn_KI270744v1 LN:168472
@SQ SN:chrUn_KI270745v1 LN:41891
@SQ SN:chrUn_KI270746v1 LN:66486
@SQ SN:chrUn_KI270747v1 LN:198735
@SQ SN:chrUn_KI270748v1 LN:93321
@SQ SN:chrUn_KI270749v1 LN:158759
@SQ SN:chrUn_KI270750v1 LN:148850
@SQ SN:chrUn_KI270751v1 LN:150742
@SQ SN:chrUn_KI270752v1 LN:27745
@SQ SN:chrUn_KI270753v1 LN:62944
@SQ SN:chrUn_KI270754v1 LN:40191
@SQ SN:chrUn_KI270755v1 LN:36723
@SQ SN:chrUn_KI270756v1 LN:79590
@SQ SN:chrUn_KI270757v1 LN:71251
@SQ SN:chrUn_GL000214v1 LN:137718
@SQ SN:chrUn_KI270742v1 LN:186739
@SQ SN:chrUn_GL000216v2 LN:176608
@SQ SN:chrUn_GL000218v1 LN:161147
@SQ SN:chr1_KI270762v1_alt LN:354444
@SQ SN:chr1_KI270766v1_alt LN:256271
@SQ SN:chr1_KI270760v1_alt LN:109528
@SQ SN:chr1_KI270765v1_alt LN:185285
@SQ SN:chr1_GL383518v1_alt LN:182439
@SQ SN:chr1_GL383519v1_alt LN:110268
@SQ SN:chr1_GL383520v2_alt LN:366580
@SQ SN:chr1_KI270764v1_alt LN:50258
@SQ SN:chr1_KI270763v1_alt LN:911658
@SQ SN:chr1_KI270759v1_alt LN:425601
@SQ SN:chr1_KI270761v1_alt LN:165834
@SQ SN:chr2_KI270770v1_alt LN:136240
@SQ SN:chr2_KI270773v1_alt LN:70887
@SQ SN:chr2_KI270774v1_alt LN:223625
@SQ SN:chr2_KI270769v1_alt LN:120616
@SQ SN:chr2_GL383521v1_alt LN:143390
@SQ SN:chr2_KI270772v1_alt LN:133041
@SQ SN:chr2_KI270775v1_alt LN:138019
@SQ SN:chr2_KI270771v1_alt LN:110395
@SQ SN:chr2_KI270768v1_alt LN:110099
@SQ SN:chr2_GL582966v2_alt LN:96131
@SQ SN:chr2_GL383522v1_alt LN:123821
@SQ SN:chr2_KI270776v1_alt LN:174166
@SQ SN:chr2_KI270767v1_alt LN:161578
@SQ SN:chr3_JH636055v2_alt LN:173151
@SQ SN:chr3_KI270783v1_alt LN:109187
@SQ SN:chr3_KI270780v1_alt LN:224108
@SQ SN:chr3_GL383526v1_alt LN:180671
@SQ SN:chr3_KI270777v1_alt LN:173649
@SQ SN:chr3_KI270778v1_alt LN:248252
@SQ SN:chr3_KI270781v1_alt LN:113034
@SQ SN:chr3_KI270779v1_alt LN:205312
@SQ SN:chr3_KI270782v1_alt LN:162429
@SQ SN:chr3_KI270784v1_alt LN:184404
@SQ SN:chr4_KI270790v1_alt LN:220246
@SQ SN:chr4_GL383528v1_alt LN:376187
@SQ SN:chr4_KI270787v1_alt LN:111943
@SQ SN:chr4_GL000257v2_alt LN:586476
@SQ SN:chr4_KI270788v1_alt LN:158965
@SQ SN:chr4_GL383527v1_alt LN:164536
@SQ SN:chr4_KI270785v1_alt LN:119912
@SQ SN:chr4_KI270789v1_alt LN:205944
@SQ SN:chr4_KI270786v1_alt LN:244096
@SQ SN:chr5_KI270793v1_alt LN:126136
@SQ SN:chr5_KI270792v1_alt LN:179043
@SQ SN:chr5_KI270791v1_alt LN:195710
@SQ SN:chr5_GL383532v1_alt LN:82728
@SQ SN:chr5_GL949742v1_alt LN:226852
@SQ SN:chr5_KI270794v1_alt LN:164558
@SQ SN:chr5_GL339449v2_alt LN:1612928
@SQ SN:chr5_GL383530v1_alt LN:101241
@SQ SN:chr5_KI270796v1_alt LN:172708
@SQ SN:chr5_GL383531v1_alt LN:173459
@SQ SN:chr5_KI270795v1_alt LN:131892
@SQ SN:chr6_GL000250v2_alt LN:4672374
@SQ SN:chr6_KI270800v1_alt LN:175808
@SQ SN:chr6_KI270799v1_alt LN:152148
@SQ SN:chr6_GL383533v1_alt LN:124736
@SQ SN:chr6_KI270801v1_alt LN:870480
@SQ SN:chr6_KI270802v1_alt LN:75005
@SQ SN:chr6_KB021644v2_alt LN:185823
@SQ SN:chr6_KI270797v1_alt LN:197536
@SQ SN:chr6_KI270798v1_alt LN:271782
@SQ SN:chr7_KI270804v1_alt LN:157952
@SQ SN:chr7_KI270809v1_alt LN:209586
@SQ SN:chr7_KI270806v1_alt LN:158166
@SQ SN:chr7_GL383534v2_alt LN:119183
@SQ SN:chr7_KI270803v1_alt LN:1111570
@SQ SN:chr7_KI270808v1_alt LN:271455
@SQ SN:chr7_KI270807v1_alt LN:126434
@SQ SN:chr7_KI270805v1_alt LN:209988
@SQ SN:chr8_KI270818v1_alt LN:145606
@SQ SN:chr8_KI270812v1_alt LN:282736
@SQ SN:chr8_KI270811v1_alt LN:292436
@SQ SN:chr8_KI270821v1_alt LN:985506
@SQ SN:chr8_KI270813v1_alt LN:300230
@SQ SN:chr8_KI270822v1_alt LN:624492
@SQ SN:chr8_KI270814v1_alt LN:141812
@SQ SN:chr8_KI270810v1_alt LN:374415
@SQ SN:chr8_KI270819v1_alt LN:133535
@SQ SN:chr8_KI270820v1_alt LN:36640
@SQ SN:chr8_KI270817v1_alt LN:158983
@SQ SN:chr8_KI270816v1_alt LN:305841
@SQ SN:chr8_KI270815v1_alt LN:132244
@SQ SN:chr9_GL383539v1_alt LN:162988
@SQ SN:chr9_GL383540v1_alt LN:71551
@SQ SN:chr9_GL383541v1_alt LN:171286
@SQ SN:chr9_GL383542v1_alt LN:60032
@SQ SN:chr9_KI270823v1_alt LN:439082
@SQ SN:chr10_GL383545v1_alt LN:179254
@SQ SN:chr10_KI270824v1_alt LN:181496
@SQ SN:chr10_GL383546v1_alt LN:309802
@SQ SN:chr10_KI270825v1_alt LN:188315
@SQ SN:chr11_KI270832v1_alt LN:210133
@SQ SN:chr11_KI270830v1_alt LN:177092
@SQ SN:chr11_KI270831v1_alt LN:296895
@SQ SN:chr11_KI270829v1_alt LN:204059
@SQ SN:chr11_GL383547v1_alt LN:154407
@SQ SN:chr11_JH159136v1_alt LN:200998
@SQ SN:chr11_JH159137v1_alt LN:191409
@SQ SN:chr11_KI270827v1_alt LN:67707
@SQ SN:chr11_KI270826v1_alt LN:186169
@SQ SN:chr12_GL877875v1_alt LN:167313
@SQ SN:chr12_GL877876v1_alt LN:408271
@SQ SN:chr12_KI270837v1_alt LN:40090
@SQ SN:chr12_GL383549v1_alt LN:120804
@SQ SN:chr12_KI270835v1_alt LN:238139
@SQ SN:chr12_GL383550v2_alt LN:169178
@SQ SN:chr12_GL383552v1_alt LN:138655
@SQ SN:chr12_GL383553v2_alt LN:152874
@SQ SN:chr12_KI270834v1_alt LN:119498
@SQ SN:chr12_GL383551v1_alt LN:184319
@SQ SN:chr12_KI270833v1_alt LN:76061
@SQ SN:chr12_KI270836v1_alt LN:56134
@SQ SN:chr13_KI270840v1_alt LN:191684
@SQ SN:chr13_KI270839v1_alt LN:180306
@SQ SN:chr13_KI270843v1_alt LN:103832
@SQ SN:chr13_KI270841v1_alt LN:169134
@SQ SN:chr13_KI270838v1_alt LN:306913
@SQ SN:chr13_KI270842v1_alt LN:37287
@SQ SN:chr14_KI270844v1_alt LN:322166
@SQ SN:chr14_KI270847v1_alt LN:1511111
@SQ SN:chr14_KI270845v1_alt LN:180703
@SQ SN:chr14_KI270846v1_alt LN:1351393
@SQ SN:chr15_KI270852v1_alt LN:478999
@SQ SN:chr15_KI270851v1_alt LN:263054
@SQ SN:chr15_KI270848v1_alt LN:327382
@SQ SN:chr15_GL383554v1_alt LN:296527
@SQ SN:chr15_KI270849v1_alt LN:244917
@SQ SN:chr15_GL383555v2_alt LN:388773
@SQ SN:chr15_KI270850v1_alt LN:430880
@SQ SN:chr16_KI270854v1_alt LN:134193
@SQ SN:chr16_KI270856v1_alt LN:63982
@SQ SN:chr16_KI270855v1_alt LN:232857
@SQ SN:chr16_KI270853v1_alt LN:2659700
@SQ SN:chr16_GL383556v1_alt LN:192462
@SQ SN:chr16_GL383557v1_alt LN:89672
@SQ SN:chr17_GL383563v3_alt LN:375691
@SQ SN:chr17_KI270862v1_alt LN:391357
@SQ SN:chr17_KI270861v1_alt LN:196688
@SQ SN:chr17_KI270857v1_alt LN:2877074
@SQ SN:chr17_JH159146v1_alt LN:278131
@SQ SN:chr17_JH159147v1_alt LN:70345
@SQ SN:chr17_GL383564v2_alt LN:133151
@SQ SN:chr17_GL000258v2_alt LN:1821992
@SQ SN:chr17_GL383565v1_alt LN:223995
@SQ SN:chr17_KI270858v1_alt LN:235827
@SQ SN:chr17_KI270859v1_alt LN:108763
@SQ SN:chr17_GL383566v1_alt LN:90219
@SQ SN:chr17_KI270860v1_alt LN:178921
@SQ SN:chr18_KI270864v1_alt LN:111737
@SQ SN:chr18_GL383567v1_alt LN:289831
@SQ SN:chr18_GL383570v1_alt LN:164789
@SQ SN:chr18_GL383571v1_alt LN:198278
@SQ SN:chr18_GL383568v1_alt LN:104552
@SQ SN:chr18_GL383569v1_alt LN:167950
@SQ SN:chr18_GL383572v1_alt LN:159547
@SQ SN:chr18_KI270863v1_alt LN:167999
@SQ SN:chr19_KI270868v1_alt LN:61734
@SQ SN:chr19_KI270865v1_alt LN:52969
@SQ SN:chr19_GL383573v1_alt LN:385657
@SQ SN:chr19_GL383575v2_alt LN:170222
@SQ SN:chr19_GL383576v1_alt LN:188024
@SQ SN:chr19_GL383574v1_alt LN:155864
@SQ SN:chr19_KI270866v1_alt LN:43156
@SQ SN:chr19_KI270867v1_alt LN:233762
@SQ SN:chr19_GL949746v1_alt LN:987716
@SQ SN:chr20_GL383577v2_alt LN:128386
@SQ SN:chr20_KI270869v1_alt LN:118774
@SQ SN:chr20_KI270871v1_alt LN:58661
@SQ SN:chr20_KI270870v1_alt LN:183433
@SQ SN:chr21_GL383578v2_alt LN:63917
@SQ SN:chr21_KI270874v1_alt LN:166743
@SQ SN:chr21_KI270873v1_alt LN:143900
@SQ SN:chr21_GL383579v2_alt LN:201197
@SQ SN:chr21_GL383580v2_alt LN:74653
@SQ SN:chr21_GL383581v2_alt LN:116689
@SQ SN:chr21_KI270872v1_alt LN:82692
@SQ SN:chr22_KI270875v1_alt LN:259914
@SQ SN:chr22_KI270878v1_alt LN:186262
@SQ SN:chr22_KI270879v1_alt LN:304135
@SQ SN:chr22_KI270876v1_alt LN:263666
@SQ SN:chr22_KI270877v1_alt LN:101331
@SQ SN:chr22_GL383583v2_alt LN:96924
@SQ SN:chr22_GL383582v2_alt LN:162811
@SQ SN:chrX_KI270880v1_alt LN:284869
@SQ SN:chrX_KI270881v1_alt LN:144206
@SQ SN:chr19_KI270882v1_alt LN:248807
@SQ SN:chr19_KI270883v1_alt LN:170399
@SQ SN:chr19_KI270884v1_alt LN:157053
@SQ SN:chr19_KI270885v1_alt LN:171027
@SQ SN:chr19_KI270886v1_alt LN:204239
@SQ SN:chr19_KI270887v1_alt LN:209512
@SQ SN:chr19_KI270888v1_alt LN:155532
@SQ SN:chr19_KI270889v1_alt LN:170698
@SQ SN:chr19_KI270890v1_alt LN:184499
@SQ SN:chr19_KI270891v1_alt LN:170680
@SQ SN:chr1_KI270892v1_alt LN:162212
@SQ SN:chr2_KI270894v1_alt LN:214158
@SQ SN:chr2_KI270893v1_alt LN:161218
@SQ SN:chr3_KI270895v1_alt LN:162896
@SQ SN:chr4_KI270896v1_alt LN:378547
@SQ SN:chr5_KI270897v1_alt LN:1144418
@SQ SN:chr5_KI270898v1_alt LN:130957
@SQ SN:chr6_GL000251v2_alt LN:4795265
@SQ SN:chr7_KI270899v1_alt LN:190869
@SQ SN:chr8_KI270901v1_alt LN:136959
@SQ SN:chr8_KI270900v1_alt LN:318687
@SQ SN:chr11_KI270902v1_alt LN:106711
@SQ SN:chr11_KI270903v1_alt LN:214625
@SQ SN:chr12_KI270904v1_alt LN:572349
@SQ SN:chr15_KI270906v1_alt LN:196384
@SQ SN:chr15_KI270905v1_alt LN:5161414
@SQ SN:chr17_KI270907v1_alt LN:137721
@SQ SN:chr17_KI270910v1_alt LN:157099
@SQ SN:chr17_KI270909v1_alt LN:325800
@SQ SN:chr17_JH159148v1_alt LN:88070
@SQ SN:chr17_KI270908v1_alt LN:1423190
@SQ SN:chr18_KI270912v1_alt LN:174061
@SQ SN:chr18_KI270911v1_alt LN:157710
@SQ SN:chr19_GL949747v2_alt LN:729520
@SQ SN:chr22_KB663609v1_alt LN:74013
@SQ SN:chrX_KI270913v1_alt LN:274009
@SQ SN:chr19_KI270914v1_alt LN:205194
@SQ SN:chr19_KI270915v1_alt LN:170665
@SQ SN:chr19_KI270916v1_alt LN:184516
@SQ SN:chr19_KI270917v1_alt LN:190932
@SQ SN:chr19_KI270918v1_alt LN:123111
@SQ SN:chr19_KI270919v1_alt LN:170701
@SQ SN:chr19_KI270920v1_alt LN:198005
@SQ SN:chr19_KI270921v1_alt LN:282224
@SQ SN:chr19_KI270922v1_alt LN:187935
@SQ SN:chr19_KI270923v1_alt LN:189352
@SQ SN:chr3_KI270924v1_alt LN:166540
@SQ SN:chr4_KI270925v1_alt LN:555799
@SQ SN:chr6_GL000252v2_alt LN:4604811
@SQ SN:chr8_KI270926v1_alt LN:229282
@SQ SN:chr11_KI270927v1_alt LN:218612
@SQ SN:chr19_GL949748v2_alt LN:1064304
@SQ SN:chr22_KI270928v1_alt LN:176103
@SQ SN:chr19_KI270929v1_alt LN:186203
@SQ SN:chr19_KI270930v1_alt LN:200773
@SQ SN:chr19_KI270931v1_alt LN:170148
@SQ SN:chr19_KI270932v1_alt LN:215732
@SQ SN:chr19_KI270933v1_alt LN:170537
@SQ SN:chr19_GL000209v2_alt LN:177381
@SQ SN:chr3_KI270934v1_alt LN:163458
@SQ SN:chr6_GL000253v2_alt LN:4677643
@SQ SN:chr19_GL949749v2_alt LN:1091841
@SQ SN:chr3_KI270935v1_alt LN:197351
@SQ SN:chr6_GL000254v2_alt LN:4827813
@SQ SN:chr19_GL949750v2_alt LN:1066390
@SQ SN:chr3_KI270936v1_alt LN:164170
@SQ SN:chr6_GL000255v2_alt LN:4606388
@SQ SN:chr19_GL949751v2_alt LN:1002683
@SQ SN:chr3_KI270937v1_alt LN:165607
@SQ SN:chr6_GL000256v2_alt LN:4929269
@SQ SN:chr19_GL949752v1_alt LN:987100
@SQ SN:chr6_KI270758v1_alt LN:76752
@SQ SN:chr19_GL949753v2_alt LN:796479
@SQ SN:chr19_KI270938v1_alt LN:1066800
@SQ SN:chrEBV LN:171823
@PG ID:bwa PN:bwa VN:0.7.13-r1126 CL:bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h20.fastq
SRR003161.1 0 chr1 143217889 0 4S35M85S * 0 0 TCAGATGCAATCATCGAATGGTCTCGAATGGAATCNTCTANAGAGATGGAATGTATCNCTCGCCANACGACACNCGAACAGGGNAAGGCAAGCAGNAGGNAGNNNANNNNNNNNNNNNNNNNNN AAAAAAAAAAAAAAAA:::BAAFAABAAB?>>=44!39=<!:866699888220862!08:8002!0200000!022200800!20660000600!000!06!!!6!!!!!!!!!!!!!!!!!! NM:i:1 MD:Z:31A3 AS:i:3XS:i:33 XA:Z:chr10,+42092546,4S35M85S,1;chr1,+143217421,4S35M85S,1;chr1,+143239587,4S35M85S,1;chr1,-143252938,85S35M4S,1;chr1,+143220601,4S35M85S,1;chr1,+143219665,4S35M85S,1;chr1,-143210830,85S35M4S,1;chr10,+42075371,4S35M85S,1;chr10,+42101425,4S35M85S,1;chr1,+143272381,4S35M85S,1;chr1,-143204112,85S35M4S,1;chr1,+143189975,4S35M85S,1;chr10,+42080829,4S35M85S,1;chr10,+42067652,4S35M85S,1;chr1,+143236600,4S35M85S,1;chr10,+42071261,4S35M85S,1;chr1,+143202568,4S35M85S,1;chr1,+143262016,4S35M85S,1;chr10,+42094445,4S35M85S,1;chr1,+143229991,4S35M85S,1;chr1,+143194906,4S35M85S,1;chr10,+42098197,4S35M85S,1;chr1,+143229325,4S35M85S,1;chr1,+143273144,4S35M85S,1;chr1,+143236132,4S35M85S,1;chr3,-196898795,85S35M4S,1;chr1,-125173710,85S35M4S,1;chr10,+42074903,4S35M85S,1;chr1,+143193143,4S35M85S,1;chr1,+143190443,4S35M85S,1;chr10,+42085796,4S35M85S,1;chr1,+143224622,4S35M85S,1;chr1,+143267943,4S35M85S,1;chr10,+42103854,4S35M85S,1;chr1,+143225093,4S35M85S,1;chr1,-143249828,85S35M4S,1;chr1,+143231300,4S35M85S,1;chr1,-143256486,85S35M4S,1;chr1,-143209440,85S35M4S,1;chr1,+143228021,4S35M85S,1;chr1,+143185063,4S35M85S,1;chr10,-41852367,85S35M4S,1;chr1,-143251629,85S35M4S,1;chr1,+143233540,4S35M85S,1;chr10,+42093977,4S35M85S,1;chr1,+143200517,4S35M85S,1;chr1,+143194441,4S35M85S,1;chr10,+42070793,4S35M85S,1;chr1,+143206914,4S35M85S,1;chr1,+143237811,4S35M85S,1;chr1,+143227553,4S35M85S,1;chr1,-143255189,85S35M4S,1;chr1,+143231768,4S35M85S,1;chr1,+143271341,4S35M85S,1;chr10,+42080361,4S35M85S,1;chr1,+143213870,4S35M85S,1;chr10,+42074435,4S35M85S,1;chr1,+143263324,4S35M85S,1;chr10,+42097745,4S35M85S,1;chr10,+42090276,4S35M85S,1;chr1,-125180284,85S35M4S,1;chr1,+143240055,4S35M85S,1;chr1,+143265756,4S35M85S,1;chr1,+143216113,4S35M85S,1;chr1,-125169985,85S35M4S,1;chr1,+143219197,4S35M85S,1;chr1,+143192675,4S35M85S,1;chr10,+42095848,4S35M85S,1;chr1,+143195374,4S35M85S,1;chr1,+143214338,4S35M85S,1;chr1,+143270772,4S35M85S,1;chr1,-125166285,85S35M4S,1;chr1,+143275099,4S35M85S,1;chr1,+143226451,4S35M85S,1;chr10,+42104319,4S35M85S,1;chr1,+143232233,4S35M85S,1;chr1,+143211626,4S35M85S,1;chr1,+143220133,4S35M85S,1;chr1,+143215645,4S35M85S,1;chr10,+42100036,4S35M85S,1;chr10,-41846998,85S35M4S,1;chr1,-125168084,85S35M4S,1;chr1,-125179816,85S35M4S,1;chr1,+143240523,4S35M85S,1;chr1,+143264771,4S35M85S,1;chr1,+143212094,4S34M86S,1;chr10,-41845898,86S34M4S,1;chr1,+143191375,4S31M89S,0;chr1,-125182919,89S31M4S,0;chr1,+143221908,4S31M89S,0;chr1,+143190911,4S31M89S,0;chr10,-41843753,89S31M4S,0;chr10_KI270824v1_alt,+1080,4S35M85S,1;chr10_KI270824v1_alt,+615,4S35M85S,1;
SRR003161.2 0 chr7 41381016 60 4S153M1D132M1D5M1D28M1D73M3I12M1I40M54S * 0 0 TCAGTTTGAGATGGAGTTTCATTCTTGTTGCCCAGGCTGGAGTGCAATGGCGCAATCTCAGCTCACAGCAACCTCCGCCTCCCGGGTTCAAGCGATTCTCCTGCCTCAGCCTCTCGAGTAGCTGGGATTACAGGCATGCACCATCACGCCCAGCTAATTTGCATTTTTTATTAGAGATGGGGTTTCTCCACATTGGTCAGGCTGATCTCGAACTCCTGACCTCAGGTGATCTGCCTGCCTTGGCCTCCCAAAGTGCTGGGATTACAGGCATGAGCCTGAGCCCAACCTATTTACTTTCAATCCATCTTTTCAATAACTTAAATACAAGTGTCAATATATACAATCTTTTCCTCCCTGGTTATCAAGCTTTCTAATATATATGGATGTATCTTCCAAGGTTTTTGATCCCATTTTACTTTACAGGCTCACTGCTGTGGAACCCAGAGAGCAGTCTCTTTTCAAGGNGGGCTGAGACNCGCAACAGGGGATTAGGCCAAGGCNCAGG CCCCCCCCCCCCCCCC@@@CCCFEEEFEEG888EEEFFEEEEFGGGGGGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCA<777@@CCCBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCAAACCCCCCCCCCCCCCCCCCCCCCC:93339@A>77//39AC666666C22CAAAA93333///7-0017>9999>>A???ACCCCCCC2239322>9977<?????CCCCCCCCC877777777111111::::5555:555:::::::::;:555:;;::::0040-----***--467::::;;;;;;:::511155555:555:::;::::::7777744-------///245::;;;::::::;;;;;;;;:55554774----------44-----064---------6---522451115247644255-----,4---24464422---------!,,,4464224!11:::7:::111111--7777---!---- NM:i:1MD:Z:153^T40T91^T5^T28^G73G23C0G26 AS:i:379 XS:i:88
SRR003161.3 0 chr7 59292897 0 4S28M1I35M577S * 0 0 TCAGCATGCTAGACAGAAGAATTCTCAGTAACTTTCTTTGTGCTGTGTGTATTCAACTCACAGAGTTGGAACCGTTCCTTTGTCAACAGAGCTAGAATTTGAAACCNCTCTTGAGGACTACGCGAAANAGGGGANAAGGTCCAAAGGCCAGTANAGGGNTCGGANGTANAAGATNCTNAAAATAAAACNGANAGAATCATTCTNAAGAAACTTNTTGNATGTNTGCCCTTTCAAACTCAACAGGAGTTTACCAAACCTTTTCTTTTCTAAAGGAGACTAAGGTTTTAAGAAAACCACTTACTCGGTCTTTGGTTAATGTCTGCAAAGGTGGATTATTGGACCTTCTTGAGGTCCCTTTCGTTGCGTAAAACCGGGGTTTCTTCCTTTCACTTAGTCGTACGTAACGTAAACGTAAAAGGTAAAGGTTACGTTACGTTAACGTTTAAACGTTTTTTTAACGTTTTGGTTTGGTTTGGTTGTTAGTTTACTTAACCTTAACCTAACCTAAACGTAAAGGTTTAACGGTTAAACCGTTAACGTTACGTTTAACGTTAAGGTAAGGAAGGACGAGTAAGTTAAGTTAAACTAAACTACTAGTAGACGACGACAACGAAGGAGAGAGAGACGACACGAGGAGGAGNGNNN AAAAAAAAAAAAAAAAAAAAAAIFAABA?7792222.,,:3<<<<:0222276:220::20020028662222022000002,220006666=9000669600000!0699788...4877873...!,.333.!......4447........!....!....4!...!..66.!..!....4+++*.!..!.33333686--!---------!--3!332,!,,,,,,,,*,,,,2,,,,,,,,,2,,,,,,,,,,,,.,,((((,(,,,,,),,,,,,,,,,..000----,,(,,,,,,,,,,,,,,,,)),,,,,10,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1,),,,1..,,,,,,,,,,,,)),,,,,,,,,,,,,,,,,,,03330,,,,,,,)))),,0(((,,,,,100,,,,,,,,0,,,,,,-03,----)))),,'''',,(((,,))),,)),,,,,,,,,,))00,,,,,,,,000,,,,,,,,,))),,,,,)),,,)),,,,,0,,)),,11133-,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-,,))),,),,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,10000,,,,,,,,,,!,!!! NM:i:1 MD:Z:63 AS:i:56 XS:i:56
SRR003161.4 0 chr6 71805157 60 4S88M495S * 0 0 TCAGGTTTGGAATGTGGGCTCTGAAGCCATACAACACAGTTTCTACTCTTTATCTTACACCTCCTGACTTTGTGACATTGGTTAAATATTTTATTTATTATNNCATAACTTACTACTTTGTTAAATTAGAAGTACGACTGTCTACACTCTTAGGTAGTTGGTCTGTTGAAATTAAATAATAGNACTTTAACTTACTTAAATAGANATACACACGACTTAGTTAGTTGTTGGCTGGAAATTAGGTATNTGTTTTAGTTCCTACACCTTACTTAACCCTAACCTACCATNTAATACTTTTACTTGTTCTCNGANANATNATAGTNTCTACGTTGAGTATATTACTTATATTACACGGTACGACGGACCGACGTCGTACACGTCTCGTCTTCTNCNANNATGTAGTGAGTCTNTTTATTNTTTCTTAACTACTACTACTCGTTGTAGTAAGTAATAATAANTNNTCTACACCTACGACTGTATTGTAAGTACAAGAAGGACCGACGTTTCGTTACCTTTCTTCTTCGTCCTCTACTTAACCTGTTACTACGTACGCGAACACGGACGTAGGAGGAGGAGGACACGAACGG AAAAAAAAAAAAAAAAAAAAAAIEEAIIIIIIAAIIIA:666AAE???<<<@AA===A=>>AAAAAAAAAAAAA?@???980000040....0/**04490!!00000600.........,,.....,.....74..............33.....7.....4..............++664!.000000.135855----*--!3------------33,,,,,,,,,2222222,,,,*,,,,,!,,,,,,3,((,,,,00,,,,,,,,,,,,,1,,)),,,,01!333001,,,,03((,,,,,,!,,!,!,,!,,3,,!,1,,,,,,,,,,,,,,,,,,,,,,,,3,,,,,433,,,,,,,,13,,,,,,,,,04,,,,,,,,,,,!,!,!!,,,,10,,,311,!,,,1))!,,,,)),,30,,,0330,,,,,,003333,,,,,0003,,!,!!,,01,,,033,,,,,1,,,,,,,,00,,,,,,,,,1331313/.,,,)),,,,,,,)),,,,,,,,,,010,,,,,,,,,,3303,,,,0000000,,,,03,,,,,0,,,,,,,,34333,,,,, NM:i:0 MD:Z:88 AS:i:88 XS:i:20
SRR003161.5 4 * 0 0 * * 0 0 TCAGATGAATGGAGTGGATGGAGTTGAGTGGAATGGAATGGAATGGATGGAGTGGAATTGGAAGTAGTANAGGTNTTGAATGGATGGAATGGATATAGAGGAATGGATGGATGATGGANGTAGNGAGTGAGTAGGAGTACGNGTAGGTACCGTGGAGTTGGGAAGTGTTANAGAGTAAGGNTTGGAGGTAGGTTTGTTAGGTAAAGNGTTACCGGTAAGGTAGGTGGTAAAGTAGGGGGAAAATTTGGGTAAAGGGGTGGGTAAACCGTAGTGGAAAAACCCTTACGGGCGTAAAACGNGGGGGGGAAAATTTAGGGGGGGGTACCCTAAAAGGGGGGGTAACGTAACGGGAAAAACTAGGGGGGGGGTAAATTAAAGGGGGAGAGGAGGTGGGTAAGTAGGTAAAAGTAGGGAGGTGTAAAGTAGGGAGTGGGTAGTAGGGAAATGGGGTAAAAGTGGGGGGTAAAAAGTTTAGGAGGTTTTTAAGGTAGGGAAGGTAAGTAAAAGTAGGGGTAAAAGTAGGGTAGGGTAGGGAAAGTGGGGTAAATTTAGGGGTAGTAGGGGTAAAAGTAGGGTATTTAGGGGGTATAACGGGGTTAGGGGAAGGGTTAGGGTAAGTAAGGTAAGTAGGGAAGTAGGGTAGGTAGGTAGGTTAGGGTAAGTAGGGTAAGGAAGAAGGTAGGGTAAAGTAGGGTAAAGTAGGTAAAGGGTAAGGGTAGTAGGGTAGGTAGGGTAGGTTTTTAGGTAGGTAGGGTAAAAAAAGGTGGTAAAGGTAGGGTAAAGTGGGTAGGGTAGGTAAAAAAAGGGTGTAAGTTTTTAGTAGTTTTAGG AAAAAAAAAAAAAAA446=BAA????B?E????@@BB@B::::?A=<93220;::22222288:85666!0222!00222<<<<96666690000000000009:999777;666;;;!4483!34...............!.............4744.....,....4!3433.....!..44........---*.3-------!-33------3855,,,,,,,,,,252,,,,,,,,00,,,,,8.333444470003/.,11-3,,,,,,,,,,,,,33),,,,0.65565..!//222,,,,--,,,(,,,,,,,,),,,,,,,,,,,,,,,,,((,',,0,,,,,,,,,,,(,,/..33.,,,-,,,,,,,)(,000101,,,,,,),,,,,,,,,,,(,,,,,,1,,,,,,0,.,,,),,,,,,),,,,,,,,,,,,,,,,,,(,,,,,,,,-22,,(,,,(,,,,,,,,,,,,,,(,,,,,,)),,,00----33)...,,,,,,(,),,,33,,,33-)),,,)1...,,,,,,,.,..,,),(30----0,,,,(,(,,,,,,,/0--2,,),,,,,,,,,,,,((((,,)))110,)),,,),,,,,,,,030/..00,10.,,,,)),0)),,,,,,,,,,,,,,30,,,,,,)),,,,,,,,,,,/32,,)0,,,,,,,,),),,,,,,,,,,))))),,)0,,,,,,))0,,,,00,,,,,,,0,,(300,,,,,,,,,,,,,))30,,,,,,,3,,,,3/,,),,./,,))),,,,3----------0),,,,,,,,,00)30,,,,,,, AS:i:0 XS:i:0
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161h20.sam |wc -l
462
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161h20.sam 2.fastq数据：SRR003161.fastq 的头1000行，即250条reads
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161.fastq |head -1000 >SRR003161h1000.fastq
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h1000.fastq >SRR003161h1000.sam
[M::bwa_idx_load_from_disk] read 261 ALT contigs
[M::process] read 250 sequences (161179 bp)...
[M::mem_process_seqs] Processed 250 reads in 10.963 CPU sec, 1525.205 real sec
[main] Version: 0.7.13-r1126
[main] CMD: bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h1000.fastq
[main] Real time: 1805.888 sec; CPU: 18.755 sec
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ wc -l SRR003161h1000.*1000 SRR003161h1000.fastq734 SRR003161h1000.sam1734 total
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161h1000.sam |head -10
@SQ SN:chr1 LN:248956422
@SQ SN:chr2 LN:242193529
@SQ SN:chr3 LN:198295559
@SQ SN:chr4 LN:190214555
@SQ SN:chr5 LN:181538259
@SQ SN:chr6 LN:170805979
@SQ SN:chr7 LN:159345973
@SQ SN:chr8 LN:145138636
@SQ SN:chr9 LN:138394717
@SQ SN:chr10 LN:1337974223.fastq数据：SRR003161.fastq 的头10000行，即2500条reads
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161.fastq |head -10000 >SRR003161h10000.fastq
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h10000.fastq >SRR003161h10000.sam
[M::bwa_idx_load_from_disk] read 261 ALT contigs
[M::process] read 2500 sequences (1499370 bp)...
[M::mem_process_seqs] Processed 2500 reads in 55.162 CPU sec, 6750.093 real sec
[main] Version: 0.7.13-r1126
[main] CMD: bwa mem GCA_000001405.15_GRCh38/GCA_000001405.15_GRCh38_full_analysis_set.fna SRR003161h10000.fastq
[main] Real time: 7224.315 sec; CPU: 69.348 sec
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ cat SRR003161h1000* |wc -l
14798
hadoop@Master:~/cloud/adam/xubo/data/test20160310$ wc -l SRR003161h1000* 10000 SRR003161h10000.fastq3064 SRR003161h10000.sam1000 SRR003161h1000.fastq734 SRR003161h1000.sam14798 total
参考：
【1】 http://blog.csdn.net/xubo245/article/details/50858334
基因数据处理8之BWA_MEM小数据集处理（成功）相关推荐
- 为了提升在小数据集上的性能，有学者让神经网络像生物一样“进化”了｜CVPR2021 Oral...
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达 作者丨二玖 审稿丨邓富城 编辑丨极市平台 导读 如何在较小的数据集上训练神经网络?马里兰大学的学者提出 ...
- 小数据集同样重要！掌握处理它的7种方法
2019-10-20 12:35:14 全文共2427字,预计学习时长5分钟 在现实生活中,通常只能使用一个小数据集.基于少量观测数据所训练出的模型往往会过度拟合,产生不准确的结果.所以即使可用的数据 ...
- 如何使用小数据集对大模型进行微调（迁移学习）-微迁移
1.首先使用小数据集训练大模型一定要有大模型的优化器停止状态参数(没有的话尽量将学习率设置的小点),和停止时候权重参数,也就是要有预训模型 2,最好不要冻结参数,冻结虽然会节约时间但是未必能达到好的效 ...
- 超越ImageNet预训练，Meta AI提出SplitMask，小数据集也能自监督预训练
©作者 | 陈萍 来源 | 机器之心 大规模数据集对自监督预训练是必要的吗?Meta AI 认为,小数据集也能自监督预训练,效果还不错. 目前,计算机视觉神经网络被大量参数化:它们通常有数千万或数亿个 ...
- 朴素贝叶斯(NaiveBayes)针对小数据集中文文本分类预测
转自相国大人的博客, http://blog.csdn.net/github_36326955/article/details/54891204 做个笔记 代码按照1 2 3 4的顺序进行即可: 1. ...
- vs xgb 神经网络_业界 | 深度学习与XGBoost在小数据集上的测评，你怎么看？（附源码）...
作者:Max Brggen机器之心编译参与:蒋思源 近来,部分机器学习从业者对深度学习不能训练小数据集这一观点表示怀疑,他们普遍认为如果深度学习经过优良的调参,那么就不会出现过拟合和过训练情况,也就能 ...
- 基因数据处理12之samtool的tview来查看sam的匹配文件
基因数据处理12之samtool的tview来查看sam的匹配文件 具体的之前有文章讲过:http://blog.csdn.net/xubo245/article/details/50836185 记 ...
- 基因数据处理1之mapping_to_cram
基因数据处理1之mapping_to_cram 参考资料: A Worked Example Obtain some public data We will use the first 100,000 ...
- 基于Keras2《面向小数据集构建图像分类模型》——Kaggle猫狗数据集
概述 在本文中,将使用VGG-16模型提供一种面向小数据集(几百张到几千张图片)构造高效.实用的图像分类器的方法并给出试验结果. 本文将探讨如下几种方法: 从图片中直接训练一个小网络(作为基准方法) ...
最新文章
- 龙岩学院计算机二级报名时间,福建龙岩学院2015年12月计算机等级考试报名通知...
- POJ1942-Paths on a Grid
- 一条SQL语句执行得很慢的原因有哪些？
- java 十进制转十六进制、十进制转二进制、二进制转十进制、二进制转十六进制...
- 在Vue中为什么需要组合API
- android实现标题栏弹框,Android：Dialog对话框、Builder、showDialog、模板方法设计模式...
- L2TP/L2TP over IPSec
- 调用打码平台api获取验证码 (C#版)
- 三星手机和计算机如何连接打印机,WIFI当道 手把手教你如何实现无线打印
- android studio Emulator is outdated
- 水平放滑轮组计算机械效率,初中物理中考常用公式-总结-2
- opencv图像合成
- 医疗his系统值不值得投入使用
- 融券余额大增，市场情绪极其悲观
- Oracle学生表练习
- Aiml智能标记语言规范（20201216）
- python 获取计算机的网卡信息
- Java项目:设备台账管理系统(java+SSM+HTML+JSP+JS+mysql)
- Google Seo基础之泛页面
- ThinkPHP框架下载