运行记录

hadoop@Mcnode5:~/disk2/home/hadoop/xubo/ref/buildIndex$ bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna
[bwa_index] Pack FASTA... 33.14 sec
[bwa_index] Construct BWT for the packed sequence...
[BWTIncCreate] textLength=6418915856, availableWord=463658232
[BWTIncConstructFromPacked] 10 iterations done. 100000000 characters processed.
[BWTIncConstructFromPacked] 20 iterations done. 200000000 characters processed.
[BWTIncConstructFromPacked] 30 iterations done. 300000000 characters processed.
[BWTIncConstructFromPacked] 40 iterations done. 400000000 characters processed.
[BWTIncConstructFromPacked] 50 iterations done. 500000000 characters processed.
[BWTIncConstructFromPacked] 60 iterations done. 600000000 characters processed.
[BWTIncConstructFromPacked] 70 iterations done. 700000000 characters processed.
[BWTIncConstructFromPacked] 80 iterations done. 800000000 characters processed.
[BWTIncConstructFromPacked] 90 iterations done. 900000000 characters processed.
[BWTIncConstructFromPacked] 100 iterations done. 1000000000 characters processed.
[BWTIncConstructFromPacked] 110 iterations done. 1100000000 characters processed.
[BWTIncConstructFromPacked] 120 iterations done. 1200000000 characters processed.
[BWTIncConstructFromPacked] 130 iterations done. 1300000000 characters processed.
[BWTIncConstructFromPacked] 140 iterations done. 1400000000 characters processed.
[BWTIncConstructFromPacked] 150 iterations done. 1500000000 characters processed.
[BWTIncConstructFromPacked] 160 iterations done. 1600000000 characters processed.
[BWTIncConstructFromPacked] 170 iterations done. 1700000000 characters processed.
[BWTIncConstructFromPacked] 180 iterations done. 1800000000 characters processed.
[BWTIncConstructFromPacked] 190 iterations done. 1900000000 characters processed.
[BWTIncConstructFromPacked] 200 iterations done. 2000000000 characters processed.
[BWTIncConstructFromPacked] 210 iterations done. 2100000000 characters processed.
[BWTIncConstructFromPacked] 220 iterations done. 2200000000 characters processed.
[BWTIncConstructFromPacked] 230 iterations done. 2300000000 characters processed.
[BWTIncConstructFromPacked] 240 iterations done. 2400000000 characters processed.
[BWTIncConstructFromPacked] 250 iterations done. 2500000000 characters processed.
[BWTIncConstructFromPacked] 260 iterations done. 2600000000 characters processed.
[BWTIncConstructFromPacked] 270 iterations done. 2700000000 characters processed.
[BWTIncConstructFromPacked] 280 iterations done. 2800000000 characters processed.
[BWTIncConstructFromPacked] 290 iterations done. 2900000000 characters processed.
[BWTIncConstructFromPacked] 300 iterations done. 3000000000 characters processed.
[BWTIncConstructFromPacked] 310 iterations done. 3100000000 characters processed.
[BWTIncConstructFromPacked] 320 iterations done. 3200000000 characters processed.
[BWTIncConstructFromPacked] 330 iterations done. 3300000000 characters processed.
[BWTIncConstructFromPacked] 340 iterations done. 3400000000 characters processed.
[BWTIncConstructFromPacked] 350 iterations done. 3500000000 characters processed.
[BWTIncConstructFromPacked] 360 iterations done. 3600000000 characters processed.
[BWTIncConstructFromPacked] 370 iterations done. 3700000000 characters processed.
[BWTIncConstructFromPacked] 380 iterations done. 3800000000 characters processed.
[BWTIncConstructFromPacked] 390 iterations done. 3900000000 characters processed.
[BWTIncConstructFromPacked] 400 iterations done. 4000000000 characters processed.
[BWTIncConstructFromPacked] 410 iterations done. 4100000000 characters processed.
[BWTIncConstructFromPacked] 420 iterations done. 4200000000 characters processed.
[BWTIncConstructFromPacked] 430 iterations done. 4300000000 characters processed.
[BWTIncConstructFromPacked] 440 iterations done. 4400000000 characters processed.
[BWTIncConstructFromPacked] 450 iterations done. 4500000000 characters processed.
[BWTIncConstructFromPacked] 460 iterations done. 4600000000 characters processed.
[BWTIncConstructFromPacked] 470 iterations done. 4700000000 characters processed.
[BWTIncConstructFromPacked] 480 iterations done. 4800000000 characters processed.
[BWTIncConstructFromPacked] 490 iterations done. 4900000000 characters processed.
[BWTIncConstructFromPacked] 500 iterations done. 5000000000 characters processed.
[BWTIncConstructFromPacked] 510 iterations done. 5100000000 characters processed.
[BWTIncConstructFromPacked] 520 iterations done. 5200000000 characters processed.
[BWTIncConstructFromPacked] 530 iterations done. 5300000000 characters processed.
[BWTIncConstructFromPacked] 540 iterations done. 5400000000 characters processed.
[BWTIncConstructFromPacked] 550 iterations done. 5500000000 characters processed.
[BWTIncConstructFromPacked] 560 iterations done. 5600000000 characters processed.
[BWTIncConstructFromPacked] 570 iterations done. 5700000000 characters processed.
[BWTIncConstructFromPacked] 580 iterations done. 5798188880 characters processed.
[BWTIncConstructFromPacked] 590 iterations done. 5886472096 characters processed.
[BWTIncConstructFromPacked] 600 iterations done. 5964934432 characters processed.
[BWTIncConstructFromPacked] 610 iterations done. 6034667936 characters processed.
[BWTIncConstructFromPacked] 620 iterations done. 6096643264 characters processed.
[BWTIncConstructFromPacked] 630 iterations done. 6151723072 characters processed.
[BWTIncConstructFromPacked] 640 iterations done. 6200674128 characters processed.
[BWTIncConstructFromPacked] 650 iterations done. 6244177920 characters processed.
[BWTIncConstructFromPacked] 660 iterations done. 6282840176 characters processed.
[BWTIncConstructFromPacked] 670 iterations done. 6317199264 characters processed.
[BWTIncConstructFromPacked] 680 iterations done. 6347733664 characters processed.
[BWTIncConstructFromPacked] 690 iterations done. 6374868704 characters processed.
[BWTIncConstructFromPacked] 700 iterations done. 6398982368 characters processed.
[BWTIncConstructFromPacked] 710 iterations done. 6418915856 characters processed.
[bwt_gen] Finished constructing BWT in 710 iterations.
[bwa_index] 3649.78 seconds elapse.
[bwa_index] Update BWT... 23.62 sec
[bwa_index] Pack forward-only FASTA... 21.46 sec
[bwa_index] Construct SA from BWT and Occ... 1015.61 sec
[main] Version: 0.7.12-r1039
[main] CMD: bwa index GCA_000001405.15_GRCh38_full_analysis_set.fna
[main] Real time: 4891.025 sec; CPU: 4743.604 sec
hadoop@Mcnode5:~/disk2/home/hadoop/xubo/ref/buildIndex$ ls
GCA_000001405.15_GRCh38_full_analysis_set.fna      GCA_000001405.15_GRCh38_full_analysis_set.fna.ann  GCA_000001405.15_GRCh38_full_analysis_set.fna.pac
GCA_000001405.15_GRCh38_full_analysis_set.fna.amb  GCA_000001405.15_GRCh38_full_analysis_set.fna.bwt  GCA_000001405.15_GRCh38_full_analysis_set.fna.sa

内存使用：

16:39:13  memtot memfree buffers   cached  slabmem      swptot swpfree  _mem_
16:39:13  14023M    713M    231M    8247M     571M       6133M   6133M
16:39:14  14023M    711M    231M    8247M     571M       6133M   6133M
16:39:15  14023M    711M    231M    8247M     571M       6133M   6133M
16:39:16  14023M    666M    231M    8247M     571M       6133M   6133M
16:39:17  14023M    372M    231M    8247M     571M       6133M   6133M
***
17:31:09  14023M    358M     89M    5002M     424M       6133M   6089M
17:31:10  14023M    173M     89M    5182M     428M       6133M   6089M
17:31:11  14023M    171M     89M    5186M     425M       6133M   6089M
17:31:12  14023M    154M     89M    5205M     424M       6133M   6089M
17:31:13  14023M    154M     89M    5203M     425M       6133M   6089M
17:31:14  14023M    154M     89M    5204M     425M       6133M   6089M
17:31:15  14023M    154M     89M    5204M     425M       6133M   6089M
17:31:16  14023M    154M     89M    5204M     425M       6133M   6089M
17:31:17  14023M    170M     89M    5188M     425M       6133M   6089M
17:31:18  14023M    154M     89M    5204M     425M       6133M   6089M
17:31:19  14023M    154M     89M    5204M     425M       6133M   6089M
17:31:20  14023M    155M     89M    5204M     425M       6133M   6089M
17:31:21  14023M    176M     89M    5182M     425M       6133M   6089M
17:31:22  14023M    172M     89M    5182M     425M       6133M   6089M
17:31:23  14023M    172M     89M    5182M     425M       6133M   6089M
17:31:24  14023M    172M     89M    5182M     424M       6133M   6089M
17:31:25  14023M   1081M     89M    5182M     424M       6133M   6089M
17:31:26  14023M   4776M     89M    5182M     424M       6133M   6089M
17:31:27  14023M   4767M     89M    5182M     424M       6133M   6089M
17:31:28  14023M   4768M     89M    5182M     424M       6133M   6089M
17:31:29  14023M   4768M     89M    5182M     424M       6133M   6089M
17:31:30  14023M   4768M     89M    5182M     424M       6133M   6089M
17:31:31  14023M   4768M     89M    5182M     424M       6133M   6089M
17:31:32  14023M   4768M     89M    5182M     424M       6133M   6089M
17:31:33  14023M   4768M     89M    5182M     424M       6133M   6089M

参考

【1】https://github.com/xubo245/AdamLearning
【2】https://github.com/bigdatagenomics/adam/
【3】https://github.com/xubo245/SparkLearning
【4】http://spark.apache.org

研究成果：

【1】 [BIBM] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Chao Wang, and Xuehai Zhou, "Distributed Gene Clinical Decision Support System Based on Cloud Computing", in IEEE International Conference on Bioinformatics and Biomedicine. (BIBM 2017, CCF B)
【2】 [IEEE CLOUD] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Xuehai Zhou. Efficient Distributed Smith-Waterman Algorithm Based on Apache Spark (CLOUD 2017, CCF-C).
【3】 [CCGrid] Bo Xu, Changlong Li, Hang Zhuang, Jiali Wang, Qingfeng Wang, Jinhong Zhou, Xuehai Zhou. DSA: Scalable Distributed Sequence Alignment System Using SIMD Instructions. (CCGrid 2017, CCF-C).
【4】more: https://github.com/xubo245/Publications

Help

If you have any questions or suggestions, please write it in the issue of this project or send an e-mail to me: xubo245@mail.ustc.edu.cn
Wechat: xu601450868
QQ: 601450868

基因数据处理114之BWA建立全基因组索引成功相关推荐

  1. 水稻PHP基因,科学网—38个水稻全基因组序列 - 闫双勇的博文

    我搜集的水稻基因组序列,如果有新序列出来,请告诉我下 16个下面这篇文章提到的水稻基因组 包括这些品种: Azucena_GJ-trop1 IR64_XI-1B1 IRGC_109232-1_XI-3 ...

  2. 基因数据处理8之BWA_MEM小数据集处理（成功）

    基因数据处理8之BWA_MEM小数据集处理 环境:ubuntu14.04 6G内存 参考基因:GRCH38 来源请参考[1] 1.fastq数据:SRR003161.fastq 的头20行,即5条re ...

  3. 基因数据处理22之对GRCH38全基因建立BWA索引

    环境: ubuntu 14.04 内存 6G bwa 0.7.12 结论: 建立索引大概4500秒左右 节点2运行: hadoop@Mcnode2:~/cloud/adam/xubo/data/tes ...

  4. 基因数据处理56之bwa运行paird-end（1千万条100bp的reads）.md

    (1)pair1.fq>sai bwa aln GRCH38BWAindex/GRCH38chr1L3556522.fasta g38L100c10000000Nhs20Paired1.fq & ...

  5. 易基因｜手把手教你做全基因组DNA甲基化测序分析

    大家好,这是专注表观组学十余年,领跑多组学科研服务的易基因. 本期,我们讲讲全基因组DNA甲基化实验怎么做,从技术原理.建库测序流程.信息分析流程和研究套路等四方面详细介绍. 一.全基因组甲基化测序技 ...

  6. 关于基因家族的全基因组鉴定和表达分析的研究步骤

    关于基因家族的全基因组鉴定和表达分析的研究步骤大致包括以下几点: 样本收集: 从相应的生物材料中提取DNA/RNA样本. 测序: 进行全基因组测序或转录组测序. 数据分析: 对测序得到的数据进行预处理 ...

  7. cfDNA（circulating cell free DNA）全基因组测序

    参考资料: [cfDNA专题]cell-free DNA在非肿瘤疾病中的临床价值(好) ctDNA, cfDNA和CTCs有什么区别吗? cfDNA你懂多少? 新发现 | 基因是否表达,做个cfDNA ...

  8. 1-1 GWAS（全基因组关联分析基本概念和材料选择）

    先把GWAS系列课程看一遍,后面再把不懂的东西再补充上来 一.概念和理论基础 全基因组关联分析定义 是对多个个体在全基因组范围的遗传变异(标记)多态性进行检测,获得基因型,进而将基因型与可观测的性状, ...

  9. 大熊猫源致病大肠杆菌CCHTP全基因组测序及耐药和毒力基因分析

    大熊猫源致病大肠杆菌CCHTP全基因组测序及耐药和毒力基因分析 邓雯文1,李才武1,赵思越2,李仁贵1,何永果1,吴代福1,杨盛智2,黄炎1,张和民1,邹立扣2 1. 中国大熊猫保护研究中心,大熊猫国 ...

最新文章

  1. 手机怎样投屏到电脑_手机有线投屏到Windows电脑
  2. Lanecat网猫的延伸使用
  3. Build Docker image of a Python Flask app【转载】
  4. Uniform String
  5. python做什么方向好_Python工程师的择业方向有哪些？你想好做什么工作了吗？
  6. linux 文本操作
  7. ADSL、SRA、HDSL
  8. 在Windows 7中安装、配置和使用IIS7和ASP
  9. python语言的读法-Python语言的特点及自学建议
  10. 使用plist文件进行ipa的安装
  11. vue3.x自定义换肤
  12. Android：一个妹zhi的学习之路_心得体会
  13. ps批量修改图片大小
  14. 索尼6400夜景测试 镜头索尼18-55
  15. eclipse官网下载不了eclipse开发工具的解决方法
  16. UI设计师=美工？不同是人眼里UI设计师~
  17. 河南在郑州开启5G网络全城试用
  18. 蜂鸣器播放音乐《好运来》^_^
  19. 疯狂Java讲义（五）----第一部分
  20. 常用的计算机有哪些台式的还有哪些,电脑有哪些常用快捷键？70个电脑常用的快捷键大全...

热门文章

  1. java基础巩固-宇宙第一AiYWM：为了维持生计，架构知识+分布式微服务+高并发高可用高性能知识序幕就此拉开（三：注册中心、补充CAP定理、BASE 理论）~整起
  2. 阿里移动技术峰会的一些体会 2015-07-04
  3. 超声加工技术的研究现状及其发展趋势
  4. 什么是Android？
  5. 看到了Pixel 3的刘海，互联网都笑了。。
  6. java 解压缩文件
  7. [ubuntu]用SSH实现ubuntu系统互联并传输文件（无图形界面）
  8. 动态解析ipv6地址，实现域名访问家里网络
  9. OSPF及一类LSA、二类LSA
  10. 达人评测i5 1340p和i7 1360p选哪个 i51340p和i71360p区别