昨天看到一个同事在通过PL/SQL Developer工具把文本数据往oracle表;有两个文本;一个有30万条记录;一个7万多条记录。
在导入到过程中;出现错误记录还需要点击确认。不过使用黑科技(屏幕精灵)自动点击。单一个7万多记录的文本需要10分钟左右的时间。

看到这种情况;我热心肠爆发;我帮你来提快导入速度。我想到两种方案;
    1:oracle的sqlldr命令
    2:oracle的外部表;
    由于文本存在错误记录;就选择1.
    
文本格式如下:存在有76760条记录

[oracle@oracle234 ~]$ wc -l lottu.txt
76761 lottu.txt
[oracle@oracle234 ~]$ head lottu.txt
stat_user_stay_info.rowkey,stat_user_stay_info.appkey,stat_user_stay_info.phone_softversion,stat_user_stay_info.dim_type,stat_user_stay_info.dim_code,stat_user_stay_info.time_peroid,stat_user_stay_info.stat_date,stat_user_stay_info.indicator,stat_user_stay_info.stat_time,stat_user_stay_info.value
3a00997_7c34d20170108,307A5C626E6C2F6472636E6E6A2F736460656473,2.14.0,cpid,blf1298_12243_001,1,20170105,stay3day,20170109102339,1
3a00997_bf86b20170108,307A5C626E6C2F6472636E6E6A2F736460656473,2.13.0,cpid,blp1375_13621_001,1,20170105,stay3day,20170109102339,7
3a00e87_4b11a20170126,337A5C626E6C2F6472636E6E6A2F736460656473,1.4.0,cpid,all,1,20170123,stay3day,20170127095931,6
3a0129a_6575220170118,307A5C626E6C2F6460726E742F716D7472,all,cpid,bsf1389_10917_001,1,20170116,stay2day,20170119094145,1
3a0183b_5764a20170202,307A5C626E6C2F6472636E6E6A2F736460656473,1.91,cpid,blf1298_12523_001,1,20170128,stay5day,20170203094327,1
3a01b9b_54b4720170123,307A5C626E6C2F6472636E6E6A2F736460656473,2.13.0,cpid,blp1375_13641_001,1,20170122,stay1day,20170124102457,3
3a0230d_7464120170126,307A5C626E6C2F6460726E742F606F65736E686569646D716473,all,cpid,bsp1405_13363_001,1,20170122,stay4day,20170127100446,18
3a02bed_3ea3320170206,307A5C626E6C2F6472636E6E6A2F736460656473,2.15.0,cpid,blp1375_14217_001,1,20170130,stay7day,20170207135438,1
3a03fe3_4c5fe20170119,307A5C21626E6C2F6472776865646E21,all,cpid,bvf1328_10885_001,1,20170116,stay3day,20170120093733,1   

导入表结果如下:

SQL> desc STAT_USER_STAY_INFO1;Name                                      Null?    Type----------------------------------------- -------- ----------------------------JOBID                                              VARCHAR2(64)APPKEY                                    NOT NULL VARCHAR2(200)PHONE_SOFTVERSION                         NOT NULL VARCHAR2(32)DIM_TYPE                                  NOT NULL VARCHAR2(64)DIM_CODE                                  NOT NULL VARCHAR2(64)TIME_PEROID                               NOT NULL VARCHAR2(4)STAT_DATE                                 NOT NULL VARCHAR2(500)INDICATOR                                 NOT NULL VARCHAR2(200)STAT_TIME                                          VARCHAR2(500)VALUE                                     NOT NULL NUMBER

执行sqlldr命令;但结果呢?只导入5万条记录;结果出乎意料。

sqlldr 'lottu/li0924' control=/home/oracle/stay_info.ctl log=/home/oracle/stay_info.log bad=/home/oracle/stay_info.bad

查看log文件;由于篇幅的问题;只取关键部分。

.......
Record 55076: Rejected - Error on table STAT_USER_STAY_INFO1, column DIM_CODE.
ORA-01400: cannot insert NULL into ("LOTTU"."STAT_USER_STAY_INFO1"."DIM_CODE")MAXIMUM ERROR COUNT EXCEEDED - Above statistics reflect partial run.Table STAT_USER_STAY_INFO1:55025 Rows successfully loaded.        51 Rows not loaded due to data errors.0 Rows not loaded because all WHEN clauses were failed.0 Rows not loaded because all fields were null.Space allocated for bind array:                 165120 bytes(64 rows)
Read   buffer bytes: 1048576Total logical records skipped:          0
Total logical records read:         55105
Total logical records rejected:        51
Total logical records discarded:        0Run began on Fri Feb 24 10:51:02 2017
Run ended on Fri Feb 24 10:51:09 2017Elapsed time was:     00:00:06.87
CPU time was:         00:00:00.46  

日志提示;只导入了“55025 Rows successfully loaded. ”;加上拒绝的51条;这跟76761条记录远远不够。
日志记录是不会骗人的;进入表中查看确实是55025条记录

SQL> select count(*) from STAT_USER_STAY_INFO1;COUNT(*)
----------55025

奇了怪;明明76761条记录;为什么oracle只认可5万条记录 。还有2万多条记录为什么不认可呢?
其实oracle给了提示;就在刚刚那个log文件中。只是我忽略这样一句话"MAXIMUM ERROR COUNT EXCEEDED - Above statistics reflect partial run."。
这个意思是说;超过最大所容限错误数。
既然说得这么清楚了;我们在看看sqlldr命令

[oracle@oracle234 ~]$ sqlldrSQL*Loader: Release 11.2.0.1.0 - Production on Fri Feb 24 11:00:08 2017Copyright (c) 1982, 2009, Oracle and/or its affiliates.  All rights reserved.Usage: SQLLDR keyword=value [,keyword=value,...]Valid Keywords:userid -- ORACLE username/password           control -- control file name                  log -- log file name                      bad -- bad file name                      data -- data file name                     discard -- discard file name
discardmax -- number of discards to allow          (Default all)skip -- number of logical records to skip    (Default 0)load -- number of logical records to load    (Default all)errors -- number of errors to allow            (Default 50)rows -- number of rows in conventional path bind array or between direct path data saves(Default: Conventional path 64, Direct path all)bindsize -- size of conventional path bind array in bytes  (Default 256000)silent -- suppress messages during run (header,feedback,errors,discards,partitions)direct -- use direct path                      (Default FALSE)parfile -- parameter file: name of file that contains parameter specificationsparallel -- do parallel load                     (Default FALSE)file -- file to allocate extents from
skip_unusable_indexes -- disallow/allow unusable indexes or index partitions  (Default FALSE)
skip_index_maintenance -- do not maintain indexes, mark affected indexes as unusable  (Default FALSE)
commit_discontinued -- commit loaded rows when load is discontinued  (Default FALSE)readsize -- size of read buffer                  (Default 1048576)
external_table -- use external table for load; NOT_USED, GENERATE_ONLY, EXECUTE  (Default NOT_USED)
columnarrayrows -- number of rows for direct path column array  (Default 5000)
streamsize -- size of direct path stream buffer in bytes  (Default 256000)
multithreading -- use multithreading in direct path  resumable -- enable or disable resumable for current session  (Default FALSE)
resumable_name -- text string to help identify resumable statement
resumable_timeout -- wait time (in seconds) for RESUMABLE  (Default 7200)
date_cache -- size (in entries) of date conversion cache  (Default 1000)
no_index_errors -- abort load on any index errors  (Default FALSE)PLEASE NOTE: Command-line parameters may be specified either by
position or by keywords.  An example of the former case is 'sqlldr
scott/tiger foo'; an example of the latter is 'sqlldr control=foo
userid=scott/tiger'.  One may specify parameters by position before
but not after parameters specified by keywords.  For example,
'sqlldr scott/tiger control=foo logfile=log' is allowed, but
'sqlldr scott/tiger control=foo log' is not, even though the
position of the parameter 'log' is correct. 

其中有一行内容“    errors -- number of errors to allow            (Default 50)”  
所以说上面出现问题就不奇怪了。

最后将sqlldr命令加errors参数即可。

sqlldr 'lottu/li0924' control=/home/oracle/stay_info.ctl log=/home/oracle/stay_info.log bad=/home/oracle/stay_info.bad errors=1000

整个过程20秒处理完成。相比之下;我那同事果断选择这种方法。

Oracle sqlldr导入之“MAXIMUM ERROR COUNT EXCEEDED”相关推荐

  1. oracle sqlldr 数据导入超长,sqlldr 数据导入问题 超长

    本帖最后由 hlwjxsg 于 2012-6-19 10:59 编辑 sql脚本代码: call.sql set echo off set term off set line 1000 pages 0 ...

  2. oracle sqlldr 数据导入错误Field in data file exceeds maximum length解决

    使用oracle sqlldr进行数据导入时报Field in data file exceeds maximum length错误的解决办法: 一种是数据字段确实比数据库中的字段要长,这中错误需要调 ...

  3. oracle sqlldr 数据导入时报错:Field in data file exceeds maximum length

    使用oracle sqlldr进行数据导入时报Field in data file exceeds maximum length错误: 在ctl文件中设置的数据长度小于实际数据长度 如: column ...

  4. oracle sqlldr 数据导入时报错:Field in data file exceeds maximum length完美解决

    使用oracle sqlldr进行数据导入时报Field in data file exceeds maximum length错误的解决办法: 一种是数据字段确实比数据库中的字段要长,这中错误需要调 ...

  5. sqlldr 导入乱码,Oracle客户端字符集问题

    2019独角兽企业重金招聘Python工程师标准>>> 1,查Oracle数据库创建时候的字符集: Oracle服务器端执行 SQL> select name, value$ ...

  6. oracle impdp导入时卡住,Oracle:impdp导入等待statement suspended, wait error to be cleared

    用数据泵impdp往开发数据库导数据,但导入到INDEX时感觉卡住不动了Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX ----查看状态, ...

  7. Oracle数据库导入csv文件(sqlldr命令行)

    1.说明 Oracle数据库导入csv文件, 当csv文件较小时, 可以使用数据库管理工具, 比如DBevaer导入到数据库, 当csv文件很大时, 可以使用Oracle提供的sqlldr命令行工具, ...

  8. ORACLE数据库超出游标异常解决(maximum open cursors exceeded)

    ORACLE数据库超出游标异常解决(maximum open cursors exceeded) 问题背景 ​ 在项目上线实际运行中,有某处功能涉及到批量增加的实现.之前客户在实际使用中每50条增加一 ...

  9. oracle ORA-01000: maximum open cursors exceeded问题的解决方法

    oracle ORA-01000: maximum open cursors exceeded问题的解决方法 参考文章: (1)oracle ORA-01000: maximum open curso ...

最新文章

  1. Hadoop控制输出文件命名
  2. 【Alpha】第一次Daily Scrum Meeting
  3. Linux提升系统安全性:自动注销 TMOUT
  4. c语言实现线性表的算法,数据结构算法代码实现——线性表的定义(一)
  5. 10分钟上线 - API网关 + 函数计算实现图片处理服务
  6. 学Python前途如何,带你了解Python的优势!
  7. python pillow无法安装_Pillow
  8. python数据挖掘学习笔记】十三.WordCloud词云配置过程及词频分析
  9. UIPikerView的属性和使用方法
  10. 使用C#客户端访问FTP服务的一个解决方案
  11. C/C++作用域运算符::
  12. C# WinForm程序中使用Unity3D控件
  13. Cannot load php5apache2_4.dll into server解决办法;
  14. 有关BT5破解wifi密码的流程及当中经历问题的总结——从寻找ISO镜像到破解wifi密码
  15. Linux服务器远程连接
  16. 【Python】P2191 小Z的情书
  17. 高云fpga.Tang Nano 4k(GW1NSR-4C)呼吸灯
  18. CT图像之Hu值变换与窗宽窗位调整
  19. bugku never give up
  20. 中职教资证计算机应用,中职计算机教师资格证只能教中职学校的吗

热门文章

  1. 被割韭菜还是来割韭菜
  2. Realsense相关
  3. Oneday01 | ~scanf、冒泡排序、sort排序
  4. 手工轻松发送匿名邮件
  5. Unity3D教程:2D游戏技能特效
  6. 我用Python量化了1000万次散户操作,然后反着来,胜率竟然高达...?! | 你可以永远相信散户!【量化投资邢不行啊】...
  7. .net 微信开发
  8. 利用pypdf2 安装包 基于 python 制作的PDF 文档合并脚本
  9. 认证资料大全(十三)------ Juniper Networks认证列表
  10. xpath爬取链家二手房