VSCrawler 爬虫 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 解决方案
长长的报错信息如下:
13:39:15.345 [main] WARN c.v.vscrawler.core.event.EventLoop - 程序已停止
13:39:15.376 [main] INFO c.v.v.core.config.DirectoryWatcher - 注册事件:ENTRY_MODIFY
13:39:15.376 [main] INFO c.v.v.core.config.DirectoryWatcher - 注册事件:ENTRY_DELETE
13:39:15.376 [main] INFO c.v.v.core.config.DirectoryWatcher - 监控目录:E:\ideaspace\spider\Demo\target\classes
13:39:15.376 [main] INFO c.v.v.c.seed.BerkeleyDBSeedManager - vsCrawler配置工作目录:classpath:work
13:39:15.392 [main] INFO c.v.v.c.seed.BerkeleyDBSeedManager - vsCrawler实际工作目录:E:\ideaspace\spider\Demo\target\classes\work
13:39:15.830 [main] INFO c.v.v.core.seed.LocalFileSeedSource - 没有配置初始种子
13:39:15.830 [main] INFO c.v.v.c.seed.BerkeleyDBSeedManager - import new init seeds:0
################################################
############## VSCrawler ##############
############## 0.0.1 ##############
############## 你有一个有意思的灵魂 ##############
################################################
############## virjar ##############
################################################
13:39:16.111 [VSCrawler-Dispatch] INFO com.virjar.vscrawler.core.VSCrawler - Spider started!
Exception in thread "VSCrawlerWorker-thread-1" Exception in thread "VSCrawlerWorker-thread-2" Exception in thread "VSCrawlerWorker-thread-6" Exception in thread "VSCrawlerWorker-thread-3" Exception in thread "createNewSession" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactoryat org.apache.http.impl.client.DefaultRedirectStrategy.<init>(DefaultRedirectStrategy.java:76)at org.apache.http.impl.client.DefaultRedirectStrategy.<clinit>(DefaultRedirectStrategy.java:84)at com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactoryat java.net.URLClassLoader.findClass(URLClassLoader.java:381)at java.lang.ClassLoader.loadClass(ClassLoader.java:424)at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)at java.lang.ClassLoader.loadClass(ClassLoader.java:357)... 11 more
java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
Exception in thread "VSCrawlerWorker-thread-4" java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
Exception in thread "VSCrawlerWorker-thread-5" java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.borrowOne(CrawlerSessionPool.java:157)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.processSeed(VSCrawler.java:234)at com.virjar.vscrawler.core.VSCrawler$SeedProcessTask.run(VSCrawler.java:222)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:748)
java.lang.NoClassDefFoundError: Could not initialize class org.apache.http.impl.client.LaxRedirectStrategyat com.virjar.vscrawler.core.net.DefaultHttpClientGenerator.gen(DefaultHttpClientGenerator.java:22)at com.virjar.vscrawler.core.net.session.CrawlerSession.<init>(CrawlerSession.java:69)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.createNewSession(CrawlerSessionPool.java:126)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool.access$600(CrawlerSessionPool.java:28)at com.virjar.vscrawler.core.net.session.CrawlerSessionPool$CreateSessionThread.run(CrawlerSessionPool.java:287)
尝试停止爬虫江城子 . 程序员之歌13:39:31.191 [vsCrawlerEventLoop] INFO com.virjar.vscrawler.core.VSCrawler - 爬虫停止,发送爬虫停止事件消息:com.virjar.vscrawler.event.systemevent.CrawlerEndEvent十年生死两茫茫,写程序,到天亮。
13:39:31.191 [VSCrawler-Dispatch] WARN com.virjar.vscrawler.core.VSCrawler - 爬虫线程休眠被打断千行代码,Bug何处藏。
13:39:31.191 [vsCrawlerEventLoop] INFO c.v.vscrawler.core.event.EventLoop - 收到爬虫结束消息,停止事件循环,未处理将被忽略,当前待处理事件个数:2
13:39:31.191 [vsCrawlerEventLoop] INFO c.v.v.c.seed.BerkeleyDBSeedManager - 收到爬虫结束消息,开始关闭资源
13:39:31.191 [vsCrawlerEventLoop] INFO c.v.v.c.seed.BerkeleyDBSeedManager - 拒绝抓取结果入库...纵使上线又怎样,朝令改,夕断肠。
13:39:31.191 [vsCrawlerEventLoop] INFO c.v.v.c.seed.BerkeleyDBSeedManager - 缓存中未分发数据重新入库,正在执行的爬虫任务,不等待结果,重新入库...13:39:31.191 [VSCrawler-Dispatch] INFO com.virjar.vscrawler.core.VSCrawler - 爬虫已经停止,不需要发生爬虫停止事件消息领导每天新想法,天天改,日日忙。
13:39:31.191 [VSCrawler-Dispatch] INFO com.virjar.vscrawler.core.VSCrawler - 爬虫结束相顾无言,惟有泪千行。
13:39:31.191 [vsCrawler-resource-clean] WARN com.virjar.vscrawler.core.VSCrawler - 爬虫被外部中断,尝试进行资源关闭等收尾工作每晚灯火阑珊处,夜难寐,加班狂。
13:39:31.191 [vsCrawler-resource-clean] INFO com.virjar.vscrawler.core.VSCrawler - 爬虫已经停止,不需要发生爬虫停止事件消息
笔者在win10旗舰版,intellij idea 2017.2 预览版,jdk 1.8.0_131 下按照官方教程http://vscrawler.scumall.com/进行配置,试图跑Demo时候报了以上错误。
至于bug原因,是没有导入commons-logging-1.0.4包!
解决方案我放两个链接:
1.IntelliJ IDEA添加jar包
2.commons-logging-1.0.4.jar下载地址
VSCrawler 爬虫 java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 解决方案相关推荐
- java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory解决方案
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory解决方案 参考文章: (1)java.lang.NoClass ...
- java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 解决方案
java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 解决方案 NoClassDefFoundErrorLogFa ...
- struts2与struts1整合,java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
原因:我往项目的WEB-INF/lib中导入了struts2基本的包,还有struts1的core包,以及struts2-strut1-plugin的包,但是没有导入commons-loggin-1. ...
- Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
1.错误叙述性说明 2014-7-12 0:38:57 org.apache.catalina.core.ApplicationContext log 信息: No Spring WebApplica ...
- DRP问题集结(一)-Tomcat无法启动,报错java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory...
问题一: Tomcat无法启动,报错java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory 问题二:[Error]Jav ...
- Tomcat无法启动,报错java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory
前面一段时间看到Tomcat7.0发布了几个测试版,由于没有稳定,也就没有测试了,今天看到新闻,看到Tomcat7.0正式版已经发布了,到官网上下载下来,看看效果如何. 下面列出Tomcat 7的一些 ...
- java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory
如果最近你也在学习SpringBoot,当项目运行时,你有可能会遇到这样的异常:java.lang.NoClassDefFoundError: org/apache/juli/logging/LogF ...
- 运行错误:Exception in thread “main“ java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactor
这是我们在做OSS文件上传时容易发生的错误,提示说没有日志相关的类,是因为没有导入相关的jar包,所以我们只需要导入一个commons-logging就好了 <!-- https://mvnre ...
- Java_异常_03_ java.lang.NoClassDefFoundError: org/apache/commons/pool/KeyedObjectPoolFactory
异常信息: java.lang.NoClassDefFoundError: org/apache/commons/pool/KeyedObjectPoolFactory 原因: 我用的是commons ...
最新文章
- 最近喜欢用markdown写笔记,贴个语法说明
- 怎样Selenium IDE选择combo box中值
- go中 -strconv包的使用
- 数据库中间件mycat
- 温故而知新 Vue 原来也有this.$forceUpdate();
- lazyload延迟加载组件
- GWT更改元素样式属性
- html自动广告业代码,html+javascript实现广告窗自由浮动
- linux调度程序是根据进程的动态优先级还是静态优先级来调度进程的,LINUX内核面试题...
- 连通图遍历策略之广度优先搜索(C语言)
- 关于NavigateUrl中绑定Eval()方法时出现服务器标记的格式不正确的解决方法
- bash 不是内部或外部命令_Python安装时提示“Python不是内部或外部命令...”解决方法...
- 20200318_抓取51job招聘数据存数据库
- HL-USB TO RS232驱动
- 三端食堂售饭管理系统
- 课程设计matlab仿真,MATLAB与仿真系统课程设计报告
- 温控-Thermal
- android本地视频播放demo,基于Android的本地视频播放器
- dentity在Java里是什么意思_Function.identity()
- 质量运营在美团点评智能支付业务测试中的初步实践