CSDN 数据访问可视化，写给CSDN 群友们用用
2024-04-27 20:18:56
文章目录
- CSDN 数据访问可视化，写给CSDN 群友们用用
- 1、大致界面
- 2、GitHub 项目地址
- 3、如何使用
- 4、具体代码
CSDN 数据访问可视化，写给CSDN 群友们用用
1、大致界面
内页就不贴图了
2、GitHub 项目地址
这里是传送门：https://github.com/FrankZuozuo/csdn-chart
有兴趣的话点个小星星吧，虽然没啥用
3、如何使用
数据库脚本先跑一下，包括建立数据库，建立表
## 创建数据库
CREATE DATABASE `csdn` /*!40100 COLLATE 'utf8mb4_general_ci' */;## 创建表
CREATE TABLE `archival_storage`
(`pk_id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,`gmt_create` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,`gmt_modified` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,`remark` VARCHAR(300) NOT NULL DEFAULT '',`sort` INT(11) UNSIGNED NOT NULL DEFAULT '0',`deleted` TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',`article_number` INT(10) UNSIGNED NOT NULL DEFAULT '0',`year` INT(10) UNSIGNED NOT NULL DEFAULT '0',`month` INT(10) UNSIGNED NOT NULL DEFAULT '0',PRIMARY KEY (`pk_id`),UNIQUE INDEX `year_month` (`year`, `month`)
)COLLATE = 'utf8mb4_general_ci'ENGINE = InnoDBAUTO_INCREMENT = 163
;CREATE TABLE `article_info`
(`pk_id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,`gmt_create` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,`gmt_modified` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,`remark` VARCHAR(300) NOT NULL DEFAULT '',`sort` INT(11) UNSIGNED NOT NULL DEFAULT '0',`deleted` TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',`url` VARCHAR(500) NOT NULL DEFAULT '',`article_name` VARCHAR(500) NOT NULL DEFAULT '',`read_number` INT(11) NOT NULL DEFAULT '0',`comment_number` INT(11) NOT NULL DEFAULT '0',PRIMARY KEY (`pk_id`),UNIQUE INDEX `url` (`url`)
)COLLATE = 'utf8mb4_general_ci'ENGINE = InnoDBAUTO_INCREMENT = 1325
;CREATE TABLE `data`
(`pk_id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,`gmt_create` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,`gmt_modified` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,`remark` VARCHAR(300) NOT NULL DEFAULT '',`sort` INT(11) UNSIGNED NOT NULL DEFAULT '0',`deleted` TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',`level` INT(10) UNSIGNED NOT NULL DEFAULT '0',`visit_number` INT(10) UNSIGNED NOT NULL DEFAULT '0',`integral` INT(10) UNSIGNED NOT NULL DEFAULT '0',`top` INT(10) UNSIGNED NOT NULL DEFAULT '0',`article_number` INT(10) UNSIGNED NOT NULL DEFAULT '0',`fans` INT(10) UNSIGNED NOT NULL DEFAULT '0',`like_number` INT(10) UNSIGNED NOT NULL DEFAULT '0',`comment_number` INT(10) UNSIGNED NOT NULL DEFAULT '0',`time_point` INT(10) UNSIGNED NOT NULL DEFAULT '0',PRIMARY KEY (`pk_id`)
)COLLATE = 'utf8mb4_general_ci'ENGINE = InnoDBAUTO_INCREMENT = 197
;CREATE TABLE `user_info`
(`pk_id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,`gmt_create` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,`gmt_modified` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,`remark` VARCHAR(300) NOT NULL DEFAULT '',`sort` INT(11) UNSIGNED NOT NULL DEFAULT '0',`deleted` TINYINT(1) UNSIGNED NOT NULL DEFAULT '0',`nickname` VARCHAR(100) NOT NULL DEFAULT '',`head_img_url` VARCHAR(200) NOT NULL DEFAULT '',`motto` VARCHAR(300) NOT NULL DEFAULT '',`version` BIGINT(20) NOT NULL DEFAULT '0',PRIMARY KEY (`pk_id`)
)COLLATE = 'utf8mb4_general_ci'ENGINE = InnoDBAUTO_INCREMENT = 555
;然后把项目里面的application.yml数据库连接信息改成你自己的，博客地址csdn.csdn-url改成你自己的就可以了。
spring:application:name: csdn-chartdatasource:driver-class-name: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://localhost:3306/csdn?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT%2B8username: rootpassword: rootthymeleaf:cache: falseexecute:core-pool-size: 10max-pool-size: 200queue-capacity: 10server:port: 9253csdn:csdn-url: https://wretchant.blog.csdn.net
报表的数据是每5分钟抓取一次，你可以根据自己的实际情况修改
50万访问以下的，建议1-2个小时抓取一次，50-500万的，建议半个小时到1个小时抓取一次，500万以上的，建议10-20分钟左右抓取一次
修改一下DataTimer 类的 fixedRate 即可
4、具体代码
package com.wretchant.csdnchart.annotation;import java.lang.annotation.*;@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE, ElementType.FIELD})
@Documented
public @interface InfoLog {String value();
}package com.wretchant.csdnchart.configuration;import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;@EnableAspectJAutoProxy
@Configuration
public class AspectConfiguration {}package com.wretchant.csdnchart.configuration;import com.wretchant.csdnchart.annotation.InfoLog;
import lombok.Setter;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableAsync;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;import java.util.concurrent.Executor;@ConfigurationProperties("spring.execute")
@EnableAsync
@Configuration
@Setter
public class ThreadConfiguration {private int corePoolSize;private int maxPoolSize;private int queueCapacity;@Bean@InfoLog("开始设置线程池")Executor executor() {ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();executor.setCorePoolSize(corePoolSize);executor.setMaxPoolSize(maxPoolSize);executor.setQueueCapacity(queueCapacity);return executor;}
}package com.wretchant.csdnchart.configuration;import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;@EnableScheduling
@Configuration
public class TimerConfiguration {}package com.wretchant.csdnchart.core;import com.wretchant.csdnchart.annotation.InfoLog;public interface Converter<IN, OUT> {@InfoLog("进行对象互转")OUT converter(IN in);
}package com.wretchant.csdnchart.core;import lombok.AllArgsConstructor;
import lombok.Data;@AllArgsConstructor
@Data
public class R<T> {private T t;private String msg;private int code;public static <T> R<T> success(T t) {return new R<T>(t, "success", 200);}public static R<String> success() {return new R<String>("", "success", 200);}public static R<String> fail() {return new R<String>("", "fail", 500);}public static R<String> fail(String msg) {return new R<String>("", msg, 500);}
}package com.wretchant.csdnchart.ctrl.base;public abstract class BaseCtrl {}package com.wretchant.csdnchart.ctrl.business.api;import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.core.R;
import com.wretchant.csdnchart.entity.DataTable;
import com.wretchant.csdnchart.entity.DataTableEnum;
import com.wretchant.csdnchart.service.DataTableService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;import java.time.format.DateTimeFormatter;
import java.util.Collections;
import java.util.List;
import java.util.Map;@RestController
@RequestMapping("/api/data")
public class DataCtrl {private final DataTableService dataTableService;public DataCtrl(DataTableService dataTableService) {this.dataTableService = dataTableService;}@InfoLog("前端获取统计数据")@GetMapping("/t/{size}")public R t(DataTableEnum dataTableEnum, @PathVariable(name = "size") Integer size) {List<DataTable> list = dataTableService.list(size);Map<String, Object> map = Maps.newHashMapWithExpectedSize(2);List<String> datetime = Lists.newArrayListWithExpectedSize(size);List<Integer> data = Lists.newArrayListWithExpectedSize(size);list.forEach(dataTable -> {String pattern = "yyyy-MM-dd HH:mm:ss";String format = DateTimeFormatter.ofPattern(pattern).format(dataTable.getGmtCreate());datetime.add(format);switch (dataTableEnum) {case TOP:data.add(dataTable.getTop());break;case FANS:data.add(dataTable.getFans());break;case LEVEL:data.add(dataTable.getLevel());break;case INTEGRAL:data.add(dataTable.getIntegral());break;case LIKE_NUMBER:data.add(dataTable.getLikeNumber());break;case VISIT_NUMBER:data.add(dataTable.getVisitNumber());break;case ARTICLE_NUMBER:data.add(dataTable.getArticleNumber());break;case COMMENT_NUMBER:data.add(dataTable.getCommentNumber());break;default:}});Integer max = Collections.max(data);Integer min = Collections.min(data);map.put("max", max);map.put("min", min);map.put("datetime", Lists.reverse(datetime));map.put("data", Lists.reverse(data));return R.success(map);}
}package com.wretchant.csdnchart.ctrl.business.web;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.service.DataTableService;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;@RequestMapping("/web/add")
@Controller
public class WebAddCtrl {private final DataTableService dataTableService;public WebAddCtrl(DataTableService dataTableService) {this.dataTableService = dataTableService;}@InfoLog("查看数据增量")@GetMapping("/{size}")public String add(Model model, @PathVariable(name = "size") Integer size) {model.addAttribute("list", dataTableService.count(size));return "add";}
}package com.wretchant.csdnchart.ctrl.business.web;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.service.ArticleInfoService;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;@RequestMapping("/web/article")
@Controller
public class WebArticleCtrl {private final ArticleInfoService articleInfoService;public WebArticleCtrl(ArticleInfoService articleInfoService) {this.articleInfoService = articleInfoService;}@GetMapping@InfoLog("进入报表页面")public String chart(Model model) {model.addAttribute("list", articleInfoService.list());return "article";}
}package com.wretchant.csdnchart.ctrl.business.web;import com.wretchant.csdnchart.annotation.InfoLog;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;@RequestMapping("/web/chart")
@Controller
public class WebChartCtrl {@GetMapping("/{size}")@InfoLog("进入报表页面")public String chart(Model model, @PathVariable(name = "size") Integer size) {model.addAttribute("size", size);return "chart";}
}package com.wretchant.csdnchart.ctrl.business.web;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ConduitEnum;
import com.wretchant.csdnchart.service.ArticleInfoService;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;@RequestMapping("/web/conduit")
@Controller
public class WebConduitCtrl {private final ArticleInfoService articleInfoService;public WebConduitCtrl(ArticleInfoService articleInfoService) {this.articleInfoService = articleInfoService;}@InfoLog("查看Conduit 文章列表")@GetMapping("/{conduit}")public String add(Model model, @PathVariable(name = "conduit") ConduitEnum conduit) {model.addAttribute("list", articleInfoService.list(conduit));return "article";}
}package com.wretchant.csdnchart.ctrl.business.web;import cn.hutool.core.bean.BeanUtil;
import com.google.common.collect.Lists;
import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.DataTable;
import com.wretchant.csdnchart.entity.DataTableVo;
import com.wretchant.csdnchart.service.DataTableService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;import java.util.List;@RequestMapping("/web/data")
@Slf4j
@RestController
public class WebDataCtrl {private final DataTableService dataTableService;public WebDataCtrl(DataTableService dataTableService) {this.dataTableService = dataTableService;}@InfoLog("前端查看全部的统计数据")@GetMappingpublic String index(Model model) {List<DataTableVo> vos = Lists.newArrayList();List<DataTable> list = dataTableService.list();for (int i = 0; i < list.size(); i++) {DataTable dataTable = list.get(i);DataTableVo dataTableVo = new DataTableVo();BeanUtil.copyProperties(dataTable, dataTableVo);if (i != 0) {DataTable old = list.get(i - 1);dataTableVo.setLevelAdd(dataTable.getLevel() - old.getLevel());dataTableVo.setVisitNumberAdd(dataTable.getVisitNumber() - old.getVisitNumber());dataTableVo.setIntegralAdd(dataTable.getIntegral() - old.getIntegral());dataTableVo.setTopAdd(dataTable.getTop() - old.getTop());dataTableVo.setArticleNumberAdd(dataTable.getArticleNumber() - old.getArticleNumber());dataTableVo.setFansAdd(dataTable.getFans() - old.getFans());dataTableVo.setLikeNumberAdd(dataTable.getLikeNumber() - old.getLikeNumber());dataTableVo.setCommentNumberAdd(dataTable.getCommentNumber() - old.getCommentNumber());}vos.add(dataTableVo);}model.addAttribute("list", vos);return "data";}
}package com.wretchant.csdnchart.ctrl.business;import com.google.common.collect.Lists;
import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ArchivalStorage;
import com.wretchant.csdnchart.service.ArchivalStorageService;
import com.wretchant.csdnchart.service.ArticleInfoService;
import com.wretchant.csdnchart.service.DataTableService;
import com.wretchant.csdnchart.service.UserInfoService;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;import java.util.Collections;
import java.util.List;@RequestMapping("/")
@Controller
public class IndexCtrl {private final ArchivalStorageService archivalStorageService;private final ArticleInfoService articleInfoService;private final DataTableService dataTableService;private final UserInfoService userInfoService;public IndexCtrl(UserInfoService userInfoService,ArchivalStorageService archivalStorageService,ArticleInfoService articleInfoService,DataTableService dataTableService) {this.userInfoService = userInfoService;this.archivalStorageService = archivalStorageService;this.articleInfoService = articleInfoService;this.dataTableService = dataTableService;}@InfoLog("进入首页")@GetMappingpublic String index(Model model) {model.addAttribute("userInfo", userInfoService.findTop());List<ArchivalStorage> list = archivalStorageService.list();List<String> dates = Lists.newArrayListWithExpectedSize(list.size());List<Integer> datas = Lists.newArrayListWithExpectedSize(list.size());list = Lists.reverse(list);list.forEach(archivalStorage -> {datas.add(archivalStorage.getArticleNumber());dates.add(archivalStorage.getYear() + "." + archivalStorage.getMonth());});model.addAttribute("userInfo", userInfoService.findTop());model.addAttribute("dates", dates);model.addAttribute("datas", datas);model.addAttribute("max", Collections.max(datas));model.addAttribute("min", Collections.min(datas));model.addAttribute("last", dataTableService.last());model.addAttribute("count", articleInfoService.count());return "index";}
}package com.wretchant.csdnchart.entity;import lombok.Data;
import org.hibernate.annotations.DynamicInsert;
import org.hibernate.annotations.DynamicUpdate;import javax.persistence.*;
import java.io.Serializable;
import java.time.LocalDateTime;@DynamicUpdate
@DynamicInsert
@Table(name = "archival_storage")
@Entity
@Data
public class ArchivalStorage implements Serializable {private static final long serialVersionUID = -5445657205714414658L;@Id@GeneratedValue(strategy = GenerationType.IDENTITY)private Long pkId;private LocalDateTime gmtCreate;private LocalDateTime gmtModified;private Integer sort;private String remark;private Boolean deleted;/** 文章数量 */private Integer articleNumber;private Integer year;private Integer month;
}package com.wretchant.csdnchart.entity;import lombok.Data;
import org.hibernate.annotations.DynamicInsert;
import org.hibernate.annotations.DynamicUpdate;import javax.persistence.*;
import java.io.Serializable;
import java.time.LocalDateTime;@DynamicUpdate
@DynamicInsert
@Table(name = "article_info")
@Entity
@Data
public class ArticleInfo implements Serializable {private static final long serialVersionUID = -8837487575308630301L;@Id@GeneratedValue(strategy = GenerationType.IDENTITY)private Long pkId;private LocalDateTime gmtCreate;private LocalDateTime gmtModified;private Integer sort;private String remark;private Boolean deleted;/** 文章名称 */private String articleName;/** 文章地址 */private String url;/** 阅读数 */private Integer readNumber;/** 评论数 */private Integer commentNumber;
}package com.wretchant.csdnchart.entity;public enum ConduitEnum {VISIT_MOST,COMMENT_MOST,NEW_MOST
}package com.wretchant.csdnchart.entity;import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;@Configuration
@Data
@ConfigurationProperties("csdn")
public class CsdnConfig {/** csdn 博客主页的链接 */private String csdnUrl;/** 爬取文章数据时，需要的Restful url 路径 */public static final String ARTICLE_LIST = "/article/list/";
}package com.wretchant.csdnchart.entity;import com.google.common.collect.Lists;
import lombok.Data;
import org.hibernate.annotations.DynamicInsert;
import org.hibernate.annotations.DynamicUpdate;import javax.persistence.*;
import java.io.Serializable;
import java.lang.reflect.Field;
import java.time.LocalDateTime;
import java.util.List;@DynamicUpdate
@DynamicInsert
@Table(name = "data")
@Entity
@Data
public class DataTable implements Serializable {private static final long serialVersionUID = -2669109119732793870L;@Id@GeneratedValue(strategy = GenerationType.IDENTITY)private Long pkId;private LocalDateTime gmtCreate;private LocalDateTime gmtModified;private Integer sort;private String remark;private Boolean deleted;/** 时间点 */private Integer timePoint;/** 等级 */private Integer level;/** 访问量 */private Integer visitNumber;/** 积分 */private Integer integral;/** 排名 */private Integer top;/** 原创文章数量 */private Integer articleNumber;/** 粉丝数量 */private Integer fans;/** 喜欢数量 */private Integer likeNumber;/** 评论数量 */private Integer commentNumber;public static void main(String[] args) {Field[] fields = DataTableViewEnum.class.getDeclaredFields();List<Field> fs = Lists.newArrayList(fields);fs.stream().filter(field -> !"$VALUES".equals(field.getName()) && !"field".equals(field.getName())).forEach(field -> {try {DataTableViewEnum dataTableViewEnum =(DataTableViewEnum) field.get(field.getName());System.out.println(dataTableViewEnum.getField());} catch (IllegalAccessException e) {e.printStackTrace();}System.out.println(field.getName());});}
}package com.wretchant.csdnchart.entity;import lombok.Data;@Data
public class DataTableDto {/** 等级 */private Integer level;/** 访问量 */private Integer visitNumber;/** 积分 */private Integer integral;/** 排名 */private Integer top;/** 原创文章数量 */private Integer articleNumber;/** 粉丝数量 */private Integer fans;/** 喜欢数量 */private Integer likeNumber;/** 评论数量 */private Integer commentNumber;
}package com.wretchant.csdnchart.entity;import cn.hutool.core.bean.BeanUtil;
import com.wretchant.csdnchart.core.Converter;import java.time.LocalDateTime;public class DataTableDto2DataTableConverter implements Converter<DataTableDto, DataTable> {@Overridepublic DataTable converter(DataTableDto dataTableDto) {DataTable dataTable = new DataTable();BeanUtil.copyProperties(dataTableDto, dataTable);LocalDateTime now = LocalDateTime.now();int hour = now.getHour();dataTable.setTimePoint(hour);return dataTable;}
}package com.wretchant.csdnchart.entity;public enum DataTableEnum {LEVEL,VISIT_NUMBER,INTEGRAL,TOP,ARTICLE_NUMBER,FANS,LIKE_NUMBER,COMMENT_NUMBER
}package com.wretchant.csdnchart.entity;import lombok.Getter;@Getter
public enum DataTableViewEnum {LEVEL("level"),VISIT_NUMBER("visitNumber"),INTEGRAL("integral"),TOP("top"),ARTICLE_NUMBER("articleNumber"),FANS("fans"),LIKE_NUMBER("likeNumber"),COMMENT_NUMBER("commentNumber");private String field;DataTableViewEnum(String field) {this.field = field;}
}package com.wretchant.csdnchart.entity;import lombok.Data;import java.time.LocalDateTime;@Data
public class DataTableVo {private LocalDateTime gmtCreate;/** 等级 */private Integer level;/** 访问量 */private Integer visitNumber;/** 积分 */private Integer integral;/** 排名 */private Integer top;/** 原创文章数量 */private Integer articleNumber;/** 粉丝数量 */private Integer fans;/** 喜欢数量 */private Integer likeNumber;/** 评论数量 */private Integer commentNumber;// ---- 增量/** 等级 */private Integer levelAdd = 0;/** 访问量 */private Integer visitNumberAdd = 0;/** 积分 */private Integer integralAdd = 0;/** 排名 */private Integer topAdd = 0;/** 原创文章数量 */private Integer articleNumberAdd = 0;/** 粉丝数量 */private Integer fansAdd = 0;/** 喜欢数量 */private Integer likeNumberAdd = 0;/** 评论数量 */private Integer commentNumberAdd = 0;
}package com.wretchant.csdnchart.entity;import lombok.Data;
import org.hibernate.annotations.DynamicInsert;
import org.hibernate.annotations.DynamicUpdate;import javax.persistence.*;
import java.io.Serializable;
import java.time.LocalDateTime;@DynamicUpdate
@DynamicInsert
@Table(name = "user_info")
@Entity
@Data
public class UserInfo implements Serializable {private static final long serialVersionUID = 1118676562861971250L;@Id@GeneratedValue(strategy = GenerationType.IDENTITY)private Long pkId;private LocalDateTime gmtCreate;private LocalDateTime gmtModified;private Integer sort;private String remark;private Boolean deleted;private String nickname;private String headImgUrl;private String motto;private Long version;
}package com.wretchant.csdnchart.framework.aop;import cn.hutool.core.date.DateUtil;
import cn.hutool.core.date.TimeInterval;
import com.wretchant.csdnchart.annotation.InfoLog;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.stereotype.Component;import java.lang.reflect.Method;@Slf4j
@Component
@Aspect
public class InfoLogAspect {@Pointcut("@annotation(com.wretchant.csdnchart.annotation.InfoLog)")public void infoLog() {}@Around("infoLog()")public Object before(ProceedingJoinPoint joinPoint) throws Throwable {MethodSignature signature = (MethodSignature) joinPoint.getSignature();Method method = signature.getMethod();InfoLog annotation = method.getAnnotation(InfoLog.class);String value = annotation.value();TimeInterval timer = DateUtil.timer();Object proceed = joinPoint.proceed();if (log.isInfoEnabled()) {log.info("[{}] : [{}]", timer.interval(), value);}return proceed;}
}package com.wretchant.csdnchart.framework;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ArchivalStorage;
import com.wretchant.csdnchart.entity.CsdnConfig;
import com.wretchant.csdnchart.service.ArchivalStorageService;
import com.wretchant.csdnchart.service.ConnService;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;@Slf4j
@Component
public class ArchivalStorageInitRunner implements ApplicationRunner {private final ArchivalStorageService archivalStorageService;private final CsdnConfig csdnConfig;private final ConnService connService;public ArchivalStorageInitRunner(CsdnConfig csdnConfig,ConnService connService,ArchivalStorageService archivalStorageService) {this.csdnConfig = csdnConfig;this.connService = connService;this.archivalStorageService = archivalStorageService;}@InfoLog("抓取文章在每月的归档存储数据")@Overridepublic void run(ApplicationArguments args) throws Exception {Document document = connService.conn(csdnConfig.getCsdnUrl());Elements elements = document.body().select(".archive-list");Elements select = elements.select("ul a");select.forEach(element -> {String href = element.attr("href");if (StringUtils.isNotBlank(href)) {String datetime = href.substring(href.length() - 7);String[] split = datetime.split("/");Integer year = Integer.valueOf(split[0]);Integer month = Integer.valueOf(split[1]);String span = element.select("span").get(0).html();if (StringUtils.isNotBlank(span)) {String articleNumber = span.substring(0, span.length() - 1);create(Integer.valueOf(articleNumber), year, month);}}});}private void create(Integer articleNumber, Integer year, Integer month) {ArchivalStorage archivalStorage = new ArchivalStorage();archivalStorage.setArticleNumber(articleNumber);archivalStorage.setYear(year);archivalStorage.setMonth(month);archivalStorageService.create(archivalStorage);}
}package com.wretchant.csdnchart.framework;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ArticleInfo;
import com.wretchant.csdnchart.entity.CsdnConfig;
import com.wretchant.csdnchart.service.ArticleInfoService;
import com.wretchant.csdnchart.service.ConnService;
import lombok.extern.slf4j.Slf4j;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;@Slf4j
@Component
public class ArticleInfoInitRunner implements ApplicationRunner {/** 在不知道作者有多少篇的文章的时候，设定一个最大值，即20000篇文章 */private static final int PAGE_MAX = 1000;private final ArticleInfoService articleInfoService;private final CsdnConfig csdnConfig;private final ConnService connService;public ArticleInfoInitRunner(CsdnConfig csdnConfig, ArticleInfoService articleInfoService, ConnService connService) {this.csdnConfig = csdnConfig;this.articleInfoService = articleInfoService;this.connService = connService;}@InfoLog("抓取全部的文章数据")@Overridepublic void run(ApplicationArguments args) throws Exception {// 页数从1开始for (int i = 1; i < PAGE_MAX; i++) {String s = csdnConfig.getCsdnUrl() + CsdnConfig.ARTICLE_LIST + i;Document document = connService.conn(s);Elements elements = document.body().select(".article-list");Elements select = elements.select(".article-item-box");if (select.isEmpty()) {// 当最后一页结束时，全循环结束break;}select.forEach(element -> {Elements h4A = element.select("h4 a");String url = h4A.attr("href");String articleName = h4A.text();Elements numbers = element.select(".num");String read = numbers.get(0).html();String comment = numbers.get(1).html();create(articleName, url, Integer.valueOf(read), Integer.valueOf(comment));});}}private void create(String articleName, String url, Integer read, Integer comment) {ArticleInfo articleInfo = new ArticleInfo();articleInfo.setArticleName(articleName);articleInfo.setUrl(url);articleInfo.setReadNumber(read);articleInfo.setCommentNumber(comment);articleInfoService.create(articleInfo);}
}package com.wretchant.csdnchart.framework;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.CsdnConfig;
import com.wretchant.csdnchart.entity.UserInfo;
import com.wretchant.csdnchart.service.ConnService;
import com.wretchant.csdnchart.service.UserInfoService;
import org.jsoup.nodes.Document;
import org.jsoup.select.Elements;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;@Component
public class UserInfoInitRunner implements ApplicationRunner {private final CsdnConfig csdnConfig;private final UserInfoService userInfoService;private final ConnService connService;public UserInfoInitRunner(CsdnConfig csdnConfig, UserInfoService userInfoService, ConnService connService) {this.csdnConfig = csdnConfig;this.userInfoService = userInfoService;this.connService = connService;}@InfoLog("抓取用户的个人信息")@Overridepublic void run(ApplicationArguments args) throws Exception {Document document = connService.conn(csdnConfig.getCsdnUrl());Elements elements = document.body().select(".title-box");String nickname = elements.select("a").get(0).html();String motto = elements.select("p").get(0).html();Elements select = document.body().select(".avatar_pic");String src = select.get(0).attr("src");// 因为这里抓的头像图片太小 - 只有三分之一大小，不好看，我们把它变成大的String headImgUrl = src.replace("3_", "1_");create(nickname, motto, headImgUrl);}private void create(String nickname, String motto, String headImgUrl) {UserInfo userInfo = new UserInfo();userInfo.setHeadImgUrl(headImgUrl);userInfo.setNickname(nickname);userInfo.setMotto(motto);userInfoService.create(userInfo);}
}package com.wretchant.csdnchart.repo;import com.wretchant.csdnchart.entity.ArchivalStorage;
import org.springframework.data.jpa.repository.JpaRepository;import java.util.Optional;public interface ArchivalStorageRepo extends JpaRepository<ArchivalStorage, Long> {Optional<ArchivalStorage> findByYearAndMonth(Integer year, Integer month);
}package com.wretchant.csdnchart.repo;import com.wretchant.csdnchart.entity.ArticleInfo;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;import java.util.List;
import java.util.Optional;public interface ArticleInfoRepo extends JpaRepository<ArticleInfo, Long> {Optional<ArticleInfo> findByUrl(String url);@Query(nativeQuery = true,value = "select * from article_info a order by a.read_number desc limit ?1")List<ArticleInfo> listOrderByReadNumberLimit(Integer limit);@Query(nativeQuery = true,value = "select * from article_info a order by a.comment_number desc limit ?1")List<ArticleInfo> listOrderByCommentNumberLimit(Integer limit);@Query(nativeQuery = true,value = "select * from article_info a order by a.gmt_create desc limit ?1")List<ArticleInfo> listOrderByGmtCreateLimit(Integer limit);
}package com.wretchant.csdnchart.repo;import com.wretchant.csdnchart.entity.DataTable;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;import java.util.List;public interface DataTableRepo extends JpaRepository<DataTable, Long> {@Query(nativeQuery = true, value = "select * from data d order by d.gmt_create desc limit ?1")List<DataTable> list(Integer limit);@Query(nativeQuery = true,value ="SELECT ANY_VALUE(DATE_FORMAT(d.gmt_create,'%Y-%m-%d')) AS `gmt_create`,\n"+ "ANY_VALUE(d.pk_id) AS `pk_id`,\n"+ "ANY_VALUE(d.deleted) AS `deleted`,\n"+ "ANY_VALUE(d.gmt_modified) AS `gmt_modified`,\n"+ "ANY_VALUE(d.remark) AS `remark`,\n"+ "ANY_VALUE(d.sort) AS `sort`,\n"+ "d.time_point,\n"+ "ANY_VALUE((MAX(d.visit_number) - MIN(d.visit_number))) AS `visit_number`,\n"+ "ANY_VALUE((MAX(d.level) - MIN(d.level))) AS `level`,\n"+ "ANY_VALUE((MAX(d.integral) - MIN(d.integral))) AS `integral`,\n"+ "ANY_VALUE((MAX(d.top) - MIN(d.top))) AS `top`,\n"+ "ANY_VALUE((MAX(d.article_number) - MIN(d.article_number))) AS `article_number`,\n"+ "ANY_VALUE((MAX(d.fans) - MIN(d.fans))) AS `fans`,\n"+ "ANY_VALUE((MAX(d.like_number) - MIN(d.like_number))) AS `like_number`,\n"+ "ANY_VALUE((MAX(d.comment_number) - MIN(d.comment_number))) AS `comment_number`\n"+ "FROM `data` d\n"+ "GROUP BY d.time_point\n"+ "ORDER BY gmt_create DESC, d.time_point DESC\n"+ "LIMIT ?1")List<DataTable> count(Integer limit);
}package com.wretchant.csdnchart.repo;import com.wretchant.csdnchart.entity.UserInfo;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Query;public interface UserInfoRepo extends JpaRepository<UserInfo, Long> {@Query(nativeQuery = true, value = "select * from user_info u order by u.version desc limit 1")UserInfo findTop();
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ArchivalStorage;import java.util.List;public interface ArchivalStorageService {@InfoLog("创建文章数据归档统计")ArchivalStorage create(ArchivalStorage archivalStorage);@InfoLog("查询全部的文章归档数据")List<ArchivalStorage> list();
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.entity.ArchivalStorage;
import com.wretchant.csdnchart.repo.ArchivalStorageRepo;
import org.springframework.stereotype.Service;import java.util.List;
import java.util.Optional;@Service
public class ArchivalStorageServiceImpl implements ArchivalStorageService {private final ArchivalStorageRepo archivalStorageRepo;public ArchivalStorageServiceImpl(ArchivalStorageRepo archivalStorageRepo) {this.archivalStorageRepo = archivalStorageRepo;}@Overridepublic ArchivalStorage create(ArchivalStorage archivalStorage) {Integer year = archivalStorage.getYear();Integer month = archivalStorage.getMonth();Optional<ArchivalStorage> dbOne = archivalStorageRepo.findByYearAndMonth(year, month);// 有则更新，无则创建if (dbOne.isPresent()) {ArchivalStorage storage = dbOne.get();if (archivalStorage.getArticleNumber().equals(storage.getArticleNumber())) {return storage;}storage.setArticleNumber(archivalStorage.getArticleNumber());return archivalStorageRepo.saveAndFlush(storage);}return archivalStorageRepo.saveAndFlush(archivalStorage);}@Overridepublic List<ArchivalStorage> list() {return archivalStorageRepo.findAll();}
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.ArticleInfo;
import com.wretchant.csdnchart.entity.ConduitEnum;import java.util.List;public interface ArticleInfoService {@InfoLog("创建文章信息")ArticleInfo create(ArticleInfo articleInfo);@InfoLog("查询全部的文章")List<ArticleInfo> list();@InfoLog("统计文章一共有多少篇")long count();@InfoLog("service：查看Conduit 文章列表")List<ArticleInfo> list(ConduitEnum conduit);
}package com.wretchant.csdnchart.service;import com.google.common.collect.Lists;
import com.wretchant.csdnchart.entity.ArticleInfo;
import com.wretchant.csdnchart.entity.ConduitEnum;
import com.wretchant.csdnchart.repo.ArticleInfoRepo;
import org.springframework.stereotype.Service;import java.util.List;
import java.util.Optional;@Service
public class ArticleInfoServiceImpl implements ArticleInfoService {private final ArticleInfoRepo articleInfoRepo;public ArticleInfoServiceImpl(ArticleInfoRepo articleInfoRepo) {this.articleInfoRepo = articleInfoRepo;}@Overridepublic ArticleInfo create(ArticleInfo articleInfo) {Optional<ArticleInfo> byUrl = articleInfoRepo.findByUrl(articleInfo.getUrl());if (byUrl.isPresent()) {ArticleInfo article = byUrl.get();boolean b =article.getCommentNumber().equals(articleInfo.getCommentNumber())&& article.getReadNumber().equals(articleInfo.getReadNumber());if (b) {return article;}article.setCommentNumber(articleInfo.getCommentNumber());article.setReadNumber(articleInfo.getReadNumber());return articleInfoRepo.saveAndFlush(article);}return articleInfoRepo.saveAndFlush(articleInfo);}@Overridepublic List<ArticleInfo> list() {return articleInfoRepo.findAll();}@Overridepublic long count() {return articleInfoRepo.count();}@Overridepublic List<ArticleInfo> list(ConduitEnum conduit) {int number = 10;switch (conduit) {case NEW_MOST:return articleInfoRepo.listOrderByGmtCreateLimit(number);case VISIT_MOST:return articleInfoRepo.listOrderByReadNumberLimit(number);case COMMENT_MOST:return articleInfoRepo.listOrderByCommentNumberLimit(number);default:}return Lists.newArrayList();}
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.annotation.InfoLog;
import org.jsoup.nodes.Document;public interface ConnService {@InfoLog("开始抓取数据")Document conn(String url) throws Exception;
}package com.wretchant.csdnchart.service;import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.springframework.stereotype.Service;@Service
public class ConnServiceImpl implements ConnService {@Overridepublic Document conn(String url) throws Exception {Connection conn = Jsoup.connect(url);conn.header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");conn.header("Accept-Encoding", "gzip, deflate, sdch");conn.header("Accept-Language", "zh-CN,zh;q=0.8");conn.header("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36");return conn.get();}
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.DataTable;
import com.wretchant.csdnchart.entity.DataTableDto;import java.util.List;public interface DataTableService {@InfoLog("爬取数据入库")DataTable create(DataTableDto dataTableDto);@InfoLog("查询所有的统计数据")List<DataTable> list();@InfoLog("查询指定数量的统计数据")List<DataTable> list(Integer limit);@InfoLog("查询指标数据增量")List<DataTable> count(Integer limit);@InfoLog("查询最近的一条数据")DataTable last();
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.core.Converter;
import com.wretchant.csdnchart.entity.DataTable;
import com.wretchant.csdnchart.entity.DataTableDto;
import com.wretchant.csdnchart.entity.DataTableDto2DataTableConverter;
import com.wretchant.csdnchart.repo.DataTableRepo;
import org.springframework.stereotype.Service;import java.util.List;@Service
public class DataTableServiceImpl implements DataTableService {private final DataTableRepo dataTableRepo;public DataTableServiceImpl(DataTableRepo dataTableRepo) {this.dataTableRepo = dataTableRepo;}@Overridepublic DataTable create(DataTableDto dataTableDto) {Converter<DataTableDto, DataTable> converter = new DataTableDto2DataTableConverter();return dataTableRepo.saveAndFlush(converter.converter(dataTableDto));}@Overridepublic List<DataTable> list() {return dataTableRepo.findAll();}@Overridepublic List<DataTable> list(Integer limit) {return dataTableRepo.list(limit);}@Overridepublic List<DataTable> count(Integer limit) {return dataTableRepo.count(limit);}@Overridepublic DataTable last() {return dataTableRepo.list(1).get(0);}
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.UserInfo;public interface UserInfoService {@InfoLog("创建用户信息")UserInfo create(UserInfo userInfo);@InfoLog("查询最新的用户信息")UserInfo findTop();
}package com.wretchant.csdnchart.service;import com.wretchant.csdnchart.entity.UserInfo;
import com.wretchant.csdnchart.repo.UserInfoRepo;
import org.springframework.stereotype.Service;@Service
public class UserInfoServiceImpl implements UserInfoService {private final UserInfoRepo userInfoRepo;public UserInfoServiceImpl(UserInfoRepo userInfoRepo) {this.userInfoRepo = userInfoRepo;}@Overridepublic UserInfo create(UserInfo userInfo) {userInfo.setVersion(userInfoRepo.count());return userInfoRepo.saveAndFlush(userInfo);}@Overridepublic UserInfo findTop() {return userInfoRepo.findTop();}
}package com.wretchant.csdnchart.timer;import com.wretchant.csdnchart.annotation.InfoLog;
import com.wretchant.csdnchart.entity.CsdnConfig;
import com.wretchant.csdnchart.entity.DataTableDto;
import com.wretchant.csdnchart.service.ConnService;
import com.wretchant.csdnchart.service.DataTableService;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;@Service
public class DataTimer {private final CsdnConfig csdnConfig;private final ConnService connService;private final DataTableService dataTableService;public DataTimer(ConnService connService, CsdnConfig csdnConfig, DataTableService dataTableService) {this.connService = connService;this.csdnConfig = csdnConfig;this.dataTableService = dataTableService;}@InfoLog("开始爬取数据")@Async@Scheduled(fixedRate = 1000 * 60 * 5)public void execute() throws Exception {Document document = connService.conn(csdnConfig.getCsdnUrl());Element asideProfile = document.body().getElementById("asideProfile");Elements gradeBox = asideProfile.select(".grade-box");Elements info = asideProfile.select(".item-tiling");Elements gradeColumn = gradeBox.select("dl");String level = gradeColumn.get(0).select("dd a").attr("title").substring(0, 1);String visitNumber = gradeColumn.get(1).select("dd").attr("title");String integral = gradeColumn.get(2).select("dd").attr("title");String top = gradeColumn.get(3).attr("title");Elements infoColumn = info.select("dl");String htmlValue = infoColumn.select("dd span").html();String[] values = htmlValue.split("\n");String articleNumber = values[0];String fans = values[1];String likeNumber = values[2];String commentNumber = values[3];DataTableDto dto = new DataTableDto();dto.setLevel(Integer.valueOf(level));dto.setVisitNumber(Integer.valueOf(visitNumber));dto.setIntegral(Integer.valueOf(integral));dto.setTop(Integer.valueOf(top));dto.setArticleNumber(Integer.valueOf(articleNumber));dto.setFans(Integer.valueOf(fans));dto.setLikeNumber(Integer.valueOf(likeNumber));dto.setCommentNumber(Integer.valueOf(commentNumber));dataTableService.create(dto);}
}package com.wretchant.csdnchart;import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;@SpringBootApplication
public class CsdnChartApplication {public static void main(String[] args) {SpringApplication.run(CsdnChartApplication.class, args);}
}<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"xmlns:th="http://www.thymeleaf.org">
<head><meta charset="UTF-8"><title>数据</title><style>body {text-align: center;width: 100%;}table {padding-right: 20px;width: 100%;min-width: 700px;text-align: center;border-spacing: unset;}td {border: none;}thead {background: rgba(0, 57, 117, 0.9);color: aliceblue;font-size: 14px;font-weight: 700;}thead td {padding-left: 10px;padding-right: 10px;padding-top: 5px;padding-bottom: 5px;}tbody {background: rgba(3, 21, 46, 0.9);font-size: 12px;font-weight: 400;}tbody td {color: azure;padding: 3px;border-bottom: blueviolet 1px solid;border-right: blueviolet 1px solid;}</style>
</head>
<body><table><thead><!--0,57,117--><!--3,21,46--><tr><td>时间</td><td>原创</td><td>粉丝</td><td>喜欢</td><td>评论</td><td>等级</td><td>访问</td><td>积分</td><td>排名</td></tr></thead><tbody><tr th:each="data:${list}"><td th:text="${#temporals.format(data.gmtCreate,'yyyy-MM-dd') +' - '+ data.timePoint+'点'}">0</td><td><span th:text="${'+'+data.articleNumber}"></span></td><td><span th:text="${'+'+data.fans}"></span></td><td><span th:text="${'+'+data.likeNumber}"></span></td><td><span th:text="${'+'+data.commentNumber}"></span></td><td><span th:text="${'+'+data.level}"></span></td><td><span th:text="${'+'+data.visitNumber}"></span></td><td><span th:text="${'+'+data.integral}"></span></td><td><span th:text="${'+'+data.top}"></span></td></tr></tbody></table></body>
</html>
<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"xmlns:th="http://www.thymeleaf.org">
<head><meta charset="UTF-8"><title>数据</title><link href="https://cdn.bootcss.com/twitter-bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet"><style>body {text-align: center;width: 100%;}table {padding-right: 20px;width: 60%;min-width: 700px;text-align: center;border-spacing: unset;}td {border: none;}thead {background: rgba(0, 57, 117, 0.9);color: aliceblue;font-size: 14px;font-weight: 700;}thead td {padding-left: 10px;padding-right: 10px;padding-top: 5px;padding-bottom: 5px;}tbody {background: rgba(3, 21, 46, 0.9);font-size: 12px;font-weight: 400;}tbody td {color: azure;padding: 3px;border-bottom: blueviolet 1px solid;border-right: blueviolet 1px solid;}a {color: chartreuse;cursor: pointer;}td {max-width: 650px;}</style>
</head>
<body><table class="m-auto"><thead><!--0,57,117--><!--3,21,46--><tr><td>名称</td><td>评论</td><td>访问</td><td>文章地址</td></tr></thead><tbody><tr th:each="data:${list}"><td th:text="${data.articleName}"></td><td th:text="${data.commentNumber}"></td><td th:text="${data.readNumber }"></td><td><a th:href="${data.url}" target="_blank" th:text="${data.url}"></a></td></tr></tbody></table></body>
</html>
<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"
>
<head><meta charset="UTF-8"><title>报表</title><script src="https://cdn.bootcss.com/echarts/4.3.0-rc.2/echarts-en.common.js"></script><script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script><script>function setChart(id, option) {var myChart = echarts.init(document.getElementById(id));option.backgroundColor = "#2c343c";option.textStyle = {color: 'rgba(255, 255, 255, 0.3)'};option.toolbox = {show: true,feature: {dataZoom: {yAxisIndex: 'none'},dataView: {readOnly: false},magicType: {type: ['line', 'bar']},restore: {},saveAsImage: {}}};option.tooltip = {trigger: 'axis'};myChart.setOption(option);}function url(position) {return '/api/data/t/' + [[${size}]] + '?' + 'dataTableEnum=' + position;}</script><style>.box {width: 49%;min-width: 400px;height: 350px;padding-bottom: 100px;}.left {float: left;}.right {float: right;}.title {font-size: 14px;font-weight: 300;color: blueviolet;padding-bottom: 10px;}</style></head>
<body><div class="box left"><div class="title">访问数据增长图</div><div id="visitNumber" style="height:400px;"></div><script>$.get(url('VISIT_NUMBER'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('visitNumber', option);});</script>
</div><div class="box right"><div class="title">积分增长曲线图</div><div id="integral" style="height:400px;"></div><script>$.get(url('INTEGRAL'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('integral', option);});</script>
</div><div class="box left"><div class="title">排名增长图</div><div id="top" style="height:400px;"></div><script>$.get(url('TOP'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('top', option);});</script>
</div><div class="box right"><div class="title">等级增长曲线图</div><div id="level" style="height:400px;"></div><script>$.get(url('LEVEL'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('level', option);});</script>
</div><div class="box left"><div class="title">原创文章增长图</div><div id="articleNumber" style="height:400px;"></div><script>$.get(url('ARTICLE_NUMBER'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('articleNumber', option);});</script>
</div><div class="box right"><div class="title">粉丝增长曲线图</div><div id="fans" style="height:400px;"></div><script>$.get(url('FANS'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('fans', option);});</script>
</div><div class="box left"><div class="title">喜欢增长图</div><div id="likeNumber" style="height:400px;"></div><script>$.get(url('LIKE_NUMBER'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('likeNumber', option);});</script>
</div><div class="box right"><div class="title">评论增长曲线图</div><div id="commentNumber" style="height:400px;"></div><script>$.get(url('COMMENT_NUMBER'), function (res) {option = {xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('commentNumber', option);});</script>
</div></body>
</html>
<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"xmlns:th="http://www.thymeleaf.org">
<head><meta charset="UTF-8"><title>数据</title><style>body {text-align: center;width: 100%;}table {padding-right: 20px;width: 100%;min-width: 700px;text-align: center;border-spacing: unset;}td {border: none;}thead {background: rgba(0, 57, 117, 0.9);color: aliceblue;font-size: 14px;font-weight: 700;}thead td {padding-left: 10px;padding-right: 10px;padding-top: 5px;padding-bottom: 5px;}tbody {background: rgba(3, 21, 46, 0.9);font-size: 12px;font-weight: 400;}tbody td {color: azure;padding: 3px;border-bottom: blueviolet 1px solid;border-right: blueviolet 1px solid;}</style>
</head>
<body><table><thead><!--0,57,117--><!--3,21,46--><tr><td>时间</td><td>原创</td><td>粉丝</td><td>喜欢</td><td>评论</td><td>等级</td><td>访问</td><td>积分</td><td>排名</td></tr></thead><tbody><tr th:each="data:${list}"><td th:text="${#temporals.format(data.gmtCreate,'yyyy-MM-dd HH:mm:ss')}">0</td><td><span th:text="${data.articleNumber}"></span><spanth:text="${data.articleNumberAdd eq 0 ? '':'(+'+data.articleNumberAdd+')'}"></span></td><td><span th:text="${data.fans}"></span><span th:text="${data.fansAdd eq 0 ? '':'(+'+data.fansAdd+')'}"></span></td><td><span th:text="${data.likeNumber}"></span><spanth:text="${data.likeNumberAdd eq 0 ? '':'(+'+data.likeNumberAdd+')'}"></span></td><td><span th:text="${data.commentNumber}"></span><spanth:text="${data.commentNumberAdd eq 0 ? '':'(+'+data.commentNumberAdd+')'}"></span></td><td><span th:text="${data.level}"></span><spanth:text="${data.levelAdd eq 0 ? '':'(+'+data.levelAdd+')'}"></span></td><td><span th:text="${data.visitNumber}"></span><spanth:text="${data.visitNumberAdd eq 0 ? '':'(+'+data.visitNumberAdd+')'}"></span></td><td><span th:text="${data.integral}"></span><spanth:text="${data.integralAdd eq 0 ? '':'(+'+data.integralAdd+')'}"></span></td><td><span th:text="${data.top}"></span><span th:text="${data.topAdd eq 0 ? '':'(+'+data.topAdd+')'}"></span></td></tr></tbody></table></body>
</html>
<!DOCTYPE html>
<html lang="en" xmlns="http://www.w3.org/1999/xhtml"xmlns:th="http://www.thymeleaf.org">
<head><meta charset="UTF-8"><link href="https://cdn.bootcss.com/twitter-bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet"><script src="https://cdn.bootcss.com/echarts/4.3.0-rc.2/echarts-en.common.js"></script><script src="https://cdn.bootcss.com/jquery/3.4.1/jquery.min.js"></script><title>首页</title><script>function setChart(id, option) {var myChart = echarts.init(document.getElementById(id));option.backgroundColor = "#2c343c";option.textStyle = {color: 'rgba(255, 255, 255, 0.3)'};option.toolbox = {show: true,feature: {dataZoom: {yAxisIndex: 'none'},dataView: {readOnly: false},magicType: {type: ['line', 'bar']},restore: {},saveAsImage: {}}};option.tooltip = {trigger: 'axis'};myChart.setOption(option);}function url(position) {return '/api/data/t/' + 30 + '?' + 'dataTableEnum=' + position;}</script><style>.main {width: 60%;min-width: 900px;min-height: 1080px;background: beige;}.info-box {height: 170px;width: 100%;background: aliceblue;}.grand-box {padding: 30px;width: 100%;background: aliceblue;}.info-box img {width: 160px;height: 160px;}.head-img {padding-left: 50px;padding-top: 10px;}h1 {margin-top: 10px;}h6 {margin-top: 10px;}.table {background: darkseagreen;height: 380px;}.title {padding-top: 15px;font-size: 14px;font-weight: 300;color: blueviolet;padding-bottom: 10px;}.all-table {float: right;padding-right: 20px;}.all-table input {width: 50px;border: none;border-bottom: blueviolet 1px solid;text-align: center;background: beige;}a {font-size: 14px;font-weight: 300;cursor: pointer;color: red;}</style>
</head>
<body><div class="main m-auto"><!-- 个人信息 --><div class="info-box"><div class="row"><div class="col-md-3"><img src="http://b-ssl.duitang.com/uploads/item/201511/07/20151107201311_TYGkm.jpeg" alt="头像"class="head-img"></div><div class="col-md-9"><h1 th:text="${userInfo.nickname}">简简单单OnlineZuozuo</h1><h6 th:text="${userInfo.motto}">未闻万里蓬莱，而窥先圣遗智。故，以此生筑梦，奔而逐之；以泰山之伟，攀而登之；以静雅素心，处世为人。------zuozuo 著...</h6><h6 class="title"><a href="/web/article">查看全部的文章 - - ></a></h6></div></div></div><div class="title">具体的数据</div><div class="grand-box"><div class="row"><div class="col-md-3" th:text="${'原创：'+last.articleNumber}">0</div><div class="col-md-3" th:text="${'粉丝：'+last.fans}">0</div><div class="col-md-3" th:text="${'喜欢：'+last.likeNumber}">0</div><div class="col-md-3" th:text="${'评论：'+last.commentNumber}">0</div></div><div class="row"><div class="col-md-3" th:text="${'等级：'+last.level}">0</div><div class="col-md-3" th:text="${'访问：'+last.visitNumber}">0</div><div class="col-md-3" th:text="${'积分：'+last.integral}">0</div><div class="col-md-3" th:text="${'排名：'+last.top}">0</div></div><br><div class="row"><div class="col-md-12"th:text="${'原转比：'+last.articleNumber+'(原创) / ' + count+'(总文章数) = ' + #numbers.formatPercent(last.articleNumber / (count * 1.00) ,3,2) }">0</div></div></div><div class="title">访问通道</div><div class="grand-box"><div class="row"><div class="col-md-3"><a href="/web/conduit/VISIT_MOST">最高访问10篇 - - ></a></div><div class="col-md-3"><a href="/web/conduit/NEW_MOST">最新文章10篇 - - ></a></div><div class="col-md-3"><a href="/web/conduit/COMMENT_MOST">评论最多10篇 - - ></a></div></div></div><!-- 访问简报 --><div class="title">访问增量简报<div class="all-table">以<input id="chart-data-length" type="number" value="30"><a onclick="chart()">条数据长度查看全部报表- - ></a></div></div><div class="table" id="visitNumber"></div><script>$.get(url('VISIT_NUMBER'), function (res) {option = {toolbox: {feature: {saveAsImage: {}}},tooltip: {trigger: 'axis'},xAxis: {type: 'category',data: res.t.datetime},yAxis: {min: res.t.min - 2,max: res.t.max + 2},series: [{data: res.t.data,type: 'line'}]};setChart('visitNumber', option);});</script><!-- 访问简报 --><div class="title">写作趋势</div><div class="table" id="archival_storage"></div><script>$.get(url('VISIT_NUMBER'), function (res) {option = {xAxis: {type: 'category',data: [[${dates}]]},yAxis: {min: [[${min}]] - 2,max: [[${max}]] + 2},series: [{data: [[${datas}]],type: 'line',markPoint: {data: [{type: 'max', name: '最大值'},{type: 'min', name: '最小值'}]},markLine: {data: [{type: 'average', name: '平均值'}]}}]};setChart('archival_storage', option);});</script></div><script>function chart() {var choose = $("#chart-data-length");window.location.href = "/web/chart/" + choose.val();}
</script></body>
</html>
spring:application:name: csdn-chartdatasource:driver-class-name: com.mysql.cj.jdbc.Driverurl: jdbc:mysql://localhost:3306/csdn?useUnicode=true&characterEncoding=utf-8&useSSL=false&serverTimezone=GMT%2B8username: rootpassword: rootthymeleaf:cache: falseexecute:core-pool-size: 10max-pool-size: 200queue-capacity: 10server:port: 9253csdn:csdn-url: https://wretchant.blog.csdn.net
CSDN 数据访问可视化，写给CSDN 群友们用用相关推荐
- 测试 ClownFish、CYQ、Entity Framework、Moon、MySoft、NHibernate、PDF、XCode数据访问组件性能...
下期预告: 由于很多园友反馈,有的组件不应该缺席.测试复杂度不够.测试还缺乏一定的公平. 因此考虑在下一个版本中,确保在更加公平的前提下进行更高复杂度的测试 . 同时将分为2组测试,纯SQL组件及纯O ...
- 测试 ClownFish、CYQ、Entity Framework、Moon、MySoft、NHibernate、PDF、XCode数据访问组件性能
"啊!你在用ORM?会不会性能很差啊?" 用数字来说话,打破模糊的.传言的印象. 标题提到的组件"增删改查"都实现了测试代码,所以除了测试外,也可以把此项目作为 ...
- CSDN：借助工具对【本博客访问来源】进行数据图表可视化(网友主要来自美国、新加坡、日本、英德加澳等)——记录数据来源截止日期20200718晚上22点
CSDN:借助工具对[本博客访问来源]进行数据图表可视化(网友主要来自美国.新加坡.日本.英德加澳等) 目录 粉丝增量 记录数据来源截止日期20201210晚上24点 记录数据来源截止日期202007 ...
- CSDN：借助工具对【本博客访问来源】进行数据图表可视化(网友主要来自欧美和印度等)——记录数据来源截止日期20190811
CSDN:借助工具对[本博客访问来源]进行数据图表可视化(网友主要来自欧美和印度等)--记录数据来源截止日期20190811 目录 地区统计 按照国家进行数据统计 按照省市地区进行数据统计 年龄统计 ...
- switchyomega规则列表备份_求人不如求己，自己动手写一个CSDN博客备份小工具？...
前提概要 背景 因为笔者在上个月的时候,突然想扩展一下技术栈,不能仅仅局限于Java,还是得掌握一门工具语言,不然显得太low.所以也就对Python和Golang类的语言有了一些兴趣.也就在上个月简 ...
- 如何写出高性能代码(四)优化数据访问
同一份逻辑,不同人的实现的代码性能会出现数量级的差异: 同一份代码,你可能微调几个字符或者某行代码的顺序,就会有数倍的性能提升:同一份代码,也可能在不同处理器上运行也会有几倍的性能差异:十倍程序员 ...
- 11 年了，我在 CSDN 被访问 800 万+次！ 被点赞 3.5 万+次，被收藏 9.6 万+次！
CSDN 的小伙伴们,大家好呀,我是沉默王二,一枚沉默但有趣的程序员.今天我们来掏心掏肺地聊一聊有效学习这件事. 刚看了一眼 CSDN 的主页,发现我已经写了 893 篇原创文章了,加上这一篇就是 8 ...
- 写一个通用数据访问组件
出处:http://www.csharp-corner.com willsound(翻译) 我收到过好多Email来问我如何用一个通用的数据提供者(data provider)在不失自然数据提供者(n ...
- R语言基于可视化进行多变量离群（Mulltivariate outliers）点检测识别：散点图可视化多变量离群点、模型平滑多变量异常检测、使用平行坐标图查看钻石数据集中的异常值
R语言基于可视化进行多变量离群(Mulltivariate outliers)点检测识别:散点图可视化多变量离群点.模型平滑多变量异常检测.使用平行坐标图查看钻石数据集中的异常值 目录
最新文章
- 2013年10月1日C#随机数
- oracle安装清单过不去,oracle 11g（二）安装过程
- WEB文档在线预览解决方案
- 重温名篇《康托尔、哥德尔、图灵——永恒的金色对角线》
- 【IT资讯】年薪170万码农征友，条件让网友炸锅……
- python保存模型与参数_Pytorch - 模型和参数的保存与恢复
- CLR via C# ver4.0 读书笔记
- 常用智能小车电机驱动模块选型
- 怎么录制电脑桌面视频 具体的操作方法
- 计算机最最最底层的工作原理是怎么运行的
- 支付宝云支付如何开通？
- Latex——连乘符号
- Leetcode刷题——每日一题题目汇总
- JS实现文件下载的几种方法
- 我的.Net Core 3.0 windows 桌面程序界面绘制 -- 从.net framework复制代码法
- termius 连接内网服务器
- 博睿数据作为AIOps代表厂商入选《2021年中国ICT技术成熟度曲线报告》
- 解决[服务器证书无效, 连接伪装服务器]问题
- p5.js的“运动”主题创作
- 2TB U盘格式化后，显示31GB，恢复如下：
热门文章
- 35条心理暗示让你摆脱坏心情，做快乐的自己
- 共享单车登录显示服务器未响应,ofo共享单车服务为什么出现故障
- STM32学习笔记（六 定时器及应用 4 光敏传感器实验 ）
- 读 Herbert George Wells 之 A Short History of The World
- VRTK 初识vrtk，如何在自己的项目中使用vrtk
- windows7 安装.Net Framework 4.6.2微软官方版(离线安装包)
- 计算机硬件故障可分为哪几类,计算机故障可分为硬件和非硬件故障
- 申请DUNS编码最新规则
- http状态码大全100-599状态详解
- java base64 转图片不现实_BASE64编码的图片在网页中的显示问题的解决