FloodlightProvider

  1. 处理交换机之间的连接并将 OpenFlow 的消息转化成其他模块可以监听的时间
  2. 决定某些特定的 OpenFLow 消息(即 PacketIn,FlowRemove,PortStatus 等)被分派到该侦听消息的模块的顺序,模块可以决定允许该消息进入下一个监听对象或者停止处理消息

FloodlightProvider 如何工作?

FloodlightProvider 使用 Netty 库来处理到交换机的线程和连接。
每个 OpenFlow 消息将通过一个 Netty 的线程进行处理,并执行与所有模块的消息相关联的所有逻辑
其他模块也可以注册类似交换机连接或断开和端口状态通知特定时间。
为了使模块注册为基于 OpenFlow 消息的,必须实现 IOFMessageListener 接口

要监听 OpenFlow 消息,要先向 FloodlightProvider 注册
调用 IFloodlightProviderService(具体由 Controller 类实现)的 addOFMessageListener 方法进行注册订阅
核心工作是在 ListenerDispatcher 类来完成。
每次增加观察者都会判断是否是终结点(也就是不被其他的 Listener 所依赖),因为最终确定这些观察者顺序的时候就是由这些终结点开始往前进行 DFS 遍历得到

Controller中实现 IFloodlightProviderService 的方法

    @Overridepublic synchronized void addOFMessageListener(OFType type, IOFMessageListener listener) {//先判断与type对应的 ListenerDispatcher对象是否存在ListenerDispatcher<OFType, IOFMessageListener> ldd =messageListeners.get(type);if (ldd == null) {ldd = new ListenerDispatcher<OFType, IOFMessageListener>();messageListeners.put(type, ldd);}//注册监听type这个消息;ldd.addListener(type, listener);}

ListenerDispatcher 维护这些观察者,有依赖关系

    volatile List<T> listeners = new ArrayList<T>();

//每个OF msg都有唯一的ListenerDispatcher对象,观察者存在listeners链表中

    private boolean ispre(U type, T l1, T l2) {return (l2.isCallbackOrderingPrereq(type, l1.getName()) ||l1.isCallbackOrderingPostreq(type, l2.getName()));}

返回两个传入的监听器的顺序

     public void addListener(U type, T listener) {List<T> newlisteners = new ArrayList<T>();if (listeners != null)newlisteners.addAll(listeners);newlisteners.add(listener);// Find nodes without outgoing edges// 查找没有出边的节点List<T> terminals = new ArrayList<T>();for (T i : newlisteners) {boolean isterm = true;for (T j : newlisteners) {if (ispre(type, i, j)) {//两个都不关心前后顺序的时候isterm = false;break;}}if (isterm) {//关乎有前后顺序的监听模块存入terminals.add(i);}}if (terminals.size() == 0) {logger.error("No listener dependency solution: " +"No listeners without incoming dependencies");listeners = newlisteners;return;}// visit depth-first traversing in the opposite order from// the dependencies.  Note we will not generally detect cycles/*** 以相反顺序访问深度优先遍历依赖。 注意我们通常不会检测周期*/HashSet<T> visited = new HashSet<T>();List<T> ordering = new ArrayList<T>();for (T term : terminals) {//进行排序visit(newlisteners, type, visited, ordering, term);}listeners = ordering;}

    private void visit(List<T> newlisteners, U type, HashSet<T> visited,List<T> ordering, T listener) {if (!visited.contains(listener)) {visited.add(listener);for (T i : newlisteners) {if (ispre(type, i, listener)) {visit(newlisteners, type, visited, ordering, i);}}ordering.add(listener);//}}

监听器具有的方法

public interface IListener<T>public enum Command {CONTINUE, STOP}
状态值,用来判断是否继续执行public String getName(); //用来判断 name 的这个模块是否要在当前对象之前执行public boolean isCallbackOrderingPrereq(T type, String name);//用来判断 name 的这个模块是否要在当前对象之后执行public boolean isCallbackOrderingPostreq(T type, String name);IOFMessageListener接口继承了 IListener 接口,同时定义了 receive 方法public Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx);
返回 CONTINUE 或者 STOP,继续看每个继承这个接口的模块的重写

查看继承了 IOFMessageListener 的Type Hierarchy

TopologyManager 模块的IOFMessageListener 重写的方法:

    @Overridepublic String getName() {return MODULE_NAME; //此处为 topology,每个模块都有自己的 MODULE_NAME}@Overridepublic boolean isCallbackOrderingPrereq(OFType type, String name) {//从此处可以看出,在执行这个模块之前,需要先执行 MODULE_NAME 为 linkiscovery 的模块return "linkdiscovery".equals(name);}@Overridepublic boolean isCallbackOrderingPostreq(OFType type, String name) {return false;}@Overridepublic Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:ctrIncoming.increment();//计数器,加一//调用这里的执行方法return this.processPacketInMessage(sw, (OFPacketIn) msg, cntx);default:break;}return Command.CONTINUE;}

通过 Type Hierarchy 可以找到Packet-In消息处理顺序的几个模块

FloodlightContextStore 数据结构

  • FloodlightContextStore 代表的是一种缓存模型(利用的是ConcurrentHashMap,线程安全的 HashMap)
  • 里面存储的是上下文相关的对象,能够根据相应的key得到具体的 Object
  • 存在的意义是Floodlight中注册监听某个事件的listener可以在被调用的时候直接从中取出上下文信息(context information)

基本数据结构,这是一个上下文对象,Floodlight代码监听器可以注册它,稍后可以检索与事件相关联的上下文信息

    public class FloodlightContext {protected ConcurrentHashMap<String, Object> storage =new ConcurrentHashMap<String, Object>();public ConcurrentHashMap<String, Object> getStorage() {return storage;}}

创建了一个 HashMap storage,

    public class FloodlightContextStore<V> {@SuppressWarnings("unchecked")public V get(FloodlightContext bc, String key) {return (V)bc.storage.get(key);}public void put(FloodlightContext bc, String key, V value) {bc.storage.put(key, value);}public void remove(FloodlightContext bc, String key) {bc.storage.remove(key);}}

一个FloodlightContextStore对象,可用于PACKET-IN有效内容,消息对象是Ethernet类型

    public static final FloodlightContextStore<Ethernet> bcStore =new FloodlightContextStore<Ethernet>();

LinkDiscoveryManager 模块

  • 链接发现服务负责发现和维护 OpenFlow 网络中的网络连接的状态

IOFMessageListener 的 receive 方法

    @Overridepublic Command receive(IOFSwitch sw, OFMessage msg,FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:ctrIncoming.increment();return this.handlePacketIn(sw.getId(), (OFPacketIn) msg,cntx);default:break;}return Command.CONTINUE;}

主要使用了 handlePacketIn()方法

    protected Command handlePacketIn(DatapathId sw, OFPacketIn pi,FloodlightContext cntx) {//提取 Packet-In 的有效分组内容Ethernet eth = IFloodlightProviderService.bcStore.get(cntx,IFloodlightProviderService.CONTEXT_PI_PAYLOAD);OFPort inPort = (pi.getVersion().compareTo(OFVersion.OF_12) < 0 ? pi.getInPort() : pi.getMatch().get(MatchField.IN_PORT));if (eth.getPayload() instanceof BSN) {BSN bsn = (BSN) eth.getPayload();if (bsn == null) return Command.STOP;if (bsn.getPayload() == null) return Command.STOP;// It could be a packet other than BSN LLDP, therefore// continue with the regular processing.// 它可以是除BSN LLDP之外的分组,因此继续进行常规处理。if (bsn.getPayload() instanceof LLDP == false)return Command.CONTINUE;return handleLldp((LLDP) bsn.getPayload(), sw, inPort, false, cntx);} else if (eth.getPayload() instanceof LLDP) {return handleLldp((LLDP) eth.getPayload(), sw, inPort, true, cntx);} else if (eth.getEtherType().getValue() < 1536 && eth.getEtherType().getValue() >= 17) {long destMac = eth.getDestinationMACAddress().getLong();if ((destMac & LINK_LOCAL_MASK) == LINK_LOCAL_VALUE) {ctrLinkLocalDrops.increment();if (log.isTraceEnabled()) {log.trace("Ignoring packet addressed to 802.1D/Q "+ "reserved address.");}return Command.STOP;}} else if (eth.getEtherType().getValue() < 17) {log.error("Received invalid ethertype of {}.", eth.getEtherType());return Command.STOP;}if (ignorePacketInFromSource(eth.getSourceMACAddress())) {ctrIgnoreSrcMacDrops.increment();return Command.STOP;}// If packet-in is from a quarantine port, stop processing.NodePortTuple npt = new NodePortTuple(sw, inPort);if (quarantineQueue.contains(npt)) {ctrQuarantineDrops.increment();return Command.STOP;}return Command.CONTINUE;}

TopolopyManager

  • 为控制器维护拓扑信息,以及在网络中寻找路由
IOFMessageListener 的 receive 方法@Overridepublic Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:ctrIncoming.increment();return this.processPacketInMessage(sw, (OFPacketIn) msg, cntx);default:break;}return Command.CONTINUE;}
主要使用了processPacketInMessage()方法protected Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, FloodlightContext cntx) {// get the packet-in switch.Ethernet eth =IFloodlightProviderService.bcStore.get(cntx,IFloodlightProviderService.CONTEXT_PI_PAYLOAD);if (eth.getPayload() instanceof BSN) {BSN bsn = (BSN) eth.getPayload();if (bsn == null) return Command.STOP;if (bsn.getPayload() == null) return Command.STOP;// 可能不是 BSN LLDP,继续常规处理if (bsn.getPayload() instanceof LLDP == false)return Command.CONTINUE;doFloodBDDP(sw.getId(), pi, cntx);return Command.STOP;} else {return dropFilter(sw.getId(), pi, cntx);}}

DeviceManagerImpl

  • DeviceManager基于在网络中看到的MAC地址创建设备
  • 它跟踪映射到设备的任何网络地址及其在网络中的位置

设备管理器通过 PACKET-IN 消息请求了解设备,通过 PACKET-IN 消息获取信息,根据实体如何建立进行分类。默认情况下,entity classifies 使用 MAC 地址和 VLAN 来识别设备。这两个属性定义一个独一无二的设备。设备管理器将了解其他属性,如 IP 地址。
信息中的一个重要的部分是设备的连接点,如果一个交换机接受到一个 PACKET-IN 消息,则交换机将会创建一个连接点,设备也会根据时间清空连接点,IP 地址,以及设备本身,最近看到的时间戳是用来保持清空过程的控制

IOFMessageListener 的 receive 方法

    @Overridepublic Command receive(IOFSwitch sw, OFMessage msg,FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:cntIncoming.increment();return this.processPacketInMessage(sw, (OFPacketIn) msg, cntx);default:break;}return Command.CONTINUE;}

主要使用了processPacketInMessage()方法

    protected Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, FloodlightContext cntx) {Ethernet eth = IFloodlightProviderService.bcStore.get(cntx,IFloodlightProviderService.CONTEXT_PI_PAYLOAD);OFPort inPort = (pi.getVersion().compareTo(OFVersion.OF_12) < 0 ? pi.getInPort() : pi.getMatch().get(MatchField.IN_PORT));// Extract source entity informationEntity srcEntity = getSourceEntityFromPacket(eth, sw.getId(), inPort);if (srcEntity == null) {cntInvalidSource.increment();return Command.STOP;}// Learn from ARP packet for special VRRP settings.// In VRRP settings, the source MAC address and sender MAC// addresses can be different.  In such cases, we need to learn// the IP to MAC mapping of the VRRP IP address.  The source// entity will not have that information.  Hence, a separate call// to learn devices in such cases.learnDeviceFromArpResponseData(eth, sw.getId(), inPort);// Learn/lookup device informationDevice srcDevice = learnDeviceByEntity(srcEntity);if (srcDevice == null) {cntNoSource.increment();return Command.STOP;}// Store the source device in the contextfcStore.put(cntx, CONTEXT_SRC_DEVICE, srcDevice);// Find the device matching the destination from the entity// classes of the source.if (eth.getDestinationMACAddress().getLong() == 0) {cntInvalidDest.increment();return Command.STOP;}Entity dstEntity = getDestEntityFromPacket(eth);Device dstDevice = null;if (dstEntity != null) {dstDevice = findDestByEntity(srcDevice.getEntityClass(), dstEntity);if (dstDevice != null)fcStore.put(cntx, CONTEXT_DST_DEVICE, dstDevice);elsecntNoDest.increment();} else {cntNoDest.increment();}if (logger.isTraceEnabled()) {logger.trace("Received PI: {} on switch {}, port {} *** eth={}" +" *** srcDev={} *** dstDev={} *** ",new Object[] { pi, sw.getId().toString(), inPort, eth,srcDevice, dstDevice });}snoopDHCPClientName(eth, srcDevice);return Command.CONTINUE;}

vitualNetworkFilter

  • 虚拟网络过滤器模块是基于虚拟化网络的数据链路层,它允许你在独立的数据链路层上创建多个逻辑链路
  • 若是使用 floodlightdefault.properties 则没有这个模块

如何工作

在 Floodlight 启动时,没有虚拟网络创建,这时主机之间不能相互通信。
一旦用户创建虚拟网络,则主机就能够被添加。
在 PACKET-IN 消息转发实现前,模块将启动。
一旦,一条 PACKET-IN 消息被接受,模块将查看源 MAC 地址和目的 MAC 地址,如果2个 MAC 地址是同一个虚拟网络,模块将返回 Command.CONINUE消息,并且继续处理流。如果MAC 地址不在同一个虚拟网络则返回 Command.STOP 消息,并丢弃包

限制

  • 必须在同一个物理数据链路层中
  • 每个虚拟网络只能拥有一个网关()【一个网关可被多个虚拟网络共享】
  • 多播和广播没有被隔离
  • 允许所有的 DHCP 路径

配置

该模块可用于 OpenStack 的部署
包含此模块的默认配置文件位置:
src/main/resources/neutron.properties

IOFMessageListener 的 receive 方法

    @Overridepublic Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:return processPacketIn(sw, (OFPacketIn)msg, cntx);default:break;}log.warn("Received unexpected message {}", msg);return Command.CONTINUE;}

主要使用了processPacketIn()方法

    protected Command processPacketIn(IOFSwitch sw, OFPacketIn msg, FloodlightContext cntx) {Ethernet eth = IFloodlightProviderService.bcStore.get(cntx,IFloodlightProviderService.CONTEXT_PI_PAYLOAD);Command ret = Command.STOP;String srcNetwork = macToGuid.get(eth.getSourceMACAddress());// If the host is on an unknown network we deny it.// We make exceptions for ARP and DHCP.if (eth.isBroadcast() || eth.isMulticast() || isDefaultGateway(eth) || isDhcpPacket(eth)) {ret = Command.CONTINUE;} else if (srcNetwork == null) {log.trace("Blocking traffic from host {} because it is not attached to any network.",eth.getSourceMACAddress().toString());ret = Command.STOP;} else if (oneSameNetwork(eth.getSourceMACAddress(), eth.getDestinationMACAddress())) {// if they are on the same network continueret = Command.CONTINUE;}if (log.isTraceEnabled())log.trace("Results for flow between {} and {} is {}",new Object[] {eth.getSourceMACAddress(), eth.getDestinationMACAddress(), ret});/** TODO - figure out how to still detect gateways while using* drop modsif (ret == Command.STOP) {if (!(eth.getPayload() instanceof ARP))doDropFlow(sw, msg, cntx);}*/return ret;}

LoadBalancer

IOFMessageListener 的 receive 方法

    @Overridepublic net.floodlightcontroller.core.IListener.Commandreceive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:return processPacketIn(sw, (OFPacketIn)msg, cntx);default:break;}log.warn("Received unexpected message {}", msg);return Command.CONTINUE;}

主要使用了processPacketIn()方法

    private net.floodlightcontroller.core.IListener.Command processPacketIn(IOFSwitch sw, OFPacketIn pi, FloodlightContext cntx) {Ethernet eth = IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD);IPacket pkt = eth.getPayload(); if (eth.isBroadcast() || eth.isMulticast()) {// handle ARP for VIPif (pkt instanceof ARP) {// retrieve arp to determine target IP address                                                       ARP arpRequest = (ARP) eth.getPayload();IPv4Address targetProtocolAddress = arpRequest.getTargetProtocolAddress();if (vipIpToId.containsKey(targetProtocolAddress.getInt())) {String vipId = vipIpToId.get(targetProtocolAddress.getInt());vipProxyArpReply(sw, pi, cntx, vipId);return Command.STOP;}}} else {// currently only load balance IPv4 packets - no-op for other traffic if (pkt instanceof IPv4) {IPv4 ip_pkt = (IPv4) pkt;// If match Vip and port, check pool and choose memberint destIpAddress = ip_pkt.getDestinationAddress().getInt();if (vipIpToId.containsKey(destIpAddress)){IPClient client = new IPClient();client.ipAddress = ip_pkt.getSourceAddress();client.nw_proto = ip_pkt.getProtocol();if (ip_pkt.getPayload() instanceof TCP) {TCP tcp_pkt = (TCP) ip_pkt.getPayload();client.srcPort = tcp_pkt.getSourcePort();client.targetPort = tcp_pkt.getDestinationPort();}if (ip_pkt.getPayload() instanceof UDP) {UDP udp_pkt = (UDP) ip_pkt.getPayload();client.srcPort = udp_pkt.getSourcePort();client.targetPort = udp_pkt.getDestinationPort();}if (ip_pkt.getPayload() instanceof ICMP) {client.srcPort = TransportPort.of(8); client.targetPort = TransportPort.of(0); }LBVip vip = vips.get(vipIpToId.get(destIpAddress));if (vip == null)            // fix dereference violations           return Command.CONTINUE;LBPool pool = pools.get(vip.pickPool(client));if (pool == null)           // fix dereference violationsreturn Command.CONTINUE;LBMember member = members.get(pool.pickMember(client));if(member == null)          //fix dereference violationsreturn Command.CONTINUE;// for chosen member, check device manager and find and push routes, in both directions                    pushBidirectionalVipRoutes(sw, pi, cntx, client, member);// packet out based on table rulepushPacket(pkt, sw, pi.getBufferId(), (pi.getVersion().compareTo(OFVersion.OF_12) < 0) ? pi.getInPort() : pi.getMatch().get(MatchField.IN_PORT), OFPort.TABLE,cntx, true);return Command.STOP;}}}// bypass non-load-balanced traffic for normal processing (forwarding)return Command.CONTINUE;}

ForwardingBase

IOFMessageListener 的 receive 方法

     @Overridepublic Command receive(IOFSwitch sw, OFMessage msg, FloodlightContext cntx) {switch (msg.getType()) {case PACKET_IN:IRoutingDecision decision = null;if (cntx != null) {decision = RoutingDecision.rtStore.get(cntx, IRoutingDecision.CONTEXT_DECISION);}return this.processPacketInMessage(sw, (OFPacketIn) msg, decision, cntx);default:break;}return Command.CONTINUE;}

主要使用了processPacketInMessage()方法

    public abstract Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx);

所有继承了 ForwardingBase 的子类Forwarding重写了这个方法,实现具体的操作

    @Overridepublic Command processPacketInMessage(IOFSwitch sw, OFPacketIn pi, IRoutingDecision decision, FloodlightContext cntx) {Ethernet eth = IFloodlightProviderService.bcStore.get(cntx, IFloodlightProviderService.CONTEXT_PI_PAYLOAD);// We found a routing decision (i.e. Firewall is enabled... it's the only thing that makes RoutingDecisions)if (decision != null) {if (log.isTraceEnabled()) {log.trace("Forwarding decision={} was made for PacketIn={}", decision.getRoutingAction().toString(), pi);}switch(decision.getRoutingAction()) {case NONE:// don't do anythingreturn Command.CONTINUE;case FORWARD_OR_FLOOD:case FORWARD:doForwardFlow(sw, pi, decision, cntx, false);return Command.CONTINUE;case MULTICAST:// treat as broadcastdoFlood(sw, pi, decision, cntx);return Command.CONTINUE;case DROP:doDropFlow(sw, pi, decision, cntx);return Command.CONTINUE;default:log.error("Unexpected decision made for this packet-in={}", pi, decision.getRoutingAction());return Command.CONTINUE;}} else { // No routing decision was found. Forward to destination or flood if bcast or mcast.if (log.isTraceEnabled()) {log.trace("No decision was made for PacketIn={}, forwarding", pi);}if (eth.isBroadcast() || eth.isMulticast()) {doFlood(sw, pi, decision, cntx);} else {doForwardFlow(sw, pi, decision, cntx, false);}}return Command.CONTINUE;}

PACKET-IN

Floodlight 源码解读:FloodlightProvider相关推荐

  1. Bert系列(二)——源码解读之模型主体

    本篇文章主要是解读模型主体代码modeling.py.在阅读这篇文章之前希望读者们对bert的相关理论有一定的了解,尤其是transformer的结构原理,网上的资料很多,本文内容对原理部分就不做过多 ...

  2. Bert系列(三)——源码解读之Pre-train

    https://www.jianshu.com/p/22e462f01d8c pre-train是迁移学习的基础,虽然Google已经发布了各种预训练好的模型,而且因为资源消耗巨大,自己再预训练也不现 ...

  3. linux下free源码,linux命令free源码解读:Procps free.c

    linux命令free源码解读 linux命令free源码解读:Procps free.c 作者:isayme 发布时间:September 26, 2011 分类:Linux 我们讨论的是linux ...

  4. nodeJS之eventproxy源码解读

    1.源码缩影 !(function (name, definition) { var hasDefine = typeof define === 'function', //检查上下文环境是否为AMD ...

  5. PyTorch 源码解读之即时编译篇

    点击上方"AI遇见机器学习",选择"星标"公众号 重磅干货,第一时间送达 作者丨OpenMMLab 来源丨https://zhuanlan.zhihu.com/ ...

  6. Alamofire源码解读系列(九)之响应封装(Response)

    本篇主要带来Alamofire中Response的解读 前言 在每篇文章的前言部分,我都会把我认为的本篇最重要的内容提前讲一下.我更想同大家分享这些顶级框架在设计和编码层次究竟有哪些过人的地方?当然, ...

  7. Feflow 源码解读

    Feflow 源码解读 Feflow(Front-end flow)是腾讯IVWEB团队的前端工程化解决方案,致力于改善多类型项目的开发流程中的规范和非业务相关的问题,可以让开发者将绝大部分精力集中在 ...

  8. spring-session源码解读 sesion

    2019独角兽企业重金招聘Python工程师标准>>> spring-session源码解读 sesion 博客分类: java spring 摘要: session通用策略 Ses ...

  9. 前端日报-20160527-underscore 源码解读

    underscore 源码解读 API文档浏览器 JavaScript 中加号操作符细节 抛弃 jQuery,拥抱原生 JS 从 0 开始学习 GitHub 系列之「加入 GitHub」 js实现克隆 ...

最新文章

  1. kvm(十)虚拟机存储池
  2. 八大排序算法图文讲解
  3. c语言程序设计实验报告2,C语言程序设计实验报告2.docx
  4. Android之Bitmap的内存优化方案总结
  5. oracle之数据处理之约束1
  6. 纯CSS实现的滑动的可折叠菜单
  7. python输出文字和数字加法_用c语言或者python将文件中特定字符串后面的数字相加...
  8. TCP/IP学习笔记(3)----IP,ARP,RARP协议
  9. iso是什么意思/iso9001质量管理体系认证有哪些标准
  10. 两种单例模式总结--(饿汉式、懒汉式)(java代码)
  11. 标准c语言教程gary,C语言标准教程
  12. 可靠性测试的定义、测试点及用例设计方法
  13. iOS 判断是否为iPhoneX以上设备
  14. aws cloudformation 堆栈集的创建和使用
  15. 基于springboot的在线小说阅读平台设计与实现-计算机毕业设计源码+LW文档
  16. “缺钱”的女人,掩饰不住以下三个特征,别不信
  17. 零基础学C语言之初识C语言(上)
  18. javascript时间差插件
  19. 经典文献阅读之--Cam2BEV
  20. 金仓KFS for Oracle RAC one node安装部署

热门文章

  1. flask(一)配置和路由
  2. 拨号上网外网使用NETMEETING
  3. Jodd 5.0 使用自定义WebApp及设置默认拦截器
  4. 简述3DES加密算法
  5. 王者荣耀服务器维护到几点,王者荣耀维护到几点?6.23维护时间介绍[多图]
  6. 逻辑回归模型(Logistic Regression)
  7. C/C++ 运行环境,编写代码工具软件安装
  8. 夸赞计算机文化节短句,赞美艺术与生活的句子
  9. HTTP 401 错误
  10. Grafana系列:Gauge下面的Legend怎么不显示了