大聊Python----Select解析

1、首先列一下，sellect、poll、epoll三者的区别
select
select最早于1983年出现在4.2BSD中，它通过一个select()系统调用来监视多个文件描述符的数组，当select()返回后，该数组中就绪的文件描述符便会被内核修改标志位，使得进程可以获得这些文件描述符从而进行后续的读写操作。

select目前几乎在所有的平台上支持，其良好跨平台支持也是它的一个优点，事实上从现在看来，这也是它所剩不多的优点之一。

select的一个缺点在于单个进程能够监视的文件描述符的数量存在最大限制，在Linux上一般为1024，不过可以通过修改宏定义甚至重新编译内核的方式提升这一限制。

另外，select()所维护的存储大量文件描述符的数据结构，随着文件描述符数量的增大，其复制的开销也线性增长。同时，由于网络响应时间的延迟使得大量TCP连接处于非活跃状态，但调用select()会对所有socket进行一次线性扫描，所以这也浪费了一定的开销。

poll
poll在1986年诞生于System V Release 3，它和select在本质上没有多大差别，但是poll没有最大文件描述符数量的限制。

poll和select同样存在一个缺点就是，包含大量文件描述符的数组被整体复制于用户态和内核的地址空间之间，而不论这些文件描述符是否就绪，它的开销随着文件描述符数量的增加而线性增大。

另外，select()和poll()将就绪的文件描述符告诉进程后，如果进程没有对其进行IO操作，那么下次调用select()和poll()的时候将再次报告这些文件描述符，所以它们一般不会丢失就绪的消息，这种方式称为水平触发（Level Triggered）。

epoll
直到Linux2.6才出现了由内核直接支持的实现方法，那就是epoll，它几乎具备了之前所说的一切优点，被公认为Linux2.6下性能最好的多路I/O就绪通知方法。

epoll可以同时支持水平触发和边缘触发（Edge Triggered，只告诉进程哪些文件描述符刚刚变为就绪状态，它只说一遍，如果我们没有采取行动，那么它将不会再次告知，这种方式称为边缘触发），理论上边缘触发的性能要更高一些，但是代码实现相当复杂。

epoll同样只告知那些就绪的文件描述符，而且当我们调用epoll_wait()获得就绪文件描述符时，返回的不是实际的描述符，而是一个代表就绪描述符数量的值，你只需要去epoll指定的一个数组中依次取得相应数量的文件描述符即可，这里也使用了内存映射（mmap）技术，这样便彻底省掉了这些文件描述符在系统调用时复制的开销。

另一个本质的改进在于epoll采用基于事件的就绪通知方式。在select/poll中，进程只有在调用一定的方法后，内核才对所有监视的文件描述符进行扫描，而epoll事先通过epoll_ctl()来注册一个文件描述符，一旦基于某个文件描述符就绪时，内核会采用类似callback的回调机制，迅速激活这个文件描述符，当进程调用epoll_wait()时便得到通知。

2、Python select

Python的select()方法直接调用操作系统的IO接口，它监控sockets,open files, and pipes(所有带fileno()方法的文件句柄)何时变成readable 和writeable, 或者通信错误，select()使得同时监控多个连接变的简单，并且这比写一个长循环来等待和监控多客户端连接要高效，因为select直接通过操作系统提供的C的网络接口进行操作，而不是通过Python的解释器。

注意：Using Python’s file objects with select() works for Unix, but is not supported under Windows.

接下来通过echo server例子要以了解select 是如何通过单进程实现同时处理多个非阻塞的socket连接的

## socket_client #3import socketHOST = 'HW-20180425SPSL'  # The remote host
PORT = 9000  # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
while True:msg = bytes(input(">>:"), encoding="utf8")s.sendall(msg)data = s.recv(1024)#
    print('Received', data)
s.close()

## socket_server ##import select
import socket
import queueserver = socket.socket()
server.bind(('HW-20180425SPSL',9000))
server.listen(1000)server.setblocking(False) #不阻塞

msg_dic = {}inputs = [server,]
#inputs = [server,conn] #[conn,]
#inputs = [server,conn,conn2] #[conn2,]
outputs = [] #
#outputs = [r1,] #
while True:readable ,writeable,exceptional= select.select(inputs, outputs, inputs )print(readable,writeable,exceptional)for r in readable:if r is server: #代表来了一个新连接conn,addr = server.accept()print("来了个新连接",addr)inputs.append(conn) #是因为这个新建立的连接还没发数据过来，现在就接收的话程序就报错了，#所以要想实现这个客户端发数据来时server端能知道，就需要让select再监测这个connmsg_dic[conn] = queue.Queue() #初始化一个队列，后面存要返回给这个客户端的数据else: #conn2data = r.recv(1024)print("收到数据",data)msg_dic[r].put(data)outputs.append(r) #放入返回的连接队列里# r.send(data)# print("send done....")for w in writeable: #要返回给客户端的连接列表data_to_client = msg_dic[w].get()w.send(data_to_client) #返回给客户端源数据
outputs.remove(w) #确保下次循环的时候writeable,不返回这个已经处理完的连接了for e in exceptional:if e in outputs:outputs.remove(e)inputs.remove(e)del msg_dic[e]

程序执行后的结果为：

socket_client.py

>>:aa
Received b'aa'
>>:cc
Received b'cc'
>>:

socket_server.py

[<socket.socket fd=236, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9000)>] [] []
来了个新连接 ('10.3.9.223', 55065)
[<socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9000), raddr=('10.3.9.223', 55065)>] [] []
收到数据 b'aa'
[] [<socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9000), raddr=('10.3.9.223', 55065)>] []
[<socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9000), raddr=('10.3.9.223', 55065)>] [] []
收到数据 b'cc'
[] [<socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9000), raddr=('10.3.9.223', 55065)>] []

3、selectors模块

该模块封装了select、epoll，在windows里默认使用select，而在Linux里，默认使用epoll

## socket_client ##import socket
import sysmessages = [ b'This is the message. ',b'It will be sent ',b'in parts.',]
server_address = ('HW-20180425SPSL', 9998)# Create a TCP/IP socket
socks = [ socket.socket(socket.AF_INET, socket.SOCK_STREAM) for i in range(2)]
print(socks)
# Connect the socket to the port where the server is listening
print('connecting to %s port %s' % server_address)
for s in socks:s.connect(server_address)for message in messages:# Send messages on both socketsfor s in socks:print('%s: sending "%s"' % (s.getsockname(), message) )s.send(message)# Read responses on both socketsfor s in socks:data = s.recv(1024)print( '%s: received "%s"' % (s.getsockname(), data) )if not data:print( 'closing socket', s.getsockname() )

## socket_server ##import selectors
import socketsel = selectors.DefaultSelector()def accept(sock, mask):conn, addr = sock.accept()  # Should be readyprint('accepted', conn, 'from', addr,mask)conn.setblocking(False)sel.register(conn, selectors.EVENT_READ, read) #新连接注册read回调函数def read(conn, mask):data = conn.recv(1024)  # Should be readyif data:print('echoing', repr(data), 'to', conn)conn.send(data)  # Hope it won't blockelse:print('closing', conn)sel.unregister(conn)conn.close()sock = socket.socket()
sock.bind(('HW-20180425SPSL', 9998))
sock.listen(100)
sock.setblocking(False)
sel.register(sock, selectors.EVENT_READ, accept)while True:events = sel.select() #默认阻塞，有活动连接就返回活动的连接列表for key, mask in events:callback = key.data #acceptcallback(key.fileobj, mask) #key.fileobj=  文件句柄

程序执行后的结果为：

socket_client.py

[<socket.socket fd=236, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>, <socket.socket fd=248, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0>]
connecting to HW-20180425SPSL port 9998
('10.3.9.223', 58356): sending "b'This is the message. '"
('10.3.9.223', 58357): sending "b'This is the message. '"
('10.3.9.223', 58356): received "b'This is the message. '"
('10.3.9.223', 58357): received "b'This is the message. '"
('10.3.9.223', 58356): sending "b'It will be sent '"
('10.3.9.223', 58357): sending "b'It will be sent '"
('10.3.9.223', 58356): received "b'It will be sent '"
('10.3.9.223', 58357): received "b'It will be sent '"
('10.3.9.223', 58356): sending "b'in parts.'"
('10.3.9.223', 58357): sending "b'in parts.'"
('10.3.9.223', 58356): received "b'in parts.'"
('10.3.9.223', 58357): received "b'in parts.'"

socket_server.py

accepted <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58344)> from ('10.3.9.223', 58344) 1
accepted <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58345)> from ('10.3.9.223', 58345) 1
accepted <socket.socket fd=304, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58346)> from ('10.3.9.223', 58346) 1
accepted <socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58347)> from ('10.3.9.223', 58347) 1
accepted <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58348)> from ('10.3.9.223', 58348) 1
accepted <socket.socket fd=316, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58349)> from ('10.3.9.223', 58349) 1
accepted <socket.socket fd=320, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58350)> from ('10.3.9.223', 58350) 1
accepted <socket.socket fd=324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58351)> from ('10.3.9.223', 58351) 1
accepted <socket.socket fd=328, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58352)> from ('10.3.9.223', 58352) 1
accepted <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58353)> from ('10.3.9.223', 58353) 1
echoing b'This is the message. ' to <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58344)>
echoing b'This is the message. ' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58345)>
echoing b'This is the message. ' to <socket.socket fd=304, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58346)>
echoing b'This is the message. ' to <socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58347)>
echoing b'This is the message. ' to <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58348)>
echoing b'This is the message. ' to <socket.socket fd=316, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58349)>
echoing b'This is the message. ' to <socket.socket fd=320, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58350)>
echoing b'This is the message. ' to <socket.socket fd=324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58351)>
echoing b'This is the message. ' to <socket.socket fd=328, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58352)>
echoing b'This is the message. ' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58353)>
echoing b'It will be sent ' to <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58344)>
echoing b'It will be sent ' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58345)>
echoing b'It will be sent ' to <socket.socket fd=304, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58346)>
echoing b'It will be sent ' to <socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58347)>
echoing b'It will be sent ' to <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58348)>
echoing b'It will be sent ' to <socket.socket fd=316, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58349)>
echoing b'It will be sent ' to <socket.socket fd=320, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58350)>
echoing b'It will be sent ' to <socket.socket fd=324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58351)>
echoing b'It will be sent ' to <socket.socket fd=328, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58352)>
echoing b'It will be sent ' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58353)>
echoing b'in parts.' to <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58344)>
echoing b'in parts.' to <socket.socket fd=304, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58346)>
echoing b'in parts.' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58345)>
echoing b'in parts.' to <socket.socket fd=320, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58350)>
echoing b'in parts.' to <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58348)>
echoing b'in parts.' to <socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58347)>
echoing b'in parts.' to <socket.socket fd=316, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58349)>
echoing b'in parts.' to <socket.socket fd=328, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58352)>
echoing b'in parts.' to <socket.socket fd=324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58351)>
echoing b'in parts.' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58353)>
closing <socket.socket fd=328, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58352)>
closing <socket.socket fd=320, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58350)>
closing <socket.socket fd=324, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58351)>
closing <socket.socket fd=304, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58346)>
closing <socket.socket fd=312, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58348)>
closing <socket.socket fd=308, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58347)>
closing <socket.socket fd=316, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58349)>
closing <socket.socket fd=296, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58344)>
closing <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58345)>
closing <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58353)>
accepted <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58356)> from ('10.3.9.223', 58356) 1
accepted <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58357)> from ('10.3.9.223', 58357) 1
echoing b'This is the message. ' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58356)>
echoing b'This is the message. ' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58357)>
echoing b'It will be sent ' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58356)>
echoing b'It will be sent ' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58357)>
echoing b'in parts.' to <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58356)>
echoing b'in parts.' to <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58357)>
closing <socket.socket fd=332, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58356)>
closing <socket.socket fd=300, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.3.9.223', 9998), raddr=('10.3.9.223', 58357)>

转载于:https://www.cnblogs.com/zhuifeng-mayi/p/9303877.html

大聊Python----Select解析相关推荐

Python select解析
一.首先列一下,sellect.poll.epoll三者的区别 1.select a.select最早于1983年出现在4.2BSD中,它通过一个select()系统调用来监控多个文件描述符的数组,当 ...
python select模块_Python之select模块解析
首先列一下,sellect.poll.epoll三者的区别 select select最早于1983年出现在4.2BSD中,它通过一个select()系统调用来监视多个文件描述符的数组,当select ...
vscode如何添加本地python解释器、解析器 Interpreter？（Python: Select Interpreter）
先安装python扩展然后点ctrl+shift+p搜索python:select,选择解析器(或者也可以直接点左下方的) 然后360报毒了,允许通过,然后选择你想要的解释器然后就好了
python 管道异步读取 select_python之异步select解析
一.I/O多路复用 I/O的含义:在计算机领域常说的IO包括磁盘IO和网络IO,我们所说的IO复用主要是指网络IO:在Linux中一切皆文件,因此网络IO也经常用文件描述符FD来表示. 复用的含义:在 ...
python配置文件解析_Python中配置文件解析模块-ConfigParser
Python中有ConfigParser类,可以很方便的从配置文件中读取数据(如DB的配置,路径的配置). 配置文件的格式是: []包含的叫section, section 下有option=valu ...
命名管道 win7未响应_大数据分析Python建立分析数据管道
如果您曾经想通过流数据或快速变化的数据在线学习Python,那么您可能会熟悉数据管道的概念.数据管道允许您通过一系列步骤将数据从一种表示形式转换为另一种表示形式.数据管道是数据工程的关键部分,我们将在 ...
Python爬虫解析网页的4种方式值得收藏
用Python写爬虫工具在现在是一种司空见惯的事情,每个人都希望能够写一段程序去互联网上扒一点资料下来,用于数据分析或者干点别的事情. 我们知道,爬虫的原理无非是把目标网址的内容下载下来存储到内存中, ...
python 列表操作详解,Python列表解析操作实例总结
本文实例讲述了Python列表解析操作.分享给大家供大家参考,具体如下: 列表解析 Python 的强大特性之一是其对 list 的解析,它提供一种紧凑的方法,可以通过对 list 中的每个元素应用一 ...
大数据Python爬取B站电影排行榜——爬取信息
大数据Python爬取B站电影排行榜-信息爬取前言一.配置环境二.爬取B站电影排行榜top100 1.找到B站电影排行榜top100网页 2.用URL进行爬取信息三.建立Excel表并导出总 ...

大聊Python----Select解析

大聊Python----Select解析相关推荐

最新文章

热门文章