python-Day5-深入正则表达式--冒泡排序-时间复杂度 --常用模块学习:自定义模块--random模块:随机验证码--time datetime模块...
正则表达式
mport re #导入模块名 p= re.compile("^[0-9]") #生成要匹配的正则对象 , ^代表从开头匹配,[0-9]代表匹配0至9的任意一个数字, 所以这里的意思是对传进来的字符串进行匹配,如果这个字符串的开头第一个字符是数字,就代表匹配上了 m= p.match('14534Abc') #按上面生成的正则对象 去匹配 字符串, 如果能匹配成功,这个m就会有值, 否则m为None<br><br>if m: #不为空代表匹配上了print(m.group()) #m.group()返回匹配上的结果,此处为1,因为匹配上的是1这个字符<br>else:<br> print("doesn't match.")
上面的第2 和第3行也可以合并成一行来写:
m = p.match("^[0-9]",'14534Abc')
效果是一样的,区别在于,第一种方式是提前对要匹配的格式进行了编译(对匹配公式进行解析),这样再去匹配的时候就不用在编译匹配的格式,第2种简写是每次匹配的时候 都 要进行一次匹配公式的编译,所以,如果你需要从一个5w行的文件中匹配出所有以数字开头的行,建议先把正则公式进行编译再匹配,这样速度会快点。
匹配格式
模式 | 描述 |
---|---|
^ | 匹配字符串的开头 |
$ | 匹配字符串的末尾。 |
. | 匹配任意字符,除了换行符,当re.DOTALL标记被指定时,则可以匹配包括换行符的任意字符。 |
[...] | 用来表示一组字符,单独列出:[amk] 匹配 'a','m'或'k' |
[^...] | 不在[]中的字符:[^abc] 匹配除了a,b,c之外的字符。 |
re* | 匹配0个或多个的表达式。 |
re+ | 匹配1个或多个的表达式。 |
re? | 匹配0个或1个由前面的正则表达式定义的片段,非贪婪方式 |
re{ n} |
importre a= '211.222.233.244'w= re.match("([0-9]{1,3}\.){3}\d{1,3}",a)print(w.group()) |
re{ n,} | 精确匹配n个前面表达式。 |
re{ n, m} | 匹配 n 到 m 次由前面的正则表达式定义的片段,贪婪方式 |
a| b | 匹配a或b |
(re) | G匹配括号内的表达式,也表示一个组 |
(?imx) | 正则表达式包含三种可选标志:i, m, 或 x 。只影响括号中的区域。 |
(?-imx) | 正则表达式关闭 i, m, 或 x 可选标志。只影响括号中的区域。 |
(?: re) | 类似 (...), 但是不表示一个组 |
(?imx: re) | 在括号中使用i, m, 或 x 可选标志 |
(?-imx: re) | 在括号中不使用i, m, 或 x 可选标志 |
(?#...) | 注释. |
(?= re) | 前向肯定界定符。如果所含正则表达式,以 ... 表示,在当前位置成功匹配时成功,否则失败。但一旦所含表达式已经尝试,匹配引擎根本没有提高;模式的剩余部分还要尝试界定符的右边。 |
(?! re) | 前向否定界定符。与肯定界定符相反;当所含表达式不能在字符串当前位置匹配时成功 |
(?> re) | 匹配的独立模式,省去回溯。 |
\w | 匹配字母数字 |
\W | 匹配非字母数字 |
\s | 匹配任意空白字符,等价于 [\t\n\r\f]. |
\S | 匹配任意非空字符 |
\d | 匹配任意数字,等价于 [0-9]. |
\D | 匹配任意非数字 |
\A | 匹配字符串开始 |
\Z | 匹配字符串结束,如果是存在换行,只匹配到换行前的结束字符串。c |
\z | 匹配字符串结束 |
\G | 匹配最后匹配完成的位置。 |
\b | 匹配一个单词边界,也就是指单词和空格间的位置。例如, 'er\b' 可以匹配"never" 中的 'er',但不能匹配 "verb" 中的 'er'。 |
\B | 匹配非单词边界。'er\B' 能匹配 "verb" 中的 'er',但不能匹配 "never" 中的 'er'。 |
\n, \t, 等. | 匹配一个换行符。匹配一个制表符。等 |
\1...\9 | 匹配第n个分组的子表达式。 |
\10 | 匹配第n个分组的子表达式,如果它经匹配。否则指的是八进制字符码的表达式。 |
正则表达式常用5种操作
re.match(pattern, string) # 从头匹配
re.search(pattern, string) # 匹配整个字符串,直到找到一个匹配
re.split() # 将匹配到的格式当做分割点对字符串分割成列表
m = re.split("[0-9]", "aaa1bbb2ccc3ddd eel8")print(m)----------------------输出:
['aaa', 'bbb', 'ccc', 'ddd eel', '']
re.findall() # 找到所有要匹配的字符并返回列表格式
m = re.findall("[0-9]", "aaa1bbb2ccc3ddd eel8")print(m,type(m))------------------------------------输出: ['1', '2', '3', '8'] <class 'list'>
re.sub(pattern, repl, string, count,flag) # 替换匹配到的字符,count=2 表示只替换两次
m = re.sub("[0-9]","|", "aaa1bbb2ccc3ddd eel8",count=2)# m = re.findall("[0-9]", "aaa1bbb2ccc3ddd eel8")print(m,type(m))
------------------------------输出: aaa|bbb|ccc3ddd eel8<class 'str'>
正则表达式实例
字符匹配
实例 | 描述 |
---|---|
python | 匹配 "python". |
字符类
实例 | 描述 |
---|---|
[Pp]ython | 匹配 "Python" 或 "python" |
rub[ye] | 匹配 "ruby" 或 "rube" |
[aeiou] | 匹配中括号内的任意一个字母 |
[0-9] | 匹配任何数字。类似于 [0123456789] |
[a-z] | 匹配任何小写字母 |
[A-Z] | 匹配任何大写字母 |
[a-zA-Z0-9] | 匹配任何字母及数字 |
[^aeiou] | 除了aeiou字母以外的所有字符 |
[^0-9] | 匹配除了数字外的字符 |
特殊字符类
实例 | 描述 |
---|---|
. | 匹配除 "\n" 之外的任何单个字符。要匹配包括 '\n' 在内的任何字符,请使用象 '[.\n]' 的模式。 |
\d | 匹配一个数字字符。等价于 [0-9]。 |
\D | 匹配一个非数字字符。等价于 [^0-9]。 |
\s | 匹配任何空白字符,包括空格、制表符、换页符等等。等价于 [ \f\n\r\t\v]。 |
\S | 匹配任何非空白字符。等价于 [^ \f\n\r\t\v]。 |
\w | 匹配包括下划线的任何单词字符。等价于'[A-Za-z0-9_]'。 |
\W | 匹配任何非单词字符。等价于 '[^A-Za-z0-9_]'。 |
re.match与re.search的区别
re.match只匹配字符串的开始,如果字符串开始不符合正则表达式,则匹配失败,函数返回None;而re.search匹配整个字符串,直到找到一个匹配
1 data = [10,4,33,21,54,3,8,11,5,22,2,1,17,13,6,99,1231,435,634573,45,78,234]2 print(data)3 #定义循环range(len(data)-1)次 4 #为了减少循环次数这里定义,range(1,len(data)): 1位起始位置len(data)为结束位置 5 for j in range(1,len(data)):6 #这里range(len(data)-j): 定义为循环一次减少一次 7 for i in range(len(data)-j):8 #如果data[i]虾标位置的4 大于 data[i+1]虾标位置加一,也就是 虾标位置的数值与虾标位置+1的数值相比较 9 #如果虾标数值大于虾标数值+1 10 if data[i] > data[i+1]:11 #如果大于为真则将虾标位置+1对应的数值 做一个变量tmp 12 tmp = data[i+1]13 #把虾标+1位置的数值放在第一个虾标位置 14 data[i+1] =data[i]15 #把虾标位置的数值替换为 tmp数值 16 data[i] =tmp17 18 print(data)19 ---------------------------------- 20 输出:21 [10, 4, 33, 21, 54, 3, 8, 11, 5, 22, 2, 1, 17, 13, 6]22 [1, 2, 3, 4, 5, 6, 8, 10, 11, 13, 17, 21, 22, 33, 54]
冒泡排序
(1)时间频度 一个算法执行所耗费的时间,从理论上是不能算出来的,必须上机运行测试才能知道。但我们不可能也没有必要对每个算法都上机测试,只需知道哪个算法花费的时间多,哪个算法花费的时间少就可以了。并且一个算法花费的时间与算法中语句的执行次数成正比例,哪个算法中语句执行次数多,它花费时间就多。一个算法中的语句执行次数称为语句频度或时间频度。记为T(n)。
for (i=1; i<=n; i++)x++;for (i=1; i<=n; i++)for (j=1; j<=n; j++)x++;
第一个for循环的时间复杂度为Ο(n),第二个for循环的时间复杂度为Ο(n2),则整个算法的时间复杂度为Ο(n+n2)=Ο(n2)。
常数时间
若对于一个算法,的上界与输入大小无关,则称其具有常数时间,记作时间。一个例子是访问数组中的单个元素,因为访问它只需要一条指令。但是,找到无序数组中的最小元素则不是,因为这需要遍历所有元素来找出最小值。这是一项线性时间的操作,或称时间。但如果预先知道元素的数量并假设数量保持不变,则该操作也可被称为具有常数时间。
对数时间
若算法的T(n) = O(log n),则称其具有对数时间
常见的具有对数时间的算法有二叉树的相关操作和二分搜索。
对数时间的算法是非常有效的,因为每增加一个输入,其所需要的额外计算时间会变小。
递归地将字符串砍半并且输出是这个类别函数的一个简单例子。它需要O(log n)的时间因为每次输出之前我们都将字符串砍半。 这意味着,如果我们想增加输出的次数,我们需要将字符串长度加倍。
线性时间
如果一个算法的时间复杂度为O(n),则称这个算法具有线性时间,或O(n)时间。非正式地说,这意味着对于足够大的输入,运行时间增加的大小与输入成线性关系。例如,一个计算列表所有元素的和的程序,需要的时间与列表的长度成正比。
时间模块:time &datetime
随机数模块:random
系统交互:OS
sys系统模块
文件copy相关模块:shutil
- json & picle
- shelve
- xml处理
- yaml处理
- configparser
- hashlib
- subprocess
- logging模块
模块,用一砣代码实现了某个功能的代码集合。
类似于函数式编程和面向过程编程,函数式编程则完成一个功能,其他代码用来调用即可,提供了代码的重用性和代码间的耦合。而对于一个复杂的功能来,可能需要多个函数才能完成(函数又可以在不同的.py文件中),n个 .py 文件组成的代码集合就称为模块。
如:os 是系统相关的模块;file是文件操作相关的模块
模块分为三种:
- 自定义模块
- 内置标准模块(又称标准库)
- 开源模块
自定义模块 和开源模块的使用参考 http://www.cnblogs.com/wupeiqi/articles/4963027.html
time & datetime模块
开源模块
一、下载安装
下载安装有两种方式:
1 yum2 pip3 apt-get4 ...
方式一
1 下载源码2 解压源码3 进入目录4 编译源码 python setup.py build5 安装源码 python setup.py install
下载方式二
注:在使用源码安装时,需要使用到gcc编译和python开发环境,所以,需要先执行:
yum install gcc yum install python-devel 或 apt-get python-dev
安装成功后,模块会自动安装到 sys.path 中的某个目录中,如:
/usr/lib/python2.7/site-packages/
- 自定义模块
图一:
图二:
图三:
目录结构如下图:
打印当前程序的绝对路径
print(__file__)
import os print(os.path.abspath(__file__))
去除文件名,只打印目录名
importos#打印的时候每多加一层os.path.dirname 将会少打印一级目录 print(os.path.dirname(os.path.abspath(__file__)))print(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))===============================输出: F:\python\day5\dj F:\python\day5
#程序主入口 __author__ = 'Administrator' importsys#导入backend目录下logic目录下的handle函数 from backend.logic importhandle#执行handle中的home函数 handle.home()--------------------------------------------------------handle内容:__author__ = 'Administrator' #-*- coding:utf-8 -*-#导入backend目录下的db目录下的sql_api程序里的 select 函数 from backend.db.sql_api importselect#from backend.db.sql_api import selectdefhome():print("welcome to home page...")#给select函数传入参数q_data = select("user",'ddd')print("qeury res:",q_data)deftv():print("welcome to tv page...")defmovie():print("welcome to movie page...")-------------------------------------------------------------sql_api内容:__author__ = 'Administrator' #-*- coding:utf-8 -*-#导入config目录下的 settings 程序内容 from config importsettings#定义一个函数db_auth并且定义一个参数configs用来接收下边传来的settings参数 defdb_auth(configs):#判断settings程序中的DATABASE里定义的user用户是否等于root,密码是否等于123if configs.DATABASE["user"] == "root" and configs.DATABASE["password"] == '123':#如果上边两个条件都符合则为真,打印如下内容print("db authentication passed!")#上面内容都为真则通过return返回TruereturnTrueelse:#如果上边的判断内容 用户名和密码有一项不为真,则打印如下内容print("db login error...")defselect(table, column):ifdb_auth(settings):if table == 'user':user_info={"001": ['aaa', 22, 'engineer'],"002": ['bbb', 23, 'chef'],"003": ['ccc', 24, 'ermao'],}returnuser_info------------------------------------------------- #setting内容:__author__ = 'Administrator' #-*- CODING:UTF-8 -*- DATABASE ={"engine":"mysql","host":"localhost","port":"3306","user":"root","password":"123"}
模块别名:
importsysimport multiprocessing as mult
2、导入模块
Python之所以应用越来越广泛,在一定程度上也依赖于其为程序员提供了大量的模块以供使用,如果想要使用模块,则需要导入。导入模块有一下几种方法:
importmodulefrom module.xx.xx importxxfrom module.xx.xx importxx as renamefrom module.xx.xx import *
导入模块其实就是告诉Python解释器去解释那个py文件
- 导入一个py文件,解释器解释该py文件
- 导入一个包,解释器解释该包下的 __init__.py 文件
那么问题来了,导入模块时是根据那个路径作为基准来进行的呢?即:sys.path
importsysprint sys.path
如果sys.path路径列表没有你想要的路径,可以通过 sys.path.append('路径') 添加。
通过os模块可以获取各种目录,例如:
importsysimportospre_path= os.path.abspath('../') sys.path.append(pre_path)
time & datetime模块
1 #_*_coding:utf-8_*_ 2 importtime3 importdatetime4 5 print(time.clock()) #返回处理器时间,3.3开始已废弃 6 print(time.process_time()) #返回处理器时间,3.3开始已废弃 7 print(time.time()) #返回当前系统时间戳 8 print(time.ctime()) #输出Tue Jan 26 18:23:48 2016 ,当前系统时间 9 print(time.ctime(time.time()-86640)) #将时间戳转为字符串格式 10 print(time.gmtime(time.time()-86640)) #将时间戳转换成struct_time格式 11 print(time.localtime(time.time()-86640)) #将时间戳转换成struct_time格式,但返回 的本地时间 12 print(time.mktime(time.localtime())) #与time.localtime()功能相反,将struct_time格式转回成时间戳格式 13 #time.sleep(4) #sleep 14 print(time.strftime("%Y-%m-%d %H:%M:%S",time.gmtime()) ) #将struct_time格式转成指定的字符串格式 15 print(time.strptime("2016-01-28","%Y-%m-%d") ) #将字符串格式转换成struct_time格式 16 17 #datetime module 18 19 print(datetime.date.today()) #输出格式 2016-01-26 20 print(datetime.date.fromtimestamp(time.time()-864400) ) #2016-01-16 将时间戳转成日期格式 21 current_time = datetime.datetime.now() #22 print(current_time) #输出2016-01-26 19:04:30.335935 23 print(current_time.timetuple()) #返回struct_time格式 24 25 #datetime.replace([year[, month[, day[, hour[, minute[, second[, microsecond[, tzinfo]]]]]]]]) 26 print(current_time.replace(2014,9,12)) #输出2014-09-12 19:06:24.074900,返回当前时间,但指定的值将被替换 27 28 str_to_date = datetime.datetime.strptime("21/11/06 16:30", "%d/%m/%y %H:%M") #将字符串转换成日期格式 29 new_date = datetime.datetime.now() + datetime.timedelta(days=10) #比现在加10天 30 new_date = datetime.datetime.now() + datetime.timedelta(days=-10) #比现在减10天 31 new_date = datetime.datetime.now() + datetime.timedelta(hours=-10) #比现在减10小时 32 new_date = datetime.datetime.now() + datetime.timedelta(seconds=120) #比现在+120s 33 print(new_date)
Directive | Meaning | Notes |
---|---|---|
%a
|
Locale’s abbreviated weekday name. | |
%A
|
Locale’s full weekday name. | |
%b
|
Locale’s abbreviated month name. | |
%B
|
Locale’s full month name. | |
%c
|
Locale’s appropriate date and time representation. | |
%d
|
Day of the month as a decimal number [01,31]. | |
%H
|
Hour (24-hour clock) as a decimal number [00,23]. | |
%I
|
Hour (12-hour clock) as a decimal number [01,12]. | |
%j
|
Day of the year as a decimal number [001,366]. | |
%m
|
Month as a decimal number [01,12]. | |
%M
|
Minute as a decimal number [00,59]. | |
%p
|
Locale’s equivalent of either AM or PM. | (1) |
%S
|
Second as a decimal number [00,61]. | (2) |
%U
|
Week number of the year (Sunday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Sunday are considered to be in week 0. | (3) |
%w
|
Weekday as a decimal number [0(Sunday),6]. | |
%W
|
Week number of the year (Monday as the first day of the week) as a decimal number [00,53]. All days in a new year preceding the first Monday are considered to be in week 0. | (3) |
%x
|
Locale’s appropriate date representation. | |
%X
|
Locale’s appropriate time representation. | |
%y
|
Year without century as a decimal number [00,99]. | |
%Y
|
Year with century as a decimal number. | |
%z
|
Time zone offset indicating a positive or negative time difference from UTC/GMT of the form +HHMM or -HHMM, where H represents decimal hour digits and M represents decimal minute digits [-23:59, +23:59]. | |
%Z
|
Time zone name (no characters if no time zone exists). | |
%%
|
A literal |
random模块
随机数
mport randomprintrandom.random()print random.randint(1,2)print random.randrange(1,10)
生成随机验证码
importrandom#定义一个空值的验证码 check_code = "" #定义一个循环,循环4次 for i in range(4):current= random.randint(0,4)if current !=i:#chr 把一个数字转换为asscc码里对应的字母取出来temp = chr(random.randint(65,90))else:temp= random.randint(0,9)check_code+=str(temp)print(check_code)
OS模块
提供对操作系统进行调用的接口
os.getcwd() 获取当前工作目录,即当前python脚本工作的目录路径 os.chdir("dirname") 改变当前脚本工作目录;相当于shell下cd os.curdir 返回当前目录: ('.') os.pardir 获取当前目录的父目录字符串名:('..') os.makedirs('dirname1/dirname2') 可生成多层递归目录 os.removedirs('dirname1') 若目录为空,则删除,并递归到上一级目录,如若也为空,则删除,依此类推 os.mkdir('dirname') 生成单级目录;相当于shell中mkdir dirname os.rmdir('dirname') 删除单级空目录,若目录不为空则无法删除,报错;相当于shell中rmdir dirname os.listdir('dirname') 列出指定目录下的所有文件和子目录,包括隐藏文件,并以列表方式打印 os.remove() 删除一个文件 os.rename("oldname","newname") 重命名文件/目录 os.stat('path/filename') 获取文件/目录信息 os.sep 输出操作系统特定的路径分隔符,win下为"\\",Linux下为"/"os.linesep 输出当前平台使用的行终止符,win下为"\t\n",Linux下为"\n"os.pathsep 输出用于分割文件路径的字符串 os.name 输出字符串指示当前使用平台。win->'nt'; Linux->'posix'os.system("bash command") 运行shell命令,直接显示 os.environ 获取系统环境变量 os.path.abspath(path) 返回path规范化的绝对路径 os.path.split(path) 将path分割成目录和文件名二元组返回 os.path.dirname(path) 返回path的目录。其实就是os.path.split(path)的第一个元素 os.path.basename(path) 返回path最后的文件名。如何path以/或\结尾,那么就会返回空值。即os.path.split(path)的第二个元素 os.path.exists(path) 如果path存在,返回True;如果path不存在,返回False os.path.isabs(path) 如果path是绝对路径,返回True os.path.isfile(path) 如果path是一个存在的文件,返回True。否则返回False os.path.isdir(path) 如果path是一个存在的目录,则返回True。否则返回False os.path.join(path1[, path2[, ...]]) 将多个路径组合后返回,第一个绝对路径之前的参数将被忽略 os.path.getatime(path) 返回path所指向的文件或者目录的最后存取时间 os.path.getmtime(path) 返回path所指向的文件或者目录的最后修改时间
更多猛击这里
sys模块
sys.argv 命令行参数List,第一个元素是程序本身路径 sys.exit(n) 退出程序,正常退出时exit(0) sys.version 获取Python解释程序的版本信息 sys.maxint 最大的Int值 sys.path 返回模块的搜索路径,初始化时使用PYTHONPATH环境变量的值 sys.platform 返回操作系统平台名称 sys.stdout.write('please:')__author__ = 'Administrator'
import sysimport timefor i in range(10): sys.stdout.write("*") sys.stdout.flush() time.sleep(0.3)
val= sys.stdin.readline()[:-1]
shutil 模块
高级的 文件、文件夹、压缩包 处理模块
shutil.copyfileobj(fsrc, fdst[, length])
将文件内容拷贝到另一个文件中,可以部分内容
def copyfileobj(fsrc, fdst, length=16*1024):"""copy data from file-like object fsrc to file-like object fdst"""while 1:buf=fsrc.read(length)if notbuf:breakfdst.write(buf)
shutil.copyfile(src, dst)
拷贝文件
1 defcopyfile(src, dst):2 """Copy data from src to dst""" 3 if_samefile(src, dst):4 raise Error("`%s` and `%s` are the same file" %(src, dst))5 6 for fn in[src, dst]:7 try:8 st =os.stat(fn)9 exceptOSError:10 #File most likely does not exist 11 pass 12 else:13 #XXX What about other special files? (sockets, devices...) 14 ifstat.S_ISFIFO(st.st_mode):15 raise SpecialFileError("`%s` is a named pipe" %fn)16 17 with open(src, 'rb') as fsrc:18 with open(dst, 'wb') as fdst:19 copyfileobj(fsrc, fdst)
copyfile
shutil.copymode(src, dst)
仅拷贝权限。内容、组、用户均不变
1 defcopymode(src, dst):2 """Copy mode bits from src to dst""" 3 if hasattr(os, 'chmod'):4 st =os.stat(src)5 mode =stat.S_IMODE(st.st_mode)6 os.chmod(dst, mode)
shutil.copymode(src, dst)
shutil.copystat(src, dst)
拷贝状态的信息,包括:mode bits, atime, mtime, flags
1 defcopystat(src, dst):2 """Copy all stat info (mode bits, atime, mtime, flags) from src to dst""" 3 st =os.stat(src)4 mode =stat.S_IMODE(st.st_mode)5 if hasattr(os, 'utime'):6 os.utime(dst, (st.st_atime, st.st_mtime))7 if hasattr(os, 'chmod'):8 os.chmod(dst, mode)9 if hasattr(os, 'chflags') and hasattr(st, 'st_flags'):10 try:11 os.chflags(dst, st.st_flags)12 exceptOSError, why:13 for err in 'EOPNOTSUPP', 'ENOTSUP':14 if hasattr(errno, err) and why.errno ==getattr(errno, err):15 break 16 else:17 raise
shutil.copystat(src, dst)
shutil.copy(src, dst)
拷贝文件和权限
1 defcopy(src, dst):2 """Copy data and mode bits ("cp src dst").3 4 The destination may be a directory.5 6 """ 7 ifos.path.isdir(dst):8 dst =os.path.join(dst, os.path.basename(src))9 copyfile(src, dst)10 copymode(src, dst)
shutil.copy(src, dst)
shutil.copy2(src, dst)
拷贝文件和状态信息
1 defcopy2(src, dst):2 """Copy data and all stat info ("cp -p src dst").3 4 The destination may be a directory.5 6 """ 7 ifos.path.isdir(dst):8 dst =os.path.join(dst, os.path.basename(src))9 copyfile(src, dst)10 copystat(src, dst)
shutil.copy2(src, dst)
shutil.ignore_patterns(*patterns)
shutil.copytree(src, dst, symlinks=False, ignore=None)
递归的去拷贝文件
例如:copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))
1 def ignore_patterns(*patterns):2 """Function that can be used as copytree() ignore parameter.3 4 Patterns is a sequence of glob-style patterns5 that are used to exclude files""" 6 def_ignore_patterns(path, names):7 ignored_names =[]8 for pattern inpatterns:9 ignored_names.extend(fnmatch.filter(names, pattern))10 returnset(ignored_names)11 return_ignore_patterns12 13 def copytree(src, dst, symlinks=False, ignore=None):14 """Recursively copy a directory tree using copy2().15 16 The destination directory must not already exist.17 If exception(s) occur, an Error is raised with a list of reasons.18 19 If the optional symlinks flag is true, symbolic links in the20 source tree result in symbolic links in the destination tree; if21 it is false, the contents of the files pointed to by symbolic22 links are copied.23 24 The optional ignore argument is a callable. If given, it25 is called with the `src` parameter, which is the directory26 being visited by copytree(), and `names` which is the list of27 `src` contents, as returned by os.listdir():28 29 callable(src, names) -> ignored_names30 31 Since copytree() is called recursively, the callable will be32 called once for each directory that is copied. It returns a33 list of names relative to the `src` directory that should34 not be copied.35 36 XXX Consider this example code rather than the ultimate tool.37 38 """ 39 names =os.listdir(src)40 if ignore is notNone:41 ignored_names =ignore(src, names)42 else:43 ignored_names =set()44 45 os.makedirs(dst)46 errors =[]47 for name innames:48 if name inignored_names:49 continue 50 srcname =os.path.join(src, name)51 dstname =os.path.join(dst, name)52 try:53 if symlinks andos.path.islink(srcname):54 linkto =os.readlink(srcname)55 os.symlink(linkto, dstname)56 elifos.path.isdir(srcname):57 copytree(srcname, dstname, symlinks, ignore)58 else:59 #Will raise a SpecialFileError for unsupported file types 60 copy2(srcname, dstname)61 #catch the Error from the recursive copytree so that we can 62 #continue with other files 63 exceptError, err:64 errors.extend(err.args[0])65 exceptEnvironmentError, why:66 errors.append((srcname, dstname, str(why)))67 try:68 copystat(src, dst)69 exceptOSError, why:70 if WindowsError is not None andisinstance(why, WindowsError):71 #Copying file access times may fail on Windows 72 pass 73 else:74 errors.append((src, dst, str(why)))75 iferrors:76 raise Error, errors
shutil.ignore_patterns(*patterns) shutil.copytree(src, dst, symlinks=False, ignore=None
shutil.rmtree(path[, ignore_errors[, onerror]])
递归的去删除文件
1 def rmtree(path, ignore_errors=False, οnerrοr=None):2 """Recursively delete a directory tree.3 4 If ignore_errors is set, errors are ignored; otherwise, if onerror5 is set, it is called to handle the error with arguments (func,6 path, exc_info) where func is os.listdir, os.remove, or os.rmdir;7 path is the argument to that function that caused it to fail; and8 exc_info is a tuple returned by sys.exc_info(). If ignore_errors9 is false and onerror is None, an exception is raised.10 11 """ 12 ifignore_errors:13 def onerror(*args):14 pass 15 elif onerror isNone:16 def onerror(*args):17 raise 18 try:19 ifos.path.islink(path):20 #symlinks to directories are forbidden, see bug #1669 21 raise OSError("Cannot call rmtree on a symbolic link")22 exceptOSError:23 onerror(os.path.islink, path, sys.exc_info())24 #can't continue even if onerror hook returns 25 return 26 names =[]27 try:28 names =os.listdir(path)29 exceptos.error, err:30 onerror(os.listdir, path, sys.exc_info())31 for name innames:32 fullname =os.path.join(path, name)33 try:34 mode =os.lstat(fullname).st_mode35 exceptos.error:36 mode =037 ifstat.S_ISDIR(mode):38 rmtree(fullname, ignore_errors, onerror)39 else:40 try:41 os.remove(fullname)42 exceptos.error, err:43 onerror(os.remove, fullname, sys.exc_info())44 try:45 os.rmdir(path)46 exceptos.error:47 onerror(os.rmdir, path, sys.exc_info())
shutil.rmtree(path[, ignore_errors[, onerror]])
shutil.move(src, dst)
递归的去移动文件
1 defmove(src, dst):2 """Recursively move a file or directory to another location. This is3 similar to the Unix "mv" command.4 5 If the destination is a directory or a symlink to a directory, the source6 is moved inside the directory. The destination path must not already7 exist.8 9 If the destination already exists but is not a directory, it may be10 overwritten depending on os.rename() semantics.11 12 If the destination is on our current filesystem, then rename() is used.13 Otherwise, src is copied to the destination and then removed.14 A lot more could be done here... A look at a mv.c shows a lot of15 the issues this implementation glosses over.16 17 """ 18 real_dst =dst19 ifos.path.isdir(dst):20 if_samefile(src, dst):21 #We might be on a case insensitive filesystem, 22 #perform the rename anyway. 23 os.rename(src, dst)24 return 25 26 real_dst =os.path.join(dst, _basename(src))27 ifos.path.exists(real_dst):28 raise Error, "Destination path '%s' already exists" %real_dst29 try:30 os.rename(src, real_dst)31 exceptOSError:32 ifos.path.isdir(src):33 if_destinsrc(src, dst):34 raise Error, "Cannot move a directory '%s' into itself '%s'." %(src, dst)35 copytree(src, real_dst, symlinks=True)36 rmtree(src)37 else:38 copy2(src, real_dst)39 os.unlink(src)
shutil.move(src, dst)
shutil.make_archive(base_name, format,...)
创建压缩包并返回文件路径,例如:zip、tar
- base_name: 压缩包的文件名,也可以是压缩包的路径。只是文件名时,则保存至当前目录,否则保存至指定路径,
如:www =>保存至当前路径
如:/Users/wupeiqi/www =>保存至/Users/wupeiqi/ - format: 压缩包种类,“zip”, “tar”, “bztar”,“gztar”
- root_dir: 要压缩的文件夹路径(默认当前目录)
- owner: 用户,默认当前用户
- group: 组,默认当前组
- logger: 用于记录日志,通常是logging.Logger对象
#将 /Users/wupeiqi/Downloads/test 下的文件打包放置当前程序目录importshutil ret= shutil.make_archive("wwwwwwwwww", 'gztar', root_dir='/Users/wupeiqi/Downloads/test')#将 /Users/wupeiqi/Downloads/test 下的文件打包放置 /Users/wupeiqi/目录 importshutil ret= shutil.make_archive("/Users/wupeiqi/wwwwwwwwww", 'gztar', root_dir='/Users/wupeiqi/Downloads/test')
1 def make_archive(base_name, format, root_dir=None, base_dir=None, verbose=0,2 dry_run=0, owner=None, group=None, logger=None):3 """Create an archive file (eg. zip or tar).4 5 'base_name' is the name of the file to create, minus any format-specific6 extension; 'format' is the archive format: one of "zip", "tar", "bztar"7 or "gztar".8 9 'root_dir' is a directory that will be the root directory of the10 archive; ie. we typically chdir into 'root_dir' before creating the11 archive. 'base_dir' is the directory where we start archiving from;12 ie. 'base_dir' will be the common prefix of all files and13 directories in the archive. 'root_dir' and 'base_dir' both default14 to the current directory. Returns the name of the archive file.15 16 'owner' and 'group' are used when creating a tar archive. By default,17 uses the current owner and group.18 """ 19 save_cwd =os.getcwd()20 if root_dir is notNone:21 if logger is notNone:22 logger.debug("changing into '%s'", root_dir)23 base_name =os.path.abspath(base_name)24 if notdry_run:25 os.chdir(root_dir)26 27 if base_dir isNone:28 base_dir =os.curdir29 30 kwargs = {'dry_run': dry_run, 'logger': logger}31 32 try:33 format_info =_ARCHIVE_FORMATS[format]34 exceptKeyError:35 raise ValueError, "unknown archive format '%s'" %format36 37 func =format_info[0]38 for arg, val in format_info[1]:39 kwargs[arg] =val40 41 if format != 'zip':42 kwargs['owner'] =owner43 kwargs['group'] =group44 45 try:46 filename = func(base_name, base_dir, **kwargs)47 finally:48 if root_dir is notNone:49 if logger is notNone:50 logger.debug("changing back to '%s'", save_cwd)51 os.chdir(save_cwd)52 53 return filename
View Code
shutil 对压缩包的处理是调用 ZipFile 和 TarFile 两个模块来进行的,详细:
1 importzipfile2 3 #压缩 4 z = zipfile.ZipFile('laxi.zip', 'w')5 z.write('a.log')6 z.write('data.data')7 z.close()8 9 #解压 10 z = zipfile.ZipFile('laxi.zip', 'r')11 z.extractall()12 z.close()13 14 zipfile 压缩解压
对压缩包的处理是调用 ZipFile 和 TarFile 两个模块来进行的
1 importtarfile2 3 #压缩 4 tar = tarfile.open('your.tar','w')5 tar.add('/Users/wupeiqi/PycharmProjects/bbs2.zip', arcname='bbs2.zip')6 tar.add('/Users/wupeiqi/PycharmProjects/cmdb.zip', arcname='cmdb.zip')7 tar.close()8 9 #解压 10 tar = tarfile.open('your.tar','r')11 tar.extractall() #可设置解压地址 12 tar.close()13 14 tarfile 压缩解压
tarfile 压缩解压
1 classZipFile(object):2 """Class with methods to open, read, write, close, list zip files.3 4 z = ZipFile(file, mode="r", compression=ZIP_STORED, allowZip64=False)5 6 file: Either the path to the file, or a file-like object.7 If it is a path, the file will be opened and closed by ZipFile.8 mode: The mode can be either read "r", write "w" or append "a".9 compression: ZIP_STORED (no compression) or ZIP_DEFLATED (requires zlib).10 allowZip64: if True ZipFile will create files with ZIP64 extensions when11 needed, otherwise it will raise an exception when this would12 be necessary.13 14 """ 15 16 fp = None #Set here since __del__ checks it 17 18 def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False):19 """Open the ZIP file with mode read "r", write "w" or append "a".""" 20 if mode not in ("r", "w", "a"):21 raise RuntimeError('ZipFile() requires mode "r", "w", or "a"')22 23 if compression ==ZIP_STORED:24 pass 25 elif compression ==ZIP_DEFLATED:26 if notzlib:27 raiseRuntimeError,\28 "Compression requires the (missing) zlib module" 29 else:30 raise RuntimeError, "That compression method is not supported" 31 32 self._allowZip64 =allowZip6433 self._didModify =False34 self.debug = 0 #Level of printing: 0 through 3 35 self.NameToInfo = {} #Find file info given name 36 self.filelist = [] #List of ZipInfo instances for archive 37 self.compression = compression #Method of compression 38 self.mode = key = mode.replace('b', '')[0]39 self.pwd =None40 self._comment = '' 41 42 #Check if we were passed a file-like object 43 ifisinstance(file, basestring):44 self._filePassed =045 self.filename =file46 modeDict = {'r' : 'rb', 'w': 'wb', 'a' : 'r+b'}47 try:48 self.fp =open(file, modeDict[mode])49 exceptIOError:50 if mode == 'a':51 mode = key = 'w' 52 self.fp =open(file, modeDict[mode])53 else:54 raise 55 else:56 self._filePassed = 1 57 self.fp =file58 self.filename = getattr(file, 'name', None)59 60 try:61 if key == 'r':62 self._RealGetContents()63 elif key == 'w':64 #set the modified flag so central directory gets written 65 #even if no files are added to the archive 66 self._didModify =True67 elif key == 'a':68 try:69 #See if file is a zip file 70 self._RealGetContents()71 #seek to start of directory and overwrite 72 self.fp.seek(self.start_dir, 0)73 exceptBadZipfile:74 #file is not a zip file, just append 75 self.fp.seek(0, 2)76 77 #set the modified flag so central directory gets written 78 #even if no files are added to the archive 79 self._didModify =True80 else:81 raise RuntimeError('Mode must be "r", "w" or "a"')82 except:83 fp =self.fp84 self.fp =None85 if notself._filePassed:86 fp.close()87 raise 88 89 def __enter__(self):90 returnself91 92 def __exit__(self, type, value, traceback):93 self.close()94 95 def_RealGetContents(self):96 """Read in the table of contents for the ZIP file.""" 97 fp =self.fp98 try:99 endrec =_EndRecData(fp)100 exceptIOError:101 raise BadZipfile("File is not a zip file")102 if notendrec:103 raise BadZipfile, "File is not a zip file" 104 if self.debug > 1:105 printendrec106 size_cd = endrec[_ECD_SIZE] #bytes in central directory 107 offset_cd = endrec[_ECD_OFFSET] #offset of central directory 108 self._comment = endrec[_ECD_COMMENT] #archive comment 109 110 #"concat" is zero, unless zip was concatenated to another file 111 concat = endrec[_ECD_LOCATION] - size_cd -offset_cd112 if endrec[_ECD_SIGNATURE] ==stringEndArchive64:113 #If Zip64 extension structures are present, account for them 114 concat -= (sizeEndCentDir64 +sizeEndCentDir64Locator)115 116 if self.debug > 2:117 inferred = concat +offset_cd118 print "given, inferred, offset", offset_cd, inferred, concat119 #self.start_dir: Position of start of central directory 120 self.start_dir = offset_cd +concat121 fp.seek(self.start_dir, 0)122 data =fp.read(size_cd)123 fp =cStringIO.StringIO(data)124 total =0125 while total <size_cd:126 centdir =fp.read(sizeCentralDir)127 if len(centdir) !=sizeCentralDir:128 raise BadZipfile("Truncated central directory")129 centdir =struct.unpack(structCentralDir, centdir)130 if centdir[_CD_SIGNATURE] !=stringCentralDir:131 raise BadZipfile("Bad magic number for central directory")132 if self.debug > 2:133 printcentdir134 filename =fp.read(centdir[_CD_FILENAME_LENGTH])135 #Create ZipInfo instance to store file information 136 x =ZipInfo(filename)137 x.extra =fp.read(centdir[_CD_EXTRA_FIELD_LENGTH])138 x.comment =fp.read(centdir[_CD_COMMENT_LENGTH])139 x.header_offset =centdir[_CD_LOCAL_HEADER_OFFSET]140 (x.create_version, x.create_system, x.extract_version, x.reserved,141 x.flag_bits, x.compress_type, t, d,142 x.CRC, x.compress_size, x.file_size) = centdir[1:12]143 x.volume, x.internal_attr, x.external_attr = centdir[15:18]144 #Convert date/time code to (year, month, day, hour, min, sec) 145 x._raw_time =t146 x.date_time = ( (d>>9)+1980, (d>>5)&0xF, d&0x1F,147 t>>11, (t>>5)&0x3F, (t&0x1F) * 2)148 149 x._decodeExtra()150 x.header_offset = x.header_offset +concat151 x.filename =x._decodeFilename()152 self.filelist.append(x)153 self.NameToInfo[x.filename] =x154 155 #update total bytes read from central directory 156 total = (total + sizeCentralDir +centdir[_CD_FILENAME_LENGTH]157 +centdir[_CD_EXTRA_FIELD_LENGTH]158 +centdir[_CD_COMMENT_LENGTH])159 160 if self.debug > 2:161 print "total", total162 163 164 defnamelist(self):165 """Return a list of file names in the archive.""" 166 l =[]167 for data inself.filelist:168 l.append(data.filename)169 returnl170 171 definfolist(self):172 """Return a list of class ZipInfo instances for files in the173 archive.""" 174 returnself.filelist175 176 defprintdir(self):177 """Print a table of contents for the zip file.""" 178 print "%-46s %19s %12s" % ("File Name", "Modified", "Size")179 for zinfo inself.filelist:180 date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]181 print "%-46s %s %12d" %(zinfo.filename, date, zinfo.file_size)182 183 deftestzip(self):184 """Read all the files and check the CRC.""" 185 chunk_size = 2 ** 20 186 for zinfo inself.filelist:187 try:188 #Read by chunks, to avoid an OverflowError or a 189 #MemoryError with very large embedded files. 190 with self.open(zinfo.filename, "r") as f:191 while f.read(chunk_size): #Check CRC-32 192 pass 193 exceptBadZipfile:194 returnzinfo.filename195 196 defgetinfo(self, name):197 """Return the instance of ZipInfo given 'name'.""" 198 info =self.NameToInfo.get(name)199 if info isNone:200 raiseKeyError(201 'There is no item named %r in the archive' %name)202 203 returninfo204 205 defsetpassword(self, pwd):206 """Set default password for encrypted files.""" 207 self.pwd =pwd208 209 @property210 defcomment(self):211 """The comment text associated with the ZIP file.""" 212 returnself._comment213 214 @comment.setter215 defcomment(self, comment):216 #check for valid comment length 217 if len(comment) >ZIP_MAX_COMMENT:218 importwarnings219 warnings.warn('Archive comment is too long; truncating to %d bytes' 220 % ZIP_MAX_COMMENT, stacklevel=2)221 comment =comment[:ZIP_MAX_COMMENT]222 self._comment =comment223 self._didModify =True224 225 def read(self, name, pwd=None):226 """Return file bytes (as a string) for name.""" 227 return self.open(name, "r", pwd).read()228 229 def open(self, name, mode="r", pwd=None):230 """Return file-like object for 'name'.""" 231 if mode not in ("r", "U", "rU"):232 raise RuntimeError, 'open() requires mode "r", "U", or "rU"' 233 if notself.fp:234 raiseRuntimeError, \235 "Attempt to read ZIP archive that was already closed" 236 237 #Only open a new file for instances where we were not 238 #given a file object in the constructor 239 ifself._filePassed:240 zef_file =self.fp241 should_close =False242 else:243 zef_file = open(self.filename, 'rb')244 should_close =True245 246 try:247 #Make sure we have an info object 248 ifisinstance(name, ZipInfo):249 #'name' is already an info object 250 zinfo =name251 else:252 #Get info object for name 253 zinfo =self.getinfo(name)254 255 zef_file.seek(zinfo.header_offset, 0)256 257 #Skip the file header: 258 fheader =zef_file.read(sizeFileHeader)259 if len(fheader) !=sizeFileHeader:260 raise BadZipfile("Truncated file header")261 fheader =struct.unpack(structFileHeader, fheader)262 if fheader[_FH_SIGNATURE] !=stringFileHeader:263 raise BadZipfile("Bad magic number for file header")264 265 fname =zef_file.read(fheader[_FH_FILENAME_LENGTH])266 iffheader[_FH_EXTRA_FIELD_LENGTH]:267 zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH])268 269 if fname !=zinfo.orig_filename:270 raiseBadZipfile, \271 'File name in directory "%s" and header "%s" differ.' %(272 zinfo.orig_filename, fname)273 274 #check for encrypted flag & handle password 275 is_encrypted = zinfo.flag_bits & 0x1 276 zd =None277 ifis_encrypted:278 if notpwd:279 pwd =self.pwd280 if notpwd:281 raise RuntimeError, "File %s is encrypted,"\282 "password required for extraction" %name283 284 zd =_ZipDecrypter(pwd)285 #The first 12 bytes in the cypher stream is an encryption header 286 #used to strengthen the algorithm. The first 11 bytes are 287 #completely random, while the 12th contains the MSB of the CRC, 288 #or the MSB of the file time depending on the header type 289 #and is used to check the correctness of the password. 290 bytes = zef_file.read(12)291 h = map(zd, bytes[0:12])292 if zinfo.flag_bits & 0x8:293 #compare against the file type from extended local headers 294 check_byte = (zinfo._raw_time >> 8) & 0xff 295 else:296 #compare against the CRC otherwise 297 check_byte = (zinfo.CRC >> 24) & 0xff 298 if ord(h[11]) !=check_byte:299 raise RuntimeError("Bad password for file", name)300 301 returnZipExtFile(zef_file, mode, zinfo, zd,302 close_fileobj=should_close)303 except:304 ifshould_close:305 zef_file.close()306 raise 307 308 def extract(self, member, path=None, pwd=None):309 """Extract a member from the archive to the current working directory,310 using its full name. Its file information is extracted as accurately311 as possible. `member' may be a filename or a ZipInfo object. You can312 specify a different directory using `path'.313 """ 314 if notisinstance(member, ZipInfo):315 member =self.getinfo(member)316 317 if path isNone:318 path =os.getcwd()319 320 returnself._extract_member(member, path, pwd)321 322 def extractall(self, path=None, members=None, pwd=None):323 """Extract all members from the archive to the current working324 directory. `path' specifies a different directory to extract to.325 `members' is optional and must be a subset of the list returned326 by namelist().327 """ 328 if members isNone:329 members =self.namelist()330 331 for zipinfo inmembers:332 self.extract(zipinfo, path, pwd)333 334 def_extract_member(self, member, targetpath, pwd):335 """Extract the ZipInfo object 'member' to a physical336 file on the path targetpath.337 """ 338 #build the destination pathname, replacing 339 #forward slashes to platform specific separators. 340 arcname = member.filename.replace('/', os.path.sep)341 342 ifos.path.altsep:343 arcname =arcname.replace(os.path.altsep, os.path.sep)344 #interpret absolute pathname as relative, remove drive letter or 345 #UNC path, redundant separators, "." and ".." components. 346 arcname = os.path.splitdrive(arcname)[1]347 arcname = os.path.sep.join(x for x inarcname.split(os.path.sep)348 if x not in ('', os.path.curdir, os.path.pardir))349 if os.path.sep == '\\':350 #filter illegal characters on Windows 351 illegal = ':<>|"?*' 352 ifisinstance(arcname, unicode):353 table = {ord(c): ord('_') for c inillegal}354 else:355 table = string.maketrans(illegal, '_' *len(illegal))356 arcname =arcname.translate(table)357 #remove trailing dots 358 arcname = (x.rstrip('.') for x inarcname.split(os.path.sep))359 arcname = os.path.sep.join(x for x in arcname ifx)360 361 targetpath =os.path.join(targetpath, arcname)362 targetpath =os.path.normpath(targetpath)363 364 #Create all upper directories if necessary. 365 upperdirs =os.path.dirname(targetpath)366 if upperdirs and notos.path.exists(upperdirs):367 os.makedirs(upperdirs)368 369 if member.filename[-1] == '/':370 if notos.path.isdir(targetpath):371 os.mkdir(targetpath)372 returntargetpath373 374 with self.open(member, pwd=pwd) as source, \375 file(targetpath, "wb") as target:376 shutil.copyfileobj(source, target)377 378 returntargetpath379 380 def_writecheck(self, zinfo):381 """Check for errors before writing a file to the archive.""" 382 if zinfo.filename inself.NameToInfo:383 importwarnings384 warnings.warn('Duplicate name: %r' % zinfo.filename, stacklevel=3)385 if self.mode not in ("w", "a"):386 raise RuntimeError, 'write() requires mode "w" or "a"' 387 if notself.fp:388 raiseRuntimeError, \389 "Attempt to write ZIP archive that was already closed" 390 if zinfo.compress_type == ZIP_DEFLATED and notzlib:391 raiseRuntimeError, \392 "Compression requires the (missing) zlib module" 393 if zinfo.compress_type not in(ZIP_STORED, ZIP_DEFLATED):394 raiseRuntimeError, \395 "That compression method is not supported" 396 if notself._allowZip64:397 requires_zip64 =None398 if len(self.filelist) >=ZIP_FILECOUNT_LIMIT:399 requires_zip64 = "Files count" 400 elif zinfo.file_size >ZIP64_LIMIT:401 requires_zip64 = "Filesize" 402 elif zinfo.header_offset >ZIP64_LIMIT:403 requires_zip64 = "Zipfile size" 404 ifrequires_zip64:405 raise LargeZipFile(requires_zip64 + 406 "would require ZIP64 extensions")407 408 def write(self, filename, arcname=None, compress_type=None):409 """Put the bytes from filename into the archive under the name410 arcname.""" 411 if notself.fp:412 raiseRuntimeError(413 "Attempt to write to ZIP archive that was already closed")414 415 st =os.stat(filename)416 isdir =stat.S_ISDIR(st.st_mode)417 mtime =time.localtime(st.st_mtime)418 date_time = mtime[0:6]419 #Create ZipInfo instance to store file information 420 if arcname isNone:421 arcname =filename422 arcname = os.path.normpath(os.path.splitdrive(arcname)[1])423 while arcname[0] in(os.sep, os.altsep):424 arcname = arcname[1:]425 ifisdir:426 arcname += '/' 427 zinfo =ZipInfo(arcname, date_time)428 zinfo.external_attr = (st[0] & 0xFFFF) << 16L #Unix attributes 429 if compress_type isNone:430 zinfo.compress_type =self.compression431 else:432 zinfo.compress_type =compress_type433 434 zinfo.file_size =st.st_size435 zinfo.flag_bits = 0x00 436 zinfo.header_offset = self.fp.tell() #Start of header bytes 437 438 self._writecheck(zinfo)439 self._didModify =True440 441 ifisdir:442 zinfo.file_size =0443 zinfo.compress_size =0444 zinfo.CRC =0445 zinfo.external_attr |= 0x10 #MS-DOS directory flag 446 self.filelist.append(zinfo)447 self.NameToInfo[zinfo.filename] =zinfo448 self.fp.write(zinfo.FileHeader(False))449 return 450 451 with open(filename, "rb") as fp:452 #Must overwrite CRC and sizes with correct data later 453 zinfo.CRC = CRC =0454 zinfo.compress_size = compress_size =0455 #Compressed size can be larger than uncompressed size 456 zip64 = self._allowZip64 and\457 zinfo.file_size * 1.05 >ZIP64_LIMIT458 self.fp.write(zinfo.FileHeader(zip64))459 if zinfo.compress_type ==ZIP_DEFLATED:460 cmpr =zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,461 zlib.DEFLATED, -15)462 else:463 cmpr =None464 file_size =0465 while 1:466 buf = fp.read(1024 * 8)467 if notbuf:468 break 469 file_size = file_size +len(buf)470 CRC = crc32(buf, CRC) & 0xffffffff 471 ifcmpr:472 buf =cmpr.compress(buf)473 compress_size = compress_size +len(buf)474 self.fp.write(buf)475 ifcmpr:476 buf =cmpr.flush()477 compress_size = compress_size +len(buf)478 self.fp.write(buf)479 zinfo.compress_size =compress_size480 else:481 zinfo.compress_size =file_size482 zinfo.CRC =CRC483 zinfo.file_size =file_size484 if not zip64 andself._allowZip64:485 if file_size >ZIP64_LIMIT:486 raise RuntimeError('File size has increased during compressing')487 if compress_size >ZIP64_LIMIT:488 raise RuntimeError('Compressed size larger than uncompressed size')489 #Seek backwards and write file header (which will now include 490 #correct CRC and file sizes) 491 position = self.fp.tell() #Preserve current position in file 492 self.fp.seek(zinfo.header_offset, 0)493 self.fp.write(zinfo.FileHeader(zip64))494 self.fp.seek(position, 0)495 self.filelist.append(zinfo)496 self.NameToInfo[zinfo.filename] =zinfo497 498 def writestr(self, zinfo_or_arcname, bytes, compress_type=None):499 """Write a file into the archive. The contents is the string500 'bytes'. 'zinfo_or_arcname' is either a ZipInfo instance or501 the name of the file in the archive.""" 502 if notisinstance(zinfo_or_arcname, ZipInfo):503 zinfo = ZipInfo(filename=zinfo_or_arcname,504 date_time=time.localtime(time.time())[:6])505 506 zinfo.compress_type =self.compression507 if zinfo.filename[-1] == '/':508 zinfo.external_attr = 0o40775 << 16 #drwxrwxr-x 509 zinfo.external_attr |= 0x10 #MS-DOS directory flag 510 else:511 zinfo.external_attr = 0o600 << 16 #?rw------- 512 else:513 zinfo =zinfo_or_arcname514 515 if notself.fp:516 raiseRuntimeError(517 "Attempt to write to ZIP archive that was already closed")518 519 if compress_type is notNone:520 zinfo.compress_type =compress_type521 522 zinfo.file_size = len(bytes) #Uncompressed size 523 zinfo.header_offset = self.fp.tell() #Start of header bytes 524 self._writecheck(zinfo)525 self._didModify =True526 zinfo.CRC = crc32(bytes) & 0xffffffff #CRC-32 checksum 527 if zinfo.compress_type ==ZIP_DEFLATED:528 co =zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,529 zlib.DEFLATED, -15)530 bytes = co.compress(bytes) +co.flush()531 zinfo.compress_size = len(bytes) #Compressed size 532 else:533 zinfo.compress_size =zinfo.file_size534 zip64 = zinfo.file_size > ZIP64_LIMIT or\535 zinfo.compress_size >ZIP64_LIMIT536 if zip64 and notself._allowZip64:537 raise LargeZipFile("Filesize would require ZIP64 extensions")538 self.fp.write(zinfo.FileHeader(zip64))539 self.fp.write(bytes)540 if zinfo.flag_bits & 0x08:541 #Write CRC and file sizes after the file data 542 fmt = '<LQQ' if zip64 else '<LLL' 543 self.fp.write(struct.pack(fmt, zinfo.CRC, zinfo.compress_size,544 zinfo.file_size))545 self.fp.flush()546 self.filelist.append(zinfo)547 self.NameToInfo[zinfo.filename] =zinfo548 549 def __del__(self):550 """Call the "close()" method in case the user forgot.""" 551 self.close()552 553 defclose(self):554 """Close the file, and for mode "w" and "a" write the ending555 records.""" 556 if self.fp isNone:557 return 558 559 try:560 if self.mode in ("w", "a") and self._didModify: #write ending records 561 pos1 =self.fp.tell()562 for zinfo in self.filelist: #write central directory 563 dt =zinfo.date_time564 dosdate = (dt[0] - 1980) << 9 | dt[1] << 5 | dt[2]565 dostime = dt[3] << 11 | dt[4] << 5 | (dt[5] // 2)566 extra =[]567 if zinfo.file_size >ZIP64_LIMIT \568 or zinfo.compress_size >ZIP64_LIMIT:569 extra.append(zinfo.file_size)570 extra.append(zinfo.compress_size)571 file_size = 0xffffffff 572 compress_size = 0xffffffff 573 else:574 file_size =zinfo.file_size575 compress_size =zinfo.compress_size576 577 if zinfo.header_offset >ZIP64_LIMIT:578 extra.append(zinfo.header_offset)579 header_offset = 0xffffffffL 580 else:581 header_offset =zinfo.header_offset582 583 extra_data =zinfo.extra584 ifextra:585 #Append a ZIP64 field to the extra's 586 extra_data =struct.pack(587 '<HH' + 'Q'*len(extra),588 1, 8*len(extra), *extra) +extra_data589 590 extract_version = max(45, zinfo.extract_version)591 create_version = max(45, zinfo.create_version)592 else:593 extract_version =zinfo.extract_version594 create_version =zinfo.create_version595 596 try:597 filename, flag_bits =zinfo._encodeFilenameFlags()598 centdir =struct.pack(structCentralDir,599 stringCentralDir, create_version,600 zinfo.create_system, extract_version, zinfo.reserved,601 flag_bits, zinfo.compress_type, dostime, dosdate,602 zinfo.CRC, compress_size, file_size,603 len(filename), len(extra_data), len(zinfo.comment),604 0, zinfo.internal_attr, zinfo.external_attr,605 header_offset)606 exceptDeprecationWarning:607 print >>sys.stderr, (structCentralDir,608 stringCentralDir, create_version,609 zinfo.create_system, extract_version, zinfo.reserved,610 zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,611 zinfo.CRC, compress_size, file_size,612 len(zinfo.filename), len(extra_data), len(zinfo.comment),613 0, zinfo.internal_attr, zinfo.external_attr,614 header_offset)615 raise 616 self.fp.write(centdir)617 self.fp.write(filename)618 self.fp.write(extra_data)619 self.fp.write(zinfo.comment)620 621 pos2 =self.fp.tell()622 #Write end-of-zip-archive record 623 centDirCount =len(self.filelist)624 centDirSize = pos2 -pos1625 centDirOffset =pos1626 requires_zip64 =None627 if centDirCount >ZIP_FILECOUNT_LIMIT:628 requires_zip64 = "Files count" 629 elif centDirOffset >ZIP64_LIMIT:630 requires_zip64 = "Central directory offset" 631 elif centDirSize >ZIP64_LIMIT:632 requires_zip64 = "Central directory size" 633 ifrequires_zip64:634 #Need to write the ZIP64 end-of-archive records 635 if notself._allowZip64:636 raise LargeZipFile(requires_zip64 + 637 "would require ZIP64 extensions")638 zip64endrec =struct.pack(639 structEndArchive64, stringEndArchive64,640 44, 45, 45, 0, 0, centDirCount, centDirCount,641 centDirSize, centDirOffset)642 self.fp.write(zip64endrec)643 644 zip64locrec =struct.pack(645 structEndArchive64Locator,646 stringEndArchive64Locator, 0, pos2, 1)647 self.fp.write(zip64locrec)648 centDirCount = min(centDirCount, 0xFFFF)649 centDirSize = min(centDirSize, 0xFFFFFFFF)650 centDirOffset = min(centDirOffset, 0xFFFFFFFF)651 652 endrec =struct.pack(structEndArchive, stringEndArchive,653 0, 0, centDirCount, centDirCount,654 centDirSize, centDirOffset, len(self._comment))655 self.fp.write(endrec)656 self.fp.write(self._comment)657 self.fp.flush()658 finally:659 fp =self.fp660 self.fp =None661 if notself._filePassed:662 fp.close()663 664 ZipFile
ZipFile
1 classTarFile(object):2 """The TarFile Class provides an interface to tar archives.3 """ 4 5 debug = 0 #May be set from 0 (no msgs) to 3 (all msgs) 6 7 dereference = False #If true, add content of linked file to the 8 #tar file, else the link. 9 10 ignore_zeros = False #If true, skips empty or invalid blocks and 11 #continues processing. 12 13 errorlevel = 1 #If 0, fatal errors only appear in debug 14 #messages (if debug >= 0). If > 0, errors 15 #are passed to the caller as exceptions. 16 17 format = DEFAULT_FORMAT #The format to use when creating an archive. 18 19 encoding = ENCODING #Encoding for 8-bit character strings. 20 21 errors = None #Error handler for unicode conversion. 22 23 tarinfo = TarInfo #The default TarInfo class to use. 24 25 fileobject = ExFileObject #The default ExFileObject class to use. 26 27 def __init__(self, name=None, mode="r", fileobj=None, format=None,28 tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,29 errors=None, pax_headers=None, debug=None, errorlevel=None):30 """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to31 read from an existing archive, 'a' to append data to an existing32 file or 'w' to create a new file overwriting an existing one. `mode'33 defaults to 'r'.34 If `fileobj' is given, it is used for reading or writing data. If it35 can be determined, `mode' is overridden by `fileobj's mode.36 `fileobj' is not closed, when TarFile is closed.37 """ 38 modes = {"r": "rb", "a": "r+b", "w": "wb"}39 if mode not inmodes:40 raise ValueError("mode must be 'r', 'a' or 'w'")41 self.mode =mode42 self._mode =modes[mode]43 44 if notfileobj:45 if self.mode == "a" and notos.path.exists(name):46 #Create nonexistent files in append mode. 47 self.mode = "w" 48 self._mode = "wb" 49 fileobj =bltn_open(name, self._mode)50 self._extfileobj =False51 else:52 if name is None and hasattr(fileobj, "name"):53 name =fileobj.name54 if hasattr(fileobj, "mode"):55 self._mode =fileobj.mode56 self._extfileobj =True57 self.name = os.path.abspath(name) if name elseNone58 self.fileobj =fileobj59 60 #Init attributes. 61 if format is notNone:62 self.format =format63 if tarinfo is notNone:64 self.tarinfo =tarinfo65 if dereference is notNone:66 self.dereference =dereference67 if ignore_zeros is notNone:68 self.ignore_zeros =ignore_zeros69 if encoding is notNone:70 self.encoding =encoding71 72 if errors is notNone:73 self.errors =errors74 elif mode == "r":75 self.errors = "utf-8" 76 else:77 self.errors = "strict" 78 79 if pax_headers is not None and self.format ==PAX_FORMAT:80 self.pax_headers =pax_headers81 else:82 self.pax_headers ={}83 84 if debug is notNone:85 self.debug =debug86 if errorlevel is notNone:87 self.errorlevel =errorlevel88 89 #Init datastructures. 90 self.closed =False91 self.members = [] #list of members as TarInfo objects 92 self._loaded = False #flag if all members have been read 93 self.offset =self.fileobj.tell()94 #current position in the archive file 95 self.inodes = {} #dictionary caching the inodes of 96 #archive members already added 97 98 try:99 if self.mode == "r":100 self.firstmember =None101 self.firstmember =self.next()102 103 if self.mode == "a":104 #Move to the end of the archive, 105 #before the first empty block. 106 whileTrue:107 self.fileobj.seek(self.offset)108 try:109 tarinfo =self.tarinfo.fromtarfile(self)110 self.members.append(tarinfo)111 exceptEOFHeaderError:112 self.fileobj.seek(self.offset)113 break 114 exceptHeaderError, e:115 raiseReadError(str(e))116 117 if self.mode in "aw":118 self._loaded =True119 120 ifself.pax_headers:121 buf =self.tarinfo.create_pax_global_header(self.pax_headers.copy())122 self.fileobj.write(buf)123 self.offset +=len(buf)124 except:125 if notself._extfileobj:126 self.fileobj.close()127 self.closed =True128 raise 129 130 def_getposix(self):131 return self.format ==USTAR_FORMAT132 def_setposix(self, value):133 importwarnings134 warnings.warn("use the format attribute instead", DeprecationWarning,135 2)136 ifvalue:137 self.format =USTAR_FORMAT138 else:139 self.format =GNU_FORMAT140 posix =property(_getposix, _setposix)141 142 #-------------------------------------------------------------------------- 143 #Below are the classmethods which act as alternate constructors to the 144 #TarFile class. The open() method is the only one that is needed for 145 #public use; it is the "super"-constructor and is able to select an 146 #adequate "sub"-constructor for a particular compression using the mapping 147 #from OPEN_METH. 148 #149 #This concept allows one to subclass TarFile without losing the comfort of 150 #the super-constructor. A sub-constructor is registered and made available 151 #by adding it to the mapping in OPEN_METH. 152 153 @classmethod154 def open(cls, name=None, mode="r", fileobj=None, bufsize=RECORDSIZE, **kwargs):155 """Open a tar archive for reading, writing or appending. Return156 an appropriate TarFile class.157 158 mode:159 'r' or 'r:*' open for reading with transparent compression160 'r:' open for reading exclusively uncompressed161 'r:gz' open for reading with gzip compression162 'r:bz2' open for reading with bzip2 compression163 'a' or 'a:' open for appending, creating the file if necessary164 'w' or 'w:' open for writing without compression165 'w:gz' open for writing with gzip compression166 'w:bz2' open for writing with bzip2 compression167 168 'r|*' open a stream of tar blocks with transparent compression169 'r|' open an uncompressed stream of tar blocks for reading170 'r|gz' open a gzip compressed stream of tar blocks171 'r|bz2' open a bzip2 compressed stream of tar blocks172 'w|' open an uncompressed stream for writing173 'w|gz' open a gzip compressed stream for writing174 'w|bz2' open a bzip2 compressed stream for writing175 """ 176 177 if not name and notfileobj:178 raise ValueError("nothing to open")179 180 if mode in ("r", "r:*"):181 #Find out which *open() is appropriate for opening the file. 182 for comptype incls.OPEN_METH:183 func =getattr(cls, cls.OPEN_METH[comptype])184 if fileobj is notNone:185 saved_pos =fileobj.tell()186 try:187 return func(name, "r", fileobj, **kwargs)188 except(ReadError, CompressionError), e:189 if fileobj is notNone:190 fileobj.seek(saved_pos)191 continue 192 raise ReadError("file could not be opened successfully")193 194 elif ":" inmode:195 filemode, comptype = mode.split(":", 1)196 filemode = filemode or "r" 197 comptype = comptype or "tar" 198 199 #Select the *open() function according to 200 #given compression. 201 if comptype incls.OPEN_METH:202 func =getattr(cls, cls.OPEN_METH[comptype])203 else:204 raise CompressionError("unknown compression type %r" %comptype)205 return func(name, filemode, fileobj, **kwargs)206 207 elif "|" inmode:208 filemode, comptype = mode.split("|", 1)209 filemode = filemode or "r" 210 comptype = comptype or "tar" 211 212 if filemode not in ("r", "w"):213 raise ValueError("mode must be 'r' or 'w'")214 215 stream =_Stream(name, filemode, comptype, fileobj, bufsize)216 try:217 t = cls(name, filemode, stream, **kwargs)218 except:219 stream.close()220 raise 221 t._extfileobj =False222 returnt223 224 elif mode in ("a", "w"):225 return cls.taropen(name, mode, fileobj, **kwargs)226 227 raise ValueError("undiscernible mode")228 229 @classmethod230 def taropen(cls, name, mode="r", fileobj=None, **kwargs):231 """Open uncompressed tar archive name for reading or writing.232 """ 233 if mode not in ("r", "a", "w"):234 raise ValueError("mode must be 'r', 'a' or 'w'")235 return cls(name, mode, fileobj, **kwargs)236 237 @classmethod238 def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):239 """Open gzip compressed tar archive name for reading or writing.240 Appending is not allowed.241 """ 242 if mode not in ("r", "w"):243 raise ValueError("mode must be 'r' or 'w'")244 245 try:246 importgzip247 gzip.GzipFile248 except(ImportError, AttributeError):249 raise CompressionError("gzip module is not available")250 251 try:252 fileobj =gzip.GzipFile(name, mode, compresslevel, fileobj)253 exceptOSError:254 if fileobj is not None and mode == 'r':255 raise ReadError("not a gzip file")256 raise 257 258 try:259 t = cls.taropen(name, mode, fileobj, **kwargs)260 exceptIOError:261 fileobj.close()262 if mode == 'r':263 raise ReadError("not a gzip file")264 raise 265 except:266 fileobj.close()267 raise 268 t._extfileobj =False269 returnt270 271 @classmethod272 def bz2open(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):273 """Open bzip2 compressed tar archive name for reading or writing.274 Appending is not allowed.275 """ 276 if mode not in ("r", "w"):277 raise ValueError("mode must be 'r' or 'w'.")278 279 try:280 importbz2281 exceptImportError:282 raise CompressionError("bz2 module is not available")283 284 if fileobj is notNone:285 fileobj =_BZ2Proxy(fileobj, mode)286 else:287 fileobj = bz2.BZ2File(name, mode, compresslevel=compresslevel)288 289 try:290 t = cls.taropen(name, mode, fileobj, **kwargs)291 except(IOError, EOFError):292 fileobj.close()293 if mode == 'r':294 raise ReadError("not a bzip2 file")295 raise 296 except:297 fileobj.close()298 raise 299 t._extfileobj =False300 returnt301 302 #All *open() methods are registered here. 303 OPEN_METH ={304 "tar": "taropen", #uncompressed tar 305 "gz": "gzopen", #gzip compressed tar 306 "bz2": "bz2open" #bzip2 compressed tar 307 }308 309 #-------------------------------------------------------------------------- 310 #The public methods which TarFile provides: 311 312 defclose(self):313 """Close the TarFile. In write-mode, two finishing zero blocks are314 appended to the archive.315 """ 316 ifself.closed:317 return 318 319 if self.mode in "aw":320 self.fileobj.write(NUL * (BLOCKSIZE * 2))321 self.offset += (BLOCKSIZE * 2)322 #fill up the end with zero-blocks 323 #(like option -b20 for tar does) 324 blocks, remainder =divmod(self.offset, RECORDSIZE)325 if remainder >0:326 self.fileobj.write(NUL * (RECORDSIZE -remainder))327 328 if notself._extfileobj:329 self.fileobj.close()330 self.closed =True331 332 defgetmember(self, name):333 """Return a TarInfo object for member `name'. If `name' can not be334 found in the archive, KeyError is raised. If a member occurs more335 than once in the archive, its last occurrence is assumed to be the336 most up-to-date version.337 """ 338 tarinfo =self._getmember(name)339 if tarinfo isNone:340 raise KeyError("filename %r not found" %name)341 returntarinfo342 343 defgetmembers(self):344 """Return the members of the archive as a list of TarInfo objects. The345 list has the same order as the members in the archive.346 """ 347 self._check()348 if not self._loaded: #if we want to obtain a list of 349 self._load() #all members, we first have to 350 #scan the whole archive. 351 returnself.members352 353 defgetnames(self):354 """Return the members of the archive as a list of their names. It has355 the same order as the list returned by getmembers().356 """ 357 return [tarinfo.name for tarinfo inself.getmembers()]358 359 def gettarinfo(self, name=None, arcname=None, fileobj=None):360 """Create a TarInfo object for either the file `name' or the file361 object `fileobj' (using os.fstat on its file descriptor). You can362 modify some of the TarInfo's attributes before you add it using363 addfile(). If given, `arcname' specifies an alternative name for the364 file in the archive.365 """ 366 self._check("aw")367 368 #When fileobj is given, replace name by 369 #fileobj's real name. 370 if fileobj is notNone:371 name =fileobj.name372 373 #Building the name of the member in the archive. 374 #Backward slashes are converted to forward slashes, 375 #Absolute paths are turned to relative paths. 376 if arcname isNone:377 arcname =name378 drv, arcname =os.path.splitdrive(arcname)379 arcname = arcname.replace(os.sep, "/")380 arcname = arcname.lstrip("/")381 382 #Now, fill the TarInfo object with 383 #information specific for the file. 384 tarinfo =self.tarinfo()385 tarinfo.tarfile =self386 387 #Use os.stat or os.lstat, depending on platform 388 #and if symlinks shall be resolved. 389 if fileobj isNone:390 if hasattr(os, "lstat") and notself.dereference:391 statres =os.lstat(name)392 else:393 statres =os.stat(name)394 else:395 statres =os.fstat(fileobj.fileno())396 linkname = "" 397 398 stmd =statres.st_mode399 ifstat.S_ISREG(stmd):400 inode =(statres.st_ino, statres.st_dev)401 if not self.dereference and statres.st_nlink > 1 and\402 inode in self.inodes and arcname !=self.inodes[inode]:403 #Is it a hardlink to an already 404 #archived file? 405 type =LNKTYPE406 linkname =self.inodes[inode]407 else:408 #The inode is added only if its valid. 409 #For win32 it is always 0. 410 type =REGTYPE411 ifinode[0]:412 self.inodes[inode] =arcname413 elifstat.S_ISDIR(stmd):414 type =DIRTYPE415 elifstat.S_ISFIFO(stmd):416 type =FIFOTYPE417 elifstat.S_ISLNK(stmd):418 type =SYMTYPE419 linkname =os.readlink(name)420 elifstat.S_ISCHR(stmd):421 type =CHRTYPE422 elifstat.S_ISBLK(stmd):423 type =BLKTYPE424 else:425 returnNone426 427 #Fill the TarInfo object with all 428 #information we can get. 429 tarinfo.name =arcname430 tarinfo.mode =stmd431 tarinfo.uid =statres.st_uid432 tarinfo.gid =statres.st_gid433 if type ==REGTYPE:434 tarinfo.size =statres.st_size435 else:436 tarinfo.size =0L437 tarinfo.mtime =statres.st_mtime438 tarinfo.type =type439 tarinfo.linkname =linkname440 ifpwd:441 try:442 tarinfo.uname =pwd.getpwuid(tarinfo.uid)[0]443 exceptKeyError:444 pass 445 ifgrp:446 try:447 tarinfo.gname =grp.getgrgid(tarinfo.gid)[0]448 exceptKeyError:449 pass 450 451 if type in(CHRTYPE, BLKTYPE):452 if hasattr(os, "major") and hasattr(os, "minor"):453 tarinfo.devmajor =os.major(statres.st_rdev)454 tarinfo.devminor =os.minor(statres.st_rdev)455 returntarinfo456 457 def list(self, verbose=True):458 """Print a table of contents to sys.stdout. If `verbose' is False, only459 the names of the members are printed. If it is True, an `ls -l'-like460 output is produced.461 """ 462 self._check()463 464 for tarinfo inself:465 ifverbose:466 printfilemode(tarinfo.mode),467 print "%s/%s" % (tarinfo.uname ortarinfo.uid,468 tarinfo.gname ortarinfo.gid),469 if tarinfo.ischr() ortarinfo.isblk():470 print "%10s" % ("%d,%d"\471 %(tarinfo.devmajor, tarinfo.devminor)),472 else:473 print "%10d" %tarinfo.size,474 print "%d-%02d-%02d %02d:%02d:%02d"\475 % time.localtime(tarinfo.mtime)[:6],476 477 print tarinfo.name + ("/" if tarinfo.isdir() else ""),478 479 ifverbose:480 iftarinfo.issym():481 print "->", tarinfo.linkname,482 iftarinfo.islnk():483 print "link to", tarinfo.linkname,484 print 485 486 def add(self, name, arcname=None, recursive=True, exclude=None, filter=None):487 """Add the file `name' to the archive. `name' may be any type of file488 (directory, fifo, symbolic link, etc.). If given, `arcname'489 specifies an alternative name for the file in the archive.490 Directories are added recursively by default. This can be avoided by491 setting `recursive' to False. `exclude' is a function that should492 return True for each filename to be excluded. `filter' is a function493 that expects a TarInfo object argument and returns the changed494 TarInfo object, if it returns None the TarInfo object will be495 excluded from the archive.496 """ 497 self._check("aw")498 499 if arcname isNone:500 arcname =name501 502 #Exclude pathnames. 503 if exclude is notNone:504 importwarnings505 warnings.warn("use the filter argument instead",506 DeprecationWarning, 2)507 ifexclude(name):508 self._dbg(2, "tarfile: Excluded %r" %name)509 return 510 511 #Skip if somebody tries to archive the archive... 512 if self.name is not None and os.path.abspath(name) ==self.name:513 self._dbg(2, "tarfile: Skipped %r" %name)514 return 515 516 self._dbg(1, name)517 518 #Create a TarInfo object from the file. 519 tarinfo =self.gettarinfo(name, arcname)520 521 if tarinfo isNone:522 self._dbg(1, "tarfile: Unsupported type %r" %name)523 return 524 525 #Change or exclude the TarInfo object. 526 if filter is notNone:527 tarinfo =filter(tarinfo)528 if tarinfo isNone:529 self._dbg(2, "tarfile: Excluded %r" %name)530 return 531 532 #Append the tar header and data to the archive. 533 iftarinfo.isreg():534 with bltn_open(name, "rb") as f:535 self.addfile(tarinfo, f)536 537 eliftarinfo.isdir():538 self.addfile(tarinfo)539 ifrecursive:540 for f inos.listdir(name):541 self.add(os.path.join(name, f), os.path.join(arcname, f),542 recursive, exclude, filter)543 544 else:545 self.addfile(tarinfo)546 547 def addfile(self, tarinfo, fileobj=None):548 """Add the TarInfo object `tarinfo' to the archive. If `fileobj' is549 given, tarinfo.size bytes are read from it and added to the archive.550 You can create TarInfo objects using gettarinfo().551 On Windows platforms, `fileobj' should always be opened with mode552 'rb' to avoid irritation about the file size.553 """ 554 self._check("aw")555 556 tarinfo =copy.copy(tarinfo)557 558 buf =tarinfo.tobuf(self.format, self.encoding, self.errors)559 self.fileobj.write(buf)560 self.offset +=len(buf)561 562 #If there's data to follow, append it. 563 if fileobj is notNone:564 copyfileobj(fileobj, self.fileobj, tarinfo.size)565 blocks, remainder =divmod(tarinfo.size, BLOCKSIZE)566 if remainder >0:567 self.fileobj.write(NUL * (BLOCKSIZE -remainder))568 blocks += 1 569 self.offset += blocks *BLOCKSIZE570 571 self.members.append(tarinfo)572 573 def extractall(self, path=".", members=None):574 """Extract all members from the archive to the current working575 directory and set owner, modification time and permissions on576 directories afterwards. `path' specifies a different directory577 to extract to. `members' is optional and must be a subset of the578 list returned by getmembers().579 """ 580 directories =[]581 582 if members isNone:583 members =self584 585 for tarinfo inmembers:586 iftarinfo.isdir():587 #Extract directories with a safe mode. 588 directories.append(tarinfo)589 tarinfo =copy.copy(tarinfo)590 tarinfo.mode = 0700 591 self.extract(tarinfo, path)592 593 #Reverse sort directories. 594 directories.sort(key=operator.attrgetter('name'))595 directories.reverse()596 597 #Set correct owner, mtime and filemode on directories. 598 for tarinfo indirectories:599 dirpath =os.path.join(path, tarinfo.name)600 try:601 self.chown(tarinfo, dirpath)602 self.utime(tarinfo, dirpath)603 self.chmod(tarinfo, dirpath)604 exceptExtractError, e:605 if self.errorlevel > 1:606 raise 607 else:608 self._dbg(1, "tarfile: %s" %e)609 610 def extract(self, member, path=""):611 """Extract a member from the archive to the current working directory,612 using its full name. Its file information is extracted as accurately613 as possible. `member' may be a filename or a TarInfo object. You can614 specify a different directory using `path'.615 """ 616 self._check("r")617 618 ifisinstance(member, basestring):619 tarinfo =self.getmember(member)620 else:621 tarinfo =member622 623 #Prepare the link target for makelink(). 624 iftarinfo.islnk():625 tarinfo._link_target =os.path.join(path, tarinfo.linkname)626 627 try:628 self._extract_member(tarinfo, os.path.join(path, tarinfo.name))629 exceptEnvironmentError, e:630 if self.errorlevel >0:631 raise 632 else:633 if e.filename isNone:634 self._dbg(1, "tarfile: %s" %e.strerror)635 else:636 self._dbg(1, "tarfile: %s %r" %(e.strerror, e.filename))637 exceptExtractError, e:638 if self.errorlevel > 1:639 raise 640 else:641 self._dbg(1, "tarfile: %s" %e)642 643 defextractfile(self, member):644 """Extract a member from the archive as a file object. `member' may be645 a filename or a TarInfo object. If `member' is a regular file, a646 file-like object is returned. If `member' is a link, a file-like647 object is constructed from the link's target. If `member' is none of648 the above, None is returned.649 The file-like object is read-only and provides the following650 methods: read(), readline(), readlines(), seek() and tell()651 """ 652 self._check("r")653 654 ifisinstance(member, basestring):655 tarinfo =self.getmember(member)656 else:657 tarinfo =member658 659 iftarinfo.isreg():660 returnself.fileobject(self, tarinfo)661 662 elif tarinfo.type not inSUPPORTED_TYPES:663 #If a member's type is unknown, it is treated as a 664 #regular file. 665 returnself.fileobject(self, tarinfo)666 667 elif tarinfo.islnk() ortarinfo.issym():668 ifisinstance(self.fileobj, _Stream):669 #A small but ugly workaround for the case that someone tries 670 #to extract a (sym)link as a file-object from a non-seekable 671 #stream of tar blocks. 672 raise StreamError("cannot extract (sym)link as file object")673 else:674 #A (sym)link's file object is its target's file object. 675 returnself.extractfile(self._find_link_target(tarinfo))676 else:677 #If there's no data associated with the member (directory, chrdev, 678 #blkdev, etc.), return None instead of a file object. 679 returnNone680 681 def_extract_member(self, tarinfo, targetpath):682 """Extract the TarInfo object tarinfo to a physical683 file called targetpath.684 """ 685 #Fetch the TarInfo object for the given name 686 #and build the destination pathname, replacing 687 #forward slashes to platform specific separators. 688 targetpath = targetpath.rstrip("/")689 targetpath = targetpath.replace("/", os.sep)690 691 #Create all upper directories. 692 upperdirs =os.path.dirname(targetpath)693 if upperdirs and notos.path.exists(upperdirs):694 #Create directories that are not part of the archive with 695 #default permissions. 696 os.makedirs(upperdirs)697 698 if tarinfo.islnk() ortarinfo.issym():699 self._dbg(1, "%s -> %s" %(tarinfo.name, tarinfo.linkname))700 else:701 self._dbg(1, tarinfo.name)702 703 iftarinfo.isreg():704 self.makefile(tarinfo, targetpath)705 eliftarinfo.isdir():706 self.makedir(tarinfo, targetpath)707 eliftarinfo.isfifo():708 self.makefifo(tarinfo, targetpath)709 elif tarinfo.ischr() ortarinfo.isblk():710 self.makedev(tarinfo, targetpath)711 elif tarinfo.islnk() ortarinfo.issym():712 self.makelink(tarinfo, targetpath)713 elif tarinfo.type not inSUPPORTED_TYPES:714 self.makeunknown(tarinfo, targetpath)715 else:716 self.makefile(tarinfo, targetpath)717 718 self.chown(tarinfo, targetpath)719 if nottarinfo.issym():720 self.chmod(tarinfo, targetpath)721 self.utime(tarinfo, targetpath)722 723 #-------------------------------------------------------------------------- 724 #Below are the different file methods. They are called via 725 #_extract_member() when extract() is called. They can be replaced in a 726 #subclass to implement other functionality. 727 728 defmakedir(self, tarinfo, targetpath):729 """Make a directory called targetpath.730 """ 731 try:732 #Use a safe mode for the directory, the real mode is set 733 #later in _extract_member(). 734 os.mkdir(targetpath, 0700)735 exceptEnvironmentError, e:736 if e.errno !=errno.EEXIST:737 raise 738 739 defmakefile(self, tarinfo, targetpath):740 """Make a file called targetpath.741 """ 742 source =self.extractfile(tarinfo)743 try:744 with bltn_open(targetpath, "wb") as target:745 copyfileobj(source, target)746 finally:747 source.close()748 749 defmakeunknown(self, tarinfo, targetpath):750 """Make a file from a TarInfo object with an unknown type751 at targetpath.752 """ 753 self.makefile(tarinfo, targetpath)754 self._dbg(1, "tarfile: Unknown file type %r,"\755 "extracted as regular file." %tarinfo.type)756 757 defmakefifo(self, tarinfo, targetpath):758 """Make a fifo called targetpath.759 """ 760 if hasattr(os, "mkfifo"):761 os.mkfifo(targetpath)762 else:763 raise ExtractError("fifo not supported by system")764 765 defmakedev(self, tarinfo, targetpath):766 """Make a character or block device called targetpath.767 """ 768 if not hasattr(os, "mknod") or not hasattr(os, "makedev"):769 raise ExtractError("special devices not supported by system")770 771 mode =tarinfo.mode772 iftarinfo.isblk():773 mode |=stat.S_IFBLK774 else:775 mode |=stat.S_IFCHR776 777 os.mknod(targetpath, mode,778 os.makedev(tarinfo.devmajor, tarinfo.devminor))779 780 defmakelink(self, tarinfo, targetpath):781 """Make a (symbolic) link called targetpath. If it cannot be created782 (platform limitation), we try to make a copy of the referenced file783 instead of a link.784 """ 785 if hasattr(os, "symlink") and hasattr(os, "link"):786 #For systems that support symbolic and hard links. 787 iftarinfo.issym():788 ifos.path.lexists(targetpath):789 os.unlink(targetpath)790 os.symlink(tarinfo.linkname, targetpath)791 else:792 #See extract(). 793 ifos.path.exists(tarinfo._link_target):794 ifos.path.lexists(targetpath):795 os.unlink(targetpath)796 os.link(tarinfo._link_target, targetpath)797 else:798 self._extract_member(self._find_link_target(tarinfo), targetpath)799 else:800 try:801 self._extract_member(self._find_link_target(tarinfo), targetpath)802 exceptKeyError:803 raise ExtractError("unable to resolve link inside archive")804 805 defchown(self, tarinfo, targetpath):806 """Set owner of targetpath according to tarinfo.807 """ 808 if pwd and hasattr(os, "geteuid") and os.geteuid() ==0:809 #We have to be root to do so. 810 try:811 g = grp.getgrnam(tarinfo.gname)[2]812 exceptKeyError:813 g =tarinfo.gid814 try:815 u = pwd.getpwnam(tarinfo.uname)[2]816 exceptKeyError:817 u =tarinfo.uid818 try:819 if tarinfo.issym() and hasattr(os, "lchown"):820 os.lchown(targetpath, u, g)821 else:822 if sys.platform != "os2emx":823 os.chown(targetpath, u, g)824 exceptEnvironmentError, e:825 raise ExtractError("could not change owner")826 827 defchmod(self, tarinfo, targetpath):828 """Set file permissions of targetpath according to tarinfo.829 """ 830 if hasattr(os, 'chmod'):831 try:832 os.chmod(targetpath, tarinfo.mode)833 exceptEnvironmentError, e:834 raise ExtractError("could not change mode")835 836 defutime(self, tarinfo, targetpath):837 """Set modification time of targetpath according to tarinfo.838 """ 839 if not hasattr(os, 'utime'):840 return 841 try:842 os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))843 exceptEnvironmentError, e:844 raise ExtractError("could not change modification time")845 846 #-------------------------------------------------------------------------- 847 defnext(self):848 """Return the next member of the archive as a TarInfo object, when849 TarFile is opened for reading. Return None if there is no more850 available.851 """ 852 self._check("ra")853 if self.firstmember is notNone:854 m =self.firstmember855 self.firstmember =None856 returnm857 858 #Read the next block. 859 self.fileobj.seek(self.offset)860 tarinfo =None861 whileTrue:862 try:863 tarinfo =self.tarinfo.fromtarfile(self)864 exceptEOFHeaderError, e:865 ifself.ignore_zeros:866 self._dbg(2, "0x%X: %s" %(self.offset, e))867 self.offset +=BLOCKSIZE868 continue 869 exceptInvalidHeaderError, e:870 ifself.ignore_zeros:871 self._dbg(2, "0x%X: %s" %(self.offset, e))872 self.offset +=BLOCKSIZE873 continue 874 elif self.offset ==0:875 raiseReadError(str(e))876 exceptEmptyHeaderError:877 if self.offset ==0:878 raise ReadError("empty file")879 exceptTruncatedHeaderError, e:880 if self.offset ==0:881 raiseReadError(str(e))882 exceptSubsequentHeaderError, e:883 raiseReadError(str(e))884 break 885 886 if tarinfo is notNone:887 self.members.append(tarinfo)888 else:889 self._loaded =True890 891 returntarinfo892 893 #-------------------------------------------------------------------------- 894 #Little helper methods: 895 896 def _getmember(self, name, tarinfo=None, normalize=False):897 """Find an archive member by name from bottom to top.898 If tarinfo is given, it is used as the starting point.899 """ 900 #Ensure that all members have been loaded. 901 members =self.getmembers()902 903 #Limit the member search list up to tarinfo. 904 if tarinfo is notNone:905 members =members[:members.index(tarinfo)]906 907 ifnormalize:908 name =os.path.normpath(name)909 910 for member inreversed(members):911 ifnormalize:912 member_name =os.path.normpath(member.name)913 else:914 member_name =member.name915 916 if name ==member_name:917 returnmember918 919 def_load(self):920 """Read through the entire archive file and look for readable921 members.922 """ 923 whileTrue:924 tarinfo =self.next()925 if tarinfo isNone:926 break 927 self._loaded =True928 929 def _check(self, mode=None):930 """Check if TarFile is still open, and if the operation's mode931 corresponds to TarFile's mode.932 """ 933 ifself.closed:934 raise IOError("%s is closed" % self.__class__.__name__)935 if mode is not None and self.mode not inmode:936 raise IOError("bad operation for mode %r" %self.mode)937 938 def_find_link_target(self, tarinfo):939 """Find the target member of a symlink or hardlink member in the940 archive.941 """ 942 iftarinfo.issym():943 #Always search the entire archive. 944 linkname = "/".join(filter(None, (os.path.dirname(tarinfo.name), tarinfo.linkname)))945 limit =None946 else:947 #Search the archive before the link, because a hard link is 948 #just a reference to an already archived file. 949 linkname =tarinfo.linkname950 limit =tarinfo951 952 member = self._getmember(linkname, tarinfo=limit, normalize=True)953 if member isNone:954 raise KeyError("linkname %r not found" %linkname)955 returnmember956 957 def __iter__(self):958 """Provide an iterator object.959 """ 960 ifself._loaded:961 returniter(self.members)962 else:963 returnTarIter(self)964 965 def_dbg(self, level, msg):966 """Write debugging output to sys.stderr.967 """ 968 if level <=self.debug:969 print >>sys.stderr, msg970 971 def __enter__(self):972 self._check()973 returnself974 975 def __exit__(self, type, value, traceback):976 if type isNone:977 self.close()978 else:979 #An exception occurred. We must not call close() because 980 #it would try to write end-of-archive blocks and padding. 981 if notself._extfileobj:982 self.fileobj.close()983 self.closed =True984 #class TarFile 985 986 TarFile
TarFile
json & pickle 模块
用于序列化的两个模块
- json,用于字符串 和 python数据类型间进行转换
- pickle,用于python特有的类型 和 python的数据类型间进行转换
Json模块提供了四个功能:dumps、dump、loads、load
pickle模块提供了四个功能:dumps、dump、loads、load
Python json 是所有的语言都支持的数据交换
Python pickle 是Python独有的只有Python可以使用的数据交换
本节作业
作业需求:
模拟实现一个ATM + 购物商城程序
- 额度 15000或自定义
- 实现购物商城,买东西加入 购物车,调用信用卡接口结账
- 可以提现,手续费5%
- 每月22号出账单,每月10号为还款日,过期未还,按欠款总额 万分之5 每日计息
- 支持多账户登录
- 支持账户间转账
- 记录每月日常消费流水
- 提供还款接口
- ATM记录操作日志
- 提供管理接口,包括添加账户、用户额度,冻结账户等。。。
转载于:https://www.cnblogs.com/nb-blog/p/5197924.html
python-Day5-深入正则表达式--冒泡排序-时间复杂度 --常用模块学习:自定义模块--random模块:随机验证码--time datetime模块...相关推荐
- python中datetime模块是以什么时间为基础_Python基础之datetime模块
Outline 构建时间对象实例 date实例的构造 time实例的构造 datetime实例的构造 timedelta对象的构造 tzinfo介绍 时间转换 时间对象转字符串 字符串转时间对象 时间 ...
- python 各种模块学习
from:https://blog.csdn.net/weiwangchao_/article/details/70570508 转载:.... Python的模块大全,很全,有详细介绍! 另外附Py ...
- Python 之路 Day5 - 常用模块学习
模块介绍 time &datetime模块 random os sys shutil json & picle shelve xml处理 yaml处理 configparser has ...
- Python中re(正则表达式)模块函数学习
2019独角兽企业重金招聘Python工程师标准>>> Python正则表达式指南 今天学习了Python中有关正则表达式的知识.关于正则表达式的语法,不作过多解释,网上有许多学习的 ...
- python中datetime模块常用方法_Python中datetime的使用和常用时间处理
datetime在python中比较常用,主要用来处理时间日期,使用前先倒入datetime模块.下面总结下本人想到的几个常用功能. 1.当前时间: >>> print dateti ...
- python中re_Python中re(正则表达式)模块学习
今天学习了Python中有关正则表达式的知识.关于正则表达式的语法,不作过多解释,网上有许多学习的资料.这里主要介绍Python中常用的正则表达式处理函数. re.match re.match 尝试从 ...
- python中re模块_Python中re(正则表达式)模块学习
今天学习了Python中有关正则表达式的知识.关于正则表达式的语法,不作过多解释,网上有许多学习的资料.这里主要介绍Python中常用的正则表达式处理函数. re.match re.match 尝试从 ...
- 想做Python开发,这8种常用Python模块,你必须得知道!
8种常用Python模块 前言 time模块 1.时间戳(timestamp) 2.格式化的时间字符串(Format String) 3.结构化的时间(struct time) datetime模块 ...
- 【正则表达式】正则表达式及python的re模块学习
以前经常听正则表达式,但自己从来没用过,这次刚好需要,就学习一下. 参考链接: https://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb ...
最新文章
- 一分钟了解负载均衡的一切
- iptables防DDOS***和CC***设置
- golang中base64编码_MySQL中如何将字符串转为base64编码?
- python压缩包怎么安装-Python学习笔记-打包发布Python模块或程序,安装包
- PHP 5.5.38 + mysql 5.0.11 + zabbix3.0 + nginx 安装
- 天鹅给癞蛤蟆的回信[转贴]
- 实现DDD领域驱动设计: Part 1
- 从Gradle自动将工件提升到Maven Central
- 纯java分布式内存数据库_最新Java岗面试清单:分布式+Dubbo+线程+Redis+数据库+JVM+并发...
- JS手动实现一个new操作符
- php如何修改二维数组的值,php如何改变二维数组的值
- 凉山州计算机等级考试时间,2020年四川凉山中考考试时间及科目安排(已公布)...
- oracle decode函数
- java线程池的好处_Java 线程池的使用好处
- HashMap源码分析(保姆式注解):三大方法(构造、Put、Remove) ;附带面试考点及博主免费答疑
- win10 cannot open clipboard 解决办法
- chm电子书字体大小的调整
- 聚沙成塔——VBA术语 (VBA Glossary)
- 40几岁读研究生计算机,年近四十岁,还有必要去考研和继续考博吗?不建议考全日制研究生...
- 客户端修改opc服务器的数据,客户端读取opc服务器数据
热门文章
- 高配游戏组装电脑配置清单表 2021 组装电脑配置推荐2021
- php薄饼,薄饼的做法大全
- amcharts图表使用总结
- 1994年的计算机安全法规,1994年2月18日,我国颁布了(),这是我国第一个计算机安全法规,是我国计算机安全工作的总体纲领。...
- Spring框架汇总(大学生的天堂 码农的圣殿)
- 外贸Newer必备外贸流程
- 【PMP】PMBOK 笔记 第12章 项目采购管理
- 国内疫情统计及预测分析系统
- c语言解矩阵方程ax=b,用初等变换的方法求解矩阵方程AX=B
- 高通820(msm8996)camera hal源码分析