BeautifulSoup用法详解

注：本文代码以网站http://www.pythonscraping.com/pages/page3.html为例

1.获取网页HTML内容，传到BeautifulSoup对象。

import requests
from bs4 import BeautifulSoup
url = 'http://www.pythonscraping.com/pages/page3.html'
response = requests.get(url)
soup = BeautifulSoup(response.text)


<html><head><style>img{width:75px;
}
table{width:50%;
}
td{margin:10px;padding:10px;
}
.wrapper{width:800px;
}
.excitingNote{font-style:italic;font-weight:bold;
}</style></head><body><div id="wrapper"><img src="../img/gifts/logo.jpg" style="float:left;"/><h1>Totally Normal Gifts</h1><div id="content">Here is a collection of totally normal, totally reasonable gifts that your friends are sure to love! Our collection is
hand-curated by well-paid, free-range Tibetan monks.<p>We haven't figured out how to make online shopping carts yet, but you can send us a check to:<br/>123 Main St.<br/>Abuja, Nigeria
We will then send your totally amazing gift, pronto! Please include an extra $5.00 for gift wrapping.</p></div><table id="giftList"><tr><th>Item Title</th><th>Description</th><th>Cost</th><th>Image</th></tr><tr class="gift" id="gift1"><td>Vegetable Basket</td><td>This vegetable basket is the perfect gift for your health conscious (or overweight) friends!<span class="excitingNote">Now with super-colorful bell peppers!</span></td><td>$15.00</td><td><img src="../img/gifts/img1.jpg"/></td></tr><tr class="gift" id="gift2"><td>Russian Nesting Dolls</td><td>Hand-painted by trained monkeys, these exquisite dolls are priceless! And by "priceless," we mean "extremely expensive"!<span class="excitingNote">8 entire dolls per set! Octuple the presents!</span></td><td>$10,000.52</td><td><img src="../img/gifts/img2.jpg"/></td></tr><tr class="gift" id="gift3"><td>Fish Painting</td><td>If something seems fishy about this painting, it's because it's a fish!<span class="excitingNote">Also hand-painted by trained monkeys!</span></td><td>$10,005.00</td><td><img src="../img/gifts/img3.jpg"/></td></tr><tr class="gift" id="gift4"><td>Dead Parrot</td><td>This is an ex-parrot!<span class="excitingNote">Or maybe he's only resting?</span></td><td>$0.50</td><td><img src="../img/gifts/img4.jpg"/></td></tr><tr class="gift" id="gift5"><td>Mystery Box</td><td>If you love suprises, this mystery box is for you! Do not place on light-colored surfaces. May cause oil staining.<span class="excitingNote">Keep your friends guessing!</span></td><td>$1.50</td><td><img src="../img/gifts/img6.jpg"/></td></tr></table><div id="footer">© Totally Normal Gifts, Inc.<br/>+234 (617) 863-0736</div></div></body>
</html>

2.提取指定的标签如 h1

print(soup.h1)

<h1>Totally Normal Gifts</h1>

.get_text()将所有标签清除，返回一个只包含文本的字符串：

print(soup.h1.get_text())

Totally Normal Gifts

3.find()和findAll()

BeautifulSoup中对两者的定义：

findAll(tag,attributes,recursive,text,limit,keywords)

find(tag,attributes,recursive,text,keywords)

----返回所有标题标签的列表

print(soup.findAll({'h1'}))

----attributes是一个用Python字典封装一个标签的若干属性和对应属性值，例如下面会返回tr标签中属性class为gift的内容：

print(soup.findAll('tr',{'class':'gift'}))

----参数recursive为布尔变量，为true则查找所有子标签，false则只查找文档的一级标签。

----参数text匹配标签的文本内容，如果我们想要知道网页中包含‘Totally Normal Gifts’内容的标签的数量，可以这样：

name = soup.findAll(text='Totally Normal Gifts')
print(name)
print(len(name))

['Totally Normal Gifts']
1

----参数limit=x表示你只对网页中获取的前x项感兴趣。limit=1时findAll相当于find

----参数keyword为冗余功能，此处不做介绍。

4.处理子标签以及其他后代标签(.children)

查找第一个tr标签的子标签：

name = soup.find('tr').children

5.处理兄弟标签(.next_siblings)(.previous_siblings)

6.父标签的处理(.parents)

BeautifulSoup用法详解相关推荐

Python BS4解析库用法详解
Python BS4解析库用法详解 Beautiful Soup 简称 BS4(其中 4 表示版本号)是一个 Python 第三方库,它可以从 HTML 或 XML 文档中快速地提取指定的数据.Bea ...
Python re模块用法详解
Python re模块用法详解在 Python 爬虫过程中,实现网页元素解析的方法有很多,正则解析只是其中之一,常见的还有 BeautifulSoup 和 lxml,它们都支持网页 HTML 元素的 ...
python argv 详解_Python3 sys.argv[ ]用法详解
sys.argv[]说白了就是一个从程序外部获取参数的桥梁,这个"外部"很关键,因为我们从外部取得的参数可以是多个,所以获得的是一个列表(list),也就是说sys.argv其实可 ...
oracle中的exists 和 not exists 用法详解
from:http://blog.sina.com.cn/s/blog_601d1ce30100cyrb.html oracle中的exists 和 not exists 用法详解 (2009-05- ...
ROW_NUMBER() OVER()函数用法详解（分组排序例子多）
ROW_NUMBER() OVER()函数用法详解 (分组排序例子多) https://blog.csdn.net/qq_25221835/article/details/82762416 post ...
python的继承用法_【后端开发】python中继承有什么用法？python继承的用法详解
本篇文章给大家带来的内容是关于python中继承有什么用法?python继承的用法详解,有一定的参考价值,有需要的朋友可以参考一下,希望对你有所帮助. 面向对象三大特征 1.封装:根据职责将属性和方法 ...
C++中substr()函数用法详解
C++中substr()函数用法详解原型: string substr (size_t pos = 0, size_t len = npos) const; 返回一个新构造的string对象,其值初 ...
php theme_path,PHP_Yii2主题(Theme)用法详解，本文实例讲述了Yii2主题(Theme) - phpStudy
Yii2主题(Theme)用法详解本文实例讲述了Yii2主题(Theme)用法.分享给大家供大家参考,具体如下: 首先看看主要的配置方式: 'components' => [ 'view' = ...
LayoutInflater的inflate函数用法详解
LayoutInflater的inflate函数用法详解 LayoutInflater作用是将layout的xml布局文件实例化为View类对象. 获取LayoutInflater的方法有如下三种: ...

BeautifulSoup用法详解

BeautifulSoup用法详解相关推荐

最新文章

热门文章