How To Make Money, Using Web Scraping

"""
文章核心思想就是利用现在的网页爬取技术在网络中寻找赚钱的机会，思路是好的，但是一目前的国内情况来看，一方面可能触碰到法律红线，另一方面可能是面临爬去成本与收益之比过低的问题，不过机会还是有，很多，的。
如果希望掌握爬取技术，可以文章后留言，楼主会非常乐意传给你爬取通用技术代码，同时也非常欢迎感兴趣的朋友一起技术交流
"""

I’m going to show you 3 unique ways to make money, using Web Scraping

Christopher Zita

Feb 10 · 7 min read

Did you know what your reading right now is data. It may just seem like a few words to you but on the back end everything you read online is data that can be taken, picked apart, and manipulated with. Simplified this is what a Web Scraper is. They go through the code that was created to make a website (HTML code) or database and take the data they want. Virtually any website can be scraped . Some sites do involve measures that stop these scrapers from taking their data, but if your good enough you can essentially scrape 99% of websites online.

If you didn’t know what a Web Scraper is, well now you have an idea and we can get to the point of why your reading this article. Money. Web Scraping can be a unique way to make money that isn’t as difficult as it sounds. In fact all the methods and examples I'm going to show you took less than 50 lines of code to make, and can be learned in only a couple of hours. So with that said let me show you...

3 ways to make Money Using Web Scraping

1. Creating Bots

A bot is just a technical term for a program that does a specific action. Depending what you make this action to be, you can sell it to those who don’t have the technical abilities to make it themselves.

To show how you can create a bot and sell it, I created an Airbnb bot. This bot allows the user to input a location and it will return all the houses that Airbnb offers at that location including the price, rating, number of guests allowed, bedrooms, beds, and baths. All of this being done by web scraping the data of each posting on the Airbnb website.

To demonstrate the bot in action I’m going to input a location. Lets say I want to search for Airbnb’s in Rome, Italy. I simply input Rome into the bot, and it returns 272 unique Airbnb’s within seconds in an organized excel spreadsheet.

It is now much easier to see all the houses/features and their comparisons to other postings. It is also much easier to filter through. I live in a family of 4 and if we were to go to Rome we would look for an Airbnb with at least 2 beds at a decent price. Now with this clean organized spreadsheet, excel makes it extremely easy to filter to match my needs. And out of 272 results 7 returned with my matching needs.

Within these 7 the one I would pick is the Vatican St.Peter Daniel, it has a very good rating and is cheapest out of the 7 with a cost of $61 per night. So after I pick the one I want, I would simply copy the link of the posting into a browser and book it then.

Looking for places to stay can be an extremly daunting task when going on vacation, I’m sure most of us have felt that at one time or another. Because of this there are those that are willing to pay just to make this process easier. With this bot I made the process easier. You just saw me book a room with all my matching needs at a good price within 5 minutes.

Trust me people are willing to pay to make their lives just a bit easier.

2. Reselling

One of the most common uses of web scraping, is getting prices off websites. There are those who create web scraping programs that run everyday and return the price of a specific product, and when the price drops to a certain amount the program will automatically buy the product before its sold out. Then since the demand for the product will be higher than the supply they resell the product at a higher price to make a profit. This is just one example of the many reselling tactics that web scrapers use.

Another one which I will show you an example of can save you a lot of money and make a lot for you too.

Every retail website has limited deals and sales, where they will display the original price and the sale price. But what they don’t do is show how much is actually discounted off the original price. For example if a watch originally costs $350 and the sale price is $300 you would think $50 off would be a lot of money but it’s actually only a 14.2% discount. Now if a T-shirt originally costs $50 and the sale price is $40, you might see $10 being not that much off the original price, but in fact the discount is larger than the watch at 20%. Therefore you can save/make money by buying the products with the highest discounted %.

Using Hudson's’ Bay, a department store that has numerous of sales on all kinds of brands, were going to use web scraping to get the original and sale price of all the products and find the product with the highest discount.

After scraping the website it returned over 900 products and as you can see there is only 1 product out of the 900 with over a 50% discount. That would be the Perry Ellis Solid Non-Iron Dress Shirt.

This sale price is only a limited time offer, so the price for this shirt will eventually go back up to around $90. So if I were to buy it now at $40 than sell it at $30 below its original at $60 when the limited sale ends, I would still make a profit of $20.

This is a method where if you find the right niche to do this is in, there is a potential to make a large amount of money.

3. Selling Data

There are millions of datasets online that are free and accessible to everyone. This data is often easily gathered and thereby offered to anyone who wants to use them. On the other hand some data is not as easy to get, and takes either time or a lot of work to put in a nice clean dataset. This has become the evolution of selling data. There are companies that focus on getting data that may be hard to obtain and structuring that data into a nice clean spreadsheet or dashboard that others can use at a certain cost.

BigDataBall is a sports data website that sells player logs, play-by-play data, and other stats at a price of $30 for a single seasons worth of data. The reason they can ask for this price is not because there the only ones that have this data, but there one of the only websites out there that offer this data in a very structured and clean dashboard that is easy to read.

Now what I’m going to do is get the same data as BigDataBall has for free and I’m going to put it into a structured dataset like the ones I did before.

Like I said before they aren’t the only ones with this type of data. Basketball-Reference.com has all the same data but its not structured meaning its data is all over the place and hard to read, and you simply cannot just download the dataset you want. This is where web scraping comes in. I’m going to web scrape the website of all the players logs for each game and put it into a structured dataset like BigDataBall.

Structured Dataset of all the NBA Player Logs

After web scraping Basketball-Reference.com we got over 16000 player logs for the season so far. You can see why this data in a nice clean format can be monetized, because no one in their right mind would manually get 16000 logs of data and put it into their own dataset. But with the help of web scraping we were able to get this data in a couple of minutes and save ourselves $30.

Needless to say though you can do what BigDataBall does. Find data that is hard to obtain manually, let your computer do the work, and than sell it to those interested in having that data in a structured dataset.

Conclusion

In the world where everyone wants to make money Web Scraping has become a very unique and new way to make money on the side. Where if you apply it to the right situations it can make you a ton of money, and it is easier to do than most people think. With that said if any of you are interested in how I made the programs for the examples above, send me a message and I’ll be happy to take you through the code and how I did it, and hopefully teach you a few things or too. Thanks for reading :)

Towards Data Science

Data Science | Analytics | Sports

如何利用网络技术赚钱的相关推荐

利用SEO技术赚钱经典案例分享
利用SEO技术赚钱相信很多SEOer都想过,平时上上班闲暇时间赚点外快,多爽.首先通过利用SEO技术来做一些项目,是可以赚钱的,而且渠道也是非常多的,但是想要马上实现SEO赚钱也是不易的,因为骗子太多 ...
如何在家利用网络副业赚钱，这三个网上副业，做好比主业更好
副业怎么赚钱?这三个副业项目可以在家里做,比主业好! 你知道我们做副业要做什么吗? 建议不要做体力相关的副业,很难获得太多的成长.即使你一辈子都在工作,你也有收入上限.如果你想做,你可以做一个能发展壮 ...
计算机及网络技术发展趋势,网络技术发展对计算机技术的影响
当前形势下,网络技术的不断发展对计算机和信息技术发展产生重要影响,计算机和信息技术借助网络技术发展实现更高标准的发展,对人们的生活方式和社会发展都产生重要影响.为了弄清网络技术发展对计算机及信息技术的 ...
对计算机技术的发展方向研究,网络技术发展对计算机技术的影响
当前形势下,网络技术的不断发展对计算机和信息技术发展产生重要影响,计算机和信息技术借助网络技术发展实现更高标准的发展,对人们的生活方式和社会发展都产生重要影响.为了弄清网络技术发展对计算机及信息技术的 ...
张大哥笔记-如何利用网络赚钱（20种网络赚钱方法推荐）
如何利用网络赚钱(20种网络赚钱方法推荐) 很多人都知道网络可以赚钱,但是如何利用网络赚钱,具体应如何赚钱,很多人就不太清楚了.下面就与大家探讨交流下网络赚钱的方法: 1. 做网站,搞流量,赚广告费 ...
黑客如何利用技术赚钱_是中国利用黑客在技术战争中取得成功
黑客如何利用技术赚钱 This appeared in The Millennial Source 这出现在千禧年的来源 Chinese state-backed hackers have pilla ...
企业级Python开发大佬利用网络爬虫技术实现自动发送天气预告邮件
前天小编带大家利用Python网络爬虫采集了天气网的实时信息,今天小编带大家更进一步,将采集到的天气信息直接发送到邮箱,带大家一起嗨~~拓展来说,这个功能放在企业级角度来看,只要我们拥有客户的邮箱,之 ...
如何利用网络赚钱之一
如何利用网络赚钱之一 1.准备一个域名和一个网站空间. 要在网络上赚钱,就要有自己的网站! 2.准备一张银行卡用来收钱. 3.注册联盟,获取代码国内的联盟有很多,你直接做一个就ok. 4.制 ...
利用计算机技术网络技术,计算机技术和网络技术在机房管理中的应用研究
摘要:当前,科技技术在我国的发展迅速,尤其是计算机技术.互联网技术的发展速度更是突飞猛进,之所以其发展迅速,实质上是因为其能够为广大的群众提供丰富且全面的信息资源.当前,社会迫切需要的就是具有信息技术 ...

如何利用网络技术赚钱的