Scrapy - extracting the data you need from websites
Scrapy
extracting the data you need from websites
Screenshots
Hunter's comment
'scraping' websites is pretty a common thing that's out there in the world. i knew companies many years back that did this at a micro level for people that just wanted information FAST that they could populate their CRM's with just to be able to cold call a bunch of execs and 'decision makers' in a business.
not cheap as well, they could run the reports and charge them quite a lot of money to deliver this data -- often the data was collected illegally or at least let's say it was a 'grey area' until laws came into place around it.
i'm sure it could be incredibly useful as well for someone wanting to build some web spiders that actually use this scraping technology in a productive way too, maybe for building exports or collecting together social media content to store away as legacy items.
Link
Contributors
Hunter: @teamhumble
This is posted on Steemhunt - A place where you can dig products and earn STEEM.
View on Steemhunt.com
Please read our posting guidelines. If you have any questions, please join our Discord Group.
It's been useful actually, but i believe that many sites actually are blocking the accesses from those APIs. You may need to use it with VPN. Also im not sure if react-based websites nowadays would be working well with these types of scraping tools because they generate html tags after the server call (or only in a client page).
Indeed a grey zone, but the website owner is in charge of the privacy in my opinion. Great product and hunt!
No doubt @teamhumble, "Scrapy" is very useful hunt you introduce here.
Steemhunt is great social media platform where we enjoy daily wonderful products, applications and other software.
Scrapy is very helpful scraping technology due to which we easily extract data we need from websites. Thanks a lot for always sharing useful hunts. stay blessed and keep sharing.
hey, stop making silly comments.
so you just have your own f***ing comment format in SH like,
and just copied and pasted from the hunting post like this,
and again the format
Seriously, shame on you f***ing penny pickers.
With some sites really loaded with information and ads, Scrapy can help a user to select valuable pieces of data. I will try it out soon.
We all use data every day. Extracting it from website is really cool. I think I like this hunt.
It's really good hunt
Great app written in Python and running on all systems to extract data from websites easily and quickly. Thanks for shating it @teamhumble, very useful.
A great website scraping tool. Definately a tool to bookmark when you are looking for extracting data from different websites. Thanks for sharing.
Scrapy is a very good and innovative product through this product we can get data which type of material we want or required very fast. It is useful product and Great hunt.
You really think this comment is helpful for SH? If you thought this hunt was cool, then you could just say "Cool hunt!". THB already mentioned all the info what you just repeated. You're clearly a penny pickers who constantly collecting f***** pennies from SH's comment voting pool. Shame on you.
This is quite a good tool for extracting only relevant data from sites or other sources. The most important this is if it can get the delta, so I will need to take a look on this.