Web scraping is more important now than ever. Gathering and selling data can be lucrative, and analyzing it can help businesses in any industry. As web scraping becomes more and more common, measures to stop bots have evolved. Google’s reCAPTCHA v2 is a great example of this. In this article, I will briefly describe how this system works, […]

Uploading/downloading large files can be tedious, especially when you’re unable to view the progress and status of the request. Using the requests library alongside clint, it’s easy to visually display progress in a console application. We’ll be able to specify the chunk size and monitor the download speed. In the snippets below, you’ll see a function ‘progress.bar’ called. […]

Python is frequently used for web scraping. Often times, the ‘requests’ library is sufficient. However, it is typically only used for basic requests. We can send a GET request to a website, but what if the actual page is loaded via javascript? Using a real browser/web driver allows us to load the page completely. Instead of simply sending […]