Battling the GPU shortage with a webscrapper
Intorduction
A friend of mine recently asked me if I could help him acquire a RTX 4090. I decided to build a web scraper that would check the stock of a GPU and send a notification to a discord channel using a webhook. The web scraper is written in python and uses the requests library to get the html of the website and the BeautifulSoup library to parse the html. Once the HTML is parsed the script checks if the GPU is in stock using elements in the dom tree. If the GPU is in stock the script sends a notification to a discord channel using a webhook. The web scraper is hosted on a server and is run every 5 minutes using a cron job. The web scraper is run every 5 minutes to prevent the website from blocking the IP address of the server.
The location of the server was chosen based on the location of the website that is being scraped. The website is hosted in the UK so the server is hosted in the UK (London) as well.
The source code can be found here
Another example of a web scraper that I made can be found here It checks the stock of a website that sells servers.
Demo
This is the discord channel that the bot would post in if it found any stock, it would tag the role @everyone so everyone would get a notification. The bot would post a message every 5 minutes if the GPU was in stock. The bot would stop posting once the GPU was out of stock.
And after ordering and waiting a few days, the GPU arrived.
