Web scrapers I build, run and fix in production.
I build scrapers, publish them on Apify, and keep them running when the sites change underneath them. Each post here takes one real problem, such as every dentist in a city with an email address, and solves it with a real run: the input, the output, what it cost, and where it stops working.
The code for every post is on GitHub.
Posts
- October 30, 2026How to compare Facebook engagement between pages (7 London museums, 56 posts)The latest 8 Facebook posts of 7 London museums. The National Gallery's typical post drew 1,331 reactions, comments and shares; the Science Museum's drew 15. Under 10 cents.
- October 29, 2026How much more do Edinburgh hotels cost on New Year's Eve? (Booking.com prices for the same hotels, two weeks apart)Edinburgh on Booking.com, one night for two. On New Year's Eve 140 places were left and the same 32 hotels cost a median 5.2 times what they cost on 14 January.
- October 28, 2026How to find a business's website, email and Instagram from just its name (8 New York bagel shops)Eight bagel shop names in, eight rows out. Google rated all 8, the websites gave 3 emails, 4 Instagram accounts came back with follower counts. 5 minutes 51 seconds, 15.9 cents.
- October 27, 2026How to tell how busy a restaurant is from OpenTable (50 steak restaurants in Chicago, ranked)OpenTable's "steak" search in Chicago, ranked by its own "Booked N times today": the busiest of 50 was Italian, at 269, and listed 28th. One search, 69 seconds, 2.75 cents.
- October 26, 2026How to find Creative Commons photos big enough to print (26 of Lisbon's Tram 28)300 Google Images results for Lisbon's Tram 28 held 9 photos 3,000 px wide. Two filters found 26 with a Creative Commons label, for 1.7 cents. What came back wrong.
- October 25, 2026How to find the email address for a list of company websites (36 accountants in Manchester)36 Manchester accountancy firms' websites. The home page alone gave an email for 27, five pages per site gave 32, for 12 cents. What the extra pages are worth, and what they cost.
- October 24, 2026How to get past the Google Maps results limit (153 coffee shops in Austin became 471)One Google Maps search for coffee shops in Austin stops at 153 places. Six neighbourhood searches found 471 distinct ones, for 42 cents. The code, the overlap, and what it costs.
- October 24, 2026How to get the email of every dentist in Miami from Google MapsGoogle Maps lists 182 dentists for one Miami search and shows an email for none of them. One run found 81 addresses in five minutes for 31 cents. The code, the output, and where it falls short.