alyaza [they/she]@beehaw.orgM to Technology@beehaw.orgEnglish · 23 days ago99% of My Website Traffic Is Bots: A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.patronview.comexternal-linkmessage-square12linkfedilinkarrow-up1100arrow-down16
arrow-up194arrow-down1external-link99% of My Website Traffic Is Bots: A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.patronview.comalyaza [they/she]@beehaw.orgM to Technology@beehaw.orgEnglish · 23 days agomessage-square12linkfedilink
minus-squaremasterofn001@lemmy.calinkfedilinkarrow-up8·23 days agoOne thing I’ve noticed that doesn’t need to know page structure - scroll all the way down and look for the last section with the title Conclusion. Just noticed this a few daus ago. So many sites I’ve never heard of about niche topics have the exact same layouts and all ended with a Conclusion. Why is there a conclusion on a page for swapping out a circuit breaker or building a fence gate or how to properly shoot heroin? Ai slop.
One thing I’ve noticed that doesn’t need to know page structure - scroll all the way down and look for the last section with the title Conclusion.
Just noticed this a few daus ago. So many sites I’ve never heard of about niche topics have the exact same layouts and all ended with a Conclusion.
Why is there a conclusion on a page for swapping out a circuit breaker or building a fence gate or how to properly shoot heroin? Ai slop.