Help me get rid of bots traffic on my e-commerce Shopify website
In April, we started noticing that on certain days our website's traffic spiked. The issue disappeared but then noticed few weeks later that issue returned. I did some research and noticed that the traffic is from California and am pretty sure someone is…
In April, we started noticing that on certain days our website's traffic spiked. The issue disappeared but then noticed few weeks later that issue returned. I did some research and noticed that the traffic is from California and am pretty sure someone is scraping our website (we are an e-commerce company) I looked for solutions and installed Negate. It did seem to help but now the issue is back. I am wondering if I should go through Negate's log manually and block IP addresses or should I get Cloudflare's paid plan or is there a better solution?
Collected discussion
Sorry but can you explain what do you mean by “kind of bot traffic”?
Manual IP blocking is useless since scrapers rotate addresses constantly. Cloudflare's free plan handles a surprising amount of this through its bot fight mode and rate limiting rules. The paid Pro plan adds the Bot Analytics dashboard which makes it much easier to see what you are actually dealing with before deciding on rules. If you want to stay within Shopify, setting a rate limit on your storefront through Cloudflare's WAF rules (block IPs hitting more than X requests per minute) cuts off most unsophisticated scrapers without touching real users.
Its scraping of pricing/inventory data - the traffic is mainly only collections page.
Thank you. I do want to stay within Shopify. I need Shopify's paid plan to set WAF rules?
It is hammering collections page specifically. In March, one page was getting continuous hits but it stopped after a week. In May (when I implemented Negate), I noticed collections pages are getting hammered (with multiple filter combinations).
I did check GA4 - most of the traffic is direct on collections pages. The sessions are hardly 1 second with 100% bounce rate. It visits the same page with different filter combinations I don't have access to server logs but Shopify analytics also shows bot traffic The spike is usually at night - 2 am to 7 am. It's from all over the world (countries I have not heard of before but there is a substantial amount from California)
Might be a noob question, but how do I find the ASN? Or do you recommend setting up free cloudlfare account first and then build a list of ASNs and research?
Yes, the spike traffic is direct with 100% bounce rate and 0s session time. It usually happens Tuesday-Thursday from 2 am to 7 am ET. I did initial investigation and it is the Shopify collections pages that are getting hit. I am sure it is scraper traffic
Honestly... I wouldn't start by manually blocking IPs. If it's actually scraping traffic, you'll probably end up playing whack-a-mole because the IPs keep changing. I'd first verify where the traffic is coming from: Check GA4 for source, medium, country, landing pages, and engagement metrics. Check Shopify analytics and server logs if available. Look for patterns like 100% bounce rate, very short sessions, or visits hitting the same pages repeatedly. If the traffic is genuinely causing problems, Cloudflare is usually where I'd start before manually blocking hundreds of IPs. It gives you much better bot management, rate limiting, and firewall controls. Also... be careful not to block legitimate crawlers or customers while trying to stop scrapers. The California traffic is interesting, but I'd want to confirm it's actually malicious bot traffic before spending time chasing IP addresses.
I wouldn't block specific IPs. I would look at an aggregated count of requests, grouped by ASN (the current provider routing the IP addresses). I would build a list of the ASNs with the highest amount of requests that match the patterns of the abusive traffic, the pages you mentioned and the timeframes. I would research the list of ASNs and using the security rules option in your free cloudflare account, to create a rule to block ASNs that are primarily providers to datacenters. I would also determine the primary countries that your customers are from or that you expect to do business in and create rules to restrict requests from IP addresses in countries outside of those areas..
If it's coming back after Negate and it's concentrated from California, it's almost certainly scrapers/datacenter traffic hitting you at the network level, so app-layer blockers will always be playing catch-up. A few things that actually hold: put Cloudflare in front of the store and turn on Bot Fight Mode (it challenges the datacenter IP ranges most scrapers use before they ever reach Shopify), and in GA4 the traffic won't pollute your reports if you mark "known bots" filtering on and set up a segment excluding that California datacenter ASN. The reason it "disappears then returns" is usually a scraper rotating IPs on a schedule, so check whether the spikes line up to the same weekday/time, because that pattern is the giveaway it's automated, not real demand. Quick q so I'm not guessing: is the spike showing up as direct/none traffic with near-100% bounce and 0s session time? That'd confirm scraper vs. a real referral surge.
you can't do anything without ruining the actual customer experience, you would need captcha to even open the website to have a chance to block this. Just ignore it
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.
This comment was deleted.