0%

Aggressive Chinese Bot/Crawer

Recently, I’m seeing massive weird HTTP requests on our company production servers, most of them are using weird User-Agent
For example:

  • LiebaoFast (50%~60%)
  • MicroMessenger
  • Kinza
  • Some weird random Huawei cellphone
  • New android versions with super old Chrome
  • Having zh_CN in their User-Agent string

Most of them are originated from Huawei Cloud in Hong Kong, so we might just encounter a bunch of aggressive bots.

Since our company is just a medium size web framework developer, we didn’t put too much effort into website filtering. Also, my supervisor does not want to put too much resources in this.
So, cloudflare to the rescue.

Resolution

  • It seems like those IP address are originated from the same AS number, AS136907
    So, just add a firewall rule to your cloudflare setting, and enjoy.
  • We don’t want to affect the real users coming from those region, so instead of blocking them, I choose to use “JS challenge”.
  • Also, some of the traffic are originated from China, so we’ll also add those IP into our firewall rule.
    CF_Setting