Announcement

Collapse

Welcome to the New Seqanswers!

Welcome to the new Seqanswers! We'd love your feedback, please post any you have to this topic: New Seqanswers Feedback.
See more
See less

Updates on traffic/errors...

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Updates on traffic/errors...

    So if you've been here for the last three days you've probably noticed some sporadic performance...apologies for that.

    I'm continuing to monitor it (and please continue emailing me if you run into problems), but it seems like the cause was that Baidu (search engine) has been indexing the wiki and forums with a crazy amount of robotic power. Meaning 6-8 requests per second, from ~30 different IPs, of different complex wiki pages that bring the database server to it's knees.

    I am now blocking their IPs, we'll see if it holds.

    Grrrrrrrrrr.

  • #2
    I understand that Baiduspider does not respect the robots.txt !

    http://seqanswers.com/robots.txt

    User-agent: *
    Disallow: /w/
    Disallow: /wiki/Special:


    User-agent: Baiduspider
    Disallow: /

    Comment


    • #3
      Thanks for sharing useful information.
      Water Bottle
      Home Decor Items

      Comment

      Working...
      X