Surge Protection

How it works

After experiencing high server load caused by bots and leechers often, I added some code to MoinMoin.request: surge protection.

It works like this:

• load request history (timestamp, ip_or_username, action)
• throw out outdated entries from the action histories
• add the current request
• is the request history for some action too long? Yes:
  • give 503 result code and warning msg
  • if the request history for some action is less than 2 * maximum:
    • add current request with timestamp+lockout_time
• save request history

So if you stay below count in any time interval dt (see below), you never will have problems.

If you are fast and go beyond that count in some interval, you will receive warning messages to slow down.

For any warning msg you get, it will add some rather sticky history entry (because it has a future timestamp, it will stay there for about lockout_time length).

If you react to the warning msg and slow down, this won't make you real trouble (it will just reduce the maximum for you a bit) and most of your access should leave history about a minute later.

If you don't react and continue at high rate (maybe you are a bot, not reading warnings?), those sticky entries will make the history longer than allowed and thus, you won't be able to recover until lockout_time later.

Watching it

cd data/cache/surgeprotect
watch cat surge-log

See also HelpOnConfiguration/SurgeProtection.