Exception Notifier Plugin can make you crazy as it made me yesterday.
Let me explain, I was using this plugin for one of my project and our server got a DOS attack. We were requested some URLs that doesn’t exists, like http://somedomain.com/game.rar etc… . I had registered my gmail account to get notification mails. And due to this attack I was sent around 7631*61 mails in 10 hrs. This made me crazy, because I was unable to receive any other email as (may be) my queue was busy with receiving notification mails. Then gmail started bouncing back those mails saying “The Gmail user you are trying to contact is receiving mail at a rate that prevents additional messages from being delivered. Please resend your message at a later
time”. And then it became a loop, Gmail-My Server-Gmail.
I found 303552 mails in postfix queue of my server. That were ready to send to gmail. Now my first priority was to clean that postfix queue, so I did it by issuing “sudo postsuper -d ALL”. I hope to receive mails ASAP, It is now more than 48 hrs i received any genuine mail.
So I request you to be careful while using this plugin 😉
Note: Since my gmail account not receiving mails, so please sent it to akhil at vinsol dot com
I’m guessing the big takeaway from this would be just to create a special gmail account for the exception notification plugin then in an extremely unusual circumstance such as a dos attack, you still have access to your regular email?
Tim,
I agree with you
On a side note, you might want to add a route like :
map ‘*whatever’, :controller => ‘some’, :action => ‘my_custom_not_found’
then in SomeController :
def my_custom_not_found
render :nothing => true, :status => 404
end
Also, it’s not a great idea to have default “map.connect ‘:controller/:action/:id'” route in your production applications. May be you should go RESTful or at least add a regular expression to your default route. Something like :
map.connect ‘:controller/:action/:id’, :controller => /mycontroller1/mycontrolle2/
Another option is to have your notifications sent out-bound through the company’s MSA, and have them do rate-limiting and blacklisting on that MSA. This means that in a DoS situation, you’ll limit the number of outbound messages and prevent further DoS due to the volume of email exiting your network.
very nice blog… well done!
i believe you are probably using old version of exception_notification(maybe from old repo) which sends emails on ActionController::RoutingError, the latest trunk version doesn’t do that.
Thanks Surendra, I’ll look at that
hmm, a RoutingError should be treated as 404, so no notification should be triggered…
maybe you have an older version ?
def exceptions_to_treat_as_404
exceptions = [ActiveRecord::RecordNotFound,
ActionController::UnknownController,
ActionController::UnknownAction]
exceptions
I’ve faced similar problems back in old php days – I’ve error notification that sends email when problem occurs – same pattern here. Then I’ve workaround too many emails problem using error grouping and my cron script sends 1 email with several errors each 10 minutes.
But since now I’m working on lite traffic sites I haven’t created any solution for rails yet.
[…] Many of you guys(as me) may have used Exception Notifier plugin to get Rails app exceptions right into your mailbox, and may also have faced some problem like this. […]
Monkey patch to rate limit Rails Exception Notification / Notifier
https://gist.github.com/3357795
Hi, just wanted to mention, I liked this blog post.
It was helpful. Keep on posting!