Other
See other Other Articles
Title: Server outages
Source:
[None]
URL Source: [None]
Published: Mar 20, 2012
Author: Neil
Post Date: 2012-03-20 11:28:48 by Pinguinite
Keywords: None
Views: 4033
Comments: 7
I've been dealing with some server issues which have resulted in some outages. What's been happening is that "bing", that wonderful big MS search engine, has had its spiders occasionally pounding the server so hard with page requests that it's brought the server down. I've institute a few changes recently on the server to combat that. The first countermeasure has been a simple load monitor program which shuts the web server down if it gets too busy, and restarts it once it catches up on work and calms down. When it activates, the web site is unavailable, but down time is usually limited between 1-4 minutes. This has kicked in a few times over the past week or two and it's worked well. Yesterday, however, bing was particularly aggressive and slammed the server so hard/fast the server was down before that counter measure could kick in. Manual reboots were a bit tricky because the bing page requests were coming in continuously such that as soon as the server rebooted, it would get overwhelmed again before I could even intervene. I did get in though and got a list of IP addrs which were the culprits and manually firewalled them off the server. That may well end the matter but I don't know if bing has other IP's they use which I don't yet know about. The down side of this is that bing won't be indexing anything on the server, but that's just the cost of doing business. Until those clowns at bing get their act together and stop overwhelming servers, that's probably how things will have to stay. I can add another countermeasure that automatically firewalls such "attacks", basically doing automatically what I did manually, and may do so.
Post Comment Private Reply Ignore Thread
Top • Page Up • Full Thread • Page Down • Bottom/Latest
#1. To: Pinguinite, A K A Stone, liberator (#0)
Maybe you could just put up a robots.txt. Anyway, you and Stone need to work on this whole Griftathon thing.
Thank you Neil! ;)
("Every fetus that DOESN'T go on welfare in 18 years because he/she was aborted yesterday... brings a smile to my face today" ~ GrandIsland - LibertyPost)
I am THERE!!!!
Oh, and yes that would do it but then all the search engines would avoid the site. Bing is the only problem child.
of course your management of multiple asynchronous software web-based servers sharing a single IPA could not be an issue ... such as having too much peak traffic simultaneously creating a crash and hence an unexplained "reset" once the "glitch" is discovered. just a thought.
It has nothing to do with the shared use of an IP address. It certainly is because of "too much peak traffic" but what I am seeing is that traffic is primarily coming in from one of a half dozen class C IP addrs with "bingbot" signatures. The result is much the same as a Denial of Service (DOS) attack. I don't know what you mean about "reset" and "glitch".
considering the content of 'this site'...would that be a 'bad thing'. I still think you are a spook. {{{chuckle}}}
Never swear "allegiance" to anything other than the 'right to change your mind'!
#2. To: Pinguinite (#0)
#3. To: TooConservative (#1)
Pull the plug on the server every few hours.
Demand $80,000 to keep it running for another 3 months.
. . .
Profit!
#4. To: TooConservative (#1)
Maybe you could just put up a robots.txt.
#5. To: Pinguinite (#4)
Bing is the only problem child.
#6. To: HP Lovecraft (#5)
of course your management of multiple asynchronous software web-based servers sharing a single IPA could not be an issue ... such as having too much peak traffic simultaneously creating a crash and hence an unexplained "reset" once the "glitch" is discovered. just a thought.
#7. To: Pinguinite (#4)
all the search engines would avoid the site
Top • Page Up • Full Thread • Page Down • Bottom/Latest
[Home] [Headlines] [Latest Articles] [Latest Comments] [Post] [Mail] [Sign-in] [Setup] [Help] [Register]