WordPress is NOT a lightweight system. Especially if you have a number of plugins installed, some of which may not be written to the best standards. wp-cache is a HUGE help, no question. But it’s not a magic bullet. Plus the main problem isn’t system load due to the code – it’s due to the number of people hitting your site at the same time.

I have WordPress running on my own dedicated server. When the comment spambots were in full battle mode, I’d see swarms of connections hit at the same time and Apache would run out of available processes due to the ‘burst’. It took a fair amount of tweaking to ensure they couldn’t take the server down (though they were running the comment code which spiked memory consumption badly)

Server tuning for swarms like this is tricky and the tweaks are often not those you’d do on a shared system.

So give pair a little break. They kept the server going for all the other people it hosted by blocking your swarm at the firewall. It sucks because you lost that chance to shine, but they did the right thing. WordPress.com can handle swarms better because it runs on a rack full of servers that can distribute load and the systems are tuned to run WordPress, not whatever site some user may decide to run in their shared account. Most shared hosting outfits try to fit as many people as they can onto each box since the margins are so low (Host a site for $2/mon!) Not saying it’s right and too many over provision their shared boxen, but you can have dedicated servers crash under a Digg if they aren’t configured for it.

FYI – I’m not a pair customer. I’ve just been administering web servers for years and have endured my share of Diggs and Slashdottings.

Then again, I’ve had my hand in several hosting services that went from great to hate, so I can’t say I have much faith in any service…

Hello Jason,

We never want to block traffic to a customer’s site – that is a last
resort. The problem is that WordPress is extremely intensive; when used
in a shared-hosting environment it can cause issues for everyone on the
server; not just the site owner themselves. With hundreds of other users
on the same server, it is in the best interest of everyone on the server
to disable a site temporarily when nothing else can be done. The other
people on the server are paying customers just as is the owner and
operator of the blog you visited. They do not deserve to have their sites
down because one domain on the server is experiencing traffic levels far
beyond average.

Another item to note is that if we receive prior notice that a certain
site may be expecting a large influx of traffic we can work to accomodate
that site on a temporary dedicated host. This has been done in the past
for a number of customers, most often for no charge. Unfortunately,
moving a site in the midst of the traffic surge is not a viable option -
this can cause more problems that it would solve. Also, we certainly can
not move an account from one server to another without the approval of the
customer – it cannot be assumed.

In this specific case, it appears that the subject of the Reddit and Digg
submissions was a picture – linking directly to the image instead of the
WordPress blog would have avoided this issue entirely. Further, since the
image was garnished from CollegeHumor.com the link could have went
directly there, avoiding any downtime of the site in question. Further,
we have various account levels; this was not an issue of a site being
disabled on a High-Volume account or higher. If one has a site that is
going to be receving a certain amount of traffic, the appropriate hosting
plan should be chosen.

I apologize for your concern and your hesitancy to host with pair
Networks, however, this is not an issue that is specific to pair Networks.
As someone who frequents social sites such as Digg.com, I see quite a few
accounts that are often disabled or terminated due to traffic woes. This
is not pair Networks’ viewpoint – accounts are not disaabled due to
traffic, only the domain or script in question. When this occurs, the
user is notified immeadiately. Unfortunately, there is no way that we can
see into the future to alert a customer to a problem that has not occurred
yet.

Thank you,

Jon C.
pair Networks
support@pair.com

- pairNIC: Online Domain Name Management – pairNIC.com
- Free Newsletter @ http://insider.pair.com/

>
> This message has been sent from the form on:
> http://www.pair.com/contact/general.html
>
> My name is: Jason
> I am not a current customer.
> My e-mail address is: treker@umich.edu
> My phone number is:
> My username, domain name, or billing account number:
>
> Message:
>
> Hello,
>
> I recently read a blog entry by the owner of nsharp.org located here:
>
> http://nsharporg.wordpress.com/2007/08/15/congrats-you-are-on-digg-and-reddit-we-know-this-is-a-special-moment-for-you-and-as-your-web-host-for-the-past-8-years-we-are-going-to-help-you-celebrate-by-killing-your-site/
>
> I just have to say that I really think it’s wrong what pair.com did to
> his site as soon as it became popular. I mean, that’s when you need a
> reliable host the most! Instead of re-provisioning resources to
> accomodate the load when it was featured on reddit.com, you simply
> pulled the plug. Are you going to do the same thing to your site if it
> is fortunate enough to be featured on a social networking site? I
> don’t think so.
>
> How about you get behind your customers and support them instead of
> pulling the plug when it actually matters whether or not they have a
> reliable host.
>
> I was considering moving my site to pair.com, after all a lot of
> people have recommended it to me, but after hearing this it’s not
> longer in the deck of cards.
>
> Sincerely,
>
> Jason
>
> –
>
> Remote Address: xxx.xxx.xxx.xx
>

http://www.nearlyfreespeech.net/

What’s more you can actually make contact with *humans* there when something goes wrong.

Also you only pay for what you use too- $1 per gig transferred. Greatest host in the world if you ask me.

They’re only going to set themselves up to handle the digg/slashdot/reddit effect when they realize that NOT handling it hurts their wallet. I left myHosting that come that month and moved to Dreamhost, who told me in an email before signing up, “When it comes to the slashdot.org effect, we never shut anyone down.”

So I’d say switch from Pair.com to dreamhost and be sure to let them know.

I probably spend 80% of my time tuning our firewall to stop the latest 12,000+ member botnet attack, or the latest wave of spam, or the latest unexplained traffic spike, or the latest equipment failure … and the other 20% of my time answering the phone.

Pair probably has a lot of the same problems. Probably even more so, because they host a /lot/ more websites than we do, which naturally makes their network the target for a lot more abuse.

While I would be happy for one of our customers if they made it to reddit, digg, or slashdot, I’d also have to face the fact that one customer’s sudden popularity is causing timeouts for 500 other customers, and all of those customers are setting my phone aflame. At that point, the decision becomes very easy: shut down and piss off one customer, to make 500 other customers happy.

You say that you weren’t approaching your 200GB monthly limit, and I believe you, but 200 gigs/month has little to do with how much minute-to-minute bandwidth you have available. They could probably handle a moderate sustained throughput for a month and be OK with it, but they can’t handle having an OC-3 swamped out for an hour. (I suspect they were having a bandwidth issue, not a processor utilization issue.)

You said, “There is no excuse in the universe for not being able to handle the d/r/s effect.” Just curious, have you worked for a hosting company in any kind of network administration capacity?

You keep mentioning that, “WordPress is free, and they can do it…” Well, almost. On http://wordpress.com/about/, you find that WordPress.com is hosted by Automattic, and that ISPs using things like Akismet to stop the influx of spam (we use a Barracuda instead, but…) help to foot the bill for your free blog. Pure hosting services is a hell of a tight margin industry these days, and I can’t understate how much of a difference it makes to be able to cover bandwidth fees with product revenue. So, again, I think Pair is at a reasonable disadvantage here.

Also, as a webmaster, some of this is your responsibility, too. Once you realized you were drawing that much traffic, you might have moved your site and set up an htaccess file to handle the redirect.

Anyway, congratulations.

Reply: Thank you for your comment. I’m sad about all of this (about the fact that this has become such a big issue). This was initially a rant, and it became popular somehow.

To answer your question: Yes, I did use htaccess to redirect traffic, and at the time, my site was just a static html page with a jpg. It was a very low impact on pair. When the site was dead I kept emailing them to at least let me enable a htaccess redirect for the whole site, which they eventually did.

I’m honestly telling you, the traffic wasn’t that much. The main site used wordpress with cache, so it was mostly static content.

The issue pair were experiencing was processor related, not bandwidth related. I still feel they could have worked some sort of solution. As a last resort, they could have disabled the problem files or folders instead of killing the whole site outright.

Anyway, thank you for your insightful comment!