My Future of Web Apps post made the front page of Digg yesterday, so I got to learn what it’s like to ride the Digg wave. It was an interesting trip to say the least.

I posted the entry yesterday morning, Dugg it once myself, and messaged a few friends to check it out. I then went about my day and in the early afternoon came back to find my site was down. I checked Digg and there it was: #6 in the list. Whoo hoo!

A Diggaggle(1) had already started flaming the site in the comments for being down. Now I myself appreciated the irony of the article ending in “…Break Your Website”. However, I didn’t really anticipate it making the front page, especially when others in the blogosphere had better coverage of the event and had posted about it much earlier.

I emailed Dreamhost as soon as possible about the site being down and called up Duane (the other half of AU Interactive) to redirect the DNS to our own server so we could put up a temporary page. (This blog is hosted on a separate shared account which usually suffices for small projects.) My email to Dreamhost had the following subject line: “Site Dugg! Is Down! Please send reinforcements!” and body: “I just got Dugg! Site is down! Please help!”

I was a little surprised to get the following email back from them about 15 minutes later:

Can you please explain what Dugg is? It appears you are being DDOS’ed … Your apache server was using up almost all of the system resources … Server uptime: 5 minutes 57 seconds … Total accesses: 5675 – Total Traffic: 4.5 MB … CPU Usage: 93.3% CPU load. 15.9 requests/sec – 13.0 kB/second – 835 B/request … 250 requests currently being processed … in short it is crashing the server.

It was kind of unexpected that they weren’t aware of Digg and the “Digg Effect”– I sort of assumed that a large hosting company would be on top of things like this. They renamed the folder to prevent the server from crashing as a temporary fix.

As soon as the DNS propagated, our servers were taking a brunt of the hits – within about an hour, we had received 4,000+ unique visitors. Duane informed me that the page was pretty bloated (300K+). AHA! That was my mistake. I had installed a number of plugins without thinking too much about it and it really “obesified” my pages. I had the wp-cache plugin activated which helped, but the extra plugins, namely Lightbox V2 (which uses prototype and scriptaculous libraries, both of which are a bit hefty ) were really to blame.

So I took out the unnecessary JS and waited for the traffic to die down. Duane let me know when the server relaxed a bit so I renamed the folder back at Dreamhost and changed back the DNS to point to its proper home.

Later in the evening Dreamhost responded again, explaining the situation and noting that currently the server appeared to be having a normal level of traffic. This is the only problem I’ve ever had with them, so I’ll probably stick with them, even after this fiasco. Plus they have a nice affilate program. *wink*

Now I’m just moderating comments and having fun with it. Thank you to everyone who took the time to give their feedback. I’ll leave you with some fun screenshots from yesterday:

This is kind of funny: The Digg mirror advertising the very service that couldn’t handle the site traffic.

Yes, I just coined a new term:

Diggaggle: n. 1. A gaggle of Diggers. 2. A rowdy group of commenters who leave snide remarks. Usage: “I just had a pile of virtual poo tossed my way by a bloodthirsty Diggaggle” [Origin: 2006 from Digg (n.) social networking site, and gaggle (n.), a “flock of geese when not flying”. Hence Diggaggle: a "flock of geeks".]

{ 5 comments… read them below or add one }

Allen September 28, 2006 at 4:44 pm

Congrats – I can only hope to one day be dugg :) I was pretty close with 22 diggs on one story but that was it.

Reply

Sam Jackson September 29, 2006 at 1:40 pm

What’s that fancy looking middle statistics program? Seems cute.

DH has been good to me.

Reply

Markus September 29, 2006 at 1:50 pm

The first 2 screenshots are Google analytics. The 3rd is our own server stats (where we moved the page temporarily).

Reply

family man September 30, 2006 at 10:38 am

Way to go Markus .. enjoyed reading your experience with Digg and Dreamhost’s response. I do a lot of dynamic stuff for some heavily trafficked sites, so I generally write light lean code .. makes a whopping improvement in performance.

Reply

Markus October 1, 2006 at 12:26 pm

Yeah, that’s the flipside of using themes and plugins that were designed by others. The theme this blog is based on (Quilm2), actually had gravatar plugin code embedded into the theme itself – really bad idea.

Plus we looked into it some more and it seems like the wp-cache plugin wasn’t working properly either – because of conflict with code somewhere else (I think) – this stuff doesn’t really rear its ugly head until you tax the servers.

Reply

Leave a Comment

{ 2 trackbacks }