[lug] High load with 'w'

[rm at fabula.de]

On Tue, May 18, 2004 at 05:53:28AM -0600, John Starkey wrote:
> On Tue, 2004-05-18 at 04:38, rm at fabula.de wrote:
> > > A load of 76 on that server is usually when several harvestors, Google,
> > > Slurp and Inktomi all go wild. Nothing we really worry about. If we wait
> > > a few minutes the honeypot catches the bad guys and it comes back down
> > > to 6 or so.
> > 
> > Seems like there's room for tweaking then ... ;-)

Hi John,

> More like room for a new DB server. ;) This site puts out about 7 Mbit
> per second, 24x7, without the bad guys or spiders. Most of that is rich
> media. We also get a lot of people that want to download the whole site,
> with no delay between requests.

Have you ever investigated what really causes the high load? Since you mention
the DB server this might well be disk IO. While high IO rates per se aren't causing
high load (see a previous poster's comments on this) it most likely increases the
average time a request takes to be served. This will result in a higher number of
apache processes and that _will_ push up the load. Even worse, on older kernels
every request will "wake up" all listening (read: idle and waiting for requests)
apache children, so, unless you tweak you high/low marks for the number of
servers, even a short time high number of parallel processes will result in a lot
of (unneccessary) context switches (which again are bad for load and responsiveness).
BTW, i assume you have enough upstream bandwidth? Those 7Mbit look suspiciously 
like an almost saturated 10Mbit cable ;-)

  RalfD

> John
> _______________________________________________
> Web Page:  http://lug.boulder.co.us
> Mailing List: http://lists.lug.boulder.co.us/mailman/listinfo/lug
> Join us on IRC: lug.boulder.co.us port=6667 channel=#colug