[lug] [OT] Web Server Question

[Nate Duehr]

George's answer is a good one... like most things IT-related, the answer
is... "kinda".  (GRIN)

> My question:  From the viewpoint of a web agent (whether it is a
> browser, spider, etc.), is there any difference between static content
> and dymanically generated content?

You can code a dynamic system to spit out stuff that looks like it's
static, or dynamic... it's really up to the coder and how well they know
what they're doing.  And how bad you want something to look
non-dynamic.  I'm kinda curious as to why you need this?  Not a big deal
to me, just wondering... (GRIN).

Another typical thing you'll see on dynamic sites is the close control
of the robots.txt file TRYING to keep spiders from digging through the
site and beating up the CPU cycles and database of the machine
generating the dynamic content.  And REALLY intelligent dynamic content
generation tools could be coded up to "throttle" how fast they respond
to a particular IP or agent if they "sense" that it's a spider just
digging through the site.

There's a LOT of ways to skin the cat... it really depends on what you
want it to do.

> We are developing an application on Apache in Python using Albatross.
> Our home page is generated dynamically.

If you're just interested in making a single page look like it's a
static page and not dynamically created, one example of how to do this
should be relatively easy.  

It depends on *how* dynamic it is... if you're generating only four or
five different "dynamic" pages, one trick is to generate those ahead of
time and have them sitting on the drive and use something like PHP or
Perl or whatever your poison (oh yeah, you mentioned Python... okay...
Python it is! **GRIN**) to simply handle handing out the correct file
from the filesystem where it's already lying there waiting with a nice
old creation date and all that rot... this also has the added benefit of
having very little overhead and if your site gets slammed with traffic,
Apache can handle this probably better than having every single hit to
the home page generate a web page on the fly.  Again, it's all about
requirements... in this example, you're not generating the pages every
time someone hits Refresh... stuff like that. 

The key to web pages on dynamic systems is to start thinking in terms of
software development instead of content development to some extent.  How
much CPU will *this* code burn up, what does that do to overall system
performance?  What are the benchmarks we're going to test that
performance against?  How long should it take for a dynamic page to
generate?  How are we going to measure that as part of the software
release?  How big can the database get?  What are the hard-limits and
soft-limits on how often we want to throw away data?  How do we back up
the database?  Online or offline?  Do we put a special "please stand by"
page up when the database is down?  

You get the idea... to do it "right" requires a bigger-picture
understanding of how everything works together once you head off into
dynamic content... especially for a business site.  If I'm just messing
around at home, no -- these types of questions probably don't get asked.

That's just a really bad example I gave there, but it was just off the
top of my head... but should give a general idea of the types of things
that can be done... a good web developer knows how both the browser and
the webserver being used interact and can tweak it to do just about
whatever you want... or they'll tell you what they can't do... 

Maybe I'm being too obvious with this stuff... dunno.  I've run into
folks who had really new web developers who could barely get the files
uploaded to the server ask some pretty simple stuff that their "expert"
couldn't figure out... so I never know where to start when answering
"generic" style questions.  You may know all of the above already.

Nate, nate at natetech.com