[lug] expect like tool for a website

[David Morris]

On Tue, Aug 17, 2004 at 01:53:20AM -0400, Hugh Brown wrote:
> I have a friend that needs to automate the retrieval of data from a
> website that has a non-trivial pathway to the data.  I know expect works
> well for scripting terminal types of activity.  Is there an open source
> tool that would work with any of the browsers out there (mozilla,
> firefox, safari, ...) to do such a thing?

Try using 'wget', a command-line web-page retrieval program.
Combined with a python script and/or grep/awk/sed/etc. I
have automated retrieval of a lot of web pages.  The only
two areas I've found so far that it doesn't handle are forms
and redirections between web pages that don't have links
referencing the next page.

If you *must* have the GUI automation mechanism, I know such
things exist but I've never dealt with them...but I suspect
intelligent automation won't be possible (e.g. only a
fixed-pattern of clicks, not clicks based on application
content).

--David