Can you recommend to me easy to use free tool to harvest links from web site

I googled and tried many tools but some of them did not work because of the login screen like parsehub and other are more complicated that uses regex. I need the tool to do that:

- Go through pages 1 2 3 etc...
- Click certain link
- Enter login page user/password
- Search the page for links like that "download.com/file.zip"
- Loop again

Thanks

William654 said:

I googled and tried many tools but some of them did not work because of the login screen like parsehub and other are more complicated that uses regex. I need the tool to do that:

- Go through pages 1 2 3 etc...
- Click certain link
- Enter login page user/password
- Search the page for links like that "download.com/file.zip"
- Loop again

Thanks

Click to expand...

Hi there
Seems an interesting project this one

If you can install python (also runnable on windows) it's an EXCELLENT language for A.I and stuff like that e.g "Spider crawler" etc. You will need though to do some coding or at least understand a bit about programming although python at least for basic stuff is very easy to learn and use.

A good start is


If you want a full blown GUI then most of the good stuff is not free and those that are free only allow so many API calls per month.

I suspect if you've the time -- learning a bit of python is probably the best approach -- install and run on a VM so if projects get totally hosed up (inevitably for beginners they will) just delete the VM and begin again.

Cheers
jimbo

@jimbo45

Thanks but looking for GUI tool.