l*******9
发帖数: 177 | 1 http://www.seoprotoolz.com/tools/collector/quick.php
search any keyword will give you a list of links, but
when I open the source of this page. The div element
with id="div1" always display empty. those links should
be generated by Google AJAX API. How can I grab these
links.. //thanks |
o*o
发帖数: 404 | 2 if you only need one page, save the page as html file,
you will get those links in that file.
if you need batch process, you have to pick up a script
language to parse it.
【在 l*******9 的大作中提到】
: http://www.seoprotoolz.com/tools/collector/quick.php
: search any keyword will give you a list of links, but
: when I open the source of this page. The div element
: with id="div1" always display empty. those links should
: be generated by Google AJAX API. How can I grab these
: links.. //thanks
|
l*******9
发帖数: 177 | 3 I did save the result page, it doesnot show any URLs in the
URL list shown in my web browser.. I tried Perl WWW::Mechanize
and the result I got is the same as I saw in notepad.. No URLs
are saved? Anything wrong? Any other things involved besides AJAX??
//thanks
【在 o*o 的大作中提到】
: if you only need one page, save the page as html file,
: you will get those links in that file.
: if you need batch process, you have to pick up a script
: language to parse it.
|
o*o
发帖数: 404 | 4 If you check the link in your browser in detail, you will find all links
refer to the page itself. When you click a link, it's actually a Js call
but with different POST paras.
【在 l*******9 的大作中提到】
: I did save the result page, it doesnot show any URLs in the
: URL list shown in my web browser.. I tried Perl WWW::Mechanize
: and the result I got is the same as I saw in notepad.. No URLs
: are saved? Anything wrong? Any other things involved besides AJAX??
: //thanks
|
l*******9
发帖数: 177 | 5 Do you know how can I do this in Perl script, I cannot use a real
browser. I did submit the form with "POST" method, say
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $ret = $ua->post(
"http://www.seoprotoolz.com/tools/collector/quick.php",
[ myQuery => "abc" ]
);
die $ret->status_line if not $ret->is_success;
print Dumper $ret;
__END__
the result content doesnot contain any URL.. any hints?
I need to grab these URLs an
【在 o*o 的大作中提到】
: If you check the link in your browser in detail, you will find all links
: refer to the page itself. When you click a link, it's actually a Js call
: but with different POST paras.
|
l*******9
发帖数: 177 | 6 In fact, If I search "abc", what I really need is just the
list of text as follows:
abc news
abc kids
abc family
abc tv
abc distributing
abc radio
abc lost
abc warehouse
abc bingo
abcarcade
......
not the links, but how can I get this list with a Perl script..:)
//thanks
【在 l*******9 的大作中提到】
: Do you know how can I do this in Perl script, I cannot use a real
: browser. I did submit the form with "POST" method, say
: #!/usr/bin/perl
: use strict;
: use warnings;
: use Data::Dumper;
: use LWP::UserAgent;
: my $ua = LWP::UserAgent->new();
: my $ret = $ua->post(
: "http://www.seoprotoolz.com/tools/collector/quick.php",
|
o*o
发帖数: 404 | 7 use regular expression.
【在 l*******9 的大作中提到】
: In fact, If I search "abc", what I really need is just the
: list of text as follows:
: abc news
: abc kids
: abc family
: abc tv
: abc distributing
: abc radio
: abc lost
: abc warehouse
|
l*******9
发帖数: 177 | 8 The problem is I dont have the source which contains the
text I need..:( or none of the above listed entries can be
found in the saved page or $ret->content of my code.
【在 o*o 的大作中提到】
: use regular expression.
|
l*******9
发帖数: 177 | 9 the search string will be provided from the command line, so
I can not copy/past the list from a browser. I need a web
automation code.
【在 l*******9 的大作中提到】
: The problem is I dont have the source which contains the
: text I need..:( or none of the above listed entries can be
: found in the saved page or $ret->content of my code.
|
o*o
发帖数: 404 | 10 your query string is not correct.
【在 l*******9 的大作中提到】
: Do you know how can I do this in Perl script, I cannot use a real
: browser. I did submit the form with "POST" method, say
: #!/usr/bin/perl
: use strict;
: use warnings;
: use Data::Dumper;
: use LWP::UserAgent;
: my $ua = LWP::UserAgent->new();
: my $ret = $ua->post(
: "http://www.seoprotoolz.com/tools/collector/quick.php",
|
l*******9
发帖数: 177 | 11 why do i need query string? I am using post, not get. :)
【在 o*o 的大作中提到】
: your query string is not correct.
|
o*o
发帖数: 404 | 12 I meant query array
try replace this line
with
[xajax => 'getKeywords', xajaxargs[] => 'myQuery=abc
xjxquery>'] |
l*******9
发帖数: 177 | 13 nice, this did the tricks :) thanks a lot.
【在 o*o 的大作中提到】
: I meant query array
: try replace this line
: with
: [xajax => 'getKeywords', xajaxargs[] => 'myQuery=abc : xjxquery>']
|
