Progress indicator?

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Progress indicator?

Beitragvon Giorgos » Mi Apr 05, 2017 7:43 pm

Hi! :D

Is there any progress bar (or any other - similar kind of indicator, for crawling process?
I started crawling a website and after ~10h yacy is still crawling.
Can someone predict, when the process will be completed?

TIA! ;)
G.
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm

Re: Progress indicator?

Beitragvon jerk » Do Apr 06, 2017 12:51 am

I only joined these forums recently myself, so I am not certain how much help I can be to you. It would seem to me however that in order for the crawler to produce a progress bar, it would have to know how long the process should take. Since that determination cannot be made until the site has been crawled, I would think that a crawler progress bar would be impossible. Just a thought though.
jerk
 
Beiträge: 4
Registriert: Mo Apr 03, 2017 4:53 am

Re: Progress indicator?

Beitragvon luc » Do Apr 06, 2017 8:43 am

Hello,
I also agree with jerk, YaCy can not predict itself the end of a crawl task because it depends on the links found in the crawled pages... unknown before crawling. But maybe an approximation could be calculated when re-crawling a known website...

By the way, I believe that currently the best estimation will come from your knowledge of the website content, the "Crawling Depth" and eventual URLs filters you used, and the monitored "Crawler PPM" value on the first hours.

If your crawl appears to be too long for you, maybe some of your crawl parameters should be re-adjusted.
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: Progress indicator?

Beitragvon Giorgos » Do Apr 06, 2017 2:45 pm

THANKS guys!!! ;)
I'll look at it, again.
G.
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm

Re: Progress indicator?

Beitragvon smokingwheels » Fr Apr 21, 2017 6:45 pm

The Creator of Yacy does have a sense humour, if you depth set to 99, I thinks it displays the Text Eternity somewhere.
smokingwheels
 
Beiträge: 137
Registriert: Sa Aug 31, 2013 7:16 am

Re: Progress indicator?

Beitragvon Giorgos » Fr Apr 21, 2017 6:58 pm

:lol: :lol: :lol:
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast