Pending in collection : (What is that)

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

Pending in collection : (What is that)

Beitragvon usern » Do Nov 28, 2013 2:14 pm

Hi.

On http://localhost:8090/Crawler_p.html under Progress there is a row named "Pending in collection:" , what does that mean exactly?

//Usern
usern
 
Beiträge: 13
Registriert: So Sep 23, 2012 12:33 pm

Re: Pending in collection : (What is that)

Beitragvon freak » Do Nov 28, 2013 9:19 pm

As far as i understood, "pending in collection" shows the amount of documents which are queued for (solr) postprocessing. The postprocessing will start automatically after the crawler has done all crawl jobs (OLD: is ready to crawl documents), but i am not sure at this point.
Zuletzt geändert von freak am Do Nov 28, 2013 11:19 pm, insgesamt 1-mal geändert.
freak
 
Beiträge: 21
Registriert: Do Okt 10, 2013 10:59 pm

Re: Pending in collection : (What is that)

Beitragvon usern » Do Nov 28, 2013 10:56 pm

Thank you freak.

It seems to me that this is a HDD thrashing job, I had over 200000 pending in collection and it has been going on for hour after hour now.
Would it help if I did put some of the files on a separate disk (SSD in this case)?
usern
 
Beiträge: 13
Registriert: So Sep 23, 2012 12:33 pm

Re: Pending in collection : (What is that)

Beitragvon freak » Do Nov 28, 2013 11:23 pm

Possibly it helps if you stop the crawler manually ... the post processing job starts automatically after a while and the counter should decrease. That's what i observed ...
freak
 
Beiträge: 21
Registriert: Do Okt 10, 2013 10:59 pm

Re: Pending in collection : (What is that)

Beitragvon usern » Do Nov 28, 2013 11:56 pm

I am not crawling any sites at the moment, crawler queue is empty.
What I did was that I performed a lot of searches ~24 hours ago, I am using the option shallow crawl on http://localhost:8090/ConfigHeuristics_p.html and my intention was to fill up the crawler queue with some good stuff before i got to bed, what I did not know was that it would take me this long to process that queue.
I am now down to ~10000 pending in collection so I guess it should be finished when I wake up tomorrow.
usern
 
Beiträge: 13
Registriert: So Sep 23, 2012 12:33 pm


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 4 Gäste

cron