Es wird nicht kriechen. (English)

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

Es wird nicht kriechen. (English)

Beitragvon cnouvelle » Mi Feb 29, 2012 12:55 pm

Hello everyone! Many thanks for the terrific program and concept! Michael Christen suggested that I put my question to you here in the German forum. :)

I have put YaCy onto a workstation, that is on 192.168.1.8

I have another copy on a laptop, that is 192.168.1.5

Both computers run Ubuntu Linux 11.10

For three days, I could put URLs into the Crawler/Harvester of either one, no problem, it went right to work after a few seconds.
Starting yesterday, 1.8 exhibits a funny behavior. Over and over, the browser tab for Creaton Monitor alternates between "Connecting" and the title of the monitor page, YaCy'(name of my YaCy)': Crawler Queues, changing back and forth about once a second. It will not execute a crawl. Many hours later, it still does the same thing.

On 1.5, no problem, I can still start a crawl. But I don't want YaCy there, I want to use the workstation on 1.8

Other points: On YaCy Network diagram I see lots of red lines and lots of green lines. That suggests plenty of 'in' and plenty of 'out'.

"This Peer" now has 2.5 million documents, although that keeps growing even when I can't crawl. That must be normal.

Changing my profile name on 1.8 didn't solve it.

I rebooted the workstation on 1.8, that didn't solve it.

I am able to browse the web normally from either computer.

Thanks for your advice! :-)
cnouvelle
 
Beiträge: 32
Registriert: Mi Feb 29, 2012 12:42 pm

Re: Es wird nicht kriechen. (English)

Beitragvon cnouvelle » Mi Feb 29, 2012 10:05 pm

I have a potentially useful additional clue.

I successfully started remote crawl. It seems to crawl just fine from that queue.

I am still unable to initiate a local crawl.
cnouvelle
 
Beiträge: 32
Registriert: Mi Feb 29, 2012 12:42 pm

Re: Es wird nicht kriechen. (English)

Beitragvon cnouvelle » Mi Feb 29, 2012 11:30 pm

I set the scraping proxy to go, and I put the prefetch depth at 2. Now under creation monitor they are backing up like crazy. My installation of YaCy isn't clearing its local crawling queue.
cnouvelle
 
Beiträge: 32
Registriert: Mi Feb 29, 2012 12:42 pm

Re: Es wird nicht kriechen. (English)

Beitragvon cnouvelle » Mi Feb 29, 2012 11:52 pm

So just to review and summarize,

When I use the proxy with scraping proxy depth set to zero, the proxy catches my browsing (as long as no cookie was involved, etc.).

When I set the scraping proxy set to 2, the local crawler gets clogged, doesn't seem to clear. In other words the scraping proxy seems to rely on the local crawler.

When I use the local crawler, the entries don't clear. But, with expert crawling, I can set my crawl ideas to be done by others.

When I enable remote crawling, the entries seem to clear just fine, except for a few links that have the usual problems, dynamic content or whatever.

I don't know why my installation of YaCy can crawl for others but not for itself.
cnouvelle
 
Beiträge: 32
Registriert: Mi Feb 29, 2012 12:42 pm

Re: Es wird nicht kriechen. (English)

Beitragvon cnouvelle » Do Mär 01, 2012 4:52 am

Problem Solved.

On the Creation Monitor page, there is a chart that looks basically like this:


Local Crawler 81 Pause this queue unlimited
Limit Crawler 468 Pause this queue unlimited
Remote Crawler 47 Pause this queue unlimited
No-Load Crawler 0 Pause this queue unlimited
Loader 1 200

See where it says "Pause this queue"? That appears as a red button. When you click that, it becomes a green arrow.

I think that all this time, I had accidentally hit that with my mouse! :D All this time, the top one was a green arrow. I didn't think to look at it. But I was trying to browse, just for laughs, with pictures off, and then the browser loaded words. That's when I realized that there was a clickable option there, to pause and start the queues.

Mystery solved. Thanks. :o :D :D
cnouvelle
 
Beiträge: 32
Registriert: Mi Feb 29, 2012 12:42 pm


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast