Speichermangel und Startschwierigkeiten

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

Speichermangel und Startschwierigkeiten

Beitragvon Anmibe » So Jan 18, 2009 1:25 pm

Hallo,

irgendwie machen die 7.x Versionen keinen Spaß. Seit dem ich die benutzte geht eigentlich gar nichts mehr. Bereits nach wenigen Minuten (max 10) läuft Yacy in die Fehlermeldung
Code: Alles auswählen
>I 2009/01/17 23:10:53 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 2048 / 272 / 763 KB)

und dann geht eigentlich nichts mehr. Weder kann man über Proxy surfen, noch kann man auf die Oberfläche zugreifen. Ganz stehen tut es nicht, da ab und zu noch peer pings auftauchen aber arbeiten kann man nicht mehr mit Yacy.

Überhaupt ist mir im Laufe der Zeit aufgefallen, dass die Performance-Schwankungen von Version zu Version enorm sind. Ich hatte kürzlich eine Version (die 5485) die war richtig toll. Tagelang permanent gelaufen, man konnte über Proxy surfen und man konnte zusehen wie der Index wächst. Leider hat der Start der 7er diese Version gelöscht und ein erneuter Download ist nicht mehr möglich, da ihr sie aus dem Verzeichnis genommen habt.

Jetzt hat Yacy wieder Startschwierigkeiten, diesmal wohl mit Doubletten. Bei der ersten Meldung habe ich ihn einfach über Nacht gelassen, aber morgens hatte sich dann Yacy verabschiedet gehabt. Ich habe dann,wie dem Protokoll unten zu entehmen, ist mehrmals neu gestartet, aber Yacy reduziert zwar die Zahl der Doubletten (291.564 -> 64.857 -> 63.907 -> 63.874 -> 62.884 -> 30.546), macht dann selbstsändig (!) einen Shutdown - ich habe kein shutdown skript oder kill aufgerufen.
Code: Alles auswählen
     1   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
     2   -------------------------------------------------------------------------------
     3   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
     4   S 2009/01/17 23:30:09 STARTUP Java version: 1.5.0_16
     5   S 2009/01/17 23:30:10 STARTUP Operation system: Mac OS X
     6   S 2009/01/17 23:30:10 STARTUP Application root-path: /Applications/Kommunikation/yacy
     7   S 2009/01/17 23:30:10 STARTUP Time zone: UTC+0100; UTC+0000 is 1232231410149
     8   S 2009/01/17 23:30:10 STARTUP Maximum file system path length: 65535
     9   I 2009/01/17 23:30:10 PLASMA This is the pro-version of YaCy
    10   I 2009/01/17 23:30:11 indexContainerRAMHeap restoring rwi blob dump 'index.dhtout.blob'
    11   I 2009/01/17 23:30:19 indexContainerRAMHeap finished rwi blob restore: 100000 words, 689938 word/URL relations in 8023 milliseconds
    12   I 2009/01/17 23:30:20 indexContainerRAMHeap restoring rwi blob dump 'index.dhtin.blob'
    13   I 2009/01/17 23:30:20 indexContainerRAMHeap finished rwi blob restore: 16 words, 17 word/URL relations in 29 milliseconds
    14   D 2009/01/17 23:30:21 STARTUP OPENING COLLECTION INDEX
    15   I 2009/01/17 23:30:21 ECOTABLE initialization of /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: available RAM: 232MB, allocating space for 2271880 entries
    16   I 2009/01/17 23:30:21 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has table copy DISABLED
    17   I 2009/01/17 23:30:21 ECOTABLE initializing RAM index for EcoTable collection.index, please wait.
    18   I 2009/01/17 23:30:35 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 291564 doubles
    19   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
    20   -------------------------------------------------------------------------------
    21   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
    22   S 2009/01/18 10:26:52 STARTUP Java version: 1.5.0_16
    23   S 2009/01/18 10:26:52 STARTUP Operation system: Mac OS X
    24   S 2009/01/18 10:26:52 STARTUP Application root-path: /Applications/Kommunikation/yacy
    25   S 2009/01/18 10:26:52 STARTUP Time zone: UTC+0100; UTC+0000 is 1232270812889
    26   S 2009/01/18 10:26:52 STARTUP Maximum file system path length: 65535
    27   I 2009/01/18 10:26:53 PLASMA This is the pro-version of YaCy
    28   I 2009/01/18 10:26:54 indexContainerRAMHeap restoring rwi blob dump 'index.dhtout.blob'
    29   I 2009/01/18 10:27:05 indexContainerRAMHeap finished rwi blob restore: 100000 words, 689938 word/URL relations in 11016 milliseconds
    30   I 2009/01/18 10:27:07 indexContainerRAMHeap restoring rwi blob dump 'index.dhtin.blob'
    31   I 2009/01/18 10:27:07 indexContainerRAMHeap finished rwi blob restore: 16 words, 17 word/URL relations in 42 milliseconds
    32   D 2009/01/18 10:27:07 STARTUP OPENING COLLECTION INDEX
    33   I 2009/01/18 10:27:07 ECOTABLE initialization of /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: available RAM: 232MB, allocating space for 1998043 entries
    34   I 2009/01/18 10:27:08 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has table copy DISABLED
    35   I 2009/01/18 10:27:08 ECOTABLE initializing RAM index for EcoTable collection.index, please wait.
    36   I 2009/01/18 10:27:28 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 64857 doubles
    37   S 2009/01/18 10:27:41 SHUTDOWN goodbye. (this is the last line)
    38   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
    39   -------------------------------------------------------------------------------
    40   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
[41-54 gelöscht]
    55   I 2009/01/18 10:28:58 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 63907 doubles
    56   S 2009/01/18 10:29:56 SHUTDOWN goodbye. (this is the last line)
    57   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
    58   -------------------------------------------------------------------------------
    59   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
[60-73 gelöscht]
    74   I 2009/01/18 10:30:59 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 63874 doubles
    75   S 2009/01/18 10:31:58 SHUTDOWN goodbye. (this is the last line)
    76   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
    77   -------------------------------------------------------------------------------
    78   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
[79-92 gelöscht]
    93   I 2009/01/18 10:35:43 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 62884 doubles
    94   S 2009/01/18 10:36:53 SHUTDOWN goodbye. (this is the last line)
    95   [ YaCy v0.71, build 20090117 by Michael Christen / http://www.yacy.net ]
    96   -------------------------------------------------------------------------------
    97   STARTUP: Trying to load logging configuration from file /Applications/Kommunikation/yacy/DATA/LOG/yacy.logging
[98-111 gelöscht]
   112   I 2009/01/18 12:41:31 ECOTABLE /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /Applications/Kommunikation/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 30546 doubles
   113   S 2009/01/18 12:41:41 SHUTDOWN goodbye. (this is the last line)
Anmibe
 
Beiträge: 48
Registriert: Mo Nov 24, 2008 12:44 pm
Wohnort: Berlin

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » So Jan 18, 2009 1:39 pm

Seit der Version 0.7 wird "Maximum number of words in cache" nicht mehr so schnell geleert. Wenn hier ein großer Wert drin steht wird dem System mitunter viel Speicher abgezwackt der vorher durch das ständige entleeren vielleicht frei war.

Ansonsten wird YaCy eher immer mehr Speicher verbrauchen als weniger weil die DB ja nicht kleiner wird. Wenn es dann knapp wird den Crawler abschalten und beim DHT eine Zeit lang nur Daten verteilen.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon Anmibe » So Jan 18, 2009 7:25 pm

thq hat geschrieben:Ansonsten wird YaCy eher immer mehr Speicher verbrauchen als weniger weil die DB ja nicht kleiner wird. Wenn es dann knapp wird den Crawler abschalten und beim DHT eine Zeit lang nur Daten verteilen.

Sind genau diese Aufgaben aber nicht Aufgaben von Yacy? Mit dieser Argumentation steigt der Speicherbedarf von Yacy gegen Unendlich. Wie soll ich eigentlich den Crawler abschalten wenn ich über das Browserinterface keinen Zugriff mehr habe? Gibt's einen Befehl für die Kommandozeile?
Man kann ja auch keine neue Version mehr heruterladen, da man Link nicht kennt (Datum & Versionsnummer). Hierzu müsste man dann kurzfristig auf eine neues leeres Datenverzeichnis umbiegen udn nach dem Download wieder zurück.
Anmibe
 
Beiträge: 48
Registriert: Mo Nov 24, 2008 12:44 pm
Wohnort: Berlin

Re: Speichermangel und Startschwierigkeiten

Beitragvon awxcnx » Do Jan 22, 2009 12:48 pm

Ich habe das gleiche Problem mit 2 Peers sobald ich einen Crawl starte. Der Indexierer läuft hoch und nach wenigen Minuten geht nichts mehr - kein freier Speicher. Yacy kann auf beiden Peers 260 MB RAM nutzen. In der DB sind 4 Mio und 2 Mio Links, das ist nicht viel. Sytem ist Debian/etch mit sun-jre 1.5.

Kann man die Anzahl der URLs im Indexierer begrenzen?
awxcnx
 
Beiträge: 2
Registriert: Do Jan 22, 2009 12:40 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon Phiber » So Jan 25, 2009 12:57 am

habe das selbe Problem:

Ohne Crawl 600MB:

Neuer Crawl gestartet und nach 1min sind die 1200MB voll und er meckert, dass er den einzelnen Jobs (also coreCrawl z.b.) nichtmal mehr den Mimimalspeicher zuweisen kann.

Tjo und dann geht auch relativ schnell gar nix mehr, kein Webinterface rein gar nichts.

Ist das ein irgend ein Problem oderhabe ich einfachfür einen 20Mio-Index zu hohe Werte bzw reichen die 1200MB RAM für YaCy nicht mehr?
Wobei es eigentlich am Crawler liegen müsste, also lasse ich vielleicht viel zu schnell crawlen und habe zu grosse Queues (diese PreQueues vielleicht) ? Aber auch hier sind glaubs die Standardwerte drin, ausser vielleicht die Crawl-Geschwindigkeiten, was ja aber weniger mit dem Speicher zu tun hat.
Dateianhänge
crawl1_phiber.JPG
so sieht das aus wenn ich nen crawl starte und dann geht nix mehr
crawl1_phiber.JPG (160.99 KiB) 9124-mal betrachtet
Phiber
 
Beiträge: 96
Registriert: So Okt 05, 2008 9:04 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » So Jan 25, 2009 6:06 am

Betrifft ja auch den Indexer, der läuft ab 0.7 nachvollziehbar voll, und dann geht im Bezug auf Crawl garnichts mehr, so wie ich das sehe weil er Intern mit Speicherknappheit kämpft.

Auch ist der Speicherbedarf ab 0.7 viel höher, ich hatte im reinen DHT-Betrieb bei fast gleichem Index über 400 von 1200MB frei (bei 25mio-links), jetzt nur noch bischen mehr als 100.

Crawlen geht garnicht mehr, zumindest nicht mehr als Tiefe 2, da sofort der Indexer übervoll.

Habe so ziemlich alle Versionen ab 5494 versucht, keine Änderung, egal ob Win/Linux, grosser Peer/kleiner Peer, frisch aufgesetzt...

Siehe auch: viewtopic.php?f=6&t=1763

Gerade mal einen Crawl gestartet:
Code: Alles auswählen

Prozess
    0.710/05511
    Online seit: 2 days 21:08
Systemressourcen
    Prozessoren: 4
Speicherverbrauch
    Frei: 1,05 GB
    Total: 2 GB
    Maximal: 2 GB

E 2009/01/25 06:15:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:19 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:21 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:26 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:30 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:31 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:33 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:37 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:42 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:52 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:53 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:15:53 PLASMA Received 294 Entries 60 Words [BMi9D9Z1oHZs .. BMq64f8gGiSi]/231689437920935964 from 2O98BfRXnM2g:rrzn3/0.61505365, processed in 27 milliseconds, requesting 21/293 URLs, blocked 0 RWIs

E 2009/01/25 06:15:54 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/transferURL.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/01/25 06:15:54 FILEHANDLER Unexpected error while processing query.
Session: Session_130.75.2.29:35768#1
Query:   /yacy/transferURL.html
Client:  130.75.2.29
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:57 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:15:59 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:03 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:04 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:08 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:10 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:14 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:17 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:19 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:25 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:30 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:16:31 PLASMA Received 302 Entries 61 Words [CAItGcOaxiOT .. CARpntUVvkyJ]/115502181527195048 from Nl82Di69OOlY:gpfYACy/0.70005494, processed in 30 milliseconds, requesting 115/301 URLs, blocked 0 RWIs

E 2009/01/25 06:16:32 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:33 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/transferURL.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/01/25 06:16:33 FILEHANDLER Unexpected error while processing query.
Session: Session_85.25.141.60:58076#1
Query:   /yacy/transferURL.html
Client:  85.25.141.60
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:36 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:39 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:43 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:50 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:52 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:54 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:16:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:01 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:03 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:05 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:07 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:17:13 PLASMA Received 1000 Entries 100 Words [AH6Luy8xxYSf .. AISLBICFhVni]/385964442161273868 from DvgzXMfS8oDp:yacy-suche_de/0.7100552, processed in 86 milliseconds, requesting 334/988 URLs, blocked 0 RWIs

E 2009/01/25 06:17:14 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:15 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/transferURL.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/01/25 06:17:15 FILEHANDLER Unexpected error while processing query.
Session: Session_78.46.105.117:45764#1
Query:   /yacy/transferURL.html
Client:  78.46.105.117
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:23 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:25 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:31 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:33 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:36 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:40 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:42 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:17:46 PLASMA RankingDistribution - error transmitting file /home/vmadmin/yacy/DATA/RANKING/GLOBAL/010_owncr/CRG-A-20090122125712322.# Name=YaCy .cr.gz to 195.202.180.115:8080: remote peer failed with transfer: transfer failure

E 2009/01/25 06:17:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:17:47 PLASMA RankingDistribution - error transmitting file /home/vmadmin/yacy/DATA/RANKING/GLOBAL/010_owncr/CRG-A-20090122125712322.# Name=YaCy .cr.gz to 87.145.29.105:8080: remote peer failed with transfer: transfer failure

I 2009/01/25 06:17:48 PLASMA RankingDistribution - error transmitting file /home/vmadmin/yacy/DATA/RANKING/GLOBAL/010_owncr/CRG-A-20090122125712322.# Name=YaCy .cr.gz to 84.60.167.229:8088: remote peer failed with transfer: transfer failure

E 2009/01/25 06:17:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:17:50 PLASMA RankingDistribution - error transmitting file /home/vmadmin/yacy/DATA/RANKING/GLOBAL/010_owncr/CRG-A-20090122125712322.# Name=YaCy .cr.gz to 84.176.27.184:8080: remote peer failed with transfer: transfer failure

E 2009/01/25 06:17:51 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:53 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:17:54 PLASMA RankingDistribution - error transmitting file /home/vmadmin/yacy/DATA/RANKING/GLOBAL/010_owncr/CRG-A-20090122125712322.# Name=YaCy .cr.gz to 86.33.89.35:8080: remote peer failed with transfer: transfer failure

E 2009/01/25 06:17:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

D 2009/01/25 06:17:55 KELONDRO file '/home/vmadmin/yacy/DATA/INDEX/freeworld/NETWORK/newsProcessed.stack' closed.

D 2009/01/25 06:17:55 KELONDRO file '/home/vmadmin/yacy/DATA/INDEX/freeworld/NETWORK/newsPublished.stack' closed.

I 2009/01/25 06:17:55 YACY rulebasedUpdateInfo: not an automatic update selected

I 2009/01/25 06:17:55 RESOURCE OBSERVER df of Volume /: 187649 MB

I 2009/01/25 06:17:55 RESOURCE OBSERVER run completed; everything in order

E 2009/01/25 06:17:56 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:17:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:04 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:08 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:13 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:17 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:30 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:32 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:33 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:40 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:43 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:45 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:51 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:52 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:54 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:56 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:18:57 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:03 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:05 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:07 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:14 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

D 2009/01/25 06:19:16 ROBOTS Trying to download the robots.txt file from URL 'http://www.das-parlament.de/robots.txt'.

D 2009/01/25 06:19:16 ROBOTS robots.txt could not be downloaded from URL 'http://www.das-parlament.de/robots.txt'. [404 Object Not Found].

E 2009/01/25 06:19:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:19 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:25 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:29 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:31 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:33 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:36 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:40 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

D 2009/01/25 06:19:44 CRAWLER LOCALCRAWL[1, 0, 0, 0]: URL=http://www.das-parlament.de/2009/index.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=0, crawlDepth=4, must-match=.*, must-not-match=, permission=true

I 2009/01/25 06:19:44 CRAWLER LOCALCRAWL[1, 0, 0, 0]: enqueued for load http://www.das-parlament.de/2009/index.html [8uDApqaIZJ7C]

E 2009/01/25 06:19:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:51 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:53 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:57 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:19:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:03 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: null
java.util.ConcurrentModificationException
   at java.util.TreeMap$NavigableSubMap$SubMapIterator.nextEntry(TreeMap.java:1572)
   at java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1620)
   at java.util.TreeMap$NavigableSubMap$SubMapEntryIterator.next(TreeMap.java:1614)
   at java.util.AbstractMap$2$1.next(AbstractMap.java:360)
   at de.anomic.index.indexContainerRAMHeap$heapCacheIterator.next(indexContainerRAMHeap.java:370)
   at de.anomic.index.indexContainerRAMHeap$heapCacheIterator.next(indexContainerRAMHeap.java:343)
   at de.anomic.kelondro.kelondroRotateIterator.next(kelondroRotateIterator.java:62)
   at de.anomic.plasma.plasmaWordIndex.indexContainerSet(plasmaWordIndex.java:772)
   at de.anomic.plasma.plasmaDHTChunk.selectTransferContainersResource(plasmaDHTChunk.java:152)
   at de.anomic.plasma.plasmaDHTChunk.selectTransferContainers(plasmaDHTChunk.java:134)
   at de.anomic.plasma.plasmaDHTChunk.<init>(plasmaDHTChunk.java:114)
   at de.anomic.plasma.plasmaSwitchboard.deQueueProcess(plasmaSwitchboard.java:1187)
   at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
   at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:142)

I 2009/01/25 06:20:03 PLASMA Received 385 Entries 63 Words [_GvAEx8b7MyN .. _IuqaHnmOZJr]/530901038532764972 from 46o-9s3mgVAA:tp-work242/0.70005494, processed in 198 milliseconds, requesting 2/362 URLs, blocked 0 RWIs

E 2009/01/25 06:20:04 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/transferURL.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/01/25 06:20:04 FILEHANDLER Unexpected error while processing query.
Session: Session_217.91.22.18:61411#1
Query:   /yacy/transferURL.html
Client:  217.91.22.18
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:05 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/25 06:20:06 PLASMA Received 1000 Entries 56 Words [AYO-53OmJG61 .. AbxvjvpCAoIe]/345636633336882440 from AUWUohdhhWAA:ZZZ/0.7100552, processed in 1569 milliseconds, requesting 632/931 URLs, blocked 0 RWIs

E 2009/01/25 06:20:08 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/transferURL.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/01/25 06:20:08 FILEHANDLER Unexpected error while processing query.
Session: Session_194.204.30.253:63971#1
Query:   /yacy/transferURL.html
Client:  194.204.30.253
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:260)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:626)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:10 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:14 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:17 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:19 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:21 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:26 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/25 06:20:30 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Do Jan 29, 2009 12:11 pm

Hier 2 Auszüge aus dem Log, des erste nach ca. 2 Tagen Laufzeit, keinerlei Crawls, kein DHT, dort habe ich dann einen Crawl Tiefe 3 mit Remote-indexierung gestartet.
Starturl wird kurz gecrawlt, gelangt nicht in den Indexer, aber folgende Ausgabe im Log dann:

Code: Alles auswählen
E 2009/01/29 03:59:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 03:59:29 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 03:59:42 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 03:59:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:00:08 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:00:21 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:00:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:00:45 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:00:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:01:08 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:01:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:01:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:01:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:01:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:02:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:02:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:02:36 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:02:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:26 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:38 YACY yacyClient.queryUrlCount error asking peer '172-23-31-120-16dpnw98':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:03:38 YACY hello: responded remote junior peer '172-23-31-120-16dpnw98' from 91.113.92.7

E 2009/01/29 04:03:40 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:53 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:03:53 YACY yacyClient.queryUrlCount error asking peer '172-23-31-120-16dpnw98':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:03:53 YACY hello: responded remote junior peer '172-23-31-120-16dpnw98' from 91.113.92.7

E 2009/01/29 04:04:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:04:09 YACY yacyClient.queryUrlCount error asking peer '172-23-31-120-16dpnw98':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:04:09 YACY hello: responded remote junior peer '172-23-31-120-16dpnw98' from 91.113.92.7

E 2009/01/29 04:04:20 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:04:25 YACY yacyClient.queryUrlCount error asking peer '172-23-31-120-16dpnw98':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:04:25 YACY hello: responded remote junior peer '172-23-31-120-16dpnw98' from 91.113.92.7

I 2009/01/29 04:04:32 YACY yacyClient.publishMySeed thread 'PublishSeed_raumquantenmotor' contacted peer at 195.177.250.222:8080, received 1257 bytes, time = 297 milliseconds

I 2009/01/29 04:04:32 YACY yacyClient.publishMySeed: Peer 'raumquantenmotor' reported us as junior.

I 2009/01/29 04:04:32 YACY publish: handshaked senior peer 'raumquantenmotor' at 195.177.250.222:8080

E 2009/01/29 04:04:33 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/29 04:04:33 YACY yacyClient.publishMySeed thread 'PublishSeed_gpfYACy' contacted peer at 85.25.141.60:8080, received 9357 bytes, time = 1516 milliseconds

I 2009/01/29 04:04:34 YACY connect: SELF reference 92.205.75.164:8080

I 2009/01/29 04:04:34 YACY publish: handshaked senior peer 'gpfYACy' at 85.25.141.60:8080

E 2009/01/29 04:04:34 YACY yacyClient.permissionTransfer error:Connection refused: connect

I 2009/01/29 04:04:34 PLASMA RankingDistribution - error transmitting file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080712192745252.onrmkrons91A.cr.gz to kaskelix.de:8080: no connection to remote address kaskelix.de:8080; phase 1

W 2009/01/29 04:04:34 YACY yacyClient.publishMySeed thread 'PublishSeed_127-0-0-1-204dpnufe65' exception: Connection refused: connect; retry = 0

I 2009/01/29 04:04:34 PLASMA RankingDistribution - error transmitting file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080712192745252.onrmkrons91A.cr.gz to yacy.dyndns.org:8000: remote peer rejected transfer: denied

I 2009/01/29 04:04:34 PLASMA RankingDistribution - transmitted file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080712192745252.onrmkrons91A.cr.gz to 92.205.75.164:8091 successfully in 0 seconds

W 2009/01/29 04:04:36 YACY yacyClient.publishMySeed thread 'PublishSeed_127-0-0-1-204dpnufe65' exception: Connection refused: connect; retry = 1

D 2009/01/29 04:04:36 KELONDRO file 'D:\yacy\data\index\freeworld\NETWORK\newsProcessed.stack' closed.

D 2009/01/29 04:04:36 KELONDRO file 'D:\yacy\data\index\freeworld\NETWORK\newsPublished.stack' closed.

I 2009/01/29 04:04:36 YACY rulebasedUpdateInfo: not an automatic update selected

I 2009/01/29 04:04:36 RESOURCE OBSERVER df of Volume D: 26807 MB

I 2009/01/29 04:04:36 RESOURCE OBSERVER run completed; everything in order

W 2009/01/29 04:04:38 YACY yacyClient.publishMySeed thread 'PublishSeed_127-0-0-1-204dpnufe65' exception: Connection refused: connect; retry = 2

W 2009/01/29 04:04:40 YACY yacyClient.publishMySeed thread 'PublishSeed_127-0-0-1-204dpnufe65' exception: Connection refused: connect; retry = 3

I 2009/01/29 04:04:40 YACY publish: disconnected senior peer '127-0-0-1-204dpnufe65' from 77.243.46.253:8080: peer ping to peer resulted in error response (added < 0)

I 2009/01/29 04:04:40 YACY PeerPing: I am accessible for 2 peer(s), not accessible for 1 peer(s).

I 2009/01/29 04:04:40 YACY PeerPing: myType is senior

E 2009/01/29 04:04:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:04:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:05:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:05:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:05:33 KELONDRO KELONDRO FAILURE D:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.02.00.kca: INCONSISTENCY (get) in d:\yacy\DATA\INDEX\freeworld\TEXT\RICOLLECTION\collection.04.0028.02.00.kca: array has different chunkcount than index: index = 29, array = 36; the index has been auto-fixed (error #121)

E 2009/01/29 04:05:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:05:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:05:59 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:06:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:06:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:06:38 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:06:52 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:07:05 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:07:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:07:31 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:07:43 YACY yacyClient.queryUrlCount error asking peer 'ZZZ':java.net.SocketException: Connection reset

E 2009/01/29 04:07:44 YACY yacyClient.queryUrlCount error asking peer 'ZZZ':java.net.SocketException: Connection reset

I 2009/01/29 04:07:44 YACY hello: responded remote junior peer 'ZZZ' from 194.204.0.26

E 2009/01/29 04:07:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:07:57 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:08:10 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:08:23 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:08:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:08:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:08:59 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:09:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:09:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:09:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:09:49 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:10:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:10:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:10:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:10:36 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:10:45 YACY yacyClient.queryUrlCount error asking peer 'flexnode':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:10:45 YACY hello: responded remote junior peer 'flexnode' from 86.32.81.104

E 2009/01/29 04:10:50 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:11:00 YACY yacyClient.queryUrlCount error asking peer 'flexnode':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:11:00 YACY hello: responded remote junior peer 'flexnode' from 86.32.81.104

E 2009/01/29 04:11:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:11:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:11:15 YACY yacyClient.queryUrlCount error asking peer 'flexnode':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:11:15 YACY hello: responded remote junior peer 'flexnode' from 86.32.81.104

E 2009/01/29 04:11:29 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:11:30 YACY yacyClient.queryUrlCount error asking peer 'flexnode':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:11:30 YACY hello: responded remote junior peer 'flexnode' from 86.32.81.104

E 2009/01/29 04:11:43 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:12:00 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:12:13 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:12:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:12:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/01/29 04:12:44 YACY yacyClient.publishMySeed thread 'PublishSeed_Ananas' contacted peer at 88.153.171.159:8000, received 10089 bytes, time = 360 milliseconds

I 2009/01/29 04:12:44 YACY yacyClient.publishMySeed thread 'PublishSeed_vulcanus' contacted peer at 87.187.85.59:8080, received 10138 bytes, time = 375 milliseconds

I 2009/01/29 04:12:44 YACY connect: SELF reference 92.205.75.164:8080

I 2009/01/29 04:12:44 YACY connect: SELF reference 92.205.75.164:8080

I 2009/01/29 04:12:44 YACY publish: handshaked principal peer 'vulcanus' at 87.187.85.59:8080

I 2009/01/29 04:12:44 YACY publish: handshaked senior peer 'Ananas' at 88.153.171.159:8000

I 2009/01/29 04:12:44 YACY PeerPing: I am accessible for 3 peer(s), not accessible for 1 peer(s).

I 2009/01/29 04:12:44 YACY PeerPing: myType is senior

E 2009/01/29 04:12:45 YACY yacyClient.permissionTransfer error:Connection refused: connect

I 2009/01/29 04:12:45 PLASMA RankingDistribution - error transmitting file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080925020104969.# Name=YaCy .cr.gz to kaskelix.de:8080: no connection to remote address kaskelix.de:8080; phase 1

I 2009/01/29 04:12:46 PLASMA RankingDistribution - error transmitting file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080925020104969.# Name=YaCy .cr.gz to yacy.dyndns.org:8000: remote peer rejected transfer: denied

I 2009/01/29 04:12:49 PLASMA RankingDistribution - transmitted file d:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080925020104969.# Name=YaCy .cr.gz to 194.204.62.215:8080 successfully in 2 seconds

D 2009/01/29 04:12:50 KELONDRO file 'D:\yacy\data\index\freeworld\NETWORK\newsProcessed.stack' closed.

D 2009/01/29 04:12:50 KELONDRO file 'D:\yacy\data\index\freeworld\NETWORK\newsPublished.stack' closed.

I 2009/01/29 04:12:50 YACY rulebasedUpdateInfo: not an automatic update selected

I 2009/01/29 04:12:50 RESOURCE OBSERVER df of Volume D: 26805 MB

I 2009/01/29 04:12:50 RESOURCE OBSERVER run completed; everything in order

E 2009/01/29 04:12:54 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:13:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:13:19 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:13:27 YACY yacyClient.queryUrlCount error asking peer '152-67-136-66-62dpnw0':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:13:27 YACY hello: responded remote junior peer '152-67-136-66-62dpnw0' from 66.179.208.36

E 2009/01/29 04:13:32 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:13:42 YACY yacyClient.queryUrlCount error asking peer '152-67-136-66-62dpnw0':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:13:42 YACY hello: responded remote junior peer '152-67-136-66-62dpnw0' from 66.179.208.36

E 2009/01/29 04:13:45 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:13:58 YACY yacyClient.queryUrlCount error asking peer '152-67-136-66-62dpnw0':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:13:58 YACY hello: responded remote junior peer '152-67-136-66-62dpnw0' from 66.179.208.36

E 2009/01/29 04:13:59 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:14:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:14:14 YACY yacyClient.queryUrlCount error asking peer '152-67-136-66-62dpnw0':java.net.ConnectException: Connection timed out: connect

I 2009/01/29 04:14:14 YACY hello: responded remote junior peer '152-67-136-66-62dpnw0' from 66.179.208.36

E 2009/01/29 04:14:22 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:14:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:14:46 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:14:59 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:15:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:15:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:15:37 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:15:52 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:16:04 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:16:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:16:32 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:16:44 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:16:58 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:17:10 KELONDRO KELONDRO FAILURE D:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.00.00.kca: INCONSISTENCY (get) in d:\yacy\DATA\INDEX\freeworld\TEXT\RICOLLECTION\collection.04.0028.00.00.kca: array has different chunkcount than index: index = 2, array = 3; the index has been auto-fixed (error #122)

E 2009/01/29 04:17:11 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:17:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:17:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:17:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:18:01 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:18:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:18:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:18:43 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:18:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:19:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:19:21 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:19:34 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:19:47 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:20:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:20:17 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:20:28 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:20:41 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

D 2009/01/29 04:20:42 ROBOTS Trying to download the robots.txt file from URL 'http://gborn.blogger.de/robots.txt'.

D 2009/01/29 04:20:42 ROBOTS Robots.txt from URL 'http://gborn.blogger.de/robots.txt' was not modified. Abort downloading of new version.

E 2009/01/29 04:20:55 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

E 2009/01/29 04:21:07 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space


Auf der Maschine war zu dem Zeitpunkt ca 400MB von 1300MB frei. (Java)
Das BS hatte noch gut 600M frei, das Java aber unter w32 nicht nutzen kann.
Müsste ja aber eigentlich trotzdem ausreichen.

Mache ich jetzt einen Neustart dann startet der Crawl, diesmal richtig mit folgender Ausgabe im Log:

Code: Alles auswählen
S 2009/01/29 04:27:38 CRAWLER Starting Crawling Management

I 2009/01/29 04:27:38 ECOTABLE initialization of D:\yacy\DATA\PLASMADB\urlNoticeCoreStack9.db: available RAM: 479MB, allocating space for 0 entries

I 2009/01/29 04:27:38 ECOTABLE D:\yacy\DATA\PLASMADB\urlNoticeCoreStack9.db: EcoTable D:\yacy\DATA\PLASMADB\urlNoticeCoreStack9.db has table copy DISABLED

I 2009/01/29 04:27:38 ECOTABLE initializing RAM index for EcoTable urlNoticeCoreStack9.db, please wait.

I 2009/01/29 04:27:38 ECOTABLE initialization of D:\yacy\DATA\PLASMADB\urlNoticeLimitStack9.db: available RAM: 478MB, allocating space for 0 entries

I 2009/01/29 04:27:38 ECOTABLE D:\yacy\DATA\PLASMADB\urlNoticeLimitStack9.db: EcoTable D:\yacy\DATA\PLASMADB\urlNoticeLimitStack9.db has table copy DISABLED

I 2009/01/29 04:27:38 ECOTABLE initializing RAM index for EcoTable urlNoticeLimitStack9.db, please wait.

I 2009/01/29 04:27:38 ECOTABLE initialization of D:\yacy\DATA\PLASMADB\urlNoticeRemoteStack9.db: available RAM: 476MB, allocating space for 0 entries

I 2009/01/29 04:27:38 ECOTABLE D:\yacy\DATA\PLASMADB\urlNoticeRemoteStack9.db: EcoTable D:\yacy\DATA\PLASMADB\urlNoticeRemoteStack9.db has table copy DISABLED

I 2009/01/29 04:27:38 ECOTABLE initializing RAM index for EcoTable urlNoticeRemoteStack9.db, please wait.

I 2009/01/29 04:27:38 ECOTABLE initialization of D:\yacy\DATA\PLASMADB\urlError3.db: available RAM: 475MB, allocating space for 150078 entries

I 2009/01/29 04:27:38 ECOTABLE D:\yacy\DATA\PLASMADB\urlError3.db: EcoTable D:\yacy\DATA\PLASMADB\urlError3.db has table copy DISABLED

I 2009/01/29 04:27:38 ECOTABLE initializing RAM index for EcoTable urlError3.db, please wait.

I 2009/01/29 04:27:41 ECOTABLE initialization of D:\yacy\DATA\PLASMADB\urlDelegated3.db: available RAM: 458MB, allocating space for 0 entries

I 2009/01/29 04:27:41 ECOTABLE D:\yacy\DATA\PLASMADB\urlDelegated3.db: EcoTable D:\yacy\DATA\PLASMADB\urlDelegated3.db has table copy DISABLED

I 2009/01/29 04:27:41 ECOTABLE initializing RAM index for EcoTable urlDelegated3.db, please wait.

I 2009/01/29 04:27:41 RESOURCE OBSERVER initializing the resource observer

I 2009/01/29 04:27:42 RESOURCE OBSERVER df of Volume D: 26780 MB

I 2009/01/29 04:27:42 RESOURCE OBSERVER run completed; everything in order

I 2009/01/29 04:27:42 STACKCRAWL STACKCRAWL thread initialized.

S 2009/01/29 04:27:43 STARTUP YACY Version: 0.71005511, Built 20090122

I 2009/01/29 04:27:43 MIGRATION copied new Skinfile

D 2009/01/29 04:27:43 STARTUP deleting downloaded releases older than 30 days

I 2009/01/29 04:27:45 SERVER Initializing serverCore ...

I 2009/01/29 04:27:45 SERVER Trying to bind server to port 8080

I 2009/01/29 04:27:46 PROXY Configuring proxy access logging ...

I 2009/01/29 04:27:46 PROXY Proxy access logging configuration done.
   Filename: DATA/LOG/proxyAccess%u%g.log
   Limit: 1048576
   Count: 20

S 2009/01/29 04:27:46 HTTPDFiles Loading mime mapping file httpd.mime

I 2009/01/29 04:27:47 STARTUP Copied the default locales to D:\yacy\DATA\LOCALE\locales

S 2009/01/29 04:27:47 STARTUP Registering Shutdown Hook

I 2009/01/29 04:27:48 YACY BOOTSTRAP: 0 seeds from seed-list URL http://www.yacy.net/seed.txt, AGE=2360h

I 2009/01/29 04:27:48 YACY BOOTSTRAP: 2 seeds from seed-list URL http://home.arcor.de/hermens/yacy/seed.txt, AGE=3h

I 2009/01/29 04:27:48 YACY BOOTSTRAP: 52 seeds from seed-list URL http://low.audioattack.de/yacy/seed.txt, AGE=0h

I 2009/01/29 04:27:48 YACY BOOTSTRAP: 16 seeds from seed-list URL http://www.lulabad.de/seed.txt, AGE=0h

I 2009/01/29 04:27:48 YACY BOOTSTRAP: 4 new seeds while bootstraping.

W 2009/01/29 04:27:53 PLASMA deQueue: null entry on queue stack.

E 2009/01/29 04:27:53 PLASMA deQueue: does not shrink after pop() == null. Emergency reset.

D 2009/01/29 04:27:53 KELONDRO file 'D:\yacy\data\index\freeworld\QUEUES\urlNoticePreStack' closed.

I 2009/01/29 04:27:53 PLASMA CRAWL: ADDED 210 LINKS FROM http://gborn.blogger.de/, NEW CRAWL STACK SIZE IS 0, STACKING TIME = 47, PARSING TIME = 203

E 2009/01/29 04:27:54 SERVER receive interrupted - exception 2 = Connection reset

E 2009/01/29 04:27:54 LanguageStatistics the language statistics folder langstats cannot be found

I 2009/01/29 04:27:54 PLASMA Excluded 0 words in URL http://gborn.blogger.de/

I 2009/01/29 04:27:54 PLASMA *Indexed 524 words in URL http://gborn.blogger.de/ [hdFWwMKNSYZA]
   Description:  Borns IT und Vista Blog
   MimeType: text/html | Charset: ISO-8859-1 | Size: 10060 bytes | Anchors: 191
   LinkStorageTime: 47 ms | indexStorageTime: 47 ms

D 2009/01/29 04:27:55 CRAWLER LOCALCRAWL[1, 0, 0, 0]: URL=http://gborn.blogger.de/stories/1317595/, initiator=148qJoTxccAZ, crawlOrder=true, depth=1, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:55 CRAWLER LOCALCRAWL[1, 0, 0, 0]: enqueued for load http://gborn.blogger.de/stories/1317595/ [Hg06InKNSYZA]

I 2009/01/29 04:27:56 kelondroBLOBHeap wrote a dump for the 5802 index entries of 20081104212928.blob in 31 milliseconds.

I 2009/01/29 04:27:56 kelondroBLOBHeap wrote a dump for the 8590 index entries of 20081105015221.blob in 0 milliseconds.

I 2009/01/29 04:27:56 kelondroBLOBHeap wrote a dump for the 9134 index entries of 20081105074612.blob in 0 milliseconds.

I 2009/01/29 04:27:56 PLASMA CRAWL: ADDED 22 LINKS FROM http://gborn.blogger.de/stories/1317595/, NEW CRAWL STACK SIZE IS 1, STACKING TIME = 0, PARSING TIME = 94

I 2009/01/29 04:27:56 PLASMA Excluded 0 words in URL http://gborn.blogger.de/stories/1317595/

I 2009/01/29 04:27:56 PLASMA *Indexed 260 words in URL http://gborn.blogger.de/stories/1317595/ [Hg06InKNSYZA]
   Description:  Borns IT und Vista Blog: Spracherweiterungen für Home Premium?
   MimeType: text/html | Charset: ISO-8859-1 | Size: 4147 bytes | Anchors: 19
   LinkStorageTime: 0 ms | indexStorageTime: 31 ms

D 2009/01/29 04:27:57 CRAWLER LOCALCRAWL[3, 0, 0, 0]: URL=http://gborn.blogger.de/stories/1317595/, initiator=148qJoTxccAZ, crawlOrder=true, depth=2, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:57 CRAWLER LOCALCRAWL[3, 0, 0, 0]: enqueued for load http://gborn.blogger.de/stories/1317595/ [Hg06InKNSYZA]

D 2009/01/29 04:27:57 CRAWLER LOCALCRAWL[2, 0, 0, 0]: URL=http://www.froggie.sk/, initiator=148qJoTxccAZ, crawlOrder=true, depth=2, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:57 CRAWLER LOCALCRAWL[2, 0, 0, 0]: enqueued for load http://www.froggie.sk/ [y75wJQmdd5vA]

D 2009/01/29 04:27:57 ROBOTS Trying to download the robots.txt file from URL 'http://www.froggie.sk/robots.txt'.

I 2009/01/29 04:27:57 BALANCER forcing crawl-delay of 390 milliseconds for gborn.blogger.de

D 2009/01/29 04:27:58 ROBOTS Robots.txt successfully loaded from URL 'http://www.froggie.sk/robots.txt' in 234 ms.

D 2009/01/29 04:27:58 CRAWLER LOCALCRAWL[1, 0, 0, 0]: URL=http://gborn.blogger.de/stories/1317595/comment, initiator=148qJoTxccAZ, crawlOrder=true, depth=2, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:58 CRAWLER LOCALCRAWL[1, 0, 0, 0]: enqueued for load http://gborn.blogger.de/stories/1317595/comment [eoTIEnKNSYZA]

E 2009/01/29 04:27:58 YACY yacyClient.queryUrlCount error asking peer 'ZZZ':java.net.SocketException: Connection reset

E 2009/01/29 04:27:58 YACY yacyClient.queryUrlCount error asking peer 'ZZZ':java.net.SocketException: Connection reset

I 2009/01/29 04:27:58 YACY hello: responded remote junior peer 'ZZZ' from 194.204.0.26

I 2009/01/29 04:27:58 YACY hello: changing remote peer 'ZZZ' [194.204.0.26] peerType from 'principal' to 'junior'.

I 2009/01/29 04:27:58 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/

I 2009/01/29 04:27:58 PLASMA CRAWL: ADDED 22 LINKS FROM http://gborn.blogger.de/stories/1317595/, NEW CRAWL STACK SIZE IS 0, STACKING TIME = 0, PARSING TIME = 16

I 2009/01/29 04:27:58 PLASMA CRAWL: ADDED 24 LINKS FROM http://www.froggie.sk/, NEW CRAWL STACK SIZE IS 0, STACKING TIME = 0, PARSING TIME = 31

I 2009/01/29 04:27:58 PLASMA Excluded 0 words in URL http://gborn.blogger.de/stories/1317595/

I 2009/01/29 04:27:58 PLASMA CRAWL: ADDED 19 LINKS FROM http://gborn.blogger.de/stories/1317595/comment, NEW CRAWL STACK SIZE IS 7, STACKING TIME = 0, PARSING TIME = 63

I 2009/01/29 04:27:58 PLASMA *Indexed 260 words in URL http://gborn.blogger.de/stories/1317595/ [Hg06InKNSYZA]
   Description:  Borns IT und Vista Blog: Spracherweiterungen für Home Premium?
   MimeType: text/html | Charset: ISO-8859-1 | Size: 4147 bytes | Anchors: 19
   LinkStorageTime: 46 ms | indexStorageTime: 16 ms

I 2009/01/29 04:27:58 PLASMA Excluded 0 words in URL http://www.froggie.sk/

I 2009/01/29 04:27:58 PLASMA *Indexed 128 words in URL http://www.froggie.sk/ [y75wJQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 1957 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:27:59 PLASMA Excluded 0 words in URL http://gborn.blogger.de/stories/1317595/comment

I 2009/01/29 04:27:59 PLASMA *Indexed 100 words in URL http://gborn.blogger.de/stories/1317595/comment [eoTIEnKNSYZA]
   Description:  Login
   MimeType: text/html | Charset: ISO-8859-1 | Size: 1264 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 0 ms

D 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[18, 0, 0, 0]: URL=http://www.froggie.sk/dvd.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[18, 0, 0, 0]: enqueued for load http://www.froggie.sk/dvd.html [Vuf0_Qmdd5vA]

D 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[17, 0, 0, 0]: URL=http://gborn.blogger.de/stories/1317595/comment, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[17, 0, 0, 0]: enqueued for load http://gborn.blogger.de/stories/1317595/comment [eoTIEnKNSYZA]

D 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[16, 0, 0, 0]: URL=http://translate.google.com/translate_t?sl=en, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[16, 0, 0, 0]: enqueued for load http://translate.google.com/translate_t?sl=en [zsHVejNUA7rY]

I 2009/01/29 04:27:59 BALANCER forcing crawl-delay of 343 milliseconds for http://www.froggie.sk

I 2009/01/29 04:27:59 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/dvd.html

I 2009/01/29 04:27:59 PLASMA Excluded 0 words in URL http://www.froggie.sk/dvd.html

I 2009/01/29 04:27:59 PLASMA *Indexed 173 words in URL http://www.froggie.sk/dvd.html [Vuf0_Qmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3690 bytes | Anchors: 26
   LinkStorageTime: 16 ms | indexStorageTime: 0 ms

I 2009/01/29 04:27:59 PLASMA Excluded 0 words in URL http://gborn.blogger.de/stories/1317595/comment

I 2009/01/29 04:27:59 PLASMA *Indexed 100 words in URL http://gborn.blogger.de/stories/1317595/comment [eoTIEnKNSYZA]
   Description:  Login
   MimeType: text/html | Charset: ISO-8859-1 | Size: 1264 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:27:59 PLASMA Excluded 0 words in URL http://translate.google.com/translate_t?sl=en

D 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[15, 0, 0, 0]: URL=http://www.froggie.sk/index.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:27:59 CRAWLER LOCALCRAWL[15, 0, 0, 0]: enqueued for load http://www.froggie.sk/index.html [uuBoHQmdd5vA]

I 2009/01/29 04:27:59 PLASMA *Indexed 89 words in URL http://translate.google.com/translate_t?sl=en [zsHVejNUA7rY]
   Description:  Google Translate
   MimeType: text/html | Charset: ISO-8859-1 | Size: 945 bytes | Anchors: 25
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:27:59 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:00 CRAWLER LOCALCRAWL[14, 0, 0, 0]: URL=http://www.froggie.sk/donate.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:00 CRAWLER LOCALCRAWL[14, 0, 0, 0]: enqueued for load http://www.froggie.sk/donate.html [tSQyCQmdd5vA]

I 2009/01/29 04:28:00 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:00 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/index.html

I 2009/01/29 04:28:00 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/donate.html

I 2009/01/29 04:28:00 PLASMA Excluded 0 words in URL http://www.froggie.sk/index.html

I 2009/01/29 04:28:00 PLASMA *Indexed 130 words in URL http://www.froggie.sk/index.html [uuBoHQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 1957 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 0 ms

I 2009/01/29 04:28:00 PLASMA Excluded 0 words in URL http://www.froggie.sk/donate.html

I 2009/01/29 04:28:00 PLASMA *Indexed 54 words in URL http://www.froggie.sk/donate.html [tSQyCQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 608 bytes | Anchors: 16
   LinkStorageTime: 16 ms | indexStorageTime: 0 ms

D 2009/01/29 04:28:00 CRAWLER LOCALCRAWL[13, 0, 0, 0]: URL=http://www.froggie.sk/img/, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:00 CRAWLER LOCALCRAWL[13, 0, 0, 0]: enqueued for load http://www.froggie.sk/img/ [bqZGoQmdd5vA]

I 2009/01/29 04:28:00 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:00 CRAWLER problem loading http://www.froggie.sk/img/: REJECTED WRONG STATUS TYPE '403 Forbidden' for URL http://www.froggie.sk/img/

D 2009/01/29 04:28:01 CRAWLER LOCALCRAWL[12, 0, 0, 0]: URL=http://www.froggie.sk/, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:01 CRAWLER LOCALCRAWL[12, 0, 0, 0]: enqueued for load http://www.froggie.sk/ [y75wJQmdd5vA]

I 2009/01/29 04:28:01 BALANCER forcing crawl-delay of 438 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:01 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/

I 2009/01/29 04:28:01 PLASMA Excluded 0 words in URL http://www.froggie.sk/

I 2009/01/29 04:28:01 PLASMA *Indexed 128 words in URL http://www.froggie.sk/ [y75wJQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 1957 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 15 ms

D 2009/01/29 04:28:01 CRAWLER LOCALCRAWL[11, 0, 0, 0]: URL=http://www.froggie.sk/usage.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:01 CRAWLER LOCALCRAWL[11, 0, 0, 0]: enqueued for load http://www.froggie.sk/usage.html [eZS5FQmdd5vA]

I 2009/01/29 04:28:01 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:02 CRAWLER LOCALCRAWL[10, 0, 0, 0]: URL=http://www.froggie.sk/notes.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:02 CRAWLER LOCALCRAWL[10, 0, 0, 0]: enqueued for load http://www.froggie.sk/notes.html [vHcTKQmdd5vA]

I 2009/01/29 04:28:02 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:02 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/usage.html

I 2009/01/29 04:28:02 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/notes.html

I 2009/01/29 04:28:02 PLASMA Excluded 0 words in URL http://www.froggie.sk/usage.html

I 2009/01/29 04:28:02 PLASMA *Indexed 158 words in URL http://www.froggie.sk/usage.html [eZS5FQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3806 bytes | Anchors: 17
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:28:02 PLASMA Excluded 0 words in URL http://www.froggie.sk/notes.html

I 2009/01/29 04:28:02 PLASMA *Indexed 200 words in URL http://www.froggie.sk/notes.html [vHcTKQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 4227 bytes | Anchors: 16
   LinkStorageTime: 0 ms | indexStorageTime: 0 ms

D 2009/01/29 04:28:02 CRAWLER LOCALCRAWL[9, 0, 0, 0]: URL=http://www.froggie.sk/download.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:02 CRAWLER LOCALCRAWL[9, 0, 0, 0]: enqueued for load http://www.froggie.sk/download.html [onxuaQmdd5vA]

I 2009/01/29 04:28:02 BALANCER forcing crawl-delay of 438 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:03 CRAWLER LOCALCRAWL[8, 0, 0, 0]: URL=http://www.froggie.sk/screens.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:03 CRAWLER LOCALCRAWL[8, 0, 0, 0]: enqueued for load http://www.froggie.sk/screens.html [XdbalQmdd5vA]

I 2009/01/29 04:28:03 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:03 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/download.html

D 2009/01/29 04:28:03 CRAWLER LOCALCRAWL[7, 0, 0, 0]: URL=http://www.froggie.sk/lp32rtm.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:03 PLASMA Excluded 0 words in URL http://www.froggie.sk/download.html

I 2009/01/29 04:28:03 CRAWLER LOCALCRAWL[7, 0, 0, 0]: enqueued for load http://www.froggie.sk/lp32rtm.html [FWMhuQmdd5vA]

I 2009/01/29 04:28:03 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/screens.html

I 2009/01/29 04:28:03 PLASMA *Indexed 218 words in URL http://www.froggie.sk/download.html [onxuaQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3959 bytes | Anchors: 23
   LinkStorageTime: 15 ms | indexStorageTime: 0 ms

I 2009/01/29 04:28:03 PLASMA Excluded 0 words in URL http://www.froggie.sk/screens.html

I 2009/01/29 04:28:03 PLASMA *Indexed 92 words in URL http://www.froggie.sk/screens.html [XdbalQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 915 bytes | Anchors: 16
   LinkStorageTime: 16 ms | indexStorageTime: 0 ms

I 2009/01/29 04:28:03 BALANCER forcing crawl-delay of 438 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:04 CRAWLER LOCALCRAWL[6, 0, 0, 0]: URL=http://www.froggie.sk/lp32sp1.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:04 CRAWLER LOCALCRAWL[6, 0, 0, 0]: enqueued for load http://www.froggie.sk/lp32sp1.html [KLBpDQmdd5vA]

I 2009/01/29 04:28:04 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:04 CRAWLER LOCALCRAWL[5, 0, 0, 0]: URL=http://www.froggie.sk/lp64sp1.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:04 CRAWLER LOCALCRAWL[5, 0, 0, 0]: enqueued for load http://www.froggie.sk/lp64sp1.html [8RYKRQmdd5vA]

I 2009/01/29 04:28:04 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/lp32rtm.html

I 2009/01/29 04:28:04 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/lp32sp1.html

I 2009/01/29 04:28:04 PLASMA Excluded 0 words in URL http://www.froggie.sk/lp32sp1.html

I 2009/01/29 04:28:04 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:04 PLASMA *Indexed 239 words in URL http://www.froggie.sk/lp32sp1.html [KLBpDQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3890 bytes | Anchors: 52
   LinkStorageTime: 0 ms | indexStorageTime: 15 ms

I 2009/01/29 04:28:04 PLASMA Excluded 0 words in URL http://www.froggie.sk/lp32rtm.html

I 2009/01/29 04:28:04 PLASMA *Indexed 237 words in URL http://www.froggie.sk/lp32rtm.html [FWMhuQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3841 bytes | Anchors: 52
   LinkStorageTime: 0 ms | indexStorageTime: 0 ms

D 2009/01/29 04:28:05 CRAWLER LOCALCRAWL[4, 0, 0, 0]: URL=http://www.froggie.sk/help.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:05 CRAWLER LOCALCRAWL[4, 0, 0, 0]: enqueued for load http://www.froggie.sk/help.html [m8qOMQmdd5vA]

I 2009/01/29 04:28:05 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:05 CRAWLER LOCALCRAWL[3, 0, 0, 0]: URL=http://www.froggie.sk/details.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:05 CRAWLER LOCALCRAWL[3, 0, 0, 0]: enqueued for load http://www.froggie.sk/details.html [NE5HtQmdd5vA]

I 2009/01/29 04:28:05 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/lp64sp1.html

I 2009/01/29 04:28:05 BALANCER forcing crawl-delay of 438 milliseconds for http://www.froggie.sk

I 2009/01/29 04:28:05 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/help.html

I 2009/01/29 04:28:05 PLASMA Excluded 0 words in URL http://www.froggie.sk/help.html

I 2009/01/29 04:28:05 PLASMA *Indexed 164 words in URL http://www.froggie.sk/help.html [m8qOMQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 2366 bytes | Anchors: 16
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:28:05 PLASMA Excluded 0 words in URL http://www.froggie.sk/lp64sp1.html

I 2009/01/29 04:28:05 PLASMA *Indexed 236 words in URL http://www.froggie.sk/lp64sp1.html [8RYKRQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3868 bytes | Anchors: 53
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

D 2009/01/29 04:28:06 CRAWLER LOCALCRAWL[2, 0, 0, 0]: URL=http://www.froggie.sk/lip.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:06 CRAWLER LOCALCRAWL[2, 0, 0, 0]: enqueued for load http://www.froggie.sk/lip.html [1ZrOcQmdd5vA]

I 2009/01/29 04:28:06 BALANCER forcing crawl-delay of 453 milliseconds for http://www.froggie.sk

D 2009/01/29 04:28:06 CRAWLER LOCALCRAWL[1, 0, 0, 0]: URL=http://www.froggie.sk/lp64rtm.html, initiator=148qJoTxccAZ, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=false

I 2009/01/29 04:28:06 CRAWLER LOCALCRAWL[1, 0, 0, 0]: enqueued for load http://www.froggie.sk/lp64rtm.html [MxjYnQmdd5vA]

I 2009/01/29 04:28:06 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/details.html

I 2009/01/29 04:28:06 PLASMA Excluded 0 words in URL http://www.froggie.sk/details.html

I 2009/01/29 04:28:06 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/lip.html

I 2009/01/29 04:28:06 PLASMA *Indexed 190 words in URL http://www.froggie.sk/details.html [NE5HtQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 4012 bytes | Anchors: 16
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms

I 2009/01/29 04:28:06 PLASMA Excluded 0 words in URL http://www.froggie.sk/lip.html

I 2009/01/29 04:28:06 PLASMA *Indexed 119 words in URL http://www.froggie.sk/lip.html [1ZrOcQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 2991 bytes | Anchors: 56
   LinkStorageTime: 0 ms | indexStorageTime: 0 ms

I 2009/01/29 04:28:07 PARSER Charset transformation needed from 'windows-1252' to 'ISO-8859-1' for URL = http://www.froggie.sk/lp64rtm.html

I 2009/01/29 04:28:07 PLASMA Excluded 0 words in URL http://www.froggie.sk/lp64rtm.html

I 2009/01/29 04:28:07 PLASMA *Indexed 234 words in URL http://www.froggie.sk/lp64rtm.html [MxjYnQmdd5vA]
   Description:  Vistalizator - change display language in Vista Home and Business
   MimeType: text/html | Charset: windows-1252 | Size: 3819 bytes | Anchors: 53
   LinkStorageTime: 0 ms | indexStorageTime: 16 ms



Die Ausgabe ist abgesehen von den Fehlern wegen Erreichbarkeit ja anscheinend normal, der Crawl wird dann auch abgearbeitet.

Warum "läuft der Peer derart zu", innerhalb 2 Tage in denen er ja wohl ausser Sortierarbeiten nichts getan hat (Crawling und DHT abgeschaltet / in und out)?

Vielleicht hilft das ja weiter, die Fehlermeldungen im ersten Log kann ich nicht deuten, ausser dass ihm wohl der Speicher ausgeht.
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Do Jan 29, 2009 4:50 pm

Und nochmal ein Log, anderer Peer, grade upgedatet, ein paar Minuten nach Crawl/Peerstart.
Es wird interessanterweise im Log immer ein Indexer-Wert um die 90 angezeigt, auf den Statusseiten aber mehr (100-200)

Welche Informationen wären denn noch interessant um dem auf die Schliche zu kommen? Ich kann das bei 3 Peers mit Versionen 5531,5511,5494 (linux32/win32) reproduzieren.

Code: Alles auswählen
Process
    0.710/05531
    Uptime: 0 days 00:08
System Resources
    Processors: 2
Protection
    password-protected
Memory Usage
    free: 615.89 MB
    total: 1,016.12 MB
    max: 1,016.12 MB
Traffic [Reset]
    Proxy: 0 Bytes
    Crawler: 7.57 MB
Incoming Connections
    Active: 24 | Max: 200
Queues
    Indexing Queue:      
    117 | 80



I 2009/01/29 16:30:07 CRAWLER LOCALCRAWL[23, 35355, 0, 0]: enqueued for load http://qik.com/video/798632 [GfxMB7Crr5tY]

D 2009/01/29 16:30:07 CRAWLER LOCALCRAWL[22, 35355, 0, 0]: URL=http://wordpress.com/tag/e61/, initiator=O1VUEDy91_AA, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=true

I 2009/01/29 16:30:07 CRAWLER LOCALCRAWL[22, 35355, 0, 0]: enqueued for load http://wordpress.com/tag/e61/ [Cz6ncY3VVYOZ]

D 2009/01/29 16:30:07 ROBOTS Trying to download the robots.txt file from URL 'http://landoffruitsandnuts.wordpress.com/robots.txt'.

D 2009/01/29 16:30:08 ROBOTS Robots.txt successfully loaded from URL 'http://landoffruitsandnuts.wordpress.com/robots.txt' in 718 ms.

D 2009/01/29 16:30:08 CRAWLER LOCALCRAWL[21, 35355, 0, 0]: URL=http://landoffruitsandnuts.wordpress.com/2009/, initiator=O1VUEDy91_AA, crawlOrder=true, depth=3, crawlDepth=3, must-match=.*, must-not-match=, permission=true

I 2009/01/29 16:30:08 CRAWLER LOCALCRAWL[21, 35355, 0, 0]: enqueued for load http://landoffruitsandnuts.wordpress.com/2009/ [GgSoD19GRo4Z]

I 2009/01/29 16:30:08 CRAWLER shifted 10 jobs from global crawl to local crawl (coreCrawlJobSize()=30, limitCrawlJobSize()=35345, cluster.mode=publicpeer, robinsonMode=off

D 2009/01/29 16:30:08 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

D 2009/01/29 16:30:10 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=89)

D 2009/01/29 16:30:12 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=89)

D 2009/01/29 16:30:14 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:16 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:18 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:20 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:22 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:24 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:26 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:28 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:30 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:32 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:34 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:36 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:38 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:40 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:42 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:44 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:46 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:49 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:51 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:53 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:55 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:57 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:30:59 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:01 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:03 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:05 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:07 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:09 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:11 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:13 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:15 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:17 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:19 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:21 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:23 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:25 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

I 2009/01/29 16:31:25 PLASMA Rejecting RWIs from peer vjb47pKoI2kQ:linux_01/0.71005523. We are too busy (buffersize=129515).

D 2009/01/29 16:31:27 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:29 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:31 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:33 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:35 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:37 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:39 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:41 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:43 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

I 2009/01/29 16:31:43 PLASMA Rejecting RWIs from peer Xq3BP4LpokAA:KIT044F--/0.71005523. We are too busy (buffersize=128982).

D 2009/01/29 16:31:45 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)

D 2009/01/29 16:31:47 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=90)
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon miTreD » Fr Jan 30, 2009 6:20 pm

Mein Log ist auch voll OOM:
Code: Alles auswählen
E 2009/01/30 18:16:01 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:02 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:04 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:05 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:06 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:08 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:09 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:10 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:12 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:13 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:15 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:16 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:18 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:21 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:23 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:24 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:26 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2009/01/30 18:16:27 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
Hab' noch 'nen thread dump erzeugen können:
Code: Alles auswählen
************* Start Thread Dump Fri Jan 30 18:15:35 CET 2009 *******************

YaCy Version: 0.710/05536
Total Memory = 622198784
Used  Memory = 436991920
Free  Memory = 185206864


THREADS WITH STATES: BLOCKED


THREADS WITH STATES: RUNNABLE

Thread= Session_212.100.55.190:54190#2 id=7094 RUNNABLE
Thread= Session_141.52.175.12:48966#1 id=7110 RUNNABLE
Thread= Session_217.91.22.18:62887#1 id=7109 RUNNABLE
Thread= Session_84.171.177.210:33385#2 id=7081 RUNNABLE
Thread= Session_127.0.0.1:40075#0 id=7143 RUNNABLE
Thread= Session_80.219.133.7:34398#1 id=7139 RUNNABLE
Thread= Session_84.143.109.62:55147#0 id=7138 RUNNABLE
Thread= Session_85.177.175.217:61827#0 id=7137 RUNNABLE
Thread= Session_84.143.109.62:44853#0 id=7136 RUNNABLE
Thread= Session_141.52.175.58:52583#0 id=7135 RUNNABLE
Thread= Session_127.0.0.1:40087#0 id=7152 RUNNABLE
Thread= Session_127.0.0.1:40079#3 id=7147 RUNNABLE
Thread= Session_85.25.141.60:53588#1 id=7122 RUNNABLE
Thread= Session_92.229.210.197:4244#1 id=7120 RUNNABLE
Thread= Session_85.178.102.200:39593#1 id=7117 RUNNABLE
Thread= Session_134.107.24.49:34981#1 id=7118 RUNNABLE
Thread= Session_84.176.50.171:2632#0 id=7116 RUNNABLE
Thread= Session_91.89.16.113:63566#1 id=7133 RUNNABLE
Thread= Session_141.52.175.66:51526#1 id=7134 RUNNABLE
Thread= Session_141.52.175.83:40666#0 id=7130 RUNNABLE
Thread= Session_141.52.175.83:35646#0 id=7131 RUNNABLE
Thread= Session_85.25.147.200:36884#0 id=7128 RUNNABLE
Thread= Session_141.52.175.84:45134#1 id=7129 RUNNABLE
Thread= Session_85.25.147.200:36212#0 id=7127 RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at java.io.PushbackInputStream.read(PushbackInputStream.java:122)
at de.anomic.server.serverCore.receive(serverCore.java:863)
at de.anomic.server.serverCore$Session.readLine(serverCore.java:572)
at de.anomic.server.serverCore$Session.listen(serverCore.java:694)
at de.anomic.server.serverCore$Session.run(serverCore.java:626)


Thread= httpd:27001 id=71 RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at de.anomic.server.serverCore.job(serverCore.java:330)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:142)


Thread= Session_127.0.0.1:40093#0 id=7158 RUNNABLE
at java.lang.Thread.dumpThreads(Native Method)
at java.lang.Thread.getAllStackTraces(Thread.java:1487)
at Threaddump_p.respond(Threaddump_p.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1182)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:764)
at de.anomic.http.httpdFileHandler.doGet(httpdFileHandler.java:252)
at de.anomic.http.httpd.GET(httpd.java:488)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
at de.anomic.server.serverCore$Session.run(serverCore.java:626)



THREADS WITH STATES: TIMED_WAITING

Thread= Thread-1 id=9 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverProfiling.run(serverProfiling.java:61)


Thread= Session_127.0.0.1:40089#0 id=7154 TIMED_WAITING
Thread= Session_127.0.0.1:40088#0 id=7153 TIMED_WAITING
Thread= Session_127.0.0.1:40091#0 id=7156 TIMED_WAITING
Thread= Session_127.0.0.1:40090#0 id=7155 TIMED_WAITING
Thread= Session_127.0.0.1:40092#0 id=7157 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:1024)
at de.anomic.http.httpdFileHandler.doGet(httpdFileHandler.java:252)
at de.anomic.http.httpd.GET(httpd.java:488)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
at de.anomic.server.serverCore$Session.run(serverCore.java:626)


Thread= Timer-0 id=61 TIMED_WAITING
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:509)
at java.util.TimerThread.run(Timer.java:462)


Thread= de.anomic.yacy.yacyCore.peerPing id=69 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.remoteCrawlLoaderJob id=66 TIMED_WAITING
Thread= de.anomic.data.bookmarksDB.autoReCrawl id=23 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.cleanupJob id=62 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.dhtTransferJob id=70 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.rwiCacheFlush id=63 TIMED_WAITING
Thread= de.anomic.yacy.yacyCore.publishSeedList id=68 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.deQueueProcess id=64 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob id=65 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.coreCrawlJob id=67 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverAbstractBusyThread.ratz(serverAbstractBusyThread.java:198)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:163)



THREADS WITH STATES: WAITING

Thread= urls_pool-1-thread-16 id=90 WAITING
Thread= urls_pool-1-thread-17 id=91 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)


Thread= MultiThreadedHttpConnectionManager cleanup daemon id=18 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)


Thread= webStructureAnalysis_pool-1-thread-9 id=48 WAITING
Thread= webStructureAnalysis_pool-1-thread-10 id=50 WAITING
Thread= job_pool-1-thread-7 id=44 WAITING
Thread= job_pool-1-thread-1 id=32 WAITING
Thread= parseDocument_pool-1-thread-15 id=60 WAITING
Thread= job_pool-1-thread-3 id=36 WAITING
Thread= job_pool-1-thread-5 id=40 WAITING
Thread= parseDocument_pool-1-thread-13 id=56 WAITING
Thread= storeDocumentIndex_pool-1-thread-8 id=46 WAITING
Thread= job_pool-1-thread-2 id=34 WAITING
Thread= parseDocument_pool-1-thread-14 id=58 WAITING
Thread= job_pool-1-thread-4 id=38 WAITING
Thread= job_pool-1-thread-6 id=42 WAITING
Thread= condenseDocument_pool-1-thread-12 id=54 WAITING
Thread= condenseDocument_pool-1-thread-11 id=52 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at de.anomic.server.serverProcessor.take(serverProcessor.java:96)
at de.anomic.server.serverAbstractBlockingThread.run(serverAbstractBlockingThread.java:55)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)


Thread= main id=1 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at de.anomic.server.serverSemaphore.P(serverSemaphore.java:63)
at de.anomic.plasma.plasmaSwitchboard.waitForShutdown(plasmaSwitchboard.java:2257)
at yacy.startup(yacy.java:420)
at yacy.main(yacy.java:1037)


Thread= Finalizer daemon id=3 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)


Thread= Reference Handler daemon id=2 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)


Thread= Java2D Disposer daemon id=7141 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at sun.java2d.Disposer.run(Disposer.java:125)
at java.lang.Thread.run(Thread.java:619)



THREADS WITH STATES: NEW


THREADS WITH STATES: TERMINATED


************* End Thread Dump Fri Jan 30 18:15:35 CET 2009 *******************
miTreD
 
Beiträge: 1241
Registriert: Mi Jun 27, 2007 11:35 am
Wohnort: /home

Re: Speichermangel und Startschwierigkeiten

Beitragvon AllesMeins » So Feb 01, 2009 1:51 pm

Ich habe das gleiche Problem, wenn auch nicht ganz so drastisch. Bei mir läuft yacy oftmals über Stunden stabil (hab aber gerade auch schon erlebt, dass es nur 15 min gedauert hat), aber früher oder später hat er doch seinen kompletten Speicher (512MB) aufgefressen und gibt nur noch hilfloses
Code: Alles auswählen
I 2009/02/01 13:36:25 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 889 / -2607 KB)   

immer und immer wieder aus. Dann ist auch kein arbeiten mehr möglich, da alles nur noch sehr, sehr langsam oder gar nicht mehr geht (Oberfläche, Proxy etc).
Bis auf die angehobenen Speichergrenze müsste ich bei allem Performance-Relevanten Einstellungen noch die Standard-Werte haben (hab erst vor kurzen alles auf "Werkseinstellungen" zurückgesetzt, weil ich yacy "kaputtkonfiguriert" hatte)
AllesMeins
 
Beiträge: 28
Registriert: So Jun 29, 2008 1:42 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Lilirasa » So Feb 01, 2009 6:25 pm

Habe das selbe Problem bei einem Peer mit 900MB RAM. Derjenige mit 1400MB RAM läuft jedoch rund wie noch nie (stabile ppm bei ~40 und sehr schnelle Antwortzeiten für das Webinterface)... beides Version 0.710/05540...
Lilirasa
 
Beiträge: 71
Registriert: So Sep 14, 2008 11:28 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Orbiter » So Feb 01, 2009 7:22 pm

ich hab das an meinem eigenen Peer auch beobachtet, der aber noch mindestens 200 MB frei hatte. Bin ein wenig ratlos. Hierzu müssen wir mal gucken wo man mehr logging einbauen könnte, sonst komme ich da nicht so schnell dahinter.
Orbiter
 
Beiträge: 5796
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » So Feb 01, 2009 9:03 pm

Das Problem ist alt und ich wiederhole mich jetzt zum Xten mal ;)

Aktuelle Werte von meinen Peer, kein Crawler.
Code: Alles auswählen
                    Memory Use per Busy-Cycle (kbytes)
HTTPD Server/Proxy  32,195
YaCy Core           40,340     
Indexing            5,933
Cache Flush         33,279
Cleanup             22,126
...

Wenn jetzt die Werte bei "Minimum of Required Memory" zu klein sind und Threads ungünstig zeitgleich gestartet werden bekommt man schnell ein OOM. Deswegen habe ich bei mir "Stop Grow" auf 40 MB und "Start Shrink" auf 20 MB gesetzt, wenn ich crawlen würde, würde ich beide Werte nochmal um 5-10MB erhöhen. Das einzige was ich sehe ist, das hin und wieder ein Thread nicht gestartet werden kann weil der Wert von "Minimum of Required Memory" zu klein ist, ist aber selten. Das nächste Problem ist das ein Peer mit wenig Speicher sich immer am Limit bewegt und YaCy mehr mit dem GC beschäftigt ist als mit sich selbst. Der wiederum kann kein Speicher freigeben weil es nichts freizugeben gibt. Dadurch kann YaCy schon mal ein paar Sekunden oder gar nicht mehr erreichbar sein.

Fazit: viel zu optimistische Werte bei "Start Shrink" und "Stop Grow". Ein GC der oft zum falschen Zeitpunkt oder zu spät den Speicher freigibt.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon AllesMeins » Di Feb 03, 2009 11:15 am

Vielleicht hilft das was: Bei mir tritt der Speichermangel besonders häufig (also bisher eigentlich immer) dann auf, wenn die Indexing Queue volläuft. Dann habe ich so ein Bild wie "134 von max. 85" in der Indexing Queue und es geht quasi gar nichts mehr. Was doppelt fatal ist, weil es dann ja auch dementsprechend lange dauert bis die Schlange wieder leer ist. Ist aber natürlich schwer zu sagen, ob das jetzt Auslöser oder Symptom ist.
AllesMeins
 
Beiträge: 28
Registriert: So Jun 29, 2008 1:42 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Di Feb 03, 2009 5:51 pm

thq hat geschrieben:Deswegen habe ich bei mir "Stop Grow" auf 40 MB und "Start Shrink" auf 20 MB gesetzt,


Sorry, wo kann ich diese Werte anpassen ;) - Unter "Advanced Properties" hab ich grad nichts gefunden, oder such ich falsch?
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Di Feb 03, 2009 6:17 pm

Im SVN 5545 muss man diese Dateien ändern
Code: Alles auswählen
source/de/anomic/kelondro/blob/Cache.java
source/de/anomic/kelondro/table/CachedRecords.java
source/de/anomic/plasma/plasmaSwitchboard.java

Für meinen Peer mit dem ich suche und somit auch indexiere habe ich diese Werte
Code: Alles auswählen
private static long memStopGrow    = 50 * 1024 * 1024;
private static long memStartShrink = 25 * 1024 * 1024;

Es müsste auch reichen wenn man nur plasmaSwitchboard.java ändert
Code: Alles auswählen
CachedRecords.setCacheGrowStati(50 * 1024 * 1024, 25 * 1024 * 1024);
Cache.setCacheGrowStati(50 * 1024 * 1024, 25 * 1024 * 1024);

Nicht vergessen die Werte für "Minimum of Required Memory" auf der PerformanceQueues_p.html Seite anzupassen, da die Werte von "Memory Use per Busy-Cycle" zu Anfang erstmal kleiner sind sollte der Peer schon etwas länger laufen damit man nicht zu kleine Werte benutzt.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Mi Feb 04, 2009 5:27 am

Was ich nicht verstehe ist, warum tritt das Problem (verstärkt) erst ab ca Version 5494 auf (frühere kann ich nicht testen, mir sind solche Fehler aber definitiv nicht aufgefallen).

thq deine Änderungen habe ich ausprobiert, auf einem Peer mit ca. 10mio urls / 7mio rwi, 2GB zugewiesen, also massig frei, eigentlich, trotzdem hat er zeitweise grade mal 50MB Speicher frei, ohne dass er crawlt.

Sogar nach einem Neustart:
Speicherverbrauch
Frei: 1,05 GB
Total: 2 GB
Maximal: 2 GB
Ist das Log wieder voll mit "javaheap outofmemory", der Indexer hängt aber auch bei 700 Seiten (nach ein paar Minuten crawl und baut auch nicht merkbar ab)

Pendelt zwar dann auch mal zwischen 400MB-1000MB freiem Speicher, aber imho immer noch viel zu viel, ich habe schon 25mio mit viel weniger Speicher problemlos indexiert.

Grosse Peers (>20mio) sind so, auch unter 64 bit, extra ausprobiert, faktisch nicht mehr benutzbar.
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Mi Feb 04, 2009 11:24 am

dulcedo hat geschrieben:thq deine Änderungen habe ich ausprobiert, auf einem Peer mit ca. 10mio urls / 7mio rwi, 2GB zugewiesen, also massig frei, eigentlich, trotzdem hat er zeitweise grade mal 50MB Speicher frei, ohne dass er crawlt.
Das ist normal weil alte Speicherobjekte durch den GC nicht sofort freigegeben werden.

Ich selber crawle ja nicht und habe deswegen keine passende Werte, aber wenn es immer noch OOM gibt müssen beide Werte solange um 5MB (man kann es auch in 1er Schritten machen wenn man Zeit hat ;)) erhöht werden bis es reicht, oder die Anzahl der Threads verringern die gleichzeitig laufen (ich weiß jetzt aber nicht wie man das macht).

Wenn man es in kauft nimmt das der GC alle x Minuten läuft (Vorgabe sind 10) kann man auch plasmaSwitchboard.java ändern.
Code: Alles auswählen
moreMemory.schedule(new MoreMemory(), 300000, 600000);
geändert werden muss die 600000.

Das hier ist nur ein zusätzlicher GC der nur ausgeführt wird wenn der letzte GC Aufruf mindesten 10 Sekunden zurück liegt.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Do Feb 05, 2009 3:00 am

thq, danke für die Erklärung, wenn Zeit experimentiere ich damit noch rum.

Ich habe gestern morgen allerdings auf einem "Problempeer" ein Update von 5531 auf 5569 gemacht und dann seinen daily-crawl bei dem er regelmässig gezickt hatte, bis jetzt läuft der crawl und auch danach normale DHT-Arbeit ohne Probleme.

Keine Ahnung was da wie geändert wurde und ob reiner Zufall, ich jedenfalls nichts geändert bis auf die Programmversion, /LOG gelöscht und RAM von 1000 auf 700 reduziert, könnte das vielleicht jemand der Betroffenen gegenchecken?
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Do Feb 05, 2009 1:12 pm

Ich habe das bei mir jetzt so
Code: Alles auswählen
moreMemory.schedule(new MoreMemory(), 300000, 120000);
also alle 2 Minuten, der Vorteil ist das Peers mit viel Speicher den Speicher besser nutzen (Cache), zumindest bei mein Peer mit 1400MB.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon Orbiter » Do Feb 05, 2009 1:44 pm

thq hat geschrieben:
dulcedo hat geschrieben:thq deine Änderungen habe ich ausprobiert, auf einem Peer mit ca. 10mio urls / 7mio rwi, 2GB zugewiesen, also massig frei, eigentlich, trotzdem hat er zeitweise grade mal 50MB Speicher frei, ohne dass er crawlt.
Das ist normal weil alte Speicherobjekte durch den GC nicht sofort freigegeben werden.

das ist richtig, aber bei Anforderung von mehr Speicher wird er dann doch durch einen GC freigegeben, der nicht explizit durchgeführt werden muss. Explizite GC machen alles nur schlimmer, denn die halten die gesamte VM an, und alle Prozesse.
Orbiter
 
Beiträge: 5796
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Do Feb 05, 2009 1:58 pm

Orbiter hat geschrieben:
thq hat geschrieben:
dulcedo hat geschrieben:thq deine Änderungen habe ich ausprobiert, auf einem Peer mit ca. 10mio urls / 7mio rwi, 2GB zugewiesen, also massig frei, eigentlich, trotzdem hat er zeitweise grade mal 50MB Speicher frei, ohne dass er crawlt.
Das ist normal weil alte Speicherobjekte durch den GC nicht sofort freigegeben werden.

das ist richtig, aber bei Anforderung von mehr Speicher wird er dann doch durch einen GC freigegeben, der nicht explizit durchgeführt werden muss. Explizite GC machen alles nur schlimmer, denn die halten die gesamte VM an, und alle Prozesse.
Stimmt, deswegen läuft MoreMemory() als Vorgabe auch nur alle 10 Minuten, nur seit dem ich den jetzt auf 2 Minuten heruntergesetzt habe kann ich zugucken wie die Cachegröße bei den URLs größer wird, vorher wurde der Cache zuschnell geleert weil der "normale" GC nicht rechtzeitig eingegriffen hat. Und je weniger freizugeben ist umso schneller ist er fertig, zur Zeit möchte ich darauf nicht verzichten.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Fr Feb 06, 2009 2:21 pm

Habe da was gefunden was vielleicht die OOM's hervorruft und was auch erklärt warum manche das Problem erst haben wenn der Peer mehr Speicher bekommt. Bin gerade am testen.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon Lilirasa » Fr Feb 06, 2009 4:52 pm

dulcedo hat geschrieben:Ich habe gestern morgen allerdings auf einem "Problempeer" ein Update von 5531 auf 5569 gemacht und dann seinen daily-crawl bei dem er regelmässig gezickt hatte, bis jetzt läuft der crawl und auch danach normale DHT-Arbeit ohne Probleme.
Keine Ahnung was da wie geändert wurde und ob reiner Zufall, ich jedenfalls nichts geändert bis auf die Programmversion, /LOG gelöscht und RAM von 1000 auf 700 reduziert, könnte das vielleicht jemand der Betroffenen gegenchecken?


Einer meiner Peers produzierte nach dem Update von svn 5480 auf 5494 nur noch OOM's. Ich habe dann wieder ein downgrade auf 5480 gmacht. Lief dann wieder ohne Probleme. Gestern Abend habe ich auf deinen POST hin auf 5578 umgestellt. Wieder nur OOM's. Allerdings nachedem ich sämtliche Crawls beendet und alle Puffer geleert habe, rennt YACY seit etwa 24h wieder mit neuen Crawls ohne Probleme.

Kann es sein das seit der neuen Stable bestehende Crawls von älteren SVN's nicht richtig übernommen werden?

Grüsse
Andi

Nachtrag:
dafür läuft der Indexier-Puffer jetzt ständig voll (>100 | 80). Im Log erscheint öfters CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize:81). Ich nehme an, sbQueueSize gibt den Füllstand des Indexierpuffer wieder. Jedoch waren zu diesem Zeitpunkt 147 (und nicht 81) Einträge vorhanden.

Hängt wohl damit zusammen: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1763
Lilirasa
 
Beiträge: 71
Registriert: So Sep 14, 2008 11:28 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Sa Feb 07, 2009 10:24 am

thq hat geschrieben:Habe da was gefunden was vielleicht die OOM's hervorruft und was auch erklärt warum manche das Problem erst haben wenn der Peer mehr Speicher bekommt. Bin gerade am testen.
Ein "Fehler" behoben aber das Problem das man ein OOM bekommt obwohl z.B. noch mehr als 200MB Speicher frei sind bleibt bestehen, auch wenn der GC immer noch Speicher frei gibt.

Entweder ist es ein java BUG oder aber es sind zwar mehr als 200MB frei, die sich aber nur aus viele kleine fragmentierte Speicherblöcke zusammen setzen. Wenn YaCy jetzt mit großen Objekten arbeitet und java nicht mehr ein so großen Speicherblock hat gibt es ein OOM.

Gibt es in java eine Funktion mit der man gezielt abfragen kann wie groß der größte Speicherblock ist ?
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Sa Feb 07, 2009 1:21 pm

Lilirasa hat geschrieben:Kann es sein das seit der neuen Stable bestehende Crawls von älteren SVN's nicht richtig übernommen werden?
...
Nachtrag:
dafür läuft der Indexier-Puffer jetzt ständig voll (>100 | 80). Im Log erscheint öfters CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize:81). Ich nehme an, sbQueueSize gibt den Füllstand des Indexierpuffer wieder. Jedoch waren zu diesem Zeitpunkt 147 (und nicht 81) Einträge vorhanden.
Hängt wohl damit zusammen: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1763


Ich habe den Peer um den es in dem Link konkret ging nun auch mal ein update verpasst, auf 5582
Ebenfalls dann alle Crawls beendet und gelöscht.
Nun macht er grade daily, bis jetzt ohne Puffer-Überlauf.

Allerdings eine Menge anderer Fehler, wenn ich z.B. einen Crawl starten mag öffnet er die Startseite nich, bzw. prüft sie nicht.
Auch das log nun voll mit anderen Fehlern, BUSYTHREAD und andere, das muss ich jetzt noch genauer ansehen.

Das Indexer-Problem scheint aber wirklich mit "alten" crawls zu tun haben.

Als UPDATE Auszug aus dem Log, das sieht nicht gut aus, was könnte das sein?

Code: Alles auswählen
I 2009/02/07 19:41:27 PLASMA Received 593 Entries 74 Words [wLSCMP0wjiGt .. wVi98Ktcj0ON]/811849745714999688 from CHk1tMQFkR22:locutus/0.7100558, processed in 1451 milliseconds, requesting 1/456 URLs, blocked 63 RWIs

I 2009/02/07 19:41:27 PLASMA Received 927 Entries 31 Words [1gdE6Xmh_kL- .. 1g2yjZTVOLuy]/54702032755026724 from URtOk5AS_eqh:127-0-0-1-12dpnufe0/0.70005494, processed in 2917 milliseconds, requesting 136/911 URLs, blocked 73 RWIs

I 2009/02/07 19:41:27 PLASMA Received 0 URLs from peer CHk1tMQFkR22:locutus/0.7100558 in 0 ms, blocked 1 URLs

I 2009/02/07 19:41:28 PLASMA Received 132 URLs from peer URtOk5AS_eqh:127-0-0-1-12dpnufe0/0.70005494 in 234 ms, blocked 4 URLs

I 2009/02/07 19:41:30 PLASMA Received 45 Entries 27 Words [11CITQ81q3r6 .. 14tvMCJc3IEa]/4669849055008484 from YpBZTVzXyGbz:chefsache/0.71005577, processed in 1404 milliseconds, requesting 8/32 URLs, blocked 0 RWIs

I 2009/02/07 19:41:30 PLASMA Received 520 Entries 57 Words [WDF80jbzYFlm .. YKaSRa2stDZW]/4436486869065184192 from bAQ1k6PNpagJ:sixcooler/0.7100558, processed in 1825 milliseconds, requesting 75/507 URLs, blocked 10 RWIs

I 2009/02/07 19:41:30 PLASMA Received 6 URLs from peer YpBZTVzXyGbz:chefsache/0.71005577 in 15 ms, blocked 2 URLs

I 2009/02/07 19:41:31 PLASMA Received 72 URLs from peer bAQ1k6PNpagJ:sixcooler/0.7100558 in 47 ms, blocked 3 URLs

E 2009/02/07 19:41:32 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816596888; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:32 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816598220; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:32 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816598886; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:32 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816600218; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:32 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816600884; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816604214; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:41:33 PLASMA Received 357 Entries 51 Words [yPnHQH3Q9XIg .. yPtMGXNC7EhV]/525320528495624308 from _F09_cwHMZbP:rrzn1/0.61505365, processed in 562 milliseconds, requesting 147/357 URLs, blocked 0 RWIs

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816605546; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816606878; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816608210; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:41:33 PLASMA Received 129 URLs from peer _F09_cwHMZbP:rrzn1/0.61505365 in 93 ms, blocked 18 URLs

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816609542; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:33 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816610208; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:41:39 PLASMA Received 699 Entries 332 Words [0o0tNWD6dvDb .. 1EaypuRufZTN]/149354482065374144 from GuTqcE_nEpAA:sixcooler1/0.7100558, processed in 172 milliseconds, requesting 11/343 URLs, blocked 44 RWIs

I 2009/02/07 19:41:40 PLASMA Received 9 URLs from peer GuTqcE_nEpAA:sixcooler1/0.7100558 in 0 ms, blocked 2 URLs

E 2009/02/07 19:41:47 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816618866; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:48 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816621530; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:48 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816622862; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:41:48 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816624194; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:41:55 PLASMA Received 304 Entries 23 Words [zNQ2Vn0uf46m .. zOPfgPB9NrVt]/385497149318482772 from gTrI17m79jAA:Tigma_2/0.7100558, processed in 171 milliseconds, requesting 215/302 URLs, blocked 0 RWIs

I 2009/02/07 19:41:56 PLASMA Received 250 Entries 15 Words [1UQrOgf3FnQn .. 1USS7wiHl0d_]/82583807233428852 from qpFUcK3PiCee:lulabad-eee01/0.7100558, processed in 140 milliseconds, requesting 48/250 URLs, blocked 0 RWIs

I 2009/02/07 19:41:56 PLASMA Received 36 URLs from peer qpFUcK3PiCee:lulabad-eee01/0.7100558 in 47 ms, blocked 12 URLs

I 2009/02/07 19:41:56 PLASMA Received 185 URLs from peer gTrI17m79jAA:Tigma_2/0.7100558 in 234 ms, blocked 30 URLs

E 2009/02/07 19:42:02 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816625526; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:42:08 PLASMA Received 177 Entries 83 Words [z1fJe2K17akU .. 0AwMMxx8qmx-]/283339368908460768 from YpBZTVzXyGbz:chefsache/0.71005577, processed in 1888 milliseconds, requesting 14/57 URLs, blocked 7 RWIs

I 2009/02/07 19:42:08 PLASMA Received 13 URLs from peer YpBZTVzXyGbz:chefsache/0.71005577 in 32 ms, blocked 1 URLs

E 2009/02/07 19:42:17 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816628190; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:42:30 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816628856; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:42:32 PLASMA Received 631 Entries 67 Words [uzS9tTO0kqYk .. u0YYIZpVwNIV]/1020311183400772612 from c4GOj4NApRRf:apfelmaennchen/0.71005577, processed in 359 milliseconds, requesting 113/602 URLs, blocked 369 RWIs

I 2009/02/07 19:42:33 PLASMA Received 76 URLs from peer c4GOj4NApRRf:apfelmaennchen/0.71005577 in 94 ms, blocked 37 URLs

I 2009/02/07 19:42:39 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\010_owncr\CRG-A-20090207135149655.nfK7g7whkCHA.cr.gz to 81.169.185.185:8642: remote peer failed with transfer: transfer failure

I 2009/02/07 19:42:40 PLASMA Received 278 Entries 32 Words [0TIpbWkYtYMr .. 0UR-Un96OFGY]/227971792928767340 from yNS4n62yLvNH:Tigma/0.7100558, processed in 2122 milliseconds, requesting 145/276 URLs, blocked 28 RWIs

I 2009/02/07 19:42:40 PLASMA Received 498 Entries 61 Words [VajLNRe6Ow2M .. XGBfYQe1iil9]/4559132032925222808 from bAQ1k6PNpagJ:sixcooler/0.7100558, processed in 2402 milliseconds, requesting 131/464 URLs, blocked 34 RWIs

I 2009/02/07 19:42:41 PLASMA Received 61 URLs from peer bAQ1k6PNpagJ:sixcooler/0.7100558 in 327 ms, blocked 70 URLs

I 2009/02/07 19:42:41 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\010_owncr\CRG-A-20090207135149655.nfK7g7whkCHA.cr.gz to 141.41.64.252:8080: remote peer failed with transfer: transfer failure

I 2009/02/07 19:42:41 PLASMA Received 114 URLs from peer yNS4n62yLvNH:Tigma/0.7100558 in 109 ms, blocked 31 URLs

I 2009/02/07 19:42:42 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\010_owncr\CRG-A-20090207135149655.nfK7g7whkCHA.cr.gz to 141.52.175.82:8080: remote peer failed with transfer: transfer failure

I 2009/02/07 19:42:43 PLASMA Received 834 Entries 216 Words [u9L1kB6U8SxL .. vAVss57wPFbb]/995713891807227268 from c4GOj4NApRRf:apfelmaennchen/0.71005577, processed in 452 milliseconds, requesting 72/791 URLs, blocked 166 RWIs

I 2009/02/07 19:42:44 PLASMA Received 71 URLs from peer c4GOj4NApRRf:apfelmaennchen/0.71005577 in 47 ms, blocked 1 URLs

I 2009/02/07 19:42:45 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\010_owncr\CRG-A-20090207135149655.nfK7g7whkCHA.cr.gz to 75.151.26.164:8080: remote peer failed with transfer: transfer failure

I 2009/02/07 19:42:46 PLASMA RankingDistribution - transmitted file C:\yacy\DATA\RANKING\GLOBAL\010_owncr\CRG-A-20090207135149655.nfK7g7whkCHA.cr.gz to 194.204.33.13:8080 successfully in 1 seconds

E 2009/02/07 19:42:50 YACY yacyClient.permissionTransfer error:Connection refused: connect

I 2009/02/07 19:42:50 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080125153525166.hu_vVUcKacvQ.cr.gz to kaskelix.de:8080: no connection to remote address kaskelix.de:8080; phase 1

I 2009/02/07 19:42:50 PLASMA RankingDistribution - error transmitting file C:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080125153525166.hu_vVUcKacvQ.cr.gz to yacy.dyndns.org:8000: remote peer rejected transfer: denied

I 2009/02/07 19:42:52 PLASMA RankingDistribution - transmitted file C:\yacy\DATA\RANKING\GLOBAL\014_othercr\CRG-A-20080125153525166.hu_vVUcKacvQ.cr.gz to 85.10.210.99:8080 successfully in 1 seconds

D 2009/02/07 19:42:52 KELONDRO file 'C:\yacy\data\index\freeworld\NETWORK\newsProcessed.stack' closed.

D 2009/02/07 19:42:52 KELONDRO file 'C:\yacy\data\index\freeworld\NETWORK\newsPublished.stack' closed.

I 2009/02/07 19:42:52 YACY rulebasedUpdateInfo: not an automatic update selected

I 2009/02/07 19:43:04 RESOURCE OBSERVER df of Volume C: 35068 MB

I 2009/02/07 19:43:04 RESOURCE OBSERVER run completed; everything in order

I 2009/02/07 19:43:04 PLASMA Received 38 Entries 23 Words [1x1utjEiRZp- .. 11A13WEnFDGv]/12433201752859248 from YpBZTVzXyGbz:chefsache/0.71005577, processed in 126 milliseconds, requesting 0/25 URLs, blocked 3 RWIs

I 2009/02/07 19:43:04 PLASMA Received 163 Entries 22 Words [u2_Juk-V25dZ .. u2_uZTbx5VV8]/1013212108492818600 from CHk1tMQFkR22:locutus/0.7100558, processed in 217 milliseconds, requesting 99/163 URLs, blocked 0 RWIs

E 2009/02/07 19:43:04 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816630854; wrong FREEH has been patched, lost 1 records.

E 2009/02/07 19:43:04 kelondroAbstractRecords/C:\yacy\data\index\freeworld\text\RICOLLECTION\collection.04.0028.01.00.kca INTERNAL ERROR (DATA INCONSISTENCY): USAGE.FREEH.index = 1363899993, entryFile.length() = 816636848; wrong FREEH has been patched, lost 1 records.

I 2009/02/07 19:43:04 PLASMA Received 82 URLs from peer CHk1tMQFkR22:locutus/0.7100558 in 47 ms, blocked 17 URLs

I 2009/02/07 19:43:05 PLASMA Received 41 Entries 9 Words [zWE2OiPLvAYG .. zWOPQonxW48z]/366590044809032448 from P-CC6JJnPmAA:halbwert/0.70005494, processed in 62 milliseconds, requesting 19/39 URLs, blocked 0 RWIs

I 2009/02/07 19:43:05 PLASMA Received 16 URLs from peer P-CC6JJnPmAA:halbwert/0.70005494 in 16 ms, blocked 3 URLs
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Sa Feb 07, 2009 11:08 pm

Zwar leidet mein Peer (KSBA-YACY) nicht unter OOM's, aber es passt zum Titel :(
- Ich hatte diesem Peer 1.4 Gbyte Ram verpasst und er startete ohne mekern im Log auf und das einzige was er dann für Stunden nach LogFile tat, war zu schauen ob er AutoReCRawls auszuführen hatte.. Keine Webseite, kein Zugriff über web, keine Klage über OOM oder VerbindungsProbleme ... Dank Update auf 64-Bit Java und erhöhung auf 1.5 Gbyte startet er nun wieder normal auf.

Aber ist es normal, dass für 27Mio Links / 15 Mio Words, das Yacy bereits nimmer läuft? Vorallem störte mich aber dass er nichts von Problemen im LogFile hinterlies, nicht dass er irgendwo hängen bliebt... Oder wo im angehängten log steht das was sagt, dass er zuwenig speicher hatte ?
Code: Alles auswählen
S 2009/02/07 13:06:43 CRAWLER Starting Crawling Management
I 2009/02/07 13:06:43 ECOTABLE initialization of C:\YACY\DATA\PLASMADB\urlNoticeCoreStack9.db: available RAM: 134MB, allocating space for 0 entries
I 2009/02/07 13:06:43 ECOTABLE C:\YACY\DATA\PLASMADB\urlNoticeCoreStack9.db: EcoTable C:\YACY\DATA\PLASMADB\urlNoticeCoreStack9.db has table copy DISABLED
I 2009/02/07 13:06:43 ECOTABLE initializing RAM index for EcoTable urlNoticeCoreStack9.db, please wait.
I 2009/02/07 13:06:43 ECOTABLE initialization of C:\YACY\DATA\PLASMADB\urlNoticeLimitStack9.db: available RAM: 133MB, allocating space for 0 entries
I 2009/02/07 13:06:43 ECOTABLE C:\YACY\DATA\PLASMADB\urlNoticeLimitStack9.db: EcoTable C:\YACY\DATA\PLASMADB\urlNoticeLimitStack9.db has table copy DISABLED
I 2009/02/07 13:06:43 ECOTABLE initializing RAM index for EcoTable urlNoticeLimitStack9.db, please wait.
I 2009/02/07 13:06:43 ECOTABLE initialization of C:\YACY\DATA\PLASMADB\urlNoticeRemoteStack9.db: available RAM: 131MB, allocating space for 0 entries
I 2009/02/07 13:06:43 ECOTABLE C:\YACY\DATA\PLASMADB\urlNoticeRemoteStack9.db: EcoTable C:\YACY\DATA\PLASMADB\urlNoticeRemoteStack9.db has table copy DISABLED
I 2009/02/07 13:06:43 ECOTABLE initializing RAM index for EcoTable urlNoticeRemoteStack9.db, please wait.
I 2009/02/07 13:06:43 ECOTABLE initialization of C:\YACY\DATA\PLASMADB\urlError3.db: available RAM: 131MB, allocating space for 3714864 entries
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - reading schedules from C:\YACY\DATA\SETTINGS\autoReCrawl.conf
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - processing: /autoReCrawl/hourly
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - processing: /autoReCrawl/daily
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:25:27Z] http://www.zeitungenschweiz.com/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:23:58Z] http://www.presseportal.ch/de/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:21:24Z] http://www.tagesanzeiger.ch/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:19:43Z] http://www.nzz.ch/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:19:13Z] http://www.blick.ch/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - checking schedule for: [2008-10-23T19:17:44Z] http://www.20minuten.ch/
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - processing: /autoReCrawl/weekly
I 2009/02/07 13:08:42 BOOKMARKS autoReCrawl - processing: /autoReCrawl/monthly
I 2009/02/07 14:08:42 BOOKMARKS autoReCrawl - reading schedules from C:\YACY\DATA\SETTINGS\autoReCrawl.conf
I 2009/02/07 14:08:42 BOOKMARKS autoReCrawl - processing: /autoReCrawl/hourly
Dateianhänge
yacy01.zip
Log eines leerlaufs mit 1.4Gbyte
(31.16 KiB) 72-mal heruntergeladen
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » So Feb 08, 2009 5:56 am

Update gemacht und LOG-Verzeichnis nicht gelöscht?
Ich stelle auch grade um auf 64-bit Windows, klappt bis auf Kleinigkeiten recht gut, ich denke bei 1,4 - 1,5GB ist unter Win32 die Grenze erreicht, zur sinnvollen Nutzung.

Zu den Fehlern im Log oben nochmal: läuft nun ohne weitere Fehler nach erneutem Restart, er hat nun allerdings auch 1,4 statt 1,2 GB RAM zur Verfügung, frei davon zwischen 50 und 400MB. Hoffentlich nur einmalige "Reparatur".
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » So Feb 08, 2009 8:59 am

dulcedo hat geschrieben:Update gemacht und LOG-Verzeichnis nicht gelöscht?

Yacy habe ich nicht updated, nur Java. Wobei sich Yacy auch mit "neuem" Java im leerlauf befand, nur das erhöhen des Speichers hat es wieder zum leben erweckt. Wobei mir nicht klar ist, wesshalb man das Log-Verzeichniss löschen sollte.

Scheint aber dass ich den Peer nun "fortwerfen" kann, denn er läuft dermassen lahm. CPU Last ist 1 Core zu 100%. Über web ist er nicht mehr erreichbar, stopen kann man ihn nicht mehr und im Logfile ist seit 2 Std. nichts mehr eingetragen worden.

Folgendes bringt das stopYACY wenn ich ihn versuche zu stopen.
08.02.2009 12:09:03 de.anomic.kelondro.util.Log logSevere SCHWERWIEGEND: could not establish connection to YACY socket: Read timed out
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Lotus » So Feb 08, 2009 12:29 pm

@bluumi: versuch mal einen Thread Dump zu machen.
Lotus
 
Beiträge: 1699
Registriert: Mi Jun 27, 2007 3:33 pm
Wohnort: Hamburg

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » So Feb 08, 2009 1:54 pm

Ich weiss wirklich nicht wie man einen Thread Dump auslesen tut :)
Aber wenn Du/Ihr daraus schlau werdet?

Hier also der erzeugte Thread Dump
Dateianhänge
yacydump.zip
(71.99 KiB) 54-mal heruntergeladen
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » So Feb 08, 2009 9:52 pm

Hab nun YACY auf 1700m erhöht, damit bleibt noch 1300m fürs OS.
Damit läuft es nun scheinbar doch durch. Aber das ist ja horror ressourcenhunger.
[edit] 3 Std. Später .. Tja.. blöde nun hängt er doch wieder :-)
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Mo Feb 09, 2009 5:53 am

Bluumi, mit dem "Aussitzen" bin ich aber schon ein paarmal weitergekommen, das spricht für mich umso mehr für die Stabilität des Gesamtpakets.
Ich habe einen Peer seit Oktober laufen, immer wieder mal Probleme weil oft Updates, im Notfall dann DHT komplett abgeschaltet, keine Crawls mehr und einfach laufen lassen, ab und an ein Update und irgendwann lief er immer wieder.

Vielleicht kann mir einer der Programmierer das bestätigen, durch die ganzen Hintergrund und Update-Prozesse scheint auch ständig repariert zu werden. Anders kann ich mir dass nicht erklären.

Danke für den Java-Tipp übrigens, hinbekommen habe ich es aber noch nicht, Zeitmangel. Es ist ja auch erst wirklich interessant wenn eine VM mehr als 2GB benötigt. Wichtiger ist, dem BS überhaupt erstmal zu zeigen dass die Erde nicht flach ist.
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Mo Feb 09, 2009 10:06 pm

dulcedo hat geschrieben:Bluumi, mit dem "Aussitzen" bin ich aber schon ein paarmal weitergekommen, das spricht für mich umso mehr für die Stabilität des Gesamtpakets.

Naja.. was mich irritiert ist einfach, dass die Kiste alleine mit RemoteCrawls und ohne Update diesen Punkt erreichte dass sie nicht mehr lief. Also habe ich ja im Prinzip erst ne Woche ein laufendes System, und dann plötzlich fängt es an immer öfters zu hängen. Und mit hängen meine ich dass nichts geht, stundenlang kein Eintrag im Logfile, hdd rasseln, keine Reaktion auf StopYacy, kein Webinterface. Was solls, ein Peer weniger.
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Lilirasa » Mo Feb 09, 2009 10:43 pm

@Blumi:
Hatte auch Probleme, wenn auch nicht ganz so arg mit svn 5585 rasselt es nun wieder... :-) Hilft bei dir vieleicht auch...
Lilirasa
 
Beiträge: 71
Registriert: So Sep 14, 2008 11:28 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Di Feb 10, 2009 10:54 am

Lilirasa hat geschrieben:mit svn 5585 Hilft bei dir vieleicht auch...

Danke für den Tip. Leider habe ich diese mitlerweile auch drauf, und es vergehen jeweils nur einige Stunden und dann ist der Peer völlig unerreichbar und nur durch Task killen lässt er sich schliessen.
Ist er noch Jung und frisch kann man Ihn noch mit stopYacy erreichen, ist er aber festgefahren taugt stopYacy nichts mehr. :|
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Orbiter » Di Feb 10, 2009 2:51 pm

bluumi, dein thread dump oben hat so viele BLOCKED threads, dass man gar nicht weiss wo man anfangen soll zu gucken. Das Ding muss die totale Totenstarre gehabt haben. Eine Kleinigkeit, die ich als Resourcenfressend angesehen habe, habe ich mal bearbeitet. Es hat mit dem Internen logging für DoS-Protection zu tun. SVN 5591. Vielleicht hilft das schon ein wenig.
Orbiter
 
Beiträge: 5796
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Mi Feb 11, 2009 12:06 am

Mit total Totenstarre liegst Du nicht weit daneben :-)

Ich hab nun updated und bin ja heftig gespannt :)
Falls er morgen um 8 noch läuft ist es ein grosser erfolg.
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon Orbiter » Mi Feb 11, 2009 12:18 am

naja, so viel wie da geblockt hat, muss noch mehr schief gelaufen sein. Aber eine kleine Ursache könnte schon mal weg sein. Wenn möglich mache bitte wieder einen neuen Thread dump.
Orbiter
 
Beiträge: 5796
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Mi Feb 11, 2009 8:36 am

Ist zwar eigentlich das Indexer-Problem aber nach 1-2 Tagen reagiert dieser Peer hier in der beschriebenen Art auch nicht mehr, eben grade noch drangekommen.
Vorher meldet er im Log mehrmals dass "too many ... index ... dismissed" (ist mir leider entwischt)

Lasse ich ihn nun so weiterlaufen, auch ohne Crawl, dann hängt er nach ein paar Stunden.


Code: Alles auswählen
************* Start Thread Dump Wed Feb 11 08:29:16 CET 2009 *******************

YaCy Version: 0.710/05583
Total Memory = 2071724032
Used  Memory = 1704208096
Free  Memory = 367515936


THREADS WITH STATES: BLOCKED

Thread= Session_195.202.180.115:45209#2 id=6482 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)




Der Peer hat "nur" knapp 14mio urls. / 7mio words und 2GB in der JVM verfügbar.

10minuten später ist er schon fast nicht mehr erreichbar:
Code: Alles auswählen
************* Start Thread Dump Wed Feb 11 08:50:24 CET 2009 *******************

YaCy Version: 0.710/05583
Total Memory = 2147024896
Used  Memory = 1217040072
Free  Memory = 929984824


THREADS WITH STATES: BLOCKED

Thread= Session_213.239.219.162:54135#0 id=8184 BLOCKED
Thread= Session_91.89.16.113:33030#0 id=8182 BLOCKED
Thread= Session_141.52.175.12:40084#1 id=8228 BLOCKED
Thread= Session_84.171.151.214:44592#0 id=8266 BLOCKED
Thread= Session_80.134.192.158:40528#0 id=8279 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= Session_141.52.175.84:39013#111 id=7508 BLOCKED
Thread= Session_137.226.239.66:59850#4 id=7808 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= Session_141.52.175.82:58750#4 id=8327 BLOCKED
Thread= Session_141.52.175.82:39177#6 id=8304 BLOCKED
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at java.io.PushbackInputStream.read(PushbackInputStream.java:122)
at de.anomic.server.serverCore.receive(serverCore.java:862)
at de.anomic.server.serverCore$Session.readLine(serverCore.java:574)
at de.anomic.server.serverCore$Session.listen(serverCore.java:693)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= de.anomic.plasma.plasmaSwitchboard.rwiCacheFlush id=103 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheUntil(plasmaWordIndex.java:509) [while (System.currentTimeMillis() < timeout && dhtCache.size() > 0) {]
at de.anomic.plasma.plasmaWordIndex.flushCacheFor(plasmaWordIndex.java:505)
at de.anomic.plasma.plasmaSwitchboard.rwiCacheFlush(plasmaSwitchboard.java:1119)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= Session_194.204.62.215:50778#86 id=6637 BLOCKED
Thread= Session_85.25.147.200:43201#7 id=7407 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= Thread-2392 id=8347 BLOCKED
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:223)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:135)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at de.anomic.http.httpdByteCountInputStream.read(httpdByteCountInputStream.java:75)
at de.anomic.kelondro.util.FileUtils.copy(FileUtils.java:95)
at de.anomic.kelondro.util.FileUtils.read(FileUtils.java:262)
at de.anomic.kelondro.util.FileUtils.read(FileUtils.java:252)
at de.anomic.http.JakartaCommonsHttpResponse.getData(JakartaCommonsHttpResponse.java:97)
at de.anomic.crawler.HTTPLoader.load(HTTPLoader.java:168)
at de.anomic.crawler.HTTPLoader.load(HTTPLoader.java:103)
at de.anomic.crawler.ProtocolLoader.load(ProtocolLoader.java:98)
at de.anomic.crawler.ProtocolLoader.process(ProtocolLoader.java:120)
at de.anomic.crawler.CrawlQueues$crawlWorker.run(CrawlQueues.java:541)


Thread= Session_84.153.213.107:47063#0 id=8164 BLOCKED
at de.anomic.kelondro.index.RowSet.get(RowSet.java:110) [return get(key, 0, key.length);]
at de.anomic.kelondro.blob.Cache.has(Cache.java:210)
at de.anomic.index.indexRepositoryReference.exists(indexRepositoryReference.java:151)
at de.anomic.plasma.plasmaWordIndex.existsURL(plasmaWordIndex.java:392)
at transferRWI.respond(transferRWI.java:171)
at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= storeDocumentIndex_pool-1-thread-14 id=62 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at de.anomic.plasma.plasmaWordIndex.addPageIndex(plasmaWordIndex.java:575)
at de.anomic.plasma.plasmaWordIndex.storeDocument(plasmaWordIndex.java:872)
at de.anomic.plasma.plasmaSwitchboard.storeDocumentIndex(plasmaSwitchboard.java:1666)
at de.anomic.plasma.plasmaSwitchboard.storeDocumentIndex(plasmaSwitchboard.java:1648)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBlockingThread.job(serverInstantBlockingThread.java:87)
at de.anomic.server.serverAbstractBlockingThread.run(serverAbstractBlockingThread.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)


Thread= Session_194.204.62.215:61983#0 id=8138 BLOCKED
Thread= Session_78.49.1.217:18470#0 id=8150 BLOCKED
Thread= Session_84.153.213.107:43483#3 id=8082 BLOCKED
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515) [if (ram.size() > 0) collections.addEntries(flushContainer(ram));]
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= job_pool-1-thread-11 id=56 BLOCKED
Thread= job_pool-1-thread-13 id=60 BLOCKED
Thread= job_pool-1-thread-10 id=54 BLOCKED
Thread= job_pool-1-thread-9 id=52 BLOCKED
at de.anomic.index.indexRepositoryReference.load(indexRepositoryReference.java:101) [if (urlHash == null) return null;]
at de.anomic.plasma.plasmaWordIndex.getURL(plasmaWordIndex.java:384)
at de.anomic.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:259)
at de.anomic.crawler.CrawlStacker.job(CrawlStacker.java:122)
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBlockingThread.job(serverInstantBlockingThread.java:87)
at de.anomic.server.serverAbstractBlockingThread.run(serverAbstractBlockingThread.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)



THREADS WITH STATES: RUNNABLE

Thread= Session_141.52.175.82:47337#4 id=8331 RUNNABLE
at de.anomic.index.indexRepositoryReference.load(indexRepositoryReference.java:101) [if (urlHash == null) return null;]
at de.anomic.plasma.plasmaWordIndex.getURL(plasmaWordIndex.java:384)
at de.anomic.plasma.plasmaSwitchboard.getURL(plasmaSwitchboard.java:923)
at urls.respond(urls.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= de.anomic.crawler.CrawlQueues.coreCrawlJob id=107 RUNNABLE
at java.lang.Object.wait(Native Method)
at de.anomic.crawler.Balancer.pop(Balancer.java:591)
at de.anomic.crawler.NoticedURL.pop(NoticedURL.java:220)
at de.anomic.crawler.NoticedURL.pop(NoticedURL.java:189)
at de.anomic.crawler.CrawlQueues.coreCrawlJob(CrawlQueues.java:195)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= Session_141.52.175.12:59283#21 id=7587 RUNNABLE
Thread= Session_141.52.175.12:34633#1 id=7588 RUNNABLE
Thread= Session_141.52.175.12:37509#1 id=7581 RUNNABLE
Thread= Session_141.52.175.12:46165#12 id=7601 RUNNABLE
Thread= Session_78.46.48.180:48293#5 id=7600 RUNNABLE
Thread= Session_91.17.217.23:64726#4 id=7975 RUNNABLE
Thread= Session_84.176.53.185:2665#28 id=7914 RUNNABLE
Thread= Session_141.52.175.14:55676#1 id=8101 RUNNABLE
Thread= Session_127.0.0.1:51013#21 id=8066 RUNNABLE
Thread= Session_82.135.80.203:49521#0 id=8089 RUNNABLE
Thread= Session_127.0.0.1:40877#9 id=8167 RUNNABLE
Thread= Session_85.10.210.99:52082#0 id=8189 RUNNABLE
Thread= Session_194.204.30.253:63571#0 id=8231 RUNNABLE
Thread= Session_141.52.175.25:39330#0 id=8254 RUNNABLE
Thread= Session_141.52.175.82:58750#4 id=8327 RUNNABLE
Thread= Session_84.176.53.185:2800#0 id=8301 RUNNABLE
Thread= Session_127.0.0.1:40880#2 id=8297 RUNNABLE
Thread= Session_218.55.113.11:51789#1 id=8272 RUNNABLE
Thread= Session_213.239.219.162:53576#0 id=8277 RUNNABLE
Thread= Session_202.168.100.149:40947#6 id=6812 RUNNABLE
Thread= Session_141.52.175.12:38100#12 id=6889 RUNNABLE
Thread= Session_202.168.100.149:41332#19 id=6879 RUNNABLE
Thread= Session_202.168.100.149:41665#8 id=6878 RUNNABLE
Thread= Session_202.168.100.149:42194#12 id=6877 RUNNABLE
Thread= Session_202.168.100.149:33742#2 id=7276 RUNNABLE
Thread= Session_202.168.100.149:58294#1 id=7265 RUNNABLE
Thread= Session_202.168.100.149:58421#0 id=7242 RUNNABLE
Thread= Session_202.168.100.149:39009#0 id=7241 RUNNABLE
Thread= Session_202.168.100.149:50595#1 id=7264 RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at java.io.PushbackInputStream.read(PushbackInputStream.java:122)
at de.anomic.server.serverCore.receive(serverCore.java:862)
at de.anomic.server.serverCore$Session.readLine(serverCore.java:574)
at de.anomic.server.serverCore$Session.listen(serverCore.java:693)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= Session_127.0.0.1:35555#128 id=6083 RUNNABLE
at java.lang.Thread.dumpThreads(Native Method)
at java.lang.Thread.getAllStackTraces(Thread.java:1477)
at Threaddump_p.respond(Threaddump_p.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doGet(httpdFileHandler.java:239)
at de.anomic.http.httpd.GET(httpd.java:488)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= process reaper daemon id=113 RUNNABLE
at java.lang.UNIXProcess.waitForProcessExit(Native Method)
at java.lang.UNIXProcess.access$900(UNIXProcess.java:20)
at java.lang.UNIXProcess$1$1.run(UNIXProcess.java:132)


Thread= httpd:8092 id=111 RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at de.anomic.server.serverCore.job(serverCore.java:332)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= Session_194.204.62.215:64760#87 id=6341 RUNNABLE
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:322)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:381)
at de.anomic.kelondro.io.FileRandomAccess.readFully(FileRandomAccess.java:55)
at de.anomic.kelondro.io.BufferedIOChunks.readFully(BufferedIOChunks.java:76)
at de.anomic.kelondro.table.FullRecords$EcoNode.getValueRow(FullRecords.java:249)
at de.anomic.kelondro.table.FixedWidthArray.get(FixedWidthArray.java:121)
at de.anomic.kelondro.kelondroCollectionIndex.getwithparams(kelondroCollectionIndex.java:750)
at de.anomic.kelondro.kelondroCollectionIndex.merge(kelondroCollectionIndex.java:533)
at de.anomic.index.indexCollectionRI.addEntries(indexCollectionRI.java:164)
at de.anomic.plasma.plasmaWordIndex.flushCacheOne(plasmaWordIndex.java:515)
at de.anomic.plasma.plasmaWordIndex.dhtFlushControl(plasmaWordIndex.java:480)
at de.anomic.plasma.plasmaWordIndex.addEntry(plasmaWordIndex.java:493)
at transferRWI.respond(transferRWI.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)



THREADS WITH STATES: TIMED_WAITING

Thread= de.anomic.crawler.CrawlQueues.coreCrawlJob id=107 TIMED_WAITING
at java.lang.Object.wait(Native Method)
at de.anomic.crawler.Balancer.pop(Balancer.java:591)
at de.anomic.crawler.NoticedURL.pop(NoticedURL.java:220)
at de.anomic.crawler.NoticedURL.pop(NoticedURL.java:189)
at de.anomic.crawler.CrawlQueues.coreCrawlJob(CrawlQueues.java:195)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= de.anomic.data.bookmarksDB.autoReCrawl id=39 TIMED_WAITING
Thread= de.anomic.yacy.yacyCore.peerPing id=109 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.cleanupJob id=102 TIMED_WAITING
Thread= de.anomic.yacy.yacyCore.publishSeedList id=108 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.remoteCrawlLoaderJob id=106 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob id=105 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.dhtTransferJob id=110 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverAbstractBusyThread.ratz(serverAbstractBusyThread.java:199)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:164)


Thread= Timer-0 id=101 TIMED_WAITING
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:509)
at java.util.TimerThread.run(Timer.java:462)


Thread= Thread-1 id=9 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverProfiling.run(serverProfiling.java:63)



THREADS WITH STATES: WAITING

Thread= parseDocument_pool-1-thread-30 id=94 WAITING
Thread= parseDocument_pool-1-thread-31 id=96 WAITING
Thread= parseDocument_pool-1-thread-25 id=84 WAITING
Thread= parseDocument_pool-1-thread-33 id=100 WAITING
Thread= parseDocument_pool-1-thread-29 id=92 WAITING
Thread= parseDocument_pool-1-thread-32 id=98 WAITING
Thread= parseDocument_pool-1-thread-28 id=90 WAITING
Thread= parseDocument_pool-1-thread-27 id=88 WAITING
Thread= parseDocument_pool-1-thread-26 id=86 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:254)
at de.anomic.server.serverProcessor.enQueue(serverProcessor.java:147)
at de.anomic.crawler.CrawlStacker.enqueueEntry(CrawlStacker.java:151)
at de.anomic.plasma.plasmaSwitchboard.parseDocument(plasmaSwitchboard.java:1594)
at de.anomic.plasma.plasmaSwitchboard.parseDocument(plasmaSwitchboard.java:1527)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBlockingThread.job(serverInstantBlockingThread.java:87)
at de.anomic.server.serverAbstractBlockingThread.run(serverAbstractBlockingThread.java:65)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)


Thread= MultiThreadedHttpConnectionManager cleanup daemon id=28 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)


Thread= main id=1 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at de.anomic.server.serverSemaphore.P(serverSemaphore.java:63)
at de.anomic.plasma.plasmaSwitchboard.waitForShutdown(plasmaSwitchboard.java:2257)
at yacy.startup(yacy.java:420)
at yacy.main(yacy.java:1037)


Thread= urls_pool-1-thread-37 id=144 WAITING
Thread= urls_pool-1-thread-34 id=141 WAITING
Thread= urls_pool-1-thread-35 id=142 WAITING
Thread= urls_pool-1-thread-36 id=143 WAITING
Thread= urls_pool-1-thread-38 id=145 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
at java.lang.Thread.run(Thread.java:619)


Thread= Finalizer daemon id=3 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)


Thread= Reference Handler daemon id=2 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)


Thread= Java2D Disposer daemon id=138 WAITING
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at sun.java2d.Disposer.run(Disposer.java:125)
at java.lang.Thread.run(Thread.java:619)


Thread= de.anomic.plasma.plasmaSwitchboard.deQueueProcess id=104 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:254)
at de.anomic.server.serverProcessor.enQueue(serverProcessor.java:147)
at de.anomic.plasma.plasmaSwitchboard.deQueueProcess(plasmaSwitchboard.java:1258)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= webStructureAnalysis_pool-1-thread-15 id=64 WAITING
Thread= webStructureAnalysis_pool-1-thread-17 id=68 WAITING
Thread= webStructureAnalysis_pool-1-thread-16 id=66 WAITING
Thread= condenseDocument_pool-1-thread-23 id=80 WAITING
Thread= job_pool-1-thread-7 id=48 WAITING
Thread= condenseDocument_pool-1-thread-22 id=78 WAITING
Thread= webStructureAnalysis_pool-1-thread-19 id=72 WAITING
Thread= condenseDocument_pool-1-thread-21 id=76 WAITING
Thread= condenseDocument_pool-1-thread-24 id=82 WAITING
Thread= condenseDocument_pool-1-thread-20 id=74 WAITING
Thread= webStructureAnalysis_pool-1-thread-18 id=70 WAITING
Thread= job_pool-1-thread-8 id=50 WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at de.anomic.server.serverProcessor.take(serverProcessor.java:97)
at de.anomic.server.serverAbstractBlockingThread.run(serverAbstractBlockingThread.java:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)



THREADS WITH STATES: NEW


THREADS WITH STATES: TERMINATED

Thread= Thread-2392 id=8347 TERMINATED
at java.util.zip.Inflater.inflateBytes(Native Method)
at java.util.zip.Inflater.inflate(Inflater.java:223)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:135)
at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:92)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at de.anomic.http.httpdByteCountInputStream.read(httpdByteCountInputStream.java:75)
at de.anomic.kelondro.util.FileUtils.copy(FileUtils.java:95)
at de.anomic.kelondro.util.FileUtils.read(FileUtils.java:262)
at de.anomic.kelondro.util.FileUtils.read(FileUtils.java:252)
at de.anomic.http.JakartaCommonsHttpResponse.getData(JakartaCommonsHttpResponse.java:97)
at de.anomic.crawler.HTTPLoader.load(HTTPLoader.java:168)
at de.anomic.crawler.HTTPLoader.load(HTTPLoader.java:103)
at de.anomic.crawler.ProtocolLoader.load(ProtocolLoader.java:98)
at de.anomic.crawler.ProtocolLoader.process(ProtocolLoader.java:120)
at de.anomic.crawler.CrawlQueues$crawlWorker.run(CrawlQueues.java:541)


Thread= Thread-2391 id=8346 TERMINATED
at java.io.RandomAccessFile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessFile.java:322)
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:381)
at de.anomic.kelondro.io.CachedRandomAccess.readFully(CachedRandomAccess.java:85)
at de.anomic.kelondro.io.AbstractRandomAccess.readInt(AbstractRandomAccess.java:86)
at de.anomic.kelondro.blob.BLOBHeapModifier.remove(BLOBHeapModifier.java:154)
at de.anomic.kelondro.blob.BLOBArray.remove(BLOBArray.java:414)
at de.anomic.kelondro.blob.BLOBCompressor.remove(BLOBCompressor.java:237)
at de.anomic.kelondro.blob.BLOBCompressor.put(BLOBCompressor.java:217)
at de.anomic.plasma.plasmaHTCache.storeFile(plasmaHTCache.java:230)
at de.anomic.plasma.plasmaSwitchboard.htEntryStoreProcess(plasmaSwitchboard.java:1035)
at de.anomic.crawler.ProtocolLoader.process(ProtocolLoader.java:123)
at de.anomic.crawler.CrawlQueues$crawlWorker.run(CrawlQueues.java:541)


Thread= Session_141.52.175.82:50256#6 id=8306 TERMINATED
at de.anomic.kelondro.index.RowCollection.ensureSize(RowCollection.java:192) [byte[] newChunkcache = new byte[(int) (needed * growfactor)]; // increase space]
at de.anomic.kelondro.index.RowCollection.addUnique(RowCollection.java:304)
at de.anomic.kelondro.index.RowCollection.addUnique(RowCollection.java:282)
at de.anomic.kelondro.index.RowSet.put(RowSet.java:139)
at de.anomic.kelondro.blob.Cache.put(Cache.java:312)
at de.anomic.kelondro.blob.Cache.put(Cache.java:321)
at de.anomic.index.indexRepositoryReference.store(indexRepositoryReference.java:134)
at de.anomic.plasma.plasmaWordIndex.putURL(plasmaWordIndex.java:380)
at crawlReceipt.respond(crawlReceipt.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)


Thread= Session_141.52.175.82:60306#5 id=8312 TERMINATED
at de.anomic.index.indexRepositoryReference.store(indexRepositoryReference.java:116) [if (exists(entry.hash())) {]
at de.anomic.plasma.plasmaWordIndex.putURL(plasmaWordIndex.java:380)
at crawlReceipt.respond(crawlReceipt.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
at de.anomic.http.httpd.POST(httpd.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
at de.anomic.server.serverCore$Session.run(serverCore.java:628)



************* End Thread Dump Wed Feb 11 08:50:24 CET 2009 *******************
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Mi Feb 11, 2009 8:58 am

Zugehöriges Logfile (ausschnitt):
Code: Alles auswählen
W 2009/02/11 08:53:04 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:46794#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:04 CRAWLER LOCALCRAWL[2270, 2093, 0, 0]: enqueued for load http://www.fr-online.de/verlagsservice/fr_newsreader/ [vZhCwAOmyGuB]

I 2009/02/11 08:53:04 PLASMA Received 996 Entries 157 Words [AAG9cjVHlj6b .. AJE4OyUBFFWK]/393854832165782560 from lXmlJN3DMT9k:swissyacy/0.7100558, processed in 386871 milliseconds, requesting 126/767 URLs, blocked 0 RWIs

I 2009/02/11 08:53:04 PLASMA Received 395 Entries 21 Words [4-ETSLJLAL3s .. 4-FYqsd1hsjA]/1417335705342971108 from qpFUcK3PiCee:lulabad-eee01/0.72005594, processed in 325911 milliseconds, requesting 116/395 URLs, blocked 0 RWIs

I 2009/02/11 08:53:05 BALANCER forcing crawl-delay of 666 milliseconds for www.fr-online.de (caused by robots.txt)

E 2009/02/11 08:53:05 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:05 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:60864#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:05 PLASMA crawlReceipt: RECEIVED RECEIPT from pQ7buo4B5lAA:KIT032F-810/0.7200559 for URL jNbyMsOmyGuB:http://www.fr-online.de/_em_cms/_multifunktion/?em_cnt=1665488&em_art=galery&em_loc=3281

D 2009/02/11 08:53:07 CRAWLER LOCALCRAWL[2269, 2081, 0, 0]: URL=http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672590_Siegfrieds-plastische-Flussfahrt.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

E 2009/02/11 08:53:07 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:07 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.23:60316#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.23
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:07 CRAWLER LOCALCRAWL[2269, 2081, 0, 0]: enqueued for load http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672590_Siegfrieds-plastische-Flussfahrt.html [aqFVpUOmyGuB]

E 2009/02/11 08:53:08 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:08 BALANCER forcing crawl-delay of 658 milliseconds for www.fr-online.de (caused by robots.txt)

W 2009/02/11 08:53:08 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.24:32964#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.24
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/02/11 08:53:09 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

D 2009/02/11 08:53:09 CRAWLER LOCALCRAWL[2268, 2081, 0, 0]: URL=http://www.fr-online.de/_em_cms/_globals/send.php?em_ssc=MSwwLDEsMCwxLDAsMSww&em_cnt=1321125&em_loc=3355&em_ref=%2Fmarktplatz%2Fbaugebiete%2F&em_ivw=koop_grund, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

W 2009/02/11 08:53:09 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:42739#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:09 CRAWLER LOCALCRAWL[2268, 2081, 0, 0]: enqueued for load http://www.fr-online.de/_em_cms/_globals/send.php?em_ssc=MSwwLDEsMCwxLDAsMSww&em_cnt=1321125&em_loc=3355&em_ref=%2Fmarktplatz%2Fbaugebiete%2F&em_ivw=koop_grund [pwmPAsOmyGuB]

I 2009/02/11 08:53:10 BALANCER forcing crawl-delay of 645 milliseconds for www.fr-online.de (caused by robots.txt)

E 2009/02/11 08:53:10 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:10 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL VTnPKBOmyGuB:http://www.fr-online.de/frankfurt_und_hessen/nachrichten/darmstadt/1673573_Gute-Noten-fuer-Schulspeisung.html

W 2009/02/11 08:53:10 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:53160#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:10 PLASMA crawlReceipt: RECEIVED RECEIPT from pQ7buo4B5lAA:KIT032F-810/0.7200559 for URL 7CLTh4OmyGuB:http://www.fr-online.de/startseite/fotostrecken/fotostrecken_reise/

D 2009/02/11 08:53:12 CRAWLER LOCALCRAWL[2267, 2081, 0, 0]: URL=http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672586_Anstelle-einer-Ohrfeige.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

I 2009/02/11 08:53:12 CRAWLER LOCALCRAWL[2267, 2081, 0, 0]: enqueued for load http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672586_Anstelle-einer-Ohrfeige.html [xKx1fUOmyGuB]

E 2009/02/11 08:53:12 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/02/11 08:53:13 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:13 BALANCER forcing crawl-delay of 676 milliseconds for www.fr-online.de (caused by robots.txt)

D 2009/02/11 08:53:14 CRAWLER LOCALCRAWL[2266, 2081, 0, 0]: URL=http://www.fr-online.de/in_und_ausland/ratgeber/top_themen/1672059_Der-boese-Waehrungsfaktor.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=1, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

E 2009/02/11 08:53:14 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:14 PLASMA crawlReceipt: RECEIVED RECEIPT from VnRbvb7EhdAA:KIT16F/0.7100557 for URL eITPQ8DwDQch:https://www.fr-online.de/in_und_ausland/politik/aktuell/1671994_USA-raeumen-Fehler-in-Afghanistan-ein.html

W 2009/02/11 08:53:14 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.23:60428#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.23
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:14 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.23:43496#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.23
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:14 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:48406#1
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:14 CRAWLER LOCALCRAWL[2266, 2081, 0, 0]: enqueued for load http://www.fr-online.de/in_und_ausland/ratgeber/top_themen/1672059_Der-boese-Waehrungsfaktor.html [qcwlWUOmyGuB]

E 2009/02/11 08:53:15 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:15 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:56379#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

D 2009/02/11 08:53:16 CRAWLER LOCALCRAWL[2265, 2081, 0, 0]: URL=http://www.fr-online.de/fr/frankfurt_und_hessen/nachrichten/frankfurt/?em_index_page=2, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

E 2009/02/11 08:53:16 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:17 CRAWLER LOCALCRAWL[2265, 2081, 0, 0]: enqueued for load http://www.fr-online.de/fr/frankfurt_und_hessen/nachrichten/frankfurt/?em_index_page=2 [dVc9iyOmyGuB]

W 2009/02/11 08:53:17 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:40130#2
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/02/11 08:53:17 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:17 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:34954#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

D 2009/02/11 08:53:19 CRAWLER LOCALCRAWL[2264, 2081, 0, 0]: URL=http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672591_Im-rechten-Licht-betrachtet.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

E 2009/02/11 08:53:19 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:19 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL _HsqMUOmyGuB:http://www.fr-online.de/in_und_ausland/politik/aktuell/1673210_Stark-radioaktiv-belastet.html

W 2009/02/11 08:53:19 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:48489#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:19 CRAWLER LOCALCRAWL[2264, 2081, 0, 0]: enqueued for load http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672591_Im-rechten-Licht-betrachtet.html [uuBAVUOmyGuB]

E 2009/02/11 08:53:20 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:20 BALANCER forcing crawl-delay of 699 milliseconds for www.fr-online.de (caused by robots.txt)

W 2009/02/11 08:53:20 FILEHANDLER Unexpected error while processing query.
Session: Session_85.10.210.99:33723#0
Query:   /yacy/crawlReceipt.html
Client:  85.10.210.99
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:20 PLASMA Received 199 Entries 18 Words [_ColKr18v0bM .. _DX9xw1fxp0m]/541546383202258252 from iS7NivAgXU93:KIT002F/0.70005494, processed in 199812 milliseconds, requesting 75/194 URLs, blocked 0 RWIs

I 2009/02/11 08:53:20 PLASMA crawlReceipt: RECEIVED RECEIPT from pQ7buo4B5lAA:KIT032F-810/0.7200559 for URL x3Yd7TOmyGuB:http://www.fr-online.de/rss/fotostrecken/

I 2009/02/11 08:53:20 PLASMA crawlReceipt: RECEIVED RECEIPT from pQ7buo4B5lAA:KIT032F-810/0.7200559 for URL CFVinBOmyGuB:http://www.fr-online.de/frankfurt_und_hessen/nachrichten/darmstadt/1672421_Der-Kindergarten-Cop-empfiehlt-sich.html

D 2009/02/11 08:53:21 CRAWLER LOCALCRAWL[2263, 2081, 0, 0]: URL=http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672587_Gefaehrliche-Seilschaften.html, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

I 2009/02/11 08:53:21 CRAWLER LOCALCRAWL[2263, 2081, 0, 0]: enqueued for load http://www.fr-online.de/in_und_ausland/kultur_und_medien/feuilleton/1672587_Gefaehrliche-Seilschaften.html [OsR79UOmyGuB]

E 2009/02/11 08:53:21 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:22 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:45821#3
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/02/11 08:53:22 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/11 08:53:22 BALANCER forcing crawl-delay of 693 milliseconds for www.fr-online.de (caused by robots.txt)

W 2009/02/11 08:53:22 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:49171#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

D 2009/02/11 08:53:23 CRAWLER LOCALCRAWL[2262, 2081, 0, 0]: URL=http://www.fr-online.de/frankfurt_und_hessen/nachrichten/hanau/, initiator=Czf8mtXEXYVM, crawlOrder=true, depth=2, crawlDepth=4, must-match=.*www.fr-online.de.*, must-not-match=, permission=true

I 2009/02/11 08:53:23 CRAWLER LOCALCRAWL[2262, 2081, 0, 0]: enqueued for load http://www.fr-online.de/frankfurt_und_hessen/nachrichten/hanau/ [MTYRkBOmyGuB]

E 2009/02/11 08:53:23 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

W 2009/02/11 08:53:23 FILEHANDLER Unexpected error while processing query.
Session: Session_141.52.175.82:51831#0
Query:   /yacy/crawlReceipt.html
Client:  141.52.175.82
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space

E 2009/02/11 08:53:24 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/vmadmin/yacy/htroot/yacy/crawlReceipt.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1172)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:751)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:247)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:761)
   at de.anomic.server.serverCore$Session.run(serverCore.java:628)
Caused by: java.lang.OutOfMemoryError: Java heap space



Er arbeitet zwar weiter aber ca. 300 Einträge im Indexer, erlaubt 80.
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Mi Feb 11, 2009 11:16 am

Orbiter hat geschrieben:naja, so viel wie da geblockt hat, muss noch mehr schief gelaufen sein. Aber eine kleine Ursache könnte schon mal weg sein.

Also jetzt, nach 9 Std. läuft er noch, und sogar noch von Remote errichbar. Bin froher Hoffnung, dass da ein MainKnopf gelöst wurde.

Da ich aber viele :
Code: Alles auswählen
I 2009/02/11 09:04:44 INDEX TRANSFER DISPATCHER enqueueContainers: selected 9 targets for primary target key LE1UPuK2BtAA/0 with 4 index containers.
java.lang.reflect.InvocationTargetException
        at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
        at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)
Caused by: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space


Beim Versuch einen MultiThreadDump zu erzeugen hat er sich dann übrigends wieder in die "Toten"starre begeben :-D
Aber 2 Std. später war das Webinterface ansprechbar, aber noch sehr lahm.

Hab zwei Cosole Dump angefügt, den um 9 Uhr, vor dem MultiThreadDump Versuch, und einen von jetzt um 11:Uhr

[edit] Nach weiteren 30 min hat er nun auch den Shutdown Befehl verarbeitet und fährt mit Giftpillen runter :)
Dateianhänge
singleDump.zip
(11.87 KiB) 49-mal heruntergeladen
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon bluumi » Mi Feb 11, 2009 3:38 pm

So, und nun hab ich wieder mehr als 2/3 des ThreadDump mit Blocked voll, obwohl der Peer erst 3.5 Std lief :(

Ich update dann mal auf die letzte, da ich las dass DHT nun auch OOMs mache.
Dateianhänge
singleDump.zip
(8.53 KiB) 53-mal heruntergeladen
bluumi
 
Beiträge: 388
Registriert: Mi Okt 08, 2008 7:27 am

Re: Speichermangel und Startschwierigkeiten

Beitragvon PCA42 » Do Feb 12, 2009 5:18 pm

Mein Peer ist mir heute auch das erste Mal seit langer Zeit wegen fehlendem Speicher hängen geblieben. Und das mit 4500 MB für Yacy. Die Tabellen für die Indizes brauchen davon derzeit 1500 MB. Wenn als die 3000 MB für die "normalen" Abläufe nicht mehr reichen... autschn. Hab jetzt SVN5603 eingespielt. Ich verspreche mir durch die kleineren Transfers wieder etwas mehr Platz im Speicher.
PCA42
 
Beiträge: 621
Registriert: Mi Jan 23, 2008 4:19 pm
Wohnort: @Home

Re: Speichermangel und Startschwierigkeiten

Beitragvon PCA42 » Fr Feb 13, 2009 5:39 am

So, nach ca. 11 Stunden Laufzeit wirft der Peer wieder OOMs. Und das bei 490 MB freiem Speicher. Die OOM scheint scheint beim DHT-Transfer aufzutreten.

Code: Alles auswählen
I 2009/02/13 05:36:25 INDEX TRANSFER DISPATCHER enqueueContainers: selected 9 targets for primary target key _tfDWzkWHeAA/3 with 4 index containers.
I 2009/02/13 05:36:31 PLASMA DEQUEUE: Shutdown detected.
E 2009/02/13 05:36:31 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space


Threaddump hab ich auch mal gezogen, ich hoffe, ihr findet das Problem schnell.
Dateianhänge
threaddump.zip
(1.68 KiB) 73-mal heruntergeladen
PCA42
 
Beiträge: 621
Registriert: Mi Jan 23, 2008 4:19 pm
Wohnort: @Home

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Fr Feb 13, 2009 8:33 am

Version 5585 läuft bei mir insoweit stabil, als dass ich auch crawlen kann ohne Speicher-Fehlermeldungen.
Und er arbeitet an der Grenze, hat nur 10-200RAM frei im Betrieb.

Die einzigen Fehlermedungen dort betreffen wohl DHT-Connections:
I 2009/02/13 08:06:11 PLASMA Received 36 URLs from peer nNpdm73ecFAA:F6F_2/0.7100558 in 47 ms, blocked 26 URLs
E 2009/02/13 08:06:17 SERVER receive interrupted - exception 2 = Connection reset

Andere Versionen 5603 < V > 5585 bekomme ich nicht stabil zum laufen, auf bestehenden Indizes.

Beim peer mit update auf Version 5603 hat sich nun nach ca 24 Stunden "Tiefschlaf unter Vollast" was getan, er ist jedenfalls wieder über http erreichbar.
Aber das Dump/log nicht gut:
Code: Alles auswählen


   
************* Start Thread Dump Fri Feb 13 08:17:36 CET 2009 *******************

YaCy Version: 0.720/05603
Total Memory = 832438272
Used  Memory = 830851408
Free  Memory = 1586536


THREADS WITH STATES: BLOCKED

Thread= Session_141.52.175.84:42465#5 id=3678 BLOCKED
Thread= Session_202.168.100.149:59171#1 id=3719 BLOCKED
Thread= Session_141.52.175.24:59118#1 id=3717 BLOCKED
Thread= Session_202.168.100.149:60084#3 id=3707 BLOCKED
at java.io.PushbackInputStream.read(Unknown Source)
at de.anomic.server.serverCore.receive(serverCore.java:863)
at de.anomic.server.serverCore$Session.readLine(serverCore.java:575)
at de.anomic.server.serverCore$Session.listen(serverCore.java:694)
at de.anomic.server.serverCore$Session.run(serverCore.java:629)


Thread= de.anomic.plasma.plasmaSwitchboard.deQueueProcess id=109 BLOCKED
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverInstantBusyThread.freemem(serverInstantBusyThread.java:125)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:181)


Thread= Session_202.168.100.149:56229#0 id=3722 BLOCKED
Thread= Session_213.239.219.162:45814#0 id=3700 BLOCKED
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
at de.anomic.server.serverCore$Session.run(serverCore.java:629)


Thread= storeDocumentIndex_pool-1-thread-38 id=83 BLOCKED
at java.lang.Thread.run(Unknown Source)



THREADS WITH STATES: RUNNABLE

Thread= de.anomic.plasma.plasmaSwitchboard.cleanupJob id=107 RUNNABLE
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverInstantBusyThread.job(serverInstantBusyThread.java:96)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= de.anomic.plasma.plasmaSwitchboard.rwiCacheFlush id=108 RUNNABLE
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverAbstractBusyThread.ratz(serverAbstractBusyThread.java:199)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:164)


Thread= httpd:8088 id=116 RUNNABLE
at java.net.ServerSocket.accept(Unknown Source)
at de.anomic.server.serverCore.job(serverCore.java:334)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:143)


Thread= Session_141.52.175.12:39851#0 id=3685 RUNNABLE
Thread= Session_80.15.5.61:45281#0 id=3687 RUNNABLE
Thread= Session_194.204.62.215:56166#0 id=3686 RUNNABLE
Thread= Session_84.60.154.134:35324#0 id=3690 RUNNABLE
Thread= Session_141.41.64.254:49503#2 id=3675 RUNNABLE
Thread= Session_141.52.175.84:42465#5 id=3678 RUNNABLE
Thread= Session_141.52.175.37:49385#1 id=3681 RUNNABLE
Thread= Session_85.10.210.99:34298#2 id=3680 RUNNABLE
Thread= Session_85.177.161.29:65061#2 id=3683 RUNNABLE
Thread= Session_116.125.22.241:58352#6 id=3682 RUNNABLE
Thread= Session_141.52.175.23:43109#6 id=3671 RUNNABLE
Thread= Session_141.41.64.254:45346#2 id=3672 RUNNABLE
Thread= Session_141.41.64.254:53094#1 id=3674 RUNNABLE
Thread= Session_141.52.175.12:35868#5 id=3650 RUNNABLE
Thread= Session_202.168.100.149:59171#1 id=3719 RUNNABLE
Thread= Session_202.168.100.149:45083#0 id=3720 RUNNABLE
Thread= Session_141.52.175.24:59118#1 id=3717 RUNNABLE
Thread= Session_141.52.175.23:53674#0 id=3718 RUNNABLE
Thread= Session_116.125.22.241:59826#2 id=3713 RUNNABLE
Thread= Session_194.204.33.13:57977#0 id=3571 RUNNABLE
Thread= Session_85.10.210.99:55772#2 id=3703 RUNNABLE
Thread= Session_141.52.175.23:33943#2 id=3701 RUNNABLE
Thread= Session_127.0.0.1:53659#5 id=3698 RUNNABLE
Thread= Session_62.75.214.113:27484#0 id=3693 RUNNABLE
Thread= Session_127.0.0.1:53649#11 id=3692 RUNNABLE
Thread= Session_116.125.22.241:59761#1 id=3711 RUNNABLE
Thread= Session_116.125.22.241:59748#1 id=3710 RUNNABLE
Thread= Session_85.10.210.99:55141#0 id=3709 RUNNABLE
Thread= Session_93.198.150.233:61351#1 id=3708 RUNNABLE
Thread= Session_202.168.100.149:60084#3 id=3707 RUNNABLE
Thread= Session_116.125.22.241:59716#2 id=3706 RUNNABLE
Thread= Session_127.0.0.1:53665#0 id=3705 RUNNABLE
at java.io.PushbackInputStream.read(Unknown Source)
at de.anomic.server.serverCore.receive(serverCore.java:863)
at de.anomic.server.serverCore$Session.readLine(serverCore.java:575)
at de.anomic.server.serverCore$Session.listen(serverCore.java:694)
at de.anomic.server.serverCore$Session.run(serverCore.java:629)


Thread= Session_213.239.219.162:45814#0 id=3700 RUNNABLE
Thread= Session_127.0.0.1:53664#2 id=3704 RUNNABLE
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
at de.anomic.server.serverCore$Session.run(serverCore.java:629)


Thread= AWT-Windows daemon id=11 RUNNABLE
at java.lang.Thread.run(Unknown Source)



THREADS WITH STATES: TIMED_WAITING

Thread= de.anomic.crawler.CrawlQueues.remoteCrawlLoaderJob id=111 TIMED_WAITING
Thread= de.anomic.yacy.yacyCore.peerPing id=114 TIMED_WAITING
Thread= de.anomic.yacy.yacyCore.publishSeedList id=113 TIMED_WAITING
Thread= de.anomic.data.bookmarksDB.autoReCrawl id=62 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverAbstractBusyThread.ratz(serverAbstractBusyThread.java:199)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:164)


Thread= de.anomic.crawler.CrawlQueues.coreCrawlJob id=112 TIMED_WAITING
Thread= de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob id=110 TIMED_WAITING
Thread= de.anomic.plasma.plasmaSwitchboard.dhtTransferJob id=115 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverAbstractBusyThread.ratz(serverAbstractBusyThread.java:199)
at de.anomic.server.serverAbstractBusyThread.run(serverAbstractBusyThread.java:183)


Thread= Timer-0 id=106 TIMED_WAITING
at java.util.TimerThread.run(Unknown Source)


Thread= Thread-1 id=9 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at de.anomic.server.serverProfiling.run(serverProfiling.java:63)



THREADS WITH STATES: WAITING

Thread= MultiThreadedHttpConnectionManager cleanup daemon id=63 WAITING
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)


Thread= Finalizer daemon id=3 WAITING
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)


Thread= AWT-EventQueue-0 id=15 WAITING
at java.awt.EventDispatchThread.run(Unknown Source)


Thread= storeDocumentIndex_pool-1-thread-19 id=42 WAITING
Thread= condenseDocument_pool-1-thread-44 id=95 WAITING
Thread= urls_pool-1-thread-50 id=140 WAITING
Thread= parseDocument_pool-1-thread-48 id=103 WAITING
Thread= storeDocumentIndex_pool-1-thread-26 id=56 WAITING
Thread= parseDocument_pool-1-thread-45 id=97 WAITING
Thread= AWT-Shutdown id=12 WAITING
Thread= storeDocumentIndex_pool-1-thread-20 id=44 WAITING
Thread= Java2D Disposer daemon id=10 WAITING
Thread= webStructureAnalysis_pool-1-thread-41 id=89 WAITING
Thread= storeDocumentIndex_pool-1-thread-27 id=58 WAITING
Thread= webStructureAnalysis_pool-1-thread-40 id=87 WAITING
Thread= parseDocument_pool-1-thread-49 id=105 WAITING
Thread= job_pool-1-thread-33 id=73 WAITING
Thread= job_pool-1-thread-35 id=77 WAITING
Thread= job_pool-1-thread-37 id=81 WAITING
Thread= storeDocumentIndex_pool-1-thread-22 id=48 WAITING
Thread= storeDocumentIndex_pool-1-thread-25 id=54 WAITING
Thread= storeDocumentIndex_pool-1-thread-23 id=50 WAITING
Thread= urls_pool-1-thread-51 id=141 WAITING
Thread= job_pool-1-thread-32 id=71 WAITING
Thread= condenseDocument_pool-1-thread-43 id=93 WAITING
Thread= condenseDocument_pool-1-thread-42 id=91 WAITING
Thread= job_pool-1-thread-36 id=79 WAITING
Thread= job_pool-1-thread-34 id=75 WAITING
Thread= urls_pool-1-thread-52 id=142 WAITING
Thread= storeDocumentIndex_pool-1-thread-24 id=52 WAITING
Thread= job_pool-1-thread-31 id=69 WAITING
Thread= parseDocument_pool-1-thread-46 id=99 WAITING
Thread= parseDocument_pool-1-thread-47 id=101 WAITING
Thread= webStructureAnalysis_pool-1-thread-39 id=85 WAITING
Thread= storeDocumentIndex_pool-1-thread-21 id=46 WAITING
Thread= storeDocumentIndex_pool-1-thread-28 id=60 WAITING
at java.lang.Thread.run(Unknown Source)


Thread= Reference Handler daemon id=2 WAITING
at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)


Thread= main id=1 WAITING
at java.lang.Object.wait(Object.java:485)
at de.anomic.server.serverSemaphore.P(serverSemaphore.java:63)
at de.anomic.plasma.plasmaSwitchboard.waitForShutdown(plasmaSwitchboard.java:2116)
at yacy.startup(yacy.java:420)
at yacy.main(yacy.java:1037)



THREADS WITH STATES: NEW


THREADS WITH STATES: TERMINATED

Thread= Session_202.168.100.149:34681#0 id=3721 TERMINATED
Thread= Session_202.168.100.149:56229#0 id=3722 TERMINATED
Thread= Session_213.239.219.162:45814#0 id=3700 TERMINATED
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
at de.anomic.server.serverCore$Session.run(serverCore.java:629)



************* End Thread Dump Fri Feb 13 08:17:36 CET 2009 *******************


YaCy - Distributed Search Engine

    *
      Search & Browse
          o Search Page
          o Rich Client Search
          o Interactive local Search
          o Compare Search
          o Ranking Config
          o Surftips
          o Publication
          o Bookmarks
          o Help
    *
      Web Indexing
          o Crawl Start & Monitoring
          o Crawl Results
          o Index Administration
          o Filter & Blacklists
          o Indexing with Proxy
    *
      Monitoring
          o YaCy Network
          o Web Visualization
          o Access Tracker
          o Server Log
          o News Monitor
          o Cookies in Proxy
    *
      Peer Control
          o Admin Console
          o Customization
          o Terminal
          o Messages New Messages
          o Re-Start
          o Shutdown
    *
      The Project
          o Project Home
          o Deutsches Forum
          o English Forum
          o YaCy Project Wiki
          o YaCy Statistics

Server Log Menu

    * Server Log
    * Log Statistics
    * Thread Dump

Server Log
Lines (max. 400) reversed order

D 2009/02/13 08:20:26 CRAWLER LOCALCRAWL[28, 1460, 0, 0]: URL=http://feeds.feedburner.com/~r/blogg/ch/~3/524567943/index.php, initiator=O1VUEDy91_AA, crawlOrder=true, depth=1, crawlDepth=2, must-match=.*, must-not-match=, permission=true

I 2009/02/13 08:20:26 CRAWLER LOCALCRAWL[28, 1460, 0, 0]: enqueued for load http://feeds.feedburner.com/~r/blogg/ch/~3/524567943/index.php [Fj2LVsH-7FkZ]

I 2009/02/13 08:20:26 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3399 / -357 KB)

W 2009/02/13 08:20:26 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3395 KB, needed: 4096 KB

I 2009/02/13 08:20:26 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL w91WAxP290EA:http://www.heise.de/newsticker/BitDefender-Webseite-ebenfalls-undicht-Update--/meldung/

I 2009/02/13 08:20:28 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3702 / -357 KB)

W 2009/02/13 08:20:28 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3698 KB, needed: 6144 KB

I 2009/02/13 08:20:29 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2957 / -357 KB)

W 2009/02/13 08:20:35 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3684 KB, needed: 4096 KB

I 2009/02/13 08:20:40 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2705 / -357 KB)

W 2009/02/13 08:20:40 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2623 KB, needed: 4096 KB

I 2009/02/13 08:20:42 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL lYb7LxP290EA:http://www.heise.de/newsticker/Jobsucher-Daten-versehentlich-veroeffentlicht--/meldung/127077

I 2009/02/13 08:20:42 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL p_gbYxP290EA:http://www.heise.de/newsticker/Magazin-Streit-um-erweiterte-Befugnisse-fuer-deutsche-Geheimdienste--/meldung/

I 2009/02/13 08:20:42 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL 9n1BZ0wqcWfA:http://www.itgrl.de/tag/disk/

I 2009/02/13 08:20:45 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1990 / -357 KB)

W 2009/02/13 08:20:45 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1980 KB, needed: 4096 KB

I 2009/02/13 08:20:47 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL r8hsuxP290EA:http://www.heise.de/newsticker/FDP-Politiker-fordert-Debatte-ueber-geplantes-Gluehlampen-Verbot--/meldung/

I 2009/02/13 08:20:47 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1220 / -357 KB)

W 2009/02/13 08:20:47 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1216 KB, needed: 4096 KB

I 2009/02/13 08:20:49 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 163 / -357 KB)

W 2009/02/13 08:20:49 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 158 KB, needed: 4096 KB

I 2009/02/13 08:20:51 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5407 / -357 KB)

W 2009/02/13 08:20:51 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 5361 KB, needed: 6144 KB

D 2009/02/13 08:20:51 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:20:53 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL c5_gMqwqcWfA:http://www.itgrl.de/2009/02/06/fieser-bosewicht/trackback/

I 2009/02/13 08:20:53 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL nebkkxP290EA:http://www.heise.de/newsticker/Texas-Instrument-kauft-Know-how-zur-Steigerung-der-Energieeffizienz--/meldung/132402

I 2009/02/13 08:20:53 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1236 / -357 KB)

W 2009/02/13 08:20:53 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 410 KB, needed: 4096 KB

I 2009/02/13 08:20:57 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3332 / -357 KB)

W 2009/02/13 08:20:57 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3321 KB, needed: 6144 KB

I 2009/02/13 08:20:57 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3312 / -357 KB)

I 2009/02/13 08:20:59 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3306 / -357 KB)

W 2009/02/13 08:20:59 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1922 KB, needed: 4096 KB

W 2009/02/13 08:20:59 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 1852 KB, needed: 6144 KB

D 2009/02/13 08:21:01 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:21:01 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL _uBeNxP290EA:http://www.heise.de/newsticker/Cyber-Zoellner-machen-in-Frankreich-Jagd-auf-Internet-Kriminelle--/meldung/132294

I 2009/02/13 08:21:01 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL g-kv0CbIJ_XC:http://www.trojaner-board.de/archive/index.php/f-7-p-2.html

I 2009/02/13 08:21:01 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL B2UrAxP290EA:http://www.heise.de/newsticker/Lehrerbewertungs-Portal-Spickmich-erhaelt-Negativpreis--/meldung/

I 2009/02/13 08:21:03 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3813 / -357 KB)

W 2009/02/13 08:21:03 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3804 KB, needed: 4096 KB

I 2009/02/13 08:21:03 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3804 / -357 KB)

W 2009/02/13 08:21:03 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3711 KB, needed: 6144 KB

I 2009/02/13 08:21:07 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2860 / -357 KB)

W 2009/02/13 08:21:07 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2633 KB, needed: 4096 KB

D 2009/02/13 08:21:10 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:21:10 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL NK_5t0wqcWfA:http://www.itgrl.de/tag/ctfmonexe/

I 2009/02/13 08:21:10 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL 87WHdxP290EA:http://www.heise.de/newsticker/Domainpulse-Neue-Adresszonen-ja-aber-bitte-nicht-so-viele--/meldung/

I 2009/02/13 08:21:10 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL ONiYuyP290EA:http://www.heise.de/software/download/packettrap_perspective/59354

D 2009/02/13 08:21:12 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:21:13 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL uVbUzxP290EA:http://www.heise.de/newsticker/Activision-Blizzard-schreibt-rote-Zahlen--/meldung/

D 2009/02/13 08:21:15 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

D 2009/02/13 08:21:17 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:21:17 PLASMA Received 38 Entries 20 Words [8dGG5jK4RVMG .. 8dGu8NAd9Xnm]/2648640577065278792 from DvgzXMfS8oDp:yacy-suche_de/0.72005606, processed in 37228 milliseconds, requesting 16/38 URLs, blocked 0 RWIs

I 2009/02/13 08:21:17 PLASMA *Indexed 787 words in URL http://feeds.feedburner.com/~r/blogg/ch/~3/508040039/ [_Wv1JsH-7FkZ]
   Description:  IFPI betreibt tranceradio.ch - Schweizer Internetradios kämpfen ums Überleben (Tages-Anzeiger) - CH Internet Szene
   MimeType: text/html | Charset: ISO-8859-1 | Size: 13439 bytes | Anchors: 113
   LinkStorageTime: 9 ms | indexStorageTime: 57954 ms

I 2009/02/13 08:21:17 PLASMA Excluded 0 words in URL http://www.politikon.org/e-lernplattform/lizenzmodelle/lizenz-fuer-freie-inhalte.html

I 2009/02/13 08:21:18 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3462 / -357 KB)

W 2009/02/13 08:21:18 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3458 KB, needed: 6144 KB

I 2009/02/13 08:21:19 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3304 / -357 KB)

W 2009/02/13 08:21:19 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3300 KB, needed: 4096 KB

I 2009/02/13 08:21:23 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2035 / -357 KB)

W 2009/02/13 08:21:23 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1763 KB, needed: 4096 KB

I 2009/02/13 08:21:23 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 1759 / -357 KB)

W 2009/02/13 08:21:23 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob' runs short memory cycle. Free mem: 1755 KB, needed: 6144 KB

E 2009/02/13 08:21:31 BUSYTHREAD Runtime Error in serverInstantThread.freemem, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

I 2009/02/13 08:21:31 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 1735 / -357 KB)

I 2009/02/13 08:21:31 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1734 / -357 KB)

W 2009/02/13 08:21:31 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1719 KB, needed: 4096 KB

W 2009/02/13 08:21:31 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 1726 KB, needed: 6144 KB

D 2009/02/13 08:21:34 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=82)

I 2009/02/13 08:21:34 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL xllEdxP290EA:http://www.heise.de/newsticker/Open-Source-Manager-appellieren-an-Obama--/meldung/132296

I 2009/02/13 08:21:34 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 2172 / -357 KB)

W 2009/02/13 08:21:34 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 2168 KB, needed: 6144 KB

D 2009/02/13 08:21:37 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=81)

D 2009/02/13 08:21:39 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=81)

D 2009/02/13 08:21:42 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=81)

I 2009/02/13 08:21:42 PLASMA CRAWL: ADDED 184 LINKS FROM http://www.itgrl.de/2009/02/05/samsung-5mm-mlc-ssd/samsungssd/, NEW CRAWL STACK SIZE IS 27, STACKING TIME = 76, PARSING TIME = 179

I 2009/02/13 08:21:42 PLASMA CRAWL: ADDED 199 LINKS FROM http://feeds.feedburner.com/~r/blogg/ch/~3/474737919/, NEW CRAWL STACK SIZE IS 27, STACKING TIME = 2791, PARSING TIME = 5827

I 2009/02/13 08:21:42 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3311 / -357 KB)

W 2009/02/13 08:21:42 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3306 KB, needed: 6144 KB

D 2009/02/13 08:21:46 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

D 2009/02/13 08:21:49 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

D 2009/02/13 08:21:52 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:21:52 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5497 / -357 KB)

W 2009/02/13 08:21:52 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 5493 KB, needed: 6144 KB

I 2009/02/13 08:21:56 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5525 / -357 KB)

W 2009/02/13 08:21:56 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob' runs short memory cycle. Free mem: 2736 KB, needed: 6144 KB

D 2009/02/13 08:21:56 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

D 2009/02/13 08:22:00 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

D 2009/02/13 08:22:03 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:03 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5693 / -357 KB)

W 2009/02/13 08:22:03 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 4453 KB, needed: 6144 KB

I 2009/02/13 08:22:03 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL nhbqIxP290EA:http://www.heise.de/newsticker/heise-online-Meldungen-als-individueller-Nachrichten-Podcast--/meldung/132473

I 2009/02/13 08:22:03 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL 76eSwxP290EA:http://www.heise.de/newsticker/Plattformneutraler-MySQL-Profiler--/meldung/132303

D 2009/02/13 08:22:05 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:05 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3782 / -357 KB)

W 2009/02/13 08:22:05 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3754 KB, needed: 6144 KB

I 2009/02/13 08:22:05 PLASMA crawlReceipt: RECEIVED RECEIPT from VnRbvb7EhdAA:KIT16F/0.70005494 for URL 3K3xQxP290EA:http://www.heise.de/newsticker/GlassFish-Portfolio-Suns-neue-Plattform-zum-Entwickeln-von-Webanwendungen--/meldung/

I 2009/02/13 08:22:08 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3096 / -357 KB)

W 2009/02/13 08:22:08 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3090 KB, needed: 4096 KB

I 2009/02/13 08:22:10 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3088 / -357 KB)

W 2009/02/13 08:22:10 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 483 KB, needed: 4096 KB

I 2009/02/13 08:22:14 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2188 / -357 KB)

W 2009/02/13 08:22:14 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2149 KB, needed: 4096 KB

I 2009/02/13 08:22:16 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2117 / -357 KB)

W 2009/02/13 08:22:16 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 961 KB, needed: 4096 KB

I 2009/02/13 08:22:20 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2190 / -357 KB)

W 2009/02/13 08:22:20 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2174 KB, needed: 4096 KB

E 2009/02/13 08:22:20 BUSYTHREAD Runtime Error in serverInstantThread.freemem, thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

D 2009/02/13 08:22:22 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:22 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL oeu0RxP290EA:http://www.heise.de/newsticker/Familienministerin-kaempft-an-allen-Fronten-fuer-Kinderporno-Sperren--/meldung/

I 2009/02/13 08:22:22 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 4930 / -357 KB)

W 2009/02/13 08:22:22 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 4852 KB, needed: 6144 KB

I 2009/02/13 08:22:23 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL GqEb1xP290EA:http://www.heise.de/newsticker/Umsatz-mit-Bildtontraegern-ging-2008-zurueck--/meldung/132410

I 2009/02/13 08:22:23 PLASMA crawlReceipt: RECEIVED RECEIPT from kEMTBrB7jyrn:KIT051F300-dulcedo/0.72005594 for URL UVMSQCbIJ_XC:http://www.trojaner-board.de/archive/index.php/f-21-p-13.html

I 2009/02/13 08:22:23 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL 21gGlxP290EA:http://www.heise.de/newsticker/Wirtschaftsministerium-zeigt-Theseus-Teilanwendungen--/meldung/

I 2009/02/13 08:22:23 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL xRlS-xP290EA:http://www.heise.de/newsticker/Ex-eBay-Chefin-Whitman-will-Schwarzenegger-als-Gouverneur-beerben--/meldung/

I 2009/02/13 08:22:23 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL 1re-KCbIJ_XC:http://www.trojaner-board.de/archive/index.php/f-20-p-34.html

D 2009/02/13 08:22:25 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:25 PLASMA *Indexed 581 words in URL http://www.politikon.org/e-lernplattform/lizenzmodelle/lizenz-fuer-freie-inhalte.html [fGGijrde-3ER]
   Description:  Lizenz für freie Inhalte
   MimeType: text/html | Charset: ISO-8859-1 | Size: 16784 bytes | Anchors: 15
   LinkStorageTime: 4 ms | indexStorageTime: 67458 ms

I 2009/02/13 08:22:25 PLASMA Excluded 0 words in URL http://blog.yacy-websuche.de/2009/01/

D 2009/02/13 08:22:27 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:27 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 4044 / -357 KB)

W 2009/02/13 08:22:27 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob' runs short memory cycle. Free mem: 4039 KB, needed: 6144 KB

I 2009/02/13 08:22:29 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2313 / -357 KB)

W 2009/02/13 08:22:29 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2309 KB, needed: 4096 KB

I 2009/02/13 08:22:31 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5551 / -357 KB)

D 2009/02/13 08:22:31 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

W 2009/02/13 08:22:31 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 5543 KB, needed: 6144 KB

D 2009/02/13 08:22:33 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:33 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 4762 / -357 KB)

W 2009/02/13 08:22:33 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 4757 KB, needed: 6144 KB

I 2009/02/13 08:22:35 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 2554 / -357 KB)

W 2009/02/13 08:22:35 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 2550 KB, needed: 6144 KB

I 2009/02/13 08:22:35 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2509 / -357 KB)

W 2009/02/13 08:22:35 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2505 KB, needed: 4096 KB

D 2009/02/13 08:22:39 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:41 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 3705 / -357 KB)

W 2009/02/13 08:22:41 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 3563 KB, needed: 6144 KB

I 2009/02/13 08:22:41 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3009 / -357 KB)

W 2009/02/13 08:22:41 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3003 KB, needed: 4096 KB

I 2009/02/13 08:22:43 PLASMA crawlReceipt: RECEIVED RECEIPT from iS7NivAgXU93:KIT002F/0.720056 for URL 2q3HHCbIJ_XC:http://www.trojaner-board.de/archive/index.php/f-21-p-20.html

D 2009/02/13 08:22:43 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:45 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL bpmZoxP290EA:http://www.heise.de/newsticker/Deutsch-aegyptischer-Blogger-freigelassen--/meldung/

I 2009/02/13 08:22:45 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2196 / -357 KB)

W 2009/02/13 08:22:45 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2031 KB, needed: 4096 KB

I 2009/02/13 08:22:47 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 259 / -357 KB)

W 2009/02/13 08:22:47 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 241 KB, needed: 4096 KB

I 2009/02/13 08:22:49 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5267 / -357 KB)

W 2009/02/13 08:22:49 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 5262 KB, needed: 6144 KB

D 2009/02/13 08:22:49 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:22:51 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 2863 / -357 KB)

W 2009/02/13 08:22:51 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 2859 KB, needed: 4096 KB

I 2009/02/13 08:22:55 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 102 / -357 KB)

W 2009/02/13 08:22:55 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 97 KB, needed: 4096 KB

I 2009/02/13 08:22:57 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 404 / -357 KB)

W 2009/02/13 08:22:57 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob' runs short memory cycle. Free mem: 375 KB, needed: 6144 KB

I 2009/02/13 08:22:57 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL sdO7JxP290EA:http://www.heise.de/newsticker/Hosting-von-Tripod-laeuft-weiter--/meldung/

E 2009/02/13 08:23:03 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at C:\Users\lm\yacy\htroot\yacy\transferRWI.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1173)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:752)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:248)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:629)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/13 08:23:03 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 398 / -357 KB)

W 2009/02/13 08:23:03 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 219 KB, needed: 4096 KB

I 2009/02/13 08:23:04 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 5208 / -357 KB)

W 2009/02/13 08:23:04 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 5201 KB, needed: 6144 KB

I 2009/02/13 08:23:06 PLASMA crawlReceipt: RECEIVED RECEIPT from NSmztVm6vhRN:KIT55F300/0.7100558 for URL XQRwKxP290EA:http://www.heise.de/newsticker/Nokia-schliesst-Forschungsstandort-und-drosselt-Handy-Produktion--/meldung/

I 2009/02/13 08:23:06 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3903 / -357 KB)

W 2009/02/13 08:23:06 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3898 KB, needed: 4096 KB

W 2009/02/13 08:23:06 FILEHANDLER Unexpected error while processing query.
Session: Session_84.143.120.251:43403#0
Query:   /yacy/transferRWI.html
Client:  84.143.120.251
Reason:  java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1173)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:752)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:248)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:629)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/13 08:23:06 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL SCZB_xP290EA:http://www.heise.de/newsticker/Britische-CSR-uebernimmt-GPS-Chiphersteller-SiRF--/meldung/132354

I 2009/02/13 08:23:06 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL 1lRVexP290EA:http://www.heise.de/newsticker/Weiter-Hickhack-um-Digital-TV-Umstieg-in-den-USA--/meldung/132336

I 2009/02/13 08:23:06 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL nZF1_xP290EA:http://www.heise.de/newsticker/Touch-MP3-Spieler-von-Samsung--/meldung/132416

I 2009/02/13 08:23:06 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL 6NUDnxP290EA:http://www.heise.de/newsticker/60-Prozent-Umsatzrueckgang-bei-Nvidia--/meldung/132318

D 2009/02/13 08:23:09 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:23:14 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 226 / -357 KB)

I 2009/02/13 08:23:14 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 226 / -357 KB)

E 2009/02/13 08:23:19 BLOCKINGTHREAD Runtime Error in serverInstantThread.job, thread 'java.lang.reflect.Method.storeDocumentIndex.17': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space

W 2009/02/13 08:23:19 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 164 KB, needed: 4096 KB

W 2009/02/13 08:23:19 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 135 KB, needed: 6144 KB

I 2009/02/13 08:23:21 PLASMA crawlReceipt: RECEIVED RECEIPT from NFbwe-AtQkNc:suma-ev6/0.61505365 for URL cqWxZxP290EA:http://www.heise.de/newsticker/IT-Distributor-Devil-entzieht-sich-dem-Zugriff-der-Mehrheitseigner--/meldung/

I 2009/02/13 08:23:21 PLASMA Excluded 0 words in URL http://www.heise.de/newsticker/heise-de-angeblich-boesartig--/meldung/127068

I 2009/02/13 08:23:23 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 4005 / -357 KB)

W 2009/02/13 08:23:23 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 4001 KB, needed: 4096 KB

I 2009/02/13 08:23:23 PLASMA Received 33 Entries 19 Words [skeR03fNXDCi .. skeyNIK96PUF]/-4285499375070995792 from c4GOj4NApRRf:apfelmaennchen/0.720056, processed in 14523 milliseconds, requesting 22/33 URLs, blocked 0 RWIs

D 2009/02/13 08:23:25 CRAWLER CoreCrawl: too many processes in indexing queue, dismissed (sbQueueSize=80)

I 2009/02/13 08:23:25 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 4398 / -357 KB)

W 2009/02/13 08:23:25 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.deQueueProcess' runs short memory cycle. Free mem: 4355 KB, needed: 6144 KB

I 2009/02/13 08:23:27 PLASMA Received 22 URLs from peer c4GOj4NApRRf:apfelmaennchen/0.720056 in 1861 ms, blocked 0 URLs

I 2009/02/13 08:23:27 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 3974 / -357 KB)

W 2009/02/13 08:23:27 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 3970 KB, needed: 4096 KB

I 2009/02/13 08:23:27 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 1217 / -357 KB)

W 2009/02/13 08:23:27 PLASMA Thread 'de.anomic.plasma.plasmaSwitchboard.dhtTransferJob' runs short memory cycle. Free mem: 1213 KB, needed: 6144 KB

I 2009/02/13 08:23:29 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 340 / -357 KB)

W 2009/02/13 08:23:29 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 336 KB, needed: 4096 KB

I 2009/02/13 08:23:33 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1317 / -357 KB)

W 2009/02/13 08:23:33 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1312 KB, needed: 4096 KB

E 2009/02/13 08:23:37 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at C:\Users\lm\yacy\htroot\yacy\transferRWI.class: java.lang.OutOfMemoryError: Java heap space:Java heap space
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.http.httpdFileHandler.invokeServlet(httpdFileHandler.java:1173)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:752)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:248)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:629)
Caused by: java.lang.OutOfMemoryError: Java heap space

I 2009/02/13 08:23:37 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 4096 / 1235 / -357 KB)

W 2009/02/13 08:23:37 PLASMA Thread 'de.anomic.crawler.CrawlQueues.coreCrawlJob' runs short memory cycle. Free mem: 1225 KB, needed: 4096 KB

I 2009/02/13 08:23:37 MEMORY former GCs indicate to not be able to free enough memory (requested/available/average: 6144 / 1209 / -357 KB)

W 2009/02/13 08:23:37 PLASMA Thread 'de.anomic.crawler.CrawlQueues.remoteTriggeredCrawlJob' runs short memory cycle. Free mem: 1198 KB, needed: 6144 KB

E 2009/02/13 08:23:37 HTTPD Unexpected Error. java.net.SocketException: Software caused connection abort: recv failed
java.net.SocketException: Software caused connection abort: recv failed
   at java.net.SocketInputStream.socketRead0(Native Method)
   at java.net.SocketInputStream.read(Unknown Source)
   at java.net.SocketInputStream.read(Unknown Source)
   at java.io.FilterInputStream.read(Unknown Source)
   at java.io.PushbackInputStream.read(Unknown Source)
   at org.apache.commons.httpclient.ChunkedInputStream.readCRLF(ChunkedInputStream.java:204)
   at org.apache.commons.httpclient.ChunkedInputStream.nextChunk(ChunkedInputStream.java:219)
   at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:176)
   at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:196)
   at org.apache.commons.httpclient.ChunkedInputStream.exhaustInputStream(ChunkedInputStream.java:369)
   at org.apache.commons.httpclient.ChunkedInputStream.close(ChunkedInputStream.java:346)
   at de.anomic.http.httpd.POST(httpd.java:647)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:629)

W 2009/02/13 08:23:37 FILEHANDLER Unexpected error while processing query.
Session: Session_202.168.100.149:58298#3
Query:   /yacy/transferRWI.html
Client:  202.168.100.149
Reason:  java.net.SocketTimeoutException: Read timed out
java.net.SocketTimeoutException: Read timed out
   at java.net.SocketInputStream.socketRead0(Native Method)
   at java.net.SocketInputStream.read(Unknown Source)
   at java.net.SocketInputStream.read(Unknown Source)
   at java.io.FilterInputStream.read(Unknown Source)
   at java.io.PushbackInputStream.read(Unknown Source)
   at org.apache.commons.httpclient.ChunkedInputStream.read(ChunkedInputStream.java:152)
   at java.util.zip.CheckedInputStream.read(Unknown Source)
   at java.util.zip.GZIPInputStream.readUByte(Unknown Source)
   at java.util.zip.GZIPInputStream.readUShort(Unknown Source)
   at java.util.zip.GZIPInputStream.readHeader(Unknown Source)
   at java.util.zip.GZIPInputStream.<init>(Unknown Source)
   at java.util.zip.GZIPInputStream.<init>(Unknown Source)
   at de.anomic.http.httpd.prepareBody(httpd.java:951)
   at de.anomic.http.httpd.parseMultipart(httpd.java:892)
   at de.anomic.http.httpdFileHandler.doResponse(httpdFileHandler.java:352)
   at de.anomic.http.httpdFileHandler.doPost(httpdFileHandler.java:248)
   at de.anomic.http.httpd.POST(httpd.java:629)
   at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:762)
   at de.anomic.server.serverCore$Session.run(serverCore.java:629)





Beides normale Bürorechner (cor2duo) unter Win32 oder 64.

Weiterhin ist mir aufgefallen das Yacy nun auch permanent mehr RAM braucht. Ich vermute weil er mehr DHT-in hat, was ich auch an der Index-Grösse sehe, die wächst auch auf älteren(!) Peers zur Zeit sehr stark.
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon thq » Fr Feb 13, 2009 10:00 am

5603 hat diesmal ein richtigen OOM BUG wie es hier schon steht viewtopic.php?f=6&t=1844
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: Speichermangel und Startschwierigkeiten

Beitragvon Hermes » Fr Feb 13, 2009 11:43 am

Da der Fehler nur beim Versenden auftritt, reicht es, fuer Versionen ab SVN 5586 die Indexverteilung abzuschalten und yacy neu zu starten. Wenn der Peer aber bereits OOM-Symptome zeigt wird der Shutdown nicht mehr richtig funktionieren.
Hermes
 
Beiträge: 50
Registriert: Mi Jun 27, 2007 11:10 pm
Wohnort: Ratingen

Re: Speichermangel und Startschwierigkeiten

Beitragvon dulcedo » Fr Feb 13, 2009 12:20 pm

Ja das ist das Problem, ich kann ja dann auch nicht vom web-interface aus updaten wenn YaCy sich nicht beenden lässt.
Reicht es ganz runterzufahren und nach Systemstart das update-Archiv einzukopieren? Egal welchen Versions-Sprung?
dulcedo
 
Beiträge: 1006
Registriert: Do Okt 16, 2008 6:36 pm
Wohnort: Bei Karlsruhe

Re: Speichermangel und Startschwierigkeiten

Beitragvon Lotus » Sa Feb 14, 2009 9:21 am

Ja, rüberkopieren reicht. Über das Webinterface passiert nichts anderes. ;)
Lotus
 
Beiträge: 1699
Registriert: Mi Jun 27, 2007 3:33 pm
Wohnort: Hamburg

Nächste

Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 3 Gäste