Kein regelmäßiger Flush des RAM chaches (r7388)

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Di Dez 21, 2010 4:42 pm

Mein Peer crawlt lustig vor sich hin und füllt dabei den RAM word cache. Die Größe hatte ich auf 50000 gesetzt und jetzt zum testen auf 25000 verringert, aber es findet kein Flush statt. Der Cache ist bei mir im Moment mit ca. 40000 Wörtern gefüllt. Der Speicher läuft immer weiter voll und der Peer wird immer langsamer. Heute Morgen war er irgendwann mal komplett weg und ich konnte ihn nur durch eine kill wiederbeleben.

Kann das noch jemand beobachten?
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Orbiter » Mi Dez 22, 2010 12:01 am

so ein Mist. Hast du irgendwo eine Exception im Log? Sollte dann in der Klasse IndexCell vorkommen, so um Zeile 496. Wenn du das hättest dann hätte ich den Bug gefunden, sonst muss ich weiter suchen.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Mi Dez 22, 2010 12:40 am

Ich hätte das hier anzubieten (beim Herunterfahren für Neustart):
Code: Alles auswählen
I 2010/12/21 19:56:10 HeapReader using a dump of the index of /home/yacy/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20101221185529450.blob.
W 2010/12/21 19:56:10 IODispatcher emergency dump of file text.index.20101221185529522.blob
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at net.yacy.kelondro.blob.ArrayStack.mountBLOB(ArrayStack.java:245)
        at net.yacy.kelondro.blob.ArrayStack.mergeMount(ArrayStack.java:833)
        at net.yacy.kelondro.rwi.IODispatcher.merge(IODispatcher.java:117)
        at net.yacy.kelondro.rwi.ReferenceContainerArray.shrink(ReferenceContainerArray.java:323)
        at net.yacy.kelondro.rwi.IndexCell.cleanCache(IndexCell.java:512)
        at net.yacy.kelondro.rwi.IndexCell.add(IndexCell.java:138)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
I 2010/12/21 19:56:10 indexContainerRAMHeap creating rwi heap dump 'text.index.20101221185529522.blob', 61629 rwi's
W 2010/12/21 19:56:10 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'E8GxKqhrY2KR' is known.
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.urlMetadata(Segments.java:172)
        at transferRWI.respond(transferRWI.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.termIndex(Segments.java:176)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.termIndex(Segments.java:176)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'GLsiuChrY2KR' is known.
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.urlMetadata(Segments.java:172)
        at transferRWI.respond(transferRWI.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.termIndex(Segments.java:176)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'GdL4bChrY2KR' is known.
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.urlMetadata(Segments.java:172)
        at transferRWI.respond(transferRWI.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.termIndex(Segments.java:176)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'HB6PMxhrY2KR' is known.
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.urlMetadata(Segments.java:172)
        at transferRWI.respond(transferRWI.java:193)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/21 19:56:10 StackTrace null
java.lang.NullPointerException
        at de.anomic.search.Segments.segment(Segments.java:132)
        at de.anomic.search.Segments.termIndex(Segments.java:176)
        at transferRWI.respond(transferRWI.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
        at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
        at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
        at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
        at de.anomic.server.serverCore$Session.run(serverCore.java:630)


Und so geht es noch einige Zeit weiter, bis YaCy dann irgendwann gestoppt wird. Nach dem Neustart findet sich aber kein "IndexCell" mehr in den Logs, einen Flush gibt es aber trotzdem nicht,
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Mi Dez 22, 2010 12:49 am

Eben beim erneuten Neustart:

Code: Alles auswählen
I 2010/12/22 00:42:18 PLASMA Saving Web Structure File
I 2010/12/22 00:42:19 NoticedURL CLOSING ALL STACKS
I 2010/12/22 00:42:19 Heap closing heap /home/yacy/yacy/DATA/INDEX/freeworld/QUEUES/crawlProfilesActive.heap
I 2010/12/22 00:42:19 kelondroBLOBHeap wrote a dump for the 11 index entries of crawlProfilesActive.heap in 27 milliseconds.
I 2010/12/22 00:42:19 Heap closing heap /home/yacy/yacy/DATA/INDEX/freeworld/QUEUES/crawlProfilesPassive.heap
I 2010/12/22 00:42:19 kelondroBLOBHeapWriter wrote a dump for the 59627 index entries of text.index.20101221190248022.blob in 134 milliseconds.
I 2010/12/22 00:42:21 HeapReader saturation of text.index.20101221190248022.blob.Yon6phvyndew.idx: keylength = 4, vallength = 4, possible saving: 0 MB
I 2010/12/22 00:42:21 HeapReader using a dump of the index of /home/yacy/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20101221190248022.blob.
I 2010/12/22 00:42:21 BLOBArray merged text.index.20101217112700047.blob with text.index.20101217113701086.blob into /home/yacy/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20101221190248022.blob
W 2010/12/22 00:42:21 IODispatcher emergency dump of file text.index.20101221191402598.blob
I 2010/12/22 00:42:21 indexContainerRAMHeap creating rwi heap dump 'text.index.20101221191402598.blob', 3001 rwi's
I 2010/12/22 00:42:21 RICELL-shrink1 unmountBestMatch(2.0, 268435456)
I 2010/12/22 00:42:21 kelondroBLOBHeapWriter wrote a dump for the 3001 index entries of text.index.20101221191402598.blob in 7 milliseconds.
I 2010/12/22 00:42:21 indexContainerRAMHeap finished rwi heap dump: 3001 words, 0 word/URL relations in 57 milliseconds
E 2010/12/22 00:42:37 STARTUP Unexpected Error: java.lang.NullPointerException
java.lang.NullPointerException
   at net.yacy.kelondro.rwi.ReferenceContainerCache.remove(ReferenceContainerCache.java:368)
   at net.yacy.kelondro.rwi.IndexCell.remove(IndexCell.java:325)
   at net.yacy.kelondro.rwi.IndexCell.removeDelayed(IndexCell.java:311)
   at net.yacy.kelondro.rwi.IndexCell.close(IndexCell.java:426)
   at de.anomic.search.Segment.close(Segment.java:262)
   at de.anomic.search.Segments.close(Segments.java:192)
   at de.anomic.search.Switchboard.close(Switchboard.java:1198)
   at net.yacy.yacy.startup(yacy.java:419)
   at net.yacy.yacy.main(yacy.java:1020)
S 2010/12/22 00:42:37 SHUTDOWN goodbye. (this is the last line)


Warum da jetzt vor dem entgültigen Shutdown was von STARTUP steht, verstehe ich nicht, es war aber so. Beim Start danach sind keine Exceptions aufgetreten.
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Orbiter » Mi Dez 22, 2010 1:05 am

für den letzten trace gibts einen fix in SVN 7390
bei den ganz oben blicke ich noch nicht durch.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Mi Dez 22, 2010 2:01 am

Habe jetzt 7391 installiert. Es wurde geflusht, weil einige Zeit vergangen war. Ich habe dann den Crawler gestartet und dann wurde wieder nicht mehr geflusht. Ich habe dann den Peer heruntergefahren und wieder sowas wie oben im Log gefunden:
Code: Alles auswählen
I 2010/12/22 01:43:58 Heap closing heap /home/yacy/yacy/DATA/HTCACHE/responseHeader.heap
I 2010/12/22 01:43:58 HeapReader saturation of text.index.20101222004324363.blob.00psnaH-j25P.idx: keylength = 4, vallength = 4, possible saving: 0 MB
I 2010/12/22 01:43:58 HeapReader using a dump of the index of /home/yacy/yacy/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20101222004324363.blob.
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at net.yacy.kelondro.blob.ArrayStack.mountBLOB(ArrayStack.java:245)
   at net.yacy.kelondro.blob.ArrayStack.mergeMount(ArrayStack.java:833)
   at net.yacy.kelondro.rwi.IODispatcher.merge(IODispatcher.java:117)
   at net.yacy.kelondro.rwi.ReferenceContainerArray.shrink(ReferenceContainerArray.java:323)
   at net.yacy.kelondro.rwi.IndexCell.cleanCache(IndexCell.java:512)
   at net.yacy.kelondro.rwi.IndexCell.add(IndexCell.java:138)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'FM0JxZYHYYyR' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'HaaNyLuEXo6Y' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'HvIzHd8FRczR' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'H09s8LuEXo6Y' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'EEExipG782lR' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'Eb5LSpG782lR' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 StackTrace null
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:185)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
W 2010/12/22 01:43:58 PLASMA transferRWI: DB-Error while trying to determine if URL with hash 'EcpL9pG782lR' is known.
java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.urlMetadata(Segments.java:172)
   at transferRWI.respond(transferRWI.java:193)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
I 2010/12/22 01:43:58 PLASMA Received 7 RWIs, 5 Words [udVhEZ4--EoT .. ueJlis3ev-AP]/3594327050424488856, blocked 0, requesting 0/0 URLs from MaOCn_Y45e8l:nasenblut/0.9900731
E 2010/12/22 01:43:58 FILEHANDLER INTERNAL ERROR: java.lang.reflect.InvocationTargetException:null target exception at /home/yacy/yacy/htroot/yacy/transferRWI.class: java.lang.NullPointerException:null
java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1194)
   at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:786)
   at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:239)
   at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:542)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:597)
   at de.anomic.server.serverCore$Session.listen(serverCore.java:731)
   at de.anomic.server.serverCore$Session.run(serverCore.java:630)
Caused by: java.lang.NullPointerException
   at de.anomic.search.Segments.segment(Segments.java:132)
   at de.anomic.search.Segments.termIndex(Segments.java:176)
   at transferRWI.respond(transferRWI.java:226)
   ... 14 more
I 2010/12/22 01:43:59 kelondroBLOBHeap wrote a dump for the 55961 index entries of text.index.20101222004324363.blob in 30 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 6537310 index entries of responseHeader.heap in 1254 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 905 index entries of HZ8eH4RRA0__.20101212122003369.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1238 index entries of HZ8eH4RRA0__.20101212133151829.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 562 index entries of HZ8eH4RRA0__.20101212145949862.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 937 index entries of HZ8eH4RRA0__.20101212153629914.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 831 index entries of HZ8eH4RRA0__.20101212163919365.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 2070 index entries of HZ8eH4RRA0__.20101212173608028.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 955 index entries of HZ8eH4RRA0__.20101212193912765.blob in 3 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1282 index entries of HZ8eH4RRA0__.20101212203320835.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 818 index entries of HZ8eH4RRA0__.20101212220636091.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 626 index entries of HZ8eH4RRA0__.20101212230551111.blob in 14 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 681 index entries of HZ8eH4RRA0__.20101212233311589.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 700 index entries of HZ8eH4RRA0__.20101213004740340.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1211 index entries of HZ8eH4RRA0__.20101213015111652.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1474 index entries of HZ8eH4RRA0__.20101213031128630.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1228 index entries of HZ8eH4RRA0__.20101213044856948.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 761 index entries of HZ8eH4RRA0__.20101213064232806.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 669 index entries of HZ8eH4RRA0__.20101213074422802.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 295 index entries of HZ8eH4RRA0__.20101213081926027.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 542 index entries of HZ8eH4RRA0__.20101213083258326.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 207 index entries of HZ8eH4RRA0__.20101213090635086.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 822 index entries of HZ8eH4RRA0__.20101213091946240.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 809 index entries of HZ8eH4RRA0__.20101213100219488.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 820 index entries of HZ8eH4RRA0__.20101213105214557.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 483 index entries of HZ8eH4RRA0__.20101213113418634.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 857 index entries of HZ8eH4RRA0__.20101213115702627.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 820 index entries of HZ8eH4RRA0__.20101213124841074.blob in 3 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 625 index entries of HZ8eH4RRA0__.20101213133344331.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 615 index entries of HZ8eH4RRA0__.20101213141123878.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 701 index entries of HZ8eH4RRA0__.20101213143007486.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 985 index entries of HZ8eH4RRA0__.20101213144000379.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 75 index entries of HZ8eH4RRA0__.20101213151103626.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 461 index entries of HZ8eH4RRA0__.20101213151212175.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 199 index entries of HZ8eH4RRA0__.20101213153051085.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 4 index entries of HZ8eH4RRA0__.20101213153247813.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 351 index entries of HZ8eH4RRA0__.20101213153318965.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1036 index entries of HZ8eH4RRA0__.20101213155556015.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 2223 index entries of HZ8eH4RRA0__.20101214082944574.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1325 index entries of HZ8eH4RRA0__.20101215221949209.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 245 index entries of HZ8eH4RRA0__.20101216100845004.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1268 index entries of HZ8eH4RRA0__.20101216101655557.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 939 index entries of HZ8eH4RRA0__.20101217091329736.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 994 index entries of HZ8eH4RRA0__.20101217141020189.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1077 index entries of HZ8eH4RRA0__.20101217143749679.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1084 index entries of HZ8eH4RRA0__.20101217145749468.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 479 index entries of HZ8eH4RRA0__.20101217234654398.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 648 index entries of HZ8eH4RRA0__.20101218005837240.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 661 index entries of HZ8eH4RRA0__.20101218045021888.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 518 index entries of HZ8eH4RRA0__.20101218093701565.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 697 index entries of HZ8eH4RRA0__.20101218094222861.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 949 index entries of HZ8eH4RRA0__.20101218163131327.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1441 index entries of HZ8eH4RRA0__.20101219103002118.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1181 index entries of HZ8eH4RRA0__.20101220080213389.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 816 index entries of HZ8eH4RRA0__.20101220092440391.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 887 index entries of HZ8eH4RRA0__.20101220102126483.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 984 index entries of HZ8eH4RRA0__.20101220113156414.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1004 index entries of HZ8eH4RRA0__.20101220124831395.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 417 index entries of HZ8eH4RRA0__.20101220140832971.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 326 index entries of HZ8eH4RRA0__.20101220142110428.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 528 index entries of HZ8eH4RRA0__.20101220142912048.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 785 index entries of HZ8eH4RRA0__.20101220144701394.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 961 index entries of HZ8eH4RRA0__.20101220154821729.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1420 index entries of HZ8eH4RRA0__.20101220163301314.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 404 index entries of HZ8eH4RRA0__.20101220220226639.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 2198 index entries of HZ8eH4RRA0__.20101220224626137.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 45 index entries of HZ8eH4RRA0__.20101221040038969.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 25 index entries of HZ8eH4RRA0__.20101221040653126.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 22 index entries of HZ8eH4RRA0__.20101221041351973.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 30 index entries of HZ8eH4RRA0__.20101221042010347.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 107 index entries of HZ8eH4RRA0__.20101221042644177.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 21 index entries of HZ8eH4RRA0__.20101221043200181.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 382 index entries of HZ8eH4RRA0__.20101221043514607.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 1883 index entries of HZ8eH4RRA0__.20101221063140743.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 224 index entries of HZ8eH4RRA0__.20101221084529571.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 148 index entries of HZ8eH4RRA0__.20101221090158374.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 255 index entries of HZ8eH4RRA0__.20101221091614103.blob in 11 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 192 index entries of HZ8eH4RRA0__.20101221093640113.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 159 index entries of HZ8eH4RRA0__.20101221095745234.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 598 index entries of HZ8eH4RRA0__.20101221101643892.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 618 index entries of HZ8eH4RRA0__.20101221140842130.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 265 index entries of HZ8eH4RRA0__.20101221143908509.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 93 index entries of HZ8eH4RRA0__.20101221151523150.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 132 index entries of HZ8eH4RRA0__.20101221151930418.blob in 1 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 70 index entries of HZ8eH4RRA0__.20101221152333018.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 309 index entries of HZ8eH4RRA0__.20101221152655827.blob in 2 milliseconds.
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 778 index entries of HZ8eH4RRA0__.20101221153646815.blob in 2 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/HTCACHE/file.array/HZ8eH4RRA0__.20101221212724677.blob
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 948 index entries of HZ8eH4RRA0__.20101221212724677.blob in 2 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/HTCACHE/file.array/HZ8eH4RRA0__.20101222004155342.blob
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 568 index entries of HZ8eH4RRA0__.20101222004155342.blob in 1 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/WORK/pkcounter.bheap
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/WORK/robots.bheap
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 342 index entries of robots.bheap in 18 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/WORK/rss.bheap
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 408418 index entries of rss.bheap in 132 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/WORK/api.bheap
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 296 index entries of api.bheap in 2 milliseconds.
I 2010/12/22 01:44:00 Heap closing heap /home/yacy/yacy/DATA/WORK/searchfl.bheap
I 2010/12/22 01:44:00 kelondroBLOBHeap wrote a dump for the 4320 index entries of searchfl.bheap in 13 milliseconds.
S 2010/12/22 01:44:00 SHUTDOWN goodbye. (this is the last line)


Ich habe jetzt nochmal neu gestartet und der Peer hat gleich gecrawlt. Als die maximale Größe des Caches erreicht war, wurde einmal geflusht, aber danach nicht wieder.
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Quix0r » Mi Dez 22, 2010 10:21 am

Mit der NPE in mountBLOB() ist das Attribut blobs null, und bei der NPE in der Segments-Klasse (Zeile 132) ist das Attribut segments null.

Vielleicht so als Denkansatz. :) Vorher das kommt, koennte nur sein, dass beim Shutdown viele Instanzen auf null gesetzt werden. Die einzige gute Loesung ist hier ein Visitor-Pattern, der alle zu beendenen Objekte "besucht" und die sich dann sauber beenden.

Mein Peer verhaelt sich auch kaum anders. Nur dass zwar geflusht wird (unterschiedliche Dateigroessen), aber nicht germergt wird.
Quix0r
 
Beiträge: 1345
Registriert: Di Jul 31, 2007 9:22 am
Wohnort: Krefeld

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Orbiter » Do Dez 30, 2010 1:09 am

vielleicht hilft svn 7414
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Sa Jan 01, 2011 4:35 pm

Hat bei mir leider nichts gebracht. Es wird weiter nur der erste Flush ausgeführt, danach passiert nichts mehr.
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Lotus » So Jan 02, 2011 10:00 pm

Ich habe eine NPE im Debugger bemerkt, ansonsten nicht wahrnehmbar:

Code: Alles auswählen
net.yacy.kelondro.io.Records

Daemon System Thread [Finalizer] (Suspended (exception NullPointerException))   
   Records.flushBuffer() line: 182   
   Records.close() line: 191   
   BufferedRecords.close() line: 87   
   BufferedRecords.finalize() line: 91   
   Finalizer.invokeFinalizeMethod(Object) line: not available [native method]   
   Finalizer.runFinalizer() line: 101   
   Finalizer.access$100(Finalizer) line: 32   
   Finalizer$FinalizerThread.run() line: 178   

raf.seek(raf.length());
raf ist hier null.

Ich habe keine Ahnung ob das hiermit zu tun haben könnte.
Lotus
 
Beiträge: 1699
Registriert: Mi Jun 27, 2007 3:33 pm
Wohnort: Hamburg

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Orbiter » Di Jan 04, 2011 12:06 am

guter Hinweis! nächster Fixversuch in SVN 7422
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Di Jan 04, 2011 9:54 am

Hat leider bei meinem Peer nichts gebracht. Ich habe im Moment auch leider keine Zeit, da genauer reinzuschauen. :-(
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Di Jan 04, 2011 3:42 pm

Mit r7424 hat mein Peer jetzt zumindest versucht, Blobs zu mergen. Das ging mit einem Hinweis auf zu wenig HeapSpace schief, so dass ich nun den Speicher für YaCy hochgesetzt habe und einen Teil der Blobs in ein temporäres Verzeichnis verschoben habe. Nach einem Neustart mergt mein Peer jetzt mit recht gutem Erfolg vor sich hin. Ich muss nun Stück für Stück die vorher verschobenen Blobs zurück schieben und werde dann überprüfen, ob wieder korrekt geflusht wird.
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: Kein regelmäßiger Flush des RAM chaches (r7388)

Beitragvon Low012 » Mi Jan 05, 2011 12:08 am

So, meine Blobs sind von über 300 auf ein erträgliches Maß zusammengeschrumpft und es wird wieder geflusht. Ich bin nicht ganz sicher, ob das Mergen immer korrekt klappt ("out of heap space" sogar bei maximaler Ausnutzung des RAMs), aber erstmal kann es weiter gehen. :)
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast