WARNING - EcoTable....collection.index has 15517 doubles

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Fr Feb 13, 2009 11:56 am

Hallo,

ich bekomme mittlerweile bei jedem Start folgende Fehlermeldung....

Code: Alles auswählen
I 2009/02/13 11:47:47 ECOTABLE /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 15517 doubles


Da ich nicht beurteilen kann, ob YaCy noch was macht, breche ich den Startvorgang ab und baue den Index neu auf.
Der Neuaufbau des Collection-Index hält max. einen Start, danach tritt das Problem wieder auf.
Der Logeintrag zeigt ein I (Information) im Text der Meldung steht "WARNING" und es gibt keinen Hinweis, ob noch was passiert, oder nicht...

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » Fr Feb 13, 2009 12:13 pm

Da bin gleich mal dabei, auch ich baue den Index immer wieder neu auf weil es schneller geht. Habe jetzt kein Thread Dump zur Hand, weiß aber noch das da einige Threads innerhalb von java den Status (parking) hatten.

Das dürfte mit der neuen Version von removeDoubles zusammen hängen.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » Fr Feb 13, 2009 2:01 pm

Code: Alles auswählen
Full thread dump Java HotSpot(TM) Client VM (11.2-b01 mixed mode):

"sorting_pool-1-thread-6" prio=6 tid=0x6306f400 nid=0x554 waiting on condition [0x633ff000..0x633ffc94]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0300> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"sorting_pool-1-thread-5" prio=6 tid=0x6305d400 nid=0x4e0 waiting on condition [0x633af000..0x633afd94]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0300> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"sorting_pool-1-thread-4" prio=6 tid=0x62c19400 nid=0x5d0 waiting on condition [0x6335f000..0x6335fd14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0300> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"sorting_pool-1-thread-3" prio=6 tid=0x62f2c400 nid=0x70c waiting on condition [0x6330f000..0x6330fa14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0300> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"partition_pool-1-thread-2" prio=6 tid=0x62c42c00 nid=0x5e8 waiting on condition [0x632bf000..0x632bfa94]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0350> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"partition_pool-1-thread-1" prio=6 tid=0x62f70800 nid=0x564 waiting on condition [0x6326f000..0x6326fb14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x146f0350> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"Thread-1" prio=6 tid=0x62f72400 nid=0x714 waiting on condition [0x631af000..0x631afb94]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at de.anomic.server.serverProfiling.run(serverProfiling.java:63)

"Low Memory Detector" daemon prio=6 tid=0x62c05000 nid=0x20c runnable [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x62c02400 nid=0x5f0 waiting on condition [0x00000000..0x62e6f9bc]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x62bfd400 nid=0x5e0 runnable [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x62bfc000 nid=0x978 waiting on condition [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x62bec800 nid=0x918 in Object.wait() [0x62d5f000..0x62d5fa94]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09860a70> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        - locked <0x09860a70> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

"Reference Handler" daemon prio=10 tid=0x62be7c00 nid=0x930 in Object.wait() [0x62d0f000..0x62d0fb14]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09860978> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
        - locked <0x09860978> (a java.lang.ref.Reference$Lock)

"main" prio=6 tid=0x002b7000 nid=0x914 runnable [0x0090f000..0x0090fe54]
   java.lang.Thread.State: RUNNABLE
        at de.anomic.kelondro.index.RowSet.remove(RowSet.java:153)
        - locked <0x155c7e48> (a de.anomic.kelondro.index.RowSet)
        at de.anomic.kelondro.index.RowSet.remove(RowSet.java:165)
        at de.anomic.kelondro.index.RAMIndex.put(RAMIndex.java:93)
        - locked <0x1558a770> (a de.anomic.kelondro.index.RAMIndex)
        at de.anomic.kelondro.index.BytesIntMap.puti(BytesIntMap.java:76)
        - locked <0x1553c468> (a de.anomic.kelondro.index.BytesIntMap)
        at de.anomic.kelondro.table.EcoTable.removeInFile(EcoTable.java:439)
        at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:199)
        at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
        at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
        at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
        at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
        at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:319)
        at yacy.startup(yacy.java:234)
        at yacy.main(yacy.java:1422)

"VM Thread" prio=10 tid=0x62be2c00 nid=0x708 runnable

"VM Periodic Task Thread" prio=10 tid=0x62c0f800 nid=0x5dc waiting on condition

JNI global references: 1032

Heap
def new generation   total 102080K, used 86889K [0x029a0000, 0x09860000, 0x09860000)
  eden space 90752K,  85% used [0x029a0000, 0x075c5bb0, 0x08240000)
  from space 11328K,  78% used [0x08d50000, 0x096049d8, 0x09860000)
  to   space 11328K,   0% used [0x08240000, 0x08240000, 0x08d50000)
tenured generation   total 1361152K, used 204720K [0x09860000, 0x5c9a0000, 0x5c9a0000)
   the space 1361152K,  15% used [0x09860000, 0x1604c1e0, 0x1604c200, 0x5c9a0000)
compacting perm gen  total 12288K, used 4185K [0x5c9a0000, 0x5d5a0000, 0x609a0000)
   the space 12288K,  34% used [0x5c9a0000, 0x5cdb67b0, 0x5cdb6800, 0x5d5a0000)
No shared spaces configured.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Lilirasa » Fr Feb 13, 2009 7:25 pm

Hatte ich auch schon. Einfach laufen lassen, dauert mehrere Stunden bis YACY wieder startet. Beim letzten mal, waren es bei mir ~900'000 Doubles, hat fast 24 Stunden gedauert...
Collection.index löschen funktioniert nicht.

Grüsse
Andi
Lilirasa
 
Beiträge: 71
Registriert: So Sep 14, 2008 11:28 am

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Sa Feb 14, 2009 11:35 am

Hallo,

einfach warten funktioniert bei mir nur bedingt...

Code: Alles auswählen
W 2009/02/14 11:06:26 kelondroRowCollection null
java.lang.ArrayIndexOutOfBoundsException
E 2009/02/14 11:06:32 STARTUP FATAL ERROR: kelondroBufferedEcoFS.put(44067229) outside bounds (44050384)
java.lang.IndexOutOfBoundsException: kelondroBufferedEcoFS.put(44067229) outside bounds (44050384)
        at de.anomic.kelondro.io.BufferedEcoFS.put(BufferedEcoFS.java:99)
        at de.anomic.kelondro.table.EcoTable.removeInFile(EcoTable.java:436)
        at de.anomic.kelondro.table.EcoTable.removeDoubles(EcoTable.java:320)
        at de.anomic.kelondro.kelondroCollectionIndex.openAllArrayFiles(kelondroCollectionIndex.java:259)
        at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:150)
        at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
        at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
        at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:311)
        at yacy.startup(yacy.java:224)
        at yacy.main(yacy.java:1037)
S 2009/02/14 11:06:32 SHUTDOWN goodbye. (this is the last line)


YaCy machtzwar nach oben genannter Warnmeldung weiter, geht dann aber nach einigen Stunden direkt in den SHUTDOWN!

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » So Feb 15, 2009 2:28 pm

Also mein collection.index hat 61964 doubles und läuft schon über eine Stunde ohne das man weiß ob und was YaCy macht, das kann so nicht bleiben !
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Orbiter » So Feb 15, 2009 3:51 pm

ich hab diese Probleme nicht, daher weiss ich hier nicht was ich debuggen kann. In so einem Fall immer thread dump machen, wisst ihr doch.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » So Feb 15, 2009 5:02 pm

Habe nach über 2 Stunden abgebrochen, nach ein Neustart sollten es dann 59580 doppelte sein. Thread dump sieht immer noch so aus wie weiter oben.
Code: Alles auswählen
2009-02-15 16:56:24
Full thread dump Java HotSpot(TM) Client VM (11.2-b01 mixed mode):

"sorting_pool-1-thread-33" prio=6 tid=0x62c19400 nid=0xa5c waiting on condition [0x6335f000..0x6335fb14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x1435f338> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"sorting_pool-1-thread-31" prio=6 tid=0x62fea800 nid=0x170 waiting on condition [0x6330f000..0x6330fb94]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x1435f338> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"sorting_pool-1-thread-30" prio=6 tid=0x62bd2400 nid=0xd0c waiting on condition [0x633af000..0x633afd14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x1435f338> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"partition_pool-1-thread-2" prio=6 tid=0x62c4a400 nid=0x9a0 waiting on condition [0x632bf000..0x632bfa94]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x1435f420> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"partition_pool-1-thread-1" prio=6 tid=0x62bd7400 nid=0xfa0 waiting on condition [0x6326f000..0x6326fb14]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x1435f420> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(Unknown Source)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(Unknown Source)
        at java.util.concurrent.SynchronousQueue.poll(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

"Thread-1" prio=6 tid=0x62f7a400 nid=0x2d4 waiting on condition [0x631af000..0x631afb94]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at de.anomic.server.serverProfiling.run(serverProfiling.java:63)

"Low Memory Detector" daemon prio=6 tid=0x62c05000 nid=0xa2c runnable [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x62c00800 nid=0xd18 waiting on condition [0x00000000..0x62e6f9bc]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x62bfd400 nid=0xe40 runnable [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x62bfc000 nid=0xd90 waiting on condition [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x62bec800 nid=0x8c4 in Object.wait() [0x62d5f000..0x62d5fa94]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09860a70> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        - locked <0x09860a70> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

"Reference Handler" daemon prio=10 tid=0x62be7c00 nid=0xce4 in Object.wait() [0x62d0f000..0x62d0fb14]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09860978> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
        - locked <0x09860978> (a java.lang.ref.Reference$Lock)

"main" prio=6 tid=0x002b7000 nid=0xf60 runnable [0x0090f000..0x0090fe54]
   java.lang.Thread.State: RUNNABLE
        at de.anomic.kelondro.index.RowSet.remove(RowSet.java:153)
        - locked <0x14364a10> (a de.anomic.kelondro.index.RowSet)
        at de.anomic.kelondro.index.RowSet.remove(RowSet.java:165)
        at de.anomic.kelondro.index.RAMIndex.put(RAMIndex.java:93)
        - locked <0x1435f5d8> (a de.anomic.kelondro.index.RAMIndex)
        at de.anomic.kelondro.index.BytesIntMap.puti(BytesIntMap.java:76)
        - locked <0x1435f5f0> (a de.anomic.kelondro.index.BytesIntMap)
        at de.anomic.kelondro.table.EcoTable.removeInFile(EcoTable.java:439)
        at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:199)
        at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
        at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
        at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
        at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
        at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:321)
        at yacy.startup(yacy.java:234)
        at yacy.main(yacy.java:1422)

"VM Thread" prio=10 tid=0x62be2c00 nid=0xf70 runnable

"VM Periodic Task Thread" prio=10 tid=0x62c0f400 nid=0x860 waiting on condition

JNI global references: 992

Heap
def new generation   total 102080K, used 33642K [0x029a0000, 0x09860000, 0x09860000)
  eden space 90752K,  28% used [0x029a0000, 0x04283840, 0x08240000)
  from space 11328K,  72% used [0x08240000, 0x08a371b0, 0x08d50000)
  to   space 11328K,   0% used [0x08d50000, 0x08d50000, 0x09860000)
tenured generation   total 1361152K, used 175624K [0x09860000, 0x5c9a0000, 0x5c9a0000)
   the space 1361152K,  12% used [0x09860000, 0x143e23d0, 0x143e2400, 0x5c9a0000)
compacting perm gen  total 12288K, used 4178K [0x5c9a0000, 0x5d5a0000, 0x609a0000)
   the space 12288K,  34% used [0x5c9a0000, 0x5cdb4810, 0x5cdb4a00, 0x5d5a0000)
No shared spaces configured.
Jede Minute eine Zeile wie weit YaCy ist würde schon helfen.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » So Feb 15, 2009 9:23 pm

Orbiter hat geschrieben:In so einem Fall immer thread dump machen, wisst ihr doch.


Sorry, war im Urlaub.....hier jetzt Fehlermeldung und Thread Dump:

LOG:
Code: Alles auswählen
STARTUP: Trying to load logging configuration from file /media/yacy/yacy/DATA/LOG/yacy.logging
S 2009/02/15 21:29:14 STARTUP Java version: 1.6.0_0
S 2009/02/15 21:29:14 STARTUP Operation system: Linux
S 2009/02/15 21:29:14 STARTUP Application root-path: /media/yacy/yacy
S 2009/02/15 21:29:14 STARTUP Time zone: UTC+0100; UTC+0000 is 1234729754463
S 2009/02/15 21:29:14 STARTUP Maximum file system path length: 65535
E 2009/02/15 21:29:14 STARTUP WARNING: the file /media/yacy/yacy/DATA/yacy.running exists, this usually means that a YaCy instance is still running
I 2009/02/15 21:29:14 PLASMA This is the pro-version of YaCy
I 2009/02/15 21:29:14 indexContainerRAMHeap restoring rwi blob dump 'index.dhtout.blob'
I 2009/02/15 21:29:21 indexContainerRAMHeap finished rwi blob restore: 304183 words, 6068012 word/URL relations in 6549 milliseconds
D 2009/02/15 21:29:22 STARTUP OPENING COLLECTION INDEX
I 2009/02/15 21:29:22 ECOTABLE initialization of /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: available RAM: 3100MB, allocating space for 44158963 entries
I 2009/02/15 21:29:23 ECOTABLE /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: EcoTable /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has table copy DISABLED
I 2009/02/15 21:29:23 ECOTABLE initializing RAM index for EcoTable collection.index, please wait.
java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: -49
   at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
   at java.util.concurrent.FutureTask.get(FutureTask.java:111)
   at de.anomic.kelondro.index.RowCollection.sort(RowCollection.java:518)
   at de.anomic.kelondro.index.RowCollection.removeDoubles(RowCollection.java:799)
   at de.anomic.kelondro.index.RAMIndex.removeDoubles(RAMIndex.java:132)
   at de.anomic.kelondro.index.BytesIntMap.removeDoubles(BytesIntMap.java:95)
   at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:174)
   at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
   at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
   at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
   at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
   at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:318)
   at yacy.startup(yacy.java:224)
   at yacy.main(yacy.java:1037)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -49
   at de.anomic.kelondro.order.Base64Order.compares(Base64Order.java:327)
   at de.anomic.kelondro.order.Base64Order.compare0(Base64Order.java:297)
   at de.anomic.kelondro.order.Base64Order.compare(Base64Order.java:293)
   at de.anomic.kelondro.index.RowCollection.compare(RowCollection.java:867)
   at de.anomic.kelondro.index.RowCollection.min(RowCollection.java:735)
   at de.anomic.kelondro.index.RowCollection.picMiddle(RowCollection.java:719)
   at de.anomic.kelondro.index.RowCollection.pivot(RowCollection.java:695)
   at de.anomic.kelondro.index.RowCollection.partition(RowCollection.java:629)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:593)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection$qsortthread.call(RowCollection.java:582)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)
W 2009/02/15 21:29:39 kelondroRowCollection -90
java.lang.ArrayIndexOutOfBoundsException: -90
   at de.anomic.kelondro.order.Base64Order.compares(Base64Order.java:329)
   at de.anomic.kelondro.order.Base64Order.compare0(Base64Order.java:297)
   at de.anomic.kelondro.order.Base64Order.compare(Base64Order.java:293)
   at de.anomic.kelondro.index.RowCollection.compare(RowCollection.java:867)
   at de.anomic.kelondro.index.RowCollection.removeDoubles(RowCollection.java:808)
   at de.anomic.kelondro.index.RAMIndex.removeDoubles(RAMIndex.java:132)
   at de.anomic.kelondro.index.BytesIntMap.removeDoubles(BytesIntMap.java:95)
   at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:174)
   at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
   at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
   at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
   at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
   at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:318)
   at yacy.startup(yacy.java:224)
   at yacy.main(yacy.java:1037)
java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: -89
   at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
   at java.util.concurrent.FutureTask.get(FutureTask.java:111)
   at de.anomic.kelondro.index.RowCollection.sort(RowCollection.java:518)
   at de.anomic.kelondro.index.RowCollection.removeDoubles(RowCollection.java:828)
   at de.anomic.kelondro.index.RAMIndex.removeDoubles(RAMIndex.java:132)
   at de.anomic.kelondro.index.BytesIntMap.removeDoubles(BytesIntMap.java:95)
   at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:174)
   at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
   at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
   at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
   at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
   at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:318)
   at yacy.startup(yacy.java:224)
   at yacy.main(yacy.java:1037)
Caused by: java.lang.ArrayIndexOutOfBoundsException: -89
   at de.anomic.kelondro.order.Base64Order.compares(Base64Order.java:329)
   at de.anomic.kelondro.order.Base64Order.compare0(Base64Order.java:297)
   at de.anomic.kelondro.order.Base64Order.compare(Base64Order.java:293)
   at de.anomic.kelondro.index.RowCollection.compare(RowCollection.java:867)
   at de.anomic.kelondro.index.RowCollection.min(RowCollection.java:735)
   at de.anomic.kelondro.index.RowCollection.picMiddle(RowCollection.java:719)
   at de.anomic.kelondro.index.RowCollection.pivot(RowCollection.java:695)
   at de.anomic.kelondro.index.RowCollection.partition(RowCollection.java:629)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:593)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection.qsort(RowCollection.java:596)
   at de.anomic.kelondro.index.RowCollection$qsortthread.call(RowCollection.java:582)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)
I 2009/02/15 21:29:41 ECOTABLE /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index: WARNING - EcoTable /media/yacy/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 4504 doubles



Thread Dump:
Code: Alles auswählen
Full thread dump OpenJDK 64-Bit Server VM (1.6.0_0-b12 mixed mode):

"sorting_pool-1-thread-6" prio=10 tid=0x00007f16ec006800 nid=0x1309 waiting on condition [0x0000000041cf5000..0x0000000041cf5d00]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16fef0b330> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"sorting_pool-1-thread-5" prio=10 tid=0x00007f16ec006400 nid=0x1308 waiting on condition [0x0000000041bf4000..0x0000000041bf4c80]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16fef0b330> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"sorting_pool-1-thread-4" prio=10 tid=0x00007f16ec296000 nid=0x1307 waiting on condition [0x0000000040b52000..0x0000000040b52c00]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16fef0b330> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"sorting_pool-1-thread-3" prio=10 tid=0x00007f16ec294c00 nid=0x1306 waiting on condition [0x0000000040a51000..0x0000000040a51b80]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16fef0b330> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"partition_pool-1-thread-2" prio=10 tid=0x00007f16ec294000 nid=0x1305 waiting on condition [0x0000000041af3000..0x0000000041af3b00]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16feef4158> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"partition_pool-1-thread-1" prio=10 tid=0x00007f16ec293800 nid=0x1304 waiting on condition [0x00000000419f2000..0x00000000419f2a80]
   java.lang.Thread.State: TIMED_WAITING (parking)
   at sun.misc.Unsafe.park(Native Method)
   - parking to wait for  <0x00007f16feef4158> (a java.util.concurrent.SynchronousQueue$TransferStack)
   at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
   at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:453)
   at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:352)
   at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
   at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:636)

"Thread-1" prio=10 tid=0x00007f16ec0f6000 nid=0x1301 waiting on condition [0x00000000418f1000..0x00000000418f1a00]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
   at java.lang.Thread.sleep(Native Method)
   at de.anomic.server.serverProfiling.run(serverProfiling.java:63)

"Low Memory Detector" daemon prio=10 tid=0x00007f16ec094400 nid=0x12ff runnable [0x0000000000000000..0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x00007f16ec091800 nid=0x12fe waiting on condition [0x0000000000000000..0x00000000413bc5c0]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x00007f16ec090000 nid=0x12fd waiting on condition [0x0000000000000000..0x00000000412bb540]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00007f16ec08e800 nid=0x12fc runnable [0x0000000000000000..0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00007f16ec065400 nid=0x12fb in Object.wait() [0x00000000417f0000..0x00000000417f0b00]
   java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on <0x00007f16f7c9e238> (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
   - locked <0x00007f16f7c9e238> (a java.lang.ref.ReferenceQueue$Lock)
   at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
   at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)

"Reference Handler" daemon prio=10 tid=0x00007f16ec063c00 nid=0x12fa in Object.wait() [0x00000000416ef000..0x00000000416efa80]
   java.lang.Thread.State: WAITING (on object monitor)
   at java.lang.Object.wait(Native Method)
   - waiting on <0x00007f16f7c9e270> (a java.lang.ref.Reference$Lock)
   at java.lang.Object.wait(Object.java:502)
   at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
   - locked <0x00007f16f7c9e270> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x0000000000e14800 nid=0x12f4 runnable [0x00000000410b9000..0x00000000410bae70]
   java.lang.Thread.State: RUNNABLE
   at de.anomic.kelondro.index.RowSet.remove(RowSet.java:153)
   - locked <0x00007f1738022fe0> (a de.anomic.kelondro.index.RowSet)
   at de.anomic.kelondro.index.RowSet.remove(RowSet.java:165)
   at de.anomic.kelondro.index.RAMIndex.put(RAMIndex.java:93)
   - locked <0x00007f1738022fb0> (a de.anomic.kelondro.index.RAMIndex)
   at de.anomic.kelondro.index.BytesIntMap.puti(BytesIntMap.java:76)
   - locked <0x00007f1738022e80> (a de.anomic.kelondro.index.BytesIntMap)
   at de.anomic.kelondro.table.EcoTable.removeInFile(EcoTable.java:439)
   at de.anomic.kelondro.table.EcoTable.<init>(EcoTable.java:199)
   at de.anomic.kelondro.kelondroCollectionIndex.openIndexFile(kelondroCollectionIndex.java:293)
   at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:149)
   at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
   at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
   at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:318)
   at yacy.startup(yacy.java:224)
   at yacy.main(yacy.java:1037)

"VM Thread" prio=10 tid=0x00007f16ec05ec00 nid=0x12f9 runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000000e1f000 nid=0x12f5 runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000000e20400 nid=0x12f6 runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000000e21800 nid=0x12f7 runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000000e22c00 nid=0x12f8 runnable

"VM Periodic Task Thread" prio=10 tid=0x00007f16ec09e800 nid=0x1300 waiting on condition

JNI global references: 892

Heap
PSYoungGen      total 1035648K, used 641860K [0x00007f179e740000, 0x00007f17f1c90000, 0x00007f17f1c90000)
  eden space 697408K, 92% used [0x00007f179e740000,0x00007f17c5a09118,0x00007f17c9050000)
  from space 338240K, 0% used [0x00007f17dd240000,0x00007f17dd248000,0x00007f17f1c90000)
  to   space 329664K, 0% used [0x00007f17c9050000,0x00007f17c9050000,0x00007f17dd240000)
PSOldGen        total 2730688K, used 1097505K [0x00007f16f7c90000, 0x00007f179e740000, 0x00007f179e740000)
  object space 2730688K, 40% used [0x00007f16f7c90000,0x00007f173ac58758,0x00007f179e740000)
PSPermGen       total 21248K, used 6947K [0x00007f16f2890000, 0x00007f16f3d50000, 0x00007f16f7c90000)
  object space 21248K, 32% used [0x00007f16f2890000,0x00007f16f2f58dd8,0x00007f16f3d50000)


Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Orbiter » Mo Feb 16, 2009 10:34 am

arbeite jetzt an einer nachhaltigen Lösung, d.h.: doubles in collections löschen. Dann muss ich noch gucken woher die eigentlich kommen. Die haben sich bei euch ja offenbar im Laufe der Zeit ganz schön angesammelt. Ich habe auch welche, aber nur ein paar hundert, das ist mir bislang nicht aufgefallen.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » Mo Feb 16, 2009 2:02 pm

Es war/ist kein BUG, es hat nur _viel_ zu lange gedauert und von daher ist es gut wenn die doppelten Einträge demnächst richtig gelöscht werden.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Di Feb 17, 2009 9:43 pm

Tja, meinem Peer hat svn 5618 irgendwie das Genick gebrochen...nach dem Neuaufbau der collection.index erhalte ich folgendes:

Code: Alles auswählen
W 2009/02/17 20:50:47 kelondroRowCollection null
java.lang.ArrayIndexOutOfBoundsException
W 2009/02/17 20:51:49 kelondroRowCollection null
java.lang.ArrayIndexOutOfBoundsException
E 2009/02/17 20:51:49 STARTUP FATAL ERROR: -24
java.lang.ArrayIndexOutOfBoundsException: -24
   at de.anomic.kelondro.order.Base64Order.compares(Base64Order.java:329)
   at de.anomic.kelondro.order.Base64Order.compare0(Base64Order.java:297)
   at de.anomic.kelondro.order.Base64Order.compare(Base64Order.java:293)
   at de.anomic.kelondro.index.RowCollection.compare(RowCollection.java:873)
   at de.anomic.kelondro.index.RowCollection.min(RowCollection.java:741)
   at de.anomic.kelondro.index.RowCollection.picMiddle(RowCollection.java:725)
   at de.anomic.kelondro.index.RowCollection.pivot(RowCollection.java:709)
   at de.anomic.kelondro.index.RowCollection.partition(RowCollection.java:635)
   at de.anomic.kelondro.index.RowCollection.sort(RowCollection.java:509)
   at de.anomic.kelondro.index.RowCollection.uniq(RowCollection.java:796)
   at de.anomic.kelondro.index.RAMIndex.finishInitialization(RAMIndex.java:66)
   at de.anomic.kelondro.index.RAMIndex.put(RAMIndex.java:90)
   at de.anomic.kelondro.index.BytesIntMap.puti(BytesIntMap.java:76)
   at de.anomic.kelondro.table.EcoTable.removeInFile(EcoTable.java:443)
   at de.anomic.kelondro.table.EcoTable.removeDoubles(EcoTable.java:320)
   at de.anomic.kelondro.kelondroCollectionIndex.openAllArrayFiles(kelondroCollectionIndex.java:265)
   at de.anomic.kelondro.kelondroCollectionIndex.<init>(kelondroCollectionIndex.java:181)
   at de.anomic.index.indexCollectionRI.<init>(indexCollectionRI.java:49)
   at de.anomic.plasma.plasmaWordIndex.<init>(plasmaWordIndex.java:164)
   at de.anomic.plasma.plasmaSwitchboard.<init>(plasmaSwitchboard.java:313)
   at yacy.startup(yacy.java:224)
   at yacy.main(yacy.java:1037)
S 2009/02/17 20:51:49 SHUTDOWN goodbye. (this is the last line)


YaCy startet nicht mehr!

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon thq » Di Feb 17, 2009 11:07 pm

Wenn ich die java.lang.ArrayIndexOutOfBoundsException: -24 sehe tippe ich auch eher darauf das ein teil deiner DB kaputt ist. Vielleicht kann man den kaputten Datensatz überspringen/löschen, aber das kann bestimmt nur Orbiter beantworten.
thq
 
Beiträge: 651
Registriert: So Jul 08, 2007 12:23 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Orbiter » Di Feb 17, 2009 11:21 pm

naja thq hat recht, da ist was kaputt. Ich frage mich natürlich warum und wann, aktueller Fehler oder alter. Jedenfalls hat es hier keine Verbindung mit den letzten Änderungen, und ich bin auch ein wenig ratlos was man machen kann. Das ArrayIndexOutOfBoundsException bei einem Base64Order.compares tritt halt auf, wenn man Zeichen im key hat, die nicht den Zeichen des b64-Alphabetes entspricht. Ein entsprechendes assert fängt das bei mir ab. Ich weiss nicht ob es temporär hilft, hier yacy mit einem -ea (enable asserts) zu starten, dann wird an der entsprechenden Stelle hat eine andere Exception geworfen. Eine 'lazyness' an dieser Stelle einzubauen, die auf solche Fehler testet, ist enorm kostspielig, weil sie im Normalfall CPU frisst und nie greift. hm.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Di Feb 17, 2009 11:40 pm

Mit jedem Start bekomme ich ein anderes Bild....jetzt beschwert YaCy sich wieder über "collection.index has 21643 doubles" ....
Bisher hatte ich auch schon ArrayIndexOutOfBoundsExceptions, allerdings ist YaCy da beim Start einfach drüber weggegangen.
Wo der Fehler her kommt weiss ich nicht, ich hatte keinen Absurz oder ähnliches und die Kiste läuft seit Monaten stabil.
Ich versuche noch ein Weilchen, ansonsten heisst es nach sehr langer Zeit und rd. 60 Mio. URLs mal wieder "Gehe zurück auf Los und ziehe nicht DM 400 ein..."

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Orbiter » Di Feb 17, 2009 11:53 pm

oh nein, ich weiss du hast die zweitdickste DB im Netz. Der Fehler ist ja auch nicht bei den URLs, sondern in den RWIs. Wenn du dir das antun möchtest kannst du auch über alle kca-Dateien gehen, jeweils eins heraus nehmen, und damit die collection.index aufbauen, in der Hoffnung das du dann den Fehler nicht mehr bekommst. Das kostet aber ziemlich viel Arbeit. Sorry. Wenn du dann dabei die kca indentifizierst, die den Fehler drin hat, dann verlierst du keine URLs, sondern nur 'ein paar' Referenzen darauf.. :-(
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Mi Feb 18, 2009 11:11 pm

Mit einem Downgrade auf 0.720/05608 und Neuaufbau der collection.index läuft mein Peer wieder.
Einzigster Nachteil, ich muss bei jedem Start den collection.index neu aufbauen...sonst hängt YaCy wegen der Doubles.
Also irgendwas muss sich schon verändert haben zwischen den Versionen.

Positiv für YaCy freeworld, ich hab mal einen zweiten Peer gestartet, mittlerweile bin ich auf YaCy angewiesen, habe schließlich meine ganzen Bookmarks da... :-)

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon soleil » So Feb 22, 2009 10:05 pm

Zur Frage, woher die "doubles" kommen: Bei mir sind vermutlich einmal welche aufgetreten, nachdem ich die Platte, auf der Yacy lief, nicht korrekt ungemountet, sondern im laufenden Betrieb abgezogen hatte.

Genauer gesagt, waren von einem Tag auf den anderen etwa die Hälfte meiner RWIs verschwunden. Nach Lektüre des Forums habe ich den collection.index gelöscht. In der Tat waren beim Neustart alle RWIs wieder da; beim folgen Start wurden dann mehrere Tausend Doubletten gemeldet. Yacy lief dann in einer Schleife und verkürzte den Index jeweils um 30 Byte. Der ganze Prozeß hätte Tage gedauert, und so habe ich im Endeffekt meine ganze Datenbank weggeschmissen und von null angefangen. Seitdem mache ich regelmäßige Backups.

Mir schien das Problem in einer der letzten Versionen gelöst. Letztens hatte ich wieder Doubletten, die Indexdateien wurden aber in sehr viel größeren Schritten verkürzt, und Yacy lief nach absehbarer Zeit wieder.

Wäre es eventuell möglich, einen Datenbanktest und gegebenenfalls eine Reparatur im Schnelldurchlauf zu implementieren? Im obigen Szenario wurde ich stutzig, weil die Anzahl der RWIs sich auf einmal halbiert hatte. Aber wer weiß, in welchem Zustand sich meine Datenbank gerade befindet? Ich bin nur sicher, daß ich zwischenzeitlich wieder meine Festplatte versehentlich abgestöpselt habe...

Thomas
soleil
 
Beiträge: 22
Registriert: Di Jan 13, 2009 11:36 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon apfelmaennchen » Mi Feb 25, 2009 11:02 pm

Tja, es war die collection.04.0028.00.00.kca mit ca. 37 Mio. RWIs (='ein paar').....Murphy lässt grüßen.
Die collection.04.0028.07.00.kca ist mit 61 GB fast zehn mal so groß, wäre hier die Wahrscheinlichkeit für einen Fehler nicht viel größer gewesen?
Ich werf dann mit den verbleibenden 7 Mio. RWIs mal den Crawler wieder an, ..... die Rohdaten (world wiede web) sind ja noch da :-)

Gruß!
apfelmaennchen
apfelmaennchen
 
Beiträge: 429
Registriert: Mo Aug 20, 2007 7:06 pm

Re: WARNING - EcoTable....collection.index has 15517 doubles

Beitragvon Copro » Do Feb 26, 2009 1:48 am

Hehe, 7 Mio. ist ja immernoch ein kleiner Haufen ;-)
Bei mir hat er nach einer Stunde aber ebenfalls weitergemacht nach einer Meldung:
WARNING - EcoTable /home/mb/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.index has 124919 doubles

Passiert bei mir nach einem lange fälligen Update eines 0.59 Releases auf die neueste Version 0.7.
Copro
 
Beiträge: 207
Registriert: Do Dez 13, 2007 3:37 am


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste