Langsamer Tod...

Keine Scheu, hier darf alles gefragt und diskutiert werden. Das ist das Forum für YaCy-Anfänger. Hier kann man 'wo muss man klicken' fragen und sich über Grundlagen zur Suchmaschinentechnik unterhalten.
Forumsregeln
Hier werden Fragen beantwortet und wir versuchen die Probleme von YaCy-Newbies zu klären. Bitte beantwortete Fragen im YaCy-Wiki http://wiki.yacy.de dokumentieren!

Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 2:36 pm

Hallo zusammen.
Nach längerer Abstinenz habe ich yacy auf einem älteren Rechner installiert um es mal wieder auszuprobieren (Peer vdbweb).
Der Rechner ist ein P4, 1,7GHz, 1,5GB Ram. Er läuft unter SuSE Linux 11.4, kein X-Server.

Sollte dies für einen kleinen Yacy-Peer reichen?
Yacy lässt sich problemos installieren und starten.
Nach einigen Stunden bis Tagen ist die Weboberfläche von Yacy dann nicht mehr erreichbar.
Das das gestartete STOP-Script läuft auch nach mehreren Tagen nicht durch.
Das Log produziert aber weiter fleissig viele Einträge.

Ich würde Yacy gern durch betreiben eines Peers unterstützen habe aber leider nur begrenzt Zeit mich darum zu kümmern.
Gibt es eine Diagnosemöglichkeit???


Vielen Dank
Matthias Vor dem Berge
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 3:35 pm

Ergänzung:
Jetzt startet Yacy garnicht mehr, ich poste mal den Teil des LOG´s den ich als interessant erachte:
I 2013/08/04 16:29:23 org.apache.solr.search.SolrIndexSearcher Opening Searcher@1ad093c main
I 2013/08/04 16:29:24 org.apache.solr.handler.component.SpellCheckComponent Initializing spell checkers
I 2013/08/04 16:29:24 org.apache.solr.spelling.DirectSolrSpellChecker init: {name=default,field=name,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
I 2013/08/04 16:29:24 org.apache.solr.handler.component.SpellCheckComponent No queryConverter defined, using default converter
I 2013/08/04 16:29:24 org.apache.solr.handler.component.QueryElevationComponent Loading QueryElevation from: /srv/www/htdocs/yacy/DATA/INDEX/freeworld/SEGMENTS/solr_40/webgraph/conf/elevate.xml
D 2013/08/04 16:29:24 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.QueryComponent@1abcd9b
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.FacetComponent@47b35d
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.MoreLikeThisComponent@150b45a
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.HighlightComponent@170a650
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.StatsComponent@b6f7f5
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding debug component:org.apache.solr.handler.component.DebugComponent@5113f0
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.QueryComponent@1abcd9b
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.FacetComponent@47b35d
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.MoreLikeThisComponent@150b45a
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.HighlightComponent@170a650
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.StatsComponent@b6f7f5
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding debug component:org.apache.solr.handler.component.DebugComponent@5113f0
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.RealTimeGetComponent@cb1edc
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.QueryComponent@1abcd9b
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.FacetComponent@47b35d
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.MoreLikeThisComponent@150b45a
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.HighlightComponent@170a650
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.StatsComponent@b6f7f5
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding component:org.apache.solr.handler.component.SpellCheckComponent@1570f5c
D 2013/08/04 16:29:25 org.apache.solr.handler.component.SearchHandler Adding debug component:org.apache.solr.handler.component.DebugComponent@5113f0
I 2013/08/04 16:29:25 org.apache.solr.handler.ReplicationHandler Commits will be reserved for 10000
I 2013/08/04 16:29:25 org.apache.solr.handler.component.SpellCheckComponent Loading spell index for spellchecker: default
I 2013/08/04 16:29:25 org.apache.solr.handler.component.SpellCheckComponent Loading spell index for spellchecker: wordbreak
I 2013/08/04 16:29:25 org.apache.solr.core.CoreContainer registering core: webgraph
I 2013/08/04 16:29:26 org.apache.solr.update.SolrCoreState Closing SolrCoreState
I 2013/08/04 16:29:26 org.apache.solr.update.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
E 2013/08/04 16:29:27 org.apache.solr.core.CoreContainer Unable to create core: collection1
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:821)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1434)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1546)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error opening Reader
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:183)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:179)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1410)
... 15 more
Caused by: org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=1882861359 vs expected header=1071082519 (resource: NIOFSIndexInput(path="/srv/www/htdocs/yacy/DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/_1j5v_Lucene41_0.pos"))
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:128)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.<init>(Lucene41PostingsReader.java:83)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:430)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:127)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
... 18 more
E 2013/08/04 16:29:27 org.apache.solr.core.CoreContainer null:org.apache.solr.common.SolrException: Unable to create core: collection1
at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1450)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:993)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:821)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
... 10 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1434)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1546)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error opening Reader
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:183)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:179)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1410)
... 15 more
Caused by: org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=1882861359 vs expected header=1071082519 (resource: NIOFSIndexInput(path="/srv/www/htdocs/yacy/DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/_1j5v_Lucene41_0.pos"))
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:128)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.<init>(Lucene41PostingsReader.java:83)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:430)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:127)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
... 18 more

I 2013/08/04 16:29:27 SolrEmbeddedInstance detected default solr core: collection1
E 2013/08/04 16:29:27 STARTUP YaCy cannot start: SolrCore 'collection1' is not available due to init failure: Error opening new searcher
org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Error opening new searcher
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1212)
at net.yacy.cora.federate.solr.instance.EmbeddedInstance.<init>(EmbeddedInstance.java:89)
at net.yacy.search.index.Fulltext.connectLocalSolr(Fulltext.java:162)
at net.yacy.search.Switchboard.<init>(Switchboard.java:486)
at net.yacy.yacy.startup(yacy.java:225)
at net.yacy.yacy.main(yacy.java:655)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:821)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:618)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:949)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:984)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:597)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:592)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1434)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1546)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:796)
... 13 more
Caused by: org.apache.solr.common.SolrException: Error opening Reader
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:172)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:183)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:179)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1410)
... 15 more
Caused by: org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=1882861359 vs expected header=1071082519 (resource: NIOFSIndexInput(path="/srv/www/htdocs/yacy/DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/_1j5v_Lucene41_0.pos"))
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:128)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.<init>(Lucene41PostingsReader.java:83)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:430)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:127)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:88)
at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:34)
at org.apache.solr.search.SolrIndexSearcher.getReader(SolrIndexSearcher.java:169)
... 18 more

Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Ruhe » So Aug 04, 2013 3:40 pm

Da scheint nun was kaputt zu sein.
Beim Lesen von CorruptIndexException ist mir das eingefallen: viewtopic.php?f=20&t=4727

Du solltest auf die Fachleute warten. Oder aber so mutig sein und das Kommando aus dem 1. Posting in o.g. Thema ausführen.
Ruhe
 
Beiträge: 75
Registriert: So Jul 21, 2013 10:37 am

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 3:44 pm

Danke für den Tipp, ich warte glaube ich lieber auf die Fachleute... :-)
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Ruhe » So Aug 04, 2013 3:53 pm

Ein Problem bei dir scheint nun zu sein, dass die Datenbank einen abbekommen hat


Hab aber auch bereits so meine Erfahrungen:

- YaCy läuft, aber das Webinterface ist nicht erreichbar; entweder erst nach einigen Stunden nicht mehr oder auch direkt nach dem Programmstart. Bei Rechnern mit wenig RAM könnte genau das der Grund sein, kein Speicher mehr
- YaCy läßt sich nicht beenden - nur per killyacy.sh (stopyacy.sh schafft es nicht die Anwendung zu beenden)
Ruhe
 
Beiträge: 75
Registriert: So Jul 21, 2013 10:37 am

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 3:57 pm

Wie viel Ram braucht so ein Rechner denn bis er stabil mit Yacy läuft?
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Ruhe » So Aug 04, 2013 4:04 pm

Gute Frage und vielleicht auch gar nicht so leicht zu beantworten, meiner Meinung nach.
1.5 GB halte ich jedenfalls für zu wenig.
Ruhe
 
Beiträge: 75
Registriert: So Jul 21, 2013 10:37 am

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 5:32 pm

Ich bin Deinem Tipp dann doch gefolgt:
Hier die Ausgabe:
yacy@maxtest:/srv/www/htdocs/yacy> java -cp 'lib/*' org.apache.lucene.index.CheckIndex DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/ -fix

NOTE: testing will be more thorough if you run java with '-ea:org.apache.lucene...', so assertions are enabled

Opening index @ DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/

Segments file=segments_y9g numSegments=31 version=4.3 format= userData={commitTimeMSec=1375425710266}
1 of 31: name=_54e docCount=327893
codec=Lucene42
compound=false
numFiles=11
size (MB)=744.864
diagnostics = {timestamp=1374272942189, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=47]
test: open reader.........OK [98 deleted docs]
test: fields..............OK [68 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [5999586 terms; 52957090 terms/docs pairs; 110673361 tokens]
test (ignoring deletes): terms, freq, prox...OK [6003452 terms; 53025106 terms/docs pairs; 110836468 tokens]
test: stored fields.......OK [15767830 total field count; avg 48.103 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

2 of 31: name=_340 docCount=192562
codec=Lucene42
compound=false
numFiles=11
size (MB)=303.441
diagnostics = {timestamp=1374189503784, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=155]
test: open reader.........OK [5606 deleted docs]
test: fields..............OK [68 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [2627581 terms; 18252080 terms/docs pairs; 31435953 tokens]
test (ignoring deletes): terms, freq, prox...OK [3088040 terms; 22895517 terms/docs pairs; 43293921 tokens]
test: stored fields.......OK [6404516 total field count; avg 34.257 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

3 of 31: name=_ifi docCount=884547
codec=Lucene42
compound=false
numFiles=11
size (MB)=959.228
diagnostics = {timestamp=1374462502759, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=1577]
test: open reader.........OK [1951 deleted docs]
test: fields..............OK [65 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [7766548 terms; 87658222 terms/docs pairs; 133772305 tokens]
test (ignoring deletes): terms, freq, prox...OK [7788761 terms; 88686071 terms/docs pairs; 136186018 tokens]
test: stored fields.......OK [29504202 total field count; avg 33.429 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

4 of 31: name=_x8l docCount=1135066
codec=Lucene42
compound=false
numFiles=11
size (MB)=1,218.117
diagnostics = {timestamp=1374748662803, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=3097]
test: open reader.........OK [4592 deleted docs]
test: fields..............OK [68 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [9393128 terms; 111823914 terms/docs pairs; 170537932 tokens]
test (ignoring deletes): terms, freq, prox...OK [9434347 terms; 114016879 terms/docs pairs; 175438021 tokens]
test: stored fields.......OK [37578063 total field count; avg 33.241 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

5 of 31: name=_10wv docCount=292556
codec=Lucene42
compound=false
numFiles=11
size (MB)=196.993
diagnostics = {timestamp=1375208598620, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=1897]
test: open reader.........OK [14934 deleted docs]
test: fields..............OK [64 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [2675627 terms; 19532587 terms/docs pairs; 21377879 tokens]
test (ignoring deletes): terms, freq, prox...OK [2782810 terms; 20658045 terms/docs pairs; 22656178 tokens]
test: stored fields.......OK [7621763 total field count; avg 27.454 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

6 of 31: name=_1fz4 docCount=495908
codec=Lucene42
compound=false
numFiles=11
size (MB)=676.03
diagnostics = {timestamp=1375380395702, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=1558]
test: open reader.........OK [3675 deleted docs]
test: fields..............OK [65 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [4798703 terms; 59936474 terms/docs pairs; 100799939 tokens]
test (ignoring deletes): terms, freq, prox...OK [4832522 terms; 61637103 terms/docs pairs; 104457137 tokens]
test: stored fields.......OK [18268502 total field count; avg 37.114 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

7 of 31: name=_18du docCount=182440
codec=Lucene42
compound=false
numFiles=11
size (MB)=226.737
diagnostics = {timestamp=1375293882949, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=2200]
test: open reader.........OK [3232 deleted docs]
test: fields..............OK [65 fields]
test: field norms.........OK [20 fields]
test: terms, freq, prox...OK [2043265 terms; 19025002 terms/docs pairs; 29972805 tokens]
test (ignoring deletes): terms, freq, prox...OK [2082980 terms; 20513236 terms/docs pairs; 33129450 tokens]
test: stored fields.......OK [6157396 total field count; avg 34.359 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

8 of 31: name=_156i docCount=262076
codec=Lucene42
compound=false
numFiles=11
size (MB)=168.659
diagnostics = {timestamp=1375262291086, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=1537]
test: open reader.........OK [9379 deleted docs]
test: fields..............OK [64 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [2338111 terms; 17087792 terms/docs pairs; 17747501 tokens]
test (ignoring deletes): terms, freq, prox...OK [2412842 terms; 18017605 terms/docs pairs; 19073584 tokens]
test: stored fields.......OK [6864090 total field count; avg 27.163 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

9 of 31: name=_1d84 docCount=24157
codec=Lucene42
compound=false
numFiles=11
size (MB)=155.978
diagnostics = {timestamp=1375347023529, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=1648]
test: open reader.........OK [2879 deleted docs]
test: fields..............OK [64 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [634672 terms; 11067115 terms/docs pairs; 25252721 tokens]
test (ignoring deletes): terms, freq, prox...OK [672985 terms; 12405029 terms/docs pairs; 28158328 tokens]
test: stored fields.......OK [2357720 total field count; avg 110.806 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

10 of 31: name=_1h3n docCount=10445
codec=Lucene42
compound=false
numFiles=11
size (MB)=65.858
diagnostics = {timestamp=1375393656563, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=544]
test: open reader.........OK [867 deleted docs]
test: fields..............OK [64 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [348352 terms; 4843078 terms/docs pairs; 10856301 tokens]
test (ignoring deletes): terms, freq, prox...OK [363148 terms; 5234881 terms/docs pairs; 11712413 tokens]
test: stored fields.......OK [1042806 total field count; avg 108.875 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

11 of 31: name=_1ips docCount=24553
codec=Lucene42
compound=false
numFiles=11
size (MB)=154.407
diagnostics = {timestamp=1375413188952, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=168]
test: open reader.........OK [428 deleted docs]
test: fields..............OK [64 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [650136 terms; 12126252 terms/docs pairs; 27399360 tokens]
test (ignoring deletes): terms, freq, prox...OK [655404 terms; 12325307 terms/docs pairs; 27845986 tokens]
test: stored fields.......OK [2609347 total field count; avg 108.159 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

12 of 31: name=_1i60 docCount=846
codec=Lucene42
compound=false
numFiles=11
size (MB)=5.859
diagnostics = {timestamp=1375406978927, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=53]
test: open reader.........OK [73 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [57606 terms; 401619 terms/docs pairs; 909430 tokens]
test (ignoring deletes): terms, freq, prox...OK [60362 terms; 435788 terms/docs pairs; 987626 tokens]
test: stored fields.......OK [88325 total field count; avg 114.263 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

13 of 31: name=_1ip7 docCount=537
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.699
diagnostics = {timestamp=1375412894526, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=12]
test: open reader.........OK [13 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [42164 terms; 261606 terms/docs pairs; 586236 tokens]
test (ignoring deletes): terms, freq, prox...OK [43148 terms; 269884 terms/docs pairs; 603722 tokens]
test: stored fields.......OK [56111 total field count; avg 107.082 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

14 of 31: name=_1ipi docCount=566
codec=Lucene42
compound=false
numFiles=11
size (MB)=4.006
diagnostics = {timestamp=1375413062186, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=8]
test: open reader.........OK [8 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [43102 terms; 284216 terms/docs pairs; 652002 tokens]
test (ignoring deletes): terms, freq, prox...OK [43831 terms; 289493 terms/docs pairs; 662839 tokens]
test: stored fields.......OK [61206 total field count; avg 109.688 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

15 of 31: name=_1ivb docCount=510
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.49
diagnostics = {timestamp=1375415088658, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=12]
test: open reader.........OK [16 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [39728 terms; 245932 terms/docs pairs; 541729 tokens]
test (ignoring deletes): terms, freq, prox...OK [40838 terms; 255221 terms/docs pairs; 562182 tokens]
test: stored fields.......OK [51997 total field count; avg 105.257 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

16 of 31: name=_1izi docCount=466
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.218
diagnostics = {timestamp=1375416437774, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=7]
test: open reader.........OK [7 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [37784 terms; 229565 terms/docs pairs; 515659 tokens]
test (ignoring deletes): terms, freq, prox...OK [38240 terms; 233729 terms/docs pairs; 525545 tokens]
test: stored fields.......OK [49342 total field count; avg 107.499 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

17 of 31: name=_1iwf docCount=568
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.873
diagnostics = {timestamp=1375415377366, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=8]
test: open reader.........OK [8 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [42520 terms; 278368 terms/docs pairs; 623772 tokens]
test (ignoring deletes): terms, freq, prox...OK [43529 terms; 284099 terms/docs pairs; 639256 tokens]
test: stored fields.......OK [60305 total field count; avg 107.688 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

18 of 31: name=_1j01 docCount=409
codec=Lucene42
compound=false
numFiles=11
size (MB)=2.876
diagnostics = {timestamp=1375416648878, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=9]
test: open reader.........OK [9 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [35003 terms; 200340 terms/docs pairs; 447729 tokens]
test (ignoring deletes): terms, freq, prox...OK [35831 terms; 207027 terms/docs pairs; 462730 tokens]
test: stored fields.......OK [43411 total field count; avg 108.527 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

19 of 31: name=_1izs docCount=465
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.188
diagnostics = {timestamp=1375416533898, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=3]
test: open reader.........OK [3 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [36790 terms; 229738 terms/docs pairs; 526553 tokens]
test (ignoring deletes): terms, freq, prox...OK [37103 terms; 231877 terms/docs pairs; 530696 tokens]
test: stored fields.......OK [50268 total field count; avg 108.805 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

20 of 31: name=_1j0v docCount=1194
codec=Lucene42
compound=false
numFiles=11
size (MB)=7.735
diagnostics = {timestamp=1375417950953, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=5]
test: open reader.........OK [5 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [71894 terms; 579563 terms/docs pairs; 1276635 tokens]
test (ignoring deletes): terms, freq, prox...OK [72186 terms; 582586 terms/docs pairs; 1284066 tokens]
test: stored fields.......OK [124565 total field count; avg 104.765 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

21 of 31: name=_1iz7 docCount=446
codec=Lucene42
compound=false
numFiles=10
size (MB)=3.01
diagnostics = {timestamp=1375416326340, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [59 fields]
test: field norms.........OK [18 fields]
test: terms, freq, prox...OK [35744 terms; 218658 terms/docs pairs; 488325 tokens]
test: stored fields.......OK [47847 total field count; avg 107.28 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

22 of 31: name=_1j5v docCount=391
codec=Lucene42
compound=false
numFiles=10
size (MB)=2.675
diagnostics = {timestamp=1375425507464, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........FAILED
WARNING: fixIndex() would remove reference to this segment; full exception:
org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=1882861359 vs expected header=1071082519 (resource: NIOFSIndexInput(path="/srv/www/htdocs/yacy/DATA/INDEX/freeworld/SEGMENTS/solr_40/collection1/data/index/_1j5v_Lucene41_0.pos"))
at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:128)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.<init>(Lucene41PostingsReader.java:83)
at org.apache.lucene.codecs.lucene41.Lucene41PostingsFormat.fieldsProducer(Lucene41PostingsFormat.java:430)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:194)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:233)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:127)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:56)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:543)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1854)

23 of 31: name=_1j66 docCount=436
codec=Lucene42
compound=false
numFiles=11
size (MB)=3.224
diagnostics = {timestamp=1375425628866, os=Linux, os.version=2.6.37.1-1.2-default, mergeFactor=10, source=merge, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, mergeMaxNumSegments=-1, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
has deletions [delGen=3]
test: open reader.........OK [5 deleted docs]
test: fields..............OK [60 fields]
test: field norms.........OK [19 fields]
test: terms, freq, prox...OK [37857 terms; 227087 terms/docs pairs; 530976 tokens]
test (ignoring deletes): terms, freq, prox...OK [38384 terms; 230880 terms/docs pairs; 540951 tokens]
test: stored fields.......OK [48639 total field count; avg 112.852 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

24 of 31: name=_1j65 docCount=4
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.043
diagnostics = {timestamp=1375425629747, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [47 fields]
test: field norms.........OK [12 fields]
test: terms, freq, prox...OK [1507 terms; 1813 terms/docs pairs; 3582 tokens]
test: stored fields.......OK [364 total field count; avg 91 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

25 of 31: name=_1j67 docCount=18
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.16
diagnostics = {timestamp=1375425638877, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [57 fields]
test: field norms.........OK [17 fields]
test: terms, freq, prox...OK [4418 terms; 8246 terms/docs pairs; 21327 tokens]
test: stored fields.......OK [1869 total field count; avg 103.833 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

26 of 31: name=_1j68 docCount=1
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.017
diagnostics = {timestamp=1375425640444, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [41 fields]
test: field norms.........OK [9 fields]
test: terms, freq, prox...OK [542 terms; 542 terms/docs pairs; 863 tokens]
test: stored fields.......OK [109 total field count; avg 109 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

27 of 31: name=_1j69 docCount=23
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.254
diagnostics = {timestamp=1375425650380, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [57 fields]
test: field norms.........OK [18 fields]
test: terms, freq, prox...OK [5537 terms; 14089 terms/docs pairs; 34393 tokens]
test: stored fields.......ERROR [Corrupted: docID=1, docBase=0, chunkDocs=1, numDocs=23]
org.apache.lucene.index.CorruptIndexException: Corrupted: docID=1, docBase=0, chunkDocs=1, numDocs=23
at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:203)
at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:133)
at org.apache.lucene.index.IndexReader.document(IndexReader.java:436)
at org.apache.lucene.index.CheckIndex.testStoredFields(CheckIndex.java:1252)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:610)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1854)
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]
FAILED
WARNING: fixIndex() would remove reference to this segment; full exception:
java.lang.RuntimeException: Stored Field test failed
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:624)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1854)

28 of 31: name=_1j6a docCount=3
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.038
diagnostics = {timestamp=1375425651445, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [45 fields]
test: field norms.........OK [10 fields]
test: terms, freq, prox...OK [1287 terms; 1397 terms/docs pairs; 3121 tokens]
test: stored fields.......OK [336 total field count; avg 112 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

29 of 31: name=_1j6b docCount=19
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.226
diagnostics = {timestamp=1375425660341, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [55 fields]
test: field norms.........OK [17 fields]
test: terms, freq, prox...OK [5357 terms; 12066 terms/docs pairs; 31851 tokens]
test: stored fields.......OK [2712 total field count; avg 142.737 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

30 of 31: name=_1j6c docCount=11
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.151
diagnostics = {timestamp=1375425671208, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [54 fields]
test: field norms.........OK [16 fields]
test: terms, freq, prox...OK [4589 terms; 7340 terms/docs pairs; 18470 tokens]
test: stored fields.......OK [1442 total field count; avg 131.091 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

31 of 31: name=_1j6d docCount=22
codec=Lucene42
compound=false
numFiles=10
size (MB)=0.238
diagnostics = {timestamp=1375425710665, os=Linux, os.version=2.6.37.1-1.2-default, source=flush, lucene.version=4.3.0 1477023 - simonw - 2013-04-29 14:50:23, os.arch=i386, java.version=1.6.0_20, java.vendor=Sun Microsystems Inc.}
no deletions
test: open reader.........OK
test: fields..............OK [56 fields]
test: field norms.........OK [18 fields]
test: terms, freq, prox...OK [6261 terms; 12810 terms/docs pairs; 33144 tokens]
test: stored fields.......OK [2759 total field count; avg 125.409 fields per doc]
test: term vectors........OK [0 total vector count; avg 0 term/freq vector fields per doc]
test: docvalues...........OK [0 total doc count; 0 docvalues fields]

WARNING: 2 broken segments (containing 414 documents) detected
WARNING: 414 documents will be lost

NOTE: will write new segments file in 5 seconds; this will remove 414 docs from the index. THIS IS YOUR LAST CHANCE TO CTRL+C!
5...
4...
3...
2...
1...
Writing...
OK
Wrote new segments file "segments_y9h"
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 5:43 pm

Ob der Datenbestand noch der Gleiche ist wie vorher weiß ich nicht, doch Yacy startet wieder...
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Ruhe » So Aug 04, 2013 5:45 pm

Laut Ausgabe, siehe am Ende, hast du 414 Dokumente verloren.
Ruhe
 
Beiträge: 75
Registriert: So Jul 21, 2013 10:37 am

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 04, 2013 5:52 pm

Leider sagt mir das nicht wirklich viel...

Habe ich Daten zu 414 Seiten verloren???

Aber Danke Dir, der Tipp hat mir geholfen!!
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Ruhe » So Aug 04, 2013 5:55 pm

Llandon hat geschrieben:Habe ich Daten zu 414 Seiten verloren???

Muss ich leider passen. Kann nicht sagen was bei solr "Documents" bedeutet, oder in welcher Form YaCy "Documents" in solr erzeugt bzw. was in einem "Document" abgelegt wird.
Ruhe
 
Beiträge: 75
Registriert: So Jul 21, 2013 10:37 am

Re: Langsamer Tod...

Beitragvon Vega » Mo Aug 05, 2013 8:54 am

Hallo,

4 GB und mehr.... es kommt halt immer darauf an was der Peer machen soll, es gibt da auch sicherlich noch Optimierungspotential in Sachen Speicher.
Mann kann auch mit 600MB zugeordnetem Speicher eine einzelne Webseite/Forum crawlen und durchsuchbar machen - solange nicht mehrere 1000 Suchanfragen
am Tag auf die Kiste ein-prasseln. Ich habe bei dem SUMA-EV Perr auch das Problem das sobald ich Crawle mir die KIste abschmiert, und das auf einem Server mit 32GB Ram - YaCy hat dabei 12 GB aktuell zugewiesen - da smuss ich aber mit dem aktuellem Relase nochmal gegen testen.

Gruß
Thomas

Llandon hat geschrieben:Wie viel Ram braucht so ein Rechner denn bis er stabil mit Yacy läuft?
Vega
 
Beiträge: 824
Registriert: Mi Jun 27, 2007 3:34 pm
Wohnort: Dresden

Re: Langsamer Tod...

Beitragvon Llandon » So Aug 11, 2013 8:24 am

Moin moin...

4GB und mehr...
UPS!

Dann ist die Idee den "abgelegten Rechner" zu verwenden wohl doch keine so tolle....
Okay, war ein Versuch...

Danke
Matthias
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Vega » Mo Aug 12, 2013 12:29 am

Naja, auch mit einen kleinen Peer kannst Du schon nette Sachen machen, mach halt einen Spezialisierten Index von Seiten die Dir wichtig sind...

Thomas

Llandon hat geschrieben:Moin moin...

4GB und mehr...
UPS!

Dann ist die Idee den "abgelegten Rechner" zu verwenden wohl doch keine so tolle....
Okay, war ein Versuch...

Danke
Matthias
Vega
 
Beiträge: 824
Registriert: Mi Jun 27, 2007 3:34 pm
Wohnort: Dresden

Re: Langsamer Tod...

Beitragvon Llandon » Di Aug 13, 2013 3:35 pm

Hallo,

Naja, das war der Plan...
Ein Peer der Fotoseiten indiziert...
Aber eine Seite (traumflieger.de) reicht schon um regelmäßige Abstürze zu provozieren...

Gruß
Matthias
Llandon
 
Beiträge: 25
Registriert: Di Feb 17, 2009 10:28 am
Wohnort: Borgholzhausen

Re: Langsamer Tod...

Beitragvon Vega » Di Sep 03, 2013 9:34 pm

Hallo,

ich stelle das mal am WE nach und teste mal mit wieviel Speicher ich das crawlen kann.... Mit welcher Linktiefe / Einstellungen hast Du das gemacht?

Thomas

Llandon hat geschrieben:Hallo,

Naja, das war der Plan...
Ein Peer der Fotoseiten indiziert...
Aber eine Seite (traumflieger.de) reicht schon um regelmäßige Abstürze zu provozieren...

Gruß
Matthias
Vega
 
Beiträge: 824
Registriert: Mi Jun 27, 2007 3:34 pm
Wohnort: Dresden


Zurück zu Hilfe für Einsteiger und Anwender

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron