poison pill ?

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

poison pill ?

Beitragvon miTreD » Fr Jan 30, 2009 6:23 pm

Beim Shutdown zeigt mir das Log Einträge zu einer ominösen "poison pill". Ist das normal?
Code: Alles auswählen
S 2009/01/30 18:16:37 SHUTDOWN server has terminated
I 2009/01/30 18:16:37 YACY publish: Interruption detected while publishing my seed.
I 2009/01/30 18:16:37 YACY publish: Signaling shutdown to 1 remaining publishing threads ...
I 2009/01/30 18:16:38 YACY publish: Shutdown off all remaining publishing thread finished.
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue parseDocument, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.parseDocument.12' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue parseDocument, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue parseDocument, thread 1
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.parseDocument.13' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue parseDocument, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue parseDocument, thread 2
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.parseDocument.14' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue parseDocument, thread 2. awaiting termination
I 2009/01/30 18:16:38 serverProcessor queue parseDocument: shutdown.
I 2009/01/30 18:16:38 STACKCRAWL Flushing remaining 0 crawl stacker job entries.
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerFast, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.0' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerFast, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerFast, thread 1
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.1' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerFast, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.2' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 1
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.3' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 2
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.4' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 2. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 3
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.5' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 3. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 4
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.job.6' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 4. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue condenseDocument, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.condenseDocument.10' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue condenseDocument, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue condenseDocument, thread 1
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.condenseDocument.11' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue condenseDocument, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue webStructureAnalysis, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.webStructureAnalysis.8' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue webStructureAnalysis, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue webStructureAnalysis, thread 1
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.webStructureAnalysis.9' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue webStructureAnalysis, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue storeDocumentIndex, thread 0
S 2009/01/30 18:16:38 THREAD-CONTROL thread 'java.lang.reflect.Method.storeDocumentIndex.7' terminated.
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue storeDocumentIndex, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor queue condenseDocument: shutdown.
I 2009/01/30 18:16:38 serverProcessor queue webStructureAnalysis: shutdown.
I 2009/01/30 18:16:38 serverProcessor queue storeDocumentIndex: shutdown.
I 2009/01/30 18:16:38 STACKCRAWL Shutdown. waiting for remaining 0 crawl stacker job entries. please wait.
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerFast, thread 0
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerFast, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerFast, thread 1
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerFast, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 0
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 0. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 1
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 1. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 2
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 2. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 3
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 3. awaiting termination
I 2009/01/30 18:16:38 serverProcessor putting poison pill in queue CrawlStackerSlow, thread 4
I 2009/01/30 18:16:38 serverProcessor .. poison pill is in queue CrawlStackerSlow, thread 4. awaiting termination
I 2009/01/30 18:16:38 serverProcessor queue CrawlStackerFast: shutdown.
I 2009/01/30 18:16:38 serverProcessor queue CrawlStackerSlow: shutdown.
I 2009/01/30 18:16:38 STACKCRAWL Shutdown. Closing stackCrawl queue.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/wiki.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/wiki-bkp.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/blog.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/blogComment.db/col.000.list' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/blogComment.db/col.001.list' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/SETTINGS/user.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/bookmarks.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/bookmarkTags.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/bookmarkDates.db' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/WORK/message.db' closed.
I 2009/01/30 18:16:38 kelondroBLOBHeap wrote a dump for the 21570 index entries of crawlRobotsTxt.heap in 184 milliseconds.
I 2009/01/30 18:16:38 PLASMA Saving Web Structure File
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/PLASMADB/urlNoticeCoreStack9.stack' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/PLASMADB/urlNoticeLimitStack9.stack' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/PLASMADB/urlNoticeRemoteStack9.stack' closed.
I 2009/01/30 18:16:38 indexContainerRAMHeap creating alternative rwi heap dump 'index.dhtout.blob', 898 rwi's
I 2009/01/30 18:16:38 kelondroBLOBHeapWriter wrote a dump for the 898 index entries of index.dhtout.blob in 23 milliseconds.
I 2009/01/30 18:16:38 indexContainerRAMHeap finished alternative rwi heap dump: 898 words, 5376 word/URL relations in 49 milliseconds
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.01.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.07.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.04.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.00.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.03.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.02.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.06.00.kca' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/TEXT/RICOLLECTION/collection.04.0028.05.00.kca' closed.
I 2009/01/30 18:16:38 kelondroBLOBHeap wrote a dump for the 82 index entries of seed.new.heap in 6 milliseconds.
I 2009/01/30 18:16:38 kelondroBLOBHeap wrote a dump for the 55 index entries of seed.old.heap in 41 milliseconds.
I 2009/01/30 18:16:38 kelondroBLOBHeap wrote a dump for the 31 index entries of seed.pot.heap in 28 milliseconds.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/NETWORK/newsOut.stack' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/NETWORK/newsPublished.stack' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/NETWORK/newsIn.stack' closed.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/NETWORK/newsProcessed.stack' closed.
I 2009/01/30 18:16:38 kelondroBLOBHeap wrote a dump for the 6 index entries of crawlProfilesActive.heap in 70 milliseconds.
D 2009/01/30 18:16:38 KELONDRO file '/yacy/DATA/INDEX/freeworld/QUEUES/urlNoticePreStack' closed.
S 2009/01/30 18:16:38 SHUTDOWN goodbye. (this is the last line)
miTreD
 
Beiträge: 1241
Registriert: Mi Jun 27, 2007 11:35 am
Wohnort: /home

Re: poison pill ?

Beitragvon Orbiter » Fr Jan 30, 2009 9:59 pm

uh, hab mich schon gefragt wie lange es dauert bis jemand fragt. Das ist eine Methode, um bei Concurrency eine Queue stabil abzuarbeiten und den Worker Threads zu sagen das sie sich beenden sollen. Ich habe die Methode aus dem Buch "Java Concurrency in Practice", und dazu gibt es auch Beispielprogramme online. Das hier scheint relevant zu sein:
http://www.javaconcurrencyinpractice.co ... rvice.java
So ein Zufall das das Beispiel auch noch 'sachverwandt' ist.

Das ganze geht so: stell dir eine FIFO-Queue vor, in die du Jobs reinschiebst, und die von x Worker Threads parallel abgearbeitet wird. Wenn man nun die Threads stabil beenden will, und vorher die Queue noch leer haben will, schiebt man einfach x poison pills vorne in die queue. Die werden dann als letztes bearbeitet. Die Poison Pill sagt dem Thread einfach das es sich beenden soll. Da damit jeder Thread sich auch nur eine Pill nimmt, bekommt jeder Thread eine, und zwar insgesamt nachdem alle anderen Jobs noch abgearbeitet wurden.

In YaCy gibt es seit einiger Zeit so eine Indexing-Queue, die aber vier Zwischenstationen hat. Die Kontrollmechanismen dazu sieht man in dem Blocking Queue Monitor. Diese Queue wird eben auch durch Poison Pills beendet.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: poison pill ?

Beitragvon miTreD » Fr Jan 30, 2009 10:10 pm

OK, das macht Sinn :-) Kenne die "poison pill" nur von Hochverfügbarkeits-Clustern. Das erfüllen sie eine ganz ähnliche Funktion. Wenn ein Mitglied des Clusters einen undefinierten Zustand einnimmt, bekommt es von den anderen Cluster-Mitgliedern eine "poison pill" untergeschoben, worauf hin sich dieses Mitglied sofort herunterfährt. Damit werden ziemlich effektiv Inkosistenzen im Cluster vermieden.
miTreD
 
Beiträge: 1241
Registriert: Mi Jun 27, 2007 11:35 am
Wohnort: /home


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: Google [Bot] und 2 Gäste