very high load >7 on crawling

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

very high load >7 on crawling

Beitragvon lucipher » Mi Jun 28, 2017 11:16 am

I can't register on mantis bug tracker so I am posting here

Few times a day search engine stops with very high load (from 7 to 8). Stopping crawling and waiting do not resolve the problem

Only one way is kill -9 on yacy process
Below info from yacy and system (ubuntu 16_04 LTS). This same problem is on openjdk8


System
YaCy version: 1.92/9000
Uptime: 0 days 04:06
Java version: 1.7.0_95
Processors: 8
Load: 7.54
Threads: 271/14, peak:427, total:4127

------------------

************* Start Thread Dump Wed Jun 28 04:49:56 GMT-05:00 2017 *******************

YaCy Version: 1.92/9000
Assigned Memory = 12525240320
Used Memory = 10503683528
Available Memory = 2021556792


this thread dump function can find threads that lock others, to enable this function start YaCy with 'startYACY.sh -l'

THREADS WITH STATES: BLOCKED


THREADS WITH STATES: RUNNABLE

Thread= qtp162548417-23108 id=23108 RUNNABLE
at net.yacy.server.serverAccessTracker.clearTooOldAccess(serverAccessTracker.java:120) [l = e.next();]
at net.yacy.server.serverAccessTracker.track(serverAccessTracker.java:141)
at net.yacy.http.Jetty9YaCySecurityHandler.prepareConstraintInfo(Jetty9YaCySecurityHandler.java:68)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:478)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at net.yacy.http.CrashProtectionHandler.handle(CrashProtectionHandler.java:33)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= Reference Handler daemon id=2 RUNNABLE
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:146)


Thread= qtp162548417-23111 id=23111 RUNNABLE
at net.yacy.cora.storage.ConcurrentARC.clear(ConcurrentARC.java:183) [for (final ARC a: this.arc) a.clear();]
at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.clearCaches(ConcurrentUpdateSolrConnector.java:130)
at net.yacy.cora.federate.solr.instance.InstanceMirror.clearCaches(InstanceMirror.java:192)
at net.yacy.search.index.Fulltext.clearCaches(Fulltext.java:218)
at net.yacy.search.index.Segment.clearCaches(Segment.java:473)
at Crawler_p.respond(Crawler_p.java:88)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(YaCyDefaultServlet.java:670)
at net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(YaCyDefaultServlet.java:879)
at net.yacy.http.servlets.YaCyDefaultServlet.doGet(YaCyDefaultServlet.java:312)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at net.yacy.http.CrashProtectionHandler.handle(CrashProtectionHandler.java:33)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= qtp162548417-81-selector-ServerConnectorManager@3acf9dfc/1 id=81 RUNNABLE
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:601)
at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:550)
at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= Finalizer daemon id=3 RUNNABLE
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)


Thread= parsing + http://www.prawo.vulcan.edu.pl/przegdok ... &qplikid=2 id=20705 RUNNABLE
at java.lang.String.toLowerCase(String.java:2519)
at net.yacy.cora.document.id.MultiProtocolURL.toNormalform(MultiProtocolURL.java:1150)
at net.yacy.cora.document.id.MultiProtocolURL.toNormalform(MultiProtocolURL.java:1115)
at net.yacy.document.parser.html.ContentScraper.addAnchor(ContentScraper.java:763)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:791)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.document.parser.html.ContentScraper.recursiveParse(ContentScraper.java:780)
at net.yacy.document.parser.html.ContentScraper.scrapeTag1(ContentScraper.java:670)
at net.yacy.document.parser.html.TransformerWriter.filterTagCloseing(TransformerWriter.java:310)
at net.yacy.document.parser.html.TransformerWriter.filterTag(TransformerWriter.java:281)
at net.yacy.document.parser.html.TransformerWriter.tokenProcessor(TransformerWriter.java:199)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:431)
at net.yacy.document.parser.html.TransformerWriter.write(TransformerWriter.java:479)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:199)
at net.yacy.kelondro.util.FileUtils.copy(FileUtils.java:156)
at net.yacy.document.parser.htmlParser.parseToScraper(htmlParser.java:256)
at net.yacy.document.parser.htmlParser.parse(htmlParser.java:106)
at net.yacy.document.TextParser.parseSource(TextParser.java:313)
at net.yacy.document.TextParser.parseSource(TextParser.java:214)
at net.yacy.search.Switchboard.parseDocument(Switchboard.java:2746)
at net.yacy.search.Switchboard.parseDocument(Switchboard.java:2703)
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Thread= qtp162548417-23115 id=23115 RUNNABLE
at java.lang.Thread.getAllStackTraces(Thread.java:1640)
at net.yacy.kelondro.logging.ThreadDump.getAllStackTraces(ThreadDump.java:100)
at Threaddump_p.respond(Threaddump_p.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at net.yacy.http.servlets.YaCyDefaultServlet.invokeServlet(YaCyDefaultServlet.java:670)
at net.yacy.http.servlets.YaCyDefaultServlet.handleTemplate(YaCyDefaultServlet.java:881)
at net.yacy.http.servlets.YaCyDefaultServlet.doGet(YaCyDefaultServlet.java:312)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at net.yacy.http.CrashProtectionHandler.handle(CrashProtectionHandler.java:33)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= qtp162548417-84-acceptor-0@27c90d11-httpd:8090@f8495b0{HTTP/1.1}{0.0.0.0:8090} id=84 RUNNABLE
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)



THREADS WITH STATES: TIMED_WAITING

Thread= MemoryTracker id=11 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at net.yacy.search.MemoryTracker.run(MemoryTracker.java:60)


Thread= qtp162548417-23108 id=23108 TIMED_WAITING
at net.yacy.server.serverAccessTracker.clearTooOldAccess(serverAccessTracker.java:120) [l = e.next();]
at net.yacy.server.serverAccessTracker.track(serverAccessTracker.java:141)
at net.yacy.http.Jetty9YaCySecurityHandler.prepareConstraintInfo(Jetty9YaCySecurityHandler.java:68)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:478)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at net.yacy.http.CrashProtectionHandler.handle(CrashProtectionHandler.java:33)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= qtp162548417-23081 id=23081 TIMED_WAITING
at java.util.AbstractQueue.add(AbstractQueue.java:95)
at net.yacy.server.serverAccessTracker.track(serverAccessTracker.java:140)
at net.yacy.http.Jetty9YaCySecurityHandler.prepareConstraintInfo(Jetty9YaCySecurityHandler.java:68)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:478)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at net.yacy.http.CrashProtectionHandler.handle(CrashProtectionHandler.java:33)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)


Thread= net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector_ProcessHandler id=64 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector$CommitHandler.run(ConcurrentUpdateSolrConnector.java:62)


Thread= IndexCell.FlushThread(/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/default) id=22 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at net.yacy.kelondro.rwi.IndexCell$FlushThread.run(IndexCell.java:126)


Thread= HTTPClient.IdleConnectionMonitorThread id=77 TIMED_WAITING
at java.lang.Object.wait(Native Method)
at net.yacy.cora.protocol.http.HTTPClient$IdleConnectionMonitorThread.run(HTTPClient.java:926)


Thread= Scheduler-615475072 id=95 TIMED_WAITING
at java.lang.Thread.run(Thread.java:745)


Thread= BusyThread net.yacy.contentcontrol.ContentControlFilterUpdateThread.run daemon id=76 TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at net.yacy.kelondro.workflow.AbstractBusyThread.ratz(AbstractBusyThread.java:266)
at net.yacy.kelondro.workflow.AbstractBusyThread.run(AbstractBusyThread.java:208)


Thread= qtp162548417-23116 id=23116 TIMED_WAITING
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
at java.lang.Thread.run(Thread.java:745)



THREADS WITH STATES: WAITING

Thread= condenseDocument_pool-1-thread-677 id=4068 WAITING
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at net.yacy.kelondro.workflow.WorkflowProcessor.take(WorkflowProcessor.java:131)
at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:73)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Thread= Array.SortJobWorker id=15 WAITING
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at net.yacy.cora.sorting.Array$SortJobWorker.run(Array.java:71)


Thread= WebStructureGraph.PublicRefDNSResolvingProcess id=41 WAITING
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at net.yacy.peers.graphics.WebStructureGraph$PublicRefDNSResolvingProcess.run(WebStructureGraph.java:148)


Thread= Thread-10 id=25 WAITING
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1240)


Thread= IODispatcher id=21 WAITING
at java.util.concurrent.Semaphore.acquire(Semaphore.java:317)
at net.yacy.kelondro.rwi.IODispatcher.run(IODispatcher.java:155)


Thread= Balancer waiting for koszyk.wydawnictwofronda.pl: 35 milliseconds daemon id=69 WAITING
at java.lang.Object.wait(Object.java:503)
at net.yacy.crawler.data.CrawlQueues.isPaused(CrawlQueues.java:408)
at net.yacy.crawler.data.CrawlQueues.coreCrawlJob(CrawlQueues.java:289)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at net.yacy.kelondro.workflow.InstantBusyThread.job(InstantBusyThread.java:105)
at net.yacy.kelondro.workflow.AbstractBusyThread.run(AbstractBusyThread.java:215)


Thread= main id=1 WAITING
at java.util.concurrent.Semaphore.acquire(Semaphore.java:317)
at net.yacy.search.Switchboard.waitForShutdown(Switchboard.java:4062)
at net.yacy.yacy.startup(yacy.java:381)
at net.yacy.yacy.main(yacy.java:773)


Thread= RobotsTxt_pool-1-thread-97 id=249 WAITING
at java.lang.Thread.run(Thread.java:745)


Thread= Log Worker id=9 WAITING
at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
at net.yacy.cora.util.ConcurrentLog$Worker.run(ConcurrentLog.java:355)



THREADS WITH STATES: NEW


THREADS WITH STATES: TERMINATED


************* End Thread Dump Wed Jun 28 04:49:56 GMT-05:00 2017 *******************

Thread list from ThreadMXBean, 272 threads:
qtp162548417-23119
qtp162548417-23117
qtp162548417-23116
qtp162548417-23115
qtp162548417-23112
qtp162548417-23111
qtp162548417-23110
qtp162548417-23108
qtp162548417-23106
qtp162548417-23105
qtp162548417-23081
qtp162548417-23080
qtp162548417-22622
webStructureAnalysis_pool-1-thread-2859
webStructureAnalysis_pool-1-thread-2852
webStructureAnalysis_pool-1-thread-2844
parseDocument_pool-1-thread-2662
parsing + http://www.prawo.vulcan.edu.pl/przegdok ... &qplikid=2
condenseDocument_pool-1-thread-677
condenseDocument_pool-1-thread-523
RobotsTxt_pool-1-thread-367
RobotsTxt_pool-1-thread-366
RobotsTxt_pool-1-thread-365
RobotsTxt_pool-1-thread-364
RobotsTxt_pool-1-thread-363
RobotsTxt_pool-1-thread-354
RobotsTxt_pool-1-thread-353
RobotsTxt_pool-1-thread-351
RobotsTxt_pool-1-thread-349
RobotsTxt_pool-1-thread-347
RobotsTxt_pool-1-thread-345
RobotsTxt_pool-1-thread-344
RobotsTxt_pool-1-thread-343
RobotsTxt_pool-1-thread-341
RobotsTxt_pool-1-thread-340
RobotsTxt_pool-1-thread-339
RobotsTxt_pool-1-thread-338
RobotsTxt_pool-1-thread-337
RobotsTxt_pool-1-thread-336
RobotsTxt_pool-1-thread-335
RobotsTxt_pool-1-thread-334
RobotsTxt_pool-1-thread-333
RobotsTxt_pool-1-thread-332
RobotsTxt_pool-1-thread-331
RobotsTxt_pool-1-thread-330
RobotsTxt_pool-1-thread-329
RobotsTxt_pool-1-thread-328
RobotsTxt_pool-1-thread-327
RobotsTxt_pool-1-thread-326
RobotsTxt_pool-1-thread-325
RobotsTxt_pool-1-thread-323
RobotsTxt_pool-1-thread-322
RobotsTxt_pool-1-thread-321
RobotsTxt_pool-1-thread-319
RobotsTxt_pool-1-thread-318
RobotsTxt_pool-1-thread-317
RobotsTxt_pool-1-thread-316
RobotsTxt_pool-1-thread-315
RobotsTxt_pool-1-thread-314
RobotsTxt_pool-1-thread-313
RobotsTxt_pool-1-thread-312
RobotsTxt_pool-1-thread-311
RobotsTxt_pool-1-thread-310
RobotsTxt_pool-1-thread-309
RobotsTxt_pool-1-thread-308
RobotsTxt_pool-1-thread-301
RobotsTxt_pool-1-thread-300
RobotsTxt_pool-1-thread-299
RobotsTxt_pool-1-thread-298
RobotsTxt_pool-1-thread-295
RobotsTxt_pool-1-thread-294
RobotsTxt_pool-1-thread-293
RobotsTxt_pool-1-thread-292
RobotsTxt_pool-1-thread-289
RobotsTxt_pool-1-thread-288
RobotsTxt_pool-1-thread-287
RobotsTxt_pool-1-thread-286
RobotsTxt_pool-1-thread-285
RobotsTxt_pool-1-thread-284
RobotsTxt_pool-1-thread-283
RobotsTxt_pool-1-thread-282
RobotsTxt_pool-1-thread-281
RobotsTxt_pool-1-thread-279
RobotsTxt_pool-1-thread-278
RobotsTxt_pool-1-thread-277
RobotsTxt_pool-1-thread-275
RobotsTxt_pool-1-thread-272
RobotsTxt_pool-1-thread-269
RobotsTxt_pool-1-thread-268
RobotsTxt_pool-1-thread-267
RobotsTxt_pool-1-thread-266
RobotsTxt_pool-1-thread-265
RobotsTxt_pool-1-thread-262
RobotsTxt_pool-1-thread-256
RobotsTxt_pool-1-thread-254
RobotsTxt_pool-1-thread-252
RobotsTxt_pool-1-thread-251
RobotsTxt_pool-1-thread-248
RobotsTxt_pool-1-thread-247
RobotsTxt_pool-1-thread-244
RobotsTxt_pool-1-thread-242
RobotsTxt_pool-1-thread-241
RobotsTxt_pool-1-thread-239
RobotsTxt_pool-1-thread-236
RobotsTxt_pool-1-thread-234
RobotsTxt_pool-1-thread-233
RobotsTxt_pool-1-thread-232
RobotsTxt_pool-1-thread-231
RobotsTxt_pool-1-thread-230
RobotsTxt_pool-1-thread-229
RobotsTxt_pool-1-thread-228
RobotsTxt_pool-1-thread-225
RobotsTxt_pool-1-thread-224
RobotsTxt_pool-1-thread-223
RobotsTxt_pool-1-thread-222
RobotsTxt_pool-1-thread-221
RobotsTxt_pool-1-thread-219
RobotsTxt_pool-1-thread-216
RobotsTxt_pool-1-thread-214
RobotsTxt_pool-1-thread-212
RobotsTxt_pool-1-thread-211
RobotsTxt_pool-1-thread-209
RobotsTxt_pool-1-thread-207
RobotsTxt_pool-1-thread-205
RobotsTxt_pool-1-thread-203
RobotsTxt_pool-1-thread-201
RobotsTxt_pool-1-thread-199
RobotsTxt_pool-1-thread-196
RobotsTxt_pool-1-thread-195
RobotsTxt_pool-1-thread-193
RobotsTxt_pool-1-thread-191
RobotsTxt_pool-1-thread-190
RobotsTxt_pool-1-thread-189
RobotsTxt_pool-1-thread-188
RobotsTxt_pool-1-thread-187
RobotsTxt_pool-1-thread-186
RobotsTxt_pool-1-thread-185
RobotsTxt_pool-1-thread-183
RobotsTxt_pool-1-thread-181
RobotsTxt_pool-1-thread-179
RobotsTxt_pool-1-thread-178
RobotsTxt_pool-1-thread-176
RobotsTxt_pool-1-thread-173
RobotsTxt_pool-1-thread-172
RobotsTxt_pool-1-thread-168
RobotsTxt_pool-1-thread-167
RobotsTxt_pool-1-thread-166
RobotsTxt_pool-1-thread-164
RobotsTxt_pool-1-thread-161
RobotsTxt_pool-1-thread-159
RobotsTxt_pool-1-thread-158
RobotsTxt_pool-1-thread-155
RobotsTxt_pool-1-thread-154
RobotsTxt_pool-1-thread-152
RobotsTxt_pool-1-thread-150
RobotsTxt_pool-1-thread-148
RobotsTxt_pool-1-thread-146
RobotsTxt_pool-1-thread-144
RobotsTxt_pool-1-thread-143
RobotsTxt_pool-1-thread-142
RobotsTxt_pool-1-thread-141
RobotsTxt_pool-1-thread-140
RobotsTxt_pool-1-thread-139
RobotsTxt_pool-1-thread-138
RobotsTxt_pool-1-thread-137
RobotsTxt_pool-1-thread-136
RobotsTxt_pool-1-thread-135
RobotsTxt_pool-1-thread-134
RobotsTxt_pool-1-thread-133
RobotsTxt_pool-1-thread-132
RobotsTxt_pool-1-thread-131
RobotsTxt_pool-1-thread-130
RobotsTxt_pool-1-thread-129
RobotsTxt_pool-1-thread-128
RobotsTxt_pool-1-thread-111
RobotsTxt_pool-1-thread-110
RobotsTxt_pool-1-thread-109
RobotsTxt_pool-1-thread-108
RobotsTxt_pool-1-thread-107
RobotsTxt_pool-1-thread-102
RobotsTxt_pool-1-thread-101
RobotsTxt_pool-1-thread-99
RobotsTxt_pool-1-thread-97
RobotsTxt_pool-1-thread-94
RobotsTxt_pool-1-thread-92
RobotsTxt_pool-1-thread-91
RobotsTxt_pool-1-thread-89
RobotsTxt_pool-1-thread-87
RobotsTxt_pool-1-thread-86
RobotsTxt_pool-1-thread-85
RobotsTxt_pool-1-thread-84
RobotsTxt_pool-1-thread-83
RobotsTxt_pool-1-thread-82
RobotsTxt_pool-1-thread-81
RobotsTxt_pool-1-thread-80
RobotsTxt_pool-1-thread-78
RobotsTxt_pool-1-thread-77
RobotsTxt_pool-1-thread-75
webStructureAnalysis_pool-1-thread-73
RobotsTxt_pool-1-thread-72
RobotsTxt_pool-1-thread-70
RobotsTxt_pool-1-thread-68
condenseDocument_pool-1-thread-67
RobotsTxt_pool-1-thread-66
RobotsTxt_pool-1-thread-65
RobotsTxt_pool-1-thread-64
parseDocument_pool-1-thread-63
RobotsTxt_pool-1-thread-60
RobotsTxt_pool-1-thread-59
RobotsTxt_pool-1-thread-58
RobotsTxt_pool-1-thread-57
RobotsTxt_pool-1-thread-56
RobotsTxt_pool-1-thread-52
RobotsTxt_pool-1-thread-49
RobotsTxt_pool-1-thread-48
RobotsTxt_pool-1-thread-47
RobotsTxt_pool-1-thread-46
RobotsTxt_pool-1-thread-44
storeDocumentIndex_pool-1-thread-43
webStructureAnalysis_pool-1-thread-42
RobotsTxt_pool-1-thread-41
condenseDocument_pool-1-thread-40
RobotsTxt_pool-1-thread-39
job_pool-1-thread-38
job_pool-1-thread-37
job_pool-1-thread-36
job_pool-1-thread-35
job_pool-1-thread-34
job_pool-1-thread-33
job_pool-1-thread-32
job_pool-1-thread-31
parseDocument_pool-1-thread-30
RobotsTxt_pool-1-thread-28
RobotsTxt_pool-1-thread-27
RobotsTxt_pool-1-thread-25
commitScheduler-11-thread-1
Java2D Disposer
Scheduler-615475072
org.eclipse.jetty.server.session.HashSessionManager@2560c30bTimer
qtp162548417-84-acceptor-0@27c90d11-httpd:8090@f8495b0{HTTP/1.1}{0.0.0.0:8090}
qtp162548417-83-selector-ServerConnectorManager@3acf9dfc/3
qtp162548417-82-selector-ServerConnectorManager@3acf9dfc/2
qtp162548417-81-selector-ServerConnectorManager@3acf9dfc/1
qtp162548417-80-selector-ServerConnectorManager@3acf9dfc/0
HTTPClient.IdleConnectionMonitorThread
BusyThread net.yacy.contentcontrol.ContentControlFilterUpdateThread.run
BusyThread net.yacy.contentcontrol.SMWListSyncThread.run
BusyThread net.yacy.search.Switchboard.dhtTransferJob
BusyThread net.yacy.peers.Network.peerPing
BusyThread net.yacy.peers.Network.publishSeedList
Balancer waiting for koszyk.wydawnictwofronda.pl: 35 milliseconds
BusyThread net.yacy.search.Switchboard.surrogateProcess
BusyThread net.yacy.search.Switchboard.schedulerJob
BusyThread net.yacy.search.Switchboard.cleanupJob
net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector_ProcessHandler
vjmNe1BLY8uA_pool-1-thread-8
WebStructureGraph.PublicRefDNSResolvingProcess
searcherExecutor-7-thread-1
searcherExecutor-8-thread-1
Thread-10
IndexCell.FlushThread(/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/default)
IODispatcher
Array.SortJobWorker
Array.SortJobWorker
Array.SortJobWorker
Array.SortJobWorker
MemoryTracker
Log Worker
Signal Dispatcher
Finalizer
Reference Handler
main


------------

I 2017/06/28 04:46:20 BLOBArray merged citation.index.20170628094556872.blob with citation.index.20170628093630425.blob into /home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/default/citation.index.20170628094620561.blob

I 2017/06/28 04:46:20 HeapReader using a dump of the index of /home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/default/citation.index.20170628094620561.blob.

I 2017/06/28 04:46:20 HeapReader saturation of citation.index.20170628094620561.blob.580itVZmWpb3.idx: keylength = 6, vallength = 3, size = 246962, maximum saving for index-compression = 1 MB, exact saving for value-compression = 0 MB

I 2017/06/28 04:46:20 HeapWriter wrote a dump for the 246962 index entries of citation.index.20170628094620561.blob in 86 milliseconds.

I 2017/06/28 04:46:20 BLOBArray merging citation.index.20170628094556872.blob with citation.index.20170628093630425.blob

I 2017/06/28 04:46:20 IODispatcher appended merge job of files citation.index.20170628094556872.blob, citation.index.20170628093630425.blob to citation.index.20170628094620561.blob

I 2017/06/28 04:46:20 RICELL-shrink2 unmountSmallest(67108864)

I 2017/06/28 04:46:20 HeapReader close HeapFile citation.index.20170628093630425.blob; trace: net.yacy.kelondro.blob.HeapModifier.close(HeapModifier.java:82) -> net.yacy.kelondro.blob.ArrayStack.unmountBLOB(ArrayStack.java:276) -> net.yacy.kelondro.blob.ArrayStack.unmountSmallest(ArrayStack.java:344) -> net.yacy.kelondro.rwi.ReferenceContainerArray.shrinkAnySmallFiles(ReferenceContainerArray.java:411) -> net.yacy.kelondro.rwi.IndexCell.shrink(IndexCell.java:198) -> net.yacy.kelondro.rwi.IndexCell.access$900(IndexCell.java:65) -> net.yacy.kelondro.rwi.IndexCell$FlushThread.flushBuffer(IndexCell.java:172) -> net.yacy.kelondro.rwi.IndexCell$FlushThread.run(IndexCell.java:122)

I 2017/06/28 04:46:20 HeapReader close HeapFile citation.index.20170628094556872.blob; trace: net.yacy.kelondro.blob.HeapModifier.close(HeapModifier.java:82) -> net.yacy.kelondro.blob.ArrayStack.unmountBLOB(ArrayStack.java:276) -> net.yacy.kelondro.blob.ArrayStack.unmountSmallest(ArrayStack.java:343) -> net.yacy.kelondro.rwi.ReferenceContainerArray.shrinkAnySmallFiles(ReferenceContainerArray.java:411) -> net.yacy.kelondro.rwi.IndexCell.shrink(IndexCell.java:198) -> net.yacy.kelondro.rwi.IndexCell.access$900(IndexCell.java:65) -> net.yacy.kelondro.rwi.IndexCell$FlushThread.flushBuffer(IndexCell.java:172) -> net.yacy.kelondro.rwi.IndexCell$FlushThread.run(IndexCell.java:122)

I 2017/06/28 04:45:56 HeapReader using a dump of the index of /home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/default/citation.index.20170628094556872.blob.

I 2017/06/28 04:45:56 HeapReader saturation of citation.index.20170628094556872.blob.n9aegJkRth9x.idx: keylength = 3, vallength = 2, size = 684, maximum saving for index-compression = 0 MB, exact saving for value-compression = 0 MB

I 2017/06/28 04:45:56 ReferenceContainerCache finished rwi heap dump: 684 terms, 0 term/data relations in 6 milliseconds

I 2017/06/28 04:45:56 HeapWriter wrote a dump for the 684 index entries of citation.index.20170628094556872.blob in 1 milliseconds.

I 2017/06/28 04:45:56 ReferenceContainerCache creating rwi heap dump 'citation.index.20170628094556872.blob', 684 rwi's

I 2017/06/28 04:45:56 IODispatcher appended dump job for file citation.index.20170628094556872.blob

I 2017/06/28 04:37:40 org.apache.solr.core.QuerySenderListener QuerySenderListener done.

I 2017/06/28 04:37:40 org.apache.solr.core.QuerySenderListener QuerySenderListener sending requests to Searcher@7ca8bc7b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9pb(5.5.2):C327111/126322:delGen=6602) Uninverting(_ev4(5.5.2):C345348/101093:delGen=5040) Uninverting(_kur(5.5.2):C333330/97611:delGen=3659) Uninverting(_sd6(5.5.2):C347474/47765:delGen=1757) Uninverting(_usf(5.5.2):c81588/9779:delGen=510) Uninverting(_szg(5.5.2):c43661/9710:delGen=906) Uninverting(_tyl(5.5.2):c65639/9852:delGen=562) Uninverting(_vj8(5.5.2):c27359/1650:delGen=493) Uninverting(_uux(5.5.2):c8768/1095:delGen=105) Uninverting(_v0q(5.5.2):c9101/1570:delGen=129) Uninverting(_wk3(5.5.2):c4715/1140:delGen=1) Uninverting(_vsl(5.5.2):c4051/1005:delGen=1) Uninverting(_ws5(5.5.2):c10161) Uninverting(_wvh(5.5.2):c2745) Uninverting(_wyj(5.5.2):c3828) Uninverting(_x1c(5.5.2):c3994) Uninverting(_wyt(5.5.2):c198) Uninverting(_x0h(5.5.2):c548) Uninverting(_x2g(5.5.2):c597) Uninverting(_x1l(5.5.2):c156) Uninverting(_x1v(5.5.2):c389) Uninverting(_x25(5.5.2):c509) Uninverting(_x2p(5.5.2):c478) Uninverting(_x1s(5.5.2):c100) Uninverting(_x3a(5.5.2):c567) Uninverting(_x3j(5.5.2):c400) Uninverting(_x2z(5.5.2):c515) Uninverting(_x3k(5.5.2):c4) Uninverting(_x3l(5.5.2):c8)))}

I 2017/06/28 04:37:40 org.apache.solr.update.DirectUpdateHandler2 end_commit_flush

I 2017/06/28 04:37:40 org.apache.solr.core.SolrDeletionPolicy newest commit generation = 12916

I 2017/06/28 04:37:40 org.apache.solr.core.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2 commit{dir=/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_5_5/collection1/data/index,segFN=segments_9yr,generation=12915} commit{dir=/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_5_5/collection1/data/index,segFN=segments_9ys,generation=12916}

I 2017/06/28 04:37:40 org.apache.solr.update.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}

I 2017/06/28 04:37:17 org.apache.solr.core.QuerySenderListener QuerySenderListener done.

I 2017/06/28 04:37:17 org.apache.solr.core.QuerySenderListener QuerySenderListener sending requests to Searcher@14b2b174[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9pb(5.5.2):C327111/126322:delGen=6602) Uninverting(_ev4(5.5.2):C345348/101093:delGen=5040) Uninverting(_kur(5.5.2):C333330/97611:delGen=3659) Uninverting(_sd6(5.5.2):C347474/47765:delGen=1756) Uninverting(_usf(5.5.2):c81588/9779:delGen=510) Uninverting(_szg(5.5.2):c43661/9710:delGen=906) Uninverting(_tyl(5.5.2):c65639/9852:delGen=562) Uninverting(_vj8(5.5.2):c27359/1650:delGen=493) Uninverting(_uux(5.5.2):c8768/1095:delGen=105) Uninverting(_v0q(5.5.2):c9101/1570:delGen=129) Uninverting(_wk3(5.5.2):c4715/1140:delGen=1) Uninverting(_vsl(5.5.2):c4051/1005:delGen=1) Uninverting(_ws5(5.5.2):c10161) Uninverting(_wvh(5.5.2):c2745) Uninverting(_wyj(5.5.2):c3828) Uninverting(_x1c(5.5.2):c3994) Uninverting(_wyt(5.5.2):c198) Uninverting(_x0h(5.5.2):c548) Uninverting(_x2g(5.5.2):c597) Uninverting(_x1l(5.5.2):c156) Uninverting(_x1v(5.5.2):c389) Uninverting(_x25(5.5.2):c509) Uninverting(_x2p(5.5.2):c478) Uninverting(_x1s(5.5.2):c100) Uninverting(_x3a(5.5.2):c567) Uninverting(_x3j(5.5.2):c400) Uninverting(_x2z(5.5.2):c515) Uninverting(_x3k(5.5.2):c4) Uninverting(_x3l(5.5.2):c8)))}

I 2017/06/28 04:37:17 org.apache.solr.update.DirectUpdateHandler2 end_commit_flush

I 2017/06/28 04:37:17 org.apache.solr.core.SolrDeletionPolicy newest commit generation = 12915

I 2017/06/28 04:37:17 org.apache.solr.core.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2 commit{dir=/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_5_5/collection1/data/index,segFN=segments_9yq,generation=12914} commit{dir=/home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_5_5/collection1/data/index,segFN=segments_9yr,generation=12915}

W 2017/06/28 04:37:17 SWITCHBOARD Crawl job '50_localcrawl' is paused: user request in Crawler_p from 158.69.250.130

I 2017/06/28 04:37:17 Crawl Start pause=localcrawler
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: very high load >7 on crawling

Beitragvon smokingwheels » Do Jun 29, 2017 11:33 pm

Could we have a more detailed information on your setup please?
smokingwheels
 
Beiträge: 137
Registriert: Sa Aug 31, 2013 7:16 am

Re: very high load >7 on crawling

Beitragvon lucipher » Fr Jun 30, 2017 7:40 am

server 4 cores, 8 thread, 16GB ram, 12GB for yacy, 2TB hdd in raid
ubuntu 16_04_2 LTS 64bit
parser selected and active - Streaming HTML Parser
~2mln Documents in index takes 40GB
Yacy in robinson mode
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: very high load >7 on crawling

Beitragvon smokingwheels » Fr Jun 30, 2017 8:21 am

I have only hired a cloud server of lower capacity and used it short term.

Are you using Desktop or Server version of Ubuntu?
A txt or screen dump of (top) if you could post that would be useful?

As an experiment if you wish edit (/DATA/SETTINGS/yacy.config) with yacy shutdown.

javastart_Xmx=Xmx10112m
javastart_Xms=Xms2592m

If you have a recent version of Yacy running and you ask it to crawl it will consume CPU time, if the load is too much I might suggest lowering the PPM for a crawling.

I only have a 4 core but when I run a crawl the average load can sitting at or around ~8 to 12 (16 to 24 on your rig). Its Linux it can take it.
smokingwheels
 
Beiträge: 137
Registriert: Sa Aug 31, 2013 7:16 am

Re: very high load >7 on crawling

Beitragvon lucipher » Fr Jun 30, 2017 9:35 am

I have server version of linux of course. PPM is low <3000

The problem is different:
When load takes all cores on 100% - crawling and all other actions stops. Queue is clear
on logs there is only info I've send before

Whit not working crawling heavy load is not decreasing

It looks like something stuck on parsing and yacy cannot stops on this and cannot go forward

Maybe webpage visible on logs have something that parser cannot handle properly?
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: very high load >7 on crawling

Beitragvon smokingwheels » Fr Jun 30, 2017 12:54 pm

Have you tried and SEO web site tools on the sites you are trying crawl?
smokingwheels
 
Beiträge: 137
Registriert: Sa Aug 31, 2013 7:16 am

Re: very high load >7 on crawling

Beitragvon luc » Fr Jun 30, 2017 6:48 pm

Hi lucipher,
I tried to parse the URL mentioned in your logs (at http://www.prawo.vulcan.edu.pl) and I confirm that as you guessed it looks like the content makes YaCy html parser fall in a kind of endless loop. I tried with both latest sources from GitHub and 1.92/9000 release and got the same behavior.
It looks like an interesting case. I will try to figure out what is happening and check how this can be fixed.
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: very high load >7 on crawling

Beitragvon luc » Mo Jul 03, 2017 9:40 am

Solved : fix is available on latest YaCy sources on GitHub (with technical details commented), and on the Docker image.

Happy crawling!
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: very high load >7 on crawling

Beitragvon lucipher » Mo Jul 03, 2017 10:25 am

Thank you for help
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron