java.lang.OutOfMemoryError: unable to create new native t...

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

java.lang.OutOfMemoryError: unable to create new native t...

Beitragvon davide » Di Mär 17, 2015 5:07 am

Hello,

After two hours of uptime since I launched a crawler, YaCy quit, leaving the following log file.

System specs:
Code: Alles auswählen
$ uname -r
2.6.32-042stab076.8

$ java -version
java version "1.7.0_65"
OpenJDK Runtime Environment (IcedTea 2.5.1) (7u65-2.5.1-5~deb7u1)
OpenJDK Client VM (build 24.65-b04, mixed mode, sharing)

# YacY version 1.82


./DATA/LOG/yacy00.log
Code: Alles auswählen
W 2015/03/17 04:39:49 ConcurrentLog java.lang.OutOfMemoryError: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
E 2015/03/17 04:39:49 BLOCKINGTHREAD Runtime Error in serverInstantThread.job, thread 'java.lang.reflect.Method.job.1': null
I 2015/03/17 04:39:49 HostQueue opened HostQueue /usr/local/share/java/yacy/DATA/INDEX/webportal/QUEUES/CrawlerCoreStacks/www.baodagroup.com.80 with 0 urls.
E 2015/03/17 04:39:49 BLOCKINGTHREAD Internal Error in serverInstantThread.job: null
E 2015/03/17 04:39:49 BLOCKINGTHREAD shutting down thread 'java.lang.reflect.Method.job.1'
W 2015/03/17 04:39:49 ConcurrentLog java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   ... 10 more
W 2015/03/17 04:39:49 ConcurrentLog java.lang.OutOfMemoryError: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
W 2015/03/17 04:39:49 ConcurrentLog java.lang.OutOfMemoryError: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
E 2015/03/17 04:39:49 BLOCKINGTHREAD Runtime Error in serverInstantThread.job, thread 'java.lang.reflect.Method.job.1': null
I 2015/03/17 04:39:49 HostQueue opened HostQueue /usr/local/share/java/yacy/DATA/INDEX/webportal/QUEUES/CrawlerCoreStacks/www.juicedsystems.com.80 with 0 urls.
E 2015/03/17 04:39:49 BLOCKINGTHREAD Internal Error in serverInstantThread.job: null
E 2015/03/17 04:39:49 BLOCKINGTHREAD shutting down thread 'java.lang.reflect.Method.job.1'
W 2015/03/17 04:39:49 ConcurrentLog java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   ... 10 more
W 2015/03/17 04:39:49 ConcurrentLog java.lang.OutOfMemoryError: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
W 2015/03/17 04:39:49 ConcurrentLog java.lang.OutOfMemoryError: unable to create new native thread
java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
E 2015/03/17 04:39:49 BLOCKINGTHREAD Runtime Error in serverInstantThread.job, thread 'java.lang.reflect.Method.job.1': null
I 2015/03/17 04:39:49 HostQueue opened HostQueue /usr/local/share/java/yacy/DATA/INDEX/webportal/QUEUES/CrawlerCoreStacks/judaqipei.en.ec21.com.80 with 0 urls.
E 2015/03/17 04:39:49 BLOCKINGTHREAD Internal Error in serverInstantThread.job: null
E 2015/03/17 04:39:49 BLOCKINGTHREAD shutting down thread 'java.lang.reflect.Method.job.1'
W 2015/03/17 04:39:49 ConcurrentLog java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
   at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
   at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
   at java.lang.Thread.start0(Native Method)
   at java.lang.Thread.start(Thread.java:714)
   at net.yacy.crawler.robots.RobotsTxt.ensureExist(RobotsTxt.java:249)
   at net.yacy.crawler.HostBalancer.push(HostBalancer.java:226)
   at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:167)
   at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:336)
   at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
   ... 10 more


The log file continues for about another 1MB.

Is this a system limit, YaCy's, or OpenJDK's?
How can I work this around?
davide
 
Beiträge: 78
Registriert: Fr Feb 15, 2013 8:03 am

Re: java.lang.OutOfMemoryError: unable to create new native

Beitragvon sixcooler » Di Mär 17, 2015 8:21 pm

Hello Davide,

you can try to rise the amount of used Memory at /Performance_p.html

cu, sixcooler.
sixcooler
 
Beiträge: 479
Registriert: Do Aug 14, 2008 5:22 pm

Re: java.lang.OutOfMemoryError: unable to create new native

Beitragvon davide » Mi Mär 18, 2015 12:06 am

From ./DATA/SETTINGS/yacy.conf

Code: Alles auswählen
javastart_Xmx=Xmx2500m
javastart_Xms=Xms200m


These configs were already in place at the time of the crash. These should be close to the upper limit for 32 bit hardware.
davide
 
Beiträge: 78
Registriert: Fr Feb 15, 2013 8:03 am

Re: java.lang.OutOfMemoryError: unable to create new native

Beitragvon smokingwheels » Mo Apr 06, 2015 9:52 am

davide hat geschrieben:From ./DATA/SETTINGS/yacy.conf

Code: Alles auswählen
javastart_Xmx=Xmx2500m
javastart_Xms=Xms200m


These configs were already in place at the time of the crash. These should be close to the upper limit for 32 bit hardware.


Linux 32 bit should be able to have 4gb Java memory but windows uses double overhead process and it maxes out at 1.6 GB and crashes.
There is a topic some where on here I will see if I can find it.

If you are running Linux Set your crawl running and try logging the system processes eg top > system.txt with ctrl-c to end it after the yacy server crashes.

It looks like you copied a windows yacy to Linux please let us know if this is true?
Should work but would need more experienced person to let me know.

If on a Virtual machine you must enable a swap file yacy will work for a little while without one.
https://www.digitalocean.com/community/ ... e-on-a-vps
smokingwheels
 
Beiträge: 102
Registriert: Sa Aug 31, 2013 7:16 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron