web server not responding

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

web server not responding

Beitragvon nstaudt » Di Aug 31, 2010 4:08 pm

has anyone else noticed their web server stops responding occasionally? (the yacy process continues to run, the web server just doesn't respond)

(sorry for the english on the german forum, but more likely to get a response here :roll: )
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon nstaudt » Do Sep 02, 2010 6:57 am

This happens regularly now, at least once a day. I'm running windows vista (ultimate edition) on a quad-core AMD with 8Gb of ram. It's a 64 bit machine, but the jvm is 32 bit. (JRE version 6: 1.6.0_21)
Just a broadband connection. My yacy peer is version 0.96/7083 and is always turned on (even has a UPS :D ), however this problem means it's not always available. my peer address is: http://abbot.is-a-geek.net:8080 (yacy peer name is "abbot").

As I said in my last post, the yacy javaw process continues to run (and consume cpu and bandwidth), the local web server just stops running. I can't find any error messages in the server log tab at http://localhost:8080/yacy/ui/, so I'm guessing the web server thread just exited (or failed to log an error).

I would really appreciate some help on this one - I could of course just re-install yacy, but if this is a chance to help find a bug then I'd rather take that option :D
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon Low012 » Do Sep 02, 2010 9:48 am

Looking at the memory graph on the status page of your peer it seems like your peer is at the maximum of data which it can handle with the assigned amount of memeory (blue graph is always on top of the table). You could try to assign a few additional MBs (I would try 200 or 300 as a start). Maybe this helps already. Or are you already at the maximum amout which the 32bit JVM can handle?

edit: If you can't assign more RAM, you could try to decrease the value for "maximum number of words in cache" on http://localhost:8080/PerformanceQueues_p.html. (You could try 25000 as a start) This will lead to more HDD activity, but it also should free a considerable amount of RAM.
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: web server not responding

Beitragvon nstaudt » Do Sep 02, 2010 10:05 pm

Thanks for your reply, I'll give that a try :)
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon nstaudt » Fr Sep 03, 2010 3:11 pm

Well, it would seem my peer is down again (I'm in work just now so can't say much more, or restart it for that matter). I reduced the cache to 15000 words. I also removed a few of the crawls that I had running... however, I noticed that it continued to run at peak memory usage. What could be using all that memory?
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon Orbiter » Fr Sep 03, 2010 4:26 pm

I experienced something similar on my peer but it was not caused by memory but by too much connections on the server that could not get closed. I made a patch that removes the no-more-connection when too many connections are there in SVN 7096 to see if this was the cause. If yes, then we may struggle then with too many dead connections and we need to see how to get them away.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: web server not responding

Beitragvon nstaudt » Fr Sep 03, 2010 4:54 pm

I've found two error reports in my log files:

W 2010/09/03 16:44:08 FILEHANDLER Unexpected error while processing query.
Session: Session_87.139.17.238:1557#0
Query: /yacy/hello.html
Client: 87.139.17.238
Reason: null
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.http.server.HTTPDFileHandler.invokeServlet(HTTPDFileHandler.java:1202)
at de.anomic.http.server.HTTPDFileHandler.doResponse(HTTPDFileHandler.java:780)
at de.anomic.http.server.HTTPDFileHandler.doPost(HTTPDFileHandler.java:246)
at de.anomic.http.server.HTTPDemon.POST(HTTPDemon.java:543)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at de.anomic.server.serverCore$Session.listen(serverCore.java:727)
at de.anomic.server.serverCore$Session.run(serverCore.java:626)


and this one

2010/09/03 15:23:35 StackTrace null
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at net.yacy.kelondro.workflow.InstantBusyThread.job(InstantBusyThread.java:108)
at net.yacy.kelondro.workflow.AbstractBusyThread.run(AbstractBusyThread.java:161)
Caused by: java.lang.OutOfMemoryError: Java heap space
W 2010/09/03 15:23:35 StackTrace Java heap space
java.lang.OutOfMemoryError: Java heap space
E 2010/09/03 15:23:35 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'de.anomic.search.Switchboard.cleanupJob': null; target exception: Java heap space
java.lang.OutOfMemoryError: Java heap space
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon nstaudt » Fr Sep 03, 2010 5:38 pm

Here's a screen shot of my peer's memory usage - does that seem normal? (pardon the low quality, had to make it under 256Kb)

screenshot.jpg
Memory Usage
screenshot.jpg (233.55 KiB) 1746-mal betrachtet
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon nstaudt » So Sep 05, 2010 11:23 am

ok, I've just installed the 64 bit JRE on my machine. (http://download.java.net/jdk6/)

I modified the windows %PATH% environment variable to point to the 64 bit version.

This caused an error: "GC Overhead limit exceeded" in the yacy log, so modified yacystart.bat as follows:

the line that reads:

Code: Alles auswählen
set javacmd=-Xmx600m -Xms180m


changed to:

Code: Alles auswählen
set javacmd=-Xmx600m -Xms180m -XX:-UseGCOverheadLimit


Started my yacy peer and Voila, it started!! I changed my peer config to use 4000Mb of memory (couldn't when using 32 bit) and there we have it: YaCy on 64bit JRE with ~4Gb of ram available!

I'll post here if that has any problems. :D
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon Orbiter » So Sep 05, 2010 3:20 pm

this is a really good hint and I added this option in SVN 7108 in the start script.
Does anybody know what happens if this is used in Java 1.5 (because "-XX:-UseGCOverheadLimit" is not included in Java 1.5)?
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: web server not responding

Beitragvon Vega » So Sep 05, 2010 8:51 pm

@Orbiter - see http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

Use a policy that limits the proportion of the VM's time that is spent in GC before an OutOfMemory error is thrown. (Introduced in 6.)
Vega
 
Beiträge: 824
Registriert: Mi Jun 27, 2007 3:34 pm
Wohnort: Dresden

Re: web server not responding

Beitragvon nstaudt » Mo Sep 06, 2010 7:40 am

I'll leave you Java experts to determine if that's a good option to use (I haven't used Java much since university - over 10 years ago... but this project is making me think I should get back into it a bit :) On a side note, I saw SWARM is using SCALA... anyone fancy porting YaCy to Scala for its advanced features? :shock: )

Back to the topic -
My yacy peer is running still this morning (and the last few mornings it has required a restart, so this is a good sign!)
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am

Re: web server not responding

Beitragvon nstaudt » Mo Sep 06, 2010 7:43 am

Orbiter hat geschrieben:this is a really good hint and I added this option in SVN 7108 in the start script.
Does anybody know what happens if this is used in Java 1.5 (because "-XX:-UseGCOverheadLimit" is not included in Java 1.5)?



Does anybody still use Java 1.5? If yes, then maybe it would be a good thing to break it... it's up to 6.x now right? ;)
nstaudt
 
Beiträge: 73
Registriert: Fr Aug 13, 2010 10:54 am


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 3 Gäste

cron