Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Beitragvon usern » Di Okt 06, 2015 3:53 pm

Hi.

I am running a server with 2 X AMD Opteron 2373 Quadcore 2,1Ghz (8 cores in total) and 64 GB of ram mostly dedicated to Yacy (16GB of ram for yacy) and I have been doing a few tweaks here and there to try and get it up to speed with no luck.
OS : Ubuntu 14.04 Server

Main issue right now is that I a only getting ~500 ppm whilst my CPU;s are almost maxed out, that can't be right?

Anyone out there with some suggestions of where things might have gone wrong?
usern
 
Beiträge: 13
Registriert: So Sep 23, 2012 12:33 pm

Re: Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Beitragvon davide » Mi Okt 07, 2015 9:50 am

Maybe the garbage collector?
The intro documentation for SOLR recommends to never exceed 16GB of RAM per single SOLR instance, to avoid GC maxing out the CPU. Try to reduce the VM RAM to something lower.

For comparison, during crawling I have a constant load factor of 2 on a xeon L5410 with 4GB for the VM.
davide
 
Beiträge: 84
Registriert: Fr Feb 15, 2013 8:03 am

Re: Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Beitragvon usern » Mi Okt 07, 2015 5:03 pm

Thank you for your answer.

I lowered yacys memory to 10000 instead of 16000MB and noticed that I am running low on disk so I am setting up another disk in the server, nothing to do with the high CPU though.
Gonna have to post back once I am up and running.

Wow, a load of only 2, what kind of PPM are you getting in the long run?

BTW : I am running Yacy directly on the server, not in a VM.
Zuletzt geändert von usern am Mi Okt 07, 2015 5:17 pm, insgesamt 1-mal geändert.
usern
 
Beiträge: 13
Registriert: So Sep 23, 2012 12:33 pm

Re: Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Beitragvon davide » Mi Okt 07, 2015 5:12 pm

I guess a daily mean of 180 PPM, peaking at 500 with load of 6 on 4 cores.
Running at 200 PPM the load is about 2÷2,5, with most of it being disk IO latency (CPU wait time).

EDIT: BTW, I would try to further lower your VM memory, maybe to 4 GB, just to have a valid comparison and see if the load decreases. In case, you might consider using a different java VM with better CG.

Also, check what makes up the load. It might just be IO latency.
davide
 
Beiträge: 84
Registriert: Fr Feb 15, 2013 8:03 am

Re: Slow 2 X AMD Opteron 2373 Quadcore 2,1Ghz 64GB server

Beitragvon smokingwheels » Do Okt 08, 2015 2:55 am

May help not to sure because I had a system under 50% CPU load and the crawler would pause all the time.
I had to fight with the settings below to get them to survive a restart.
Editing yacy.config and the web interface.

Go to /PerformanceQueues_p.html

Local Crawl milliseconds milliseconds kbytes load

Try increasing the load setting to see if this stabeizes the ppm.
For a single CPU core in Debian mine is set at 6.
My P4 HT 3 is set at 32.

You could change the milliseconds but you may loose your reputation for being a well behaved robot...
You could setup a few remote VM's and enable remote crawl but then everybody can use them.
smokingwheels
 
Beiträge: 137
Registriert: Sa Aug 31, 2013 7:16 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron