Has anyone ever tuned Java for YaCy.

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Has anyone ever tuned Java for YaCy.

Beitragvon smokingwheels » Fr Jul 07, 2017 11:41 pm

I started to watch a few videos on the tube and realized if you have a large machine it might need help tuning Java to suit. There are lots of command line options.

Seek professional help, I think the email address is present at the peer shutdown page.

I am experimenting with some settings and I have noticed my machine sometimes goes wild on the swap partition, I usually hit the reset on the machine and am unsure of the exact cause.
smokingwheels
 
Beiträge: 131
Registriert: Sa Aug 31, 2013 7:16 am

Re: Has anyone ever tuned Java for YaCy.

Beitragvon lucipher » Mo Jul 10, 2017 8:29 am

On my machine swapping occurs when indexing few txt files with links (2MB each - hundreds of thousands domains)
Resolution was run only one file in time
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: Has anyone ever tuned Java for YaCy.

Beitragvon lucipher » Di Jul 11, 2017 2:36 pm

have you try OS tuning?

After setting swappiness to 10 (was 60)
sudo sysctl vm.swappiness=10
my yacy is working much better
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: Has anyone ever tuned Java for YaCy.

Beitragvon smokingwheels » Di Jul 11, 2017 6:08 pm

No I have not tried that it could be something to look into thanks.

I plotted the top output overnight of java peak and the average loads of my system see https://twitter.com/smokingwheels/status/868705075778117632
There is one result where the load of java goes near 1000 on a Quad core CPU.
I have noticed sometimes my swap drive goes full on. I can see this because I am running Yacy on some old SCSI drives in Raid 0.
On my old install I could put Ubuntu 14 in a death bash loop of doom and had the average load at 140 using a quad core before the system was hung.
I have read that the average load ideally on a server should never go above 0.7 per CPU under normal running to handle the transient loads.
I found this on Ubuntu about swappiness https://sites.google.com/site/tipsandtricksforubuntu/system-tips/swappiness will have a look around as well.

I read a while ago you can also change the priority the swap files if you have more that 1 drive. https://ubuntuforums.org/showthread.php?t=1746599
smokingwheels
 
Beiträge: 131
Registriert: Sa Aug 31, 2013 7:16 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron