How Build Public Peer in cluster

Hier finden YaCy User Hilfe wenn was nicht funktioniert oder anders funktioniert als man dachte. Bei offensichtlichen Fehlern diese bitte gleich in die Bugs (http://bugs.yacy.net) eintragen.
Forumsregeln
In diesem Forum geht es um Benutzungsprobleme und Anfragen für Hilfe. Wird dabei ein Bug identifiziert, wird der thread zur Bearbeitung in die Bug-Sektion verschoben. Wer hier also einen Thread eingestellt hat und ihn vermisst, wird ihn sicherlich in der Bug-Sektion wiederfinden.

How Build Public Peer in cluster

Beitragvon Guims » Fr Sep 13, 2013 1:55 pm

Hello,

I YaCy used for 3 months, but I would like to set up multiple server cluster.
Here is my situation:
I currently 2 Dedicated Server with YaCy mode robinson private peer.
Each index server 150 sites each

Is it be possible to set up two (or more) YaCy cluster mode?
Is there a tutorial that explains how to do?

Regards
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon Guims » So Sep 15, 2013 10:00 am

hello,

I found a solution on each server YaCy I have defined external Solr.
this is not very practical because I have to manage multiple list site crawler, but the result is centralized on a single server.
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon Guims » Mo Sep 30, 2013 12:31 pm

Up
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon Orbiter » Mo Sep 30, 2013 1:08 pm

your solution is something I would suggest now if the number of peers is small.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: How Build Public Peer in cluster

Beitragvon Guims » Mo Sep 30, 2013 8:06 pm

Thx for ur reply Orbiter,
is that it is possible robinson Fashion> Public Cluster have the following diagram?

Website<--Server-Crawl-1 <------->|
Website<--Server-Crawl-2 <------->|<--->Server-Data
Website<--Server-Crawl-3 <------->|


The Server-Data: have a List site crawler (example1.com, example2.com, example3.com,...) and store Data with external solr or solr/yacy.
The Server-Crawl-X is a pool of server in cluster.

I have try to read this http://www.yacy-websuche.de/wiki/index.php/En:Performance ( Switch to Robinson Mode ) but it does not really explain how.
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon Guims » So Okt 06, 2013 8:33 pm

Up
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon David » Mi Okt 09, 2013 10:05 pm

Do I have this right? You want to use three yacys as crawlers, and another yacy tells the other three what sites to crawl?
David
 
Beiträge: 170
Registriert: Di Mär 05, 2013 5:35 pm

Re: How Build Public Peer in cluster

Beitragvon Guims » Mo Okt 21, 2013 11:40 am

Hello David,

I now work this way.
1 solr server for data
2 YaCy servers with each list a different site crawler

what I want:
1 solr server for data
YaCy 1 server with a single list of site crawler
YaCy X server that will crawler my list

Basically I want to do to increase cluster performance / redundancy as an example ScienceNet
Guims
 
Beiträge: 40
Registriert: Mo Sep 02, 2013 5:03 pm

Re: How Build Public Peer in cluster

Beitragvon Orbiter » Di Okt 22, 2013 2:55 pm

Yes you can easily set up three YaCy without the built-in Solr and assign the same Solr to all three YaCy instances. But you cannot distribute the crawl list this way. This works only for YaCy networks and you would need to define your own YaCy network within these Peers.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main


Zurück zu Fragen und Antworten

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron