Using yacy for create local distributed crawler

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Using yacy for create local distributed crawler

Beitragvon mr_aliagha » Mo Mai 05, 2014 9:59 pm

Hi,
I am a new user to yacy. I was wondering is there any way to configure yacy in a way that it can be installed on local cluster without any help from other yacy peers around the world? I mean I want to have my own peer list (on local network) and I need some kind of load balancer for this cluster of yacy. I dont want to give each crawler peer the static list of web sites that should be crawled! I want to give them a list of whole sites and load balancer assign them to each peer. Also is there any way that I can integrate yacy with NOSQL databases such as Hbase? If yes is there any tutorial available for doing that?
Regards.
mr_aliagha
 
Beiträge: 6
Registriert: Mo Mai 05, 2014 9:49 pm

Re: Using yacy for create local distributed crawler

Beitragvon David » Fr Mai 09, 2014 6:19 pm

On the settings page "Network Configuration", theres an option "Public Cluster". Maybe this is the right thing for you.

http://localhost:8090/ConfigNetwork_p.html

mr_aliagha hat geschrieben:Also is there any way that I can integrate yacy with NOSQL databases such as Hbase?

I have no clue.
David
 
Beiträge: 170
Registriert: Di Mär 05, 2013 5:35 pm


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: Google [Bot], YaCy [Bot] und 1 Gast

cron