Spreading YaCy on web servers

Forum for developers

Spreading YaCy on web servers

Beitragvon luc » Fr Okt 23, 2015 12:01 am

Imagine a world where each Apache HTTP server instance would also act as a YaCy peer, providing fresh index of its own domain but also parts of the distributed index...
As Apache HTTP Servers are very widespread, don't you think it would really be a BIG step forward for decentralized index and search? Reliability and responsiness of YaCy network could be greatly improved... Even if it would be one little percent of all Apache HTTP servers.
This idea is now far from reality. A first step would maybe to implement an Apache module (http://httpd.apache.org/docs/2.4/mod/) able to act as a YaCy peer? And maybe to formalize YaCy protocol to make it easier for anyone or any company to implement its own YaCy peer software, or at least to provide a minimal YaCy core module (YaCy 2 architecture?)?
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: Spreading YaCy on web servers

Beitragvon luc » Mo Okt 26, 2015 7:42 pm

I continue thinking on the idea of moving the indexing effort to each web server : for a web server, without going as far as being a YaCy node, it would be really helpful if it exposed its own index in a standard way. Then any YaCy node or any software or people could get it (fully or parts) and use it without having to recrawl the entire web server exposed content. Does anyone has information on such an eventual existing standard : url, index format...?
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: Spreading YaCy on web servers

Beitragvon luc » Mo Nov 16, 2015 9:34 am

It seems this post doesn't interest much people... Anyway, reconsidering what I said, I think now that developing a Apache http module would not be the most releveant idea. But I still think what would really make sense would be to be able to deploy YaCy as a standard war in any J2EE compliant server (Tomcat, JBoss...). For any organization already having applications deployed on this kind of server it would really be helpful if they wish to expose some website content on YaCy network. Is it this way YaCy pro section proceed? I don't find any document on the wiki or in the build.xml to help building a standard war...
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am

Re: Spreading YaCy on web servers

Beitragvon reger » So Nov 22, 2015 1:07 am

I think too, a standard war distribution would be a nice option
and tackled it once in a while.
But as it includes major rewirte of existing features, I'm not sure if it is worse while the effort.
The development is likely only feasible as separate branch with not much chance to be joined together later, splitting possibly the available development resources.

luc hat geschrieben: I don't find any document on the wiki or in the build.xml to help building a standard war...

Simply, because there is no way easy way of getting this done (at least I don't see it).

I probably would join the club to make it work, once a starter skeleton is defined. But I see other things which would benefit the main target/user group more.
But still, for spreading YaCy it would be a nice/additional option and imho, the way to go (compared to httpd module).
reger
 
Beiträge: 46
Registriert: Mi Jan 02, 2013 9:23 am

Re: Spreading YaCy on web servers

Beitragvon luc » Fr Dez 04, 2015 2:32 am

For sure it would not be a good idea to start splitting resources...
And another bad news for this idea : it appears even Solr deployment as a standard war is no more supported since Solr 5.0 (see https://cwiki.apache.org/confluence/dis ... +on+Tomcat)
luc
 
Beiträge: 300
Registriert: Mi Aug 26, 2015 1:04 am


Zurück zu YaCy Coding & Architecture

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron