Preventing "super-indexing"

Ereignisse, Vorschläge und Aktionen

Preventing "super-indexing"

Beitragvon tinkerphone » Fr Okt 10, 2014 10:58 am

Hi,
does Yacy has a mechanism to prevent the indexing of the same content "too many times"? I would call it "super-indexing" if for example 100 servers index wikipedia. The index would be spread anyway between the servers. Is there a mechanism implemented which can be used to avoid the indexing of widely available content?
tinkerphone
 
Beiträge: 26
Registriert: Fr Okt 10, 2014 10:38 am

Zurück zu Mitmachen

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 3 Gäste