security: what guards are there against fake contributions

Keine Scheu, hier darf alles gefragt und diskutiert werden. Das ist das Forum für YaCy-Anfänger. Hier kann man 'wo muss man klicken' fragen und sich über Grundlagen zur Suchmaschinentechnik unterhalten.
Forumsregeln
Hier werden Fragen beantwortet und wir versuchen die Probleme von YaCy-Newbies zu klären. Bitte beantwortete Fragen im YaCy-Wiki http://wiki.yacy.de dokumentieren!

security: what guards are there against fake contributions

Beitragvon hk3 » Di Nov 29, 2011 2:11 pm

Hello,

What guards are there against fake or manipulated contributions? For instance, persons engaging in the YaCi network with very fast computers returning links to commecial pages very fast on all search queries? Or serving links to malware sites?

Thanks,

HK
hk3
 
Beiträge: 5
Registriert: Di Nov 29, 2011 2:05 pm

Re: security: what guards are there against fake contributio

Beitragvon Lotus » Mi Nov 30, 2011 4:04 pm

All links received are verified before they get into the results. YaCy loads the respective links and checks whether all terms are valid for it. Additionally there is a stepwise ranking of all received results before they are displayed on the result page of the peer. (I don't know the exact process, but I think it will answer your question anyway.)
Lotus
 
Beiträge: 1699
Registriert: Mi Jun 27, 2007 3:33 pm
Wohnort: Hamburg

Re: security: what guards are there against fake contributio

Beitragvon hk3 » Mo Dez 05, 2011 2:44 pm

Thank you, Lotus.

Is there a public statement by YaCi about security? Has there been conducted a security analysis of the YaCi working model? Security is a complex issue.

Checking links that are returned by rogue participants has little value if it is done for instance all the time by the same YaCi service (recognizable by ip number).

A fast working rogue participant might very well be economically feasable if it returns only 1% fake links. Or if a self-constructed result page returns commercial information only 1% of the times it is visited.

As for the stepwise ranking, I have read somewhere that the fastest returned link will be presented as a query result?

A rogue participant machine could enter popular search queries itself and reuse high ranking results for its own goals.

I am not a security expert. I am just asking questions.

with kind regards,

HK
hk3
 
Beiträge: 5
Registriert: Di Nov 29, 2011 2:05 pm

Re: security: what guards are there against fake contributio

Beitragvon Orbiter » Mo Dez 05, 2011 3:04 pm

I made a explanation movie about that issue!
http://vimeo.com/33025433
it explains how links are verified and how spam is impossible in YaCy
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: security: what guards are there against fake contributio

Beitragvon hk3 » Di Dez 06, 2011 11:32 am

thank you Orbiter. I watched the video.

As far as security is concerned I see only this function/process step: before a search result is returned the link is/might be checked to see whether the search words still exist in the returned webpage.

If that is really all for the security model, I personally find it very poor.

The only thing you know is that for that http request at that moment a page was returned with the search words in it. That's all. You don't check whether the search words are visible in the page. You don't know whether the page is still similar to the cached page. You don't seem to be checking whether malware is in the page or is referenced by the page.

For a page serving website it might be very easy to detect whether a request comes from a yaci server, for instance by calling on the yaci peer port of the requester, or by other means (joining in the network and receiving a list of peers?).

As I mentioned in my former reaction, the model does not seem to take into account that malicious peers will join the network, who hide their evil carefully or make it unobtrusive.

For a page serving website it is very annoying as all the yaci peers who join in a search query will collect the targeted page and that for each similar search request... After all the yaci peers have checked the page, than comes the page request from the original searcher, if he chooses this link from the search result. If we presume that for each request an average of 3 yaci peers joins in, and the searchers chooses 1 in 10 search results, that means that pages are request 31 times, for 1 deliberate request by a interested individual.

I am just commenting on the security model and not on the good intentions of the whole project.

with kind regards,

HK
hk3
 
Beiträge: 5
Registriert: Di Nov 29, 2011 2:05 pm

Re: security: what guards are there against fake contributio

Beitragvon Lotus » Di Dez 06, 2011 12:42 pm

hk3 hat geschrieben:The only thing you know is that for that http request at that moment a page was returned with the search words in it. That's all. You don't check whether the search words are visible in the page. You don't know whether the page is still similar to the cached page. You don't seem to be checking whether malware is in the page or is referenced by the page.

A very popular search engine neither does check for visibility. I use it myself to fool it.

For a page serving website it might be very easy to detect whether a request comes from a yaci server, for instance by calling on the yaci peer port of the requester, or by other means (joining in the network and receiving a list of peers?).

Also valid for other search engines and usually done on the web.

For a page serving website it is very annoying as all the yaci peers who join in a search query will collect the targeted page and that for each similar search request... After all the yaci peers have checked the page, than comes the page request from the original searcher, if he chooses this link from the search result. If we presume that for each request an average of 3 yaci peers joins in, and the searchers chooses 1 in 10 search results, that means that pages are request 31 times, for 1 deliberate request by a interested individual.

in sum minimum 1 request (chosen result), maximum 2 requests (verified result) is true for this scenario.
Lotus
 
Beiträge: 1699
Registriert: Mi Jun 27, 2007 3:33 pm
Wohnort: Hamburg

Re: security: what guards are there against fake contributio

Beitragvon hk3 » Mi Dez 07, 2011 10:36 am

Thank you, Lotus, for the corrections.

2:1 for useful requests might still be very annoying. Especially when some of the pages require a lot of processing to provide the data. I was for some years webmaster of a website that was treated like that by a redirecting service.

you are right that yaci does not have to be more secure than is generally accepted for search engines.

may I conclude that my point still stands that you have not taken into account the possibility of malicious peers joining in?

regards,

HK
hk3
 
Beiträge: 5
Registriert: Di Nov 29, 2011 2:05 pm

Re: security: what guards are there against fake contributio

Beitragvon Orbiter » Mi Dez 07, 2011 10:46 am

hk3 hat geschrieben:may I conclude that my point still stands that you have not taken into account the possibility of malicious peers joining in?

Yes and no: This is a difficult question because who shall distribute the information that someone is malicious? Send out a message like "please ignore peer xxx" is like doing a censoring. Therefore we do the following:

- in the network configuration file is a property which can be used to exclude hosts from the network. The network configuration file is like a 'central organisation point' of the network.
- the network configuration file can be hosted as central service for a network, but in case of the YaCy freeworld network it is part of the distribution. It is like a distributed central point of network configuration. At this point it cannot exclude anybody from the network
- the file can be found at defaults/yacy.network.freeworld.unit, see the property network.unit.access.blacklist

.. thats the 'no': it is possible, but we don't do it since it would be censoring

but it is also a 'yes' because YOU and everybody can start a new network with a clone of this network configuration file and you can put in any malicious host. You can even do this in your own freeworld configuration file.

This is actually done in the 'sciencenet' YaCy network of the Karlsruhe Institut for Technology.
Orbiter
 
Beiträge: 5792
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: security: what guards are there against fake contributio

Beitragvon hk3 » Fr Dez 09, 2011 10:39 am

so there is more to say about security...

I feel like leaving the subject here, since I am not really involved. Thanks for answering my question!
hk3
 
Beiträge: 5
Registriert: Di Nov 29, 2011 2:05 pm


Zurück zu Hilfe für Einsteiger und Anwender

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast