Privacy and Referer http header

Forum for developers

Privacy and Referer http header

Beitragvon luc » Mo Feb 06, 2017 1:19 pm

Hi everyone,
I am sure some YaCy users once opened a browser console and had a look at the HTTP headers transmitted when opening a search result link. By the way if you do that you will see that the 'Referer' header sent to the visited website contains the full YaCy search URL including request parameters (at least with default major browsers settings, and except when your peer is requested with https and the target link is http...).
To my mind it can be quite an issue regarding privacy. Having a look at other major search engines related strategy, it looks like most of them redirect their search result links and take the opportunity to fill this header only with the search engine host name.

What about YaCy? I believe many users are not only concerned by decentralization but also privacy. So why not set that "Referer" header empty as default for P2P and web portal modes (in Intranet mode it probably makes sense to let it as it is), and add a configuration setting for those who still want to send this header filled (some rare websites may block visitors with an empty Referer header).

Technically I propose to add a "rel='noreferrer'" attribute to search results links in yacysearch.html. Eventually finer settings will become standard (see W3C Referer Policy Candidate Recommendation).

So what do you think about it? Where would you see that new configuration setting?
luc
 
Beiträge: 283
Registriert: Mi Aug 26, 2015 1:04 am

Re: Privacy and Referer http header

Beitragvon biolizard89 » Mo Feb 06, 2017 2:17 pm

luc hat geschrieben:Hi everyone,
I am sure some YaCy users once opened a browser console and had a look at the HTTP headers transmitted when opening a search result link. By the way if you do that you will see that the 'Referer' header sent to the visited website contains the full YaCy search URL including request parameters (at least with default major browsers settings, and except when your peer is requested with https and the target link is http...).
To my mind it can be quite an issue regarding privacy. Having a look at other major search engines related strategy, it looks like most of them redirect their search result links and take the opportunity to fill this header only with the search engine host name.

What about YaCy? I believe many users are not only concerned by decentralization but also privacy. So why not set that "Referer" header empty as default for P2P and web portal modes (in Intranet mode it probably makes sense to let it as it is), and add a configuration setting for those who still want to send this header filled (some rare websites may block visitors with an empty Referer header).

Technically I propose to add a "rel='noreferrer'" attribute to search results links in yacysearch.html. Eventually finer settings will become standard (see W3C Referer Policy Candidate Recommendation).

So what do you think about it? Where would you see that new configuration setting?


DuckDuckGo has some interesting documentation about various methods of hiding this private data.

DuckDuckGo's solution is to implement a redirect, which is its own privacy problem if you don't trust DuckDuckGo's server not to spy on you. However, I don't think this is really a problem for YaCy, since it's just trusting a locally running free software application that you can inspect to make sure it's not doing something sketchy.

According to a WordPress plugin's documentation, rel="noreferrer" is supported by Firefox since version 33 and Chromium since 2009. Referrer policy meta elements are supported by Firefox since version 37 and Chromium since 2011. Mozilla's documentation says that Firefox prior to version 37 incorrectly handled rel="noreferrer". This means that the Firefox ESR releases (upon which Tor Browser is based) and the ~1.5-year-old Chromium releases (upon which Replicant WebView is based) should handle both of them without trouble. (I haven't tested this myself.)

(Coincidentally, this topic came up a few days ago on #yacy on Freenode. Cool to see that you've been thinking about this topic too. :) )
biolizard89
 
Beiträge: 61
Registriert: Do Jan 03, 2013 12:42 am

Re: Privacy and Referer http header

Beitragvon luc » Di Feb 14, 2017 8:43 am

According to caniuse.com, the referrer meta tag is indeed already supported by many browsers.

It is far more flexible than the "rel=noreferrer" link attribute, and would allow to easily (without much refactoring, using the metas.template) set this policy not only for YaCy search results but for any link to external. Personnally I think I would favor this over a redirection solution.
luc
 
Beiträge: 283
Registriert: Mi Aug 26, 2015 1:04 am

Re: Privacy and Referer http header

Beitragvon luc » Di Mär 07, 2017 12:37 pm

Hello, for those interested, you can check latest YaCy sources on GitHub : now is included a new advanced settings page related to the referrer policy, available on your peer at /Settings_p.html?page=referrer.
luc
 
Beiträge: 283
Registriert: Mi Aug 26, 2015 1:04 am


Zurück zu YaCy Coding & Architecture

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast