Removing old index entries when reindexing a page

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Removing old index entries when reindexing a page

Beitragvon ktplulo » Mi Aug 20, 2014 12:24 pm

After YaCy indexed a page, can it remove the entries for that page that had existed before? I am not talking about the manual settings of a particular crawl, that delete everything about a page without caring if it is being indexed.

When YaCy does "If verification fails, delete index reference", does it delete only the search word, or also all other words that are invalid now (because it reindexes the page anyway)?
ktplulo
 
Beiträge: 18
Registriert: Do Mär 01, 2012 11:27 am

Re: Removing old index entries when reindexing a page

Beitragvon Orbiter » Do Aug 28, 2014 11:56 am

ktplulo hat geschrieben:After YaCy indexed a page, can it remove the entries for that page that had existed before? I am not talking about the manual settings of a particular crawl, that delete everything about a page without caring if it is being indexed.

'entries for that page that had existed before': this has two aspects:
- Documents in Solr are simply overwritten
- the p2p index references stay. If they cause that a web page is considered as search hit but the search word does not then appear on the page any more, the rwi entry is removed (that happens during searches). Words that are never searched stay forever.

ktplulo hat geschrieben:When YaCy does "If verification fails, delete index reference", does it delete only the search word, or also all other words that are invalid now (because it reindexes the page anyway)?

this has also two faces: the 'index reference' now means that the Solr document is deleted. But the words in the rwi index stay and point now to a document that do not exist any more. If they are for further searches found because someone searched the word, they are deleted whenever no reference can be found.
Orbiter
 
Beiträge: 5776
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast