Results randomly vanish and reappear in consecutive searches

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Results randomly vanish and reappear in consecutive searches

Beitragvon biolizard89 » So Apr 20, 2014 8:10 am

I have a Python script set to collect all YaCy results (and their ranking) for a given query, by using the JSON API. I'm grabbing results 100 at a time (which appears to be YaCy's limit), and incrementing the pagecount until the resulting JSON has zero results.

This mostly works, but I noticed that some results randomly disappear on some of the searches, and reappear when I repeat the search. I'm doing this in Stealth Mode, so misbehaving network code or remote nodes aren't suspects. I reset the Solr ranking weights to a constant set of test weights each time I do the search (and wait 5 seconds afterward), if that matters.

Is there any known issue that would cause results to randomly appear and disappear from a search for a given query? Any suggestions on how I might be able to fix this strange behavior?

Thanks.
biolizard89
 
Beiträge: 61
Registriert: Do Jan 03, 2013 12:42 am

Re: Results randomly vanish and reappear in consecutive sear

Beitragvon Orbiter » Di Apr 29, 2014 1:12 pm

YaCy has some safety functions which protect the peer from too strong remote search requests. If you made your searches from the same IP, then that should not trigger that functions. Did you made the requests from the same or different IP?
Orbiter
 
Beiträge: 5793
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Results randomly vanish and reappear in consecutive sear

Beitragvon biolizard89 » Di Apr 29, 2014 6:04 pm

Orbiter hat geschrieben:YaCy has some safety functions which protect the peer from too strong remote search requests. If you made your searches from the same IP, then that should not trigger that functions. Did you made the requests from the same or different IP?


All of these requests were made from localhost.
biolizard89
 
Beiträge: 61
Registriert: Do Jan 03, 2013 12:42 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron