Guaranteed Opensearch Results ALWAYS ADDED to index, Please

Ideen und Vorschläge sind willkommen.

Guaranteed Opensearch Results ALWAYS ADDED to index, Please

Beitragvon xioc752 » Di Jan 27, 2015 2:24 pm

On /ConfigHeuristics_p.html

It is written
opensearch load external search result list from active systems below
When using this heuristic, then every search request line is used for a call to listed opensearch systems until enough results to fill the current search page are available. 20 results are taken from remote system and loaded simultanously, parsed and indexed immediately.


>>> We want and need the Opensearch results to BE INCLUDED in EVERY CASE.
We need to maximize our sources and Open Search engines are an important meta tool for us.

We recognize that this will increase the size of the results being stored considerably.
We want this, please.
How do we turn off the filter that decides to include Opensearch results ONLY if there are fewer than 20 results to display on a results page?

Clearly there is a filter that only adds Opensearch results if needed in the classic YaCy usage when the crawler sees it has insufficient results to fill the page - described here:
until enough results to fill the current search page


>>> While 20 Results from each Opensearch source is a good number for starting (ON or Off) for guaranteed inclusion,
>>> >>> we need to have the option to index unlimited results from each Opensearch source, please, and to be able to be load, parse and index the results automatically (from all selected Opeansearch background sources) each time a search is made for a specific search word or specific search string.

We believe that this should be pretty easy to do. It will lead to increased usability for professional big data usage of YaCy.

Thank you for making this possible as an option that we and probably some others -with tightly focussed searches- really need.
xioc752
 
Beiträge: 68
Registriert: Mo Jul 28, 2014 5:01 pm

Re: Guaranteed Opensearch Results ALWAYS ADDED to index, Ple

Beitragvon reger » So Mär 08, 2015 11:33 pm

Actually, the "until enough results" filter is not active (description Needs to be updated in this regard).
As you desire, every new search uses the active opensearch Systems.

To the Point of how many results.... that depends on the remote System and can be influenced by the URL Parameter (e.g. like &Count=100 or &Count={count} to use value from yacy search page.
reger
 
Beiträge: 45
Registriert: Mi Jan 02, 2013 9:23 am

Re: Guaranteed Opensearch Results ALWAYS ADDED to index, Ple

Beitragvon xioc752 » Mo Mär 09, 2015 12:18 pm

Thank you very kindly for the good news.

Regarding the 'harvest,' due to the use of YaCy for our project, we need to collect the maximum number of answers possible.
We know good sources frequently have many thousands of answers that can be generated in a local search on their sites,

You started to explain:
To the Point of how many results.... that depends on the remote System and can be influenced by the URL Parameter (e.g. like &Count=100 or &Count={count} to use value from yacy search page.


How and what/where do we adapt YaCy's approach to these OpeanSearch sources to harvest the most that each source has in its databases, please?
It seems we are getting less than we would expect from some big sources.
Many thanks
xioc752
 
Beiträge: 68
Registriert: Mo Jul 28, 2014 5:01 pm


Zurück zu Wunschliste

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste