YaCy and Arado Api-Interface

Ideen und Vorschläge sind willkommen.

YaCy and Arado Api-Interface

Beitragvon arado » Mo Nov 01, 2010 6:20 pm

Hello

http://arado.sf.net is just another c++/Qt decentral search engine for Websites.

YaCy is quite interesting to be queried by Arado,
while browsers allow to enter a keyword and display the results in a browser tab,
Arado could query the yacy node and display the results in a XML table shown in the Qt gui (and not a browser tab).

That would extend YaCy to make the Url-search saveable in the Arado Database and make them searchable.

Would it be possible, that YaCy is providing a XML interface API, so that Arado Nodes can add YacY Node-Adresses (the project could integrate that)
and the search terms query as well the entered YacY portals?

Arado uses this kind of search sting for html:

http://IP-ADRESS:29998/arado?request=recent

Code: Alles auswählen

GET:: messages go to the path '/arado', followed by one of
'?request=recent|range'::: can specify



For an Api, YaCy could generate this kind of xml ?:

<arado>
<aradourl>
<url>
http://www.linutop.com/linutop2.en.html
</url>
<keyword>
Energy efficient
</keyword>
<keyword>
Linutop is a fanless mini PC
</keyword>
<description>
Linutop 2
</description>
</aradourl>
</arado>


if the Yacy Url contains any api with "&api=DB", then the XML could contain Titel, URL and some keywords?
while the keywords could be the generated snippets with each word in one <> line.

That would allow to query some YaCy Portals with the Arado Database and not only with a browser, and Users would be able to save the URLS and re-search them.

For the other way from Arado to YaCy: Arado allows to export an XML from the database. An interface in yacy could add the URLs to YaCy to crawl them.

If there is a cooperation between both projects appreciated, please send some feedback into this thread.

Regards from Arado.
arado
 
Beiträge: 3
Registriert: Mo Nov 01, 2010 6:01 pm

Re: YaCy and Arado Api-Interface

Beitragvon Low012 » Di Nov 02, 2010 12:42 pm

Hello,

just a personal remark (not speaking for the whole project):

I have only taken a quick look at the Arado website and I have not downloaded the software yet. I hope that I understood everything right.

Wouldn't it make sense to use the OpenSearch (http://en.wikipedia.org/wiki/OpenSearch)? OpenSearch provides a well defind and widely known interface which is used by several search engines and other software and the best thing about it is that YaCy can deliver OpenSearch compliant search results already (http://www.yacy.net/API.html). ;)
Low012
 
Beiträge: 2214
Registriert: Mi Jun 27, 2007 12:11 pm

Re: YaCy and Arado Api-Interface

Beitragvon ribbon » Fr Dez 07, 2012 6:58 am

habe das mal getestet. Arado gefällt mir als Datenbank nicht schlecht, ein Export aus Yacy könnte man über den XML-RSS Export machen, doch der bricht immer bei 100.000 URLs ab. Wieso das? und warum schaffte er nicht meine 5 Mio Urls?
Das XML-RSS Formal hat noch Zeit und Permalink und anderes im XML, kann man nicht auch explizit einen Arado-Export machen? das wäre ja nur aus 5 zeilen code 3 Zeilen machen, so dass es ohne Aufsetzen eines Feeds importierbar wäre?
Ich denke es macht Sinn, auch eine SQL als Backup bedienen zu können. kann das jemand machen? würde gerne meine URLS in SQL backuppen.
ribbon
 
Beiträge: 212
Registriert: So Jan 06, 2008 4:23 pm


Zurück zu Wunschliste

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste