Hello, I recently found an interesting repository of public APIs descriptions in OpenApi format : https://github.com/APIs-guru/api-models
First this made me think it would be a good thing to provide a description of YaCy API in a standardized format such as OpenApi (Swagger) or RAML or in the most relevant format...
I also wonder if it could be possible to make a generic advanced crawler able to request any public API, as an alternative to classical crawling. YaCy already have MediaWiki and phpBB3 specialized crawlers and RSS and OAI-PMH importers.
It would be great to have the possibility to request other APIs with little mapping work.
My basic idea would be to specify in a new YaCy screen :
- eventual required api key
- one or more resources listing/discovery services
- which result field(s) contain a public http resource to index
- or wich result field(s) contain metadata to index
- ... any other necessary information
This mapping should be exportable to be shared with other YaCy users, eventually in a specific folder in git repository.
What do you think? Feasible?