Plugin architecture for parsers ?

Forum for developers

Plugin architecture for parsers ?

Beitragvon reger » Mo Sep 12, 2016 9:26 pm

After the 1.92 release and considering this ...
sixcooler hat geschrieben:Maybe we should disable the use the swfParser - I got some cases where it caused ContentScraper.anchors to be filled in GB-range, which may fail in many installations. I think a feature like parsing flash isn't worth scaring new user away.
(I didn't had time to fix / workaround that)


How about to adjust the parser config architecture to a plugin system, making it easy to switch parsers on and off (not dealing with all the ext and mimes).
Basically .... one parser = one plugin, easy to switch on, add more or less.
I snooped a little around looking for light weight plugin frameworks and I actually liked this on (which is actively developed) PF4J https://github.com/decebals/pf4j
(with one small thing I don't like..... external/downloaded plugins come by default in zip format :-(
The parsers would be 1a use case for this.
Any arguments to take it on the todo list ?
reger
 
Beiträge: 46
Registriert: Mi Jan 02, 2013 9:23 am

Re: Plugin architecture for parsers ?

Beitragvon luc » Mi Sep 14, 2016 7:17 am

Hi reger, to do this I am not sure we really need one more external library.

Did you consider using the extension mechanism provided by the JDK itself? It is even already used in YaCy by the generic image parser which can be extended by any ImageIO extension provided as a jar added to the classpath.
To manage plugin repository, downloads and updates we could maybe inspire from the YaCy updates mechanism.
luc
 
Beiträge: 285
Registriert: Mi Aug 26, 2015 1:04 am


Zurück zu YaCy Coding & Architecture

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast