New hardware ready to YaCy!

was weder zu YaCy noch zum Thema Suchmaschinen gehört

New hardware ready to YaCy!

Beitragvon davide » Di Jun 16, 2015 5:20 pm

I'm in the view of starting a small business and among the requirements there's a... programmable search engine w/ APIs :D

I previously relied upon Yahoo Boss API while experimenting with YaCy alongside for testing the grounds for a possible transition. Last month Yahoo triplicated its API pricing so it's become more important to properly test YaCy capabilities on suitable hardware and decide for a possible move.

For the sake of the hype, here's the hardware :mrgreen:


These are:

  • Motherboard: dual-CPU 771 Supermicro
  • RAM: 32GB ECC fully-buffered DDR2 (in picture: 24GB by the right box, 16GB mounted on MB. Final configuration will have 32GB)
  • CPU: 2 x Intel Xeon L5420, quad-core (yet to arrive)
  • Storage: 4 x 1TB, 7200 rpm (2 SATA, 2 SCSI) [2 for RAID1, 1 for backup, 1 host filesystem]

All will be assembled in a week when the Xeon's arrive.
The existing YaCy virtual machine (currently running on a smaller server) will run under OpenVZ and will be moved to this server soon as it's assembled.

The target documents count is 100M and the main sources for crawling will be Amazon, Newegg and Best Buy. The node publicly shares its indexes and supports the global community network 8-)
Beiträge: 84
Registriert: Fr Feb 15, 2013 8:03 am

Re: New hardware ready to YaCy!

Beitragvon LA_FORGE » Mo Jun 22, 2015 6:37 pm

Great hardware! Carefully selected and wisely assembled! Welcome to the community :-)
Beiträge: 559
Registriert: Sa Okt 11, 2008 5:24 pm

Re: New hardware ready to YaCy!

Beitragvon davide » Mi Jul 22, 2015 5:24 pm

Today I stopped using YaCy.

After several unfulfilled bug reports and two data loss which corrupted a 210 GB index, I'm glad I move away and repurpose my hardware for a custom crawler & indexer.
After all, today YaCy seems to be a glory of the past, mostly unmaintained, with bugs and plain missing features keeping it unsuitable for professional use, and probably it's remained nothing more than an exploit to fill up someone's resume to inflate wasted years.

For the smart one, wondering why after 11 years of development YaCy is a failure with less than 1000 users, here might lie some clues; maybe the unsuccess of YaCy was not caused by one user walking away, but by thousands of missed users sharing the same point of view as mine. But you demonstrated to not being able to interpret this. So, good luck, if possible at all.

Dependable release cycle
Proxy all crawler traffic
Language filter ineffective
Demote CMS template images
Does Amazon block YaCy?
Aggregate / Clear duplicate results
Index halved, 9M docs lost, 22,000 hours of post-process
Index a value for image proximity to keywords
Adopt English for forum language
Beiträge: 84
Registriert: Fr Feb 15, 2013 8:03 am

Zurück zu Off-Topic

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste