yacy 1.92 > 1.921 update problem

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

yacy 1.92 > 1.921 update problem

Beitragvon lucipher » So Jul 09, 2017 9:11 pm

I was using 1.92 and have 5 mln crawled pages
I made 1.921/9236 (using ant dist)
After update my index shows 0 pages!

but Citations (reverse link index) and Local Crawler queue are ok!

In logs I have some messages:

W 2017/07/09 15:58:11 org.apache.solr.handler.admin.LukeRequestHandler Error getting file length for [segments_1f]
java.nio.file.NoSuchFileException: /home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_6_6/collection1/data/index/segments_1f
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)


Should I copy some files manualy? How to update my 5mln index to new solr version?


After downgrade YaCy version: 1.92/9000 everything is ok
Documents
solr search api 5,003,993
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: yacy 1.92 > 1.921 update problem

Beitragvon luc » Mi Jul 12, 2017 8:13 am

Hi lucipher, the internal Solr index upgrade from YaCy 1.92/9000 is supposed to be automatic.

But I would strongly recommend you to update to the latest GitHub sources instead of 1.921/9236, as some Solr configuration issues related to Solr 6.6 have been solved in the meantime (see commits 4be4bfb and f6e8d71).

Hopefully this time it will work better. Don't hesitate to share your feedback here.
luc
 
Beiträge: 289
Registriert: Mi Aug 26, 2015 1:04 am

Re: yacy 1.92 > 1.921 update problem

Beitragvon lucipher » Mi Jul 12, 2017 2:55 pm

My steps:
1. download https://github.com/yacy/yacy_search_ser ... master.zip
2. unzip on my system
3. inside yacy_search_server-master run "and dist"

as results have

[tar] Building tar: /home/zmudzmar/yacy_search_server-master/RELEASE/yacy_v1.921_20170712_9000.tar.gz
[delete] Deleting directory /home/zmudzmar/yacy_search_server-master/RELEASE/MAIN

BUILD SUCCESSFUL
Total time: 16 seconds

4. copy this file to actual /yacy/DATA/RELEASE
5. run update from panel
6. after restart YaCy version: 1.921/9000
7. Documents=0, Citations as should be, queue as should be


something still is wrong :( or I am doing something in wrong style
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am

Re: yacy 1.92 > 1.921 update problem

Beitragvon luc » Mo Jul 17, 2017 2:15 pm

Hi lucipher,
as far as I know you did it right. I just reproduced the exact same steps you describe on a small "webportal" mode peer (only a few thousands documents in the index) and it worked fine (same number of documents in the index after the upgrade).

So I also wonder what was going wrong... What's more, the exception you reported (java.nio.file.NoSuchFileException) is only a non blocking warning from Solr (see mantis 727). Maybe there are some other exceptions more significant in your logs.

By the way, an alternative will maybe to export your index (/IndexExport_p.html page) and then import it in an upgraded peer with empty index... If you experience issues with the export feature (see for example mantis 682), an intermediate upgrade to 1.92/9199 (still using Solr 5.5) may be required.

Good luck and sorry for the delayed answer.
luc
 
Beiträge: 289
Registriert: Mi Aug 26, 2015 1:04 am

Re: yacy 1.92 > 1.921 update problem

Beitragvon luc » Mo Jul 17, 2017 2:37 pm

Just one complementary info : one step of the automated Solr upgrade performed at startup is renaming of the folder 'DATA/INDEX/webportal/SEGMENTS/solr_5_5/' to 'DATA/INDEX/webportal/SEGMENTS/solr_6_6/'.

Unfortunately, up to now if the operation failed for some reason, there was no trace in the logs.

So you could also check if this folder is still in your DATA after the upgrade, and eventually shut down your peer, rename it manually, then restart.
luc
 
Beiträge: 289
Registriert: Mi Aug 26, 2015 1:04 am

Re: yacy 1.92 > 1.921 update problem

Beitragvon lucipher » Mo Jul 17, 2017 9:30 pm

bingo!

problem was with copying 5_5 to 6_6 specially with big files, part of small files was copied ok

after manually rename everything is ok!
lucipher
 
Beiträge: 11
Registriert: Mi Jun 28, 2017 11:07 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron