Seite 1 von 1

yacy 1.92 > 1.921 update problem

BeitragVerfasst: So Jul 09, 2017 9:11 pm
von lucipher
I was using 1.92 and have 5 mln crawled pages
I made 1.921/9236 (using ant dist)
After update my index shows 0 pages!

but Citations (reverse link index) and Local Crawler queue are ok!

In logs I have some messages:

W 2017/07/09 15:58:11 org.apache.solr.handler.admin.LukeRequestHandler Error getting file length for [segments_1f]
java.nio.file.NoSuchFileException: /home/zmudzmar/yacy/DATA/INDEX/webportal/SEGMENTS/solr_6_6/collection1/data/index/segments_1f
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)


Should I copy some files manualy? How to update my 5mln index to new solr version?


After downgrade YaCy version: 1.92/9000 everything is ok
Documents
solr search api 5,003,993

Re: yacy 1.92 > 1.921 update problem

BeitragVerfasst: Mi Jul 12, 2017 8:13 am
von luc
Hi lucipher, the internal Solr index upgrade from YaCy 1.92/9000 is supposed to be automatic.

But I would strongly recommend you to update to the latest GitHub sources instead of 1.921/9236, as some Solr configuration issues related to Solr 6.6 have been solved in the meantime (see commits 4be4bfb and f6e8d71).

Hopefully this time it will work better. Don't hesitate to share your feedback here.

Re: yacy 1.92 > 1.921 update problem

BeitragVerfasst: Mi Jul 12, 2017 2:55 pm
von lucipher
My steps:
1. download https://github.com/yacy/yacy_search_ser ... master.zip
2. unzip on my system
3. inside yacy_search_server-master run "and dist"

as results have

[tar] Building tar: /home/zmudzmar/yacy_search_server-master/RELEASE/yacy_v1.921_20170712_9000.tar.gz
[delete] Deleting directory /home/zmudzmar/yacy_search_server-master/RELEASE/MAIN

BUILD SUCCESSFUL
Total time: 16 seconds

4. copy this file to actual /yacy/DATA/RELEASE
5. run update from panel
6. after restart YaCy version: 1.921/9000
7. Documents=0, Citations as should be, queue as should be


something still is wrong :( or I am doing something in wrong style

Re: yacy 1.92 > 1.921 update problem

BeitragVerfasst: Mo Jul 17, 2017 2:15 pm
von luc
Hi lucipher,
as far as I know you did it right. I just reproduced the exact same steps you describe on a small "webportal" mode peer (only a few thousands documents in the index) and it worked fine (same number of documents in the index after the upgrade).

So I also wonder what was going wrong... What's more, the exception you reported (java.nio.file.NoSuchFileException) is only a non blocking warning from Solr (see mantis 727). Maybe there are some other exceptions more significant in your logs.

By the way, an alternative will maybe to export your index (/IndexExport_p.html page) and then import it in an upgraded peer with empty index... If you experience issues with the export feature (see for example mantis 682), an intermediate upgrade to 1.92/9199 (still using Solr 5.5) may be required.

Good luck and sorry for the delayed answer.

Re: yacy 1.92 > 1.921 update problem

BeitragVerfasst: Mo Jul 17, 2017 2:37 pm
von luc
Just one complementary info : one step of the automated Solr upgrade performed at startup is renaming of the folder 'DATA/INDEX/webportal/SEGMENTS/solr_5_5/' to 'DATA/INDEX/webportal/SEGMENTS/solr_6_6/'.

Unfortunately, up to now if the operation failed for some reason, there was no trace in the logs.

So you could also check if this folder is still in your DATA after the upgrade, and eventually shut down your peer, rename it manually, then restart.

Re: yacy 1.92 > 1.921 update problem

BeitragVerfasst: Mo Jul 17, 2017 9:30 pm
von lucipher
bingo!

problem was with copying 5_5 to 6_6 specially with big files, part of small files was copied ok

after manually rename everything is ok!