recovering a directory containing crrawling data

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

recovering a directory containing crrawling data

Beitragvon jihell » Mo Mär 14, 2016 7:55 pm

Hello,

Some months before, I configured a partition dedicated for crawling data (directories from ARCHIVE to WORK). After several months, I got about 10Go data.
I reinstalled recently my Debian system and reinstalled yacy : it works, but it is crawling new data from scratch, in the system /var/lib/yacy directory. So my 10Go data are not used.

So I stopped yacy, removed this directory, created a new /var/lib/yacy directory, mounted my dedicated partition (sdb3) to it and restart yacy. But yacy does not start.

I don't understand because that worked correctly before the Debian reinstallation.

I miss probably something. Could you help ?
jihell
 
Beiträge: 6
Registriert: Mi Feb 26, 2014 11:08 am

Re: recovering a directory containing crrawling data

Beitragvon Orbiter » Mi Mär 30, 2016 9:13 am

Please have a look into your log (/var/log/yacy); is there any suspicious message (i.e. exceptions) about the startup issue?
Orbiter
 
Beiträge: 5769
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: recovering a directory containing crrawling data

Beitragvon jihell » So Jun 12, 2016 7:26 pm

Hi,
Sorry to give you a so late answer (:-(

Effectively, I have exceptions errors ; the log file returns :

2016/06/12 20:02:13 STARTUP YaCy cannot start: SolrCore 'collection1' is not available due to init failure: Error opening new searcher
org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Error opening new searcher
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1066)...

then :
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:820)...

then :
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1676)...

then :
Caused by: org.apache.lucene.index.CorruptIndexException: file mismatch, expected suffix=2tcr, got=2ti6 (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/var/lib/yacy/INDEX/freeworld/SEGMENTS/solr_5_2/collection1/data/index/segments_2tcr")))
at org.apache.lucene.codecs.CodecUtil.checkIndexHeaderSuffix(CodecUtil.java:279)
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:308)
at org.apache.lucene.index.IndexFileDeleter.<init>(IndexFileDeleter.java:171)...

I don't see the meaning of all that ; perhaps my 10 Go data ane not compatible with the yacy version ( 1.83.9857) ?
jihell
 
Beiträge: 6
Registriert: Mi Feb 26, 2014 11:08 am

Re: recovering a directory containing crrawling data

Beitragvon luc » Di Jun 14, 2016 7:44 am

Hi, to be sure it is not a Solr version issue you could try reinstalling the exact same YaCy version you used before Debian reinstall.
But to my mind, looking at your error stack and at involved Solr sources it is more probably a consistency issue with one of your index file. YaCy might have incorrectly stopped and let a Solr file in a corrupted state...
I don't think recovering your index data is desperate. Maybe can you try following a procedure such as described here : Solr fix corrupted index
Let us now if you have some success, it may be quite useful!
luc
 
Beiträge: 235
Registriert: Mi Aug 26, 2015 1:04 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron