Cache deletion needed [SOLVED].

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Cache deletion needed [SOLVED].

Beitragvon Giorgos » Mi Mär 29, 2017 4:18 pm

Hi! :-)

I'm running out of space and I temporarily need to delete Yacy's cache.
I went to RAM/Disk -> Web Cache -> Cleanup and indeed worked.
However I can see the directory /var/lib/yacy/INDEX/ holds >1.5GB data.

Do I have to delete this as well (and how this can be done), or just leave it be?
TIA! ;)
G.
Zuletzt geändert von Giorgos am So Apr 02, 2017 12:01 am, insgesamt 1-mal geändert.
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm

Re: Cache deletion needed.

Beitragvon luc » Do Mär 30, 2017 8:15 am

Hi Giorgos,

Web Cache and Index are two different things :
- /var/lib/yacy/HTCACHE/ : the web cache that stores data downloaded by the crawler, for faster next crawls, as would do a browser cache.
- /var/lib/yacy/INDEX/ : this is the file storage for your local Solr Index and YaCy internal Reverse Word Index. These are the main data, used for searching. You should not delete manually files here, but if you are running out of space, some administration pages can help you :
- in "Index Administration" > "Index Deletion" (/IndexDeletion_p.html) you will find various options and criteria to clean your Solr Index (related folder is /var/lib/yacy/INDEX/freeworld/SEGMENTS/solr_5_5)
- in "Index Administration" > "Reverse Word Index" (/IndexControlRWIs_p.html) you can for example limit the number of references per word (related folder is /var/lib/yacy/INDEX/freeworld/SEGMENTS/default)
- more radically : in "Index Administration" > "URL Database Administration" (/IndexControlURLs_p.html) in the "Cleanup" section you can delete completely your Solr Index and/or RWI Index

Complementary notes :
- if you are regularly running out of space you can schedule these tasks using the Process Scheduler (/Table_API_p.html)
- wait a little or restart your peer to see changes on your disk usage as Solr can take some time before completing its deletion task

Have a nice day
luc
 
Beiträge: 283
Registriert: Mi Aug 26, 2015 1:04 am

Re: Cache deletion needed.

Beitragvon Giorgos » Fr Mär 31, 2017 9:47 pm

THANK YOU VERY MUCH luc, for your help!!! ;)

Just one side question:
Is it possible to move these 2 directories (HTCACHE and INDEX) to another location?
My primary partition holds everything (except swap) and it isn't a very big one.
On the other hand, I have plenty of room on another partition (which holds various bits and bytes and it's intented mostly for data storage).
So, is it possible through Yacy administration to change the location of these 2 dirs (or if this isn't possible, maybe compiling Yacy from source)?
G.
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm

Re: Cache deletion needed.

Beitragvon smokingwheels » Sa Apr 01, 2017 8:16 am

Do you HDD space elsewhere?

I need someone to Second and translate this.

1. Do a Clean Shutdown of YaCy
2. Copy to new Location
3. Start YaCy in new Location

I have run YaCy on USB's and Phone's (Via PC).

You can't do it any more with a Phone because someone revoked FAT32 USB Mass Storage Device.
smokingwheels
 
Beiträge: 126
Registriert: Sa Aug 31, 2013 7:16 am

Re: Cache deletion needed.

Beitragvon luc » Sa Apr 01, 2017 2:07 pm

As said smokingwheels copying a YaCy install (eventually compiled from sources) and its DATA folder to a new location will work.

But given your previously mentioned paths I guess you installed the Debian package, so copying the various installed files is maybe not your best option.

By the way, both HTCACHE and INDEX folders can be configured.
- HTCACHE :
- with the administration page "RAM/Disk Usage & Updates" > "Web Cache" (/ConfigHTCache_p.html) : you can modify the field "The path where the cache is stored"
- or with the generalist administration page "System Administration" > "Advanced Properties" (/ConfigProperties_p.html) : "proxyCache" property
- or by editing manually the "proxyCache" property in your /etc/yacy/yacy.conf file
- INDEX :
- with the generalist administration page "System Administration" > "Advanced Properties" (/ConfigProperties_p.html) : "indexPrimaryPath" property
- or by editing manually the "indexPrimaryPath" property in your /etc/yacy/yacy.conf file

You will need to restart YaCy after modifying these settings.

Happy customization.
luc
 
Beiträge: 283
Registriert: Mi Aug 26, 2015 1:04 am

Re: Cache deletion needed.

Beitragvon Giorgos » So Apr 02, 2017 12:01 am

luc hat geschrieben:But given your previously mentioned paths I guess you installed the Debian package

Correct! :)
I'm on Debian Jessie x64 KDE and I have installed the version from Yacy repository.
luc hat geschrieben:By the way, both HTCACHE and INDEX folders can be configured.
- HTCACHE :
- with the administration page "RAM/Disk Usage & Updates" > "Web Cache" (/ConfigHTCache_p.html) : you can modify the field "The path where the cache is stored"
- or with the generalist administration page "System Administration" > "Advanced Properties" (/ConfigProperties_p.html) : "proxyCache" property
- or by editing manually the "proxyCache" property in your /etc/yacy/yacy.conf file
- INDEX :
- with the generalist administration page "System Administration" > "Advanced Properties" (/ConfigProperties_p.html) : "indexPrimaryPath" property
- or by editing manually the "indexPrimaryPath" property in your /etc/yacy/yacy.conf file

Somehow, changing one or both directories (either from web interface or from the config file), resulting yacy unable to start.
Giving from root terminal "/etc/init.d/yacy start", does nothing.
Actually (as I could see at KSysGuard), java starts running and it autounloads from memory, after 3-4 secs.

On the other hand, compiling from source is pretty straightforward and really fast! :D
(So I'll uninstall the already installed version and delete the repository and I'll keep the compiled version).

THANKS A LOT guys for your help!!! :D
G.
Giorgos
 
Beiträge: 11
Registriert: Sa Aug 24, 2013 4:28 pm


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron