Amount of RWIs no longer increasing

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Amount of RWIs no longer increasing

Beitragvon LA_FORGE » Di Feb 02, 2016 6:52 pm

Hi,

I recently discovered that on my main peer 'endeavour' the amount of the RWIs is no longer increasing. Since sixcooler helped me with the resouce observer (thank you very much) I'm able to exclude this as an issue (It's doing it's work very well now and honestly reports that 1,2 TB of space are available :-) ). I'm also doing excessive widespread webcrawls. What's the reason that the amount of RWIs is no longer increasing at this peer?

Greetings from germany

LA_FORGE

PS. I decided to start this thread in our english forum because I was too lazy to write in my native language at this moment :D
LA_FORGE
 
Beiträge: 559
Registriert: Sa Okt 11, 2008 5:24 pm

Re: Amount of RWIs no longer increasing

Beitragvon Orbiter » Do Feb 04, 2016 1:37 am

the number or RWIs can be saturated, i.e. it reached the maximum number of vocabulary of the used languge. This is a bit theoretical but can hold if the indexing domain is limited. Hower, this comment should not exclude the option that this is is a bug.
Orbiter
 
Beiträge: 5796
Registriert: Di Jun 26, 2007 10:58 pm
Wohnort: Frankfurt am Main

Re: Amount of RWIs no longer increasing

Beitragvon LA_FORGE » Sa Feb 06, 2016 9:14 am

Thanks a lot for the detailed explanation. Is there a way to rebuild all the RWIs for my index?

Code: Alles auswählen
stefan@endeavour:~# du -sh /mnt/3200gb/yacy2/DATA/INDEX/freeworld/
1.2T /mnt/3200gb/yacy2/DATA/INDEX/freeworld/


:-)
LA_FORGE
 
Beiträge: 559
Registriert: Sa Okt 11, 2008 5:24 pm

Re: Amount of RWIs no longer increasing

Beitragvon LA_FORGE » Mi Feb 10, 2016 11:32 am

Is my peer still valuable for our global network, even if the integrity of the amount of RWIs isn't corresponding with the amount of Documents in my Index / Are RWIs playing a necessary role at all when a search over DHT/P2P is done?
LA_FORGE
 
Beiträge: 559
Registriert: Sa Okt 11, 2008 5:24 pm

Re: Amount of RWIs no longer increasing

Beitragvon sixcooler » Mi Feb 10, 2016 9:56 pm

Hi LA_FORGE,

at first the count of RWIs that /Status.html tells you is just the count of RWIs of your biggest Blob.
Thats because other Blobs may include doublicates of RWIs.
Since you have a realy high count of about 22M RWIs this will be a super large Blob for your biggest Blob.
As soon as you have 2 Blobs of almost same size - these both blobs will be merged into one if the computed needed ram for that merge is available.
But even if this merge of 2 blobs of that big size will never happen (I don't remember the computation of needed RAM - that may be enormous)
your sum of all your Blob should grow till the theoretical maximum of indexed words that Orbiter mentioned.

Is your used size of DATA/INDEX/freeworld/SEGMENTS/default/ realy constant over the time?

At the end RWIs have lost its weight in getting search result since we use solr.
The main job for the RWIs is to be transfered by DHT to share your index with other peers.

cu, sixcooler.
sixcooler
 
Beiträge: 494
Registriert: Do Aug 14, 2008 5:22 pm

Re: Amount of RWIs no longer increasing

Beitragvon LA_FORGE » Do Feb 11, 2016 12:53 am

Thank you very much for the detailed explanation. Yes, the size of these

Code: Alles auswählen
-rw-r--r-- 1 yacy2 yacy2 724K Feb 11 00:30 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210233014039.blob.wlSSJAnkd9OW.idx
-rw-r--r-- 1 yacy2 yacy2    0 Feb 11 00:30 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210233014039.blob.wlSSJAnkd9OW.gap
-rw-r--r-- 1 yacy2 yacy2  17M Feb 11 00:30 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210233014039.blob
-rw-r--r-- 1 yacy2 yacy2 1.7M Feb 11 00:10 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210231011288.blob.uTAepUXSrU38.idx
-rw-r--r-- 1 yacy2 yacy2    0 Feb 11 00:10 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210231011288.blob.uTAepUXSrU38.gap
-rw-r--r-- 1 yacy2 yacy2  58M Feb 11 00:10 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210231011288.blob
-rw-r--r-- 1 yacy2 yacy2 141M Feb 10 23:57 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210215959677.blob
-rw-r--r-- 1 yacy2 yacy2 1.7G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210191956405.blob
-rw-r--r-- 1 yacy2 yacy2 5.6G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160210151156871.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160209071234302.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160209040709600.blob
-rw-r--r-- 1 yacy2 yacy2 6.6G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160209015000577.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160208072757476.blob
-rw-r--r-- 1 yacy2 yacy2 7.3G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207212536658.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207061045056.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207001704939.blob
-rw-r--r-- 1 yacy2 yacy2 4.2G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207001256382.blob
-rw-r--r-- 1 yacy2 yacy2 6.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207001213158.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160207000817020.blob
-rw-r--r-- 1 yacy2 yacy2 6.8G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160206231744144.blob
-rw-r--r-- 1 yacy2 yacy2 6.0G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160206083656389.blob
-rw-r--r-- 1 yacy2 yacy2 6.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160206011405675.blob
-rw-r--r-- 1 yacy2 yacy2 6.6G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160205131436435.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160204054928162.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:14 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160204005917322.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160203201115026.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160203152815149.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160203041407180.blob
-rw-r--r-- 1 yacy2 yacy2 7.5G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160202035646810.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160201230120154.blob
-rw-r--r-- 1 yacy2 yacy2 6.7G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160201095407044.blob
-rw-r--r-- 1 yacy2 yacy2 6.7G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160201013910251.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160130030250157.blob
-rw-r--r-- 1 yacy2 yacy2 5.2G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129220538580.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129215831877.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129215107854.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129215018757.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129213659000.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129213628208.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129211908315.blob
-rw-r--r-- 1 yacy2 yacy2 7.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129211246571.blob
-rw-r--r-- 1 yacy2 yacy2 7.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129161941171.blob
-rw-r--r-- 1 yacy2 yacy2 5.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129160439543.blob
-rw-r--r-- 1 yacy2 yacy2 7.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160129023739250.blob
-rw-r--r-- 1 yacy2 yacy2 5.2G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128231804114.blob
-rw-r--r-- 1 yacy2 yacy2 5.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128223548909.blob
-rw-r--r-- 1 yacy2 yacy2 8.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128221942207.blob
-rw-r--r-- 1 yacy2 yacy2 5.5G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128221939143.blob
-rw-r--r-- 1 yacy2 yacy2 5.5G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128221219813.blob
-rw-r--r-- 1 yacy2 yacy2 5.2G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128220502746.blob
-rw-r--r-- 1 yacy2 yacy2  20G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128213715441.blob
-rw-r--r-- 1 yacy2 yacy2 6.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128213029420.blob
-rw-r--r-- 1 yacy2 yacy2 7.5G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128211915180.blob
-rw-r--r-- 1 yacy2 yacy2  12G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128195524918.blob
-rw-r--r-- 1 yacy2 yacy2 5.2G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128194703272.blob
-rw-r--r-- 1 yacy2 yacy2  21G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128185230154.blob
-rw-r--r-- 1 yacy2 yacy2  21G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128183840887.blob
-rw-r--r-- 1 yacy2 yacy2 6.1G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128100941427.blob
-rw-r--r-- 1 yacy2 yacy2 6.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160128004912300.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160127161200873.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160127104342263.blob
-rw-r--r-- 1 yacy2 yacy2 6.2G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160127102702978.blob
-rw-r--r-- 1 yacy2 yacy2 7.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160127093512879.blob
-rw-r--r-- 1 yacy2 yacy2 6.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160126115242487.blob
-rw-r--r-- 1 yacy2 yacy2 5.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160126032256635.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160125204713682.blob
-rw-r--r-- 1 yacy2 users 6.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160125095959100.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160124205639546.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160124172934086.blob
-rw-r--r-- 1 yacy2 yacy2 6.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160124103546089.blob
-rw-r--r-- 1 yacy2 yacy2 6.8G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160123220941194.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160123104810652.blob
-rw-r--r-- 1 yacy2 yacy2 7.6G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160123072154259.blob
-rw-r--r-- 1 yacy2 yacy2 6.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160122195729633.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160122182123364.blob
-rw-r--r-- 1 yacy2 yacy2 7.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160122165351208.blob
-rw-r--r-- 1 yacy2 yacy2 7.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160122102445925.blob
-rw-r--r-- 1 yacy2 yacy2 6.5G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160121182551167.blob
-rw-r--r-- 1 yacy2 yacy2 7.2G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160120232642385.blob
-rw-r--r-- 1 yacy2 yacy2 7.3G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160120232553212.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160120232040747.blob
-rw-r--r-- 1 yacy2 yacy2 7.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160120201139795.blob
-rw-r--r-- 1 yacy2 users 6.6G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160119140618250.blob
-rw-r--r-- 1 yacy2 yacy2 6.3G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118174646102.blob
-rw-r--r-- 1 yacy2 yacy2 6.2G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118105306832.blob
-rw-r--r-- 1 yacy2 yacy2 5.9G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118064556144.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118064330997.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118064011179.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118062808625.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118062624014.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118061427983.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118061010193.blob
-rw-r--r-- 1 yacy2 yacy2 7.8G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118060530691.blob
-rw-r--r-- 1 yacy2 yacy2 7.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118055052060.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118054655982.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118053901420.blob
-rw-r--r-- 1 yacy2 yacy2 7.9G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118052803959.blob
-rw-r--r-- 1 yacy2 yacy2 8.0G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118052609803.blob
-rw-r--r-- 1 yacy2 yacy2 7.4G Feb 11 00:26 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160118051535980.blob
-rw-r--r-- 1 yacy2 yacy2 7.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160117052352080.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115161746472.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115141130920.blob
-rw-r--r-- 1 yacy2 users 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115101706851.blob
-rw-r--r-- 1 yacy2 yacy2 6.6G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115084356521.blob
-rw-r--r-- 1 yacy2 yacy2 7.5G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115084116441.blob
-rw-r--r-- 1 yacy2 yacy2 5.3G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115083734811.blob
-rw-r--r-- 1 yacy2 yacy2 7.0G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115083506366.blob
-rw-r--r-- 1 yacy2 yacy2 5.4G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115083131785.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 11 00:31 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115082814419.blob
-rw-r--r-- 1 yacy2 yacy2 6.4G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115082508876.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115081949473.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:32 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115081534043.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115081032004.blob
-rw-r--r-- 1 yacy2 yacy2 6.8G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115080656183.blob
-rw-r--r-- 1 yacy2 yacy2 7.1G Feb 10 22:42 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115080410582.blob
-rw-r--r-- 1 yacy2 yacy2 5.7G Feb 11 00:35 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115080138824.blob
-rw-r--r-- 1 yacy2 yacy2 5.8G Feb 11 00:34 /mnt/3200gb/yacy2/DATA/INDEX/freeworld/SEGMENTS/default/text.index.20160115080135764.blob


RWI blobs seemed to be really constant over the time. I've located two 21 GB chunks there and orbiter is completely right, I think the borders of the Wortschatz :-) is achieved, because 98% of my crawled sites are english and german ones. I've looked at the two peers SOUND_ENGINE and Z-Search in freeworld, since they are japanese and chinese peers, their amount of RWIs is much higher. I assume it's a correlation to their alphabet, they're having much more letters in it.

Greetings from germany ;-)

Steve
LA_FORGE
 
Beiträge: 559
Registriert: Sa Okt 11, 2008 5:24 pm


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron