Greetings, Salutations, & A Plea For Help

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Greetings, Salutations, & A Plea For Help

Beitragvon jerk » Di Apr 04, 2017 4:50 am

First off hello, I am happy to be here. I tried messing about with yacy some time ago, but gave up on it with very little effort. A bit part of my issue is that I am not much of a joiner and asking for help on forums is not something I regularly do, but I really like the project and would love to make a real go of it this time around. My primary aim is to use yacy as my goto general purpose search engine. With that in mind I have a few questions.

What are the realistic hardware requirements for this? A lot of the discussions and tutorials seem to focus on specific use cases.

Are there any simple tricks or tips to improve results and relevance for general search?

Is there any encryption or anonymity in peer searches? Is any planned? (that would be great)

Don't let the name fool you, I actually try to be kind and courteous and helpful. I can't spend long hours on forums, but I will try to reply to anything anyone posts here. I may be away from the computer so, if I don't get back right away please accept my apologies in advance and know that I will as soon as possible. Thanks for any help. Look forward to hearing from you.
jerk
 
Beiträge: 4
Registriert: Mo Apr 03, 2017 4:53 am

Re: Greetings, Salutations, & A Plea For Help

Beitragvon itchigo » Di Apr 04, 2017 9:26 pm

Hi there,
I'm pretty new to Yacy myself, so I don't think I can be much help to you. Just wanted to say: "Hello", mainly.
The forums seem awfully quit, to be honest. I am hoping for some new interest and enthusiasm in Yacy.
Greetings,
itchigo
 
Beiträge: 1
Registriert: Di Apr 04, 2017 9:17 pm

Re: Greetings, Salutations, & A Plea For Help

Beitragvon luc » Mi Apr 05, 2017 8:37 am

Hi guys, yes these times you won't be annoyed by plenty of messages on this forum...
By the way welcome! I hope you will find your way through this amazing but no so simple tool. If you want to make YaCy your main search application to my mind it requires a bit of learning time. My advice would be to experiment and play regularly with it, not expecting from it to replace immediately your usual favourite search engines.

What are the realistic hardware requirements for this? A lot of the discussions and tutorials seem to focus on specific use cases.

From my (partial) own experience, I would say the following is a good start for a small (no more than a few millions indexed documents) general purpose peer :
- 1GB RAM, a few dedicated GB space disk, 2 Ghz processor
Important : if you don't have that much disk space, you should regularly clean the index, manually or with a scheduled task (see this other thread for some more details)

Are there any simple tricks or tips to improve results and relevance for general search?

- you can struggle with ranking parameters, but another way that can work fine in many cases is to crawl the websites you are interested in, even just their index page. Of course you don't want to crawl the whole internet and that is not the idea of this project, but having in your own local index enough websites that are relevant especially for you will help. Improving the relevance of results from other peers is not an easy task.

Is there any encryption or anonymity in peer searches? Is any planned? (that would be great)

- Peer to peer communication is not exactly clear text, but basically base 64 encoded. Strong encryption between peers is something to consider, and https could do the job. But we have to be careful : I mean, does it really makes sense to encrypt communications with untrusted peers if users have no more way to analyse what is going in and out on their network? Private information retrieval techniques could also be promising, but I am not sure whether it is already accessible apart to cryptography specialists.

See you soon
luc
 
Beiträge: 284
Registriert: Mi Aug 26, 2015 1:04 am

Re: Greetings, Salutations, & A Plea For Help

Beitragvon jerk » Mi Apr 05, 2017 2:23 pm

Nice to meet you itchigo, I was thinking the same thing. It's terrible when you see a great p2p technology, die due to lack of participation. I was thinking that one good way to do this would be to get all the google-blocked urls from those "chilling effects" reports and add them to our indices. Then we could make a sort of "Here's what google is hiding from you" pitch. The issue would be making sure that those results came up the top of a search.
jerk
 
Beiträge: 4
Registriert: Mo Apr 03, 2017 4:53 am

Re: Greetings, Salutations, & A Plea For Help

Beitragvon jerk » Mi Apr 05, 2017 2:46 pm

Nice to meet you luc. I have no delusions about the difficulty or time involved in this one. I'm sure I'll always have to go back to big data for some things. Right now I'm running yacy on a vm with a single core cpu and 1 GB of RAM. I did have a crawler going at one point just to see how it would work, the system ran out of memory and crashed after one day. Works fine without the crawler though. I can probably throw it one more gig from my work station, we'll see.

Crawling sites that I like would indeed improve my index, but it seems like adding sites you already know about to your search database kind of defeats the purpose of web search anyway. Probably going to have to figure out the ranking thing

One of the things I love about yacy is that my searches do not have to leave my own local network. This means that, malware aside, I don't have to worry about any sort of MITM attack. When it leaves my LAN the data would be vulnerable. In the U.S. there is new legislation being discussed that would allow all ISPs to monitor all internet traffic and sell the information off to whomever they choose. Strong encryption would be useful in preventing this.

Anyhow, I appreciate all the advice. I will definately check out the links. Have a great day and thanks for getting back.
jerk
 
Beiträge: 4
Registriert: Mo Apr 03, 2017 4:53 am

Re: Greetings, Salutations, & A Plea For Help

Beitragvon smokingwheels » Fr Apr 21, 2017 6:17 pm

jerk hat geschrieben:Running yacy on a vm with a single core cpu and 1 GB of RAM.


https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04

If you manually Create a Swap File for VM you will find it much more reliable.
My Forum is down but just create a Text file with the command sequence that you got working, then next time its copy and paste.

Note some of the articles online beat around the bush but they are more educational now.
smokingwheels
 
Beiträge: 129
Registriert: Sa Aug 31, 2013 7:16 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 1 Gast

cron