Some questions before try out

Discussion in English language.
Forumsregeln
You can start and continue with posts in english language in all other forums as well, but if you are looking for a forum to start a discussion in english, this is the right choice.

Some questions before try out

Beitragvon wolfenstein » Do Jun 05, 2014 6:02 am

I have a few questions before I try out Yacy;

1. Is it possible to crawl only specific website(ex. only my websites)
and don't crawl other websites
and distribute my webpage to other peers?

2. Does Yacy read "Sitemap:" in robots.txt?

3. I heard that Yacy eats up CPU power - it is safe to use Yacy on a laptop?

Thanks.
wolfenstein
 
Beiträge: 2
Registriert: Do Jun 05, 2014 5:44 am

Re: Some questions before try out

Beitragvon biolizard89 » Do Jun 05, 2014 6:56 am

wolfenstein hat geschrieben:I have a few questions before I try out Yacy;

1. Is it possible to crawl only specific website(ex. only my websites)
and don't crawl other websites
and distribute my webpage to other peers?

2. Does Yacy read "Sitemap:" in robots.txt?

3. I heard that Yacy eats up CPU power - it is safe to use Yacy on a laptop?

Thanks.


1. Yes, you can choose to only crawl certain websites, and those websites will be added to the global index.
2. Pretty sure it does, but I'm not 100% certain.
3. I've used it under a variety of circumstances on a laptop, with mixed results. Under some circumstances it works great; under other (more demanding) circumstances it can be a problem. My advice: try it, see if it works for you, if it doesn't then stop using it and file a bug report.
biolizard89
 
Beiträge: 58
Registriert: Do Jan 03, 2013 12:42 am

Re: Some questions before try out

Beitragvon wolfenstein » Do Jun 05, 2014 8:43 am

Hi, thanks for a reply. I'm trying it now, but...

My Current config:
Basic Config = Search portal for your own web pages
System Administ - Remote Proxy (optional) = HTTP Proxy is set

1. Why "Robinson mode"? (Set automatically)
System Administ - Network Configuration = Robinson mode
Should I change to P2P mode to distribute my results?

2. Can I restrict what to crawl by YaCy? (ex. "Deny *.*; Allow my.domain.com)
wolfenstein
 
Beiträge: 2
Registriert: Do Jun 05, 2014 5:44 am

Re: Some questions before try out

Beitragvon biolizard89 » Do Jun 05, 2014 9:48 am

wolfenstein hat geschrieben:Hi, thanks for a reply. I'm trying it now, but...

My Current config:
Basic Config = Search portal for your own web pages
System Administ - Remote Proxy (optional) = HTTP Proxy is set

1. Why "Robinson mode"? (Set automatically)
System Administ - Network Configuration = Robinson mode
Should I change to P2P mode to distribute my results?

2. Can I restrict what to crawl by YaCy? (ex. "Deny *.*; Allow my.domain.com)


If you want your crawls to be shared with the public index then you don't want "Search portal for your own web pages"... there should be an option somewhere to join the "freeworld" network.

YaCy will only crawl what you tell it to crawl. If you use the HTTP proxy that it provides, it will index every page you visit, but that's entirely optional (and it sounds like you don't want to do that). So don't use the proxy and you should be fine. There's a place in the admin interface where you can tell it to start crawling certain websites.
biolizard89
 
Beiträge: 58
Registriert: Do Jan 03, 2013 12:42 am


Zurück zu English

Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste

cron