The Adventures of "Google [Bot]" and this Forum -off topic-

[myrtas]

Whats up with Google [Bot] and this forum ?? Ive been seeing it hanging around here a lot lately

Is this normal for a forum or what ? See pictures below

1st ofApril

Eariler today

Not to mention on and off several more times today ... Excuse my end of day ramblings

[teiglin]

It means that one of Google's webcrawlers has visited the forum recently. Basically they index pages, and follow links from those pages to index more pages. That's how Google gathers the information it uses for search results. I guess it's just displayed to distinguish such bots from actual guest users?

[myrtas]

I see their crawler accessing this forum atleast twice a day. Are you telling me that google has enough resources to crawl the world wide web with its billions of websites and mine its data on a daily basis??? Seriously, they can do that ? Or are they priotizing US based servers ?

[RexRox]

This is Google, they often have recent 4chan threads on their search engine (though they never cache for legal reasons). They're the gods of search.

[teiglin]

There are heuristics for webpages that get updated more often (among other things) to keep their search up to date, so since this forum has high activity now it will get scanned more often. I don't have any hard data for you on total google web activity, but I suspect you'd be surprised at the amount of resources they spend on it.

[Atvaark]

google bot is often accompanied by his trusty friend Yahoo bot as well. i'm sure there's more bots around, but it's not like they're doing any harm. it's probably because of them that i even discovered Eien no Aselia in the first place...

[Ghaleon]

I've seen it a few times here as well, and never anywhere else. So I'm hesitant to believe that we coincidently saw it here like that. Maybe other sites don't even list it at all?