How to detect traffic from the most common search spiders with Ruby
Ruby posted over 2 years ago by christian
This snippet detects traffic from the following bots, which is enough for me:
- Google – Googlebot/2.1 ( http://www.googlebot.com/bot.html)
- Google Image – Googlebot-Image/1.0 ( http://www.googlebot.com/bot.html)
- MSN Live – msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)
- Yahoo – Mozilla/5.0 (compatible; Yahoo! Slurp;)
The code (via):
1 user_agent = request.user_agent.downcase 2 @bot = [ 'msnbot', 'yahoo! slurp','googlebot' ].detect { |bot| user_agent.include? bot }
When the Google bot visists your site the @bot string will contain ‘googlebot’.
If you need to detect more bots than these, then the user-agents.org site contains a list of various user agents for both bots and browsers.