User-agent: * Disallow: /Datas/senat/ #Disallow: /mobile/ #Disallow: /m/ #bad-bots User-agent: Baiduspider Disallow: / User-agent: BaiDuSpider+ Disallow: / User-agent: GingerCrawler Disallow: / User-agent: nutch Disallow: / User-agent: Urlbot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: spbot Disallow: / User-agent: ScoutJet Disallow: / User-agent: 008 Disallow: / User-agent: Yandex Disallow: / User-agent: Speedy Spider Disallow: / User-agent: Accelobot Disallow: / User-agent: dotbot Disallow: / User-agent: discobot Disallow: / User-agent: AboutUsBot Disallow: / User-agent: Vocus Disallow: / User-agent: MJ12bot Disallow: / User-agent: plukkie Disallow: / User-agent: AhrefsBot Disallow: / User-agent: Avant Browser (http://www.avantbrowser.com) Disallow: / User-agent: http://ahrefs.com/robot Disallow: / User-agent: http://www.genieo.com/webfilter.html Disallow: / User-agent: ia_archiver Disallow: / User-agent: linkdex.com Disallow: / User-agent: linkdex.com/v2.0 Disallow: / User-agent: SemrushBot/0.95 Disallow: / User-agent: MJ12bot/v1.4.3 Disallow: / User-agent: Java/1.6.0_04 Disallow: / User-agent: JS-Kit URL Resolver, http://js-kit.com/ Disallow: / User-agent: JS-Kit Disallow: / User-agent: NetShelter ContentScan Disallow: / User-agent: NetShelter Disallow: / User-agent: ^NetShelter Disallow: / User-agent: Presto/2.8.119 Disallow: / User-agent: Presto Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: python-requests/1.2.0 Disallow: / User-agent: CPython/2.7.3 Disallow: / User-agent: Mozilla/5.0 Moreover/5.1 (+http://www.moreover.com; webmaster@moreover.com) Disallow: / User-agent: Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5) Disallow: / User-agent: Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm) Disallow: / User-agent: Mozilla/5.0 (X11; compatible; semantic-visions.com crawler; HTTPClient 3.1) Disallow: / User-agent: Mozilla/5.0 (compatible; YYSpider; +http://www.yunyun.com/spider.html) Disallow: / User-agent: YisouSpider Disallow: / User-agent: Mozilla/3.0 (compatible; Indy Library) Disallow: / User-agent: PagePeeker2 Disallow: / User-agent: Mozilla/4.0 (Windows 98; US) Opera 10.00 [en] Disallow: / User-agent: Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots) Disallow: / User-agent: SiteExplorer Disallow: / User-agent: Ruby Disallow: / User-agent: Abonti Disallow: / User-agent: Mozilla/5.0 (compatible; SEOkicks-Robot +http://www.seokicks.de/robot.html) Disallow: / User-agent: SEOkicks Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: BoardReader Disallow: / User-agent: FairShare-http://fairshare.cc Disallow: / User-agent: VerticalLeap-PageStatusBot/2.0(+http://www.apolloinsights.com/) Disallow: / User-agent: (compatible; aiHitBot/2.9; +https://www.aihitdata.com/about) Disallow: / User-agent: (compatible; DuckDuckGo-Favicons-Bot/1.0; +http://duckduckgo.com) Disallow: / User-agent: (compatible; MSIE 6.0; Windows NT 5.1; SV1; http://www.changedetection.com/bot.html ) Disallow: / User-agent: (compatible; OpenHoseBot/2.1; +http://www.openhose.org/bot.html) Disallow: / User-agent: LinkedIn/7.1 CFNetwork/672.0.8 Darwin/14.0.0 Disallow: / User-agent: LinkedIn/7.1 Disallow: / User-agent: ^LinkedIn/7.1 Disallow: /