Download the PHP package hryvinskyi/magento2-bot-verification-google without Composer
On this page you can find all versions of the php package hryvinskyi/magento2-bot-verification-google. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download hryvinskyi/magento2-bot-verification-google
More information about hryvinskyi/magento2-bot-verification-google
Files in hryvinskyi/magento2-bot-verification-google
Package magento2-bot-verification-google
Short Description Google crawler addresses for Hryvinskyi_BotVerification
License MIT
Informations about the package magento2-bot-verification-google
Magento 2 Bot Verification: Google
Google's crawlers and their published address lists for
hryvinskyi/magento2-bot-verification. The package is
configuration only: one etc/di.xml.
It declares two crawlers.
| Crawler | User-Agent tokens | Checked against |
|---|---|---|
google |
Googlebot, Storebot-Google, Google-InspectionTool, GoogleOther, Google-CloudVertexBot, AdsBot-Google, Mediapartners-Google, APIs-Google, DuplexWeb-Google, Google-Safety | common-crawlers.json, special-crawlers.json, user-triggered-fetchers-google.json, user-triggered-agents.json |
google_fetcher |
FeedFetcher-Google, Google-Read-Aloud, Google-Site-Verification, GoogleProducer, Google-NotebookLM, Google-Pinpoint | user-triggered-fetchers.json, user-triggered-fetchers-google.json, user-triggered-agents.json |
The lists are published at https://developers.google.com/static/crawling/ipranges/.
Googlebot and the special-case crawlers are checked against every list Google publishes as its own infrastructure,
not only the one its documentation names for each crawler. In a production shop's traffic every real Storebot-Google
request came from user-triggered-fetchers-google.json rather than common-crawlers.json, where the documentation
places it. user-triggered-fetchers.json holds the addresses of Google Cloud customers fetching on a user's behalf, so
it vouches for the fetchers only, never for Googlebot: a scraper on a Google Cloud machine stays an impostor.
A list accuses for up to 7 days after the last stored fetch. A real Google crawler that comes from a range Google started using after that fetch is refused until the next successful refresh, every 6 hours, or for up to 7 days while fetches fail.
License
MIT
All versions of magento2-bot-verification-google with dependencies
hryvinskyi/magento2-bot-verification Version ^1.0
magento/framework Version >=103.0.0