# What Is a User Agent and How It Identifies Each Bot > What a user agent is, how to read the string of a browser or Googlebot, why it does not prove who visits your store and how to check it in your logs. URL: https://clione.ai/glossary/user-agent/ Lang: en The _user agent_ is not an [AI agent](/glossary/ai-agent/) or a program you have to install: it is the line of text with which whoever requests a page from your server introduces itself. A customer's browser sends it, [Googlebot](/glossary/googlebot/) sends it and every AI company's [bot](/glossary/web-crawler/) sends it. It matters to a store for two reasons. It is **the first thing you look at to find out who visited a [product page](/glossary/product-page/)**, in the server logs or in your CDN's stats. And it is the name that the rules in [`robots.txt`](/glossary/robots-txt/) are addressed to. What it does not do is prove anything, because the one making the request writes it. ## How it works The HTTP standard, RFC 9110, defines the `User-Agent` header as information about the program that originates the request. It says a client should send it in every request and explains what servers use it for: to scope compatibility problems, to work around or tailor responses to the limitations of a particular program, and for statistics on browsers and operating systems.[^rfc9110] **The format is a list of products**, each with its name and, optionally, its version after a slash, followed by comments in parentheses, ordered from most to least significant. The standard's example is `CERN-LineMode/2.15 libwww/2.17b3`.[^rfc9110] In practice, browser strings are longer and stranger. Almost every browser starts with `Mozilla/5.0`, a fragment that today only says the browser is compatible with Mozilla and is sent for historical reasons.[^mdn-ua] The standard itself discourages using other programs' names to claim compatibility with them, because that defeats the purpose of the header.[^rfc9110] They do it anyway: that is why a bot like [OAI-SearchBot](/glossary/oai-searchbot/) sends a string that starts like Chrome on a Mac.[^openai-bots] **It says less and less about the visitor.** The RFC warns that an overly detailed string helps identify a person without their wanting it, which is known as fingerprinting.[^rfc9110] Chrome responded by reducing its own: since Chrome 101 the version arrives as `0.0.0` after the major number, since Chrome 107 the desktop string started to be trimmed and since Chrome 110 the Android one, which no longer includes the phone model but a fixed `K`. The reduction was complete in Chrome 113.[^chromium-ua-reduction] If a server needs more detail, it can ask for it with client hints, `Sec-CH-UA` headers that the browser sends only when asked.[^mdn-ua] **Bots have two names.** Google distinguishes between the string that appears in the HTTP request and the short name written in the `User-agent:` line of `robots.txt`. For Googlebot Smartphone, the string is long and changes with every Chrome version; in `robots.txt`, `Googlebot` is enough.[^google-common-crawlers] OpenAI adds a useful detail: when OAI-SearchBot or [GPTBot](/glossary/gptbot/) request `robots.txt` itself, they may include `robots.txt` in the string so those requests can be told apart in the logs.[^openai-bots] **And the name can be forged.** Google says so in its list of crawlers: the user agent string can be spoofed.[^google-common-crawlers] That is why its crawlers identify themselves in three ways at once: the `User-Agent` header, the source IP and the name returned by a reverse DNS lookup of that IP.[^google-crawlers] ## In a store Four requests to the same product page, with the strings documented by Chromium, Google and OpenAI. The store and the order are a **fictional example**; `W.X.Y.Z` is the Chrome version, which changes: | Who it says it is | User agent string | How it is checked | | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | A customer on Chrome for Android | `Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.0.0.0 Mobile Safari/537.36`[^chromium-ua-reduction] | It is not: it is a person, and the phone model no longer travels | | Googlebot Smartphone | `Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)`[^google-common-crawlers] | Reverse and forward DNS of the IP, or the ranges Google publishes[^google-crawlers] | | OAI-SearchBot | `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot`[^openai-bots] | The IP, against `openai.com/searchbot.json`[^openai-bots] | | [ChatGPT-User](/glossary/chatgpt-user/) | `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot`[^openai-bots] | The IP, against `openai.com/chatgpt-user.json`[^openai-bots] | Look at the second row: **Googlebot's string includes "Android" and "Chrome"**. A filter that looks for "Chrome" or "Android" counts it as a customer. What identifies the bot is its own name inside the string, and what proves it is that bot is the check in the last column. That tailoring the page to the _user agent_ is a legitimate way to "help" bots. Google calls _cloaking_ presenting different content to users and to search engines with the intent of manipulating rankings, and gives as an example **inserting text or keywords only when the _user agent_ requesting the page is a search engine's**. For content a search engine has trouble reading, such as content that depends on JavaScript, it points to its recommendations for making it accessible without _cloaking_.[^google-spam] ## How to check it 1. **Look at the user agent field in your logs.** On the server or the CDN, every access line carries it at the end. Separate the requests from the bots you care about by their own name, not by "Chrome" or "Mozilla". 2. **Verify bots by IP.** For Google, run the reverse and forward DNS lookup, or compare the IP with its published ranges;[^google-crawlers] OpenAI publishes a JSON file of IPs for each agent.[^openai-bots] 3. **Request your product page as someone else.** In Chrome, the Network conditions panel in developer tools lets you replace the string: it does not change how the browser works, it changes what the server hands it.[^devtools-ua] From the terminal, `curl -A "string" https://yourdomain.com/products/...` does the same. 4. **Compare what each one receives.** If the title, the price or the [structured data](/glossary/structured-data/) change depending on the string, find out what causes it before it affects the page's [indexing](/glossary/indexing/). 5. **Read bot dashboards with care.** On the free plan, Cloudflare's AI Crawl Control identifies AI crawlers by their user agent string, so it only detects the ones that identify themselves.[^cf-manage-crawlers] ## Related concept: crawler A [crawler](/glossary/web-crawler/) is a program that walks the web; the user agent is the label it presents on every request. Every crawler has one, but so does your browser, your customer's and any script that requests a URL. **Reading the user agent tells you who the visitor says it is; verifying its IP tells you whether it is.** To go a step beyond the name, some agents sign their requests with [Web Bot Auth](/glossary/web-bot-auth/), and that signature is what Cloudflare's [pay per crawl](/glossary/pay-per-crawl/), for example, requires. To keep going, the [crawling and indexing reading path](/learn/crawling-and-indexing/) orders the rest of the terms, and the Googlebot entry details how to verify Google's crawler step by step.