An agent that fetches a URL it has never seen is trusting a domain it knows nothing about. Before it scrapes a page, summarizes a feed, or files a link into a report, it can ask three cheap questions: What does this server claim about itself? Who registered this domain, and when? Is there a machine-readable feed here?
Three free endpoints in the Agent Intelligence category answer those questions as structured JSON — no API key, no signup, no HTML parsing:
| Endpoint | Query params | Returns |
|---|---|---|
GET /api/headers/check | url | Status, all response headers, a security-header score, and a curated "important" subset |
GET /api/whois/lookup | domain | Registrar, creation/expiry dates, name servers, status flags, DNSSEC |
GET /api/rss/parse | url, limit | Feed metadata plus items (title, link, date, creator, categories, summary) |
All three are plain GET requests you can call from curl, Python, n8n, or a shell script in a cron job. They run against public URLs only — the platform resolves the hostname first and refuses loopback, private, link-local, and multicast targets, so an agent can't be talked into probing your internal network.
Headers tell you how a site is configured without reading a byte of its content. The endpoint makes a single request, collects the response headers, and stops reading the body after the first kilobyte, so it stays fast on heavy pages. It does not follow redirects — a 301 or 302 comes back as-is, which is exactly what you want when you are auditing where a link lands:
curl -s "https://agentmediatools.com/api/headers/check?url=https://example.com"
A trimmed response looks like this:
{
"success": true,
"url": "https://example.com/",
"elapsed_ms": 143,
"status": 200,
"status_text": "OK",
"security": {
"level": "good",
"score": 4,
"max_score": 8,
"checks": {
"hsts": true,
"content_security_policy": false,
"x_frame_options": true,
"x_content_type_options": true,
"referrer_policy": false,
"permissions_policy": false,
"cors_origin": null,
"cors_methods": null
}
},
"important": {
"server": "ECS (dcb/7F84)",
"content_type": "text/html; charset=UTF-8",
"cache_control": "max-age=86000",
"etag": "\"84238dfc8092e5d9c0dac8ef93371a07:1736799080.121134\"",
"location": null,
"set_cookie": null
}
}
The security score is deterministic, not a guess. Each check that passes adds a fixed weight — HSTS and Content-Security-Policy are worth 2 points each, and X-Frame-Options, X-Content-Type-Options, Referrer-Policy, and Permissions-Policy are worth 1 point each:
| Score | Level |
|---|---|
| 6–8 | excellent |
| 4–5 | good |
| 2–3 | fair |
| 0–1 | poor |
The important block is the part agents usually want for reasoning: location reveals a redirect the agent is about to walk into, cache_control and etag tell it whether re-fetching is wasteful, server hints at the stack, and set_cookie is a reminder that some requests create state.
From Python, the call is a single requests.get:
import requests
r = requests.get(
"https://agentmediatools.com/api/headers/check",
params={"url": "https://example.com"},
timeout=20,
)
data = r.json()
if data.get("success") and data["security"]["level"] in ("poor", "fair"):
print("weak header posture:", data["security"]["score"], data["security"]["checks"])
Domain age is one of the cheapest trust signals available. A page on a domain registered three weeks ago deserves more skepticism than one on a domain that has existed for fifteen years. Pass a bare domain — protocol and path are stripped for you:
curl -s "https://agentmediatools.com/api/whois/lookup?domain=example.com"
{
"success": true,
"domain": "example.com",
"whois_server": "whois.verisign-grs.com",
"tld": "com",
"available": false,
"rate_limited": false,
"elapsed_ms": 512,
"raw_truncated": false,
"raw_length": 2187,
"parsed": {
"registrar": "RESERVED-Internet Assigned Numbers Authority",
"creation_date": "1995-08-14T04:00:00Z",
"expiry_date": "2027-08-13T04:00:00Z",
"name_servers": ["A.IANA-SERVERS.NET", "B.IANA-SERVERS.NET"],
"status": ["clientDeleteProhibited", "clientTransferProhibited"],
"dnssec": "signedDelegation"
}
}
Two boolean flags do most of the work for an agent:
available — true when the registry answers with a "no match" style reply, meaning the domain is unregistered.rate_limited — true when the registry itself throttles the query (Verisign and others do). Treat this as "unknown", not "no data" — retry later instead of caching a false negative.The endpoint maps the TLD to the right registry server (com/net, org, io, co, app, dev, ai, me, info, biz, tv, xyz, cloud, shop, uk, de, eu, ru, jp, cn and others) and falls back to IANA's referral server for anything else. Field extraction is best-effort across registry formats: registrar, creation_date, expiry_date, updated_date, name_servers, status, registrant_org, registrant_country, admin_email, tech_email, and dnssec are returned when present. Two honest boundaries: the raw WHOIS text is not returned (only parsed fields), and the raw reply is capped at 16 KB — when raw_truncated is true, fields that appear later in the record may be missing.
Feeds are the most agent-friendly part of the open web: no scraping, no selectors, no rendering. One call turns a feed into a list your model can reason over directly. limit defaults to 50 and is clamped to 200, and the sample below is trimmed to a single entry:
curl -s "https://agentmediatools.com/api/rss/parse?url=https%3A%2F%2Fnews.ycombinator.com%2Frss&limit=3"
{
"success": true,
"feed": {
"title": "Hacker News: Front Page",
"link": "https://news.ycombinator.com/",
"feed_url": "https://news.ycombinator.com/rss",
"language": "en-us",
"last_build_date": null,
"generator": null
},
"items": {
"total": 30,
"returned": 3,
"entries": [
{
"title": "Example entry title",
"link": "https://example.com/post",
"guid": "https://example.com/post",
"pub_date": "Fri, 10 Oct 2026 18:02:11 +0000",
"creator": null,
"categories": [],
"summary": "First 500 characters of the item text…"
}
]
},
"elapsed_ms": 361
}
Both RSS and Atom are handled by the same endpoint, including the common extension fields publishers use. Each entry's summary is truncated to 500 characters so a full feed won't blow up your context window, and items.total tells you how many entries the feed actually contained versus how many you asked for. Feed bodies are read up to a 1 MB cap, and a response smaller than 50 bytes is reported as an empty feed instead of being half-parsed.
The three endpoints compose into a short pre-flight check. This script decides whether a domain is worth trusting, then pulls its feed:
import datetime
import requests
API = "https://agentmediatools.com/api"
SOURCE = "https://news.ycombinator.com/"
headers = requests.get(f"{API}/headers/check", params={"url": SOURCE}, timeout=20).json()
domain = SOURCE.split("//", 1)[1].split("/", 1)[0]
whois = requests.get(f"{API}/whois/lookup", params={"domain": domain}, timeout=20).json()
age_note = "unknown"
created = (whois.get("parsed") or {}).get("creation_date")
if created:
try:
years = (datetime.datetime.now(datetime.timezone.utc)
- datetime.datetime.fromisoformat(created.replace("Z", "+00:00"))).days / 365.25
age_note = f"{years:.1f} years old"
except ValueError:
pass
print(f"{domain}: headers={headers['security']['level']} ({headers['security']['score']}/8), "
f"domain {age_note}, whois rate-limited={whois.get('rate_limited')}")
feed = requests.get(f"{API}/rss/parse", params={"url": SOURCE + "rss", "limit": 5}, timeout=25).json()
if feed.get("success"):
for item in feed["items"]["entries"]:
print("-", item["pub_date"], item["title"])
That's a complete trust-and-ingest step with no browser, no HTML parser, and no third-party dependency beyond requests.
429 with code: "ANON_DAILY_LIMIT" and links to register or log in.200 with {"success": false, "error": "…"}. Branch on success, not on the status code — and note that because the anonymous counter increments on any response below HTTP 400, a failed lookup can still consume one of your ten calls.These three are about inspection, not extraction. A few neighbours are worth knowing:
Inspection endpoints are cheap, boring, and enormously useful: they turn "the model guessed about a URL" into "the model checked, and here is the evidence." That is exactly the layer an autonomous agent should be built on.
Run the header inspector, WHOIS lookup, and feed parser in the browser first, then call the same endpoints from your agent.
Open toolbox