The ping rabbit hole

It started, as these things do, with a typo. I meant to paste a hostname into ping and pasted a whole URL instead:

$ ping http://git.example.org
PING lorien.example.org (185.183.181.181) 56(84) bytes of data.
64 bytes from lorien.example.org (185.183.181.181): icmp_seq=1 ttl=51 ...

Hold on a moment. That shouldn’t work. At all. http://git.example.org is a URL, not a hostname, and ping speaks ICMP, a layer-3 protocol that has never heard of HTTP, or URLs, or ports, and wouldn’t know what to do with them if it had. Hand that string to ping on most Linux boxes and you get what you’d expect:

ping: http://git.example.org: Name or service not known

And yet there it was, merrily pinging something whose name starts with a protocol scheme. No error. Actual ICMP replies coming back.

Then it got worse. git.example.org points at a machine called curzio, at 193.43.108.75. Easy enough to confirm:

$ getent hosts git.example.org
193.43.108.75   curzio.example.org

$ dig +short git.example.org
curzio.example.org.
193.43.108.75

Everything agreed: curzio. Which left me holding two impossibilities stacked one on top of the other. A URL that resolves at all, and a URL that resolves to the wrong host: not an error, not curzio, but some mystery third answer that had no business existing.

What follows is the story of chasing both of them down, all the way to two RFCs that quietly disagree about what a “name” even is, and three separate design decisions that had to line up just so.

Wrong theory #1: “ping strips the URL scheme”

First guess, and the obvious one: ping doesn’t understand URLs, so it must be lopping off the http:// and resolving the bare hostname. Perfectly plausible, and dead on arrival the second you look at the output again. If ping had resolved the bare git.example.org, it would have got curzio, like every other tool on the machine did. It got lorien. Whatever ping was resolving, it wasn’t the clean hostname.

Wrong theory #2: stale cache

Cache is always worth suspecting. Not this time, though:

$ resolvectl query --cache=no 'http://git.example.org'
http://git.example.org: 185.183.181.181     -- link: wlan0
                           (lorien.example.org)
-- Data from: network

--cache=no, and the answer comes back marked “Data from: network”. So a real DNS server, somewhere out there, was answering lorien for this query. Nothing stale about it. Something on the wire was replying to… what name, exactly?

Wrong theory #3: “:// becomes a dot”

Next idea: maybe the resolver normalizes http://git into http.git, and there’s a record for http.git.example.org sitting in the zone. Cheap to test:

$ dig +short http.git.example.org
(nothing)

$ dig +short https.git.example.org
(nothing)

Dead end. No such records. But while I was already poking around in the zone, one idle probe changed the whole picture:

$ dig +short pippo123xyz.example.org
lorien.example.org.
185.183.181.181

A completely made-up name under example.org resolves to lorien. Well then.

The first real answer: a wildcard

The zone has a wildcard record in it:

*.example.org  →  lorien.example.org

Any name under example.org without a record of its own falls through to lorien. git.example.org has an explicit CNAME to curzio, so the exact name gives you the right answer, and only the exact name does.

Which brings us to the thing about DNS labels: they split on dots, and on nothing else whatsoever. The string http://git.example.org parses like this:

http://git . example . org
└────────┘
 ONE label

:// isn’t a separator. http://git is a single label. An odd-looking one, certainly, but as far as the DNS wire format is concerned a label is just a length-prefixed run of bytes, and it has no opinion about which bytes. That label has no record of its own, the wildcard catches it, and out comes lorien.

Two things nicely confirm it. First, any invented scheme works just as well. This was never about http:

$ resolvectl query 'qualsiasi://cosa.example.org'
→ 185.183.181.181 (lorien)

Second, RFC 4592 says an existing node “shadows” the wildcard across its whole subtree, which explains why nothing under git. ever matched:

$ dig +short foo.bar.git.example.org
(nothing: the explicit 'git' node blocks the wildcard below it)

Proving ping’s innocence with strace

Still, should that string ever have reached DNS in the first place? Time to stop guessing at what ping does internally and just watch it work:

$ strace ping -c 3 http://git.example.org
execve("/usr/bin/ping", ["ping", "-c", "3", "http://git.example.org"], ...) = 0
...
socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP) = 3
...
connect(5, {sa_family=AF_UNIX, sun_path="/run/systemd/resolve/io.systemd.Resolve"}, 42) = 0
sendto(5, "{\"method\":\"io.systemd.Resolve.Re"..., 120, ...) = 120
recvfrom(5, "{\"parameters\":{\"addresses\":[{\"if"..., ...) = 132

The trace doesn’t leave much room for argument:

1. The argument turns up verbatim in argv. There’s no URL parsing anywhere in ping, because there’s no URL parsing anywhere in ping.
2. ping calls getaddrinfo(), which walks NSS, which loads libnss_resolve, which hands the raw string straight to systemd-resolved over a socket. ping never talks DNS itself at any point.
3. resolved comes back with 185.183.181.181, and ping starts firing ICMP at it without a second thought.

So ping is an innocent bystander here. It handed an opaque string to the system resolver, and the resolver said “sure, that’s 185.183.181.181”. Case closed, then?

Not quite. A friend ran the very same command on their machine and got this:

sonne@clockwork:~$ ltrace ping https://git.example.org 2>&1 | grep getaddr
getaddrinfo("https://git.example.org", nil, ...) = -2

-2 is EAI_NONAME, “Name or service not known”. Same ping, same string, same public DNS zone with the same wildcard sitting in it. Their machine flatly refuses to resolve it. Mine happily returns lorien.

This is where the rabbit hole starts getting good.

Two resolvers, two definitions of “valid name”

What differs between the machines is the NSS backend answering getaddrinfo. Mine, an Arch box with stock config:

$ grep hosts: /etc/nsswitch.conf
hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns

First backend in the list is resolve, i.e. systemd-resolved. My friend’s machine goes through the classic glibc dns backend instead. The satisfying part is that I could reproduce both behaviours on my own machine, same network, same zone, same everything, just by forcing the backend by hand:

$ getent -s resolve hosts 'https://git.example.org'
185.183.181.181 lorien.example.org        # exit 0

$ getent -s dns hosts 'https://git.example.org'
                                              # exit 2, just like clockwork

$ getent -s dns hosts 'git.example.org'
193.43.108.75   curzio.example.org        # exit 0, clean names are fine

Same question, two backends, opposite answers. So why?

Mapping the boundary by hand

That wildcard makes a lovely little oracle. Any name that actually makes it onto the wire comes back as lorien; any name rejected before it gets there comes back with nothing. So I went through it character by character:

Note that glibc also throws out + and ~, so this was never “glibc hates URLs”. It’s a plain allowlist (letters, digits, hyphen, dot, underscore) and everything outside it dies on the spot.

And where exactly does it die? strace answers that one too:

# Clean name through glibc's dns backend:
connect(3, {sin_port=htons(53), sin_addr="1.1.1.1"}) = 0 ← real DNS query

# Dirty name through glibc's dns backend:
(no socket to port 53 at all) ← rejected in-process

glibc turns the name away locally, before it so much as opens a socket. That -2 my friend saw was manufactured inside the C library and never went anywhere near a network. systemd-resolved, meanwhile, puts the strange label on the wire without blinking, and on this particular zone the wildcard rewards it with an answer.

The two RFCs behind the split

None of this is a bug in either component. It’s two components faithfully implementing two different specs.

• RFC 952 / RFC 1123 §2.1, “hostnames”: the preferred name syntax. Letters, Digits, Hyphen. LDH. This is what glibc’s res_hnok() enforces, and the function name says it out loud: “host name OK”. The glibc resolver traces its lineage back to 1980s BIND code, and it’s built around the idea of a host name.
• RFC 2181 §11, “DNS labels”: the DNS protocol places no character restrictions on a label at all. A label is an arbitrary sequence of up to 63 octets, full stop. This is what systemd-resolved’s dns_name_is_valid() checks, and all it really checks is lengths.

Which means https://git is simultaneously an illegal hostname and a perfectly legal DNS label. Both resolvers are right, each one inside its own charter.

Why each one chose as it did

glibc is strict partly out of lineage and partly out of caution. The LDH filter historically guarded the response path: a hostile DNS server could hand back a “hostname” stuffed with shell metacharacters or control bytes, which would then go on to land in logs, .rhosts files and scripts, where they’d do real damage. Filtering down to LDH was hardening. And glibc, being the C library underpinning half the planet, changes its behaviour with enormous reluctance. The underscore only got grudgingly waved through once real-world records like _dmarc and _sip._tcp made rejecting it completely untenable.

systemd-resolved is permissive because its job leaves it no choice. It isn’t a hostname-to-address shim; it’s a full system resolver that also speaks mDNS and LLMNR. DNS-SD service names legitimately contain spaces and arbitrary UTF-8. Kitchen Speaker._raop._tcp.local is a name someone’s speaker is really called. A resolver that has to cope with those can’t enforce LDH. So the permissiveness is a requirement rather than laziness, and on top of that a deliberate stance: carry whatever the protocol allows, and let the caller decide what it means.

The two of them end up on opposite sides of Postel’s law. glibc: conservative in what it sends. resolved: liberal in what it accepts.

The impedance mismatch

Neither piece is wrong, then. The surprise lives in the glue between them. NSS wired a narrow API onto a wide backend:

getaddrinfo()  ... asks a "hostname" question (RFC 1123 semantics)
     │
     └── nsswitch → libnss_resolve → systemd-resolved  (RFC 2181 semantics)

The narrow question quietly inherits the liberality of the wide backend underneath it. On a glibc-only machine the two sets of semantics happen to coincide, and the surprise never gets a chance to show itself.

Is that a misconfiguration on my part? No. My nsswitch.conf is the untouched Arch default, and precisely what man nss-resolve recommends. Is it a contract violation? Also no, and this is the part I find most interesting: POSIX never promised that getaddrinfo would reject non-LDH names. That strictness was an implementation detail of the BIND-derived resolver, not a guarantee the API ever made. And the security-relevant half of the old filter, sanitizing hostile responses, is something resolved still does, through careful record parsing, escaping and DNSSEC. What actually got dropped along the way is input-side strictness alone, and its practical impact rounds to zero. Unless, of course, someone’s zone happens to have a wildcard in it.

Why ping delegates everything

That leaves the last design decision in the chain: why doesn’t ping validate its own input? Because it deliberately holds no opinion on what a valid name looks like, and that’s the right call.

ping’s job is ICMP at layer 3. What it needs is an address; the name is a convenience for you and me. By calling getaddrinfo it inherits, for free and in lockstep with every other tool on the system, whatever the admin actually configured: /etc/hosts, search domains, mDNS, split-DNS, dual-stack. If ping enforced LDH on its own it would start rejecting perfectly good lookups the resolver handles without complaint: ping "Marco's iPhone.local", IDN names, anything with an underscore in it. A tool has no way of knowing which naming schemes the system supports. Only the resolver does. So ping treats the string as opaque and takes the verdict: -2 means unknown host, an address means start pinging.

The price of that honest abstention is that ping becomes a mirror of the entire resolution stack sitting beneath it. Which is why one mistyped URL made such a good window into how the machine really works.

Mystery solved!

What looked for a while like a haunted ping was the precise spot where RFC 1123 and RFC 2181 part ways, surfacing through an unvalidated string, a permissive resolver, and one wildcard record that had been sitting there for years, waiting patiently for someone to paste a URL into the wrong command.