[BUG] DNS resolution issues - polluted by DHCP search option

[shalak]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

curl resolves domain names for another containers via external DNS.

Expected Behavior

curl resolves domain names for another containers via internal docker networking first

Steps To Reproduce

1. Setup a LAN-wide DNS resolver and DHCP server with search option for .example.org
2. Name another container e.g. influxdb
3. Create DNS entry for influx.example.org on DNS server
4. Try to curl http://influxdb

Here's how /etc/resolv.conf looks like:

root@8134903bf950:/# cat /etc/resolv.conf
# Generated by Docker Engine.
# This file can be edited; Docker Engine will not make further changes once it
# has been modified.

nameserver 127.0.0.11
search example.org
options edns0 trust-ad ndots:0

# Based on host file: '/etc/resolv.conf' (internal resolver)
# ExtServers: [host(127.0.0.53)]
# Overrides: []
# Option ndots from: internal

The ping resolves DNS properly, via internal Docker DNS service, to 172.20.0.4:

root@8134903bf950:/# ping -c 1 influxdb
PING influxdb (172.20.0.4): 56 data bytes
64 bytes from 172.20.0.4: seq=0 ttl=64 time=0.246 ms
(...)

The curl resolves DNS incorrectly, via my LAN DNS service, I believe because before attempting to resolve influxdb it first applies the DHCP search option and adds .example.org to influxdb, i.e. resolves to my influxdb.example.org address, which is 10.0.0.131 (address of my reverse-proxy, port 8086 is not even open there):

root@8134903bf950:/# curl -v http://influxdb:8086
* Host influxdb:8086 was resolved.
* IPv6: ::ffff:10.0.0.131
* IPv4: 10.0.0.131
*   Trying 10.0.0.131:8086...
* connect to 10.0.0.131 port 8086 from 172.20.0.16 port 37192 failed: Connection refused
*   Trying [::ffff:10.0.0.131]:8086...
* connect to ::ffff:10.0.0.131 port 8086 from ::ffff:172.20.0.16 port 37200 failed: Connection refused
* Failed to connect to influxdb port 8086 after 4 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to influxdb port 8086 after 4 ms: Could not connect to server

As a workaround, I can use influxdb.services_default hostname (services_default is my compose network):

root@8134903bf950:/# curl -v http://influxdb.services_default:8086
* Host influxdb.services_default:8086 was resolved.
* IPv6: (none)
* IPv4: 172.20.0.4
*   Trying 172.20.0.4:8086...
* Connected to influxdb.services_default (172.20.0.4) port 8086
> GET / HTTP/1.1
> Host: influxdb.services_default:8086
> User-Agent: curl/8.9.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
(...)

This issue breaks the communication to influxdb, as speedtest-tracker's PHP uses curl libs under the hood.

Environment

- OS: Ubuntu 22.04.4 LTS
- How docker service was installed: curl docker.io

CPU architecture

x86-64

Docker creation

speedtest:
    image: lscr.io/linuxserver/speedtest-tracker:latest
    container_name: speedtest
    restart: unless-stopped
    environment:
        - PUID=1000
        - PGID=1000
        - DB_CONNECTION=mysql
        - DB_HOST=speedtest-db
        - DB_PORT=3306
        - DB_DATABASE=speedtest_tracker
        - DB_USERNAME=speedy
        - APP_URL=https://speedtest.example.org
        - APP_NAME=Speedtest Tracker
    env_file: /srv/fast/services/speedtest.env
    volumes:
        - '/srv/fast/services/speedtest/speedtest:/config'
    labels:
      - "traefik.enable=true"
    depends_on:
        - speedtest-db
  speedtest-db:
    image: mariadb:10
    container_name: speedtest-db
    restart: unless-stopped
    environment:
        - MARIADB_DATABASE=speedtest_tracker
        - MARIADB_USER=speedy
        - MARIADB_RANDOM_ROOT_PASSWORD=true
    env_file: /srv/fast/services/speedtest.env
    volumes:
        - /srv/fast/services/speedtest/speedtest-db:/var/lib/mysql
  influxdb:
    image: influxdb:2.7.0
    container_name: influxdb
    volumes:
      - /srv/fast/services/influxdb/data:/var/lib/influxdb2
      - /srv/fast/services/influxdb/config:/etc/influxdb2
    restart: unless-stopped
    labels:
      - "traefik.enable=true"

Container logs

[migrations] started
[migrations] 01-nginx-site-confs-default: skipped
[migrations] done
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support the app dev(s) visit:
speedtest-tracker: https://github.com/sponsors/alexjustesen

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    1000
User GID:    1000
───────────────────────────────────────
Linuxserver.io version: v0.21.2-ls43
Build-date: 2024-08-24T21:21:31+00:00
───────────────────────────────────────

using keys found in /config/keys
Waiting for DB to be available
[custom-init] No custom files found, skipping...
[ls.io-init] done.

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[shalak]

I did not notice this issue with other linuxserver.io images, e.g. FreshRSS' curl resolves domain names correctly

[aptalca]

Your issue is your search domain included, but the behavior is the expected behavior.

According to the man page, if the query has no dots, the search domain is automatically added.
https://man7.org/linux/man-pages/man5/resolv.conf.5.html

Typically in a home lab setting the search domain would be set to the local domain used on the lan. You have it set to a public domain.

[shalak]

Your issue is your search domain included, but the behavior is the expected behavior.

Why doesn't it happen in other images or binaries, only curl and only in speedtest-tracker docker image?

According to the man page, if the query has no dots, the search domain is automatically added.

I'm not sure what you mean. I believe you refer to this part?

Finally, if the hostname does not contain a '.', the root domain is assumed as the local domain name.

This refers to populating the search domain list: By default, the search list contains one entry, the local domain name. In my setup this list contains two: the "local domain name" and the one from my DHCP. The former is calculated with the "dot" logic.

Typically in a home lab setting the search domain would be set to the local domain used on the lan. You have it set to a public domain.

For this specific case, when using local DNS server the concept of "local domain" vs "public domain" disappears. Whether the search domain option is set to valid TLD, or a syntetic one, from the perspective of the client - it doesn't matter.

Even when I set the domain search to foo.bar the issue persist (assuming I have a baz container and a baz.foo.bar domain name configured in DNS).

Looks like curl libs somehow bypass the docker's name resolution 🤔

[LinuxServer-CI]

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.