What is the issue?

– One day, I tried to build a local GitLab and Jenkins server. I don’t want to remember the IP address, so decided to use the custom domain name/hostname.
– There is some tutorial on the internet telling us we should put a record into the host file, like that.
 
cat /etc/hosts
...
192.168.0.100 gitlab.mlabviet.com
– And this solution works as expected, I’m able to ping and resolve the hostname gitlab.mlabviet.com
– After some days, Jenkin server complains that it can’t not resolve the hostname `gitlab.mlabviet.com`, but WEB browser still able to access jenkins by hostname.
– So I’ve tried to use nslookup to verify, and Oop!, the system’s not able to resolve.
nslookup gitlab.mlabviet.com
Server: 8.8.8.8
Address: 8.8.8.8#53


** server can't find gitlab.mlabviet.com: NXDOMAIN
– Then I try to ping the host name, and this is the result
ping gitlab.mlabviet.com
...
64 bytes from gitlab.mlabviet.com (192.168.0.100): icmp_seq=1 ttl=64 time=0.031 ms
...

– What the hell? nslookup return failed, so how ping command knows the IP
192.168.0.100 is mapped to gitlab.mlabviet.com ????

Why nslookup failed to lookup gitlab.mlabviet.com?

– Let see what happen in the network stack when we call
nslookup [domain name] [dns server]
– As far as I know, nslookup will send a DNS query package to a domain name server to ask which IP address is mapped to the host name gitlab.mlabviet.com
 
– By using WireShark, I can see the packages is sent out to a well known DNS server 8.8.8.8. Actually DNS servers are configured in /etc/resolv.conf and these IPs are updated by  dhclient application.
– Because Google DNS server (8.8.8.8) does not have information about my custom hostname gitlab.mlabviet.com, so nslookup failed to lookup, this is correct behavior.

Why ping success to resolve gitlab.mlabviet.com?

– So, I tried to ping the hostname, it surprises me as it successes in pinging.
ping gitlab.mlabviet.com
...
64 bytes from gitlab.mlabviet.com (192.168.0.100): icmp_seq=1 ttl=64 time=0.031 ms
...
`Ping` also need to know which IP is mapped with the hostname `gitlab.mlabviet.com`. But from WireShark, there is NO DNS query was sent.

– So, The system probably lookups somewhere locally and know the IP before sending any DNS request. After some searching, I’d found it in /etc/nsswitch.conf

```
The Name Service Switch (NSS) configuration file, /etc/nsswitch.conf,
is used by the GNU C Library and certain other applications to
determine the sources from which to obtain name-service information
in a range of categories, and in what order. Each category of
information is identified by a database name.
```
– Let see what inside this file.
cat /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.
passwd: files
group: files
shadow: files
gshadow: files
hosts: files mdns4_minimal [NOTFOUND=return] dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc: db files
netgroup: nis
from man page, the config hosts is defined as below:
hosts Host names and numbers, used by gethostbyname(3) and
related functions.
– Meaning of hosts configuration:
  • files: read from the /etc/hosts files first.
  • mdns4_minimal : if failed, use mDNS
  • dns: If all the above ways are failed, send a DNS query to the DNS server.
– So, at this point, I can understand that the ping command will use the api `gethostbyname()` to get the IP, then system  will lookup `/etc/nsswitch.conf` to decide where to look up the IP address. It successfully lookups the hostname `gitlab.mlabviet.com` in `/etc/hosts` for the first time, so there is no DNS query is sent out.

How to confirm these conclusions are correct or not?

– I write small programs that use the API `gethostbyname()` to verify. In the code, I will try to query the hostname in 2 ways.
  • Use hostname existed in `/etc/hosts`: confirm that no DNS query is sent.
  • Use hostname did not exist in `/etc/hosts`: confirm that DNS query is sent.
char hostname[256] = {0};
int main(int argc, char *argv[])
{
if (argc == 2) {
snprintf(hostname, sizeof(hostname), “%s”, argv[1]);
} else {
snprintf(hostname, sizeof(hostname), “%s”, “google.com”);
}
struct hostent *p_host;
unsigned int i = 0;

 

p_host = gethostbyname(hostname);
if (!p_host) {
return 1;
}
if (p_host->h_addrtype != AF_INET) {
return 1;
}

 

printf(“%s = “, p_host->h_name);
while (p_host->h_addr_list[i] != NULL) {
printf(“%s “, inet_ntoa(*(struct in_addr *)(p_host->h_addr_list[i])));
i++;
}
printf(\n);
return 0;
}
– And the result is correct as I expected, this application use the gethostbyname()  API to resolve the domain name, and based on nsswitch.con
  • gitlab.mlabviet.com: the system had prioritized reading from the /etc/hosts file first.
  • google.com: not exited in host file, no mDNS discovried, so DNS request is sent oput to DNS server.