Karan Singh

Code Never Lies, Comments Sometime Do !!

OpenShift / K8s : DNS Configuration Explained

| Comments

openshift-route Life is easy when you have a DNS service … But you not always have DNS service handy to play with !!!

When you have an openshift local environment and do not have a DNS service in place, accessing your openshift application endpoints from outside master node could be challenging. I had the same problem in home lab, which i managed to fix.

In my setup, i have an openshift cluster hosted on a my home server. My applications which are ofcourse inside containers works fine from within openshift master node. However accessing my appliation from another host (e.g. my workstation) does not work. Reason being, DNS resolution for application endpoint missing.

Here is how you can access your application from outside openshift master noded.

Note: You must have a fully functional openshift environment with some apps running, exposed with their route configured.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[[email protected] ~]# oc get po
NAME                 READY     STATUS    RESTARTS   AGE
mapit-1-k3b15        1/1       Running   0          5h
node-hello-1-n24k8   1/1       Running   1          1d
[[email protected] ~]#
[[email protected] ~]# oc get routes
NAME         HOST/PORT                                                   PATH      SERVICES     PORT       TERMINATION   WILDCARD
mapit        mapit-first-project.router.default.svc.cluster.local                  mapit        8080-tcp                 None
node-hello   node-hello-first-project.router.default.svc.cluster.local             node-hello   8080-tcp                 None
[[email protected] ~]#
[[email protected] ~]#
[[email protected] ~]#
[[email protected] ~]# curl http://node-hello-first-project.router.default.svc.cluster.local
Hello Kubernetes!
[[email protected] ~]#
[[email protected] ~]#
  • As shown above everything works fine from within openshift master node. However i try accessing the same application from outside your openshift master node (e.g. from my workstation) it does not work.
1
2
3
karasing-OSX:~$ curl http://node-hello-first-project.router.default.svc.cluster.local
curl: (6) Could not resolve host: node-hello-first-project.router.default.svc.cluster.local
karasing-OSX:~$
So the challenge for the day is “How to fix it” ?

Step - 1 : Configure dnsmasq on openshift master node

  • By default OCP master node runs SkyDNS service on port 8053 for DNS resolution of router pods.
1
2
3
4
[[email protected] ~]# netstat -plunt | grep -i 8053
tcp        0      0 0.0.0.0:8053            0.0.0.0:*               LISTEN      2218/openshift
udp        0      0 0.0.0.0:8053            0.0.0.0:*                           2218/openshift
[[email protected] ~]#
  • Now to forward DNS queries to SkyDNS we need to use dnsmasq a lightweighted DNS server on OCP master node
  • Install dnsmasq on OCP master node
1
# yum install -y dnamasq
  • Edit default /etc/dnsmasq.conf file and add the following lines. Update subdomain domain name, hostname, IP address of your master node.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Reverse DNS record for master
host-record=beast,192.168.0.5  <== Add your OCP Master Node hostname and IP

# Wildcard DNS for OpenShift Applications - Points to Router
address=/local/192.168.0.5     <== Add your OCP Master Node IP    

# Forward .local queries to SkyDNS
server=/local/0.0.0.0#8053

# Do not read /etc/resolv.conf and forward requests to nameservers listed there:
no-resolv

# Never forward plain names (without a dot or domain part)
domain-needed

# Never forward addresses in the non-routed address spaces.
bogus-priv

Here we are using wildcard DNS resolution for domains *.local and forwarding the resolution queries to SkyDNS. i.e all local resolution queries from 192.160.0.5:53 will be forwarded to 0.0.0.0:8053

Step - 2 : Configure dnsmasq on your workstation

We will perform almost similar steps to configure the node you are trying to access your application from (in my case its macbook) for resolving OCP application endpoints to services.

  • Install dnsmasq
1
2
$ brew install dnsmasq
$ mkdir /usr/local/etc/ (create if not exists)
  • Edit /usr/local/etc/dnsmasq.conf and add the following. Update subdomain domain name, hostname, IP address of your master node.
1
2
3
4
5
6
7
8
9
10
11
# Wildcard DNS for all .local domains : Points to other DNS server
address=/local/192.168.0.5  <== Add your OCP Master IP here

# Do not read /etc/resolv.conf and forward requests to nameserver instead
no-resolv

# Never forward plain names (without a dot or domain part)
domain-needed

# Never forward addresses in the non-routed address spaces.
bogus-priv

Here we are instructing dnsmasq to resolve all the hostnames with *.local to OCP master IP address

  • Update resolver for MacOS
1
2
$ sudo mkdir /etc/resolver
$ sudo bash -c 'echo "nameserver 127.0.0.1" > /etc/resolver/local'
  • Edit /etc/resolv.conf and add the following just after search domain
1
nameserver 127.0.0.1
  • Restart dnsmasq service
1
2
3
4
5
6
$ sudo brew services start  dnsmasq
==> Successfully started `dnsmasq` (label: homebrew.mxcl.dnsmasq)
$ sudo brew services list  dnsmasq
Name    Status  User Plist
dnsmasq started root /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
$

Step - 3 : Verify DNS resolution and test your application

  • Let’s verify that our dnsmasq server is working as expected and resolving all subdomains of *.local
1
2
3
4
5
6
7
8
$ ping -c 1 mapit-first-project.router.default.svc.cluster.local
PING mapit-first-project.router.default.svc.cluster.local (192.168.0.5): 56 data bytes
64 bytes from 192.168.0.5: icmp_seq=0 ttl=64 time=0.325 ms

--- mapit-first-project.router.default.svc.cluster.local ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.325/0.325/0.325/0.000 ms
karasing-OSX:~$
  • Test DNS resolution as well
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ dig mapit-first-project.router.default.svc.cluster.local

; <<>> DiG 9.8.3-P1 <<>> mapit-first-project.router.default.svc.cluster.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48914
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;mapit-first-project.router.default.svc.cluster.local. IN A

;; ANSWER SECTION:
mapit-first-project.router.default.svc.cluster.local. 0   IN A 192.168.0.5

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Oct  4 01:09:14 2017
;; MSG SIZE  rcvd: 86

Since we are able to resolve application endpoint directly from our workstation, accessing the application should also work

1
2
3
karasing-OSX:~$ curl http://node-hello-first-project.router.default.svc.cluster.local
Hello Kubernetes!
karasing-OSX:~$

yay .. we are able to access our application from outside openshift master node !!! Long Live DNSmasq

Comments