On-premise : multiple namespaces accessible from outisde the cluster

I’m on-premise with kubernetes 1.20 . I want to put CICD in place like gitops. When a new environment or branch it’s created, I want to deploy the application in its own namespace and I want the application to be accessible. (so I’ll have the same application deployed multiple times, but it a different version)here my usecase. I want to deploy my applications on multiple namespaces

dev
qa
PR-111 (for Pull request)

I used that Kong demo GitHub - gAmUssA/quotes-service: a samle webservice renerates fake quotes at starting point.

my services could look like that

chuck-service:8080
quote-service:8080

I’m on premise with nginx-ingress and with metallb as loadbalancer
I’ll expose nginx-controler as daemonset with a external IP : 10.1.10.123 for ingress

/chuck -> chuck-service:8080
/quote -> quote-service:8080

I want to have those applications to be accessible for from outside on 10.1.10.123 (I can’t expose new IP)
the domain (inside-my-company.com) name it not register in external DNS
ex :

 dev.inside-my-company.com/chuck
 dev.inside-my-company.com/quote
 qa.inside-my-company.com/chuck
 qa.inside-my-company.com/quote

How can I do that ? For now I only have one version and it’s in the default namespace. like

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chuck
  annotations:
    # this important https://docs.konghq.com/kubernetes-ingress-controller/1.3.x/references/annotations/#konghqcomstrip-path
    konghq.com/strip-path: "true"
spec:
  ingressClassName: kong
  rules:
    - http:
        paths:
          - path: /quote
            pathType: Prefix
            backend:
              service:
                name: reactive-quote-service
                port:
                  number: 8080
          - path: /chuck
            pathType: Prefix
            backend:
              service:
                name: chuck-quote-service
                port:
                  number: 8080

Here the list of my services. Kong is my ingress controller

vagrant@enroute-master:~$ kubectl get svc --all-namespaces
NAMESPACE     NAME                      TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                      AGE
default       chuck-quote-service       ClusterIP      10.110.149.200   <none>           8080/TCP                     12h
default       exdns-k8s-gateway         LoadBalancer   10.109.67.179    192.168.50.202   53:30186/UDP                 11h
default       kubernetes                ClusterIP      10.96.0.1        <none>           443/TCP                      13h
default       reactive-quote-service    ClusterIP      10.108.148.54    <none>           80/TCP                       12h
kong          kong-proxy                LoadBalancer   10.103.175.232   192.168.50.200   80:30092/TCP,443:31098/TCP   12h
kong          kong-validation-webhook   ClusterIP      10.110.25.248    <none>           443/TCP                      12h
kube-system   exdns-2-k8s-gateway       LoadBalancer   10.105.96.51     192.168.50.203   53:30389/UDP                 11h
kube-system   ext-dns-tcp               LoadBalancer   10.111.101.102   192.168.50.201   53:32759/TCP                 11h
kube-system   ext-dns-udp               LoadBalancer   10.110.14.237    192.168.50.201   53:31119/UDP                 11h
kube-system   kube-dns                  ClusterIP      10.96.0.10       <none>           53/UDP,53/TCP,9153/TCP       13h
test          chuck-quote-service       ClusterIP      10.109.213.53    <none>           8080/TCP                     12h
test          reactive-quote-service    ClusterIP      10.106.129.43    <none>           80/TCP                       12h
vagrant@enroute-master:~$

I have 2 applications

    chuck-quote-service
    reactive-quote-service

For those 2 applications, I want to deploy them in dev, qa… namespaces and modify the ingress rules for that.and I need to access those applications from outside my cluster, like

http://dev.example.org/chuck
http://qa.example.org/chuck

I’m looking to reproduce that setup on bare-metal with kubernetes 1.20 configured with kubeadm.

This should be quite simple to handle. You’ll just need to create a copy of your Ingress definition in each of your namespaces, and then add a hostname criterion to its rules (rules with no hostname will be accessible via those paths on any hostname, which isn’t what you want).

Ingresses are only allowed to send traffic to Services within their namespaces, so you don’t need any additional configuration to ensure that; just place the Ingress with the matching hostname in that namespace and it will route to the Services in the namespace automatically.

but the problem is that I want to connect to dev.example.org from outside my cluster. If I try curl http://dev.example.org I will get a “host not found” because I don’t have a DNS Server that will redirect/forward to my cluster loadbalancer.

if my loadbalancer is : 10.1.10.123

I need to convert example.org → 10.1.10.123

and that’s where I think I need to have a private internal DNS server and add this dns server into each of my computer/VM

I made lot of progress this weekend.

I started from scratch.

I found a section on Kubernetes docs about “virtual host”. We need to pass the “Host” header

here are my ingress in a namespace dev.

root@test-pcl4014:~# kubectl -n dev get ingress
NAME               CLASS    HOSTS                       ADDRESS      PORTS   AGE
gateway            <none>   dev.kubernetes.comact.com   10.1.34.55   80      176m
production-wui     <none>   dev.kubernetes.comact.com   10.1.34.55   80      174m
twin-api-service   <none>   dev.kubernetes.comact.com   10.1.34.55   80      13m
root@test-pcl4014:~#

if I want to call gateway endpoint, I have to do that

curl -I -H 'Host: dev.kubernetes.comact.com' http://10.1.34.55/gateway

my last problem is HOW to access the UI . When I use only one namespace and no host… It’s simple

http://10.1.34.55/ui 

but now, I could have the UI deployed in QA, DEV, staging… I need to find how to pass the header when I try to access to UI. Maybe I could have a different ingress for the UI. I put the prefix in the url like :

http://10.1.34.55/dev/ui 
http://10.1.34.55/qa/ui 

Ingress can’t handle the DNS mapping side of things. If you send the Ingress-controlled proxy a request with that hostname/path combination, it will route it to the appropriate Service inside the cluster, but it doesn’t have any control over whether clients resolve that hostname to the proxy address. You will indeed want to add records mapping your hostnames to your proxy address.

You can achieve something similar to your curl Host header override (using the proxy address without creating a DNS record) by adding a local hosts file entry. This effectively creates a DNS record that only your computer will use, and should allow you to access the GUI, as your browser will honor it.

If you’re using non-root paths in your route (e.g. using example.com/dev instead of dev.example.com in your Ingress rule), you will often need to configure your application to know what its external URL is, as without this configuration many applications will attempt to build paths that expect they’re at the root. You can usually avoid this by using a hostname only in your rule.


© 2019 Kong Inc.    Terms  •  Privacy  •  FAQ