[Cilium] #5주차 (2) Cilium Geneve encapsulation + DSR(Direct Server Return)

25년도 Cilium Study 1기 정리 글입니다.

 

Geneve + DSR

(⎈|HomeLab:N/A) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --version 1.18.0 --namespace kube-system --reuse-values \
  --set tunnelProtocol=geneve --set loadBalancer.mode=dsr --set loadBalancer.dsrDispatch=geneve \
  --set loadBalancer.algorithm=maglev
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Sat Aug 16 20:03:24 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble Relay and Hubble UI.

Your release version is 1.18.0.

For any further help, visit https://docs.cilium.io/en/v1.18/gettinghelp



(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl -n kube-system rollout restart ds/cilium
daemonset.apps/cilium restarted


(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium status --verbose
...
  Mode:                  DSR
    DSR Dispatch Mode:   Geneve
  Backend Selection:     Maglev (Table Size: 16381)
  Session Affinity:      Enabled

 

설정 전에는 서비스 IP가 모든 노드에 균등하게 광고되어서 파드가 없는 노드로도 트래픽이 흘러들어가는 문제가 있었다.

하지만 externalTrafficPolicy=Local과 함께 위 설정을 적용하여 BGP 라우팅 테이블에는 실제로 Pod가 존재하는 노드만 nexthop으로 잡힌다.

외부에서 들어온 요청은 Geneve 캡슐링을 통해 적절한 파드로 도달하여 응답은 Pod → 클라이언트로 직접 나가므로 불필요한 홉이 사라지게 된다.

 

컨트롤 플레인, 워커노드0,1에서 curl 테스트

tcpdump -i eth1 -w /tmp/dsr.pcap


root@router:~# curl -s $LBIP
Hostname: webpod-697b545f57-fnz6w
IP: 127.0.0.1
IP: ::1
IP: 172.20.0.205
IP: fe80::c8f4:b0ff:fe01:9d74
RemoteAddr: 192.168.10.200:43122
GET / HTTP/1.1
Host: 172.16.1.1
User-Agent: curl/8.5.0
Accept: */*

loopy  ~/cilium-lab
 vagrant plugin install vagrant-scp
Installing the 'vagrant-scp' plugin. This can take a few minutes...
Fetching hashdiff-1.2.0.gem
Fetching crack-1.0.0.gem
Fetching public_suffix-6.0.2.gem
Fetching addressable-2.8.7.gem
Fetching webmock-3.25.1.gem
Fetching rspec-support-3.13.4.gem
Fetching diff-lcs-1.6.2.gem
Fetching rspec-expectations-3.13.5.gem
Fetching rspec-core-3.13.5.gem
Fetching rspec-its-2.0.0.gem
Fetching rspec-mocks-3.13.5.gem
Fetching rspec-3.13.1.gem
Fetching rake-13.3.0.gem
Thank you for installing the Vagrant VMware Desktop
plugin. This plugin requires the Vagrant VMware
Utility to be installed. To learn more about the
Vagrant VMware Utility, please visit:

  https://www.vagrantup.com/docs/providers/vmware/vagrant-vmware-utility

To install the Vagrant VMware Utility, please
download the appropriate installer for your
system from:

  https://www.vagrantup.com/downloads/vmware
Fetching vagrant-scp-0.5.9.gem
Successfully uninstalled addressable-2.8.7
Successfully uninstalled crack-1.0.0
Removing htmldiff
Removing ldiff
Successfully uninstalled diff-lcs-1.6.2
Successfully uninstalled hashdiff-1.2.0
Successfully uninstalled public_suffix-6.0.2
Removing rake
Successfully uninstalled rake-13.3.0
Successfully uninstalled rspec-3.13.1
Removing rspec
Successfully uninstalled rspec-core-3.13.5
Successfully uninstalled rspec-expectations-3.13.5
Successfully uninstalled rspec-its-2.0.0
Successfully uninstalled rspec-mocks-3.13.5
Successfully uninstalled rspec-support-3.13.4
Successfully uninstalled webmock-3.25.1
Installed the plugin 'vagrant-scp (0.5.9)'!



 loopy  ~/cilium-lab
 vagrant scp k8s-ctr:/tmp/dsr.pcap .

Warning: Permanently added '[127.0.0.1]:60000' (ED25519) to the list of known hosts.
dsr.pcap    100%   48KB  44.0MB/s   00:00

 

 

• Outer 헤더 (Geneve Encapsulation)
  • Src: 192.168.10.100 (컨트롤 플레인)
  • Dst: 192.168.10.101 (워커1)
  • Protocol: UDP/6081 (제네브)
• Inner 헤더 (원래 요청 패킷)
  • Src: 192.168.10.200 (라우터)
  • Dst: 172.20.1.135 (Pod IP)
  • Protocol: TCP 43124 → 80, HTTP GET

 

파드(172.20.1.135)는 Geneve 터널을 거치지 않고 직접 라우터(192.168.10.200)로 응답을 반환하는데,

이는 Cilium DSR(Direct Server Return) 모드의 특징으로 응답 트래픽은 서비스 노드를 거치지 않고 바로 클라이언트로 향하는 것을 보여주는 것이다.