We are working on an Infrastructure as Code for Hybrid Setup (GCP). As part of that we have to mount cloud network Filestore with on-premises Kubernetes cluster. We are maintaining two VPC networks in GCP for on-premises & cloud environments. We have configured the required Google cloud HA VPN that lets securely connect on-premises VPC network to cloud VPC. We have verified private connectivity over VPN from the instances in a VPC network, to reach instances in second network VPC. But we are unable to mount the Filestore through VPN.
Here is the kubectl describe pod command output.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 9m46s default-scheduler Successfully assigned axp/filestore-operations-755fd78fb6-w7z5d to gke-hybrid-onprem-kuber-app-node-pool-21f25437-nfvx
Warning FailedMount 57s (x4 over 7m43s) kubelet Unable to attach or mount volumes: unmounted volumes=[nfs-pvc], unattached volumes=[nfs-pvc kube-api-access-drvb5]: timed out waiting for the condition
Warning FailedMount 42s (x3 over 6m46s) kubelet MountVolume.SetUp failed for volume "xligw-pilot-pv" : mount failed: exit status 1
Mounting command: /home/kubernetes/containerized_mounter/mounter
Mounting arguments: mount -t nfs <ip_address>:/filestore_vol1 /var/lib/kubelet/pods/49bde489-bf00-47e1-94ba-eeccfb803ec1/volumes/kubernetes.io~nfs/xligw-pilot-pv
Output: Mount failed: mount failed: exit status 32
Mounting command: chroot
Mounting arguments: [/home/kubernetes/containerized_mounter/rootfs mount -t nfs <ip_address>:/filestore_vol1 /var/lib/kubelet/pods/49bde489-bf00-47e1-94ba-eeccfb803ec1/volumes/kubernetes.io~nfs/xligw-pilot-pv]
Output: mount.nfs: Connection timed out
Furthermore, I have advertised the IP address of the Filestore instance as a custom route during BGP configuration.
https://cloud.google.com/network-connectivity/docs/router/how-to/advertising-custom-ip
Appreciate if you can suggest a solution for this.