# Kong Ingress Crontroller in DBless mode error; failed to wait for secret caches to sync: timed out waiting for cache to be synced
**URL:** https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923
**Category:** Questions
**Tags:** kubernetes
**Created:** [March 8, 2022, 5:35pm UTC](https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923 "2022-03-08T17:35:34Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@anmolsha](https://discuss.konghq.com/u/anmolsha)
#### Post date: [March 8, 2022, 5:35pm UTC](https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923/1 "2022-03-08T17:35:35Z")
In my case, there are a large number of deployments running, that also correspond to large secret resources. The Kong pods start failing, when I do helm upgrade on those deployments, which eventually raise the number of K8S secret resource counts to approx 8K. Pasting here the debug logs from the kong `ingress-controller` (version: `2.1.1`) container:
```auto
time="2022-03-08T17:23:57Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
I0308 17:23:57.491252 1 trace.go:205] Trace[1752425436]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167 (08-Mar-2022 17:22:57.489) (total time: 60001ms):
Trace[1752425436]: [1m0.001703338s] [1m0.001703338s] END
E0308 17:23:57.491272 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167: Failed to watch *v1.Secret: failed to list *v1.Secret: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 197; INTERNAL_ERROR; received from peer
time="2022-03-08T17:24:00Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
...
...
...
time="2022-03-08T17:24:57Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for secret caches to sync: timed out waiting for cache to be synced" logger=controller.secret reconciler group= reconciler kind=Secret
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongplugin caches to sync: timed out waiting for cache to be synced" logger=controller.kongplugin reconciler group=configuration.konghq.com reconciler kind=KongPlugin
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongconsumer caches to sync: timed out waiting for cache to be synced" logger=controller.kongconsumer reconciler group=configuration.konghq.com reconciler kind=KongConsumer
time="2022-03-08T17:24:57Z" level=info msg="context done: shutting down the proxy update server" subsystem=proxy-cache-resolver
time="2022-03-08T17:24:57Z" level=error msg="context completed with error" error="context canceled" subsystem=proxy-cache-resolver
I0308 17:24:57.489210 1 trace.go:205] Trace[1371614229]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167 (08-Mar-2022 17:23:58.818) (total time: 58670ms):
Trace[1371614229]: [58.670670788s] [58.670670788s] END
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for ingress caches to sync: timed out waiting for cache to be synced" logger=controller.ingress reconciler group=networking.k8s.io reconciler kind=Ingress
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for service caches to sync: timed out waiting for cache to be synced" logger=controller.service reconciler group= reconciler kind=Service
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for tcpingress caches to sync: timed out waiting for cache to be synced" logger=controller.tcpingress reconciler group=configuration.konghq.com reconciler kind=TCPIngress
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for endpoints caches to sync: timed out waiting for cache to be synced" logger=controller.endpoints reconciler group= reconciler kind=Endpoints
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongclusterplugin caches to sync: timed out waiting for cache to be synced" logger=controller.kongclusterplugin reconciler group=configuration.konghq.com reconciler kind=KongClusterPlugin
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongconsumer caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for udpingress caches to sync: timed out waiting for cache to be synced" logger=controller.udpingress reconciler group=configuration.konghq.com reconciler kind=UDPIngress
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for ingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongingress caches to sync: timed out waiting for cache to be synced" logger=controller.kongingress reconciler group=configuration.konghq.com reconciler kind=KongIngress
time="2022-03-08T17:24:57Z" level=info msg="stop status update channel."
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for service caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for tcpingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for endpoints caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongclusterplugin caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for udpingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongplugin caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="context canceled"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="context canceled"
Error: failed to wait for secret caches to sync: timed out waiting for cache to be synced
Error: failed to wait for secret caches to sync: timed out waiting for cache to be synced
```
Can above issue be solved by implementing [Kubernetes API Concepts | Kubernetes](https://kubernetes.io/docs/reference/using-api/api-concepts/#retrieving-large-results-sets-in-chunks) somewhere in the Kong Ingress Controller code?
OR any other fix that I can try from Kong configuration options.
---
### Author:  [@krish7919](https://discuss.konghq.com/u/krish7919)
#### Post date: [March 31, 2022, 9:36am UTC](https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923/2 "2022-03-31T09:36:58Z")
I am facing the same issue due to a huge number of Secrets in the cluster, and trying to find a fix.
---
### Author:  [@krish7919](https://discuss.konghq.com/u/krish7919)
#### Post date: [March 31, 2022, 3:14pm UTC](https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923/3 "2022-03-31T15:14:37Z")
I can easily reproduct this scenario in our test cluster with 1500 Kubernetes Secrets with ~1MB of data in each of them.