We are trying to run dragonfly as a replacement for memcached. We are using the default configuration, only specifying the memcached port to turn on memcached mode. We are running it in Kubernetes.
When we go to write keys to Dragonfly, we end up with a problem where we cannot write keys more than about 200 bytes. (200 works, 256 does not, we played with some other numbers, I can do a binary search to find the exact number if necessary.
Upon further digging we see
hidden_info 12 dragonfly_connection.cc:354] Closed connection for peer hidden_ip_address
hidden_info 12 listener_interface.cc:203] sock[13] After HandleRequests
It looks like it is leaking io_uring instances.
hidden_info 10 dragonfly_listener.cc:229] Opening connection 562
It says we are opening a ton of connections but all we did was try to connect to it through a golang app sending one single key.
We also tried to run it with --force_epoll and the problem still persists:
hidden_info 9 listener_interface.cc:119] sock[14] Accepted hidden_info
hidden_info 9 dragonfly_listener.cc:288] CPU/NAPI for connection 14 is 6/0
hidden_info 9 epoll_proactor.cc:276] PRO[0] Fetched 1 cqes
hidden_info 9 listener_interface.cc:194] sock[14] Running connection
hidden_info 9 dragonfly_listener.cc:229] Opening connection 1
hidden_info 9 epoll_proactor.cc:276] PRO[0] Fetched 1 cqes
hidden_info 9 epoll_socket.cc:363] sock[14] Error system:103 on hidden_info
hidden_info 9 dragonfly_connection.cc:481] Before dispatch_fb.join()
hidden_info 9 dragonfly_connection.cc:483] After dispatch_fb.join()
hidden_info 9 dragonfly_connection.cc:354] Closed connection for peer hidden_info
hidden_info 9 listener_interface.cc:203] sock[14] After HandleRequests
hidden_info 9 dragonfly_listener.cc:256] Closing connection 1
We will spin up a github issue with more info if needed, I just figured I would start here to see if there was any idea of what was wrong. Thanks!