Cluster mode support

Hello, I would like to ask, does dragonfly now support a real cluster mode, similar to that of redis cluster?
Now there is a demand, there is a cluster with a capacity of more than 1T, and I want to migrate to dragonfly for testing, but the maximum memory of a single physical machine is generally only 384G. I also considered that dragonfly actually supports sentinel mode, and can be used as a cluster with multiple master names, but the problem is that the business needs to consider the issue of sharding, and once the cluster capacity reaches the upper limit, the expansion will be very heavy , requires reshaard operation.

I have consulted before and explained that the cluster mode is not supported for the time being. Do we have any plans to support the cluster mode in the future?

We do not have plans to provide cluster support for self hosted installations. It’s gonna be part of our managed service solution.

which cloud you are on?

Our services are on our own cloud.
You mean, to use the cluster cluster mode, you need to use your company’s cloud services? This function is not open source? Is there something wrong with my understanding? @inclusive-numbat

  1. we have not yet implemented cluster mode yet
  2. yes, when we implement it, it won’t be part of the OSS. Dragonfly cluster mode will require a third-party component similar to redis-sentinel to manage the cluster. This component will be proprietary

Ok, got it, thanks.

I would like to ask you another question. Do you mean that this separate component similar to redis-sentinel has the function of initialization and even management of fragmentation? Similar to codis? Or redis-cli --cluster mode? In my impression, redis-sentinel only has the function of managing the high availability of instances, and has no function to allocate slots. Can you give me some pointers? @inclusive-numbat

I meant at high level, management component that manages nodes, did not meant to say that redis sentinel can manage a cluster

What’s this codis?

Ah, sorry, I forgot to introduce, codis is an intermediate key for managing redis clusters, which is open sourced by the pea pod infrastructure team. There are also many businesses in our country that use it.
Project address: https://github.com/CodisLabs/codis

You mean that the management component module will be similar to redis-sentinel to manage instance switching and high availability guarantees. In fact, the functions of the cluster mode may be implemented at the server level. Is there any problem with this understanding?
Indeed, the current situation can already meet most scenarios for general cache level construction. After all, the required capacity of most clusters will not exceed 200G. My idea is that if dragonfly supports cluster mode, then maybe we can have a lot of room for optimization in terms of cost reduction and efficiency increase. Because there are many large clusters in our scenario, a simple master-slave architecture may not be able to meet the needs of large clusters on our side.
Of course, now everyone is still making software for commercialization, which is understandable. Thank you for your reply. :+1: @inclusive-numbat

Yes, we must create enough value so that companies would agree to pay for extra features

Clear, We may start to build and use it from the containerized small cluster level in the future. At the same time, I hope that dragonfly will get better and better, thank you for your hard work. :grimacing:

Thanks!

Cluster mode support

@organized-alpaca why codis does not meet your needs?

Codis is just an intermediate key. It can solve the problem of multi-instance slot allocation, but it cannot solve the problem of cost reduction and efficiency increase. If this server only has the function of managing slots, then we might as well use redis cluster directly. , also saves the middle key link. I value the performance advantages of dragonfly and the cost optimization at the memory storage level, so I want to choose dragonfly in cluster mode to try to replace some large redis clusters. This skill saves hardware costs, and may also save a certain amount of performance. If it is just to replace the small clusters in the cluster, the direction may be the container direction. Try to use the dragonfly on k8s solution to reduce costs and increase efficiency by optimizing all instances in the resource pool. But the large cluster can only maintain the original state, so we are actually very urgent for this function. @inclusive-numbat