Python脚本之操作Redis Cluster

本文为博主原创，未经授权，严禁转载及使用。
本文链接：https://blog.csdn.net/zyooooxie/article/details/123760358

之前曾经分享过2篇关于Redis命令：https://blog.csdn.net/zyooooxie/article/details/120334491 、 https://blog.csdn.net/zyooooxie/article/details/95352530，这一篇博客是讲Redis cluster对key相关的操作。

此外有些博客写到 python的redis库是不支持集群操作，现在已经支持了。

https://pypi.org/project/redis/ redis-py now supports cluster mode and provides a client for Redis Cluster.

Redis集群命令

CLUSTER INFO

CLUSTER KEYSLOT key

CLUSTER NODES

CLUSTER SLAVES node-id

CLUSTER GETKEYSINSLOT slot count

CLUSTER COUNTKEYSINSLOT slot

想要更加详细学习可以参考： https://redis.io/commands/?group=cluster

我的疑惑

在接触到集群命令后，我不仅仅 KEYSLOT(test_key)，我还实际去对应master节点去查看这个test_key，然后我就发现：公司好几个Redis集群每个master节点都有这个test_key。

这。。。很遗憾，没确切找到原因。

此外，Redis 集群不支持那些需要同时处理多个键的 Redis 命令，因为执行这些命令需要在多个 Redis 节点之间移动数据，并且在高负载的情况下，这些命令将降低 Redis 集群的性能，并导致不可预测的行为。

但我发现 redis-py-cluster 是支持mset、mget的，所以本期的代码主要讲 2个库如何连接集群。

代码：redis-py

最初搜使用Python操作 Redis cluster，所有答案都是redis-py 不支持。但我在学习 cluster commands，看到 https://redis-py.readthedocs.io/en/stable/commands.html#redis-cluster-commands，我恍惚了，又确认好几遍是redis-py，才去翻看pypi redis-py的Project description。

我在使用 redis 4.1.4。

"""
@blog: https://blog.csdn.net/zyooooxie
"""def test_rp():"""redis-py:return:"""from redis.cluster import RedisClusterfrom redis.cluster import ClusterNodehost2 = 'host'port = 6666pwd = 'pwd'nodes = [ClusterNode(host2, port)]rc = RedisCluster(startup_nodes=nodes, password=pwd, decode_responses=True)Log.info(rc)Log.info(rc.cluster_info())Log.info(rc.cluster_slots())Log.info(rc.cluster_nodes())exist_key = 'exist_key'exist_key_slot = rc.cluster_keyslot(exist_key)Log.info(exist_key_slot)Log.info(rc.cluster_countkeysinslot(exist_key_slot))Log.info(rc.cluster_get_keys_in_slot(exist_key_slot, 5))Log.info('------')Log.info('------')Log.info('------')cluster_test_str = 'test_str'Log.info(rc.set(cluster_test_str, 'zyooooxie', ex=random.randint(600, 1000)))Log.info(rc.cluster_keyslot(cluster_test_str))Log.info(rc.exists(cluster_test_str))Log.info(rc.type(cluster_test_str))Log.info(rc.ttl(cluster_test_str))Log.info(rc.expire(cluster_test_str, 2 * 60 * 60))Log.info(rc.ttl(cluster_test_str))Log.info(rc.get(cluster_test_str))Log.info(rc.getset(cluster_test_str, '新值str'))Log.info(rc.get(cluster_test_str))# 类似mset, mget这样的多个key的原生批量操作命令, redis集群只支持所有key落在同一slot的情况key1 = 'test1'key2 = 'test2'Log.info(rc.cluster_keyslot(key1))Log.info(rc.cluster_keyslot(key2))# Log.info(rc.mset({key1: 'Redis Cluster-{}'.format(key1), key2: 'Redis Cluster-{}'.format(key2)}))# # TODO 留意：redis.exceptions.RedisClusterException: MSET - all keys must map to the same key slot# Log.info(rc.mget(key1, key2))# # redis.exceptions.RedisClusterException: MGET - all keys must map to the same key slotLog.info(rc.delete(cluster_test_str, key1, key2))Log.info('------------------')cluster_test_hash = 'test_hash'Log.info(rc.hset(cluster_test_hash, mapping={'hash_key': 'hash_value', 'hash_key1': 'hash_value1','hash_key2': 'hash_value2', 'hash_key3': 'hash_value3'}))Log.info(rc.cluster_keyslot(cluster_test_hash))Log.info(rc.exists(cluster_test_hash))Log.info(rc.type(cluster_test_hash))Log.info(rc.ttl(cluster_test_hash))Log.info(rc.expire(cluster_test_hash, 60 * 60))Log.info(rc.ttl(cluster_test_hash))Log.info(rc.hget(cluster_test_hash, 'hash_key'))Log.info(rc.hget(cluster_test_hash, 'hash_key2222'))Log.info(rc.hgetall(cluster_test_hash))Log.info(rc.hexists(cluster_test_hash, 'hash_key'))Log.info(rc.hexists(cluster_test_hash, 'hash_key2222'))Log.info(rc.hkeys(cluster_test_hash))Log.info(rc.hvals(cluster_test_hash))Log.info(rc.hlen(cluster_test_hash))Log.info(rc.hdel(cluster_test_hash, 'hash_key2222', 'hash_key'))Log.info(rc.hlen(cluster_test_hash))Log.info(rc.hmget(cluster_test_hash, 'hash_key', 'hash_key2'))Log.info(rc.hmset(cluster_test_hash, {'test': 'test1', 'test2': 'test3'}))Log.info(rc.hgetall(cluster_test_hash))Log.info(rc.delete(cluster_test_hash))Log.info('------------------')cluster_test_list = 'test_list'Log.info(rc.rpush(cluster_test_list, 'list1', 'list2', 'list3'))Log.info(rc.cluster_keyslot(cluster_test_list))Log.info(rc.type(cluster_test_list))Log.info(rc.exists(cluster_test_list))Log.info(rc.ttl(cluster_test_list))Log.info(rc.expire(cluster_test_list, 5 * 60 * 60))Log.info(rc.ttl(cluster_test_list))Log.info(rc.lindex(cluster_test_list, 1))Log.info(rc.llen(cluster_test_list))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.lpush(cluster_test_list, 'list0'))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.linsert(cluster_test_list, 'BEFORE', 'list0', 'list0000'))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.linsert(cluster_test_list, 'AFTER', 'list3', 'list4'))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.lpop(cluster_test_list))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.rpop(cluster_test_list))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.lrem(cluster_test_list, 1, 'list0'))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.lset(cluster_test_list, 0, 't_list'))Log.info(rc.lrange(cluster_test_list, 0, -1))Log.info(rc.delete(cluster_test_list))Log.info('------------------')cluster_test_set = 'test_set'Log.info(rc.sadd(cluster_test_set, 'set1', 'set2', 'set3', 'set3', 'set3'))Log.info(rc.cluster_keyslot(cluster_test_set))Log.info(rc.type(cluster_test_set))Log.info(rc.exists(cluster_test_set))Log.info(rc.ttl(cluster_test_set))Log.info(rc.expire(cluster_test_set, 4 * 60 * 60))Log.info(rc.ttl(cluster_test_set))Log.info(rc.scard(cluster_test_set))Log.info(rc.smembers(cluster_test_set))Log.info(rc.sismember(cluster_test_set, 'set0'))Log.info(rc.sismember(cluster_test_set, 'set2'))Log.info(rc.srem(cluster_test_set, 'set1'))Log.info(rc.smembers(cluster_test_set))Log.info(rc.delete(cluster_test_set))rc.close()

"""
@blog: https://blog.csdn.net/zyooooxie
"""def test_rp2():from redis.cluster import RedisClusterhost2 = 'host2'port = 6666pwd = 'pwd'rc = RedisCluster(host=host2, port=port, password=pwd, decode_responses=True)Log.info(rc)rc.close()

代码：redis-py-cluster

我在使用 redis-py-cluster 2.1.3。

ERROR: redis-py-cluster 2.1.3 has requirement redis<4.0.0, >=3.0.0

"""
@blog: https://blog.csdn.net/zyooooxie
"""# Redis Cluster中有一个16384长度的槽的概念，他们的编号为0、1、2、3 …… 16382、16383。这个槽是一个虚拟的槽，并不是真正存在的。
# 正常工作的时候，Redis Cluster中的每个Master节点都会负责一部分的槽，当有某个key被映射到某个Master负责的槽，那么这个Master负责为这个key提供服务。# 对于cluster而言，不同的key 往往 由不同的node处理。# 集群使用公式 CRC16(key) % 16384来计算键key属于哪个槽。def test_rpc0():"""redis-py-cluster:return:"""from rediscluster import RedisClusterrc = RedisCluster(startup_nodes=[{'host': host1, 'port': port}, {'host': host2, 'port': port}],decode_responses=True, password=pwd)print('{}'.format(rc))# print('{}----{}'.format(type(rc), rc.__dict__))print(rc.cluster_info())                        # 集群的信息# 每个节点在集群中都有一个独一无二的 ID ， 该 ID 是一个十六进制表示的 160 位随机数， 在节点第一次启动时由 /dev/urandom 生成。# print(rc.cluster_nodes())                       # 集群当前已知的所有节点（node），以及这些节点的相关信息print(rc.cluster_slots())                       # 查看slot和节点的对应关系mec_key = 'mec_key'mec_key_slot = rc.cluster_keyslot(mec_key)print(mec_key_slot)                              # 计算key 应该被放置在哪个槽上print(rc.exists(mec_key), '存在的key-ttl是1小时', rc.ttl(mec_key))slot_len = rc.cluster_countkeysinslot(mec_key_slot)print(slot_len)                                 # 返回 槽slot_id 目前包含的键值对数量print(rc.cluster_get_keys_in_slot(mec_key_slot, 5))    # 返回 num_keys个 槽slot 的键print(rc.cluster_get_keys_in_slot(mec_key_slot, slot_len))rc.close()

"""
@blog: https://blog.csdn.net/zyooooxie
"""def test_rpc1():"""redis-py-cluster:return:"""from rediscluster import ClusterConnectionPoolfrom rediscluster import RedisClusterccp = ClusterConnectionPool(startup_nodes=[{'host': host1, 'port': port},{'host': host2, 'port': port}],decode_responses=True, password=pwd)rc = RedisCluster(connection_pool=ccp)print('{}'.format(rc))print('{}----{}'.format(type(rc), rc.__dict__))print(f'{rc.cluster_info()}')print(f'{rc.cluster_nodes()}')print(f'{rc.cluster_slots()}')rc.close()

个人常用环境是使用redis-py；虚拟环境装的redis-py-cluster。

本文链接：https://blog.csdn.net/zyooooxie/article/details/123760358

交流技术欢迎+QQ 153132336 zy
个人博客 https://blog.csdn.net/zyooooxie