1. 为什么需要改进
当 ZSet 中存在相同分数 (score) 的元素时,单纯使用分数作为偏移会导致数据漏查或重复。例如:
- 多条记录具有相同时间戳(作为分数)
- 分页查询时可能跳过相同分数的元素
- 或重复查询相同分数的元素
改进方案:结合最后分数 (lastScore) 和相同分数内的偏移量 (offset) 实现精确滚动。
2. 实现代码
2.1 依赖配置
确保 pom.xml
包含 Redis 依赖:
xml
<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
2.2 服务实现类
使用StringRedisTemplate的高级ZSet滚动查询
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.ZSetOperations;
import org.springframework.stereotype.Service;
import java.util.*;@Service
public class ZSetScrollService {private final StringRedisTemplate stringRedisTemplate;// ZSet键名(可根据业务调整)private static final String ZSET_KEY = "articles:zset";public ZSetScrollService(StringRedisTemplate stringRedisTemplate) {this.stringRedisTemplate = stringRedisTemplate;}/*** 高级滚动查询(处理相同分数场景)* @param lastScore 上一次查询的最后分数,首次查询传0* @param offset 相同分数内的偏移量,首次查询传0* @param pageSize 每页大小* @return 包含数据、最后分数、偏移量和是否有更多数据的Map*/public Map<String, Object> scrollQuery(double lastScore, int offset, int pageSize) {List<String> result = new ArrayList<>();double newLastScore = lastScore;int newOffset = 0;boolean hasMore = false;// 1. 处理上一次最后分数的剩余元素(如果有偏移)if (offset > 0) {// 获取与lastScore相同分数的所有元素Set<ZSetOperations.TypedTuple<String>> sameScoreTuples = stringRedisTemplate.opsForZSet().rangeByScoreWithScores(ZSET_KEY, lastScore, lastScore);if (sameScoreTuples != null && !sameScoreTuples.isEmpty()) {List<ZSetOperations.TypedTuple<String>> sameScoreList = new ArrayList<>(sameScoreTuples);int remaining = sameScoreList.size() - offset;// 相同分数下还有剩余元素if (remaining > 0) {int take = Math.min(remaining, pageSize);// 从偏移位置开始取元素for (int i = offset; i < offset + take; i++) {result.add(sameScoreList.get(i).getValue());}newOffset = offset + take;// 如果已取完当前分数的所有元素,重置偏移if (newOffset >= sameScoreList.size()) {newOffset = 0;} else {newLastScore = lastScore;}// 如果取的元素小于pageSize,继续从更高分数取if (take < pageSize) {pageSize -= take;} else {// 已取够一页,判断是否有更多数据hasMore = checkHasMore(newLastScore, newOffset);return createResponse(result, newLastScore, newOffset, hasMore);}} else {newOffset = 0; // 相同分数下已无元素,重置偏移}} else {newOffset = 0; // 该分数已无元素,重置偏移}}// 2. 查询更高分数的元素if (pageSize > 0) {// 查询大于lastScore的元素,多查一个用于判断是否有下一页Set<ZSetOperations.TypedTuple<String>> higherTuples = stringRedisTemplate.opsForZSet().rangeByScoreWithScores(ZSET_KEY, lastScore + 1, Double.MAX_VALUE, 0, pageSize + 1);if (higherTuples != null && !higherTuples.isEmpty()) {List<ZSetOperations.TypedTuple<String>> higherList = new ArrayList<>(higherTuples);hasMore = higherList.size() > pageSize;// 确定需要取的元素数量int take = hasMore ? pageSize : higherList.size();// 提取元素for (int i = 0; i < take; i++) {ZSetOperations.TypedTuple<String> tuple = higherList.get(i);result.add(tuple.getValue());newLastScore = tuple.getScore();}// 处理最后一个分数的偏移if (take > 0) {ZSetOperations.TypedTuple<String> lastTuple = higherList.get(take - 1);double currentScore = lastTuple.getScore();// 计算当前分数下的总元素数long totalSameScore = stringRedisTemplate.opsForZSet().count(ZSET_KEY, currentScore, currentScore);// 计算当前分数下已返回的元素数long currentScoreReturned = 0;for (int i = 0; i < take; i++) {if (higherList.get(i).getScore().equals(currentScore)) {currentScoreReturned++;}}// 设置相同分数内的偏移newOffset = (currentScoreReturned < totalSameScore) ? (int) currentScoreReturned : 0;}}}// 最终判断是否有更多数据if (!hasMore) {hasMore = checkHasMore(newLastScore, newOffset);}return createResponse(result, newLastScore, newOffset, hasMore);}/*** 检查是否有更多数据*/private boolean checkHasMore(double lastScore, int offset) {// 1. 检查当前分数下是否还有未返回的元素if (offset > 0) {Long sameScoreCount = stringRedisTemplate.opsForZSet().count(ZSET_KEY, lastScore, lastScore);if (sameScoreCount != null && sameScoreCount > offset) {return true;}}// 2. 检查是否存在更高分数的元素Long higherCount = stringRedisTemplate.opsForZSet().count(ZSET_KEY, lastScore + 1, Double.MAX_VALUE);return higherCount != null && higherCount > 0;}/*** 创建响应结果*/private Map<String, Object> createResponse(List<String> data, double lastScore, int offset, boolean hasMore) {Map<String, Object> response = new HashMap<>(4);response.put("data", data);response.put("lastScore", lastScore);response.put("offset", offset);response.put("hasMore", hasMore);return response;}// ------------------------- 辅助方法 -------------------------/*** 向ZSet添加元素* @param member 元素值(通常为JSON字符串)* @param score 排序分数(如时间戳)*/public Boolean add(String member, double score) {return stringRedisTemplate.opsForZSet().add(ZSET_KEY, member, score);}/*** 获取元素的分数*/public Double getScore(String member) {return stringRedisTemplate.opsForZSet().score(ZSET_KEY, member);}/*** 删除元素*/public Long remove(String... members) {return stringRedisTemplate.opsForZSet().remove(ZSET_KEY, members);}/*** 统计分数范围内的元素数量*/public Long count(double min, double max) {return stringRedisTemplate.opsForZSet().count(ZSET_KEY, min, max);}
}
创建时间:10:10
2.3 控制器使用示例
java
运行
@RestController
@RequestMapping("/api/zset")
public class ZSetController {private final ZSetScrollService scrollService;private final ObjectMapper objectMapper; // 用于JSON序列化public ZSetController(ZSetScrollService scrollService, ObjectMapper objectMapper) {this.scrollService = scrollService;this.objectMapper = objectMapper;}// 添加元素示例(对象转JSON字符串)@PostMapping("/add")public ResponseEntity<?> addElement(@RequestBody Article article) throws JsonProcessingException {// 将对象转为JSON字符串存储String jsonStr = objectMapper.writeValueAsString(article);// 使用时间戳作为分数(确保新元素排在前面可使用负的时间戳)double score = System.currentTimeMillis();scrollService.add(jsonStr, score);return ResponseEntity.ok("添加成功");}// 滚动查询示例@GetMapping("/scroll")public ResponseEntity<?> scroll(@RequestParam(defaultValue = "0") double lastScore,@RequestParam(defaultValue = "0") int offset,@RequestParam(defaultValue = "10") int pageSize) throws JsonProcessingException {Map<String, Object> result = scrollService.scrollQuery(lastScore, offset, pageSize);// 将JSON字符串转为对象(可选)if (result.containsKey("data")) {List<String> jsonList = (List<String>) result.get("data");List<Article> articles = jsonList.stream().map(json -> {try {return objectMapper.readValue(json, Article.class);} catch (JsonProcessingException e) {throw new RuntimeException(e);}}).collect(Collectors.toList());result.put("data", articles);}return ResponseEntity.ok(result);}
}
3. 核心原理说明
双重定位机制:
lastScore
:记录上一次查询的最后一个元素的分数offset
:记录在该分数下已经返回的元素数量,解决相同分数问题
查询流程:
- 先处理上一次最后分数中未返回的元素(基于 offset)
- 再查询更高分数的元素,直到满足分页大小
- 自动计算下一次查询所需的 lastScore 和 offset
关键 Redis 命令:
ZRANGEBYSCORE key min max [LIMIT offset count]
:按分数范围查询ZCOUNT key min max
:统计分数范围内的元素数量ZSCORE key member
:获取元素的分数
4. 使用场景
- 社交媒体时间线(按发布时间排序,可能有相同时间)
- 实时排行榜(按分数排序,可能有相同分数)
- 日志记录查询(按时间戳排序)
- 大数据量有序列表的滚动加载
这种实现既保留了 Redis ZSet 的高性能,又解决了相同分数元素的查询问题,是处理动态有序数据滚动加载的理想方案。