java - 如何跟踪在多个服务器中运行的异步任务的进度状态

Question

我在 Spring Boot 中运行多个异步任务。这些任务读取 Excel 文件并将所有数据插入数据库。

当前端发出请求时，任务就会启动。然后前端会定期持续请求任务的进度状态。

我需要跟踪每项任务的进度并知道它们何时完成。

这是接收任务请求并轮询其进度状态的 Controller 文件:

public class TaskController {

    @RequestMapping(method = RequestMethod.POST, value = "/uploadExcel")
    public ResponseEntity<?> uploadExcel(String excelFilePath) {
        String taskId = UUID.randomUUID().toString();
        taskAsyncService.AsyncManager(id, excelFilePath);

        HashMap<String, String> responseMap = new HashMap<>();
        responeMap.put("taskId",taskId);
        return new ResponseEntity<>(responseMap, HttpStatus.ACCEPTED);
    }

    // This will be polled to get progress of tasks being executed
    @RequestMapping(method = RequestMethod.GET, value = "/tasks/progress/{id}")
    public ResponseEntity<?> getTaskProgress(@PathVariable String taskId) {
        HashMap<String, String> map = new HashMap<>();

        if (taskAsyncService.containsTaskEntry(id) == null) {
            map.put("Error", "TaskId does not exist");
            return new ResponseEntity<>(map, HttpStatus.BAD_REQUEST);
        }

        boolean taskProgress = taskAsyncService.getTaskProgress(taskId);

        if (taskProgress) {
            map.put("message", "Task complete");
            taskAsyncService.removeTaskProgressEntry(taskId);
            return new ResponseEntity<>(map, HttpStatus.OK);
        }

        //Otherwise task is still running
        map.put("progressStatus", "Task running");
        return new ResponseEntity<>(map, HttpStatus.PARTIAL_CONTENT);

    }
}

这是执行异步任务的代码。

public class TaskAsyncService {
    private final AtomicReference<ConcurrentHashMap<String, Boolean>> isTaskCompleteMap = new AtomicReference<ConcurrentHashMap<String, Boolean>>();

    protected boolean containsTaskEntry(String taskId) {
        if (isTaskCompleteMap.get().get(taskId) != null) {
            return true;
        }
        return false;
    }

    protected boolean getTaskProgress(String taskId, String excelFilePath) {
        return isTaskCompleteMap.get().get(taskId);
    }

    protected void removeTaskProgressEntry(String taskId) {
        if (isTaskCompleteMap.get() != null) {
            isTaskCompleteMap.get().remove(taskId);
        }
    }

    @Async
    public CompletableFuture<?> AsyncManager(String taskId) {
        HashMap<String, String> map = new HashMap<>();

        //Add a new entry into isTaskCompleteMap
        isTaskCompleteMap.get().put(taskId, false);

        //Insert excel rows into database

        //Task completed set value to true
        isTaskCompleteMap.get().put(taskId, true);
        map.put("Success", "Task completed");

        return CompletableFuture.completedFuture(map);
    }
}

I am using AWS EC2 with a load balancer. Therefore, sometimes a polling request gets handled by a newly spawned server which cannot access the isTaskCompleteMap and returns saying that "TaskId does not exist".

在这种情况下如何跟踪任务的状态？我知道我需要一个分布式数据结构，但不知道它是什么类型以及如何实现它。

Answer 1

您可以使用Hazelcast或类似的分布式解决方案(Redis等)。

map - https://docs.hazelcast.org/docs/3.0/manual/html/ch02.html#Map

1. 使用 hazelcast 的分布式 map 而不是 CHM。
2. 从这样的映射中获取应该返回任务，即使它们正在另一个 Pod(服务器)上处理