Activiti version 8.1.x: Not getting inbound variables in connector

Question

Spring boot version 3.6 activiti version : 8.1.0

Context: I have a POC application created where i need to simulate user task in activiti via API, for which i have created a connector on update Approval as highligted in below xml file. Now, when i start worklfow, the system is sucessfully able to register the workflow and when i fetch all pending task for that given user who can approve. but when i try to push the approval using below code with setting the variables for the task, few things i'm unable to figure out how to do it the right way 1.

List<VariableInstance> variables = taskRuntime.variables(TaskPayloadBuilder.variables().withTaskId(taskId).build());

This above code is not able to get variables which were sent to workflow while it was created

workaround to which was done using below code

Map<String, Object> vars = taskService.getVariables(taskId);

Below piece of code i done see any persisted entery in RU_Variables table

taskRuntime.complete(TaskPayloadBuilder.complete()
                    .withTaskId(taskId)
                    .withVariable(VAR_REGISTRATION_REQUEST, contentToProcess)
                    .withVariable("comment", taskRequest.getComment())
                    .withVariable("approved", taskRequest.isApproved()).build());

3. The task complete triggered on 2. triggeres below code where i the size of getInBoundVariables is 0

@Bean
    public Connector updateRegistrationStatusConnector() {
        return integrationContext -> {
            Map<String, Object> inBound = integrationContext.getInBoundVariables();

            if (log.isDebugEnabled()) {
                log.debug("inBoundVariables --> {}", inBound);
            }
            Map<String, Object> inBoundVariables = (Map<String, Object>) inBound.get("inbound");
            UserRegistrationRequest registrationRequest = (UserRegistrationRequest) inBoundVariables.get(VAR_REGISTRATION_REQUEST);
            String comment = (String) inBoundVariables.get("comment");
            Boolean approved = (Boolean) inBoundVariables.get("approved");
           saveRegistrationStatus(registrationRequest.getUserId(), comment, approved);
            return integrationContext;
        };
    }

public Task completeTask(String taskId, TaskCompletionRequest taskRequest) {
        Task task = null;
        try {
            ObjectMapper objectMapper = new ObjectMapper();
            task = taskRuntime.task(taskId);

            //Map<String,Object> variables = runtimeService.getVariables(task.getProcessDefinitionId());
            List<VariableInstance> variables = taskRuntime.variables(TaskPayloadBuilder.variables().withTaskId(taskId).build());
            Map<String, Object> vars = taskService.getVariables(taskId);
            //UserRegistrationRequest contentToProcessNode = Objects.requireNonNull(variables.stream().filter(var -> VAR_REGISTRATION_REQUEST.equals(var.getName())).findFirst().orElse(null)).getValue();
            UserRegistrationRequest contentToProcess = (UserRegistrationRequest) vars.get(VAR_REGISTRATION_REQUEST);

            /*CreateTaskVariablePayloadBuilder builder = new CreateTaskVariablePayloadBuilder();
            taskRuntime.createVariable(builder.withTaskId(taskId)
                    .withVariable("comment", taskRequest.getComment())
                    .withVariable("approved", taskRequest.isApproved()).build());*/

            HashMap<String,Object> map  = new HashMap<>();
            map.put(VAR_REGISTRATION_REQUEST, contentToProcess);
            map.put("comment", taskRequest.getComment());
            map.put("approved", taskRequest.isApproved());
           CreateTaskVariablePayloadBuilder builder = new CreateTaskVariablePayloadBuilder();
           builder.withVariable("inbound",map);
            task = taskRuntime.complete(TaskPayloadBuilder.complete()
                    .withTaskId(taskId)
                    .withVariable(VAR_REGISTRATION_REQUEST, contentToProcess)
                    .withVariable("comment", taskRequest.getComment())
                    .withVariable("approved", taskRequest.isApproved()).build());
            builder.withTaskId(task.getId());
            taskRuntime.createVariable(builder.build());

        } catch (Exception e) {
            log.error("error",e);
            throw new UserRegistrationTaskException(e.getMessage());
        }

        return task;
    }

below is the Activiti.bpmn20.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<definitions xmlns="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:activiti="http://activiti.org/bpmn" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:omgdc="http://www.omg.org/spec/DD/20100524/DC" xmlns:omgdi="http://www.omg.org/spec/DD/20100524/DI" xmlns:tns="http://www.activiti.org/test" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" expressionLanguage="http://www.w3.org/1999/XPath" id="m1603026671390" name="" targetNamespace="http://www.activiti.org/test" typeLanguage="http://www.w3.org/2001/XMLSchema">
    <process id="userRegistrationProcess" isClosed="false" isExecutable="true" processType="None">
        <startEvent id="start" name="StartEvent"/>
        <userTask activiti:exclusive="true" id="sendApproval" name="Send Approval" activiti:assignee="${registrationRequest.approverId}" />
        <sequenceFlow id="_4" sourceRef="start" targetRef="sendApproval"/>
        <serviceTask activiti:exclusive="true" id="updateApprovalStatus" name="Update Approval Status" implementation="***updateRegistrationStatusConnector***" />
        <sequenceFlow id="_6" sourceRef="sendApproval" targetRef="updateApprovalStatus"/>
        <exclusiveGateway gatewayDirection="Unspecified" id="approvalCheck" name="Approved?">
        </exclusiveGateway>
        <sequenceFlow id="_8" sourceRef="updateApprovalStatus" targetRef="approvalCheck"/>
        <endEvent id="end" name="EndEvent"/>
        <sequenceFlow id="_10" name="no" sourceRef="approvalCheck" targetRef="end">
            <conditionExpression xsi:type="tFormalExpression">
                <![CDATA[${approved == false}]]>
            </conditionExpression>
        </sequenceFlow>
        <userTask activiti:exclusive="true" id="updateProfile" name="Update Profile" activiti:assignee="${registrationRequest.userId}"/>
        <sequenceFlow id="_2" name="yes" sourceRef="approvalCheck" targetRef="updateProfile">
            <conditionExpression xsi:type="tFormalExpression">
                <![CDATA[${approved == true}]]>
            </conditionExpression>
        </sequenceFlow>
        <sequenceFlow id="_3" sourceRef="updateProfile" targetRef="end"/>
    </process>
    <bpmndi:BPMNDiagram documentation="background=#3C3F41;count=1;horizontalcount=1;orientation=0;width=842.4;height=1195.2;imageableWidth=832.4;imageableHeight=1185.2;imageableX=5.0;imageableY=5.0" id="Diagram-_1" name="New Diagram">
        <bpmndi:BPMNPlane bpmnElement="userRegistrationProcess">
            <bpmndi:BPMNShape bpmnElement="start" id="Shape-start">
                <omgdc:Bounds height="32.0" width="32.0" x="75.0" y="90.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="32.0" width="32.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="sendApproval" id="Shape-sendApproval">
                <omgdc:Bounds height="55.0" width="85.0" x="205.0" y="80.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="55.0" width="85.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="updateApprovalStatus" id="Shape-updateApprovalStatus">
                <omgdc:Bounds height="55.0" width="85.0" x="365.0" y="80.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="55.0" width="85.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="approvalCheck" id="Shape-approvalCheck" isMarkerVisible="false">
                <omgdc:Bounds height="32.0" width="32.0" x="540.0" y="90.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="32.0" width="32.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="end" id="Shape-end">
                <omgdc:Bounds height="32.0" width="32.0" x="660.0" y="90.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="32.0" width="32.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNShape bpmnElement="updateProfile" id="Shape-updateProfile">
                <omgdc:Bounds height="55.0" width="85.0" x="515.0" y="215.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="55.0" width="85.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNShape>
            <bpmndi:BPMNEdge bpmnElement="_2" id="BPMNEdge__2" sourceElement="approvalCheck" targetElement="_11">
                <omgdi:waypoint x="556.0" y="122.0"/>
                <omgdi:waypoint x="556.0" y="215.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="_3" id="BPMNEdge__3" sourceElement="_11" targetElement="_9">
                <omgdi:waypoint x="600.0" y="242.5"/>
                <omgdi:waypoint x="675.0" y="175.0"/>
                <omgdi:waypoint x="675.0" y="121.96871942267131"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="_4" id="BPMNEdge__4" sourceElement="start" targetElement="sendApproval">
                <omgdi:waypoint x="107.0" y="106.0"/>
                <omgdi:waypoint x="205.0" y="107.5"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="_6" id="BPMNEdge__6" sourceElement="sendApproval" targetElement="updateApprovalStatus">
                <omgdi:waypoint x="290.0" y="107.5"/>
                <omgdi:waypoint x="365.0" y="107.5"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="_8" id="BPMNEdge__8" sourceElement="updateApprovalStatus" targetElement="approvalCheck">
                <omgdi:waypoint x="450.0" y="107.5"/>
                <omgdi:waypoint x="540.0" y="106.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
            <bpmndi:BPMNEdge bpmnElement="_10" id="BPMNEdge__10" sourceElement="approvalCheck" targetElement="_9">
                <omgdi:waypoint x="572.0" y="106.0"/>
                <omgdi:waypoint x="660.0" y="106.0"/>
                <bpmndi:BPMNLabel>
                    <omgdc:Bounds height="0.0" width="0.0" x="0.0" y="0.0"/>
                </bpmndi:BPMNLabel>
            </bpmndi:BPMNEdge>
        </bpmndi:BPMNPlane>
    </bpmndi:BPMNDiagram>
</definitions>

Note: the same code with activiti 7.1.123 & spring boot 2.1 is working perfrectly

1. I did some research around the documentation on 8.x activiti, i didn't find any. if someone can help me with that will also be helpful
2. when i dig deeper into the code the one thing i was able to find is activiti task is looking only for local variables for the code

List<VariableInstance> variables = taskRuntime.variables(TaskPayloadBuilder.variables().withTaskId(taskId).build());

for which i could get the global variables using taskService