callback of when RecyclerView has finished loading its content is loaded more than once and raising exception

Question

I am using this Listener in order to acknowledge when RecyclerView has finished loading its views but this listener is also called after I'm moving to another fragment and its raising an exception.

options = view.findViewById(R.id.poll_question_multi_choice_rv); //My RecyclerView
globalLayoutListener= () -> 
setAnswer(viewModel.getPollQuestionWithAnswer().getValue().answer.getAnswer());
options.setLayoutManager(new LinearLayoutManager(getContext()));
options.setHasFixedSize(true);
adapter = new PollQuestionMultiChoiceAdapter(viewModel,getLayoutInflater());
options.setAdapter(adapter);
options.getViewTreeObserver().addOnGlobalLayoutListener(globalLayoutListener);

setAnswer method inside the listener has an impact on the fragment I'm using, and once this listener is called right before I move to another fragment, it raises this exception:

E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.appproject, PID: 6703
    java.lang.IllegalStateException: Fragment FragmentPollQuestionMultiChoice{11c6938} (2ecb1a90-4574-40ec-a93d-7b96886507f1) not attached to a context.