Effect of each task in Multi-task learning-Task Affinity calculation?

Question

I have designed a network for multi-task learning. Here I have some shared layers and specific layers for 5 different tasks.

 input_dim_train=X_train.shape[1]
 n_input_dim = int(input_dim_train)
 shared_model = Dense(input_dim_train)(inputs)
 shared_model = Dense(n_input_dim, activation='relu')(shared_model)
 shared_model = BatchNormalization(input_dim=n_input_dim)(shared_model)
 shared_model = Dropout(0.1)(shared_model)

 shared_model = Dense(int(n_input_dim//4), activation='relu')(shared_model)
 shared_model = BatchNormalization(input_dim=n_input_dim)(shared_model)
 shared_model = Dropout(0.1)(shared_model)

 shared_model = Dense(int(n_input_dim//12), activation='relu')(shared_model)
 shared_model = BatchNormalization(input_dim=n_input_dim)(shared_model)
 shared_model = Dropout(0.1)(shared_model)

 shared_model = Dense(int(n_input_dim//20), activation='relu')(shared_model)
 shared_model = BatchNormalization()(shared_model)
 shared_model = Dropout(0.1)(shared_model)

 output1 = Dense(3, activation='softmax',name='target_1',)(shared_model)
 output2 = Dense(2, activation='softmax',name='target_2')(shared_model)
 output3 = Dense(4, activation='softmax',name='target_3')(shared_model)
 output4 = Dense(2, activation='softmax',name='target_4')(shared_model)
 output5 = Dense(3, activation='softmax',name='target_5')(shared_model)

 model = Model(inputs=inputs, outputs=[output1, output2,output3,output4,output5])

 model.compile(optimizer='adam', loss='categorical_crossentropy',metrics=['accuracy',f1_metric])



 history=model.fit(X_train, [y_train_task1, y_train_task2,y_train_task3,y_train_task4,y_train_task5],
              validation_data=(X_test,[y_test_task1,y_test_task2,y_test_task3,y_test_task4,y_test_task5]),verbose='auto',epochs=n_epoch,batch_size=n_batch_size)

 task1_pred,task2_pred,task3_pred,task4_pred,task5_pred  = model.predict(X_test)

 

The network somehow works well and gives the results as follows:

Now, my question is: How can I measure the effect of e.g., Task1 on Task2 in this multi task learning? Indeed In want to calculate the task affinity score between the tasks in this problem. Is there any way to calculate the inter-task affinity score during the training process--over different epochs?