RAFNI

RAFNI (Relabelling And Filtering Noisy Instances) is an algorithm to help neural networks learn when the training set presents label noise. It handles the instances that the algorithm consider as noisy in the training set by using three mechanisms that filter or relabel these instances. The mechanisms use the predictions, the probability of those predictions and the loss values of the instances made by the backbone network. It is important to note that the RAFNI algorithm is independet of the backbone network used. There is two filtering mechanisms and one relabelling mechanism.

The first filtering mechanism only uses the loss value of the instances. The foundation is that the noisy instances tend to have higher loss values than the rest of them. As a result, this mechanism filters out instances that have a loss value above a certain threshold. This threshold is dynamic and will change during training.

The second filtering mechanism depends on how many times an instance has been relabelled. Here we suppose that if the algorithm relabels an instance too many times is because the backbone network is unsure about its class and it is better to remove that instance. Thus, this mechanism filters an instance if it has been relabelled more than a certain number of times.

The relabelling mechanism takes into account the probability predictions of the backbone network. We suppose that if the backbone network predicts another class with a high probability as the training progresses, it is probable that the instance is noisy and its class is indeed the one predicted by the backbone network. As a consequence, the relabelling mechanism changes the class of an instance if the backbone network predicts another class with a probability that is above a certain fixed threshold.

In the following two figures we can see how the RAFNI algorithm affects the training of the backbone network and the flowchart of the algorithm.

A detailed description of the algorithm can be found in the paper:

A. Gómez-Ríos, J. Luengo, F. Herrera, “A robust approach for deep neural networks in presence of label noise: relabelling and filtering instances during training”

which you can cite as:

@misc{gómezríos2021robust,
      title={A robust approach for deep neural networks in presence of label noise: relabelling and filtering instances during training},
      author={Anabel Gómez-Ríos and Julián Luengo and Francisco Herrera},
      year={2021},
      eprint={2109.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

The code of the algorithm can be found on GitHub. If you want to reproduce our experiments using CIFAR, please download the data sets here.

Contact: Anabel Gómez