Expanding annotated data with informed labels for weak supervision

EasyChair Preprint 1582

Abstract

In this paper we present an instance of the weak supervision paradigm, the multi-uncertain learning scenario. Our multi-uncertain scenario has three facets all related to the labels and the human labelers: there are multiple labels per instance, the gold standard label may not be included in this label set, and the identity of the labelers is unknown. In order to avoid disposing of expensive and potentially useful labels, we outline a method of adding informed labels to a label set by using label propagation. Under the smoothness assumption, we are able to introduce new, informative labels into an existing training label set to improve performance under highly uncertain constraints. For complex classification tasks with three or more classes, we report that this method of adding informed labels is capable of producing classifiers with high accuracy and low complexity, despite being trained on these multi-uncertain datasets.

Keyphrases: Clustering, Multi uncertain, Uncertain Labels, label propagation, noisy labels, weak supervision

@booklet{EasyChair:1582,
  author    = {Eura Shin and Sam Berglin and Jacob Furst and Daniela Raicu},
  title     = {Expanding annotated data with informed labels for weak supervision},
  howpublished = {EasyChair Preprint 1582},
  year      = {EasyChair, 2019}}