cnlpt.cnlp_processors module

Module containing auto processor class, which infers labels, task type, and output modes for tasks and datasets that use a few conventional formats.

class cnlpt.cnlp_processors.AutoProcessor

Bases: DataProcessor

A special type of processor that tries to infer the details about the dataset from the artifacts that are present in the data directory.

TODO - add documentation of the expected file formats for json and csv defaults

__init__(data_dir, tasks=None, max_train_items=-1)

get_train_examples()

Gets a collection of [InputExample] for the train set.

get_dev_examples()

Gets a collection of [InputExample] for the dev set.

get_test_examples()

Gets a collection of [InputExample] for the test set.

get_labels()

Gets the list of labels for this data set.