cnlpt.train_system module

Finetuning the library models for sequence classification on clinical NLP tasks

class cnlpt.train_system.CnlpTrainingArguments

Bases: TrainingArguments

Additional arguments specific to this class. See all possible arguments in transformers.TrainingArguments or by passing the --help flag to this script.

__init__(output_dir, overwrite_output_dir=False, do_train=False, do_eval=False, do_predict=False, evaluation_strategy='no', prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, per_gpu_train_batch_size=None, per_gpu_eval_batch_size=None, gradient_accumulation_steps=1, eval_accumulation_steps=None, eval_delay=0, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, lr_scheduler_type='linear', warmup_ratio=0.0, warmup_steps=0, log_level='passive', log_level_replica='passive', log_on_each_node=True, logging_dir=None, logging_strategy='steps', logging_first_step=False, logging_steps=500, logging_nan_inf_filter=True, save_strategy='steps', save_steps=500, save_total_limit=None, save_on_each_node=False, no_cuda=False, use_mps_device=False, seed=42, data_seed=None, jit_mode_eval=False, use_ipex=False, bf16=False, fp16=False, fp16_opt_level='O1', half_precision_backend='auto', bf16_full_eval=False, fp16_full_eval=False, tf32=None, local_rank=-1, xpu_backend=None, tpu_num_cores=None, tpu_metrics_debug=False, debug='', dataloader_drop_last=False, eval_steps=None, dataloader_num_workers=0, past_index=-1, run_name=None, disable_tqdm=None, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, sharded_ddp='', fsdp='', fsdp_min_num_params=0, fsdp_transformer_layer_cls_to_wrap=None, deepspeed=None, label_smoothing_factor=0.0, optim='adamw_hf', adafactor=False, group_by_length=False, length_column_name='length', report_to=None, ddp_find_unused_parameters=None, ddp_bucket_cap_mb=None, dataloader_pin_memory=True, skip_memory_metrics=True, use_legacy_prediction_loop=False, push_to_hub=False, resume_from_checkpoint=None, hub_model_id=None, hub_strategy='every_save', hub_token=None, hub_private_repo=False, gradient_checkpointing=False, include_inputs_for_metrics=False, fp16_backend='auto', push_to_hub_model_id=None, push_to_hub_organization=None, push_to_hub_token=None, mp_parameters='', auto_find_batch_size=False, full_determinism=False, torchdynamo=None, ray_scope='last', ddp_timeout=1800, evals_per_epoch=-1, final_task_weight=1.0, freeze=-1.0, arg_reg=-1, bias_fit=False)
class cnlpt.train_system.ModelArguments

Bases: object

Arguments pertaining to which model/config/tokenizer we are going to fine-tune from. See all possible arguments by passing the --help flag to this script.

__init__(model='cnlpt', encoder_name='roberta-base', config_name=None, tokenizer_name=None, cache_dir=None, layer=-1, token=False, num_rel_feats=12, head_features=64, cnn_embed_dim=100, cnn_num_filters=25, cnn_filter_sizes=<factory>, lstm_embed_dim=100, lstm_hidden_size=100, use_prior_tasks=False, hier_num_layers=2, hier_hidden_dim=2048, hier_n_head=8, hier_d_k=8, hier_d_v=96, hier_dropout=0.1)
cnlpt.train_system.main(json_file=None, json_obj=None)

See all possible arguments in transformers.TrainingArguments or by passing the –help flag to this script.

We now keep distinct sets of args, for a cleaner separation of concerns.

Parameters:

  • json_file (Optional[str]) – if passed, a path to a JSON file to use as the model, data, and training arguments instead of retrieving them from the CLI (mutually exclusive with json_obj)

  • json_obj (Optional[dict]) – if passed, a JSON dictionary to use as the model, data, and training arguments instead of retrieving them from the CLI (mutually exclusive with json_file)

Return type:

Dict[str, Dict[str, Any]]

Returns:

the evaluation results (will be empty if --do_eval not passed)