modbot.training.training.clean_log

modbot.training.training.clean_log(config, infile, outfile, filt=False, links=False, correct=False, check_phrases=False, check_probs=False, bounds=None)[source]

Clean raw log file to prepare for training

Parameters

  • infile (str) – Path to log file

  • outfile (str) – Path to cleaned output file

  • config (RunConfig) – Class storing configuration parameters

  • filt (bool, optional) – Whether to filter input data for emotes and repetitions, by default False classifications, by default False

  • links (bool, optional) – Whether to label messages with links, by default False

  • correct (bool, optional) – Whether to santitize messages. e.g. remove repetitions, usernames, and some special characters

  • check_phrases (bool, optional) – Whether to check messages for reclassification, by default False

  • check_probs (bool, optional) – Whether to check messages for reclassification based on model predictions, by default False

  • bounds (list, optional) – List specifying start and end index of messages to review, by default None