Skip to content

README.md

Latest commit

 

History

History
51 lines (33 loc) · 4.28 KB

README.md

File metadata and controls

51 lines (33 loc) · 4.28 KB

Model Submodule

This submodule is the core of the experimental framework, containing the definitions of the neural network architectures, the scripts for training and evaluation, and utilities for post-experiment analysis.

Overview

The primary role of this submodule is to implement, train, and analyze various autoencoder-based models for the task of semi-supervised anomaly detection. It is structured to facilitate extensive experimentation with different model architectures, training configurations, and data scenarios (one-vs-all and many-vs-many).

Core Components

The submodule is organized into several key functional areas:

1. Model Architectures

These files define the neural network architectures used in the experiments.

  • deep_ae.py: Defines the DeepAE, a standard fully-connected autoencoder. This serves as the foundational model for many experiments.
  • deep_ae_conv.py: Defines the DeepConvAE, a convolutional autoencoder designed for image data, leveraging convolutional layers to capture spatial features.
  • deep_ae_var.py: Defines the DeepVAE, a Variational Autoencoder. This is a generative model that learns a probabilistic latent space, regularized by a KL-divergence term.
  • deep_ae_linear.py & deep_ae_tanh.py: These are variants of the DeepAE that use linear (identity) and tanh activation functions, respectively. They are used for comparative analysis to study the effect of non-linearities.

2. Training and Evaluation Scripts

These scripts are the main entry points for running experiments. They implement the complete pipeline of data loading, model training, evaluation, and results logging.

  • deep_ae_sad_trainer.py: The primary trainer for one-vs-all anomaly detection scenarios using the fully-connected autoencoder variants.
  • deep_ae_conv_sad_trainer.py: The trainer for the DeepConvAE model in a one-vs-all setup.
  • deep_ae_var_sad_trainer.py: The trainer for the DeepVAE model, which includes a VAE-specific loss function (reconstruction loss + KL divergence).
  • deep_ae_sad_multi_class_trainer.py: The main script for conducting large-scale, many-vs-many experiments, iterating through numerous undersampling techniques and anomaly configurations.

3. Analysis and Post-Processing

These scripts are used to analyze the results generated by the training scripts.

  • deep_ae_sad_latent_analysis.py: A script for visualizing the 2D latent space of trained models to qualitatively assess the separation between normal and anomalous classes.
  • json_metrics_to_latex.py: A utility script to parse the raw JSON output from experiments, aggregate results, and format them into LaTeX tables for inclusion in the thesis document.
  • plot_*.py Scripts: A collection of scripts (plot_auc_time_centr_&_rand.py, plot_auc_time_clust_based.py, plot_near_miss_v1_chart.py) that read processed JSON data and generate plots comparing the performance (AUC, Time, AUC/Time Rate) of different undersampling methods against their key hyperparameters.

Directory Structure

  • dumps/: This directory is used to store the trained model weights (.pt files) after the completion of a training run.
  • json_metrics/: This directory contains the results of all experiments.
    • staged/: Raw JSON output from individual experimental runs are saved here.
    • bests/ & multi_class/: These folders contain aggregated and structured JSON files, processed by the analysis scripts for generating tables and plots.

Workflow

  1. Run Experiments: Execute one of the trainer scripts (e.g., python -m torch-AE-SAD.model.deep_ae_sad_multi_class_trainer) to start an experimental run based on the configurations defined within the script.
  2. Collect Results: The trainer saves raw metrics from each run into the json_metrics/staged/ directory. At the end of a full suite, it aggregates these into a single timestamped JSON file in json_metrics/.
  3. Process and Analyze: Use json_metrics_to_latex.py to process the aggregated JSON files. This can generate LaTeX tables or create new structured JSON files in the bests/ and multi_class/ directories.
  4. Visualize: Run the plot_*.py scripts to generate plots from the structured JSON data, allowing for visual analysis of the experimental results.