{"name":"napari-unet-assistant","display_name":"U-Net Assistant","visibility":"public","icon":"","categories":[],"schema_version":"0.2.1","on_activate":null,"on_deactivate":null,"contributions":{"commands":[{"id":"napari-unet-assistant.toolbox","title":"U-Net Assistant Toolbox","python_name":"napari_unet_assistant.widgets.toolbox_widget:toolbox_widget","short_title":null,"category":null,"icon":null,"enablement":null}],"readers":null,"writers":null,"widgets":[{"command":"napari-unet-assistant.toolbox","display_name":"U-Net Assistant","autogenerate":false}],"sample_data":null,"themes":null,"menus":{},"submenus":null,"keybindings":null,"configuration":[]},"package_metadata":{"metadata_version":"2.4","name":"napari-unet-assistant","version":"0.6.5","dynamic":["license-file"],"platform":null,"supported_platform":null,"summary":"a napari plugin for TIFF-based 2D and 3D U-Net segmentation workflows.","description":"# napari-unet-assistant\n\n`napari-unet-assistant` is a napari plugin for supervised 2D and 3D U-Net segmentation workflows.\n\nIt is designed for users who already have image-mask training data and want to pair files, train a U-Net model, run inference, and inspect results directly inside napari.\n\nThis plugin is separate from SAM-based annotation workflows. Its focus is conventional supervised U-Net training from existing image-mask pairs.\n\n## What's new in 0.6.5\n\n- Added recursive dataset-folder auto pairing for TIFF datasets with `images`/`masks`, `raw`/`labels`, mixed-folder, and nested layouts.\n- Added folder-name role hints so dataset folders can guide image-mask pairing even when filenames are less explicit.\n- Added configurable augmentation presets: `none`, `conservative`, `balanced`, and `strong`.\n- Added custom augmentation controls for flips, rotation, shear, scale, brightness jitter, and Gaussian noise.\n- Added model-capacity options for standard, large, and xlarge U-Net widths.\n- Added a model registry with built-in U-Net, MONAI, nnU-Net, and segmentation-models-pytorch backend hooks.\n- Added separate model-family, backbone/encoder, and encoder-weight controls for clearer U-Net variant testing.\n- Added a Model Sweep tab for running multiple backend, model-family, backbone, weight, and capacity combinations sequentially.\n- Added a dedicated nnU-Net pipeline path that exports training pairs, runs nnU-Net v2 CLI training, and imports prediction TIFFs.\n- Added early stopping with one learning-rate reduction on validation plateau before stopping after a second plateau.\n- Added a Training Monitor tab with structured run, pair, metric, and output-path summaries.\n- Changed single new training runs to create timestamped child run folders under the selected output parent so repeated runs do not overwrite history.\n- Improved resume metadata loading so epochs, batch size, validation split, and early-stopping controls are restored as editable defaults.\n- Clarified model selection wording with user-facing labels: model source, architecture, feature extractor, pretrained weights, and model size.\n- Made Model Sweep write completed/failed result rows immediately from the worker thread so `sweep_results.csv` stays current through long sweeps.\n- Added a Model Sweep SMP feature-extractor checklist so users can choose small/fast, default, or larger encoders instead of sweeping every SMP encoder blindly.\n- Added an in-app Model Sweep results comparison view with a sortable table, metric bars, and a shortcut to use the selected run for inference.\n- Added multiclass mask label-range validation before training starts.\n- Added saving and loading of augmentation settings in run configuration metadata.\n- Added a training stop button that cancels after the current batch, discards the interrupted model state, and clears GPU cache when available.\n- Added a U-Net architecture preview for the selected 2D/3D mode and output-channel configuration.\n- Added short tooltips across the main controls to explain settings in place.\n- Improved the training UI layout with clearer tabs and more stable dock-widget sizing.\n\n## Highlights\n\n- TIFF-first 2D and 3D U-Net training\n- Binary and multiclass segmentation\n- Clear binary output-channel wording: binary uses one foreground channel while background remains label 0\n- Recursive smart image-mask pairing\n- Pair review with confidence, reason, and shape-check status\n- Patch-based training with configurable patch size and overlap\n- Optional empty-mask patch inclusion\n- Configurable augmentation presets and custom augmentation controls\n- Standard, large, and xlarge U-Net capacity options\n- Optional model backends for MONAI, nnU-Net, and segmentation-models-pytorch\n- Model Sweep for batch training and validation comparison across model choices\n- Training Monitor for structured current-run status, latest/best metrics, and output paths\n- 80/20 validation split\n- Continue training from a previous run\n- Training cancellation from the UI\n- Single-image and folder inference\n- 2D image and 3D volume prediction\n- TIFF prediction export\n- napari-based visualization and QC\n\n## Smart image-mask pairing\n\nThe plugin can pair training data from:\n\n- a dataset root scanned recursively\n- separate image and mask folders\n- one mixed folder containing both images and masks\n- a manual CSV file\n\nDataset-root auto scan supports common layouts such as:\n\n- `images/` + `masks/`\n- `raw/` + `labels/`\n- nested TIFF folders under one dataset root\n- one mixed folder containing `sample.tif` + `sample_mask.tif`\n\nSupported naming patterns include:\n\n- `sample.tif` + `sample_mask.tif`\n- `sample_1.tif` + `sample_2.tif`\n- `sample_image.tif` + `sample_mask.tif`\n- `sample_raw.tif` + `sample_label.tif`\n\nAfter scanning, the plugin shows each proposed pair with confidence, reason, and shape-check status. Ambiguous or invalid pairs are reported instead of being silently used for training.\n\nFolder names such as `images`, `raw`, `masks`, and `labels` can also provide role hints during recursive dataset scans.\n\n## Augmentation\n\nTraining supports augmentation presets and custom controls:\n\n- `none`: no augmentation\n- `conservative`: flips, small rotations/scales, and light brightness jitter\n- `balanced`: stronger rotation/scale, shear, brightness jitter, and light Gaussian noise\n- `strong`: wider rotation/scale/shear ranges, stronger brightness jitter, and stronger Gaussian noise\n- `custom`: user-selected flips, rotation, shear, scale, brightness, and noise settings\n\nThe selected augmentation configuration is saved in each run folder's `config.json` and restored when loading run metadata.\n\n## Training controls\n\nThe training panel includes a stop button for cancelling an active training run. Cancellation is checked between batches, so the current batch may finish before the run stops.\n\nWhen a run is stopped, the interrupted model state is discarded and GPU cache is cleared when available.\n\nThe validation tab also includes early stopping controls. When enabled, the plugin monitors validation Dice. If validation Dice does not improve by at least the configured minimum gain for the patience window, the learning rate is reduced once. If validation Dice still does not improve for another patience window after that reduction, training stops early and keeps the best validation-Dice checkpoint. Validation uses augmentation disabled, even when training augmentation is enabled. When at least two image-mask pairs are available, validation is split by image/mask pair so training and validation patches come from different source images. Single-pair datasets fall back to patch-level validation and record that fallback in `validation.json`. The per-epoch `history.csv` records `learning_rate`, `lr_reduced`, and `early_stopped`, and `validation.json` records split level plus augmentation state.\n\nWhen loading resume metadata, the plugin restores architecture/data settings for compatibility and also restores editable schedule controls such as epochs, batch size, validation split, and early-stopping settings. In continue-training mode, the restored epoch value is the default number of additional epochs to run, not a total lifetime epoch target.\n\n## Training Monitor\n\nThe `Monitor` tab in Training Options gives an organized current-run view instead of another scrolling log. It shows the active run configuration, pair-scan summary, latest validation metrics, best validation metrics, and important output paths such as `config.json`, `history.csv`, `summary.json`, `best_model.pt`, and prediction folders.\n\nThis is different from Step 4 `Results`: Step 4 remains the chronological message log for detailed events, warnings, pair reports, and command output. The Monitor tab is a compact dashboard that overwrites the same fields as training progresses, so users can leave training running and come back to the current state quickly. It also includes lightweight actions to preview one selected/first OK image-mask pair in napari and to reuse the current run folder in the Inference section.\n\n## Model size\n\nTraining can use standard, large, or xlarge U-Net widths. For 2D models, these use base channel widths of 32, 64, and 128. Larger models can learn more complex boundaries, but they need more GPU memory and may require a smaller batch size.\n\n## Model sources\n\nThe default model source is the built-in U-Net and works without extra model packages. MONAI, segmentation-models-pytorch, and nnU-Net are optional backends, so they must be installed into the same Python environment that runs napari. Prefer `python -m pip` from that environment so the package is installed where napari can import it.\n\nIf you install a release of `napari-unet-assistant` that declares the optional extras, you can use:\n\n```bash\npython -m pip install napari-unet-assistant[monai]\npython -m pip install napari-unet-assistant[smp]\npython -m pip install napari-unet-assistant[nnunet]\npython -m pip install napari-unet-assistant[models]\n```\n\nUse `models` to install all optional model backends at once. If your shell treats square brackets specially, quote the requirement:\n\n```bash\npython -m pip install \"napari-unet-assistant[models]\"\n```\n\nIf the extras command does not install the backend package, install the backend dependency directly:\n\n```bash\npython -m pip install monai\npython -m pip install segmentation-models-pytorch\npython -m pip install nnunetv2\n```\n\nFor a development checkout, install the extras from the repository root:\n\n```bash\npython -m pip install -e \".[models]\"\n```\n\nFor the GitHub version, put the extras before the `@` URL:\n\n```bash\npython -m pip install \"napari-unet-assistant[models] @ git+https://github.com/wulinteousa2-hash/napari-unet-assistant.git\"\n```\n\nYou can verify that the optional packages are available with:\n\n```bash\npython -c \"import monai; import segmentation_models_pytorch; import nnunetv2; print('optional model backends ok')\"\n```\n\nIf this import check fails, napari will still load the built-in U-Net backend, but the missing optional model source will not be usable until its package is installed in the active environment. After installing a source, restart napari and select it from the `Model source` dropdown.\n\nThe model registry lives under `src/napari_unet_assistant/models/` and separates provider code into `providers/`. MONAI and segmentation-models-pytorch models are regular `torch.nn.Module` backends. nnU-Net uses a dedicated pipeline workflow because nnU-Net manages its own data conversion, planning, preprocessing, training, and prediction commands.\n\nModel selection is split into:\n\n- `Model source`: the library or workflow provider, such as built-in plugin models, MONAI, SMP, or the separate nnU-Net pipeline. This is the ML/backend term in plainer UI wording.\n- `Architecture`: the model family inside that source, such as U-Net, U-Net++, SegResNet, or DeepLabV3+.\n- `Feature extractor`: the encoder/backbone used by encoder-style models when supported, such as ResNet34, ResNet50, EfficientNet-B0, DenseNet121, or MobileNetV2.\n- `Pretrained weights`: optional starting weights for the feature extractor, such as ImageNet weights for supported 2D SMP encoders.\n- `Model size`: width/depth capacity preset used by sources that expose size controls.\n\n## Model Sweep\n\nUse the `Model Sweep` tab when you want to compare multiple model configurations without starting each run manually. Set the shared training constants first, including data shape, task type, patch settings, augmentation, validation split, epochs, and batch size. Then check the model sources, model sizes, and SMP feature extractors to test, click `Preview model sweep`, and click `Run model sweep`. The regular `Model source`, `Architecture`, and `Feature extractor` controls in the `Run` tab are for single training runs; Model Sweep expands checked sources into valid architectures, selected feature extractors, and pretrained weights automatically.\n\nFor SMP sweeps, the `SMP feature extractors` checklist controls which encoders are expanded. The list is grouped into small/fast, good default, and larger encoders. The default checked encoders preserve the earlier sweep behavior: `resnet34`, `resnet50`, `efficientnet-b0`, `densenet121`, and `mobilenet_v2`.\n\nAfter a sweep finishes, click `Load sweep results` and choose the sweep folder to compare `sweep_results.csv` inside the app. The comparison view provides a sortable table, top-run metric bars for Dice, IoU, F1, or validation loss, and `Use selected run for inference` to load the chosen completed run into the Inference section.\n\nThe sweep runs configurations sequentially so one GPU is used predictably. Each completed configuration gets its own run folder with the usual files:\n\n- `config.json`\n- `run_summary.txt`\n- `validation.json`\n- `history.csv`\n- `summary.json`\n- `best_model.pt`\n\nThe sweep folder also writes:\n\n- `sweep_config.json`: shared settings, augmentation, selected configurations, and skipped entries\n- `sweep_results.csv`: one summary row per skipped, failed, or completed configuration\n- `pairs.csv`: image-mask pairs used by the sweep\n\nThe `sweep_results.csv` table includes source/backend, architecture/model family, model size, feature extractor/encoder, pretrained weights, best validation loss, best validation Dice/IoU/F1, run folder, and any error message. Missing optional backend packages are reported as skipped during preview/run. Failed combinations do not stop the sweep; they are recorded and the next configuration runs. Completed and failed rows are written from the sweep worker as each configuration ends, so partial results stay current even during long runs. The stop button cancels after the current batch or epoch boundary and leaves partial results on disk.\n\nnnU-Net is not trained through Model Sweep because it uses its own data-conversion, planning, training, and prediction pipeline rather than the plugin's direct `torch.nn.Module` training loop. To train nnU-Net, select `nnunet` in the `Run` tab and start a normal training run. The plugin exports the paired TIFF data to a self-contained nnU-Net work folder, runs `nnUNetv2_plan_and_preprocess`, runs `nnUNetv2_train`, and saves `nnunet_workflow.json` so the Inference panel can run `nnUNetv2_predict` and convert predictions back to TIFF.\n\n## Manual CSV pairing\n\nFor manual pairing, provide a CSV file with one image-mask pair per row.\n\nRequired columns:\n\n- `image_path`\n- `mask_path`\n\nOptional column:\n\n- `key`\n\nExample:\n\n```csv\nkey,image_path,mask_path\nsample01,/path/to/images/sample01.tif,/path/to/masks/sample01_mask.tif\nsample02,/path/to/images/sample02.tif,/path/to/masks/sample02_mask.tif\n```\n\nUse absolute paths for the clearest behavior. Relative paths are interpreted from the current working directory.\n\nEach image and mask should have matching spatial dimensions.\n\n## Installation\n\n```bash\npip install git+https://github.com/wulinteousa2-hash/napari-unet-assistant.git\n```\n\nFor editable development:\n\n```bash\ngit clone https://github.com/wulinteousa2-hash/napari-unet-assistant.git\ncd napari-unet-assistant\npip install -e .\nnapari\n```\n\n## Basic workflow\n\n1. Open napari.\n2. Launch **U-Net Assistant**.\n3. Choose a pairing mode.\n4. Scan and review image-mask pairs.\n5. Set training options.\n6. Train a 2D or 3D U-Net model.\n7. Load a saved run folder.\n8. Run inference on new images or volumes.\n9. Review prediction masks in napari.\n\n## Binary vs multiclass outputs\n\nFor binary segmentation, masks still use `0 = background` and nonzero/`1 = foreground`, but the model uses one output channel: foreground probability. Background is implicit when foreground probability is below the threshold.\n\nFor multiclass segmentation, enter the total number of labels including background. For example, three classes means `0 = background`, `1 = class 1`, and `2 = class 2`.\n\n## Supported data\n\n### 2D training\n\n- image: `(Y, X)` grayscale TIFF\n- mask: `(Y, X)` label TIFF\n- binary masks: `0 = background`, nonzero = foreground\n- multiclass masks: integer labels from `0` through `num_classes - 1`; training checks this before starting\n\n### 3D training\n\n- image: `(Z, Y, X)` grayscale TIFF\n- mask: `(Z, Y, X)` label TIFF\n- multiclass masks should use integer labels from `0` through `num_classes - 1`; training checks this before starting:\n  - `0 = background`\n  - `1 = class 1`\n  - `2 = class 2`\n  - `3 = class 3`\n\n## Patch options\n\n### 2D\n\nXY patch sizes:\n\n- 64\n- 128\n- 256\n- 512\n- 1024\n\nDefault: `256 x 256`\n\n### 3D\n\nZ patch sizes:\n\n- 8\n- 16\n- 32\n- 64\n\nXY patch sizes:\n\n- 64\n- 128\n- 256\n- 512\n- 1024\n\nDefault: `16 x 256 x 256`\n\n## Validation\n\nThe current training workflow uses a standard train/validation split. The default validation split is 20%.\n\nK-fold cross-validation is not active in this release.\n\nEach run writes `validation.json` with the active validation mode, split fraction, random seed, total patch count, train patch count, and validation patch count. The training log also reports the same split. Per-epoch validation metrics are written to `history.csv`.\n\nAt training start, the log reports the model source, architecture, feature extractor, pretrained weights, model size, patch settings, augmentation preset, validation setup, epochs, and batch size. The same user-readable summary is written to `run_summary.txt`, while full structured settings are saved in `config.json`.\n\n## Output History\n\nThe `Run output` field is treated as a parent folder for new training runs. Each new single training run creates a timestamped child run folder so repeated one-by-one experiments do not overwrite earlier `history.csv`, `summary.json`, or `best_model.pt` files.\n\nExample:\n\n```text\nmy_training_outputs/\n  run_20260520_143000_builtin_unet_large_plain_none/\n  run_20260520_151500_monai_unet_large_plain_none/\n  run_20260520_163000_smp_unetplusplus_large_resnet34_imagenet/\n  run_20260520_174500_nnunet_nnunetv2_dataset600_2d_fold0/\n```\n\nModel Sweep already uses a parent folder with child run folders under `sweep_YYYYMMDD_HHMMSS/`. Continue-training workflows still target a specific compatible run folder instead of creating a new sibling.\n\n## Outputs\n\nEach run folder can contain:\n\n- `best_model.pt`\n- `config.json`\n- `run_summary.txt`\n- `summary.json`\n- `history.csv`\n- `validation.json`\n- `pairs.csv`\n- prediction TIFF outputs\n\n## Current scope\n\nThis release is focused on TIFF-based supervised U-Net training and inference.\n\nOME-Zarr, spectral/lambda workflows, and SAM-assisted annotation are intentionally outside the scope of this version.\n\n## Reference\n\nThis project builds on U-Net-based nerve morphometry workflows described in:\n\nMoiseev D, Hu B, Li J. Morphometric Analysis of Peripheral Myelinated Nerve Fibers through Deep Learning. *Journal of the Peripheral Nervous System*. 2019;24(1):87-93.  \nhttps://pmc.ncbi.nlm.nih.gov/articles/PMC6420354/\n","description_content_type":"text/markdown","keywords":"napari,u-net,segmentation,annotation,microscopy,image analysis","home_page":null,"download_url":null,"author":"Wulin Teo","author_email":null,"maintainer":null,"maintainer_email":null,"license":"MIT","classifier":["Development Status :: 3 - Alpha","Framework :: napari","Intended Audience :: Science/Research","Programming Language :: Python :: 3","Programming Language :: Python :: 3.10","Programming Language :: Python :: 3.11","Topic :: Scientific/Engineering :: Image Processing"],"requires_dist":["napari","numpy","magicgui","qtpy","tifffile","zarr","ome-zarr","dask[array]","torch","torchvision","monai; extra == \"monai\"","nnunetv2; extra == \"nnunet\"","segmentation-models-pytorch; extra == \"smp\"","monai; extra == \"models\"","nnunetv2; extra == \"models\"","segmentation-models-pytorch; extra == \"models\""],"requires_python":">=3.10","requires_external":null,"project_url":["Homepage, https://github.com/wulinteousa2-hash/napari-unet-assistant","Repository, https://github.com/wulinteousa2-hash/napari-unet-assistant","Issues, https://github.com/wulinteousa2-hash/napari-unet-assistant/issues"],"provides_extra":["monai","nnunet","smp","models"],"provides_dist":null,"obsoletes_dist":null},"npe1_shim":false}