{"name":"frontvegV2","display_name":"FrontVeg V2","visibility":"public","icon":null,"categories":[],"schema_version":"0.3.0","on_activate":null,"on_deactivate":null,"contributions":{"commands":[{"id":"frontvegV2.make_frontveg_widget","title":"Open FrontVeg V2","python_name":"napari_frontveg:make_frontveg_widget","short_title":null,"category":null,"icon":null,"enablement":null}],"readers":null,"writers":null,"widgets":[{"command":"frontvegV2.make_frontveg_widget","display_name":"FrontVeg V2 Studio","autogenerate":false}],"sample_data":null,"themes":null,"menus":{},"submenus":null,"keybindings":null,"configurations":{}},"package_metadata":{"metadata_version":"2.4","name":"frontvegV2","version":"2.2.0","dynamic":["license-file"],"platform":null,"supported_platform":null,"summary":"Automated plant organ segmentation and leaf area estimation in trellised crops using side view monocular RGB images.","description":"# FrontVeg V2: Foreground-Aware Zero-Shot Plant Trait Segmentation in Trellised Crops\n\n\n**Abdoul Djalil Ousseini Hamza** ([ORCID](https://orcid.org/0009-0008-6172-4439)), **Herearii Metuarea** ([ORCID](https://orcid.org/0009-0008-2716-0617)), **Corentin Lothodé** ([ORCID](https://orcid.org/0000-0002-8209-317X)), **Morgane Roth** ([ORCID](https://orcid.org/0000-0002-5244-4215)), **Eric Duchêne** ([ORCID](https://orcid.org/0000-0003-2712-1892)), **Lionel Ley**, **David Alletru** ([ORCID](https://orcid.org/0009-0006-6238-6123)), and **David Rousseau*** ([ORCID](https://orcid.org/0000-0002-7935-1609))\n\n*\\* Project Supervisor* \n\n\n\n\n\n\n\n\n\"Partner\n\n[![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/frontvegv2)](https://napari-hub.org/plugins/frontvegv2.html) [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/) \n[![PyTorch](https://img.shields.io/badge/PyTorch-%23EE4C2C.svg?style=flat&logo=PyTorch&logoColor=white)](https://pytorch.org/)\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\n\n \n\n\n\n# FrontVeg V2\n\nA napari plugin for automated plant organ segmentation and leaf area estimation in trellised crops using side view monocular RGB images.\n\n \n\nSome examples of organ segmentation on different trellised crops: \n\n

\n \"FrontVeg\n

\n\n---\n\n## Prerequisites\n\n1. **Create and activate a virtual environment** \n\nWe recommend using Python 3.10 and cuda 12.1:\n\n ```bash\n conda create -n python=3.10 -y\n conda activate \n pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121\n ```\n\n2. **Request SAM3 Weights Access:**\n Request access to the SAM3 checkpoint on [Hugging Face](https://huggingface.co/facebook/sam3). Once approved, download `sam3.pt`.\n\n3. **Download Depth-Anything V2 Weights:**\n Download the Large model checkpoint (`depth_anything_v2_vitl.pth`) from the official [Depth-Anything V2 repository](https://github.com/DepthAnything/Depth-Anything-V2).\n\n4. **Place your downloaded checkpoints inside the project folder:** \n - Place (`sam3.pt`) into: `checkpoints/sam3_ckpts/`\n - Place (`depth_anything_v2_vitl.pth`) into: `checkpoints/depthanything_ckpts/`\n\n---\n\n\n\n\n\n## Installation\n\nFrontVeg V2 requires a dedicated Conda environment and two external models: **Depth-Anything V2** and **SAM3**.\n\n> **Note:** The FrontVeg V2 GitHub repository is currently private. Therefore, users should install the FrontVeg V2 package from PyPI rather than cloning the main repository.\n\n\n\n### 1. Clone and install the external models\n\nCreate a directory for the external dependencies:\n\n```bash\nmkdir external\ncd external\n```\n\n#### Depth-Anything V2\n\n```bash\ngit clone https://github.com/DepthAnything/Depth-Anything-V2.git\n```\n\nDepth-Anything V2 does not need to be installed as a package at this stage; FrontVeg V2 uses it from the `external/` directory.\n\n#### SAM3\n\n```bash\ngit clone https://github.com/facebookresearch/sam3.git\ncd sam3\npip install -e .\ncd ..\n```\n\nYou should now have:\n\n```text\nexternal/\n├── Depth-Anything-V2/\n└── sam3/\n```\n\n### 2. Install FrontVeg V2 from PyPI\n\nOnce the external dependencies have been installed, return to the directory where you want to use FrontVeg V2 and run:\n\n```bash\npip install frontvegV2\n```\n\nThis installs the FrontVeg V2 package and its Python dependencies.\n\n### 3. Check the installation\n\nYou can verify that FrontVeg V2 is available with:\n\n```bash\npython -c \"import frontveg; print('FrontVeg V2 installed successfully')\"\n```\n\n> **Important:** The external repositories must be installed before running FrontVeg V2. In particular, **SAM3 must be installed with `pip install -e .`** from its cloned repository.\n\n### Alternative: Development Installation\n\nIf you have access to the private FrontVeg V2 GitHub repository and want to modify the source code, you can clone the repository and install it in editable mode:\n\n```bash\ngit clone https://github.com/djaliloh/FrontVegV2.git\ncd FrontVegV2\n\nmkdir external\ncd external\n\n# Depth-Anything V2\ngit clone https://github.com/DepthAnything/Depth-Anything-V2.git\n\n# SAM3\ngit clone https://github.com/facebookresearch/sam3.git\ncd sam3\npip install -e .\n\ncd ../..\n\n# Install FrontVeg V2 in editable mode\npip install -e .\n```\n\nThis development installation is intended for contributors and users who need direct access to the source code.\n\n\n\n\n\n\n\n\n\n## Usage in Napari\n\n1. Launch Napari: `napari`\n2. Open an RGB image of a row crop.\n3. Select **Plugins > FrontVeg V2 Studio**.\n4. Set your prompt (e.g., \"leaf\") and adjust parameters (Tile Overlap, Sigma, etc.).\n5. Click **Run Complete Pipeline**.\n\n\n\n## Acknowledgements\n\nThis work was supported by the French **Programme et Équipements Prioritaires de Recherche (PEPR) AgroEcoNum** ([pepr-agroeconum.fr](https://www.pepr-agroeconum.fr/)), the **France 2030** program through the Agence Nationale de la Recherche (ANR), and the European Union’s **Horizon Europe** research and innovation programme (Grant No. 101094587, **PHENET**). \n\nThis research used computing resources from the **GLiCID Computing Facility** (Ligerien Group for Intensive Distributed Computing, [doi:10.60487/glicid](https://doi.org/10.60487/glicid), Pays de la Loire, France), as well as HPC and storage resources provided by **GENCI at IDRIS** on the Jean Zay supercomputer’s H100 partition (Grant 2025-AD010115553R1).\n\nThe authors would like to thank **Sirine Gharbi**, **Oumaima Karia**, **Justin Langlois**, **Paul Persello**, and **Thomas Lebouc** for their valuable assistance in annotating the image dataset used in this study.\n\n\n## Contact\n\nImhorphen team, bioimaging research group,
\nIRHS, UMR 1345, INRAE,
\n42 Rue Georges Morel, 49070 Beaucouzé, France\n\n- David Rousseau - Professor, david.rousseau@univ-angers.fr\n- Corentin Lothode - Researcher Engineer, corentin.lothode@inrae.fr\n- Herearii Metuarea - PhD student, herearii.metuarea@univ-angers.fr\n- Abdoul Djalil Ousseini Hamza - Engineer, abdoul-djalil.ousseini-hamza@inrae.fr\n\n\n## Troubleshooting\n\n- Missing Module 'triton': On Windows, install the Windows-compatible Triton build: pip install triton-windows.\n- RuntimeError: mat1 and mat2 must have the same dtype: If running on GPUs older than NVIDIA Ampere (e.g., GTX 10xx, RTX 20xx), ensure autocast is set to float16 or float32 instead of bfloat16.\n- Missing Checkpoints Error: Verify that sam3.pt exists at the expected path or set FRONTVEG_SAM3_CKPT manually.\n\n\n## License\n\nFrontVeg V2 is released under the MIT License. See the LICENSE.txt file for details.\n\nThird-party software and models used by FrontVeg V2, including SAM3 and\nDepth Anything V2, are subject to their respective licenses and terms.\n\n\n## Contributing\n\nContributions, bug reports, feature requests, and suggestions are welcome.\n\nTo contribute:\n\n1. Fork the repository.\n2. Create a new branch for your changes.\n3. Make your changes and test them locally.\n4. Commit your changes with a clear and descriptive message.\n5. Open a pull request describing your changes and their purpose.\n\nFor bugs or feature requests, please open an issue and provide enough information to reproduce the problem or evaluate the proposed feature.\n\nBy contributing to this repository, you agree that your contributions will be distributed under the same license as the project.\n\n\n