Skip to main content

Cellpose-SAM

AI Segmentation 📥 Download Script

Cellpose-SAM instance segmentation tutorial script.

This tutorial demonstrates how to automate instance segmentation workflows using the Cellpose-SAM model via the Python API.

Cellpose-SAM is a general-purpose instance segmentation model that assigns a unique integer label to each detected object. Unlike semantic segmentation models (such as TotalSegmentator, nnU-Net, or MONAI) where labels represent predefined tissue classes, Cellpose-SAM identifies individual object instances. This makes it particularly effective for cell counting, nuclei detection, and analyzing densely packed structures.

The provided script executes a complete pipeline: it performs 3D instance segmentation on the active volume, isolates the largest detected instance, filters the result to the largest connected region, and generates a 3D surface model. Finally, it applies surface cleanup operations and exports the result as an STL file.

Best Practice

For large datasets, consider testing parameters on a single 2D slice (cellpose_params.mode = '2D') before running a full 3D volumetric segmentation. This significantly reduces iteration time during parameter tuning.

Complete Example​

# Import necessary modules
import ScriptingApi as api

# Instantiate the Application class
# The Application class serves as the primary entry point for interacting with
# the software. To begin using the API, create an instance of this class.
# Use the original class name `Application` or the pythonic alias `app` or `application`
# as the variable name to improve IntelliSense support. For details, refer to the documentation.
app = api.Application()

# Close any open project
app.close_project()

# Open a specific project
project_path = R'C:\Data\MyProject.vvcx'
app.open_project(project_path)

# Get the name of the first volume
volumes = app.get_all_volume_names()
if len(volumes) == 0:
    raise RuntimeError("No volumes found in project.")
volume_name = volumes[0]
print("Using volume:", volume_name)

# Get AI segmentation instance
# Use the pythonic alias `ai_segmentation` as the variable name to improve IntelliSense support.
# Hover over the variable to view available methods and properties.
# If no tooltip appears, IntelliSense does not recognize the variable name.
# For details, refer to the documentation.
ai_segmentation = app.get_ai_segmentation()

# Set the AI segmentation model type to Cellpose-SAM
ai_segmentation.set_model_type(api.AiSegmentationModelType.Cellpose)

# Check installation status and install if necessary
installed = ai_segmentation.get_installation_status()
if not installed:
    print('Installation required: The AI model is not installed.')
    print('Please run the application as an administrator to install it.')
    print('Attempting to install the AI model now.')
    installed = ai_segmentation.install_model()
    if not installed:
        raise RuntimeError('An error occurred during the installation of the AI model. The log file has been saved for troubleshooting.')
    else:
        print('The AI model is installed and ready for use.')
else:
    print('The AI model is installed and ready for use.')

# ---------------------------------------------------------------------------
# 3D Volumetric Instance Segmentation
# ---------------------------------------------------------------------------

# Get default Cellpose-SAM parameters and configure for 3D segmentation
cellpose_params: api.CellposeParams = ai_segmentation.get_default_cellpose_params()
cellpose_params.mode = '3D'

# Select the built-in Cellpose model to use. Options:
#   'cpsam_v2'    - Latest CellposeSAM model (default, improved on low-contrast regions)
#   'cpsam'       - Original CellposeSAM model
# The selected model is downloaded automatically on first use.
cellpose_params.model = 'cpsam_v2'

# Use GPU when available for significantly faster 3D processing.
# Set to 'cpu' if no CUDA-compatible GPU is available.
cellpose_params.device = 'gpu'

# Detection sensitivity: lower flow_threshold = stricter mask quality filtering.
# Range: 0.0-5.0. Default: 0.4.
cellpose_params.flow_threshold = 0.4

# Object probability threshold: lower (negative) = detect more/larger objects.
# Range: -6.0 to 6.0. Default: 0.0.
cellpose_params.cellprob_threshold = 0.0

# Expected object diameter in pixels. Set to 0 for automatic estimation.
cellpose_params.diameter = 0

# Minimum object size in pixels. Smaller objects are removed. Set to -1 to disable.
cellpose_params.min_size = 15

# Discard masks touching the image boundary (useful for excluding incomplete objects).
cellpose_params.exclude_on_edges = False

# Number of 256x256 tiles processed simultaneously on GPU. Reduce if running out of memory.
cellpose_params.batch_size = 8

# Z-to-XY voxel spacing ratio for anisotropic volumes (e.g., 2.5 if Z is 2.5x coarser).
# Set to 1.0 for isotropic data.
cellpose_params.anisotropy = 1.0

# Stitch threshold: 0.0 = native 3D segmentation; ~0.9 = stitch 2D slices into 3D objects.
# Stitch mode is faster and uses less memory but may produce less coherent results.
cellpose_params.stitch_threshold = 0.0

# Run Cellpose-SAM 3D segmentation
# Returns a list of generated mask names. Each mask is an instance-labeled volume
# where each detected object has a unique integer label.
# multilabelOutput=True (default): all instances are packed into one multilabel mask.
# Set multilabelOutput=False to receive each detected instance as a separate mask object.
mask_names = ai_segmentation.run_cellpose([volume_name], cellpose_params)
if len(mask_names) == 0:
    raise RuntimeError("Segmentation returned no masks (operation cancelled or failed).")
print("Generated masks:", mask_names)

# Get the name of the currently active mask
active_mask = app.get_active_mask_name()
print("Active mask:", active_mask)

# Get the mask_operations object to perform operations on masks
mask_operations = app.get_mask_operations()

# Split the instance-labeled mask into individual per-object masks.
# In Cellpose-SAM output, each unique label value represents a distinct object instance.
# Setting extract_largest=True and count=1 retains only the largest instance by volume.
splitted_masks = mask_operations.split_multi_label_mask(volume_name, active_mask, True, 1)
if len(splitted_masks) == 0:
    raise RuntimeError("Split operation did not produce separate labels.")
print("Splited masks (largest instance retained):", splitted_masks)

# Retain only the largest connected region within the mask
mask_operations.filter_regions(splitted_masks, True, 1, 8)

# Show only the splited mask, hide the original instance-labeled mask
app.set_masks_visible([active_mask], False)
app.set_masks_visible(mask_names, True)

# Set 3D preview quality and generate preview of all visible masks
app.set_mask_3d_preview_quality(api.Mask3dPreviewQuality.Optimal)
app.generate_mask_3d_preview(app.get_visible_mask_names())

# Convert generated 3D preview to surface objects
surfaces = mask_operations.convert_3d_preview_to_surface_objects()
if len(surfaces) == 0:
    raise RuntimeError("No surface generated from 3D preview.")
print("Created surfaces:", surfaces)

# Get the name of the currently active surface
surface = app.get_active_surface_name()
print(surface)

# Get the surface_operations object to perform operations on surfaces
surface_operations = app.get_surface_operations()

# Filter surface to keep only the largest shell
surface_filter_shells_params = api.SurfaceFilterShellsParams()
surface_filter_shells_params.largest_shells = 1
surface_operations.filter_shells(surface, api.SurfaceFilterShellsMethod.LargestShells, surface_filter_shells_params)

# Remesh the surface for better quality
surface_operations.remesh([surface])

# Fix surface issues using diagnostics checks
surface_diagnostics_checks = api.SurfaceDiagnosticsChecks()
surface_operations.fix_surface(surface, surface_diagnostics_checks)

# Hide all volumes and masks to focus on the surface
app.set_volumes_visible(app.get_all_volume_names(), False)
app.set_masks_visible(app.get_all_mask_names(), False)

# Isolate the surface
app.isolate_surfaces([surface])

# Export the surface to disk in STL format
surface_operations.export_surface_to_disk(surface, 'C:/tmp/surface.stl')

# Set surface representation to Solid with Edges
surface_render_properties_operations = surface_operations.get_render_properties_operations()
surface_render_properties_operations.set_representation([surface], api.SurfaceRepresentation.SolidEdges)

# ---------------------------------------------------------------------------
# Alternative: 2D Single-Slice Segmentation
# ---------------------------------------------------------------------------
# To segment a single slice instead of the full volume, configure as follows:
#
#   cellpose_params.mode = '2D'
#   cellpose_params.slice_index = -1   # -1 uses the currently active slice
#   # Or specify a slice number directly:
#   # cellpose_params.slice_index = 42
#   mask_names = ai_segmentation.run_cellpose([volume_name], cellpose_params)
#   # Optional: set multilabelOutput=False to receive each instance as a separate mask.
#   # mask_names = ai_segmentation.run_cellpose([volume_name], cellpose_params, False)
#
# The output is a full-volume mask where only the segmented slice contains
# instance labels. All other slices remain empty (value 0).

print("Cellpose-SAM tutorial completed successfully.")