train_hf_semantic_seg

Train on semantic segmentation models available on Hugging Face (Segformer, BeiT, Data2Vec-vision).

🚀 Use with Ikomia API

1. Install Ikomia API

We strongly recommend using a virtual environment. If you're not sure where to start, we offer a tutorial here.

pip install ikomia

2. Create your workflow

from ikomia.dataprocess.workflow import Workflow

# Init your workflow
wf = Workflow()    

# Add data loader
coco = wf.add_task(name="dataset_coco")

coco.set_parameters({
    "json_file": "path/to/json/annotation/file",
    "image_folder": "path/to/image/folder",
    "task": "semantic_segmentation",
}) 

# Add train algorithm 
train = wf.add_task(name="train_hf_semantic_seg", auto_connect=True)

# Launch your training on your data
wf.run()

☀️ Use with Ikomia Studio

Ikomia Studio offers a friendly UI with the same features as the API.

• If you haven't started using Ikomia Studio yet, download and install it from this page.

• For additional guidance on getting started with Ikomia Studio, check out this blog post.

📝 Set algorithm parameters

• model_card (int) - default 'nvidia/segformer-b0-finetuned-ade-512-512': Name of the model.
• batch_size (int) - default '4': Number of samples processed before the model is updated.
• epochs (int) - default '50': Number of complete passes through the training dataset.
• input_size (int) - default '224': Size of the input image.
• learning_rate (float) - default '0.00006': Step size at which the model's parameters are updated during training.
• dataset_split_ratio (float) – default '0.9': Divide the dataset into train and evaluation sets ]0, 1[.
• output_folder (str, optional): path to where the model will be saved.
• config_file (str, optional): path to the training config file .yaml.

Parameters should be in strings format when added to the dictionary.

from ikomia.dataprocess.workflow import Workflow

# Init your workflow
wf = Workflow()    

# Add data loader
coco = wf.add_task(name="dataset_coco")

coco.set_parameters({
    "json_file": "path/to/json/annotation/file",
    "image_folder": "path/to/image/folder",
    "task": "semantic_segmentation",
}) 

# Add train algorithm 
train = wf.add_task(name="train_hf_semantic_seg", auto_connect=True)
train.set_parameters({
    "model_card": "nvidia/mit-b2",
    "batch_size": "4",
    "epochs": "50",
    "learning_rate": "0.00006",
    "dataset_split_ratio": "0.8",
}) 

# Launch your training on your data
wf.run()

⏩ Advanced usage

This algorithm proposes to fine-tune semantic segmentation models available on Hugging Face:

1. BEiT (from Microsoft) released with the paper BEiT: BERT Pre-Training of Image Transformers by Hangbo Bao, Li Dong, Furu Wei.

   • microsoft/beit-base-patch16-224-pt22k
   • microsoft/beit-base-patch16-224
   • microsoft/beit-base-patch16-384
   • microsoft/beit-large-patch16-224-pt22k
   • microsoft/beit-large-patch16-224
   • microsoft/beit-large-patch16-384
   • microsoft/beit-large-patch16-512

2. Data2Vec (from Facebook) released with the paper Data2Vec: A General Framework for Self-supervised Learning in Speech, Vision and Language by Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, Michael Auli.

   • facebook/data2vec-vision-base

3. SegFormer (from NVIDIA) released with the paper SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers by Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, Ping Luo.

   • nvidia/mit-b0
   • nvidia/mit-b1
   • nvidia/mit-b2
   • nvidia/mit-b3
   • nvidia/mit-b4
   • nvidia/mit-b5