Open Source Data Annotation Tool
In a machine learning project the model is the replaceable part. What is expensive and hard to recreate is the labeled data - thousands of hours of human judgment about what each image, span, or clip actually contains. That makes the annotation tool a strange thing to rent, because a hosted labeler puts both your proprietary source data and the labels you paid for on someone else's platform. Running the workflow on infrastructure you control keeps the dataset, the annotations, and the review decisions that produced them with the team that built them. Open source adds a second payoff: labels only have value if you can get them out cleanly, and open tools let you inspect and script the export formats your training code depends on.
Label Studio
Open source data labeling and annotation tool for text, images, audio, video, and time series
CVAT
Open-source data annotation platform for image, video, and 3D datasets with AI-assisted labeling
LabelMe
Offline image annotation tool with polygons, masks, and AI-assisted labeling
doccano
Open-source text annotation tool for collaborative labeling, classification, and sequence tasks
Argilla
Collaboration tool for AI engineers and domain experts to build high-quality datasets
Our picks
Pick by the data you are labeling, since each of these tools specializes.
Multi-modal labeling: Label Studio Label Studio handles text, images, audio, video, and time series in one interface and exports to a range of model formats, which makes it the safe default when a team labels more than one kind of data. It can be customized for specific datasets and connect to your own machine learning models for pre-labeling, and it self-hosts or runs in the cloud.
Computer vision at team scale: CVAT CVAT is purpose-built for image, video, and 3D annotation, with AI-assisted labeling that can run your own detection or segmentation models. It includes quality control, multi-user and multi-organization workflows with roles and task assignment, and developer APIs, so it fits a labeling operation rather than a single annotator.
Collaborative text annotation: doccano doccano focuses on text: classification, sequence labeling, and sequence-to-sequence tasks such as named entity recognition and summarization. It supports collaborative annotation and exposes a REST API, and it self-hosts with pip or Docker so the text you are labeling stays on your own infrastructure.
Offline image annotation: LabelMe LabelMe is a desktop tool for polygon, rectangle, circle, and mask annotation that runs completely offline, exporting VOC and COCO for segmentation datasets. AI-assisted labeling with SAM and YOLO-World speeds up drawing, and because nothing leaves the machine it suits air-gapped or privacy-sensitive image work.
Matching the tool to the data you are labeling
The first filter is not the interface but the kind of data you are labeling, because most of these tools are built around one modality and get awkward outside it. Label Studio is the generalist, covering text, images, audio, video, and time series in a single UI. CVAT is the computer-vision specialist, purpose-built for image, video, and 3D annotation with AI-assisted labeling. doccano handles text only, from classification to sequence labeling and sequence-to-sequence tasks like named entity recognition. LabelMe is a focused offline image tool for polygons and masks. Argilla targets NLP and LLM feedback datasets, though its own notes say it is in maintenance mode, receiving bug fixes rather than new features.
Once the modality fits, weigh the review workflow, which is where casual and serious labeling diverge. A real project needs assignment rules, reviewer queues, and a way to measure agreement between annotators. CVAT builds this in with roles, task assignment, and multi-organization review steps, and Label Studio supports team labeling that includes external annotators. If you plan to run blind double-labeling or have a senior reviewer correct work in place, confirm the tool enforces that rather than just storing labels. When outside labelers are involved, per-task permissions matter as much as drawing speed, since each should see only their assigned work.
Finally, plan the data path in and out before you load a real dataset. Check that labels export in a format your training code already reads - LabelMe writes VOC and COCO directly - and that imports preserve IDs, frame numbers, and label attributes. Model-assisted labeling is worth testing: CVAT can run your own detection or segmentation models, and LabelMe supports point-to-mask labeling with SAM and text-to-annotation with YOLO-World. For air-gapped or privacy-sensitive data, LabelMe's fully offline design keeps every asset on the machine.
Related categories
Frequently asked questions
Which of these tools fits which kind of data?+
Match the tool to the modality, because they specialize. Label Studio is the multi-modal generalist for text, images, audio, video, and time series. CVAT is for images, video, and 3D. doccano is text-only, and LabelMe is for offline image annotation with polygons and masks. Argilla is oriented toward NLP and LLM feedback datasets. Test a real sample file in your chosen tool before committing, since one modality's strength can be another's weakness.
Do these tools support AI-assisted or model-assisted labeling?+
Several do, but the payoff depends on model quality. CVAT offers AI-assisted annotation and can use your own ML models for detection, segmentation, and tracking. LabelMe supports point-to-polygon and point-to-mask labeling with SAM and EfficientSAM, plus text-to-annotation with YOLO-World. Predictions help most when annotators correct rather than redraw, so measure time per accepted label, not just time to a first draft, before assuming it saves work.
Can I label data completely offline or on an air-gapped machine?+
Yes, and LabelMe is the tool built for it. It is a desktop application in Python and Qt that runs 100% offline, so images, annotations, and model predictions never leave the machine, which suits privacy-sensitive or air-gapped work. The browser-based tools like CVAT and doccano can also be self-hosted on an isolated network, but LabelMe needs no server at all, only the local application.
Which export formats matter for a training pipeline?+
The format has to match how your model code reads data, so check this before labeling at scale. LabelMe exports VOC for semantic and instance segmentation and COCO for instance segmentation. Label Studio exports to a range of model formats, and CVAT and doccano expose APIs for custom export. Image detection usually wants JSON with boxes and class IDs, while NLP work needs token offsets or spans. Run a small round trip to confirm nothing important is dropped.
How do review and quality control work with a team of labelers?+
The pattern should fit the cost of a wrong label. CVAT provides roles, task assignment, and review steps across multi-user and multi-organization projects, which supports reviewer approval and adjudication. Common approaches include sampling, blind double-labeling, and consensus scoring by a senior annotator. For medical, legal, or safety-related data you usually want traceable review decisions and a way to compare annotators on the same item, not just random spot checks.
Is Argilla still safe to adopt while it is in maintenance mode?+
It can be, with eyes open. Argilla's own description says it is in maintenance mode, with maintainers publishing bug fixes and patches against a mature, stable codebase rather than shipping new features. For a stable NLP or LLM feedback workflow on a well-understood version, that is often fine. If you need active development, new modalities, or long-term feature growth, one of the actively maintained tools here is the safer base to build on.
Is self-hosting a labeling tool worth it for sensitive datasets?+
It usually is when the data is sensitive, large, or expensive to move, since self-hosting keeps raw assets and labels inside your own network and auth. CVAT, doccano, and Label Studio all self-host with Docker, and LabelMe runs locally with no server. The tradeoff is that upgrades, uptime, backups, and storage tuning become your team's job. For a small one-off labeling task, managed hosting can be the simpler choice.