JS Segment Annotator
Javascript image annotation tool based on image segmentation.
- Label image regions with mouse.
- Written in vanilla Javascript, with require.js dependency (packaged).
- Pure client-side implementation of image segmentation.
A browser must support HTML canvas to use this tool.
There is an online demo.
Importing data
Prepare a JSON file that looks like the following. The required fields are labels and imageURLs. The annotationURLs are for existing data and can be omitted. Place the JSON file inside the data/ directory.
{
"labels": [
"background",
"skin",
"hair",
"dress",
"glasses",
"jacket",
"skirt"
],
"imageURLs": [
"data/images/1.jpg",
"data/images/2.jpg"
],
"annotationURLs": [
"data/annotations/1.png",
"data/annotations/2.png"
]
}
Then edit main.js to point to this JSON file. Open a Web browser and visit index.html.
Known issues
Browser incompatibility
A segmentation result can greatly differ due to the difference in Javascript implementation across Web browsers. The difference stems from numerical precision of floating point numbers, and there is no easy way to produce the exact same result across browsers.
Python tips
Annotation PNG
The annotation PNG file contains label map encoded in RGB value. Do the following to encode an index map.
import numpy as np
from PIL import Image
# Decode
encoded = np.array(Image.open('data/annotations/1.png'))
annotation = np.bitwise_or(np.bitwise_or(
encoded[:, :, 0].astype(np.uint32),
encoded[:, :, 1].astype(np.uint32) << 8),
encoded[:, :, 2].astype(np.uint32) << 16)
print(np.unique(annotation))
# Encode
Image.fromarray(np.stack([
np.bitwise_and(annotation, 255),
np.bitwise_and(annotation >> 8, 255),
np.bitwise_and(annotation >> 16, 255),
], axis=2).astype(np.uint8)).save('encoded.png')
JSON
Use JSON module.
import json
with open('data/example.json', 'r') as f:
dataset = json.load(f)
Using dataURL
Do the following to convert between dataURL and NumPy format.
from PIL import Image
import base64
import io
# Encode
with io.BytesIO() as buffer:
encoded.save(buffer, format='png')
data_url = b'data:image/png;base64,' + base64.b64encode(buffer.getvalue())
# Decode
binary = base64.b64decode(data_url.replace(b'data:image/png;base64,', b''))
encoded = Image.open(io.BytesIO(binary))
Matlab tips
Annotation PNG
The annotation PNG file contains label map encoded in RGB value. Do the following to encode an index map.
% Decode
X = imread('data/annotations/0.png');
annotation = X(:, :, 1);
annotation = bitor(annotation, bitshift(X(:, :, 2), 8));
annotation = bitor(annotation, bitshift(X(:, :, 3), 16));
% Encode
X = cat(3, bitand(annotation, 255), ...
bitand(bitshift(annotation, -8), 255), ...
bitand(bitshift(annotation, -16)), 255));
imwrite(uint8(X), 'data/annotations/0.png');
JSON
Use the matlab-json package.
Using dataURL
Get the byte encoding tools.
Do the following to convert between dataURL and Matlab format.
% Decode
dataURL = 'data:image/png;base64,...';
png_data = base64decode(strrep(dataURL, 'data:image/png;base64,', ''));
annotation = imdecode(png_data, 'png');
% Encode
png_data = imencode(annotation, 'png');
dataURL = ['data:image/png;base64,', base64encode(png_data)];
Citation
We appreciate if you cite the following article in an academic paper. The tool was originally developed for this work.
@article{tangseng2017looking,
Author = {Pongsate Tangseng and Zhipeng Wu and Kota Yamaguchi},
Title = {Looking at Outfit to Parse Clothing},
Eprint = {1703.01386v1},
ArchivePrefix = {arXiv},
PrimaryClass = {cs.CV},
Year = {2017},
Month = {Mar},
Url = {http://arxiv.org/abs/1703.01386v1}
}
)
On the next picture below is annotated image from Pascal VOC 2012 for segmentation, containing RGB information + index for object and also the same for background.
1 Jan 03, 2022
1 Jan 09, 2022
9.3k Dec 31, 2022
1.2k Dec 30, 2022
1 Jan 19, 2022
24 Dec 20, 2022
6 Jun 24, 2022
2 Nov 19, 2021
149 Dec 25, 2022
3 Dec 22, 2021
3 Mar 12, 2022
2 Nov 23, 2021
1 Feb 02, 2022
6.9k Jan 03, 2023
133 Dec 27, 2022
672 Jan 05, 2023
228 Dec 17, 2022
2 Dec 28, 2021
27 Dec 28, 2022
152 Dec 13, 2022