AutoLabelMe

AutoLabelMe Project has been closed and the GUI has been rewritten in Swift and Python. Check the AutoAnnotator App.

1. Introduction

Data Collection is a major step in every Machine Learning projects. The quality and quantity of data alone can greatly impact the results of the model. AutoLabelme is an open-source automatic image annotator created using the Tkinter library in Python. It’s an extension of LabelMe, the open-source Image Annotator available in Github LabelMe. It matches the template provided by the user in an image and find same objects associating a bounding box and label for every object. Autolabelme uses Normalized Cross-Correlation to check whether two templates are similar. It is designed keeping in mind with the necessity of creating dataset for object detection purposes and currently can only find objects in the cropped image i.e the search space to find same object is the space around the current template.

2. Properties

1. AutoLabelMe is simple to use. After LabelMe Manual Annotation just run AutoLabelMe.
2. It is fast. The search space to the local neighbourhood of the template which makes it efficient for some projects.
3. AutoLabelMe can identify rotated, scaled, horizontally and vertically flipped templates. There is no need for manually annotate rotated version of any template. It assigns a new label for the rotated templates with the rotaion information added in the label.
4. AutoLabelMe also saves any meta information user stores during the LabelMe Manual Annotation step. For example, if for a bouding box, user assigns its label as "0 This is a Sun". AutoLabelMe will create a separate json file storing the new label for the bounding box and the meta information. This is helpful to save any additional information user wants for any object.

3. Tutorial

Types of Windows

1. Left Pane: Shows the image and matching for the current label. Red for the usual boxes. Blue for horizontally or vertically flipped boxes. Green for rotated. Although these are not final colors of the boxes. These are present only for checking.
2. Top-Right Pane: Buttons.
3. Bottom-Right Pane: Shows the image with all the matched templates. These image is helpful to recognise if every templates are matched or not. If any object is not annotated, it can be added using LabelMe.

Functions of Every Buttons and Boxes

1. Next Line $>>$: Template matching for next label.
2. $<<$Previous Line: Template matching for previous label.
3. $-$: Increases the threshold which results in less number of boxes.
4. $+$: Decreses the threshold which results in detection of more boxes.
5. Save Json: Saves a JSON file which can be read by LabelMe for further edits.
6. Save Images: Save the cropped vignettes from the image.
7. min: The minimum value of Rotation
8. max: The maximum value of Rotation
9. Rematch: Match again for the current label with the current setting. If min and max are provided, Rematch will match and activate rotation in template-matching.
10. Correct Label:Transform all the boxes to red boxes (no flip).

How to Run

1. Open Terminal.
2. Open Labelme. Write labelme or "path/to/labelme"
.
3. Create one bounding box per label.
4. Run AutoLabelme.py in Terminal python3 /path/to/AutoLabelme.py.
.
5. Open JSON and press Next Line $>>$ to start matching.
6. The Left Pane will show the images with boxes for current Label. The smaller bottom-right pane is for showing the image with all the matched templates.
7. Press $<<$Previous Line for viewing the matched boxes for the previous label.
8. Press $+$ for more boxes and $-$ for less boxes.
9. If you have rotated image, fill the rotation range or just enter min value. For example min=45 and max=90 will give the values 45,50,55,...90 or just enter min=45 which will only rotate the image once (45 degree).
10. The search space value is by default 2 which means the algorithm will check for the templates from two heights up to two heights down in the original image. Choose any value from 1 to 15. For example if your template starts from coordinate (200,200) and height and width is 100 and 100 respectively and search space=2, the algorithm will search for the template from (0,0) to (400,400) in the image. If you select more than 15 than it's just heights-the value. For example, if you choose 100 in the previous example, the it will be (0,100) to (300,300) where the templates will be searched in the image. In any case, value from 1 to 15 is sufficient.
11. Press Rematch button or Press Enter or Return in your keyboard.
12. (Optional) Sometimes if the template is symmetric, the algorithm picks up some templates as flipped, to fix this, press Correct Label.
13. Press Save Json to save a json file.
14. Open the saved json file in Labelme. Labelme will show the matched templates. Edit it if necessary.
15. Press Save Images in AutoLabelme if all the boxes are okay. This will save the matched templates in JPEG.

4. Future Work

In V2, AutoLabelme will be used independently to annotate manually or automatically.