Overview

In conjunction with MVA2023, we host the Small Object Detection Challenge for Spotting Birds. The task of this challenge is to detect relatively small birds with few pixel features on images, which is categorized to the Small Object Detection (SOD)[5] problem, and is based on the paper by S. Fujii, et al. [1]. In contrast to general detection, SOD has its own difficulties and thus is a hot topic in the Computer Vision community, as presented in the ECCV2020 workshop [2] and the ACCV2022 workshop [3] in recent years.

This challenge not only raises academic issues in Computer Vision but also promotes practical technology developments that are expected to be employed in real-world applications. For example, distant-bird detection is an important function for unmanned aerial vehicles (UAVs) such as drones in order to avoid bird attacks and drive away harmful birds that destroy fields and rice paddies. Thus, this Challenge is in line with MVA's policy to build bridges between academia and industry.

News

• [ May 29, 2023 ] The organizer has announced the winners to the participants. The winners will be announced on this website at a later date.
• [ April 19, 2023 ] We have decided to add the following conditions to the prize selection.
  • If a participant who has registered for both categories wins in the Research category and wins 1st place in the Development category, the organizer will ask the winner by email which category to choose.
  • Winners who receive the email are required to respond within 24 hours. If there is no answer, we will adopt the winner with the larger prize money.
• [ April 6, 2023 ] We released the Google Form for final submission.
• [ April 6, 2023 ] The papers submitted by Research Category winners will be included in the IEEE Xplore and the IEICE Proceedings Series as "MVA Workshop on Small Object Detection Challenge for Spotting Birds2023 proceedings".
• [ April 6, 2023 ] The summary paper of this challenge by the Research Category winners and the organizer will be included in the IEICE Proceedings Series as "MVA Workshop on Small Object Detection Challenge for Spotting Birds2023 proceedings".

Challenge Category

The Challenge has two categories, "Research Category" and "Development Category" in order to encourage a wide range of participants from academia and industry, from young to senior. Participants must select one or both of the categories at the time of final submission. In both categories, you can create a team and participate as a member of a team. The details of each category are as follows.

1. Research Category

In this category, participants are expected to not only have the top performance but also propose novel technical contribution(s). Accordingly, each participant of this category is required to submit a technical paper. This paper must follow the same guidelines as the main conference so that each paper consists of four pages or less excluding references. See Submission in the MVA2023 website for details on submitting your paper. In addition, the training and test codes and the trained model for the private test phase must be submitted to check the reproducibility of the results. Papers from the winners will be published in the MVA2023 proceedings. The paper submitted by participants of Research Category will be double-blind peer reviewed for contributions (private test scores are also taken into account) and scientific quality as any other MVA's paper.

2. Development Category

In this category, participants can focus solely on achieving the top performance. No novel technical contribution is required. Therefore, each participant is only required to submit the codes and the trained model.

3. Both (updated)

Submissions will be evaluated first in the Research Category, and if it does not win in the Research Category, it will be evaluated in the Development Category. note: If you win in the research category, you cannot win in the development category.

Task

The task is to detect birds on images, and there is only a single class. The goal of this task is to exhibit the state-of-the-art performance in object detection metrics.

Dataset

The dataset is split into training/public test/private test sets. Participants can download the images and annotations of the training data, and the images of the public test data. The annotations of the public test data are hidden but the participants can obtain the evaluation score once their detection results are online submitted via the CodaLab web platform. Participants cannot access the private test images. The private test will be conducted manually by the challenge organizer. The trainig data includes an extended version of the publicly available data (train1) published in [4] and newly released data for this competition (train2).

• Images and instances:
  • Train :
    • Train1 (Modified based on [4]) :Consists of 47,260 images with 60,971 annotated bird instances.
    • Train2:Consists of 9,759 images with 29,037 annotated bird instances.
  • Public test : Consists of 9,699 images.
  • Private test : Consists of approximately 20,000 images.
• Data format :
  • Input : Image
  • Annotation : COCO format

After the challenge, the public test evaluation server will continue to run on CodaLab to promote further research on small object detection.

Evaluation

The performance metrics is mAP@50. In Development Category, the rank is automatically determined based on the metric score. In Research Category, the challenge organization members evaluate the metric scores of the evaluation metrics, the novelty of the method, and the quality of the paper comprehensively.

Baseline

We release the baseline code that is based on [1].

https://github.com/IIM-TTIJ/MVA2023BirdDetection

Discussion

In this Challenge, a Discord channel where participants can discuss with each other is available. After you join this Challenge in CodaLab, you will also receive an invitation email to join the channel. Please press GOOD REACTION for discussions you find particularly useful. Discussions in this channel will be evaluated for the Best Booster Award.

Important dates

Prizes & Awards

This Challenge offers cash prizes and awards, and free admission to MVA2023. In addition, participants who are highly-ranked in the Research Category will receive the right to present their work in the special session in the main conference. The details of the prizes and awards are as follows.

1.Research Category

† The registration fee is free for oral presenters.

2.Development Category

† The registration fee is free for oral presenters.
∗ The recipient of the Winner Award receive the right to present their work in the special session in the main conference.

3.Common award

A chance to win the Best Booster Award will be given to participants in both categories. This award will be presented to the individual who are beneficially active in discussions on the Discord channel. The evaluation criteria will be based on the content of the discussion and the number of good reactions. The winner of this award is also invited to the conference free of charge.

Registration

If you wish to participate in this challenge, please register for the challenge in CodaLab to receive email notifications for the challenge, and you can then join the Discord channel via email.

CodaLab page

Submission

For submitting the results of the public test data, all participants must submit the zip file in which the detection results are discribed in the json files. In the private test phese, the training and test codes, and the trained model files must be submitted before the deadline via the Google Form. In addition, participants in the Research Category are required to submit a technical paper. See Submission in the MVA2023 website for details on submitting your paper in this category. Note that, different from papers submitted to the main conference, (1) no supplemental material is permitted and (2) the submission site is available in the Google Form. The paper submission form has been emailed to Research category participants. Please check your email. The paper for this Challenge can be short and simple (e.g., just one-page manuscript), while the page limitation is four pages. Only the brief description of the method and its novel contributions are sufficient.

Challenge organizers

Citing Small Object Detection Challenge for Spotting Birds 2023

Note: Not yet published and booktitle is tentative.
@inproceedings{mva2023_sod_challenge,
  title={{MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results}},
  author={Yuki Kondo and Norimichi Ukita and Takayuki Yamaguchi and Hao-Yu Hou and Mu-Yi Shen and Chia-Chi Hsu and En-Ming Huang and Yu-Chen Huang and Yu-Cheng Xia and Chien-Yao Wang and Chun-Yi Lee and Da Huo and Marc A. Kastner and Tingwei Liu and Yasutomo Kawanishi and Takatsugu Hirayama and Takahiro Komamizu and Ichiro Ide and  Yosuke Shinya and Xinyao Liu and Guang Liang and Syusuke Yasui},
  booktitle={2023 18th International Conference on Machine Vision and Applications (MVA)},
  note={\url{https://www.mva-org.jp/mva2023/challenge}},
  year={2023}}

@misc{baselinecode_mva2023_sod_challenge,
  title={{Baseline code for SOD4SB by IIM-TTIJ}},
  author={Kaikai Zhao and Riku Miyata and Yuki Kondo and Kazutoshi Akita},
  license={Apache-2.0},
  url={\url{https://github.com/IIM-TTIJ/MVA2023SmallObjectDetection4SpottingBirds}},
  year={2023}}

@misc{sodbchallenge2023misc,
  title={{MVA2023 Small Object Detection Challenge for Spotting Birds}},
  author={Yuki Kondo and Norimichi Ukita and Takayuki Yamaguchi},
  howpublished={\url{https://www.mva-org.jp/mva2023/challenge}},
  year={2023}}

References

[1].

S. Fujii, K. Akita, N. Ukita, "Distant Bird Detection for Safe Drone Flight and Its Dataset", 17th International Conference on Machine Vision and Applications (MVA), 2021.

[2].

X. Yu, Z. Han, Y. Gong, N. Jan, J. Zhao, Q. Ye, J. Chen, Y. Feng, B. Zhang, X. Wang, Y. Xin, J. Liu, M. Mao, S. Xu, B. Zhang, S. Han, C. Gao, W. Tang, L. Jin, M. Hong, Y. Yang, S. Li, H. Luo, Q. Zhao, H. Shi. "The 1st tiny object detection challenge: Methods and results", European Conference on Computer Vision Workshop (ECCVW), 2020. https://rlq-tod.github.io/challenge1.html

[3].

“Deep Learning-Based Small Object Detection from Images and Videos”, Asian Conference on Computer Vision Workshop (ACCVW), 2022. https://sites.google.com/view/dlsod2022

[4].

https://github.com/kakitamedia/drone_dataset

[5].

G. Chen, H. Wang, K. Chen, Z. Li, Z. Song, Y. Liu, W. Chen, A. C. Knoll, "A Survey of the Four Pillars for Small Object Detection: Multiscale Representation, Contextual Information, Super-Resolution, and Region Proposal", IEEE Trans. Syst. Man Cybern. Syst. 52(2): 936-953, 2022.

Contact

Google form

Please share!

Tweet