Journal of Global Change Data & Discovery2020.4(2):183-190

[PDF] [DATASET]

Citation:Chen, J. Y., Li, H. W. , Zhang, G., et al.Dataset of Aircraft Classification in Remote Sensing Images[J]. Journal of Global Change Data & Discovery,2020.4(2):183-190 .DOI: 10.3974/geodp.2020.02.15 .

DOI: 10

Dataset of Aircraft Classification in Remote
Sensing Images

Chen, J. Y.1,2  Li, H. W.1  Zhang, G.1*  Wang, S.1  Chen, T. Q.1

1. Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xian 710119, China;

2. University of Chinese Academy of Sciences, Beijing 100049, China

 

 

Abstract: The aircraft is a typical object in remote sensing. However, the aircraft classification is still at an early stage in remote-sensing field. Authors choose several airports around the world, which are based on the public datasets DIOR, UCAS_AOD, NWPU VHR-10, DOTA and Google Earth. And select images of airplane about 3594. Then, the authors divide the data into different types according to the attention mechanism. Next, the authors divide seven types roughly in terms of wings and propellers. And continue to divide fourteen types in terms of color of aircraft and engine position. Last, the dataset of aircraft classification in remote-sensing field (OPT-aircraft _v1.0) has been completed. The datasets include seven types of aircraft: (1) Swept-back wing aircraft; (2) Swept-back aircraft with leading edge; (3) Forward-swept wing airplane with trailing edge; (4) Delta-wing aircraft; (5) Flat-wing aircraft; (6) Propeller aircraft; (7) Helicopter. The datasets are stored in .png format, which consists of 3589 data files. Its size is 69.3MB. (Compressed to 1 file, 69.3MB). The datasets not only identify aircraft in a fine-grained manner and simulate different types of aircraft but also provide experimental data for aircraft image classification research in remote sensing field.

Keywords: Remote sensing image classification; The categorical dataset of aircraft; Google Earth; Attention mechanism

1 Introduction

With the rapid development of science and technology, the high-tech informatization degree is gradually enhanced. Aircraft is a kind of important target in the field of remote sensing. It is a hot research direction to identify the position, type and number quickly and accurately during the detection of aircraft target. The existing fine-grained visual classification dataset of aircraft is FGVC-aircraft [2], which collects 120 aircraft types in natural scenes. However, the FGVC-aircraft dataset collects aircraft images in natural scenes, and there are few datasets of aircraft classification in the field of remote sensing. It is difficult to identify abnormal aircraft in the field of remote sensing. Thus, it is urgent to solve the shortage of aircraft classification dataset in remote sensing field.

In order to speed up the research of aircraft recognition technology in remote sensing field, it is very important to sort out the aircraft classification dataset labeled with remote sensing images. In this paper, the aircraft remote sensing image classification dataset OPT-Aircraft _v1.0 is established based on the research area of several airports around the world and includes different types of aircraft data of different airports. The aircraft remote sensing data are classified according to the corresponding classification criteria. The dataset provides reference for aircraft data classification in remote sensing images, and promotes the development of aircraft type classification further in remote sensing field.

2 Metadata of Dataset

The metadata of aircraft classification in remote sensing images dataset OPT-Aircraft _v1.0[3] is summarized in Table 1. It includes the dataset full name, short name, authors, year of

 

Table 1  Summary of the OPT-Aircraft _v1.0 metadata

Items

Description

Dataset full name

Dataset of Aircraft Classification in Remote Sensing Field

Dataset short name

OPT-Aircraft _v1.0

Authors

Chen, J. Y. Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, chenjunyu2016@opt.cn

Li, H. W. Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, lihaiwei@opt.ac.cn

Zhang, G Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, gzhang@opt.ac.cn

Wang, S Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, wangshuang@opt.ac.cn

Chen, T. Q. Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, chentieqiao@opt.ac.cn

Geographical region

airports around various countries all over the world

Year

1989-2020

Data format

.png

 

 

Data size

69.3MB

 

 

Data files

The dataset is composed of seven main class and fourteen subclasses, which are respectively:

1 Swept_back_wing_aircraft (1.1 Sp_bk_I, 1.2 Sp_bk_II, 1.3 Sp_bk_III)

2 Leading_edge_sp_bk_aircraft (2.1 Ld_sp_bk_I, 2.2 Ld_sp_bk_II)

3 Trailing_edge_forward_sp_airplane

4 Delta_aircraft

5 Flat_wing_aircraft (5.1 Ft_Eg_I, 5.2 Ft_Eg_II)

6.Propeller_aircraft (6.1Propeller_airplane_I,6.2Propeller_aircraft_II,6.3Propeller_aircraft_III, 6.4 Propeller_aircraft_IV)

7 Helicopter

Foundation(s)

Western Young Scientist Program of the Chinese Academy of Sciences under Grant (XAB2017B19)

Data publisher

Global Change Research Data Publishing & Repository, http://www.geodoi.ac.cn

Address

No. 11A, Datun Road, Chaoyang District, Beijing 100101, China

Data sharing policy

 

Data from the Global Change Research Data Publishing & Repository includes metadata, datasets (data products), and publications (in this case, in the Journal of Global Change Data & Discovery). Data sharing policy includes: (1) Data are openly available and can be free downloaded via the Internet; (2) End users are encouraged to use Data subject to citation; (3) Users, who are by definition also value-added service providers, are welcome to redistribute Data subject to written permission from the GCdataPR Editorial Office and the issuance of a Data redistribution license; and (4) If Data are used to compile new datasets, the ‘ten per cent principal’ should be followed such that Data records utilized should not surpass 10% of the new dataset contents, while sources should be clearly noted in suitable places in the new dataset[1]

Communication and searchable system

DOI, DCI, CSCD, WDS/ISC, GEOSS, China GEOSS, Crossref

 

the dataset, temporal resolution, spatial resolution, data format, data size, data files, data publisher, and data sharing policy, et al.

3 Methods

3.1 Algorithmic Principles

The FGVC-aircraft dataset is an open dataset for aircraft classification in natural scenes, but there is no open dataset on aircraft classification in remote sensing field. The aircraft images in remote sensing scene and in natural scene have different characteristics. The remote sensing images are obtained by remote sensor from high altitude and the shooting distance. For the most of acquired images, it is impossible to mark the type and manufacturer clearly without prior conditions. However, the aircraft image in the natural scene is obtained from the side of the fuselage, which can see the structure of the fuselage clearly and obtain the relevant type information. Therefore, the classification rules for remote sensing aircraft can’t be the same as the FGVC-Aircraft [2] datasets. Therefore, it is necessary to reestablish classification rules for aircraft images in remote sensing area. In order to get the dataset of aircraft classification in remote sensing images (OPT-Aircraft _v1.0), firstly, we obtain the aircraft images from the public high-resolution remote sensing images. Next, the aircraft object is cut out and screened out the better images from the aircraft data. And then, we classify the aircraft into main classes and subclasses. Finally, we enhance the data and complete dataset of aircraft classification in remote sensing (OPT-Aircraft _v1.0).

3.2 Data Collection or Processing

The flowchart of the dataset processing is shown in Figure 1, which includes seven parts: initial collection of aircraft, extraction of aircraft targets, secondary selection, normalization, main classification, subclassification and data enhancement.

 

 

Figure 1  Flowchart of the dataset processing

(1) Initial collection of aircraft in remote sensing field: Public datasets DIOR[4]UCAS_AOD[5]NWPU VHR-10[6][7][8]DOTA[9][10] and high resolution RGB aircraft remote sensing images in Google Earth are chosen as aircraft dataset. The aircrafts in Google Earth are captured from Beijing Capital International Airport, Amsterdam Airport, Phoenix Sky Harbor International Airport and other airports over the world. The data collection obtains the aircraft images from different sensors, different times, seasons and different light intensities, and thus makes the data have diversity within the class.

(2) Extraction of aircraft targets: Using the tags in the public dataset and the manual annotation of Google Earth images, the aircraft data can be extracted automatically and manually according to the labeled coordinates.

(3) Secondary selection of aircraft in remote sensing: The primarily collected aircraft data has the following problems: 1. The redundant objects around the aircraft, such as boarding stairways, trucks, personnel, and adjacent aircraft, will affect the target aircraft. 2. Complex backgrounds like white zebra crossing, bright stripes and shadows of surrounding buildings, which have an impact on the spectral information of aircraft and need to be eliminated. 3. The aircraft at the edge of remote sensing images, which are acquired by the calibrated coordinate points, is not complete and needs to be manually screened out. 4. It is difficult to separate data from background in cases such as strong exposure, weak exposure, fog, and small target. These data are filtered out by professionals to extract high quality aircraft remote sensing targets.

(4) Normalization for the aircraft remote sensing data: After filtering the remote sensing aircraft data through human computer interaction, the data size is normalized to 96 × 96.

(5) The main classification of aircraft: The normalized remote sensing aircraft data are roughly classified. Since the remote sensing aircraft images are acquired from high altitude, it is necessary to redefine the criteria for its classification. According to the mechanism of human visual attention and the research results of 15 related research fields, the focus of aircraft differentiation lies in the difference of wing and propeller. And thus, the aircraft could be classified to the following types: 1. Swept-back wing aircraft; 2. Swept-back aircraft with leading edge; 3. Forward-swept wing airplane with trailing edge; 4. Delta-wing aircraft; 5. Flat-wing aircraft; 6. Propeller aircraft; 7. Helicopter.

(6) Subclassification of aircraft remote sensing data: Different types of aircraft need to be distinguished in different fields. In order to further improve the accuracy of aircraft classification, the aircraft is roughly divided according to the wing and propeller, and the subclassification of the aircraft is carried out according to the distinctive characteristics such as the different positions of the engine and the aircraft color.

The swept-back wing aircraft in the OPT-Aircraft _v1.0 dataset can be divided into three types according to wing width, engine position and fuselage color. The swept-back wing aircraft of which the wing is narrow, the fuselage has bright colors (such as white, blue, color, etc.) and the engine is at the tail is named as Sp_bk_I; while those of which the wing is narrow, the fuselage has bright colors (such as white, blue, color, etc.) and the engine is not at the tail is named as Sp_bk_II; and those with wider wings and grey or dark green fuselage are classified as Sp_bk_III.

The propeller aircraft in the OPT-Aircraft _v1.0 dataset also has its distinctive characteristics in remote sensing images. According to the ratio of propeller to fuselage and fuselage color, propeller aircraft can be divided into four categories. Propeller aircraft with small propeller and bright color (white, color, etc.) is named as Propeller_airplane_I, while propeller aircraft with small propeller and green or gray fuselage is named as Propeller_airplane_II; propeller aircraft with large propeller and bright fuselage (white, color, etc.) is named as Propeller_airplane_III, and the propeller with large propeller and green or gray fuselage is named as Propeller_airplane_IV.

As the same way, there are differences in the position of the engine in swept-back aircraft with leading edge and flat-wing aircraft in the OPT-Aircraft _v1.0 dataset, which be visually distinguished. swept-back aircraft with leading edge can be further divided into l swept-back aircraft with leading edge with engine in the tail named as Ld_sp_bk_I, and leading edge swept wing aircraft with engine not in the tail named as Ld_sp_bk_II. In the flat wing aircraft, the flat-wing aircraft with engine in the tail are named as Ft_Eg_I, and the flat wing aircraft with engine not in the tail are named as Ft_Eg_II.

The classified aircrafts involved in the whole data include all kinds of general aircraft types, involving a wide range of remote sensing aircraft applications and strong compatibility.

(7) remote sensing data enhancement: the dataset of aircraft classification in remote sensing (OPT aircraft_ V1.0) can be enhanced and promote the experimental accuracy according to different experimental requirements, i.e., the data can be amplified by 3-7 times according to the experimental conditions by image reversal, rotate by 45°, 90°, 135°, 180° and 270°.

4 Results and Validation

4.1 Data Composition

The dataset of aircraft classification in remote sensing (OPT-aircraft _V1.0) is for a classification of aircraft images in the field of remote sensing. We collect a total of 3,954 valid aircraft, which are divided into 7 main classes and 14 subclasses. The 7 main classes are: 1. Swept-back wing aircraft, 2. Swept-back aircraft with leading edge, 3. Forward-swept wing aircraft with trailing edge, 4. Delta-wing aircraft, 5. Flat wing aircraft, 6. Propeller aircraft, 7. Helicopter. For the 14 subclasses, the naming rules are divided by decimal point. The number before the decimal point represents the main classes, and the number after the decimal point represents the subclass of the same category. Specific as follows: 1.1 Sp_bk_I, 1.2 Sp_bk_II, 1.3 Sp_bk_III, 2.1 Ld_sp_bk_I, 2.2 Ld_sp_bk_II, 5.1 Ft_Eg_, 5.2 Ft_Eg_II, 6.1 Propeller_airplane_I, 6.2 Propeller_aircraft_II, 6.3 Propeller_aircraft_III, 6.4 Propeller_aircraft_IV. Table 2 shows the details of the specific classification and the English name.

Figure 2 shows the 12 subclasses in the dataset of OPT-Aircraft _v1.0. Each row displays 10 images in the same category randomly.

Table 2  Dataset of Aircraft Classification in Remote Sensing ImagesOPT-Aircraft _v1.0

main classes

subclasses

Number

1 Swept_back_ wing_aircraft

1.1 Sp_bk_I

656

1.2 Sp_bk_II

201

1.3 Sp_bk_III

320

2 Leading_edge_ sp_bk_aircraft

2.1 Ld_sp_bk_I

104

2.2 Ld_sp_bk_II

75

3 Trailing_edge_ forward_sp_airplane

 

21

4 Delta_aircraft

 

192

5 Flat_wing_aircraft

5.1 Ft_Eg_I

134

5.2 Ft_Eg_II

1088

6 Propeller_aircraft

6.1 Propeller _airplane_I

104

6.2 Propeller_aircraft_II

414

6.3 Propeller_aircraft_III

242

6.4 Propeller_aircraft_IV

39

7 Helicopter

 

4

A total of 7 classes

There are 14 subclasses in total

Total 3594

 

 

Figure 2  OPT-Aircraft _v1.0 partial data display

4.2 Data Products

The dataset of aircraft classification in remote sensing (OPT-aircraft _v1.0) consists of 7 main classes and 14 subclasses. The spatial distribution of some data is shown in Figure 2.

 

Figure 3  Visualization of data of 5 subclasses (1.3 Sp_bk_III, 2.2 Ld_sp_bk_II, 4 Delta_aircraft, 5.2 Ft_Eg_II, 6.2 Propeller_aircraft_II)) by using the t-SNE

Figure 3 shows the two-dimensional space representation of the five subclasses (1.3 Sp_bk_III, 2.2 Ld_sp_bk_II, 4 Delta_aircraft, 5.2 Ft_Eg_II, 6.2 Propeller_aircraft_II) by using the t-SNE algorithm. The same color represents the same subclass. For these five subclasses, we can see that the distance between the same class is small and the distance between different classes is large. The space after dimension reduction is separable.

Combining Figures 2 and Figures 3, we can see that each of the 14 subclasses has its own characteristics. This method of classification can cover different aircraft remote sensing data. More importantly, we can distinguish 14 subclasses further according to the color of the aircraft, engine position and so on. According to the classification rule of 3.2, we use the remote sensing image of the aircraft in TGRS-HRRSD-Dataset [11] to verify. It can be found that we can classify the aircraft of TGRS-HRRSD-Dataset into 14 subclasses accurately. Finally, the dataset is stored in. PNG format with a data size of 69.3 MB.

5 Discussion and Conclusion

In the remote sensing data processing, the fine-grained recognition of remote sensing image is still in the initial exploration stage. The shortage of classification data of aircraft in the field of remote sensing brings many difficulties to the research of scientific problems. The dataset of OPT-Aircraft _v1.0 first screens aircraft images from public datasets and high-resolution remote sensing images in Google Earth. Then, we obtain the classification dataset of aircraft remote sensing images by using the human-computer interaction and visual interpretation according to the classification criteria. This classification criteria are combination feature of wing shape, number of engines, color of the fuselage and so on. Finally, we divide the remote sensing image of aircraft into 7 main classes and 14 subclasses. It covers aircraft with different characteristics in the field of remote sensing and has a complete data division criterion relatively.

By using the t-SNE, we can see the separability of the 14 subclasses in low dimensional space. This is helpful to improve the accuracy of fine-grained recognition and classification of aircraft images. What's more, the establishment of OPT-Aircraft _v1.0 marks different labels of aircraft. In the simulation of remote sensing targets, the generative precision of different types of aircraft can be improved by using the generative adversarial networks (GAN).

The next task is to complete the realistic simulation of aircraft in the field of remote sensing and fine-grained recognition of aircraft based on OPT aircraft_ V1.0. To promote the construction of system of simulation and recognition for the different kinds of aircraft in remote sensing scene. This is very beneficial to the risk and cost estimation in the load development stage and special target identification. Of course, the dataset of OPT-Aircraft _v1.0 can also be improved. In the following stage, we can continue to collect and sort out more aircraft images according to the division criteria in this paper, or use the network of GAN to generate the required data to supplement the dataset of OPT-Aircraft _v1.0. To promote the precision of fine-grained recognition and target generation. It is a process of mutual promotion between them.

Author Contributions

Li, H. W. designed the algorithms of dataset. Chen, J. Y. and Chen, T. Q. contributed to the data processing and analysis. Li, H. W. and Chen, J. Y. designed the model and algorithm. Zhang, G., Wang, S. and Chen, J. Y. verified the data. Chen, J. Y. wrote the data paper.

Acknowledgements

This work was supported by the Western Young Scientist Program of the Chinese Academy of Sciences under Grant NO. XAB2017B19. Thanks to the editors and all the anonymous reviewers for their constructive comments.

References

[1]       GCdataPR Editorial Office. GCdataPR Data Sharing Policy [OL]. DOI: 10.3974/dp.policy.2014.05 (Updated 2017).

[2]       Maji S, Rahtu E, Kannala J, et al. Fine-Grained Visual Classification of Aircraft[J]. Computer Vision and Pattern Recognition, 2013.

[3]       Chen, J. Y., Li, H. W., Zhang, G., et al. Dataset of aircraft classification in remote sensing images [DB/OL]. Global Change Research Data Publishing & Repository, 2015. DOI: 10.3974/geodb.2020.03.25.V1.

[4]       Li, L., Xin, X. Z., Zhang, H. L., et al. A method for estimating hourly photosynthetically active radiation (PAR) in China by combining geostationary and polar-orbiting satellite data [J]. Remote Sensing of Environment, 2015, 165: 14-26.

[5]       Zhu H, Chen X, Dai W, et al. Orientation robust object detection in aerial images using deep convolutional neural network[C]// IEEE International Conference on Image Processing. IEEE, 2015.

[6]       Cheng G, Han J, Zhou P, et al. Multi-class geospatial object detection and geographic image classification based on collection of part detectors[J]. ISPRS Journal of Photogrammetry & Remote Sensing, 2014, 98(dec.): 119-132.

[7]       Cheng G, Han J. A survey on object detection in optical remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 117(jul.): 11-28.

[8]       Cheng G, Zhou P, Han J. Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12): 7405-7415.

[9]       Xia G S, Bai X, Ding J, et al. DOTA: A Large-scale Dataset for Object Detection in Aerial Images[J]. 2017.

[10]    Ding J, Xue N, Long Y, et al. Learning RoI Transformer for Detecting Oriented Objects in Aerial Images. [J]. Computer Vision and Pattern Recognition, 2018.

[11]    Zhang Y, Yuan Y, Feng Y, et al. Hierarchical and Robust Convolutional Neural Network for Very High-Resolution Remote Sensing Object Detection[J]. IEEE Transactions on Geoscience & Remote Sensing, 2019: 1-14.

Co-Sponsors
Superintend