Journal of Global Change Data & Discovery2019.3(3):290-296

[PDF] [DATASET]

Citation:Wang, D. L., Luo, W..Bayberry Tree Recognition Dataset Based on the Aerial Photos and Deep Learning Model .Journal of Global Change Data & Discovery2019.3(3):290-296 .DOI: 10.3974/geodp.2019.03.10 .

DOI: 10

Bayberry Tree Recognition Dataset Based on the Aerial Photos and Deep Learning Model

Wang, D. L.1* Luo, W.2

1. Key Laboratory of Land Surface Pattern and Simulation, Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China??

2. Center for Intelligent Manufacturing Electronics, Institute of Microelectronics, Chinese Academy of sciences, Beijing 100029, China

Abstract:Bayberry (Myricarubra) is an evergreen tree with 5-15 meters high; the diameter of the tree is about 60 cm at breast height and a crown of more than 5 meters. Bayberry tree is widely distributed in the southern of the Yangtze River Basin of China. Bayberry tree grows in acid red soil below 1500 meters above sea level and sunny mountain slopes. Its fruit is a special fruit with geographical characteristics in the southern of the Yangtze River Basin. The development of intelligent model for recognizing bayberry trees based on unmaned aerial systems (UAS) images is of great significance to extract the position and crown information of bayberry trees, which can then be used for precise pesticide spraying and yield estimation. The authors used DJI Phantom 4 unmanned aerial systems (UAS) to take the aerial photography in Dayangshan Forest Park, Yongjia County, Zhejiang Province from January 23 to 24, 2019. Polygonal markers of Bayberry trees were then labeled before training a deep learning model, Mask RCNN (Mask Region Convolutional Neural Networks), to automatic identify the bayberry trees. The recognition results were evaluated by visual interpretation method. The experimental results show that Mask RCNN has a high accuracy for bayberry tree recognition. The overall true positive rate is 90.08%, the false positive rate is 9.62%, and the loss positive rate is 9.92%. The experimental dataset of the presented bayberry tree recognition deep learning model includes: (1) 3,080 UAS images captured in Dayangshan Forest Park, Yongjia County, Zhejiang Province, with an image size of 5,472Í3648; (2) Marked bayberry tree training samples, including 284 image tiles; and (3) bayberry tree recognition results by the deep learning model, including 14 image tiles. The dataset is archived in .jpg and .json data formats, consists of 3690 data files with data size of 25.6 GB (Compressed to 71 files with data size of 25.5 GB).

Keywords:Bayberry tree; recognition; UAS images; Mask RCNN

1 Introduction

As a branch of machine learning, deep learning can automatically learn some features which are difficult to extract manually from big data, and extract more useful feature representations with a higher accuracy than those of traditional machine learning algorithms (such as ANN)[1–2]. It has been reported by journals such as Nature[2–3], PNAS[4], etc. Deep learning has become an important tool in data processing fields such as remote sensing[1], and earth system science[2]. Kellenberger et al.[5] trained a convolutional neural network (CNN, a recent family of deep learning algorithms) to recognize more than 20 types of large wildlife based on thousands of UAS images with a resolution of about 4cm (there was no distinction between species in the experiment), and achieved a higher accuracy than traditional EESVM shallow machine learning[6] (30%@80% recall vs. 10%@75% recall rate). Norouzzadeh et al[7] combined nine deep neural network models, such as AlexNet, Google LeNet and ResNet, to carry out the recognition and classification of animals in ground-based camera-trap images, and the accuracy is similar to that of volunteers (the accuracy of recognition is 96.6%). Madec et al.[8]developed a wheat ear recognition algorithm based on Faster RCNN, and the relative root mean square error (rRMSE) is 5.3%, which is even better than the first-round human-computer interaction marking accuracy[9]. Deep learning typically has a higher accuracy and better probability than traditional machine learning algorithms. However, it is a black-box solution, and extremely large training datasets and appropriate model adjustment are needed to improve the accuracy and efficiency when applying deep learning. These issues are controversial within the remote sensing community[3,5].

With the emergence of models such as Faster RCNN[8] and Mask RCNN[10], the precision and speed of deep learning models are close to or beyond human beings in object recognition and other fields[7]. This study presented a Mask RCNN-based model to identify and mark bayberry trees from high resolution UAS images based on TensorFlow platform. The recognized results were inspected by a visual interpretation method and showed that Mask RCNN can be used for identify bayberry tree with a high accuracy.

2 Metadata of Dataset

The metadata of bayberry tree recognition dataset based on the aerial photos and deep learning model[11] is summarized in Table 1. It includes the dataset full name, short name, authors, geographical region, year of the dataset, temporal resolution, spatial resolution, data format, data size, data files, data publisher, and data sharing policy, etc.

Table 1 Metadata summary of Bayberry tree recognition dataset based on the aerial photos and deep learning model

Items

Description

Dataset full name

Bayberry tree recognition dataset based on the aerial photos and deep learning model

Dataset short name

BayberryTreeRecogData

Authors

Wang, D. L. 0000-0002-1377-8394, IGSNRR/CAS, wangdongliang@igsnrr.ac.cn

Luo, W.0000-0003-2226-8414, IME/CAS, luowei2@ime.ac.cn

Geographical region

Dayangshan Forest Park, Yongjia County, Zhejiang province. 28°17¢N-28°19¢N, 120°26¢E-120°28¢E

Year

January 23-24, 2019

Temporal resolution

once

Spatial resolution

3 cm

Data format

.jpg, .JSON

   

Data size

25.6 GB

   

Data files

UAS images, marked bayberry tree training images, test images, and bayberry tree recognition results

Foundations

Ministry of Science and Technology of PR. China (2017YFC0506505, 2017YFB0503005); Chinese Academy of Sciences (XDA23100200)

Data publisher

Global Change Research Data Publishing & Repository, http://www.geodoi.ac.cn

Address

No. 11A, Datun Road, Chaoyang District, Beijing 100101, China

(To be continued on the next page)

(Continued)

Items

Description

Data sharing policy

Data from the Global Change Research Data Publishing & Repository includes metadata, datasets (data products), and publications (in this case, in the Journal ofGlobal Change Data & Discovery). Data sharing policy includes: (1) Data are openly available and can be free downloaded via the Internet; (2) End users are encouraged to use Data subject to citation; (3) Users, who are by definition also value-added service providers, are welcome to redistribute Data subject to written permission from the GCdataPR Editorial Office and the issuance of a Data redistribution license; and (4) If Data are used to compile new datasets, the ‘ten per cent principal’ should be followed such that Data records utilized should not surpass 10% of the new dataset contents, while sources should be clearly noted in suitable places in the new dataset[7]

3 Methods

3.1 Study Area

Dayangshan Forest Park is located in Yongjia County, Zhejiang province, 48.8 km away from Wenzhou City. The park has an area of 661.93 hm2. Across the park, elevation varies from 330 m to 893.1 m. The overall terrain slope across the park is about 30°. Annual mean temperature is about 17 oC and the frost-free period is about 270 days. Annual average precipitation ranges from 1,500-1,900 mm. The park is characterized as coniferous forests, bayberry gardens and Dendrobiumcandidum planting bases. Forests cover 87.4% of the park’s land surface.

3.2 Principle of Mask RCNN

Mask RCNN is a conceptually simple, flexible, and general framework for object instance segmentation, which can efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance[10]. The method is an extension of Faster RCNN that applies fully convolutional network (FCN) and Feature Pyramid Networks (FPN), and adds a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask RCNN works as follows. First, a ResNet-FPN backbone is used for extracting features from UAS images. Second, a region proposal network (RPN) is used to propose candidate object bounding boxes. Region of interests (RoI) are then determined from the candidate object bounding boxes. The target category, location, and a segmentation mask are predicted for each ROIs.

3.3 Technical Flow Chart

The survey area is divided into four blocks and flight routes are designed to obtain UAS images. Training and validation of the bayberry tree recognition model are then carried out using Mask RCNN. The three steps for training and evaluate of the bayberry tree recognition model are as follows (Figure 1).

Step 1: The UAS images are split into several sub-images and bayberry trees are labeled. It was not possible to train the model with the original 5472 x 3648 pixels images because of GPU memory limitation. The original images are therefore split into multiple sub-images (tiles) to ensure that each sub-image has a size of no more than 1,024 Í682 pixels. The bayberry trees are then labeled to construct bayberry tree samples. The samples are then divided into training samples (284 sub-images) and test samples (14 sub-images).

Step 2: The bayberry tree recognition model is built. Mask RCNN bayberry tree recognition model is built based on Google TensorFlow. The bayberry tree deep learning recognition model is obtained by iteratively training the model based on training samples.

Step 3: The model is evaluated. The trained model is used to recognize and mask bayberry trees for each image to obtain their segmentation masks in test samples. The accuracy of the model is calculated by comparing recognized results with visual interpretation results.

 

Figure 1 Technical flow chart showing generation of the dataset


4 Results and Validation

4.1 Data Composition

The experimental dataset includes: (1) UAS images captured in Yongjia County, Zhejiang Province (Table 2); (2) Bayberry tree training sub-images, tag files, and recognition results (Table 3).

4.2 Data Products

The survey area is divided into four regions. Figure 2 shows flight lines. The campaign captured 3,108 images with a resolution of approximately 3 cm and a size of 5,472Í3,648. The training images consisting of 284 sub-images (split from 18 original images) come from survey region 1, 2, and 4. The test images, consisting of 14 sub-images (split from 2 original images), are picked from survey region 4 and are used for evaluate recognition results, as shown in Figure 2. The bayberry polygons are labeled using Labelme (Figure 3). Recognition results are shown in Figure 4.

Table 2UAS images captured in Yongjia county, Zhejiang province

Items

Description

20190123DayangshanRegion01

Images of Region01

20190123 DayangshanRegion02

Images of Region02

20190123 DayangshanRegion03

Images of Region03

20190124 DayangshanRegion04

Images of Region04

20190124DayangshanRegion04continue01

Additional Images of Region04

20190124DayangshanRegion04continue02

Additional Images of Region04

Table 3 UAS bayberry tree training sub-images, bayberry tree tag files, and bayberry tree recognition results

Items

Description

Bayberry tree training sub-images, labeled bayberry tree polygons

284 training sub-images and tag files

Bayberry tree test sub-images and recognition results

14 test sub-images and 14 bayberry tree recognition resulting images

Figure 2 UAS flight lines, and distribution of training and test images

Figure 3 Example of bayberry tree polygons labeled using Labelme

Figure 4 Bayberry tree image (left) and the recognition results

4.3 Data Validation

The visual interpretation method is used to evaluate the accuracy of bayberry tree deep learning recognition model. The test samples includes 14 sub-images extracted from survey region 4.

True positive rate (TPR) is the ratio of the number of bayberry trees detected out by the model to the number of visual interpretation bayberry trees ().

(1)

False positive rate (FPR) is the number of targets () that are not bayberry trees but are identified as bayberry trees by the model, divided by the total number of targets () that are identified as bayberry trees by the model:

(2)

Loss Positive Rate (LPR) is the ratio of the number of bayberry trees are not identified as bayberry treesby the model () to the number of visual interpretation bayberry trees.

(3)

The recognition results and accuracy of bayberry tree recognition model based on MASK RCNN are shown in Table 4.

Table 4Results and accuracy of bayberry tree recognition model

Images

Number of bayberry trees

Number of true positives

Number of false positives

Number of loss positive rate

True positive rate (%)

False positive rate (%)

Loss Positive Rate (%)

2019012404DJI_c_0546-1

25

25

1

0

100.00

3.85

0.00

2019012404DJI_c_0546-2

29

27

0

2

93.10

0.00

6.90

2019012404DJI_c_0546-3

0

0

10

0

-

100.00

-

2019012404DJI_c_0546-6

39

34

0

5

87.18

0.00

12.82

2019012404DJI_c_0546-7

43

41

0

2

95.35

0.00

4.65

2019012404DJI_c_0546-8

10

7

0

3

70.00

0.00

30.00

2019012404DJI_c_0546-9

46

43

0

3

93.48

0.00

6.52

2019012404DJI_c_0546-10

34

34

0

0

100.00

0.00

0.00

2019012404DJI_c_0526-13

33

33

0

0

100.00

0.00

0.00

2019012404DJI_c_0526-18

22

19

7

3

86.36

26.92

13.64

2019012404DJI_c_0526-19

46

39

0

7

84.78

0.00

15.22

2019012404DJI_c_0526-20

34

29

0

5

85.29

0.00

14.71

2019012404DJI_c_0526-21

29

25

0

4

86.21

0.00

13.79

2019012404DJI_c_0526-22

28

25

1

3

89.29

3.85

10.71

Sum/average

418

381

19

37

90.08

9.62

9.92

5 Discussion and Conclusion

A Mask-RCNN-based bayberry tree recognition model is built based on 284 training sub-images. The recognition accuracy is relative high. In 14 test sub-images, 418 bayberry trees are found by the visual interpretation method, and 381 bayberry trees are detected out by the deep learning recognition model. The overall true positive rate is 90.08%, false positive rate is 9.62%, and loss positive rate is 9.92% (Table 4).

Where as, there is still considerable room to improve the accuracy of recognition. This paper only uses the basic MASK RCNN deep learning framework for bayberry tree recognition. The false positive rate and loss positive rate are both approximately 10%. The use of a negative sample learning method in combination with revising the model according to characteristics of bayberry trees will be addressed in our future work to improve the accuracy and reliability of recognition. The yields of bayberry trees will also be estimated according the relationship between ground-measured yields and the crown sizes of bayberry trees calculated from model-generated crown masks.

Author Contributions

Wang, D. L. contributed to the UAS data collection and processing. Wang, D. L. and Luo, W. made a general design for the development of datasets, labeled the samples, designed, trained, and evaluated the model, and wrote the data paper.

Acknowledgements

The authors would like to thank Zhejiang Senminglin Technology Co., Ltd. and the staff of Yangshan Forest Park for their support in this UAS campaign.

References

[1] Gitelson, A. A., Kaufman, Y. J., Stark, R., et al. Novel algorithms for remote estimation of vegetation fraction [J]. Remote Sensing of Environment, 2002, 80(1): 76-87.

[2] Gutman, G., Ignatov, A. Satellite-derived green vegetation fraction for the use in numerical weather prediction models [J]. Advances in Space Research, 1997, 19(3): 477-480.

[3] Zhang, L. J., Qin, Y. C., Zhang J. P., et al. County green GDP accounting based on EMA-MFA method and its spatial differentiation: A case of Henan province [J]. Journal of Natural Resources, 2013, 28(3): 504-516.

[4] Li, L., Xin, X. Z., Zhang, H. L., et al. A method for estimating hourly photosynthetically active radiation (PAR) in China by combining geostationary and polar-orbiting satellite data [J]. Remote Sensing of Environment, 2015, 165: 14-26.

[5] Liang, S., Li, X., Wang, J. Advanced Remote Sensing: Terrestrial Information Extraction and Applications [M]. Pennsylvania: Academic Press, 2012.

[6] Mu, X. H., Liu, X. H., Ruan, G. Y., et al. 1 km/5 day fractional vegetation cover product over China and the Association of Southeast Asian Nations for 2013 [DB/OL]. Global Change Research Data Publishing & Repository, 2015. DOI: 10.3974/geodb.2015.01.17.V1.

[7] GCdataPR Editorial Office. GCdataPR data sharing policy [OL]. DOI: 10.3974/dp.policy.2014.05 (Updated 2017).

[8] Serafy, E. S. The proper calculation of income from depletable natural resources [C]. Environmental Accounting for Sustainable Development: A UNDP—World Bank Symposium. Washington D C: World Bank, 1989: 10-18.

[9] Alberts, J. J., Takacs, M., Pattanayek, M. Natural organic matter from a Norwegian lake: possible structural changes resulting from lake acidification [C]. In: Ghabbour, E.A, Davies, G., (eds). Humic Substances: Versatile Components of Plants, Soil and Water. Cambridge, UK: The Royal Society of Chemistry, 2000.

[10]Ding, X. Study on distribution of crop’s structure in Heilongjiang [D]. Harbin: Northeast Agricultural University, 2014.

[11]Wang, D. L., Luo, W. Bayberry tree recognition dataset based on the aerial photos and deep learning model [DB/OL]. Global Change Research Data Publishing & Repository, 2019. DOI: 10.3974/geodb.2019.04.16.V1.

Go to Ruwix.com to learn the solution of the Rubik's Cube and other twisty puzzles like Pyraminx, Square-1 etc.

Co-Sponsors

Institute of Geographic Sciences and Natural Resources Research,Chinese Academy of Sciences

The Geographical Society of China

Parteners

Committee on Data for Science and Technology (CODATA) Task Group on Preservation of and Access to Scientific and Technical Data in/for/with Developing Countries (PASTD)

Jomo Kenyatta University of Agriculture and Technology

Digital Linchao GeoMuseum