Data Impact Score (DIS) —A Quantitative Method of Data Performance to the Data-driven Sciences

Liu, C.

Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing 100101, China

Abstract: Data Impact Score (DIS) is an index of a quantitative method of data performance for the data-driven sciences. It is based on a data citation paper published in a journal and the respective journal’s impact factor. DIS can be identified for data product (DIS_D) and data authors (DIS_A). The DIS_D models are consisted of models for data product (dataset) in a given year’s performance and multiple years or lifetime performance. The DIS_A models are consisted of models for each authors’ share for one data product DIS_D, in a given year’s sum of more than one data products share DIS_Ay and multiple years or lifetime performance DIS_Amysum. Examples were given in this paper to explain how DIS works for evaluating the datasets performance (DIS_D) published in Global Change Research Data Publishing & Repository (GCdataPR) and the authors’ impact (DIS_A) for the data-driven sciences.

Keywords: data impact score; quantitative method; dataset; data author; data citation

1.      Introduction

Data sharing principles were made by the International Council of Sciences, Committee of Data for Science and Technology, World Data System, Group of Earth Observations and the National Scientific Data Management Law of P. R. China, etc., which suggested open data as the default data sharing principle^[1–4]. It was emphasized that the data intellectual property should be protected, the data security and ethical should be guaranteed and the data quality should be controlled. Under which, more and more data were openly available and re-used^[5–7]. Following the open data and data re-use, a new issue comes, that is, how to evaluate the data performance. Besides the quantitative statistics of dataset visitors, number of data files, and data size download^[8], etc., the data re-use performance should be calculated, although few cases did so. The new challenging issues come: how to measure these data performance, how to credit the data authors’ contributions, and how to evaluate the data centers’ or data repositories’ performances.

Based on the experiences from the Global Change Research Data Publishing & Repository (GCdataPR), the definition, methodology and practices of the Data Impact Score (DIS) was descripted.   

2 Data Impact Score (DIS)—A Quantitative Method of Data Performance

2.1 Definition

Data Impact Score (DIS) is a quantitative method of data performance for the data-driven sciences; it is based on a data citation paper published in a journal and the respective journal’s impact factor. DIS could be used for quantifying the dataset (or data product) used, as well as re-used performance and for crediting the data author’s impact to the data-driven sciences.

2.2 Principles

(1) Principle of Data Re-use and Citation

The critical principle of data impact is its performance in data use and re-use. Similar to article (actually, it is journal performance), since Garfield founded the Institute for Scientific Information (ISI)^[8] and Science Citation Index (SCI)^[9–10], the journal performance was evaluated by journal impact factors through SCI, EI (Engineering Index)^[11], ISTP (Index to Scientific & Technical Proceedings)^[12], ISR (Index to Scientific Reviews)^[13] and more. Although there were differences among them, the article citation model was used for all of them. Furthermore, the Information and Documentation—Guidelines for Bibliographic References and Citations to Information Resources (ISO 690)^[14] has been adopted, which helps the National Standard of China^[15–16] , and they are also based on the documentation citation.

How to cite the dataset used or re-used? Liu, C. et al. classified the data products into five groups^[17], which are as follows: (1) data published and openly available by a publisher; (2) data archived in WDS (World Data System)^[18] and openly available; (3) data archived in a National Data Center and openly available^[19–20]; (4) data products are online by individual person or a research group without data peer review; (5) data products were internal use only. The first three groups of data products should be cited in the references when they were re-sued, and the last two groups of data products should be noticed as the footnotes when they were re-used.

(2) Principle of Differences among the Data Cited Journals

Because of the differences of journal performance, especially in quantitative performance (Impact Factor, IF)^[21], it should be considered in data impact score.

3 DIS Model for Data Product Performance (DIS_D)

Two different concepts are identified in this paper, one is DIS for data products performance (DIS_D) and one is DIS for creating data authors’ contribution (DIS_A).

3.1 DIS Model for a Data Product in a Given Year’s Performance (DIS_Dy) 

                                                                                                      (1)

where DIS_Dy is the Data Impact Score (DIS) for a dataset in a year of y; y is the year for calculating the DIS; C_i is the number of data product was cited in i journal; IF_i is the impact factor of i journal in the year of y when the paper was published and the data product was cited in the paper; n is the number of papers where the data product was cited.

3.2 DIS model for a data product for multiple year’s performance (DIS_Dmy)

                                                                                               (2)

Where DIS_Dmy is the Data Impact Score summary since the data product was published; my is the total number of years from when the data product was published to the calculating year (1 means the data published year); C_i is the number of data product was cited in i journal; IF_i is the impact factor of i journal in the year of y when the paper published and the data product was cited at the paper; n is the number of papers where the data product was cited.

3.3 Flowchart of DIS for a Data Product

The flowchart of the DIS calculation for a data product is shown in Figure 1.

3.4 Data Used for the DIS Calculation

The data products which will be assessed could be from the databases of DCI (Data Citation Index)^[22], WDS (World Data System), National Data Center or any data repositories. Datasets (data products) assessed here were published in GCdataPR from June 2014 to May 2018. The data citation statistics can be searchable through Web of Science^[13] and Google Scholar^[23] (in English), and CNKI (China Knowledge Resources Integrated Database)^[24] and CSCD (Chinese Science Citation Database) (in Chinese)^[25]. The Journal Impact Factors could be searchable through Letpub^[26], Scijournal^[27], CNKI, and CSCD.

3.5 Example of DIS for Data Product Performance

An example of DIS application is demonstrated in taking the “1 km grid population dataset of China” (DOI: 10.3974/geodb.2014.01.06.V1)^[28–29].

Table 1  Statistics of the data citation information about the “1 km grid population dataset of China”

(Web of Science, Letpub, Scijourna in English Journals, June 2014-May 2018)

The dataset was originally published by the GCdataPR in June 2014. The GCdataPR is one of the WDS (World Data System) regular members, China GEO (Group of Earth Observations), Data Publishing Center, GEOSS (Global Earth Observation System of Systems) data provider and broker. It can be searchable by DCI (Data Citation Index) of Clavate Analytics. Based on the information from the Web of Science, Letpub, Scijourna, CNKI (in Chinese) and CSCD (in Chinese), the information about the data citation and the journal impact factor are listed in the Table 1 (in English journals) and Table 2 (in Chinese journals).

The Statistics of the DIS for “1 km grid population dataset of China” performance in one year (2017) and multiple years (from June 2014 to May 2018) both in English and Chinese Journals are shown in Table 3. From which, the “1 km grid population dataset of China” (published in June 2014) had a DIS of 19.973 in English journals and 4.174 in China journals in 2017, the total DIS was 24.147 in 2017. In summary, the “1 km grid population dataset of China” had a DIS of 52.619 in English journals and 10.632,1 in Chinese journals from June 2014-May 2018, the total DIS was 63.132,1 in the 5 years. The highest DIS record in a year was 2017 in English (19.973), the third year after the data was published, and 2016 in Chinese (6.469), two years after the data was published.

Table 2  Statistics of the data citation information about the “1 km grid population dataset of China”

(CNKI and CSCD in Chinese Journals, June 2014-May 2018)

Table 3  Statistics of the DIS from June 2014-May 2017 for the “1 km grid population dataset of China”

4 DIS for Data Author(s) (DIS_A)

4.1 Data Authors (Co-authors) Share DIS for a Data Product

Based on the DIS for a data product, the DIS for individual authors (co-authors) could be calculated. The principle of DIS for individual author(s) (DIS_Ai) is all of the authors of one data product will share the data product’s DIS.

The method of the share is that the first author’s share is 50% of the DIS, the second co-author’s share is 50% of the rest, and third co-author’s share is 50% of the rest after the second co-author, etc. The last two co-authors will have the same share (Table 4)

Table 4  Authors’ share of DIS from a data product (DIS_Ai)

4.2 Data Author’s DIS_Ai for a Year

If more than one data products were published and cited by one data author in a year and then he/she has more than one data products DIS_D, the total DIS_A for the data author in a year will be the DIS_Aysum.

                                  (3)

where DIS_Aysum is the sum of data author’s DIS from more than on data products for a year; y is the year for calculating author’s DIS; DIS_Ayi is the author’s DIS from the data product i in the year of y; n is the number of data products where the author’s DIS were calculated.

4.3 Data Author’s DIS for Multiple Years

For calculating the data author’s DIS for more than one year, the total DIS for the data author in multiple years is given by sum of author’s DIS during the year period.

                          (4)

where DIS_Amysum is the sum of data author’s DIS_A from more than on data products for multiple years; y=1 means the year when the data published; n is the number of data products where the author’s DIS were calculated; my is the years for calculating author’s DIS;

4.4 Example of the Data Author’s DIS

Example: Dr. Fu, J. Y. is the first author of the “1 km grid population dataset of China” and the third co-author of the “1 km grid GDP dataset of China”^[51–52], both were published in 2014. Her DIS in 2017 is 16.597,75 (12.072 for the first published dataset and 4.525,75 for the second published dataset), Total DIS (from June 2014 to May 2018) is 34.787,75 (26.145 for the first published dataset and 8.648,75 for the second published dataset) (Table 5).

Table 5  Statistics of Fu, J.Y.’s DIS_A both for 2017 and for multiple years from June 2014 to May 2018

5 Discussion and Conclusion

The DIS method presented herein has been used for the data product performance assessment for GCdataPR. Based on the DIS result, the GCdataPR will publish its DIS ranking for both 2017 and for the period from June 2014 to May 2018. The highest DIS data author of China will be announced at the National Conference of Geographical Society of China in August 2018 in Xi’an city, Shaanxi province, China. Based on this work, further discussion on the DIS model could continue to be used for the data product performance assessment.

The data product could be cited in the conference proceedings, academic reports, books, or dissertations to strengthen the DIS model, but currently this information is not included in the DIS model. Further and more detailed discussions are therefore proposed. Regardless, data citation forms the backbone of all scientific endeavors and therefore should be emphasized in all scientific communities and scientific papers.

Acknowledgements

Dr. Shi, R. X. and Dr. Ma, J. H. from the Institute of Geographic Sciences and Natural Resources Research (IGSNRR), Chinese Academy of Sciences (CAS) supported in searching and checking data citation and journal IF information; Dr. Barbara, R., Editorial Board Member of Journal of Global Change Data & Discovery, provided the article editing and most valuable comments. All of these are greatly appreciated.

References

[1]       Science international: open data in a big data world [S]. Paris: International Council for Science (ICSU), International Social Science Council (ISSC), The World Academy of Sciences (TWAS), Inter Academy Partnership (IAP). 2015.

[2]       WDS Scientific Committee. Data Sharing Principles [S]. November 2015 V1. DOI: 10.5281/zenodo.34354.

[3]       GEO. Open Earth Observation Data [OL]. http://www.earthobservations.org/open_eo_data.php.

[4]       National Council of P. R. China. National Scientific Data Management Law [OL]. http://www.most.gov.cn/ mostinfo/xinxifenlei/fgzc/gfxwj/gfxwj2018/201804/t20180404_139023.htm. March 17, 2018.

[5]       Jiang, D., Song, X. F., Zhang, G. Y. A new milestone of China data sharing progress [R]. Journal of Global Change Data & Discovery, 2017, 1(2): 246-248. DOI: 10.3974/geodp.2017.02.22.

[6]       Wu, J. J., Liu, C., Li, B. Y. The global change research data publishing & sharing communication in 100 universities program launched in Beijing Normal University [R]. Journal of Global Change Data & Discovery, 2017, 1(3): 371-374. DOI: 10.3974/geodp.2017.03.21.

[7]       Xu, G. H. A new milestone of China scientific data sharing progress—speech at the First National Conference of Global Change Publishing and Sharing [J]. Journal of Global Change Data & Discovery, 2018, 2(1): 7-8. DOI: 10.3974/geodp.2018.02.02.

[8]       Liu, C. Acting locally, networking globally [R]. Journal of Global Change Data & Discovery, 2017, 1(3): 379-381. DOI: 10.3974/geodp.2017.03.24.

[9]       Garfield, E. The evolution of the Science Citation Index [J]. International Microbiology, 2007, 10: 65-69. DOI: 10.2436/20.1501.01.10.

[10]    Garfield, E. A new dimension in documentation through association of ideas [J]. International Journal of Epidemiology, 2006, 35(5): 1123–1127.

[11]    Elsevier. Ei compendex—engineering village flagship database [OL]. https://www.elsevier.com/solutions /engineering-village/content#databases.

[12]    White Paper: conference proceedings and their impact on global research. Enhancing proceedings coverage with cited references [OL]. http://wokinfo.com/media/pdf/proceedingswhtpaper.pdf. 2015-07-09.

[13]    Clarivate Analytics. Web of Science [OL]. https://clarivate.com/products/web-of-science/databases/.

[14]    ISO. Information and documentation—Guidelines for bibliographic references and citations to information resources [S]. International Standard. Reference Number ISO 690: 2010 (E).

[15]    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of China. Information and documentation—rules for bibliographic references and citations to information resources: GB/T 7714—2015[S]. Beijing: Standard Press of China, 2015: 5.

[16]    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of China. Scientific data citation: GB/T 35294—2017 [S]. Beijing: Standard Press of China, 2017: 12.

[17]    Liu, C., Ma, J. H., Uhlir, P. F., et al. Citation standardization of published scientific data [J]. Journal of Global Change Data & Discovery, 2018, 2(2): 123-134.

[18]    List of WDS Regular members [OL]. http://www.icsu-wds.org/community/membership/regular-members.

[19]    National Earth System Science Data Sharing Platform of China [OL]. http://www.geodata.cn/.

[20]    African Open Science Platform [OL]. http://africanopenscience.org.za/.

[21]    Impact Factor [OL]. https://en.wikipedia.org/wiki/Impact_factor.

[22]    Clarivate Analytics. Scientific and Academic Research tool, enabling researchers to accelerate discovery, evaluate impact, and benefit society worldwide, using trusted content and analytics [OL]. https://clarivate.com/product-category/scientific-academic-research/.

[23]    Google Scholar [OL]. https://scholar.google.com/.

[24]    China Knowledge Resources Integrated Database (CNKI) [OL]. http://www.cnki.net/.

[25]    Chinese Science Citation Database (CSCD) [OL]. http://sciencechina.cn/cscd_source.jsp.

[26]    Letpub [OL]. http://www.letpub.com.cn/index.php?page=journalapp&view=search.

[27]    Scijournal. Impact Factor List of Journal [OL]. https://www.scijournal.org/list-of-impact- factor-journal_1.shtml.

[28]    Fu, J.Y., Jiang, D., Huang, Y. H. 1 km grid population dataset of China. Global Change Research Data Publishing & Repository, 2014. DOI: 10.3974/geodb.2014.01.06.V1.

[29]    Fu, J. Y., Jiang, D., Huang, Y. H. 1 km grid population dataset of China [J]. Acta Geographica Sinica, 2014, 69(Sup): 41-44. DOI: 10.11821/dlxb2014S006.

[30]    Xie, Y., Wang, Y. X., Zhang, K., et al. Daily estimation of ground-level PM2.5 concentrations over Beijing using 3 km resolution MODIS AOD [J]. Environmental Science & Technology, 2015, 49(20): 12280-8.

[31]    Sun, J. T., Pan, L. L., Zhan, Y., et al. Contamination of phthalate esters, organochlorine pesticides and polybrominated diphenyl ethers in agricultural soils from the Yangtze River Delta of China [J]. Science of the Total Environment, 2016, 55: 670-676.

[32]    Tong, Y. D., Bu, X. G., Chen, C., et al. Impacts of sanitation improvement on reduction of nitrogen discharges entering the environment from human excreta in China [J]. Science of the Total Environment, 2016, 593: 439-448.

[33]    Yin, Y. Y., Tang, Q. H., Liu, X. C., et al. Water scarcity under various socio-economic pathways and its potential effects on food production in the Yellow River basin [J]. Hydrology and Earth System Sciences, 2017, 21: 791-804.

[34]    Zhou, Y. D., Zhao, Y., Mao, P., et al. Development of a high-resolution emission inventory and its evaluation and application through air quality modeling for Jiangsu Province, China [J]. Atmospheric Chemistry and Physics, 2017, 17: 211-233.

[35]    Zhou, Y., Xing, X. F., Lang, J. L., et al. A comprehensive biomass burning emission inventory with high spatial and temporal resolution in China [J]. Atmospheric Chemistry and Physics, 2017, 17: 2839-2864.

[36]    Li, S. C., Zhang, Y., Wang, Z. F. Mapping human influence intensity in the Tibetan Plateau for conservation of ecological service functions [J]. Ecosystem Services, 2018, 30: 276-286.

[37]    Zhang, N., Huang, H. Resilience analysis of countries under disasters based on multisource data [J]. Risk Analysis, 2018, 38: 31-42.

[38]    Hao, Y. F., Xie, S. D. Optimal redistribution of an urban air quality monitoring network using atmospheric dispersion model and genetic algorithm [J]. Atmospheric Environment, 2018, 177: 222-233.

[39]    Liu, L. L., Xu, H. M., Wang, Y., et al. Impacts of 1.5 and 2 ^oC global warming on water availability and extreme hydrological events in Yiluo and Beijiang River catchments in China [J]. Climate Change, 2017, 145(1-2): 145-158.

[40]    Fu, T. G., Han, L. P., Gao, H., et al. Pedodiversity and its controlling factors in mountain regions—a case study of Taihang Mountain, China [J]. Geoderma, 2018, 310: 230-237.

[41]    Li, S. F., Hughes, A. C. Su, T., et al. Fire dynamics under monsoonal climate in Yunnan, SW China: past, present and future [J]. Palaeogeography, Palaeoclimatology, Palaeoecology, 2017, 465: 168-176.

[42]    Li, S. C., Zhang, Y. L., Wang, Z. F., et al. Comparison of Socioeconomic Factors between Surrounding and Non-Surrounding Areas of the Qinghai-Tibet Railway before and after Its Construction [J]. Sustainability, 2016, 30: 276-286.

[43]    Wang, H., Wu, Q. Z., Zhou, R. J., et al. Preliminary numerical simulation of air pollution in Hefei with GIS data [J]. Acta Scientiae Circumstantiae, 2016, 36(11): 3902-3912.

[44]    Guo, S. S., Gong, J., Yin, J. F. Study on grid refinement for population distribution based on DMSP/OLS [J]. Journal of Seismological Research, 2016, 39(2): 321-326.

[45]    Tian, X. R., Dai, X., Wang, M. Y., et al. Forest fire risk assessment for China under different climate scenarios [J]. Chinese Journal of Applied Ecology, 2016, 27(3): 769-776.

[46]    Yan, Q., Kong, S. F., Liu, H. B., et al. Emission inventory of water soluble ions in fine particles from residential coal burning in China and implication for emission reduction [J]. China Environmental Science, 2017, 37(10): 3708-3721.

[47]    Nian, W., Chen, Y. M., Gao, J. X., et al. Relationship between provision and reception of ecological service of carbon fixation and oxygen release in Beijing-Tianjin-Hebei region [J]. Journal of Ecology and Rural Environment, 2017, 33(9): 783-791.

[48]    Jiang, D., Wang, Q., Ding, F. Y. Research on the geo-environment in the era of big data [J]. Science & Technology Review, 2018, 36(3): 41-48.

[49]    Zeng, J. J., Shen, C. Z., Zhou, S. L., et al. Application of the LUR model in the prediction of spatial distributions of soil heavy metals [J]. Environmental Science, 2018, 39: 371-378.

[50]    Hu, Z. Z., Zhao, Z. L., Zhang, G. X. Simulation and projection of the spatial pattern of the population in Beijing under the background of non-capital function extraction [J]. Journal of Geo-information Science, 2018. 20(2): 205-216.

[51]    Huang, Y. H., Jiang, D., Fu, J. Y. 1 km grid GDP dataset of China [DB/OL]. Global Change Research Data Publishing & Repository, 2014. DOI: 10.3974/geodb.2014.01.07.V1.

[52]    Huang, Y. H., Jiang, D., Fu, J. Y. 1 km grid GDP dataset of China [J]. Acta Geographica Sinica, 2014, 69(Sup): 45-48. DOI: 10.11821/dlxb2014S007.