Journal of Global Change Data & Discovery2018.2(4):458-463

[PDF] [DATASET]

Citation:Zhang, L. F., Sun, X. J., Zhang, X., et al.New Version for the Multi-Dimensional Data Format (MDD) Analysis Tool (MARS v2.03)[J]. Journal of Global Change Data & Discovery,2018.2(4):458-463 .DOI: 10.3974/geodp.2018.04.17 .

A Spatio-temporal-spectral Multi-dimensional Data Format (MDD) Tool: from MARS v1.0 to v2.03

Zhang, L. F.1  Sun, X. J.1*  Zhang, X.1  Wang, N.1  Zhang, M. Y.1,2  
Lin, Y. K.1,2  Huang, H.1  Cen, Y.1  Huang, C. P.1  Yang, H.1  
Zhang, H. M.1  Liu, J.1  Tong, Q. X.1

1. Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing 100101, China;

2. University of Chinese Academy of Sciences, Beijing 100049, China

 

Abstract: The MARS-Multi-dimensional Analysis of Remote Sensing V1.0 was published in June 2017, based on which the updated version MARS v2.03 is now released. The system of this version can process the spatio-temporal-spectral multi-dimensional dataset, namely MDDMulti- Dimensional Data Format, which includes Temporal Sequential in Band (TSB), Temporal Sequential in Pixel (TSP), Temporal Interleaved by Band (TIB),Temporal Interleaved by Pixel (TIP), and Temporal Interleaved by Spectrum (TIS). It can deal with tasks of input, storage, analysis, output, format transform, data preprocessing, spectral analysis, classification, vegetation index calculation, etc. with data format of .mdd. Also the redundancy and mistakes have been corrected to make the system run efficiently. The renewed version keeps the original DOI.

Keywords: MDD; data format; multi-dimensional dataset; data update; MARS2.03

1 Multi-Dimensional Dataset (MDD)

1.1 MDD Data Format

MDD is a multi-dimensional data format that can integrate the temporal, spatial, and spectral features of remote sensing data[1]. A MDD consists of spectral data cube of series of different phases. From the prospect of dataset, MDD includes temporal information compared to remote sensing dataset with only spatial and spectral information and changes into a spatio-temporal-spectral 4-dimensional remote sensing dataset with spatial dimension of 2 and spectral dimension of 1, and at the same time it has the ability to extending to 5-dimensional dataset with spatial dimension of 3 and spectral dimension of 1[2].

1.2 Physical Storage Structure of the MDD

MDD dataset consists of a header file and a data file.

The header file records information regarding the image data itself as well as descriptive information. It includes the size of each dimension, interleave, and data type. The data file includes the coordinate projection, affine transform coefficients, file name, file type, and data offset values. Data files are actually stored image data files, and are divided into five data storage structures (TSB, TSP, TIB, TIP, and TIS) according to the storage order of the data.

1.2.1 MDD Header File

The MDD header file records all metadata information used to express and read the image data file, as shown in Table 1.

 

Table 1  Definition of the fields in the header file

Field

Definition

Samples

The number of samples (pixels) per image line for each band

Lines

The number of lines per image for each band

Bands

The number of bands per image file

Time

The number of phases per image file

Header offset

Offset values for each band. When a file is read, these bytes are skipped

File type

The MDD-defined file type, e.g., MDD Standard

Data type

The type of data representation: 1 = 8-bit byte, 2 =16-bit signed integer, 3 = 32-bit signed long integer, 4 = 32-bit floating point, 5 = 64-bit double-precision floating point, 6 = 2 × 32-bit complex, real-imaginary pair of double precision, 9 = 2 × 64-bit double-precision complex, real-imaginary pair of double precision, 12 =16-bit unsigned integer, 13 = 32-bit unsigned long integer, 14 = 64-bit signed long integer, 15 = 64-bit unsigned long integer

Interleave

Refers to whether the data are TSB, TSP, TIB, TIP, or TIS

Sensor type

The type or name of sensor, e.g., Landsat TM, SPOT, MODIS, etc.

Byte order

The order of the bytes in an integer, long integer, 64-bit integer, unsigned 64-bit integer, floating point, double-precision, and complex data types. One of the following is used:

Byte order = 0 (Host (Intel) in the Header Info dialog) is least significant byte first (LSF) data (DEC and MS-DOS systems)

Byte order = 1 (Network (IEEE) in the Header Info dialog) is most significant byte first (MSF) data (all other platforms)

Map info

Lists geographic coordinate information in the order of projection name (UTM), reference pixel x location (in file coordinates), pixel y, pixel easting, pixel northing, x pixel size, y pixel size, projection zone, and north or south (UTM only)

In MDD, pixel values always refer to the upper-left corner of the pixel. Map coordinates also typically refer to the upper-left corner of the pixel

Coordinate system string

A coordinate system string field to the header file. It lists the parameters used for a geographic coordinate system or projected coordinate system

Band names

Specific names for each band of an image

Time names

Specific names for each phase of an image

1.2.2 MDD Data File

The MDD data file stores the image data. The raster image data can be stored in one of the TSB, TSP, TIB, TIP, or TIS storage structure and as with the binary data stream, the five data structures can be converted to each other. The data format has been fully demonstrated in previous publication[1].

2 MARS[3] System Update

2.1 Descriptions and Existing Problems of MARS v1.0.

2.1.1 Descriptions of MARS v1.0

Version naming rules:

Major_Version_Number.Minor_Version_Number[Revision_Number[.Build_Number]]

Version management rules:

(1) The version number of the first version of the software is 1.0 or 1.00.

(2) When the software undergoes modifications or debug, the major version number and minor version number will not be changed. Revision number adds 1.

(3) When only minor applications were added to or removed from the original software, the major version number will not be changed, the minor version number adds 1 and the revision number returns to 0.

(4) When the software has been significantly modified or too many modifications resulting in overall changes, the major version number adds 1.

2.1.2 Existing Problems of MARS v1.0

After one and half years’ applications, we found some functions in MARS v1.0 need to be improved. The updates in new version (MARS v2.03) include more precise and complete settings of system, further improvements on design and usage, and some bugs to be corrected.

2.2 Updates of MARS v2.03

Table 2  Deleted parts of MARS v2.03 from MARS v1.0

Number

Deleted parts

1

Deleting Processing from menu

2

Deleting Spectral from menu

3

Deleting Save As from menu

4

Deleting Open Vector from menu

5

Deleting Temporal Filter from menu

6

Deleting Statistics from menu

7

Deleting PCA from menu

2.2.1 Deleted Parts

The following parts (Table 2) have been deleted in the updated version to make usage easier.

2.2.2 Added Parts

There are calculation functions in the updated version including vegetation index calculation (Table 3).

2.2.3 Modified Parts

Some functions have been modified in the updated version (Table 4).

 

Table 3  Added parts of MARS v2.03 based on MARS v1.0

Number

Added parts

1

Vegetation Index calculation operation, including calculating UNVI and NDVI

2

Temporal classification and Temporal_Spectral Classification in menu
(To be developed)

 

2.2.4 Debug

During the usage of MARS v1.0, we found some bugs in the system. And in the updated version we corrected these bugs, as shown in Table 5.

 

Table 4  Modified parts of MARS v2.03 based on MARS v1.0

Number

Modified parts

1

Moving MDD Open from the menu of MDD Analysis to the menu of File and renaming it Open MDD

2

Moving MDD Export from the menu of MDD Analysis to the menu of File

3

Substituting MDD Builder in the menu of MDD Analysis with Processing and putting MDD Append, MDD Merge and MDD Split under it

4

Moving MDD Math from MDD Analysis to Basic Tools and combining it with Band math in Data Math

5

Moving all sub-items from Classification to MDD Analysis

 

Table 5  Debug of MARS v2.03 based on MARS v1.0

Number

Debug parts

1

During MDD construction, when dealing with large amount of data the speed is low and the software breaks down

2

The version of the third party QT used by Image Format Transformation is too old, which makes the software breaks down when applying the operation

3

The code of MDD Append is not fully logical

4

The speed of MDD Merge is low

5

In the operation of Masking, the mask file cannot be loaded

2.3 Main Functions of MARS v2.03

The main functions of the updated MARS v2.03 include 7 categories and 27 applications, which consists of: File, including functions of inputs, outputs, save, exit and etc.; MDD Builders transforming data format from Landsat[48] to MDD, or from HDF [910] or IMG to MDD data format; MDD Construction operations, including appending, merging and splitting; Basic Tools, including building 3D Cube, rotating image, resizing image, spatial filtering, spectral filtering, data math and masking; Transform, including MDD conversion and image format transformation; MDD Analysis, including classification and vegetation index calculation; Windows and Help, etc.

 

Table 6  Main functions of MARS v2.03

Number

Menu

Function

Description

1

File

Open MDD

Opening MDD files

2

Open Image

Opening image files

3

Edit Image Header

Editing header file of Images

4

MDD Export

Exporting MDD files

5

Save as ASCII

Saving files as ASCII data

6

Exit

Exiting

7

MDD Builder

From Landsat

Constructing MDD data from Landsat data

8

From MODIS hdf

Constructing MDD data from MODIS hdf data

9

From MODIS img

Constructing MDD data from data with format of .img

10

From ENVI Image

Constructing MDD data from ENVI Image

11

MDD Append

Appending MDD files

12

MDD Merge

Merging MDD files

13

MDD Split

Splitting MDD files

14

Basic Tools

Build 3D Cube

Building 3D image cube

15

Rotate Image

Rotating image

16

Resize Image

Resizing image

17

Spatial Filter

Filtering in the spatial dimension

18

Spectral Filter

Filtering in the spectral dimension

19

Data Math

Calculating

20

Masking

Masking the image file

21

Transform

MDD Conversion

Data format conversion of MDD

22

Image Format Transformation

Transforming image format

23

MDD Analysis

Classification

Classification operation

24

Vegetation Index

Calculating vegetation index

25

Windows

Available Data List

Showing available data list

26

Help

User Manual

Showing user manual

27

About MARS v2.03

Showing information about MARS v2.03

2.4 Comparison of Main Interface and Menu of MARS v1.0 and v2.03

For the main interface, compared with MARS v1.0 (Figure 1), the main interface of the updated MARS v2.03 (Figure 2) is more concise.

  MARS v2.03 merges, deletes and adds some menu function items based on MARS v1.0. For example, “MDD Conversion” and “Image Format Transformation” are added into the menu item of “Transform dish”, redundant menu items for processing and analyzing the MDD dataset such as “Preprocessing” are deleted, it Added menu items such as “Vegetation Index” calculation for MDD dataset. The new version will make the generation, operation and analysis of the MDD dataset more purposeful, make the software operation more reasonable, and repair the system vulnerabilities existing in the original version, and thus the software will run more smoothly.

Figure 1  Main interface and its menus of MARS v1.0

Figure 2  Main interface and its menus of MARS v2.03

3 Intellectual Property Declaration

MARS v2.03 is developed based on MARS v1.0. Despite of the difference in these two versions, the principles of academic achievements are the same and they are correlated and successive. Therefore, MARS v2.03 succeeds the DOI from MARS v1.0 as DOI: 10.3974/geodb.2017.02.20.V1. The research and development team (the authors, corresponding author, co-authors with author sequences and institute), and funding project are not changed compared to that of MARS v1.0. Thus the intellectual property of MARS v2.03 and MARS v1.0 are not changed.

References

[1]     Zhang, L. F., Chen, H., Sun X. J., et al. Designing spatial-temporal-spectral integrated storage structure of multi-dimensional remote sensing images [J]. Journal of Remote Sensing, 2017, 21(1): 62-73.

[2]     Zhang, L. F., Sun, X. J., Zhang, X., et al. New version for the multi-dimensional data format (MDD) analysis tool (MARS v2.03) [DB/OL]. Global Change Research Data Publishing & Repository, 2017. DOI: 10.3974/geodb.2017.02.20.V1.

[3]     Zhang, L. F., Sun, X. J., Zhang, X., et al. A multi-dimensional data format (MDD) and analysis tool [J]. Journal of Global Change Data & Discovery, 2017, 1(2): 121-135. DOI: 10.3974/geodp.2017.02.01.

[4]     Kennedy, R. E., Yang, Z., Cohen, W. B. Detecting trends in forest disturbance and recovery using yearly Landsat time series: 1. LandTrendr—temporal segmentation algorithms[J]. Remote Sensing of Environment, 2010, 114(12): 2897-2910.

[5]    Huang, C., Goward, S. N., Masek, J. G., et al. An automated approach for reconstructing recent forest disturbance history using dense Landsat time series stacks [J]. Remote Sensing of Environment, 2010, 114(1): 183-198.

[6]     Powell, S. L., Cohen, W. B., Healey, S. P., et al. Quantification of live aboveground forest biomass dynamics with Landsat time-series and field inventory data: a comparison of empirical modeling approaches [J]. Remote Sensing of Environment, 2010, 114(5): 1053-1068.

[7]     Margono, B. A., Potapov, P., Hansen, M. C. Mapping and monitoring deforestation and forest degradation in Sumatra (Indonesia) using Landsat time series data sets from 1990 to 2010 [J]. Environmental Research Letters, 2012, 7(3): 034010.

[8]     Townsend, P. A., Helmers, D. P., Kingdon, C. C., et al. Changes in the extent of surface mining and reclamation in the Central Appalachians detected using a 1976-2006 Landsat time series [J]. Remote Sensing of Environment, 2009, 113(1): 62-72.

[9]     Galford, G. L., Mustard, J. F., Melillo, J., et al. Wavelet analysis of MODIS time series to detect expansion and intensification of row-crop agriculture in Brazil [J]. Remote Sensing of Environment, 2008, 112(2): 576-587.

[10]  Sakamoto, T., Yokozawa, M., Toritani, H., et al. A crop phenology detection method using time-series MODIS data [J]. Remote Sensing of Environment, 2005, 96(3): 366-374. 

Co-Sponsors

Institute of Geographic Sciences and Natural Resources Research,Chinese Academy of Sciences

The Geographical Society of China

Parteners

Committee on Data for Science and Technology (CODATA) Task Group on Preservation of and Access to Scientific and Technical Data in/for/with Developing Countries (PASTD)

Jomo Kenyatta University of Agriculture and Technology

Digital Linchao GeoMuseum