Insert Menu

Once the input data has been correctly formatted into the appropriate CSV file format is can be loaded via the Insert menu which has the following options:

Files can be loaded into a STEMgis database either singularly or in batch mode.

An additional format, besides CSV, that can be loaded is an XYZ file. The ASCII (text) format is simply x, y, and z values separated by a comma, space, or tab. The file extension for these files should be .xyz.

Note: All spatial data must be loaded before attributes can be added to them. It is therefore safest not to mix the spatial and attribute files within the batch loader. Similarly, spatial and attribute data should be loaded before any key information for raster datasets.

Load File

To load a single file click on the Load File menu item, select the required file and click Open.

The file will be processed immediately:

Information about the loading file is shown including filename, projection and scale. A progress bar is also shown which passes through three stages for each file. These stages are Reading data, Processing data and Loading data. The more features in a file the longer it will take to load and as the database becomes larger, files will progressively take longer to load. This is an inevitable consequence of the amount of checking that must takes place during the loading process. For instance, the loader must check that you are not attempting to load duplicate data and that features already exist before you attempt to load attribute data onto them.

If a XYZ files has been chosen the user will be asked to complete the following information:

The XYZ file does not contain the information necessary to load the data, so the user must enter it here. The form allows the user to specify the Feature Type, the Date and a prefix to assist the naming of the UCODE and FNAME values. A projection may be selected either by typing in the projection number in the text box provided or by selected the projection from the hierarchical tree view below the text box. N.B. if you have latitude/longitude values then type -9999 or 0 in the Select Projection text box.

The remaining options on the form enable the user to provide information about the format of the file and how the list of XYZ values should be broken down into SURFACE features. The user may select a space, comma, tab or semi-colon as the separator between the x, y, and z values, choose a value to indicate no data values in the file (which will not be loaded) and indicate whether the z values should be negated.

The XYZ values are loaded as SURFACE features, which are basically multi-point features in 3D space. The user could load the whole file as one feature. However, the files may have many thousands of points and it may be inefficient to load them all as one feature. A reasonable size for a SURFACE feature is between 5000 and 20000 points. The user therefore has the option to select the number of points per feature or to arrange features on a grid basis, e.g. a new feature for every square kilometer in the example above. The latter is the default option.

A CSV file will be created and saved to disk during the loading process. This may save time later if the user ever needs to reload the data.

Batch Load

To load more than one file at a a time click on the Batch Load menu item:

Click on Browse to use the file browser to select a number of files. If you wish to remove some files from the list before loading them click on the Remove files from list button. When you are ready to load click on Load files into database.

Each file is processed in turn and progress is indicated in the same way as loading a single file (see above)

Load Metadata File(s)

To load preformatted metadata entries use the Metadata File(s) menu item. Metadata entries should be in XML format and conform to the National Geospatial Data Framework (NGDF) Discovery Metadata specifications (see www.ngdf.org.uk) and the data definition file ngdfdm2.dtd, which can be found in the data\maps directory, wherever STEMgis has been installed.

After selecting the Metadata File(s) menu item the Batch Load form is loaded. Click on Browse to use the file browser to select a number of files (the file type will be restricted to .xml files rather than .csv files). If you wish to remove some files from the list before loading then click on the Remove files from list button. When you are ready to load click on Load files into database.

Datasets

A STEMgis dataset is a collection of tables, definitions, spatial and attribute data. Usually, they make up one STEMgis database. However, it is possible to have more than one dataset within a physical database file. This may be useful for compartmentalizing data and thus making some queries much quicker for large feature types, where database indices for such data may take a long time to search.

It should be noted that multiple datasets within a database basically behave as separate databases in that they have unique feature type, attribute, dictionary definitions etc and also have unique spatial feature and attribute values. These definitions would need to be entered for the inserted dataset before the spatial and attribute CSV files for the inserted dataset were loaded.

To add a new dataset to a database select the Insert Dataset from the Insert menu and complete the following form.

Note: As a minimum you will need to provide a short name for the dataset.

To add the spatial and attribute data for the inserted dataset, first enter the definitions (feature type, attributes, metadata, dictionaries etc) using the appropriate options on the Tool Menu, and when this is complete load the spatial CSV files followed by the attribute csv files for the inserted dataset using the load options of the Insert Menu described above.