Configuration options

weectl import requires a second configuration file, the import configuration file, in addition to the standard WeeWX configuration file. The import configuration file specifies the import type and various options associated with the import type. The import configuration file is specified using the mandatory --import-config option. How you construct the import configuration file is up to you; however, the recommended method is to copy one of the example import configuration files located in the util/import directory, modify the configuration options in the newly copied file to suit the import to be performed and then use this file as the import configuration file.

Following is the definitive guide to the options available in the import configuration file. Where a default value is shown this is the value that will be used if the option is omitted from the import configuration file.

`source`¶

The source option determines the type of import to be performed by weectl import. The option is mandatory and must be set to one of the following:

CSV to import from a single CSV format file.
WU to import from a Weather Underground PWS history
Cumulus to import from one or more Cumulus monthly log files.
WD to import from one or more Weather Display monthly log files.
WeatherCat to import from one or more WeatherCat monthly .cat files.

Mandatory, there is no default.

[CSV]¶

The [CSV] section contains the options controlling the import of observational data from a CSV format file.

`file`¶

The file containing the CSV format data to be used as the source during the import. Include full path and filename.

Mandatory, there is no default.

`source_encoding`¶

The source file encoding. This parameter should only need be used if the source file uses an encoding other than UTF-8 or an ASCII compatible encoding. If used, the setting used should be a Python Standard Encoding.

Optional, the default is utf-8-sig.

`delimiter`¶

The character used to separate fields. This parameter must be included in quotation marks.

Optional, the default is ',' (comma).

`decimal`¶

The character used as the decimal point in the source files. A full stop is frequently used, but it may be another character. This parameter must be included in quotation marks.

Optional, the default is '.'.

`interval`¶

Determines how the time interval (WeeWX archive table field interval) between successive observations is derived. The interval can be derived by one of three methods:

The interval can be calculated as the time, rounded to the nearest minute, between the date-time of successive records. This method is suitable when the data was recorded at fixed intervals and there are NO missing records in the source data. Use of this method when there are missing records in the source data can compromise the integrity of the WeeWX statistical data. Select this method by setting interval = derive.
The interval can be set to the same value as the archive_interval setting under [StdArchive] in weewx.conf. This setting is useful if the data was recorded at fixed intervals, but there are some missing records and the fixed interval is the same as the archive_interval setting under [StdArchive] in weewx.conf. Select this method by setting interval = conf.
The interval can be set to a fixed number of minutes. This setting is useful if the source data was recorded at fixed intervals, but there are some missing records and the fixed interval is different to the archive_interval setting under [StdArchive] in weewx.conf. Select this method by setting interval = x where x is an integer number of minutes.

If the CSV source data records are equally spaced in time, but some records are missing, then a better result may be achieved using conf or a fixed interval setting.

Optional, the default is derive.

`qc`¶

Determines whether simple quality control checks are applied to imported data. Setting qc = True will result in weectl import applying the WeeWX StdQC minimum and maximum checks to any imported observations. Setting qc = False will result in weectl import not applying quality control checks to imported data. weectl import quality control checks use the same configuration settings, and operate in the same manner, as the StdQC service. For example, for minimum/maximum quality checks, if an observation falls outside of the quality control range for that observation, the observation will be set to None. In such cases you will be alerted through a log entry similar to:

2023-11-04 16:59:01 weectl-import[3795]: WARNING weewx.qc: 2023-10-05 18:30:00 
AEST (1696494600) Archive value 'outTemp' 194.34 outside limits (0.0, 120.0)

Note

As derived observations are calculated after the quality control check is applied, derived observations are not subject to quality control checks.

Optional, the default is True.

`calc_missing`¶

Determines whether any missing derived observations will be calculated from the imported data. Setting calc_missing = True will result in weectl import using the WeeWX StdWXCalculate service to calculate any missing derived observations from the imported data. Setting calc_missing = False will result in WeeWX leaving any missing derived observations as None. See [StdWXCalculate] for details of the observations the StdWXCalculate service can calculate.

Optional, the default is True.

`ignore_invalid_data`¶

Determines whether invalid data in a source field is ignored or the import aborted. If invalid data is found in a source field and ignore_invalid_data is True the corresponding WeeWX destination field is set to None and the import continues. The import is aborted if ignore_invalid_data is False and invalid data is found in a source field.

Optional, the default is True.

`tranche`¶

To speed up database operations imported records are committed to database in groups of records rather than individually. The size of the group is set by the tranche parameter. Increasing the tranche parameter may result in a slight speed increase, but at the expense of increased memory usage. Decreasing the tranche parameter will result in less memory usage, but at the expense of more frequent database access and likely increased time to import.

Optional, the default is 250 which should suit most users.

`UV_sensor`¶

WeeWX records a None/null for UV when no UV sensor is installed, whereas some weather station software records a value of 0 for UV index when there is no UV sensor installed. The UV_sensor parameter enables weectl import to distinguish between the case where a UV sensor is present and the UV index is 0 and the case where no UV sensor is present and UV index is 0. UV_sensor = False should be used when no UV sensor was used in producing the source data. UV_sensor = False will result in None/null being recorded in the WeeWX archive field UV irrespective of any UV observations in the source data. UV_sensor = True should be used when a UV sensor was used in producing the source data. UV_sensor = True will result in UV observations in the source data being stored in the WeeWX archive field UV.

Optional, the default is True.

`solar_sensor`¶

WeeWX records a None/null when no solar radiation sensor is installed, whereas some weather station software records a value of 0 for solar radiation when there is no solar radiation sensor installed. The solar_sensor parameter enables weectl import to distinguish between the case where a solar radiation sensor is present and solar radiation is 0 and the case where no solar radiation sensor is present and solar radiation is 0. solar_sensor = False should be used when no solar radiation sensor was used in producing the source data. solar_sensor = False will result in None/null being recorded in the WeeWX archive field radiation irrespective of any solar radiation observations in the source data. solar_sensor = True should be used when a solar radiation sensor was used in producing the source data. solar_sensor = True will result in solar radiation observations in the source data being stored in the WeeWX archive field radiation.

Optional, the default is True.

`raw_datetime_format`¶

WeeWX stores each record with a unique unix epoch timestamp, whereas many weather station applications or web sources export observational data with a human-readable date-time. This human-readable date-time is interpreted according to the format set by the raw_datetime_format option. This option consists of Python strptime() format codes and literal characters to represent the date-time data being imported.

For example, if the source data uses the format '23 January 2015 15:34' the appropriate setting for raw_datetime_format would be %d %B %Y %H:%M, '9:25:00 12/28/16' would use %H:%M:%S %m/%d/%y. If the source data provides a unix epoch timestamp as the date-time field the unix epoch timestamp is used directly and the raw_datetime_format option is ignored.

Optional, the default is %Y-%m-%d %H:%M:%S.

Note

weectl import does not support the construction of the unique record date-time stamp from separate date and time fields, rather the date-time information for each imported record must be contained in a single field. CSV data containing separate date and time fields may require further manual processing before it can be imported.

`wind_direction`¶

WeeWX records wind direction in degrees as a number from 0 to 360 inclusive (no wind direction is recorded as None/null), whereas some data sources may provide wind direction as a number over a different range (e.g., -180 to +180), or may use a particular value when there is no wind direction (e.g., 0 may represent no wind direction and 360 may represent a northerly wind, or -9999 (or some similar clearly invalid number of degrees) to represent there being no wind direction). weectl import handles such variations in data by defining a range over which imported wind direction values are accepted. Any value outside of this range is treated as there being no wind direction and is recorded as None/null. Any value inside the range is normalised to the range 0 to 360 inclusive (e.g., -180 would be normalised to 180). The wind_direction option consists of two comma separated numbers of the format lower, upper where lower and upper are inclusive. The operation of the wind_direction option is best illustrated through the following table:

Option wind_direction
wind_direction option setting	Source data wind direction value	Imported wind direction value
0, 360	0	0
	160	160
	360	360
	500	None/null
	-45	None/null
	-9999	None/null
	No data	None/null
-360, 360	0	0
	160	160
	360	360
	500	None/null
	-45	315
	-9999	None/null
	No data	None/null
-180, 180	0	0
	160	160
	360	None/null
	500	None/null
	-45	315
	-9999	None/null
	No data	None/null

The default is 0, 360.

`[[FieldMap]]`¶

The [[FieldMap]] stanza defines the mapping from the CSV source data fields to WeeWX archive fields. This allows weectl import to take a source data field, perform the appropriate unit conversion and store the resulting value in the appropriate WeeWX archive field. The map consists of one sub-stanza per WeeWX archive field being populated using the following format:

    [[[weewx_archive_field_name]]]
        source_field = csv_field_name
        unit = weewx_unit_name
        is_cumulative = True | False
        is_text = True | False

Where

weewx_archive_field_name is a field in the in-use WeeWX archive table schema
csv_field_name is the name of a field in the CSV source data
weewx_unit_name is a WeeWX unit name; e.g., degree_C

Each WeeWX archive field stanza supports the following options:

source_field. The name of the CSV field to be mapped to the WeeWX archive field. Mandatory.
unit. The WeeWX unit name of the units used by source_field. Mandatory for non-text source fields. Ignored for source text fields.
is_cumulative. Whether the WeeWX archive field is to be derived from a cumulative source field (e.g., daily rainfall) or not. Optional boolean value. Default is False.
is_text. Whether the source field is to be imported as text or not. Optional boolean. Default is False.

A mapping is not required for every WeeWX archive field and neither does every CSV field need to be included in a mapping.

Note

Importing of text data into text fields in the WeeWX archive is only supported for WeeWX archive fields that have been configured as text fields. Refer to the Wiki page Storing text in the database for details.

If the source data includes a field that contains a WeeWX unit system code (i.e. the equivalent of the WeeWX usUnits field such as may be obtained from WeeWX or wview data) then this field may be mapped to the WeeWX usUnits field and used to set the units used for all fields being imported. In such cases, except for the [[[dateTime]]] field map entry, the weewx_unit_name portion of the imported fields in the field map is not used and may be omitted.

For example, source CSV data with the following structure:

date_and_time,temp,humid,wind,dir,dayrain,rad,river,decsription
23 May 2018 13:00,17.4,56,3.0,45,10.0,956,340,'cloudy'
23 May 2018 13:05,17.6,56,1.0,22.5,10.4,746,341,

where temp is temperature in Celsius, humid is humidity in percent, wind is wind speed in km/h, dir is wind direction in degrees, rainfall is rain in mm, rad is radiation in watts per square meter, river is river height in mm and description is a text field might use a field map as follows:

[[FieldMap]]
    [[[dateTime]]]
        source_field = date_and_time
        unit = unix_epoch
    [[[outTemp]]]
        source_field = temp
        unit = degree_C
    [[[outHumidity]]]
        source_field = humid
        unit = percent
    [[[windSpeed]]]
        source = wind
        unit = km_per_hour
    [[[windDir]]]
        source_field = dir
        unit = degree_compass
    [[[rain]]]
        source_field = dayrain
        unit = mm
        is_cumulative = True
    [[[radiation]]]
        source_field = rad
        unit = watt_per_meter_squared
    [[[outlook]]]
        source_field = description
        is_text = True

If the same source CSV data included a field unit_info that contains WeeWX unit system data as follows:

date_and_time,temp,humid,wind,dir,dayrain,rad,river,decsription,unit_info
23 May 2018 13:00,17.4,56,3.0,45,0.0,956,340,'cloudy',1
23 May 2018 13:05,17.6,56,1.0,22.5,0.4,746,341,'showers developing',16

then a field map such as the following might be used:

[[FieldMap]]
    [[[dateTime]]]
        source_field = date_and_time
        unit = unix_epoch
    [[[usUnits]]]
        source_field = unit_info
    [[[outTemp]]]
        source_field = temp
    [[[outHumidity]]]
        source_field = humid
    [[[windSpeed]]]
        source = wind
    [[[windDir]]]
        source_field = dir
    [[[rain]]]
        source_field = dayrain
        is_cumulative = True
    [[[radiation]]]
        source_field = rad
    [[[outlook]]]
        source_field = description
        is_text = True

Note

Any WeeWX archive fields that are derived (e.g., dewpoint), and for which there is no field mapping, may be calculated during import by use of the calc_missing option in the [CSV] section of the import configuration file.

Note

The dateTime field map entry is a special case. Whereas other field map entries may use any supported WeeWX unit name, or no unit name if the usUnits field is populated, the dateTime field map entry must include the WeeWX unit name unix_epoch. This is because weectl import uses the raw_datetime_format config option to convert the supplied date-time field data to a Unix epoch timestamp before the field map is applied.

[WU]¶

The [WU] section contains the options relating to the import of observational data from a Weather Underground PWS history.

`station_id`¶

The Weather Underground weather station ID of the PWS from which the historical data will be imported.

Mandatory, there is no default.

`api_key`¶

The Weather Underground API key to be used to obtain the PWS history data.

Mandatory, there is no default.

Note

The API key is a seemingly random string of 32 characters used to access the new (2019) Weather Underground API. PWS contributors can obtain an API key by logging onto the Weather Underground internet site and accessing 'Member Settings'. 16 character API keys used with the previous Weather Underground API are not supported.

`interval`¶

Determines how the time interval (WeeWX database field interval) between successive observations is determined. This option is identical in operation to the CSV interval option, but applies to Weather Underground imports only. As a Weather Underground PWS history sometimes has missing records, the use of interval = derive may give incorrect or inconsistent interval values. Better results may be obtained by using interval = conf if the current WeeWX installation has the same archive_interval as the Weather Underground data, or by using interval = x where x is the time interval in minutes used to upload the Weather Underground data. The most appropriate setting will depend on the completeness and (time) accuracy of the Weather Underground data being imported.

Optional, the default is derive.

`qc`¶

Determines whether simple quality control checks are applied to imported data. This option is identical in operation to the CSV qc option but applies to Weather Underground imports only. As Weather Underground imports at times contain nonsense values, particularly for fields for which no data was uploaded to Weather Underground by the PWS, the use of quality control checks on imported data can prevent these nonsense values from being imported and contaminating the WeeWX database.

Optional, the default is True.

`calc_missing`¶

Determines whether any missing derived observations will be calculated from the imported data. This option is identical in operation to the CSV calc_missing option but applies to Weather Underground imports only.

Optional, the default is True.

`ignore_invalid_data`¶

Determines whether invalid data in a source field is ignored or the import aborted. This option is identical in operation to the CSV ignore_invalid_data option but applies to Weather Underground imports only.

Optional, the default is True.

`tranche`¶

The number of records written to the WeeWX database in each transaction. This option is identical in operation to the CSV tranche option but applies to Weather Underground imports only.

Optional, the default is 250 which should suit most users.

`wind_direction`¶

Determines the range of acceptable wind direction values in degrees. This option is identical in operation to the CSV wind_direction option but applies to Weather Underground imports only.

Optional, the default is 0, 360.

`[[FieldMap]]`¶

The [[FieldMap]] stanza defines the mapping from the Weather Underground source data fields to WeeWX archive fields. This allows weectl import to take a source data field, perform the appropriate unit conversion and store the resulting value in the appropriate WeeWX archive field. Weather Underground imports use a simplified map that consists of one sub-stanza per WeeWX archive field being populated using the following format:

    [[[weewx_archive_field_name]]]
        source_field = wu_field_name
        is_cumulative = True | False

Where

weewx_archive_field_name is a field in the in-use WeeWX archive table schema
wu_field_name is the name of a Weather Underground source field as detailed in the Available Weather Underground import field names table below.

Each WeeWX archive field stanza supports the following option:

source_field. The name of the Weather Underground source field to be
mapped to the WeeWX archive field. Mandatory.
is_cumulative. Whether the WeeWX archive field is to be derived from a cumulative source field (e.g., daily rainfall) or not. Optional boolean value. Default is False.

A mapping is not required for every WeeWX archive field and neither does every Weather Underground field need to be included in a mapping.

Current o

Available Weather Underground import field names
Field name	Description
epoch	Date and time
dewptAvg	Current dewpoint
heatindexAvg	Current heat index
humidityAvg	Current outside Humidity
precipRate	Current rain rate
precipTotal	Rainfall since midnight
pressureAvg	Current barometric pressure
solarRadiationHigh	Current solar radiation
tempAvg	Outside temperature
uvHigh	Current UV index
windchillAvg	Current windchill
winddirAvg	Current wind direction
windgustHigh	Current wind gust
windspeedAvg	Current average wind speed

Note

The above field names are internally generated by weectl import and do not represent any field names used within Weather Underground. They have only been provided for use in the field map.

For example, the following field map might be used to import outside temperature to WeeWX field outTemp, outside humidity to WeeWX field outHumidity and rainfall to WeeWX field rain:

[[FieldMap]]
    [[[dateTime]]]
        source_field = epoch
    [[[outTemp]]]
        source_field = tempAvg
    [[[outHumidity]]]
        source_field = humidityAvg
    [[[rain]]]
        source = precipTotal
        is_cumulative = True

Note

The inclusion of is-cumulative = True under [[[rain]]] as Weather Underground records rainfall as a daily cumulative value.

Note

Any WeeWX archive fields that are derived (e.g., dewpoint) and for which there is no field mapping may be calculated during import by use of the calc_missing option in the [WU] section of the import configuration file.

The example Weather Underground import configuration file located in the util/import directory contains an example field map.

[Cumulus]¶

The [Cumulus] section contains the options relating to the import of observational data from Cumulus monthly log files.

`directory`¶

The full path to the directory containing the Cumulus monthly log files to be imported. Do not include a trailing /.

Mandatory, there is no default.

`source_encoding`¶

The Cumulus monthly log file encoding. This option is identical in operation to the CSV source_encoding option but applies to Cumulus imports only.

Optional, the default is utf-8-sig.

`interval`¶

Determines how the time interval (WeeWX database field interval) between successive observations is determined. This option is identical in operation to the CSV interval option but applies to Cumulus monthly log file imports only. As Cumulus monthly log files can, at times, have missing entries, the use of interval = derive may give incorrect or inconsistent interval values. Better results may be obtained by using interval = conf if the archive_interval for the current WeeWX installation is the same as the Cumulus 'data log interval' setting used to generate the Cumulus monthly log files, or by using interval = x where x is the time interval in minutes used as the Cumulus 'data log interval' setting. The most appropriate setting will depend on the completeness and (time) accuracy of the Cumulus data being imported.

Optional, the default is derive.

`qc`¶

Determines whether simple quality control checks are applied to imported data. This option is identical in operation to the CSV qc option but applies to Cumulus imports only.

Optional, the default is True.

`calc_missing`¶

Determines whether any missing derived observations will be calculated from the imported data. This option is identical in operation to the CSV calc_missing option but applies to Cumulus imports only.

Optional, the default is True.

`separator`¶

The character used as the date field separator in the Cumulus monthly log file. A solidus (/) is frequently used, but it may be another character depending on the settings on the machine that produced the Cumulus monthly log files. This parameter must be included in quotation marks.

Optional, the default is '/'.

`delimiter`¶

The character used as the field delimiter in the Cumulus monthly log file. A comma is frequently used, but it may be another character depending on the settings on the machine that produced the Cumulus monthly log files. This parameter must be included in quotation marks.

Optional, the default is ','.

`decimal`¶

The character used as the decimal point in the Cumulus monthly log files. A period is frequently used, but it may be another character depending on the settings on the machine that produced the Cumulus monthly log files. This parameter must be included in quotation marks.

Optional, the default is '.'.

`ignore_invalid_data`¶

Determines whether invalid data in a source field is ignored or the import aborted. This option is identical in operation to the CSV ignore_invalid_data option but applies to Cumulus monthly log file imports only.

Optional, the default is True.

`tranche`¶

The number of records written to the WeeWX database in each transaction. This option is identical in operation to the CSV tranche option but applies to Cumulus monthly log file imports only.

Optional, the default is 250 which should suit most users.

`UV_sensor`¶

Enables weectl import to distinguish between the case where a UV sensor is present and the UV index is 0 and where no UV sensor is present and the UV index is 0. This option is identical in operation to the CSV UV_sensor option but applies to Cumulus monthly log file imports only.

Optional, the default is True.

`solar_sensor`¶

Enables weectl import to distinguish between the case where a solar radiation sensor is present and solar radiation is 0 and where no solar radiation sensor is present and solar radiation is 0. This option is identical in operation to the CSV solar_sensor option but applies to Cumulus monthly log file imports only.

Optional, the default is True.

`[[FieldMap]]`¶

The [[FieldMap]] stanza defines the mapping from the Cumulus source data fields to WeeWX archive fields. This allows weectl import to take a source data field, perform the appropriate unit conversion and store the resulting value in the appropriate WeeWX archive field. The map consists of one sub-stanza per WeeWX archive field being populated using the following format:

    [[[weewx_archive_field_name]]]
        source_field = cumulus_field_name
        unit = weewx_unit_name
        is_cumulative = True | False
        is_text = True | False

Where

weewx_archive_field_name is a field in the in-use WeeWX archive table schema
cumulus_field_name is the name of a Cumulus source field as detailed in the Available Cumulus import field names table below.
weewx_unit_name is a WeeWX unit name; e.g., degree_C
is_text. Whether the source field is to be imported as text or not. Optional boolean. Default is False.

Each WeeWX archive field stanza supports the following options:

source_field. The name of the Cumulus field to be mapped to the WeeWX archive field. Mandatory.
unit. The WeeWX unit name of the units used by source_field. Mandatory.
is_cumulative. Whether the WeeWX archive field is to be derived from a cumulative source field (e.g., daily rainfall) or not. Optional boolean value. Default is False.

A mapping is not required for every WeeWX archive field and neither does every Cumulus field need to be included in a mapping.

Note

The unit option setting for each field map entry will depend on the Cumulus settings used to generate the Cumulus monthly log files. Depending on the Cumulus field type, the supported WeeWX units names for that field may only be a subset of the corresponding WeeWX unit names; e.g., WeeWX supports temperatures in Celsius, Fahrenheit and Kelvin, but Cumulus logs may only include temperatures in Celsius or Fahrenheit. Refer to Units for details of available WeeWX unit names.

Available Cumulus import field names
Field name	Description
datetime	Date and time
annual_et	Annual evapotranspiration
avg_wind_speed	Average wind speed
cur_app_temp	Current apparent temperature
avg_wind_bearing	Current wind direction
cur_dewpoint	Current dew point
cur_et	Evapotranspiration
cur_heatindex	Current heat index
cur_in_hum	Current inside humidity
curr_in_temp	Current inside temperature
cur_out_hum	Current outside humidity
cur_out_temp	Current outside temperature
cur_rain_rate	Current rain rate
cur_slp	Current barometric pressure
cur_solar	Current solar radiation
cur_tmax_solar	Current theoretical maximum solar radiation
cur_uv	Current UV index
cur_wind_bearing	Current wind direction
cur_windchill	Current windchill
day_rain	Total rainfall since the daily rollover
day_rain_rg11	Today's RG-11 rainfall
day_sunshine_hours	Today's sunshine hours
gust_wind_speed	Wind gust speed
latest_wind_gust	Latest measured wind speed
midnight_rain	Total rainfall since midnight
rain_counter	Total rainfall counter

Note

The above field names are internally generated by weectl import and do not represent any field names used within Cumulus. They have only been provided for use in the field map.

For example, the following field map might be used to import outside temperature to WeeWX field outTemp, outside humidity to WeeWX field outHumidity and extra temperature 1 to WeeWX field poolTemp:

[[FieldMap]]
    [[[dateTime]]]
        source_field = datetime
        unit = unix_epoch
    [[[outTemp]]]
        source_field = temp
        unit = degree_C
    [[[outHumidity]]]
        source_field = humid
        unit = percent
    [[[poolTemp]]]
        source = temp1
        unit = degree_C

Note

Any WeeWX archive fields that are derived (e.g., dewpoint) and for which there is no field mapping may be calculated during import by use of the calc_missing option in the [Cumulus] section of the import configuration file.

Note

The dateTime field map entry is a special case. Whereas other field map entries may use any WeeWX unit name for a unit supported by the import source, the dateTime field map entry must use the WeeWX unit name unix_epoch.

The example Cumulus import configuration file located in the util/import directory contains an example field map.

[WD]¶

The [WD] section contains the options relating to the import of observational data from Weather Display monthly log files.

`directory`¶

The full path to the directory containing the Weather Display monthly log files to be imported. Do not include a trailing /.

Mandatory, there is no default.

`logs_to_process`¶

The Weather Display monthly log files to be processed. Weather Display uses multiple files to record each month of data. Which monthly log files are produced depends on the Weather Display configuration and the capabilities of the weather station. weectl import supports the following Weather Display monthly log files:

MMYYYYlg.txt
MMYYYYlgcsv.csv (csv format version of MMYYYYlg.txt)
MMYYYYvantagelog.txt
MMYYYYvantagelogcsv.csv (csv format version of MMYYYYvantagelog.txt)
MMYYYYvantageextrasensorslog.csv

where MM is a one or two-digit month and YYYY is a four digit year

The format for the logs_to_process setting is:

logs_to_process = [lg.txt, | logcsv.csv, | vantagelog.txt, | vantagelogcsv.
csv, | vantageextrasensorslog.csv]

Note

The leading MMYYYY is omitted when listing the monthly log files to be processed using the logs_to_process setting. Inclusion of the leading MMYYYY will cause the import to fail.

Note

The MMYYYYlgcsv.csv and MMYYYYvantagelogcsv.csv log files are CSV versions of MMYYYYlg.txt and MMYYYYvantagelog.txt respectively. Either the .txt or .csv version of these files should be used but not both.

The monthly log files selected for processing should be chosen carefully as the selected log files will determine the Weather Display data fields available for import. weectl import is able to import the following data from the indicated monthly log files:

MMYYYYlg.txt/MMYYlgcsv.csv:
- average wind speed
- barometer
- date and time
- dew point
- heat index
- outside humidity
- outside temperature
- rain fall
- wind direction
- wind gust speed
MMYYYYvantagelog.txt/MMYYYYvantagelogcsv.csv:
- date and time
- soil moisture
- soil temperature
- solar radiation
- UV index
MMYYYYvantageextrasensorslog.csv:
- date and time
- extra humidity 1
- extra humidity 2
- extra humidity 3
- extra humidity 4
- extra humidity 5
- extra humidity 6
- extra temperature 1
- extra temperature 2
- extra temperature 3
- extra temperature 4
- extra temperature 5
- extra temperature 6

Note

Whilst the above log files may contain the indicated data the data may only be imported subject to a suitable field map and in-use WeeWX archive table schema (refer to the [[FieldMap]] option).

Optional, the default is lg.txt, vantagelog.txt, vantageextrasensorslog.csv.

`source_encoding`¶

The Weather Display monthly log file encoding. This option is identical in operation to the CSV source_encoding option but applies to Weather Display imports only.

Optional, the default is utf-8-sig.

`interval`¶

Determines how the time interval (WeeWX database field interval) between successive observations is determined. This option is identical in operation to the CSV interval option but applies to Weather Display monthly log file imports only. As Weather Display log files nominally have entries at one minute intervals the recommended approach is to set interval = 1. As Weather Display monthly log files can, at times, have missing entries, the use of interval = derive may give incorrect or inconsistent interval values. If then archive_interval for the current WeeWX installation is 1 minute interval = conf may be used. In most cases the most appropriate setting will be interval = 1.

Optional, the default is 1.

`qc`¶

Determines whether simple quality control checks are applied to imported data. This option is identical in operation to the CSV qc option but applies to Weather Display imports only.

Optional, the default is True.

`calc_missing`¶

Determines whether any missing derived observations will be calculated from the imported data. This option is identical in operation to the CSV calc_missing option but applies to Weather Display imports only.

Optional, the default is True.

`txt_delimiter`¶

The character used as the field delimiter in Weather Display text format monthly log files (.txt files). A space is normally used but another character may be used if necessary. This parameter must be included in quotation marks.

Optional, the default is ' '.

`csv_delimiter`¶

The character used as the field delimiter in Weather Display csv format monthly log files (.csv files). A comma is normally used but another character may be used if necessary. This parameter must be included in quotation marks.

Optional, the default is ','.

`decimal`¶

The character used as the decimal point in the Weather Display monthly log files. A period is frequently used but another character may be used if necessary. This parameter must be included in quotation marks.

Optional, the default is '.'.

`ignore_missing_log`¶

Determines whether missing log files are to be ignored or the import aborted. Weather Display log files are complete in themselves and a missing log file will have no effect other than there will be no imported data for the period covered by the missing log file.

Optional, the default is True.

`ignore_invalid_data`¶

Determines whether invalid data in a source field is ignored or the import aborted. This option is identical in operation to the CSV ignore_invalid_data option but applies to Weather Display monthly log file imports only.

Optional, the default is True.

`tranche`¶

The number of records written to the WeeWX database in each transaction. This option is identical in operation to the CSV tranche option but applies to Weather Display monthly log file imports only.

Optional, the default is 250 which should suit most users.

`UV_sensor`¶

Enables weectl import to distinguish between the case where a UV sensor is present and the UV index is 0 and where no UV sensor is present and the UV index is 0. This option is identical in operation to the CSV UV_sensor option but applies to Weather Display monthly log file imports only.

Optional, the default is True.

`solar_sensor`¶

Enables weectl import to distinguish between the case where a solar radiation sensor is present and solar radiation is 0 and where no solar radiation sensor is present and solar radiation is 0. This option is identical in operation to the CSV solar_sensor option but applies to Weather Display monthly log file imports only.

Optional, the default is True.

`ignore_extreme_temp_hum`¶

Determines whether extreme temperature and humidity values are ignored. Weather Display log files record the value 255 for temperature and humidity fields if no corresponding sensor is present. Setting ignore_extreme_temp_hum = True will cause temperature and humidity values of 255 to be ignored. Setting ignore_extreme_temp_hum = False will cause temperature and humidity values of 255 to be treated as valid data to be imported.

Optional, the default is True.

Note

Setting ignore_extreme_temp_hum = False will cause temperature and humidity values of 255 to be imported; however, these values may be rejected by the simple quality control checks implemented if qc = True is used.

`[[FieldMap]]`¶

The [[FieldMap]] stanza defines the mapping from the Weather Display source data fields to WeeWX archive fields. This allows weectl import to take a source data field, perform the appropriate unit conversion and store the resulting value in the appropriate WeeWX archive field. The map consists of one sub-stanza per WeeWX archive field being populated using the following format:

    [[[weewx_archive_field_name]]]
        source_field = wd_field_name
        unit = weewx_unit_name
        is_cumulative = True | False
        is_text = True | False

Where

weewx_archive_field_name is a field in the in-use WeeWX archive table schema
wd_field_name is the name of a Weather Display source field as detailed in the Available Weather Display import field names table below.
weewx_unit_name is a WeeWX unit name; e.g., degree_C

Each WeeWX archive field stanza supports the following options:

source_field. The name of the Weather Display field to be mapped to the WeeWX archive field. Mandatory.
unit. The WeeWX unit name of the units used by source_field. Mandatory.
is_cumulative. Whether the WeeWX archive field is to be derived from a cumulative source field (e.g., daily rainfall) or not. Optional boolean value. Default is False.
is_text. Whether the source field is to be imported as text or not. Optional boolean. Default is False.

A mapping is not required for every WeeWX archive field and neither does every Weather Display field need to be included in a mapping.

Note

The unit option setting for each field map entry will depend on the Weather Display settings used to generate the Weather Display log files. Depending on the Weather Display field type, the supported WeeWX units names for that field may only be a subset of the corresponding WeeWX unit names; e.g., WeeWX supports temperatures in Celsius, Fahrenheit and Kelvin, but Weather Display log files may only include temperatures in Celsius or Fahrenheit. Refer to Units for details of available WeeWX unit names.

Available Weather Display import field names
Field name	Description
datetime	Date and time
barometer	Barometric pressure
dailyet	Daily evapotranspiration
dewpoint	Dew point
direction	Wind direction
gustspeed	Wind gust speed
heatindex	Heat index
humidity	Outside humidity
hum1	Extra humidity 1
hum2	Extra humidity 2
hum3	Extra humidity 3
hum4	Extra humidity 4
hum5	Extra humidity 5
hum6	Extra humidity 6
hum7	Extra humidity 7
radiation	Solar radiation
rainlastmin	Rainfall in the last 1 minute
soilmoist	Soil moisture
soiltemp	Soil temperature
temperature	Outside temperature
temp1	Extra temperature 1
temp2	Extra temperature 2
temp3	Extra temperature 3
temp4	Extra temperature 4
temp5	Extra temperature 5
temp6	Extra temperature 6
temp7	Extra temperature 7
uv	UV index
windspeed	Average wind speed

Note

The above field names are internally generated by weectl import and do not represent any field names used within Weather Display. They have only been provided for use in the field map.

For example, the following field map might be used to import outside temperature to WeeWX field outTemp, outside humidity to WeeWX field outHumidity and extra temperature 1 to WeeWX field poolTemp:

[[FieldMap]]
    [[[dateTime]]]
        source_field = datetime
        unit = unix_epoch
    [[[outTemp]]]
        source_field = temperature
        unit = degree_C
    [[[outHumidity]]]
        source_field = humidity
        unit = percent
    [[[poolTemp]]]
        source = temp1
        unit = degree_C

Note

Any WeeWX archive fields that are derived (e.g., dewpoint) and for which there is no field mapping may be calculated during import by use of the calc_missing option in the [WD] section of the import configuration file.

Note

The dateTime field map entry is a special case. Whereas other field map entries may use any WeeWX unit name for a unit supported by the import source, the dateTime field map entry must use the WeeWX unit name unix_epoch.

The example Weather Display import configuration file located in the util/import directory contains an example field map.

[WeatherCat]¶

The [WeatherCat] section contains the options relating to the import of observational data from WeatherCat monthly .cat files.

`directory`¶

The full path to the directory containing the year directories that contain the WeatherCat monthly .cat files to be imported. Do not include a trailing /.

Mandatory, there is no default.

`source_encoding`¶

The WeatherCat monthly .cat file encoding. This option is identical in operation to the CSV source_encoding option but applies to WeatherCat imports only.

Optional, the default is utf-8-sig.

`interval`¶

Determines how the time interval (WeeWX database field interval) between successive observations is determined. This option is identical in operation to the CSV interval option but applies to WeatherCat imports only. As WeatherCat monthly .cat files can, at times, have missing entries, the use of interval = derive may give incorrect or inconsistent interval values. Better results may be obtained by using interval = conf if the archive_interval for the current WeeWX installation is the same as the WeatherCat .cat file log interval, or by using interval = x where x is the time interval in minutes used in the WeatherCat monthly .cat file(s). The most appropriate setting will depend on the completeness and (time) accuracy of the WeatherCat data being imported.

Optional, the default is derive.

`qc`¶

Determines whether simple quality control checks are applied to imported data. This option is identical in operation to the CSV qc option but applies to WeatherCat imports only.

Optional, the default is True.

`calc_missing`¶

Determines whether any missing derived observations will be calculated from the imported data. This option is identical in operation to the CSV calc_missing option but applies to WeatherCat imports only.

Optional, the default is True.

`decimal`¶

The character used as the decimal point in the WeatherCat monthly .cat files. This parameter must be included in quotation marks.

Optional, the default is '.'.

`tranche`¶

The number of records written to the WeeWX database in each transaction. This option is identical in operation to the CSV tranche option but applies to WeatherCat imports only.

Optional, the default is 250 which should suit most users.

`UV_sensor`¶

Enables weectl import to distinguish between the case where a UV sensor is present and the UV index is 0 and where no UV sensor is present and the UV index is 0. This option is identical in operation to the CSV UV_sensor option but applies to WeatherCat imports only.

Optional, the default is True.

`solar_sensor`¶

Enables weectl import to distinguish between the case where a solar radiation sensor is present and solar radiation is 0 and where no solar radiation sensor is present and solar radiation is 0. This option is identical in operation to the CSV solar_sensor option but applies to WeatherCat imports only.

Optional, the default is True.

`[[FieldMap]]`¶

The [[FieldMap]] stanza defines the mapping from the WeatherCat source data fields to WeeWX archive fields. This allows weectl import to take a source data field, perform the appropriate unit conversion and store the resulting value in the appropriate WeeWX archive field. The map consists of one sub-stanza per WeeWX archive field being populated using the following format:

    [[[weewx_archive_field_name]]]
        source_field = wc_field_name
        unit = weewx_unit_name
        is_cumulative = True | False
        is_text = True | False

Where

weewx_archive_field_name is a field in the in-use WeeWX archive table schema
wc_field_name is the nameof a WeatherCat source field as detailed in the Available WeatherCat import field names table below.
weewx_unit_name is a WeeWX unit name; e.g., degree_C

Each WeeWX archive field stanza supports the following options:

source_field. The name of the WeatherCat field to be mapped to the WeeWX archive field. Mandatory.
unit. The WeeWX unit name of the units used by source_field. Mandatory.
is_cumulative. Whether the WeeWX archive field is to be derived from a cumulative source field (e.g., daily rainfall) or not. Optional boolean value. Default is False.
is_text. Whether the source field is to be imported as text or not. Optional boolean. Default is False.

Note

The unit option setting for each field map entry will depend on the WeatherCat settings used to generate the WeatherCat .cat files. Depending on the WeatherCat field type, the supported WeeWX unit names
for that field may only be a subset of the corresponding WeeWX unit names; e.g., WeeWX supports temperatures in Celsius, Fahrenheit and Kelvin, but WeatherCat .cat files may only include temperatures in Celsius or Fahrenheit. Refer to Units for details of available WeeWX unit names.

A mapping is not required for every WeeWX archive field and neither does every WeatherCat field need to be included in a mapping.

Available WeatherCat import field names
Field name	Description
datetime	date and time
Pr	Barometric pressure
D	Dew point
Hi	Inside humidity
Ti	Inside temperature
H1	Extra humidity 1
H2	Extra humidity 2
T1	Extra temperature 1
T2	Extra temperature 2
T3	Extra temperature 3
Lt1	Leaf temperature 1
Lt2	Leaf temperature 2
Lw1	Leaf wetness 1
Lw2	Leaf wetness 2
H	Outside humidity
T	Outside temperature
P	Precipitation
Sm1	Soil moisture 1
Sm2	Soil moisture 2
Sm3	Soil moisture 3
Sm4	Soil moisture 4
St1	Soil temperature 1
St2	Soil temperature 2
St3	Soil temperature 3
St4	Soil temperature 4
S	Solar radiation
U	UV index
Wc	Windchill
Wd	Wind direction
Wg	Wind gust speed
W	Wind speed

Note

The above field names are internally generated by weectl import and do not represent any field names used within WeatherCat. They have only been provided for use in the field map.

For example, the following field map might be used to import outside temperature to WeeWX field outTemp, outside humidity to WeeWX field outHumidity and extra temperature 1 to WeeWX field poolTemp:

[[FieldMap]]
    [[[dateTime]]]
        source_field = datetime
        unit = unix_epoch
    [[[outTemp]]]
        source_field = T
        unit = degree_C
    [[[outHumidity]]]
        source_field = H
        unit = percent
    [[[poolTemp]]]
        source = T1
        unit = degree_C

Note

Any WeeWX archive fields that are derived (e.g., dewpoint) and for which there is no field mapping may be calculated during import by use of the calc_missing option in the [WeatherCat] section of the import configuration file.

Note

The dateTime field map entry is a special case. Whereas other field map entries may use any WeeWX unit name for a unit supported by the import source, the dateTime field map entry must use the WeeWX unit name unix_epoch.

The example WeatherCat import configuration file located in the util/import directory contains an example field map.

Configuration options

source¶

[CSV]¶

file¶

source_encoding¶

delimiter¶

decimal¶

interval¶

qc¶

calc_missing¶

ignore_invalid_data¶

tranche¶

UV_sensor¶

solar_sensor¶

raw_datetime_format¶

wind_direction¶

[[FieldMap]]¶

[WU]¶

station_id¶

api_key¶

interval¶

qc¶

calc_missing¶

ignore_invalid_data¶

tranche¶

wind_direction¶

[[FieldMap]]¶

[Cumulus]¶

directory¶

source_encoding¶

interval¶

qc¶

calc_missing¶

separator¶

delimiter¶

decimal¶

ignore_invalid_data¶

tranche¶

UV_sensor¶

solar_sensor¶

[[FieldMap]]¶

[WD]¶

directory¶

logs_to_process¶

source_encoding¶

interval¶

qc¶

calc_missing¶

txt_delimiter¶

csv_delimiter¶

decimal¶

ignore_missing_log¶

ignore_invalid_data¶

tranche¶

UV_sensor¶

solar_sensor¶

ignore_extreme_temp_hum¶

[[FieldMap]]¶

[WeatherCat]¶

directory¶

source_encoding¶

interval¶

qc¶

calc_missing¶

decimal¶

tranche¶

UV_sensor¶

solar_sensor¶

[[FieldMap]]¶

`source`¶

`file`¶

`source_encoding`¶

`delimiter`¶

`decimal`¶

`interval`¶

`qc`¶

`calc_missing`¶

`ignore_invalid_data`¶

`tranche`¶

`UV_sensor`¶

`solar_sensor`¶

`raw_datetime_format`¶

`wind_direction`¶

`[[FieldMap]]`¶

`station_id`¶

`api_key`¶

`interval`¶

`qc`¶

`calc_missing`¶

`ignore_invalid_data`¶

`tranche`¶

`wind_direction`¶

`[[FieldMap]]`¶

`directory`¶

`source_encoding`¶

`interval`¶

`qc`¶

`calc_missing`¶

`separator`¶

`delimiter`¶

`decimal`¶

`ignore_invalid_data`¶

`tranche`¶

`UV_sensor`¶

`solar_sensor`¶

`[[FieldMap]]`¶

`directory`¶

`logs_to_process`¶

`source_encoding`¶

`interval`¶

`qc`¶

`calc_missing`¶

`txt_delimiter`¶

`csv_delimiter`¶

`decimal`¶

`ignore_missing_log`¶

`ignore_invalid_data`¶

`tranche`¶

`UV_sensor`¶

`solar_sensor`¶

`ignore_extreme_temp_hum`¶

`[[FieldMap]]`¶

`directory`¶

`source_encoding`¶

`interval`¶

`qc`¶

`calc_missing`¶

`decimal`¶

`tranche`¶

`UV_sensor`¶

`solar_sensor`¶

`[[FieldMap]]`¶