User's guide to the weewx weather system
Version 1.13

Table of Contents

This is the complete guide to installing weewx — all details and options are documented here. As such, it can get a bit tedious to follow.

If you are running Ubuntu, you can save some time by following the Quick Start Guide for Debian (Ubuntu) Systems.

For information on customizing weewx, see the separate document Customizing weewx.

For instructions on upgrading from various versions, see the separate document Upgrading weewx.

For instructions on porting to the SheevaPlug, see the separate document Notes on porting weewx to the SheevaPlug.

Copyright

(c) 2009, 2010, 2011 by Tom Keffer <tkeffer@gmail.com>

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses.

About weewx

weewx is a piece of software, written in Python, that interacts with your weather station to produce plots, reports, and HTML pages. It can optionally upload the reports to a remote Web server as well as publish to the WeatherUnderground or PWSweather.com. It uses modern software concepts, making it simple, robust, and easy to extend. For an example station see Hood River West.

Key features:

I wrote weewx over the winter of 2008-2009 for two reasons: it was a wet and miserable winter here in Oregon with not much else to do, so there was no good reason not to, and because I wanted a simple, easy-to-understand server to run my Davis VantagePro2 weather station on a Linux box. I had been using wview, which is a high-performance and feature rich system authored by Mark Teel with lots of users. Written in C, it's an efficient system that can run on underpowered boxes. In exchange, it's huge (45,000+ lines of code), tightly integrated in with its companion library, radlib (another 14,000+ lines), and very complex, making it difficult to understand and reliably customize. I wanted something more modern and much simpler.

Having made a career in C++ and Java, I was also interested in some more modern languages, so I thought I'd try either Python or Ruby (although, truth be told, the roots of Python are nearly as old as C++!). I ended up picking Python because its libraries are more mature and there are many mores choices for third party libraries.

Weewx weighs in at well under 5,000 lines of code. It also has another 2,900 comment lines. Because it is pure Python, it requires no makefiles, no builds, no special installs. It offers very powerful configuration and templating options, as well as an internally extensible engine, making it easy to customize. Its internal modular design and use of modern exception handling make it very robust and difficult to crash. It is also architecturally very simple and easy to understand. However, to be fair, at this point it supports only the  Davis VantagePro2 weather station.

Downloading weewx

Weewx can be downloaded from its SourceForge page:http://sourceforge.net/projects/weewx/.

Prerequisites

Python

Python V2.5, V2.6, or V2.7 is required. The newer V3.0 distribution will not work.

External packages

The following external packages are required to use weewx.

  1. configobj (Version 4.5 or greater) Manages the configuration file weewx.conf.
  2. pyserial (Version 2.3 or greater) Manages the serial connection to the weather station.
  3. Cheetah (Version 2.0 or greater) The HTML templating engine.
  4. Python Imaging Library (Version 1.1.6 or greater) Also known as PIL, this is included in many Python distributions.

In addition, there is one optional package:

There are two general strategies for installing these prerequisites:

  1. Use operating system tools, such as apt-get (or its graphical equivalent Synaptic Package Manager) for Debian/Ubuntu or yast for SuSE; or
  2. Use the Python tool pip.

Option #1 is easier, but if your Linux distribution does not come with such tools, you may have to use pip.

Brief instructions for both approaches are given below.

Installation on Debian distributions (including Ubuntu and Mint) using apt-get

The instructions that follow are for using the Debian tool apt-get, but the same package names would be used should you chose to use a graphical interface such as the Synaptic Package Manager.

To install the required packages:

sudo apt-get install python-configobj
sudo apt-get install python-serial
sudo apt-get install python-cheetah
sudo apt-get install python-imaging

Note that the python-imaging (PIL) package is often preinstalled on many Debian distributions so the install may not actually do anything.

As noted above, if you choose to install the optional package pyephem, then it will have to be installed using pip. See their website for instructions on how to install this tool. Once installed, then installing pyephem is easy:

sudo pip install pyephem

Installation on Fedora using yum

Installing on Fedora v16 can be done entirely by using yum.

sudo yum install python-configobj
sudo yum install python-imaging
sudo yum install python-cheetah
sudo yum install pyserial

The optional package pyephem can also be installed by using yum:

sudo yum install pyephem

Installation on SuSE using yast

My SuSE 11.3 system came with some of the prerequisites installed, some available through yast, and some that required pip.

Using yast, install the following:

sudo yast -i gcc
sudo yast -i python-devel
sudo yast -i python-configobj
sudo yast -i python-serial
sudo yast -i python-imaging
sudo yast -i python-setuptools

Now install pip. There are many ways to do this (see the pip website), but perhaps the simplest is by using get-pip.py:

sudo curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python

Now using pip, install Cheetah:

pip install Cheetah

Finally, if you choose to install the optional package pyephem, then install it using pip:

pip install pyephem

Installation using pip

An alternative approach to installing the required packages is by using the Python setup tool "pip". Refer to their instructions on how to install this tool.

Once pip has been installed, installing the rest of the packages is very easy.

pip install configobj
pip install pyserial
pip install Cheetah
pip install pil

If you choose to install the optional package pyephem:

pip install pyephem

Time

The time on the VantagePro is automatically synchronized with the weewx server nominally every four hours (changeable by the user). You should run a NTP daemon on your server to insure that it is synchronized with the correct time. Doing so will greatly reduce errors, especially if you send data to services such as the Weather Underground.

System requirements

I run weewx on a 500MHz system with an AMD Geode processor and 512 MB of memory.  Configured this way, it consumes about 5% of the CPU and about 40MB of total memory.

Weather station hardware requirements

At this point, the following Davis Vantage weather stations are supported:

Installing weewx

Because weewx is "Pure Python", that is it is 100% Python with no "C" modules to compile, installing it is very easy. Furthermore, it uses the standard Python distutils install method, which is very easy and flexible. Detailed instructions follow.

Unpacking

Start by unpack the tar ball (substitute your version for X.Y.Z) into any convenient directory where you have write permission

tar xvf weewx-X.Y.Z

Then change directory into it:

cd weewx-X.Y.Z

Choosing where to install weewx

Next step is figuring out where you want to install weewx. If $WEEWX_INSTALL symbolizes the root location of where the weewx directory will be installed, then the nominal directory layout is:

Subdirectory Contents
$WEEWX_INSTALL/bin Python packages and scripts
$WEEWX_INSTALL/weewx.conf Configuration file
$WEEWX_INSTALL/skins Template skins and their configuration files
$WEEWX_INSTALL/archive sqlite3 archive and statistical databases
$WEEWX_INSTALL/public_html Generated HTML and .PNG images

By default, the location for WEEWX_INSTALL is /home/weewx. However, it can be changed by editing the file setup.cfg and changing the following line

home = /home/weewx

to reflect your decision.

Build and install

Build the distribution

./setup.py build

(Because weewx is pure Python this doesn't actually build anything, but it does arrange files for the final installation)

Then install it. If you have write permission in the directory where weewx will go (i.e., $WEEWX_INSTALL), then type

./setup.py install

Otherwise, if you do not have write permission, you will have to use sudo:

sudo ./setup.py install

Upgrading

Before upgrading from a previous version of weewx, check the upgrade notes, found in file upgrading.htm, to see if there any specific actions you need to do.

You then generally follow the procedure above.

In particular, before starting, be sure to set home in the file setup.cfg.

The build and install process will do the following for you.

Permissions

Strictly speaking it is not necessary to install or run weewx with root privileges. You only need read/write access to the serial port for your hardware. For example, if your hardware has a USB interface, on Ubuntu and SuSE:

sudo chmod a+rw /dev/ttyUSB0

Once done, if you edit setup.cfg to install into a directory where you have write permissions, you can install and run weewx without any root privileges at all.

Final note on installation

This is an advanced topic. Ignore it if you are just getting started.

Because weewx is pure Python, it does not actually have to be "built" and "installed" at all! You can just simply run it out of whatever directory you unpack it into. This can be useful for testing and experimentation of new versions. If you try this, weewx must still be told where to find the skin templates and databases, as well as where to put the generated HTML files and plot images. This is done by setting WEEWX_ROOT in weewx.conf to reflect where your data files can be found (normally, the install process does this for you):

> # Edit WEEWX_ROOT to reflect the location of your data hierarchy:
> vi ./weewx-X.Y.Z/weewx.conf
> # Now run weewxd.py out of the directory you unpacked weewx into:
> ./weewx-X.Y.Z/bin/weewxd.py ./weewx-X.Y.Z/weewx.conf

Configuring weewx

This section covers configuring weewx, in particular the configuration files weewx.conf and skin.conf.

In the following, $WEEWX_ROOT refers to the weewx data root directory, generally /home/weewx. The subdirectories archive, skins, and public_html are expected to be found here.

Moving from wview

NOTE: This section is necessary only if you are moving from wview to weewx and wish to transfer your old data over. If you are starting afresh, you do not need to follow this section — the two main databases are created and populated automatically by weewx.

The main archive database can be just copied over because wview and weewx use identical schemas:

mkdir $WEEWX_ROOT/archive
cp /usr/local/var/wview/archive/wview-archive.sdb $WEEWX_ROOT/archive/weewx.sdb

The second database, the statistics, will then be automatically created and backfilled on startup of weewx. If your existing database is large, backfilling could take some time. On my modest 500 MHz fit-PC Slim with 512 MB of memory it took a little over 4 minutes for a year and a half (25 MB) of data.

Configuring your weather station hardware

Weewx comes with a configuration utlity, configure.py, that can set many of the on-board EEPROM constants in your weather station, such as its archive interval, altitude, rain bucket type, etc. It can also create and modify archive and stats databases, but this is an advanced topic that is not normally needed.

Run it with the configuration file as an argument and --help as an option to see its usage:

$WEEWX_ROOT/bin/configure.py $WEEWX_ROOT/weewx.conf --help

It is useful to run it with the --info option to see what the current EEPROM settings are on your station:

$WEEWX_ROOT/bin/configure.py $WEEWX_ROOT/weewx.conf --info

This will print out something like:

VantagePro EEPROM settings:
       
        CONSOLE FIRMWARE DATE: Nov 28 2005
       
        CONSOLE SETTINGS:
          Archive interval: 300 (seconds)
          Altitude:         700 (foot)
          Wind cup type:    large
          Rain bucket type: 0.01 inches
          Rain year start:  10
         
        CONSOLE UNITS:
          Barometer:   mbar
          Temperature: degree_F
          Rain:        inch
          Wind:        knot
         
        RECEPTION STATS:
          Total packets received:       18668
          Total packets missed:         2144
          Number of resynchronizations: 2
          Longest good stretch:         1887
          Number of CRC errors:         659
         
        BAROMETER CALIBRATION DATA:
          Current barometer reading:    30.060 inHg
          Altitude:                     700 feet
          Dew point:                    50 F
          Virtual temperature:          61 F
          Humidity correction factor:   24
          Correction ratio:             1.025
          Correction constant:          +0.027 inHg
          Gain:                         -1.000
          Offset:                       -1.000

The values highlighted in yellow above can be changed. They are:

Set the value you would like in the appropriate place in your configuration file weewx.conf (described below) and then run the utility with the option --configure:

$WEEWX_ROOT/bin/configure.py $WEEWX_ROOT/weewx.conf --configure

The utility will read the new value from the configuration file and, if it is different from the old value, ask for your permission to change it. If you agree, then it will send the necessary commands to your hardware to change the EEPROM settings.

The following discusses the details of these various parameters.

Archive interval

The archive interval is set in the main configuration file $WEEWX_ROOT/weewx.conf. Look for the entry archive_interval in the VantagePro section. Set it to the number of seconds. Valid entries are 60, 300, 600, 900, 1800, 3600, and 7200. However, if you are ftp'ing lots of files to a server, setting it to 60 seconds may not give enough time to have them all uploaded before the next archive record is due. If this is the case, you should pick an archive interval of at least 300 seconds, or trim the number of files you are using.

I have found that a five minute (300 seconds) archive interval works well for the VantagePro. Because of the large amount of onboard memory it carries, going to a larger interval really does not have any advantages.

Choose your archiving interval carefully! Once chosen, it cannot be changed without messing up your statistics (highs and lows will be OK, but averages and rms wind speed will be wrong).

Barometer and altitude

All barometer readings are referenced to sea level pressure. Hence, the console needs to know the station's altitude. You also may also want to calibrate your barometer if you have access to a known-accurate reading of a nearby station.

Look for the entry altitude in the Station section. Change it to your station's altitude, along with the units used. For example:

altitude = 213, meter

for a station at 213 meters (700 feet).

When you run the configuration utility it will give you the opportunity to type in the reference pressure of the nearby station, or select a reasonably accurate default. You can also just hit <enter> to use the previous calibration setting.

Here's a sample session:

BAROMETER AND ALTITUDE CALIBRATION
For the following, you have three choices:
1. If you have a current barometer reading from a very reliable nearby
   reference, you can use it to calibrate the barometer in your station.
   In this case, give the value and the unit it is in.
   Examples: '30.15 inHg'
             '1024.5 mbar'
   (The console is currently reading 30.062 inHg or 1017.9 mbar)
2. If you do not have such a reading, then you can use a value of zero,
   which will result in a default, sensible calibration.
3. Finally, you can simply hit <enter>, which will leave the current calibration
   value unchanged.
Response: 30.055 inHg
Calibrated pressure will be 30.055 inHg
Altitude will be 700 feet.
Proceeding will change the altitude and/or barometer calibration.
Are you sure you want to proceed? (Y/n) Y
New altitude is 700 feet, new barometer reading is 30.055 inHg

The user responses are in blue. Note how the old pressure was 30.062, but the correction constant was changed so the pressure now reads 30.055.

Rain bucket type

Normally, this is set by Davis, but if you have replaced your bucket with a different kind, you might want to reconfigure. Look for rain_bucket_type under section 'VantagePro' in the configuration file weewx.conf. You may have to uncomment the line. For example, you might change it to

rain_bucket_type = 1

if you had a 0.2 MM rain bucket.

Editing the configuration file weewx.conf

Station specific information is set in the configuration file $WEEWX_ROOT/weewx.conf. (There is another configuration file skin.conf for presentation-specific options, which is described in a separate document Customizing weewx under section Reference: The standard skin configuration file.)

Most of the important options are up near the top of the file. They are all documented in this section, although you can safely ignore most of them. The truly important ones, the ones you are likely to have to customize for your station, are shown in bold face and in blue.

Default values are provided for many of them, meaning that if they are not listed in the configuration file at all, weewx will pick sensible values. When the documentation below gives a "default value" this is what it means. However, all options have been given values in the configuration file that ships with weewx, so you can see what they look like. The value given in this shipped configuration file is not necessarily the same as the "default value".

What follows is organized by the different sections of the configuration file.

General

The options declared at the top are not actually part of any section. There are two:

debug

Set to 1 to have the program perform extra debug checks, as well as emit extra information on the log file. Otherwise, set to 0. Default is 0 (no debug).

socket_timeout

Set to how long to wait before declaring a socket time out. This is used when FTP'ing data to a web server or sending data to the Weather Underground. Twenty (20) seconds is reasonable. Default is 20.

[Station]

This section covers options relating to the entire weather station setup.

WEEWX_ROOT

Set to the root directory of the weewx data file hierarchy for this station, nominally '/home/weewx'. The weewx data subdirectories skins, archive, and public_html are expected to be found here. This value will be set automatically by the setup script setup.py to reflect the choice you made in the configuration file setup.cfg. Required. No default.

location

The station location should be a UTF-8 string that describes the geography of where you weather station is located, such as 'Hood River, Oregon'. Required. No default.

latitude
longitude

The lat/lon should be set in decimal degrees, negative for southern and western hemispheres, respectively. Required. No default.

altitude

Set to the altitude of the station and the unit used for the altitude. See additional details under configuring your station altitude. Required. No default. Example:

altitude = 700, foot

rain_year_start

If your area uses a rain year that starts on something other than the first of January, you may want to set this variable. For example, set to 10 if your rain year starts in October (as mine does). Default is 1.

week_start

Start of the week. 0=Monday, 1= Tuesday, ... , 6 = Sunday. Default is 6 (Sunday)

clock_check

How often to check the station's onboard clock for drift, in seconds. Default is 14400 (every 4 hours)

max_drift

The maximum amount of drift to tolerate, in seconds, in the onboard clock, before resetting the clock. Default is 5.

cache_loop_data

Set to 1 (one) to cache LOOP data, otherwise, set to zero. Most users will not want to change this unless you have a specialized application. Default is 1 (i.e., cache LOOP data).

station_type

Set to the type of hardware you are using. For this version, only 'VantagePro' is accepted. Required.

[VantagePro]

This section is for options relating to the VantagePro hardware.

archive_interval

Set to the desired archive interval of your station, in seconds. This variable is only used when first setting up your station using the configuration program configure.py. Otherwise, this value is read from the station hardware EEPROM. See additional details under configure your station archive interval. No default.

type

Set to either 'serial', if you have a serial or USB connection to your VantagePro (by far the most common), or to 'ethernet' if you have the WeatherLinkIP. No default.

port

If you chose 'serial' above, then set to the serial port name used by your station. Example, /dev/ttyUSB0 is a common location for USB ports under Debian, /dev/ttyS0 for serial ports. Otherwise, not required. No default.

host

If you chose 'ethernet' above, then give the either the IP address (e.g., 192.168.0.1) or hostname (e.g., console.mydomain.com) to your console. Otherwise, not required. No default.

rain_bucket_type

The type of rain bucket you have. This variable is used only by the configuration program configure.py. The rest of the time, the bucket type is read from the station hardware EEPROM. Normally, this option is left commented out and therefore the configuration program leaves it alone. However, if for some reason you change your bucket type, or if your bucket type is wrong, uncomment the line and set it to the appropriate type (0=0.01 inch bucket; 1=0.2 MM bucket; 2=0.1 MM bucket) then run the configuration program.

baudrate

Set to the baudrate of your station. The default is 19200.

tcp_port

The port where your WeatherLinkIP will be listening. Default is 22222.

tcp_send_delay

How long to block after sending a socket packet to your WeatherLinkIP. Default is 1 second.

iss_id

Set to the ID number of your Integrated Sensor Suite (ISS). This is used in the formula to calculate reception quality for wireless stations. The default is 1.

archive_delay

How long to wait in seconds after the top of an archiving interval before fetching new data off the station. For example, if your archive interval is 5 minutes and archive_delay is set to 15, then the data will be fetched at 00:00:15, 00:05:15, 00:10:15, etc. This delay is to give the station a few seconds to archive the data internally, and in case your server has any other tasks to do at the top of the minute. Default is 15 seconds.

timeout

How many seconds  to wait for a response from the station before giving up. Default is 5 seconds.

wait_before_retry

How many seconds to wait before retrying again. Unless you have a good reason to change it, this value should be left at the default, as it is long enough for the station to offer new data, but not so long as to go into a new loop packet (which arrive every 2 seconds). Default is 1.2 seconds.

max_tries

How many times to try again before giving up. Default is 4.

unit_system

What unit system is in use on your weather station hardware. Possible values are '1' (U.S. Customary) or '2' (Metric). As far as I know, all Davis instruments support only U.S.. In any case, U.S. is the only system supported by weewx at this time. Default is 1.

[RESTful]

This section is for configuring uploads to simple RESTful servers such as the Weather Underground, PWSweather.com, or CWOP.

[[Wunderground]]

Weewx can send your current data to the Weather Underground. If you do not wish to do this, comment out the two options below.

station

Set to your Weather Underground station ID (e.g., KORHOODR3). Required.

password

Set to your Weather Underground password. Required.

[[PWSweather]]

Weewx can send your current data to the PWSweather.com service. If you do not wish to do this, comment out the two options below.

station

Set to your PWSweather station ID. Required.

password

Set to your PWSweather password. Required.

[[CWOP]]

Weewx can send your data to the Citizen Weather Observer Program. If you do not wish to do this, comment out option "station" below.

station

Set to your CWOP station ID (e.g., CW1234). Required.

passcode

This is used for APRS (amateur radio) stations only. Set to the passcode given to you by the CWOP operators. Otherwise, leave this option commented out [Required for APRS stations; ignored for others]

server

A comma separated list of servers:ports to try. Unless you have a good reason to change this, use the servers listed. [Required.]

interval

The interval in seconds between posts. Because CWOP is heavily used, the operators discourage very frequent posts. Every 5 minutes (300 seconds) is fine, but they prefer every 10 minutes (600 s) or even longer. Setting this value to zero will cause every archive record to be posted. [Optional. Default is zero.]

stale

How old a record can be before it will not be used for a catch up. CWOP does not use the timestamp on a posted record. Instead, they use the wall clock time that it came in. This means that if your station is off the air for a long period of time, then weewx attempts a catch up, old data could be interpreted as the current conditions. [Optional. Default is 1800 seconds.]

[Archive]

This section is for configuring the sqlite3 database in which the station archive data is stored.

archive_file

The path, relative to the WEEWX_ROOT directory, to the database. Required

unit_system

What unit system to use inside the database. Required. The only one supported right now is '1', the U.S. Customary system

[Stats]

This section is for configuring the sqlite3 database in which the station statistics are stored.

stats_file

The path, relative to the WEEWX_ROOT directory to the statistical database. Required.

stats_types

The list of types for which statistics will be kept. Types not listed will not be available for generating HTML pages. Optional. The default is all types, resulting in a possibly much bigger than necessary stats database (do you really have four different soil moisture sensors?) The list that ships with the configuration file will work for most stations and probably will not have to be modified.

[Reports]

This section controls which reports are to be generated. While it can be highly customized for your individual situation, this documentation describes the section as shipped in the standard distribution.

Each report is represented by a sub-section, marked with double brackets (e.g., [[MyReport]]). Any options for the report should be placed under it. The standard report service will go through the sections, running each report in order. Hence, for the stock distribution, report [[StandardReport]] will be run first, then report [[FTP]] (which actually optionally uploads the results to a remote web server). Details for how to customize reports are in the section Opportunities for customizing reports, in the document Customizing weewx.

SKIN_ROOT

The directory relative to $WEEWX_ROOT where the skins live. Default is skins.

HTML_ROOT

The target directory for the generated files, relative to $WEEWX_ROOT. Generated files and images will be put here. Default is public_html.

[[StandardReport]]

This is the standard report that will be run on every archiving interval. It uses the skin "Standard", which generates four HTML pages ("day", "week", "month", and "year" observations), plot graphs for same, an RSS feed, and NOAA monthly and yearly reports. Unless changed otherwise, it uses US Customary Units and puts the results in public_html and subdirectory public_html/NOAA.

[[FTP]]

While this "report" doesn't actually generate anything, it uses the report machinery to upload files from directory $HTML_ROOT to a remote webserver. It does an incremental update, that is, it only FTPs any files that have changed, saving the outgoing bandwidth of your Internet connection.

If you do not use such a server, comment out the first four options below.

user

Set to the username you use for your FTP connection to your web server. Required. No default.

password

Set to the password you use for your FTP connection to your web server. Required. No default.

server

Set to the name of your web server (e.g., www.threefools.org, in my case). Required. No default

path

Set to the path where the weather data will be stored on your webserver (e.g., '/weather'). NB: some FTP servers require a leading slash ('/'), some don't. Required. No default.

passive

Set to 1 if you wish to use the more modern, FTP passive mode, 0 if you wish to use active mode. Passive mode generally works better through firewalls, but not all FTP servers do a good job of supporting it. See Active FTP vs. Passive FTP, a Definitive Explanation for a good explanation of the difference. Default is 1 (passive mode).

max_tries

Weewx will try up to this many times to FTP a file up to your server before giving up. Default is 3.

[Calibrate]

One of the services that weewx can run is a calibration service called StdCalibrate. This service offers an opportunity to correct for any calibration errors in your instruments. It is very general and flexible.

If you do not wish to apply any calibrations, you can leave it out of service_list, the list of services to be run, and it will not be loaded or run at all.

[[Corrections]]

In this section you list all correction expressions. For example, say that you know your outside thermometer reads high by 0.2°F. You could add the expression:

outTemp = outTemp - 0.2

Perhaps you need a linear correction around a reference temperature of 68°F:

outTemp = outTemp + (outTemp-68) * 0.02

It is even possible to do corrections involving more than one variable. Suppose you have a temperature sensitive barometer:

barometer = barometer + (outTemp-32) * 0.0091

All correction expressions are run in the order given.

The units should be in the native units of the instrument, US Customary in the case of the VantagePro.

Both LOOP data and archive data will be corrected. It is important that the calibration service, StdCalibrate, be run before the archiving service StdArchive, so that it is the corrected data that is stored.

If all you require is a simple correction offset, this can also be done in the Davis VantagePro hardware. See your manual for instructions.

[QC]

Another service that weewx can run is a quality control service called StdQC. This service is very simple and only checks that values are within a minimum and maximum range.

If you do not wish to use this service, you can leave it out of service_list, the list of services to be run, and it will not be loaded or run.

[[MinMax]]

In this section you list the observation types you wish to have checked, along with their minimum and maximum values. The units should be in the native units of the weather station hardware (e.g., US Customary in the case of the Davis VantagePro2). For example,

[[MinMax]]
  outTemp = -40, 120
  barometer = 28, 32.5
  outHumidity = 0, 100

In this example, if a temperature should fall outside of the inclusive range -40 °F through 120 °F, then it will be set to the null value, None and ignored. In a similar manner, the acceptable values for barometric pressure would be 28 through 32.5 inHg, for humidity 0 through 100%.

Both LOOP and archive data will be checked. It is important that the quality control service be run after the calibration service StdCalibrate and before the archiving service StdArchive, so that it is the calibrated data that is checked and then stored.

Knowing the details of how your hardware encodes data helps to minimize the number of observations that need to be checked. For example, the VP2 devotes only one unsigned byte to storing wind speed, and even then 0xff is devoted to a bad value, so the only possible values that could appear are 0 through 126 mph, a reasonable range. So, for the VP2, there is no real point in checking wind speed.

[Engines]

This section is used to configure the internal service engine in weewx. It is for advanced customization. Details on how to do this is found in the section Customizing the weewx service engine in the document Customizing weewx.

[[WxEngine]]

This section is for options used by the service engine.

service_list

This option is the list of services that are to be run by the service engine. After each event (such as the arrival of LOOP data, etc.), they will be run in the given order. The standard list of services run by weewx is:

service_list = weewx.wxengine.StdCalibrate, weewx.wxengine,StdQC, weewx.wxengine.StdArchive, weewx.wxengine.StdTimeSynch, weewx.wxengine.StdPrint, weewx.wxengine.StdRESTful, weewx.wxengine.StdReportService

You can leave some of these services out if you do not need them. However, this will only make a slight difference in execution speed.

Running weewx

Weewx can be run either from the command line (useful for diagnostic purposes because it will print out a summary of every LOOP data), or as a daemon. When first trying weewx, it's probably best to run it from the command line because you will be able to see command line diagnostics, as well as log messages.

Running from the command line

Weewx can easily be run from the command line. Start by making sure you have appropriate permissions to the serial port your weather station uses. For example, if you are using a plain old serial port:

sudo chmod 666 /dev/ttyS0

Then run the main loop program, weewxd.py, giving the configuration file as its only parameter:

$WEEWX_ROOT/bin/weewxd.py $WEEWX_ROOT/weewx.conf

It should start by downloading any archive data from your weather station into the database $WEEWX_ROOT/archive/weewx.sdb. As the Davis VantagePro can store a couple thousand archive records internally, this could take a minute or two. I've found this process particularly slow on SuSE for some reason.

Weewx will then start monitoring LOOP data, printing a short version of the received data on standard output, about once every two seconds.

You can tell a running instance of weewx to reread its configuration file by sending it the HUP signal. First run ps to find out the Process ID (PID) number of the instance, then send it the HUP signal:

ps -a         # Note the PID of the weewxd.py process

kill -HUP pid # Send it a HUP signal

Running as a daemon

For unattended operations it is best to have weewx run as a daemon, started automatically when the server is rebooted. Start by selecting the appropriate run script. They can be found under $WEEWX_ROOT/start_script.

SuSE: $WEEWX_ROOT/start_script/SuSE/weewx
Debian/Ubuntu: $WEEWX_ROOT/start_script/Debian/weewx

Check the chosen script to make sure the variable WEEWX_ROOT inside has been set to the proper root directory for your weewx installation (it should have been set to the correct value automatically by the install process, but it's worth checking).

Copy it to the proper location for your system:

SuSE: cp $WEEWX_ROOT/start_script/SuSE/weewx /etc/init.d
Debian/Ubuntu: cp $WEEWX_ROOT/start_script/Debian/weewx /etc/init.d

Make sure the script is executable:

SuSE: chmod +x /etc/init.d/weewx
Debian/Ubuntu: chmod +x /etc/init.d/weewx

Create symbolic links in the run level directories:

SuSE: /usr/lib/lsb/install_initd /etc/init.d/weewx
Debian/Ubuntu: update-rc.d weewx defaults 98

Weewx will now start automatically whenever your system is booted. You can also manually start, stop, and restart the weewx daemon:

/etc/init.d/weewx start

/etc/init.d/weewx stop

/etc/init.d/weewx restart

By default, the scripts are designed to have weewx run at run levels 2, 3, 4 and 5. Incidentally, a nice tool for setting run levels with Debian (Ubuntu) systems is sysv-rc-conf. It uses a curses interface to allow you to change easily which run level any of your daemons runs at. There is a similar tool on SuSE. From the start menu run the YAST Control Center, then look for Systems Services (Runlevel). Pick "Expert" mode to see the run levels.

You can also tell weewx to reread its configuration file without stopping by using the 'reload' option:

/etc/init.d/weewx reload

Compatibility with wview

The sqlite3 archive database used by weewx (nominally, weewx.sdb) is completely compatible with the database used by wview (usually called wview-archive.sdb), at least as of wview Version 5.2.X. The schema and its semantics is identical. However, the statistical file stats.sdb is different, and must be rebuilt. This will be done automatically on startup by weewx.

Monitoring weewx

Weewx logs many events to the system log. On Debian systems, this is /var/log/syslog, on SuSE, /var/log/messages. Your system may use yet another place. When troubleshooting the system, be sure to check it!

Setting the option debug in weewx.conf to 1 (one) will generate many more checks and output and can be useful for debugging.

Troubleshooting

If you get stuck, be sure to

Execution

configobj errors

These are errors in the configuration file. Two are very common. Incidentally, these errors are far easier to diagnose when weewx is run from the command line.

configobj.DuplicateError exception

This error is caused by using an identifier more than once in the configuration file. For example, you may have inadvertently listed your FTP server twice:

[Reports]
	[[FTP]]
		... (details elided)
		user = fred
		server = ftp.myhost.com
		password = mypassword
		server = ftp.myhost.com  # OOPS! Listed it twice!
		path = /weather
		... 

Generally, if you encounter this error, the log file will give you the line number it happened in:

Apr 24 12:09:15 raven weewx[11480]: wxengine: Error while parsing configuration file /home/weewx/weewx.conf
Apr 24 12:09:15 raven weewx[11480]: wxengine: Unable to initialize main loop:
Apr 24 12:09:15 raven weewx[11480]:     ****  Duplicate keyword name at line 254.
Apr 24 12:09:15 raven weewx[11480]:     ****  Exiting. 

configobj.NestingError exception

This is a very similar error, and is caused by a misformed section nesting. For example:

[Reports]
	[[FTP]]]
		... (details elided)

Note the extra closing bracket on the subsection FTP.

Hardware

Tips on a reliable system

If you are having problems keeping your weather station up for long periods of time, here are some tips, in decreasing order of importance:

cp2101 converter problems

The USB converter used in the Davis VantagePro is known to have some "noise" problems. The symptom is that the Linux kernel will disconnect from your old USB port claiming "EMI noise", and reconnect to a new and different port, where weewx can't find it. Here's a typical log output:

Nov 29 10:40:21 hummingbird kernel: [6661624.786792] hub 2-0:1.0: port 3 disabled by hub (EMI?), re-enabling... 
Nov 29 10:40:21 hummingbird kernel: [6661624.786871] usb 2-3: USB disconnect, address 2 
Nov 29 10:40:21 hummingbird kernel: [6661624.795778] cp2101 2-3:1.0: device disconnected 
Nov 29 10:40:21 hummingbird weewx[25808]: VantagePro: Max retries exceeded while getting LOOP packets
 
... (messages elided)
 
Nov 29 10:40:22 hummingbird kernel: [6661625.352340] cp2101 2-3:1.0: cp2101 converter detected 
Nov 29 10:40:22 hummingbird kernel: [6661625.528107] usb 2-3: reset full speed USB device using ohci_hcd and address 3 
Nov 29 10:40:22 hummingbird kernel: [6661625.735497] usb 2-3: cp2101 converter now attached to ttyUSB1

In this example, the VantagePro was connected to /dev/ttyUSB0, but then reconnected to /dev/ttyUSB1.

If you put ferrite coils on the USB connection, you will eliminate 90% of this problem. I did this about 18 months ago, and have not had a problem since.

However, there is one final step you can take that will really harden up your system: install a udev script that will create a symbolic link to the VantagePro USB port, whatever it might be. With this approach, if the port jumps from ttyUSB0 to ttyUSB1, the symbolic link will follow it. You just specify port /dev/vpro in the configuration file weewx.conf and be done with it.

Here's how to do this:

Installing a udev script

I have installed a file /etc/udev/rules.d/vpro.rules on my fit-PC that looks like this:

# Automount the VantagePro2 to port /dev/vpro.
# Install in /etc/udev/rules.d/vpro.rules
#
ACTION=="add", ATTRS{interface}=="CP2102 USB to UART Bridge Controller", SYMLINK+="vpro"

What this rule says is that when the USB port is plugged in (action add), and it has an attribute with name interface that is equal to "CP2102 USB to UART Bridge Controller", then add a symbolic link for its physical port to /dev/vpro.

Here's a rule that works for my Serial-to-USB cable, made by Prolific Technology, Inc.:

# Automount Serial-to-USB cable to port /dev/vpro
# Install in /etc/udev/rules.d/cable.rules
#
ACTION=="add", ATTRS{product}=="USB-Serial Controller", SYMLINK+="vpro"

Your devices may, and probably will, have different identifiers!! I can recommend this article, "Writing udev rules," for how to find and write an appropriate udev rule for your controller. (Note, however, that this article uses the old udevinfo command, rather than the newer udevadm command.) In particular, run the command

# udevadm info --attribute-walk --path $(udevadm info --query=path --name=/dev/ttyUSB0) 

where /dev/ttyUSB0 is the port (substitute your real USB port) the VP2 is attached to. It will print out various identifiers that can be useful in identifying your VP2 to udev. While the first example script above used a rule that matched attribute interface, others are possible. For example, the second example, for the serial-to-USB cable, chose to match the attribute product.

Once you've installed your udev rule, you can then set port=/dev/vpro in weewx.conf, confident that it will always point to your VantagePro2, no matter which USB port it is actually attached to!

I have tested this system many times. You can yank the USB port out of the machine and then plug it back in while also pulling out the network connection in the middle of an FTP upload: weewx will recover.

Or, at least, it should!

FreeBSD

User Fabian reports that the following had to be done to get the VantagePro2 working under FreeBSD:

I needed the uslcom Driver for the usb/rs232 Adapter used by my vantage. Also I had to reset the memory of the weatherstation.
Loading the Driver:
Put uslcom_load="YES" in /boot/loader.conf (to load it as module).
Which gives here an output like:
uslcom0:  on usbus1
And put in weewx.conf:
    port = /dev/cuaU0

Weewx generates HTML pages, but it does not update them

If you are getting a symptom that everything appears normal, that is HTML is getting generated and getting FTP'd to your webserver (look in the log to be sure!), but your web pages are not being updated, it could be because the data on board your console has gotten garbled. The way the Davis Vantage series works is that the software (weewx in this case) asks the console for all archive data "since" some time. The console then downloads the records one at a time. The software can detect when the console has sent the last one because the timestamps on the records, instead of monitonically increasing, suddenly jump backwards in time. You've wrapped around the memory space of the console.

However, if the internal memory gets garbled, the console will immediately return archives in the past, and so it looks like the timestamps have decreased in value and so weewx figures that's it: no more data.

I have received reports from a couple of users that this has happened. There seems to be two fixes:

  1. Unplug the console, take out the batteries, and wait a minute or two. This will cause the console software to internally reboot. In one case this has fixed the problem without data loss.
  2. If all else fails, clear the memory of the console using the utility configure.py. This may cause loss of data, but usually works. Adjust paths as necessary:

cd /home/weewx
./bin/configure.py weewx.conf --clear

Templates

Cheetah.NameMapper.NotFound errors

If you get errors of the sort:

Apr 12 05:12:32 raven reportengine[3074]: filegenerator: Caught exception "<class 'NameMapper.NotFound'>" 
Apr 12 05:12:32 raven reportengine[3074]:          ****  Message: "cannot find 'fubar' in template /home/weewx/skins/Standard/index.html.tmpl"
Apr 12 05:12:32 raven reportengine[3074]:          ****  Ignoring template and continuing.

you have a tag in your template that weewx does not recognize (in this example, it's the tag $fubar in the template /home/weewx/skins/Standard/index.html.tmpl.

Architectural notes

This section is not needed to get started.

Goals

The primary design goals of weewx are:

Strategies

To meet these goals, the following strategies were used:

While weewx is nowhere near as fast at generating images and HTML as its predecessor, wview (this is partially because it uses fancier fonts and a much more powerful templating engine), it is 'fast enough' for all platforms but the slowest. I run it regularly on a 500 MHz machine where generating the 9 images used in the "Current Conditions" page takes just under 2 seconds (compared with wview's 0.4 seconds).

Unfortunately, the architectural goal of one code base is likely to be broken with the arrival of Python V3.X. It has so many changes that are not backwards compatible with V2.X, that a separate code base will most likely be needed. My intention is to stick with the V2.X versions until V3.X is so widespread it cannot be ignored, then make a permanent switch. Given the slow adoption rate of V3.X this is unlikely to happen anytime soon. In any case, I doubt the transition will affect the average weewx user.

All writes to the databases are protected by transactions. You can kill the program at any time (either Control-C if run from the command line or "/etc/init.d/weewx stop" if a daemon) without fear of corrupting the databases.

The code makes ample use of exceptions to insure graceful recovery from problems such as network outages. It also monitors socket and console timeouts, restarting whatever it was working on several times before giving up. In the case of an unrecoverable console error (such as the console not responding at all), the program waits 60 seconds then restarts the program from the top.

Any "hard" exceptions, that is those that do not involve network and console timeouts and are most likely due to a logic error, are logged, reraised, and ultimately cause thread termination. If this happens in the main thread (not likely due to its simplicity), then this causes program termination. If it happens in the report processing thread (much more likely), then only the generation of reports will be affected — the main thread will continue downloading data off the instrument and putting them in the database. You can fix the problem at your leisure, without worrying about losing any data.

Terminology

This is a glossary of terminology used throughout the code.

packet Something obtained off the weather station. Frequently uses a complex internal encoding, so it requires some processing to be useful.
record Something obtained off the SQL database.
archive packet A packet obtained off the store on the weather station. For example, with a Davis VantagePro, it's obtained using their DMPAFT command.
loop packet A packet with the current observations. For example, with a Davis VantagePro, it's obtained using their LOOP command.
archive record A record obtained off the SQL database
tuple-time An instance of the Python object time.struct_time. This is a 9-wise tuple that represent a time. It could be in either local time or UTC, though usually the former. See module time for more information. They are useful because they are a little closer in format to what the Davis VantagePro uses, although they still require a bit of processing. Variables carrying tuple time usually have a suffix '_tt'.
epoch time Sometimes referred to as "unix time," or "unix epoch time." The number of seconds since the epoch, which is 1 Jan 1970 00:00:00 UTC. Hence, it always represents UTC (well.... after adding a few leap seconds. But, close enough). This is the time used on the sqlite archive and appears as type 'dateTime' in the SQL schema, perhaps an unfortunate name because of the similarity to the completely unrelated Python type 'datetime'. Very easy to manipulate, but it's an opaque big number.
time stamp A variable in unix epoch time. Always in UTC. Variables carrying a time stamp usually have a suffix '_ts'.
datetime An instance of the Python object datetime.datetime. Variables of type datetime usually have a suffix '_dt'.
SQL type A type that appears in the SQL database. This usually looks something like 'outTemp', 'barometer', 'extraTemp1', and so on.
observation type A type that can be used in the presentations. This is generally all of the SQL types, plus calculated data (such as rms or vecavg).
value tuple A 3-way tuple. First element is a value, second element the unit type the value is in, the third the unit group. An example would be (21.2, 'degree_C', 'group_temperature').

Units

In general, there are three different areas where the unit system makes a difference.:

  1. On the weather station. As far as I know, the Davis VantagePro series supports only U.S. Customary units internally.
  2. In the database. The unit system of any individual record is indicated by the "usUnits" field. The numerical value 1 indicates U.S. Customary, 2 indicates Metric.
  3. In the presentation (i.e., html and image files).

The general strategy is that measurements are stored internally in the native measurement system of the weather instrument. That is, the units used in #1 and #2 are always the same. With Version 1.5, weewx supports translations between #2 and #3, allowing any unit system to be used in the presentation layer.

It would be easy enough to support an instrument that uses metric units internally, although this has not been done yet.

Value "None"

The Python special value 'None' is used throughout to signal a missing data point. All functions expect it.

However, the time value must never be 'None'. This is because it is used as the primary key in the SQL database.

Time

Weewx stores all data in UTC (roughly, "Greenwich" or "Zulu") time. However, usually one is interested in weather events in local time and want image and HTML generation to reflect that. Furthermore, most weather stations are configured in local time. This requires that many data times be converted back and forth between UTC and local time. To avoid tripping up over time zones and daylight savings time, weewx generally uses Python routines to do this conversion. Nowhere in the code base is there any explicit recognition of DST. Instead, its presence is implicit in the conversions. At times, this can cause the code to be relatively inefficient.

For example, if one wanted to plot something every 3 hours in UTC time, it would be very simple: to get the next plot point, just add 10,800 to the epoch time:

next_ts = last_ts + 10800

But, if one wanted to plot something for every 3 hours in local time (that is, at 0000, 0300, 0600, etc.), despite a possible DST change in the middle, then things get a bit more complicated. One could modify the above to recognize whether a DST transition occurs sometime between last_ts and the next three hours and, if so, make the necessary adjustments. This is generally what wview does. Weewx takes a different approach and converts from UTC to local, does the arithmetic, then converts back. This is inefficient, but bulletproof against changes in DST algorithms, etc:

time_dt = datetime.datetime.fromtimestamp(last_ts)

delta = datetime.timedelta(seconds=10800)

next_dt = time_dt + delta

next_ts = int(time.mktime(next_dt.timetuple()))

Other time conversion problems are handled in a similar manner.