(c) 2009 by Tom Keffer <tkeffer@gmail.com>
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with
this program. If not, see
I wrote weewx over the winter of 2008-2009 for two reasons: it was a wet and miserable winter here in Oregon with not much else to do, so there was no good reason not to, and because I wanted a simple, easy-to-understand server to run my Davis VantagePro2 weather station on a Linux box. I had been using wview, which is a very high-performance, stable, and feature rich system authored by Michael Teele with lots of users. Written in C, it's an amazingly efficient system that can run on very underpowered boxes. In exchange, it's huge (55,000+ lines of code), tightly integrated in with its companion library, radlib (another 24,000+ lines), and so brittle that only Michael can reliably make modifications to it. But, if you're looking to be in good company and you want to run on inexpensive, featherweight machines such as the Linksys NSLU2, you can't beat it.
Having made a career in C++ and Java, I was also interested in some more modern languages, so I thought I'd try either Python or Ruby (although, truth be told, the roots of Python are nearly as old as C++!). I picked Python because its libraries are more mature and there are many more choices for third party libraries.
Weewx weighs in at about 7,000 lines of code, including its many comment lines although, to be fair, it is missing many features such as support for other weather stations, support for metric units, support for alarms, etc. On the other hand, it offers very powerful configuration and templating options, making it easy to customize. It is also architecturally very simple and easy to understand.
weewx can be downloaded from its SourceForge page: https://sourceforge.net/projects/weewx.
Python V2.5 or V2.6 is required. The newer V3.0 distribution will not work.
The following external packages are required to use weewx.
There are two general strategies for installing these prerequisites:
Option #1 is easier, but if your Linux distribution does not come with such tools, you may have to use easy_install. Brief instructions for both approaches are given below.
The instructions that follow are for using the Debian tool apt_get, but the same package names would be used should you chose to use a graphical interface such as the Synaptic Package Manager.
My Ubuntu 8.10 system came with V3.5.9, which works just fine. However, if you need to install:
sudo apt-get install sqlite3
Easily installed:
sudo apt-get install python-pysqlite2
Easily installed:
sudo apt-get install python-configobj
Easily installed:
sudo apt-get install python-serial
Easily installed:
sudo apt-get install python-cheetah
My version of Python came with V1.1.6, which works great. Nothing needs to be done.
My SuSE 11.1 system came with some of the prerequisites installed, some available through yast, and three that required easy_install. To start, you will have to install the gcc compiler:
sudo yast -i gcc
Then install easy_install:
sudo yast -i python-setuptools
On my system, some scripts wanted to install themselves into /usr/local/lib/python2.6/site-packages, which didn't exist. If this is the case, you may have to create these directories before running easy_install:
sudo mkdir /usr/local/lib/python2.6
sudo mkdir /usr/local/lib/python2.6/site-packages
My SuSE 11.1 system came with V3.6.4, which works just fine. However, if you need to install:
sudo yast -i sqlite3
Install using easy_install. See comments below about installing pysqlite using easy_install. On my SuSE 11.1 system, I had to install the gcc compiler and the sqlite3 development environment first:
sudo yast -i gcc
sudo yast -i sqlite-devel
Then I was able to install pysqlite using easy_install. However, because the hosting site for pysqlite had changed recently, I had to give the URL explicitly:
sudo easy_install http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz
Install using easy_install:
sudo easy_install configobj
Install using yast:
sudo yast-i python-serial
Install using easy_install (My system emitted a bunch of, apparently, benign warnings):
sudo easy_install Cheetah
Install using yast
sudo yast -i python-imaging
An alternative approach to installing the required packages is by using the Python setup tool "easy_install", part of the python-setuptools package. Refer to their instructions on how to install this tool.
Once installed, installing the rest of the packages is very easy.
My Ubuntu 8.10 system came with V3.5.9, which works just fine. If you do not have sqlite3, refer to the sqlite webpage for installation instructions.
While Version 2.3.X of pysqlite is included with many versions of Python, the more recent 2.5.X or greater is required in order to take advantage of transaction contexts. Hence, you may have to install or upgrade. Because pysqlite builds a C library, you may have to install the Python development environment first, if you have not already done so. Generally, this means installing the gcc compiler. You may also have to install the sqlite3 development environment as well.
With the development environment in place, you can easily build and install pysqlite:
easy_install pysqlite
If your system already has a version of pysqlite installed, but it is not a high enough version (easy_install will tell you the version you have), then you may have to force an upgrade:
easy_install --upgrade pysqlite
Note that at the time of this writing (24 Oct 2009), the hosting site for pysqlite has changed to one on googlecode, and easy_install could not find it. You may have to find and give the URL explicitly to easy_install (adjust version numbers as necessary):
easy_install http://pysqlite.googlecode.com/files/pysqlite-2.5.5.tar.gz
Easily installed:
easy_install configobj
easy_install pyserial
easy_install Cheetah
My version of Python came with V1.1.6, which works great.
I run weewx on a 500MHz system with an AMD Geode processor and 512 MB of memory. Configured this way, it consumes about 5% of the CPU and about 75MB of total memory.
At this point, only the Davis VantagePro2 is supported, and even then, only the "Revision B" version (firmware dated on or after 22 April 2002). It would be very easy to port to a "Revision A" station or even the original VantagePro, but I don't have access to the hardware to test it.
Because weewx is "Pure Python", that is it is 100% Python with no "C" modules to compile, installing it is very easy. Furthermore, it uses the standard Python distutils install method, which is very easy and flexible. Detailed instructions follow.
Start by unpack the tar ball (substitute your version for X.Y.Z) into any convenient directory where you have write permission
tar xvf weewx-X.Y.Z
Then change directory into it:
cd weewx-X.Y.Z
Next step is figuring out where you want to install weewx. If WEEWX_ROOT symbolizes the root location of the weewx directory hierarchy, then
By default, the location for WEEWX_ROOT is /home/weewx. However, it can be changed by editing the file setup.cfg. If you wish to install someplace else, open up setup.cfg and change the line
home = /home/weewx
to reflect your decision.
Build the distribution
./setup.py build
(Because weewx is pure Python this doesn't actually build anything, but it does arrange files for the final installation)
Then install it. If you have write permission in the directory where weewx will go (i.e., $WEEWX_ROOT), then type
./setup.py install
Otherwise, if you do not have write permission, you will have to use sudo:
sudo ./setup.py install
If you are upgrading from a previous version of weewx, the install process will
Strictly speaking it is not necessary to install or run weewx with root privileges. You only need read/write access to the serial port for your hardware. For example, if your hardware has a USB interface, on Ubuntu and SuSE:
sudo chmod a+rw /dev/ttyUSB0
Once done, if you edit setup.cfg to install into a directory where you have write permissions, you can install and run weewx without any root privileges at all.
Because weewx is pure Python, it actually does not have to be "built" and "installed" at all! You can just simply run it out of whatever directory you unpack it into (after, of course, editing weewx.conf to reflect your local environment). I do this all the time when testing. However, the setup.py script does include special provisions for updating your configuration file weewx.conf, which can be handy when upgrading to a later version.
This section covers configuring your archive and statistical database (if necessary; this step is required only if you are moving from wview to weewx), configuring your weather station, and configuring the configuration file weewx.conf.
In the following, $WEEWX_ROOT refers to the weewx root directory, generally /home/weewx.
This section is necessary only if you are moving from wview to weewx and wish to transfer your old data over. If you are starting afresh, you do not need to follow this section --- the two main databases are created and populated automatically by weewx.
Two databases are maintained by weewx:
Because wview and weewx use identical schema for the first of these (the archive database), it can be just copied over. However, the second (the statistical databases) are different --- the weewx statistical database must be built manually and backfilled. This is done using the configuration script configure.py.
Here's a summary of how to transfer your wview data to weewx.
mkdir $WEEWX_ROOT/archive
cp /usr/local/var/wview/archive/wview-archive.sdb $WEEWX_ROOT/archive/weewx.sdb
$WEEWX_ROOT/bin/configure.py --create-stats $WEEWX_ROOT/weewx.conf
$WEEWX_ROOT/bin/configure.py --backfill-stats $WEEWX_ROOT/weewx.conf
If your existing database is large, backfilling could take some time. On my modest 500 MHz fit-PC with 512 MB of memory it took a little over 4 minutes for a year and a half (25 MB) of data (while wview was running in the background).
The only two variables weewx tries to manage on the VantagePro are the time and the archive interval.
The time on the VP is automatically synchronized with the weewx server every four hours. However, you should run a NTP daemon on your server to insure that it is synchronized with the correct time. Doing so will greatly reduce errors, especially if you send data to services such as the Weather Underground.
The archive interval is set in the main configuration file $WEEWX_ROOT/weewx.conf. Look for the entry archive_interval in the VantagePro section. Set it to the number of seconds. Valid entries are 60, 300, 600, 900, 1800, 3600, and 7200. However, if you are ftp'ing lots of files to a server, setting it to 60 seconds may not give enough time to have them all uploaded before the next archive record is due. If this is the case, you should pick an archive interval of at least 300 seconds, or trim the number of files you are using.
After setting to the desired interval, run the configure.py script to set it on the VantagePro. If it differs from the old archive interval, the main memory log of the VantagePro will be cleared.
$WEEWX_ROOT/bin/configure.py --configure-VantagePro $WEEWX_ROOT/weewx.conf
Virtually every conceivable configuration option is in the configuration file $WEEWX_ROOT/weewx.conf. Most of the important ones are up near the top of the file. They are all documented in this section, although you can safely ignore most of them. The truly important ones, the ones you are likely to have to customize for your station, are shown in bold face and in blue.
Default values are provided for many of them, meaning that if they are not listed in the configuration file at all, weewx will pick sensible values. When the documentation below gives a "default value" this is what it means. However, all options have been given values in the configuration file that ships with weewx, so you can see what they look like. The value given in this shipped configuration file is not necessarily the same as the "default value".
What follows is organized by the different sections of the configuration file.
The options declared at the top are not actually part of any section. There are two:
Set to 1 to have the program perform extra debug checks, as well as emit extra information on the log file. Otherwise, set to 0. Default is 0 (no debug).
Set to how long to wait before declaring a socket time out. This is used when FTP'ing data to a web server or sending data to the Weather Underground. Twenty (20) seconds is reasonable. Default is 20.
This section covers options relating to the entire weather station setup.
Set to the root directory of the weewx file hierarchy for this station, nominally '/home/weewx'. This value will be set automatically by the setup script setup.py to reflect the choice you made in the configuration file setup.cfg. Required. No default.
The station location should be a string that describes the geography of where you weather station is located, such as 'Hood River, Oregon'. Required. No default.
The lat/lon should be set in decimal degrees, negative for southern and eastern hemispheres, respectively. Required. No default.
Should be set to the altitude of the station. In this version, the only unit accepted is feet. Required. No default.
If your area uses a rain year that starts on something other than the first of January, you may want to set this variable. For example, set to 10 if your rain year starts in October (as mine does). Default is 1.
This variable is available in the HTML templates. Set it to an appropriate URL to display a radar image for your area. No default.
Set to 1 (one) to cache LOOP data, otherwise, set to zero. By default, weewx updates the statistical database with every LOOP update. This can be as often as every two seconds. If you are using a solid-state disk as your permanent store, this can result in a lot of wear on the device. By setting cache_loop_data to 1, the LOOP data will be cached, and the stats database will be updated only when new archive data is due. This results in many fewer writes to the database, but possible loss of data should your server go down between archive updates. (The only data actually lost would be any new High/Lows found during the archive period.) The default is 0 (no caching).
Set to the base temperature for calculating heating and cooling degree-days, respectively. The default is 65.0 for both.
Set to suitable abbreviations for the four hemispheres. Default is "N", "S", "E", "W"
Set to the type of hardware you are using. For this version, only 'VantagePro' is accepted. Required.
This section is for options relating to the VantagePro hardware.
Set to the port name used by your station. Example, /dev/ttyUSB0 is a common location for USB ports under Debian, /dev/ttyS0 for serial ports. Required. No default.
Set to the baudrate of your station. The default is 19200.
Set to the desired archive interval of your station, in seconds. This variable is only used when setting up your station. Otherwise, this value is read directly from the station. Required if you configure your station. No default.
Set to the ID number of your Integrated Sensor Suite (ISS). This is used in the formula to calculate reception quality for wireless stations. The default is 1.
How long to wait in seconds after the top of an archiving interval before fetching new data off the station. For example, if your archive interval is 5 minutes and archive_delay is set to 15, then the data will be fetched at 00:00:15, 00:05:15, 00:10:15, etc. This delay is to give the station a few seconds to archive the data internally, and in case your server has any other tasks to do at the top of the minute. Default is 15 seconds.
How many seconds to wait for a response from the station before giving up. Default is 5 seconds.
How many seconds to wait before retrying again. Unless you have a good reason to, this value should not be changed from the default, as it is long enough for the station to offer new data, but not so long as to go into a new loop packet (which arrive every 2 seconds). Default is 1.2 seconds.
How many times to try again before giving up. Default is 4.
How often to check the VantagePro's onboard clock for drift, in seconds. Default is 14400 (every 4 hours)
The maximum amount of drift to tolerate, in seconds, in the VantagePro's onboard clock, before resetting the clock. Default is 5.
What unit system is in use on your weather station hardware. Possible values are '1' (Imperial) or '2' (Metric). As far as I know, all Davis instruments support only Imperial. In any case, Imperial is the only system supported by weewx. Default is 1.
If you FTP your images and HTML files to an external web server, weewx can FTP them for you. It does an incremental update, that is, it only FTPs any files that have changed, saving outgoing bandwidth with your Internet connection.
If you do not use such a server, comment out the entire section, including the '[FTP]' heading.
user
Set to the username you use for your FTP connection to your web server. Required. No default.
password
Set to the password you use for your FTP connection to your web server. Required. No default.
server
Set to the name of your web server (e.g., www.threefools.org, in my case). Required. No default
path
Set to the path where the weather data will be stored on your webserver (e.g., 'weather'). Required. No default.
Set to 1 if you wish to use the more modern, FTP passive mode, 0 if you wish to use active mode. Passive mode generally works better through firewalls, but not all FTP servers do a good job of supporting it. See Active FTP vs. Passive FTP, a Definitive Explanation for a good explanation of the difference. Default is 1 (passive mode).
Weewx will try up to this many times to FTP a file up to your server before giving up. Default is 3.
Weewx can send your current data to the Weather Underground. If you do not wish to do this, comment out the entire section, including the '[Wunderground]' heading.
Set to your Weather Underground station ID (e.g., KORHOODR3). Required.
Set to your Weather Underground password. Required.
This section is for configuring the sqlite3 database on which the station archive data is stored.
The path, relative to the WEEWX_ROOT directory, to the database. Required
What unit system to use inside the database. Required. The only one supported right now is '1', the Imperial (U.S.) system
This section is for configuring the sqlite3 database on which the station statistics are stored.
The path, relative to the WEEWX_ROOT directory to the statistical database. Required.
The list of types for which statistics will be kept. Types not listed will not be available for generating HTML pages. Optional. The default is all types, resulting in a possibly much bigger than necessary stats database (do you really have four different soil moisture sensors?) The list that ships with the configuration file will work for most stations and probably will not have to be modified.
This section, which controls which images (plots) get generated and with which options, is by far the most complicated. However, it is extremely flexible and powerful.
It consists of one or more sub-sections, one for each time period (day, week, month, and year). These sub-sections define the nature of aggregation and plot types for the time period. For example, here's a typical set of options for sub-section [[month_images]], controlling how images that cover a month period are generated:
[[month_images]]
x_label_format = %d
bottom_label_format = %m/%d/%y %H:%M
time_length = 2592000 # == 30 days
aggregate_type = avg
aggregate_interval = 10800 # == 3 hours
The option x_label_format gives a strftime() type format for the x-axis. In this example, it will only show days (format option "%d"). The bottom_label_format is the format used to time stamp the image at the bottom. In this example, it will show the time as 10/25/09 15:35. A plot will cover a nominal 30 days, and all items included in it will use an aggregate type of averaging over 3 hours.
Within each sub-section is one final nesting (!), one for each image to be generated. The title of each sub-sub-section is the filename to be used for the image. Under this is each of the SQL types to be included in the image. Values specified in the level above can be overridden. For example, here's a typical set of options for sub-sub-section [[[monthrain]]]:
[[[monthrain]]]
[[[[rain]]]]
plot_type = bar
aggregate_type = sum
aggregate_interval = 86400
label = Rain (daily avg)
This will generate an image file with name monthrain.png. In it will be a bar plot showing SQL type 'rain'. The aggregation type will be summing (overriding the averaging specified in sub-section [[month_images]], so you get the total rain over the aggregate period rather than the average) over an aggregation interval of 86,400 seconds (one day). It will also include the indicated label.
More than one SQL type can be included in a plot. For example, here's how to generate a plot with the week's outside temperature as well as dewpoint:
[[[monthtempdew]]]
[[[[outTemp]]]]
[[[[dewpoint]]]]
This would create an image in file monthtempdew.png that includes a line plot of both outside temperature and dewpoint.
Studying this section in the shipped version of weewx.conf will give you ideas about the many different image plot configurations that are possible without hacking the code.
This section controls how images are labeled. It consists of three sub-sections:
This sub-sections specifies default labels to be used for each SQL type. For example, options
inTemp = Inside Temperature
outTemp = Outside Temperature
would cause the given labels to be used for plots involving SQL types inTemp and outTemp..
This sub-section is used to specify what format to be used for y-axis labels in image plots that use Imperial (U.S.) units. It is also used for unit labels in HTML file generation. For example, the options
outTemp = %.1f
rain = %.2f
would cause the given formats to be used when formatting outside temperature and rain axes, respectively. The formatting codes are those used by Python, and are very similar to C's sprintf() codes.
This sub-section specifies what unit labels to be used for the y-axis in image plots that use Imperial (U.S.) units. For example, the options
outTemp = \xb0F
rain = ' in'
would cause outside temperature to have unit labels '°F' and rain to have labels ' in'. (NB: the code \xb0 is the hexadecimal value b0, which in many encodings encodes the degree sign.)
Section [HTML] has two options and two sub-sections. For additional information on HTML generation see the section below.
This option specifies the directory, relative to WEEWX_ROOT, where the HTML templates can be found. Required. No default.
This option specifies the directory, relative to WEEWX_ROOT, where the generated HTML files should be put. Required. No default.
This subsection is similar to its eponymous counterpart in section [Labels] above, except it is used for HTML generation. It is useful to have a separate section because HTML uses special 'entity' codes to code special characters, such as the degree sign. For example, the options
outTemp = °F
rain = ' in'
would cause outside temperature and rain to have unit labels '°F' and ' in', respectively.
This subsection is used for time labels in HTML generation. It uses strftime() formats. For example
week = %H:%M on %A
month = %d-%b-%Y %H:%M
would specify that week data should use a format such as "15:20 on Sunday", while month data should look like "06-Oct-2009 15:20"
Weewx can be run either from the command line (useful for diagnostic purposes because it will print out a summary of every LOOP data), or as a daemon. When first trying weewx, it's probably best to run it from the command line because you will be able to see command line diagnostics, as well as log messages.
Weewx can easily be run from the command line. Start by making sure you have appropriate permissions to the serial port your weather station uses. For example, if you are using a plain old serial port:
sudo chmod 666 /dev/ttyS0
Then run the main loop program, weewxd.py, giving the configuration file as its only parameter:
$WEEWX_ROOT/bin/weewxd.py $WEEWX_ROOT/weewx.conf
It should start by downloading any archive data from your weather station into the database $WEEWX_ROOT/archive/weewx.sdb. As the Davis VantagePro can store a couple thousand archive records internally, this could take a minute or two. I've found this process particularly slow on SuSE for some reason.
Weewx will then start monitoring LOOP data, printing a short version of the received data on standard output, about once every two seconds.
First, select the appropriate run script. They can be found under $WEEWX_ROOT/start_script.
| SuSE: | $WEEWX_ROOT/start_script/SuSE/weewx |
| Debian/Ubuntu: | $WEEWX_ROOT/start_script/Debian/weewx |
Check the chosen script to make sure the variable WEEWX_ROOT inside has been set to the proper root directory for your weewx installation (it should have been set to the correct value automatically by the install process, but it's worth checking).
Copy it to the proper location for your system:
| SuSE: | cp $WEEWX_ROOT/start_script/SuSE/weewx /etc/init.d |
| Debian/Ubuntu: | cp $WEEWX_ROOT/start_script/Debian/weewx /etc/init.d |
Make sure the script is executable
| SuSE: | chmod +x /etc/init.d/weewx |
| Debian/Ubuntu: | chmod +x /etc/init.d/weewx |
Create symbolic links in the run level directories:
| SuSE: | /usr/lib/lsb/install_initd /etc/init.d/weewx |
| Debian/Ubuntu: | update-rc.d weewx defaults 98 |
Weewx will now start automatically whenever your system is booted. You can also manually start, stop, and restart the weewx daemon:
/etc/init.d/weewx start
/etc/init.d/weewx stop
/etc/init.d/weewx restart
By default, the scripts are designed to have weewx run at run levels 2, 3, 4 and 5. Incidentally, a nice tool for setting run levels with Debian (Ubuntu) systems is sysv-rc-conf. It uses a curses interface to allow you to change easily which run level any of your daemons runs at. There is a similar tool on SuSE. From the start menu run the YAST Control Center, then look for Systesm Services (Runlevel). Pick "Expert" mode to see the run levels.
The sqlite3 archive database used by weewx (nominally, weewx.sdb) is completely compatible with the database used by wview (usually called wview-archive.sdb), at least as of Version 5.2.X. The schema and its semantics is identical. However, the statistical file stats.sdb is different, and must be rebuilt
HTML generation is done using the Cheetah templating engine. This is a very powerful engine, which essentially lets you have the full semantics of Python available in your templates. As this would make the templates incomprehensible to anyone but a Python programmer, weewx adopts a very small subset of its power.
Generally, any value is specified by using a 'dot' code. For example:
$month.outTemp.max
$month.outTemp.maxtime
$current.outTemp
would code the max outside temperature for the month, the time it occurred, and the current outside temperature, respectively. So, an HTML file that looks like
<html>
<head>
<title>Current conditions</title>
</head>
<body>
<p>Current temperature = $current.outTemp</p>
<p>Max for the month is $month.outTemp.max, which occurred at $month.outTemp.maxtime</p>
</body>
</html>
would be all you need for a very simple HTML page that would display the text:
Current temperature = 51.0°F
Max for the month is 68.8°F, which occurred at 07-Oct-2009 15:15
The format that was used to format the temperature (51.0) is specified in section [Labels][[ImperialFormats]]. The unit label °F is from section [HTML][[ImperialUnits]], while the time format is from [HTML][[Time]].
The "dot" code has up to three parts.
The following types are available to be used in your template (assuming your station supports them and you have specified that it be stored in your stats database. See section stats_types in the weewx.conf configuration file).
| Type | min | mintime | max | maxtime | avg | sum | rms | vecavg | vecdir |
| barometer | X | X | X | X | X | ||||
| inTemp | X | X | X | X | X | ||||
| outTemp | X | X | X | X | X | ||||
| inHumidity | X | X | X | X | X | ||||
| outHumidity | X | X | X | X | X | ||||
| wind | X | X | X | X | X | X | X | X | |
| rain | X | X | X | X | X | X | |||
| dewpoint | X | X | X | X | X | ||||
| windchill | X | X | X | X | X | ||||
| heatindex | X | X | X | X | X | ||||
| heatdeg | X | ||||||||
| cooldeg | X | ||||||||
| ET | X | X | X | X | X | ||||
| radiation | X | X | X | X | X | ||||
| UV | X | X | X | X | X | ||||
| extraTemp1 extraTemp2 extraTemp3 |
X | X | X | X | X | ||||
| soilTemp1 soilTemp2 soilTemp3 |
X | X | X | X | X | ||||
| leafTemp1 leafTemp2 |
X | X | X | X | X | ||||
| extraHumid1 extraHumid2 |
X | X | X | X | X | ||||
| soilMoist1 soilMoist2 soilMoist3 soilMoist4 |
X | X | X | X | X | ||||
| leafWet1 leafWet2 |
X | X | X | X | X | ||||
| rxCheckPercent | X | X | X | X | X |
Weewx logs many events to the system log. On Debian systems, this is /var/log/syslog, on SuSE, /var/log/messages. Your system may use yet another place. When troubleshooting the system, be sure to check it!
Setting the option debug in weewx.conf to 1 (one) will generate many more checks and output and can be useful for debugging.
The primary goals of weewx are:
To meet these goals, the following strategies were used:
While weewx is nowhere near as fast at generating images and HTML as its predecessor, wview (this is partially because it uses fancier fonts and a way more powerful templating engine), it is 'fast enough' for all platforms but the slowest. I run it regularly on a 500 MHz machine where generating the 9 images used in the "Current Conditions" page takes just under 2 seconds. Compare this with wview's 0.4 seconds.
Unfortunately, the architectural goal of one code base is likely to be broken with the arrival of Python V3.X. It has so many changes that are not backwards compatible with V2.X, that a separate code base will most likely be needed. My intention is to stick with the V2.5 and V2.6 versions until V3.X is so widespread it cannot be ignored, then make a permanent switch. I doubt this will affect the average weewx user.
Three threads are used within weewx:
Once finished with these startup chores, the main thread then puts the VantagePro in LOOP mode. In this mode, the VP offers up data every 2 seconds, sleeping in between. This is a very energy saving mode. The thread monitors the port and when new data is available, it adds it to the running statistical tally, in particular highs, lows, and wind rms data, kept in the stats database. While in LOOP mode, very little processing is done so it doesn't miss any updates.
When an archive interval is due, typically every 5 minutes or so, then the main thread cancels the LOOP mode, and then downloads the new archive data, putting it in the main database. It also uses the archive data to update averages in the stats database. It puts the new record in a Queue to be sent to the Weather Underground. It then creates and starts thread #3 to do any processing of the data.
Thread #2 interacts with the Weather Underground. It is also active the entire lifetime of the program. It monitors a Queue. When new data appears in the Queue, this thread forms the necessary URL to send it to the WU and then sends it. It then goes back to monitoring the queue. This way, posting data on the WU can happen asynchronously with other processing.
Thread #3 is responsible for generating HTML files, images, and NOAA monthly and yearly reports. Its lifetime is only as long as it takes to process a new archive record and then it dies. Because most of the tricky processing happens in this thread, this is the most likely place where an exception could occur. However, should this happen, it will only affect this short-lived thread, and not the much longer-lived main thread. Hopefully, this means that at least no data will be missed.
All writes to the databases are protected by transactions. You can kill the program at any time (either Control-C if run from the command line or "/etc/init.d/weewx stop" if a daemon) without fear of corrupting the databases.
The code makes ample use of exceptions to insure graceful recovery from problems such as network outages. It also monitors socket and console timeouts, restarting whatever it was working on several times before giving up. In the case of an unrecoverable console error (such as the console not responding at all), the program waits 60 seconds then restarts the program from the top.
Any "hard" exceptions, that is those that do not involve network and console timeouts and are most likely due to a logic error, are logged, reraised, and ultimately cause thread termination. If this happens in the main thread (not likely and hasn't happened to me yet), then this causes program termination.
This is a glossary of terminology used throughout the code.
| packet | Something obtained off the weather station. Frequently uses a complex internal encoding, so it requires some processing to be useful. |
| record | Something obtained off the SQL database. |
| archive packet | A packet obtained off the store on the weather station. For example, with a Davis VantagePro, it's obtained using their DMPAFT command. |
| loop packet | A packet with the current observations. For example, with a Davis VantagePro, it's obtained using their LOOP command. |
| archive record | A record obtained off the SQL database |
| tuple-time | An instance of the Python object time.struct_time. This is a 9-wise tuple that represent a time. It could be in either local time or UTC, though usually the former. See module time for more information. They are useful because they are a little closer in format to what the Davis VantagePro uses, although they still require a bit of processing. Variables carrying tuple time usually have a suffix '_tt'. |
| epoch time | Sometimes referred to as "unix time," or "unix epoch time." The number of seconds since the epoch, which is 1 Jan 1970 00:00:00 UTC. Hence, it always represents UTC (well.... after adding a few leap seconds. But, close enough). This is the time used on the sqlite archive and appears as type 'dateTime' in the SQL schema, perhaps an unfortunate name because of the similarity to the Python type 'datetime'. Very easy to manipulate, but it's an opaque big number. |
| time stamp | A variable in unix epoch time. Always in UTC. Variables carrying a time stamp usually have a suffix '_ts'. |
| datetime | An instance of the Python object datetime.datetime. Variables of type datetime usually have a suffix '_dt'. |
This package is imperial (U.S.) units only. However, it has been set up to make it easy to extend to metric.
In general, there are three different areas where the unit system makes a difference.:
The transition from 1 to 2, i.e., from data in the VantagePro to the database, is handled by a translation function. Right now, only one is supplied, weewx.VantagePro.translateArchiveToImperial. Others could easily be introduced.
The Python special value 'None' is used throughout to signal a missing data point. All functions expect it.
However, the time value must never be 'None'. This is because it is used as the primary key in the SQL database.
Weewx stores all data in UTC (roughly, "Greenwich" or "Zulu") time. However, usually one is interested in weather events in local time and want image and HTML generation to reflect that. Furthermore, most weather stations are configured in local time. This requires that many data times be converted back and forth between UTC and local time. To avoid tripping up over time zones and daylight savings time, weeewx generally uses Python routines to do this conversion. Nowhere in the code base is there any explicit recognition of DST. Instead, its presence is implicit in the conversions. At times, this can cause the code to be relatively inefficient.
For example, if one wanted to plot something every 3 hours in UTC time, it would be very simple: to get the next plot point, just add 10,800 to the epoch time:
next_ts = last_ts + 10800
But, if one wanted to plot something for every 3 hours in local time (that is, at 0000, 0300, 0600, etc.), despite a possible DST change in the middle, one could modify the above to recognize whether a DST transition occurs sometime between last_ts and the next three hours and, if so, make the necessary adjustments. This is generally what wview does. Weewx takes a different approach and converts from UTC to local, does the arithmetic, then converts back. This is inefficient, but bulletproof against changes in DST algorithms, etc:
time_dt = datetime.datetime.fromtimestamp(last_ts)
delta = datetime.timedelta(seconds=10800)
next_dt = time_dt + delta
next_ts = int(time.mktime(next_dt.timetuple()))
Other time conversion problems are handled in a similar manner.
Three different configurations were used to test weewx V1.1.X. Here are the package versions used in each configuration:
| Package | Configuration 1 | Configuration 2 | Configuration 3 |
| Python | V2.5.2 | V2.5.4 | V2.6.2 |
| sqlite3 | V3.5.9 | V3.6.10 | V3.6.10 |
| pysqlite | V2.5.5 | V2.5.5 | V2.5.0 |
| configobj | V4.6.0 | V4.6.0 | V4.6.0 |
| pyserial | V1.35 | V1.35 | V1.35 |
| Cheetah | V2.0.1 | V2.0.1 | V2.2.2 |
| Python Imaging Library (PIL) |
V1.1.6 | V1.1.6 | V1.1.6 |