This document covers the customization of weewx. It assumes that you have read and are reasonably familiar with the Users Guide.
It starts with an overview of the architecture of weewx. If you are only interested in customizing the generated reports you can probably skip the overview and proceed directly to the section The Standard skin configuration file. With this approach you can easily add new plot images, change the titles of images, change the units used in the reports, and so on.
However, if your goal is a specialized application, such as adding alarms, RSS feeds, etc., then it would be worth your while to read about the internal architecture and how to customize it.
Most of the guide will cover any weather hardware, but the exact data types are specific to the Davis Vantage series. Unless you are using an unusual type you are unlikely to run into trouble.
Warning!
weewx is still an experimental system and, as
such, its internal design is subject to change. Be prepared to do
updates to any code or customization you do!
At a high-level, weewx consists of an engine class called StdEngine. It is responsible for loading "services", then arranging for them to be called when key events occur, such as the arrival of LOOP data. The default install of weewx includes the following services:
Service | Function |
weewx.wxengine.StdTimeSynch | Arrange to have the clock on the station synchronized at regular intervals. |
weewx.wxengine.StdConvert | Converts the units of the input to a target unit system (such as US or Metric). |
weewx.wxengine.StdCalibrate | Adjust new LOOP and archive packets using calibration expressions. |
weewx.wxengine.StdQC | Check quality of incoming data, making sure values fall within a specified range. |
weewx.wxengine.StdArchive | Archive any new data to the SQL databases. |
weewx.restx.StdStationRegistry weewx.restx.StdWunderground weewx.restx.StdPWSweather weewx.restx.StdCWOP weewx.restx.StdWOW weewx.restx.StdAWEKAS |
Various RESTful services (simple stateless client-server protocols), such as the Weather Underground, CWOP, etc. Each launches its own, independent thread, which manages the post. |
weewx.wxengine.StdPrint | Print out new LOOP and archive packets on the console. |
weewx.wxengine.StdReport | Launch a new thread to do report processing after a new archive record arrives. Reports do things such as generate HTML files, generate images, or FTP/rsync files to a web server. New reports can be added easily by the user. |
It is easy to extend old services or to add new ones. The source distribution includes an example new service called "MyAlarm," which sends an email when an arbitrary expression evaluates True. These advanced topics are covered later in the section Customizing the weewx service engine.
For the moment, let us focus on the last service, weewx.wxengine.StdReport, the standard service for creating reports. This will be what most users will want to customize even if it means just changing a few options.
The Standard Report Service runs zero or more Reports. The specific reports which get run are set in the configuration file weewx.conf, in section [StdReport].
The default distribution of weewx includes three reports:
Report | Default functionality |
StandardReport | Generates day, week, month and year "to-date" summaries in HTML, as well as the plot images to go along with them. Also generates NOAA monthly and yearly summaries. |
FTP | Arranges to upload everything in the $HTML_ROOT directory up to a remote webserver. |
RSYNC | Like FTP, but uses rsync for transferring files to a remote webserver. |
Note that the FTP and RSYNC "reports" are a funny kind of report in that it they do not actually generate anything. Instead, they use the reporting service engine to arrange for things to be transferred to a remote server.
Each report has a skin associated with it. For most reports, the relationship with the skin is an obvious one: it contains the templates, any auxiliary files such as background GIFs or CSS style sheets, and a skin configuration file, skin.conf. If you will, the skin controls the look and feel of the report. Note that more than one report can use the same skin. For example, you might want to run a report that uses US Customary units, then run another report against the same skin, but using metric units and put the results in a different place. All this is possible by either overriding configuration options in the weewx configuration file weewx.conf or the skin configuration file skin.conf.
Like all reports, the FTP and RSYNC "Reports" also use a skin, and include a skin configuration file, although they are quite minimal.
Skins live in their own directory located in $SKIN_ROOT.
A template is a text file that is processed by weewx to create a new file. A template may be used to generate HTML, XML, CSV, javascript, or any other type of text file. A template typically contains variables that are replaced by when creating the new file. Templates may also contain programming logic.
Each template file lives in the skin directory of the skin that uses it. By convention, a template file ends with the .tmpl extension.
To create their output, skins rely on one or more Generators that actually create useful things such as HTML files or plot images. Generators can also copy files around or FTP/rsync them to remote locations. The default install of weewx includes the following generators:
Generator | Function |
weewx.cheetahgenerator.CheetahGenerator | Generates files from templates, using the Cheetah template engine. Used to generate HTML and text files. |
weewx.imagegenerator.ImageGenerator | Generates graph plots. |
weewx.reportengine.FtpGenerator | Uploads data to a remote server using FTP. |
weewx.reportengine.RsyncGenerator | Uploads data to a remote server using rsync. |
weewx.reportengine.CopyGenerator | Copies files locally. |
Note that the three generators FtpGenerator, RsyncGenerator, and CopyGenerator do not actually generate anything having to do with the presentation layer. Instead, they just move files around.
Which generators are to be run for a given skin is specified in the skin's configuration file skin.conf, in section [Generators].
There are two databases used by weewx, which can be implemented either by using SQLITE3, an open-source, lightweight SQL database, or MySQL, an open-source, full-featured database server, or some combination of the two of them.
How these abstract databases are bound to the real database is covered in the Weewx User's Guide, in section '[StdArchive]'.
The important thing to remember is that the archive database contains a record for every archive interval and, as such, represents the current conditions at the time of the observation. By contrast, the statistical database represents the aggregation of conditions over a day. That is, it contains the daily minimum, maximum, and the time of the minimum and maximum, for each observation type. As you can imagine, the statistical database is much smaller because it represents only a summary of the data.
The archive database is used for both generating plot data and in template generation (where it appears as tag $current). The statistical database is used only in template generation (where it appears as tags $day, $week, $month, $year, and $rainyear, depending on the aggregation time period).
For configuration changes, simply modify the weewx configuration file weewx.conf, and possibly modify the skin configuration file skin.conf as described later in this document. These files will be preserved when you upgrade.
Other customizations require new Python code or modifications of example code. Where should you put the code? If you simply modify the examples in place, then your changes will be overwritten the next time you do an upgrade.
A better idea is to put the code in the $BIN_ROOT/user, directory. For example, copy example code from the examples directory to the user directory, then modify it there. If your modification does not contain much code, consider putting it in the file extensions.py in the user directory. Because the user directory is preserved through upgrades, you won't have to redo any changes you might have made.
If you make changes, how do you know what the results will look like? You could just run weewx and wait until the next reporting cycle kicks off but, depending on your archive interval, that could be a 30 minute wait or more.
The tool wee_reports allows you to run a report whenever you want to. To use it, just run it from a command line, with the location of your configuration file weewx.conf as the first argument. Optionally, if you include a unix epoch timestamp as a second argument, then the report will use that as the "Current" time; otherwise, the time of the last record in the archive database will be used. Here is an example, using 1 May 2014 00:00 PDT as the "Current" time. NB: this example assumes you used the setup.py install method. Adjust paths as necessary if you used another method.
cd /home/weewx ./bin/wee_reports weewx.conf 1398927600
This section discusses the two general strategies for customizing reports: by changing options in one or more configuration file, or by changing the template files. The former is generally easier, but occasionally the latter is necessary.
Changing an option means either modifying the main configuration file weewx.conf, or the skin configuration file skin.conf.
Each skin will have a skin.conf that defines its default configuration. The examples in this guide refer to the standard skin that comes with the distribution.
With this approach, edit the skin configuration file with a text editor. Changes made in this way will be used by weewx the next time it generates reports, which is typically the next archive interval; there is no need to restart weewx to see the results of the changes.
For the standard skin that comes with weewx, the file is $SKIN_ROOT/Standard/skin.conf.
For example, suppose you wish to use metric units in the presentation layer, instead of the default US Customary Units. The section that controls units is [Units][[Groups]]. It looks like this:
[Units] [[Groups]] group_altitude = foot group_degree_day = degree_F_day group_direction = degree_compass group_moisture = centibar group_percent = percent group_pressure = inHg group_radiation = watt_per_meter_squared group_rain = inch group_rainrate = inch_per_hour group_speed = mile_per_second group_speed2 = mile_per_second2 group_temperature = degree_F group_uv = uv_index group_volt = volt
To use metric units, you would edit this section to read:
[Units] [[Groups]] group_altitude = meter group_degree_day = degree_C_day group_direction = degree_compass group_moisture = centibar group_percent = percent group_pressure = mbar group_radiation = watt_per_meter_squared group_rain = mm group_rainrate = mm_per_hour group_speed = meter_per_second group_speed2 = meter_per_second2 group_temperature = degree_C group_uv = uv_index group_volt = volt
The options that were changed have been highlighted . Details of the various unit options are given in the Appendix Units.
Other options are available, such as changing the text label for various observation types. For example, suppose your weather console is actually located in a barn, not indoors, and you want the plot for the temperature at the console to be labeled "Barn Temperature," rather than the default "Inside Temperature." This can be done by changing the "inTemp" option located in section [Labels][[Generic]] from the default
[Units] [[Generic]] inTemp = Inside Temperature outTemp = Outside Temperature ...
to:
[Units]
[[Generic]]
inTemp = Barn Temperature
outTemp = Outside Temperature
...
This approach is very similar, except that instead of changing the skin configuration file directly, you override its options by editing the main configuration file, weewx.conf. The advantage of this approach is that you can use the same skin to produce several different output, each with separate options.
With this approach, you must restart weewx to see the effects of any changes.
Revisiting our example, suppose you want two reports, one in US Customary, the other in Metric. The former will go in the directory $HTML_ROOT, the latter in a directory, $HTML_ROOT/metric. If you just simply modify skin.conf, you can get one, but not both at the same time. Alternatively, you could create a whole new skin by copying all the files to a new skin directory then editing the new skin.conf. The trouble with this approach is that you would then have two skins you would have to maintain. If you change something, you have to remember to change it in both places.
But, there's a better approach: reuse the same skin, but override some of its options. Here is what your [StdReport] section in weewx.conf would look like:
[StdReport] # # This section specifies what reports, using which skins, are to be generated. # # Where the skins reside, relative to WEEWX_ROOT: SKIN_ROOT = skins # Where the generated reports should go, relative to WEEWX_ROOT: HTML_ROOT = public_html # This report will use US Customary Units [[USReport]] # It is based on the Standard skin skin = Standard # This report will use metric units: [[MetricReport]] # It is also based on the Standard skin: skin = Standard # However, override where the results will go and put them in a directory: HTML_ROOT = public_html/metric # And override the options that were not in metric units [[[Units]]] [[[[Groups]]]] group_altitude = meter group_pressure = mbar group_rain = mm group_rainrate = mm_per_hour group_speed = meter_per_second group_speed2 = meter_per_second2 group_temperature = degree_C
We have done two things different from the stock reports. First (1), we have renamed the first report from StandardReport to USReport for clarity; and second (2), we have introduced a new report MetricReport, just like the first, except it puts its results in a different spot and uses different units. Both use the same skin, the Standard skin.
If you cannot achieve the results you need by changing a configuration option, you may have to modify the templates that come with weewx, or write your own.
Template modifications are preserved across upgrades (indeed, everything in the ./skins directory is preserved), so you don't have to worry about losing changes after an upgrade.
Template generation is done using the Cheetah templating engine. This is a very powerful engine, which essentially lets you have the full semantics of Python available in your templates. As this would make the templates incomprehensible to anyone but a Python programmer, weewx adopts a very small subset of its power.
The key construct is a 'dot' code, specifying what value you want. For example:
$month.outTemp.max $month.outTemp.maxtime $current.outTemp
would code the max outside temperature for the month, the time it occurred, and the current outside temperature, respectively. So, an HTML file that looks like
<html> <head> <title>Current conditions</title> </head> <body> <p>Current temperature = $current.outTemp</p> <p>Max for the month is $month.outTemp.max, which occurred at $month.outTemp.maxtime</p> </body> </html>
would be all you need for a very simple HTML page that would display the text (assuming that the unit group for temperature is degree_F):
Current temperature = 51.0°F
Max for the month is 68.8°F, which occurred at 07-Oct-2009 15:15
The format that was used to format the temperature (51.0) is specified in section [Units][[StringFormat]]. The unit label °F is from section [Units][[Labels]], while the time format is from [Units][[TimeFormats]].
As we saw above, the dot codes can be very simple:
## Output max outside temperature using an appropriate format and label: $month.outTemp.max
Most of the time, the dot code will "do the right thing" and is all you will need. However, weewx offers extensive customization of the generated output for specialized applications such as XML RSS feeds, or ridgidly formatted reports (such as the NOAA reports). This section specifies the various options available.
There are two different versions of the dot code, depending on whether the data is "current", or an aggregation over time. However, both versions are similar.
Time period $current represents a current observation. An example would be the current barometric pressure:
$current.barometer
The most general dot code for a "current" observation looks like:
$current.obstype[.optional_unit_conversion][.optional_formatting]
Where:
obstype is an observation type, such as barometer. See Appendix Archive Types for a table of observation types valid for time period current.
optional_unit_conversion is an optional unit conversion tag. If provided, the results will be converted into the specified units, otherwise the default units specified in the skin configuration file (in section [Units][[Groups]]) will be used. See the section Unit Conversion Options below.
optional_formatting is an optional formatting tag that controls how the value will appear. See the section Formatting Options below.
The other time periods represent an aggregation over time. In addition to the time period over which the aggregation will occur, they also require an aggregation type. An example would be the week's total precipitation (where the aggregation type is sum):
$week.rain.sum
The most general dot code for an aggregation over time looks like:
$period.statstype.aggregation[.optional_unit_conversion][.optional_formatting]
Where:
period is the time period over which the aggregation is to be done. Possible choices are day, week, month, year, rainyear.
statstype is a statistical type. See the Appendix Statistical Types, for a table of statistical types.
aggregation is an aggregation type. This is something like 'min', 'sum', 'mintime'. If you ask for $month.outTemp.avg you are asking for the average outside temperature for the month. The table Statistical Types shows what aggregation types are available for which types.
optional_unit_conversion is an optional unit conversion tag. If provided, the results will be converted into the specified units, otherwise the default units specified in the skin configuration file (in section [Units][[Groups]]) will be used. See the section Unit Conversion Options below.
optional_formatting is an optional formatting tag that controls how the value will appear. See the section Formatting Options below.
The tag optional_unit_conversion can be used with either current observations or aggregations. If supplied, the results will be converted to the specified units. For example, if you have set group_pressure to inches of mercury (inHg), then the tag
Today's average pressure=$day.barometer.avg
would normally give a result such as
Today's average pressure=30.05 inHg
However, if you add "mbar" to the end,
$day.barometer.avg.mbar
then the results will be in millibars:
Today's average pressure=1017.5 mbar
Using this method, you can output compass ordinals for wind direction. For example, the template
Current wind direction is $current.windDir ($current.windDir.ordinal_compass)
would result in:
Current wind direction is 138° (SW)
The ordinal abbreviations are set by option directions in the skin configuration file skin.conf.
If an inappropriate or nonsense conversion is asked for, e.g.,
Today's minimum pressure in mbars: $day.barometer.min.mbar or in degrees C: $day.barometer.min.degree_C or in foobar units: $day.barometer.min.foobar
then the offending tag(s) will be put in the output:
Today's minimum pressure in mbars: 1015.3
or in degrees C: $day.barometer.min.degree_C
or in foobar units: $day.barometer.min.foobar
The tag optional_formatting can be used with either current observations or aggregations. It can be one of:
Optional formatting tag | Comment |
(no tag) | Value is returned as a string, formatted using an appropriate string format from skin.conf. A unit label (e.g., °F) from skin.conf is also attached at the end. |
.string(NONE_string) | Value is returned as a string, formatted using an appropriate string format from skin.conf. If the value is None, the string NONE_string will be substituted if given, otherwise the value for NONE in [Units][[StringFormats]] will be used. A unit label (e.g., °F) from skin.conf will be attached at the end. |
.formatted | Value is returned as a string, formatted using an appropriate string format and None value from skin.conf. No unit label will be attached. |
.format(string_format, NONE_string) | Value is returned as a string, using the string format specified with string_format. If the value is None, the string NONE_string will be substituted if given, otherwise the value for NONE in [Units][[StringFormats]] will be used. A unit label (e.g., °F) from skin.conf will be attached at the end. |
.nolabel(string_format, NONE_string) | Value is returned as a string, using the string format specified with string_format. If the value is None, the string NONE_string will be substituted if given, otherwise the value for NONE in [Units][[StringFormats]] will be used. No unit label will be attached at the end. |
.raw | Value is returned "as is" without being converted to a string and without any formatting applied. This can be useful for doing arithmetic directly within the templates. You must be prepared to deal with a None value unless the value is converted directly to a string. In this case, it will be converted to the empty string ('') |
Formatting Tag | Format Used | Label Used | NONE String | Returned Value |
(no tag) | From skin.conf | From skin.conf | From skin.conf | string |
.string | From skin.conf | From skin.conf | Optional user-supplied | string |
.formatted | From skin.conf | No label | From skin.conf | string |
.format | User-supplied | From skin.conf | Optional user-supplied | string |
.nolabel | User-supplied | No label | Optional user-supplied | string |
.raw | None | No label | None | native value |
Here are some examples with the expected results:
Tag | Result | Comment |
$current.outTemp | 45.2°F | String formatting and label from skin.conf |
$current.outTemp.string | 45.2°F | String formatting and label from skin.conf |
$current.UV.string | N/A | This example assumes that the instrument has no UV sensor, resulting in a None value. The string specified by NONE in [Units][[StringFormats]] is substituted. |
$current.UV.string("No UV") | No UV | This example assumes that the instrument has no UV sensor, resulting in a None value. The string supplied by the user is substituted. |
$current.outTemp.formatted | 45.2 | String formatting from skin.conf; no label |
$current.outTemp.format("%.3f") | 45.200°F | Specified string format used; label from skin.conf. |
$current.dateTime | 02-Apr-2010 16:25 | Time formatting and label from skin.conf |
$current.dateTime.format("%H:%M") | 16:25 | Specified time format used; label from skin.conf. |
$current.dateTime.raw | 1270250700 | Unix epoch time, converted to string by template engine. |
$current.outTemp.raw | 45.2 | Float returned, converted to string by template engine. |
$month.dateTime | 01-Apr-2010 00:00 | Time formatting and label from skin.conf |
$month.outTemp.avg | 40.8°F | String formatting and label from skin.conf |
$month.outTemp.avg.string | 40.8°F | Time formatting and label from skin.conf |
$month.UV.avg.string | N/A | This example assumes that the instrument has no UV sensor, resulting in a None value. The string specified by NONE in [Units][[StringFormats]] is substituted. |
$month.UV.avg.string("No UV") | No UV | This example assumes that the instrument has no UV sensor, resulting in a None value. The string supplied by the user is substituted. |
$month.outTemp.avg.formatted | 40.8 | String formatting from skin.conf; no label |
$month.outTemp.avg.format("%.3f") | 40.759°F | Specified string format used; no label |
$month.outTemp.avg.raw | 40.7589690722 | Float returned, converted to string by template engine |
$month.UV.avg.raw | (empty) | None value converted to empty string by template engine. |
Note:
While not an observation type, in many ways the time of an observation, dateTime, can be treated as one. A tag such as $current.dateTime represents the current time (more properly, the time as of the end of the last archive interval). Similarly, a tag such as $month.dateTime represents the start time of the month. Like true observation types, explicit formats can be specified, except that they require a strftime() time format, rather than a string format:
$month.dateTime.format("%B %Y")
produces
January 2010
The returned string value will always be in local time.
The raw value of dateTime is Unix Epoch Time (number of seconds since 00:00:00 UTC 1 Jan 1970, i.e., a large number), which you must convert yourself to local time. It is guaranteed to never be None, so you don't worry have to worry about handling a None value.
The tag $trend is available for time trends, such as barometer trends. Here are some examples:
Tag | Results |
$trend.barometer | -.02 inHg |
$trend.outTemp | 1.1 °C |
$trend.time_delta | 10800 secs |
$trend.time_delta.hour | 3 hrs |
Note that the time delta over which the trend is calculated is also available. This time delta is set by an option in the skin configuration file, time_delta.
As a summary, the template expression
<p>The barometer trend over $trend.time_delta.hour is $trend.barometer.format("%+.2f").</p>
would result in
The barometer trend over 3 hrs is +.02 inHg.
The unit type, label, and string formats are also available, allowing you to do highly customized labels:
Tag | Results |
$unit.unit_type.outTemp | degree_C |
$unit.label.outTemp | °C |
$unit.format.outTemp | %.1f |
As a summary, the tag
$day.outTemp.max.formatted$unit.label.outTemp
would result in
21.2°C
(assuming metric values have been specified for group_temperature), essentially reproducing the results of the simpler tag $day.outTemp.max.
For dot codes using an aggregation (e.g., $day, $week, $month, $year, $rainyear, then the aggregation period can be iterated over by day or month. These are the only two iteration periods available as of this version.
This example uses a Cheetah 'for' loop to iterate over all months in a year, printing out each month's min and max temperature (the iteration loop is highlighted ):
<html> <head> <title>Year stats by month</title> </head> <body> <p>Min, max temperatures by month:</p> #for $month in $year.months <p>$month.dateTime.format("%B"): Min, max temperatures: $month.outTemp.min $month.outTemp.max</p> #end for </body> </html>
Produces results:
Min, max temperatures by month: January: Min, max temperatures: 30.1°F 51.5°F February: Min, max temperatures: 24.4°F 58.6°F March: Min, max temperatures: 27.3°F 64.1°F April: Min, max temperatures: 33.2°F 52.5°F May: Min, max temperatures: N/A N/A June: Min, max temperatures: N/A N/A July: Min, max temperatures: N/A N/A August: Min, max temperatures: N/A N/A September: Min, max temperatures: N/A N/A October: Min, max temperatures: N/A N/A November: Min, max temperatures: N/A N/A December: Min, max temperatures: N/A N/A
See the NOAA template files NOAA/NOAA-YYYY.txt.tmpl and NOAA/NOAA-YYYY-MM.txt.tmpl for examples using iteration, as well as explicit formatting.
If module pyephem has been installed, then weewx can generate extensive almanac information for the Sun, Moon, Venus, Mars, Jupiter, and other heavenly bodies, including their rise, transit and set times, as well as their azimuth and altitude. Other information is also available.
Here is a small sampling:
<html> <head> <title>Almanac data</title> </head> <body> <p>Current time is $current.dateTime<p> #if $almanac.hasExtras <p>Sunrise, transit, sunset: $almanac.sun.rise $almanac.sun.transit $almanac.sun.set</p> <p>Moonrise, transit, moonset: $almanac.moon.rise $almanac.moon.transit $almanac.moon.set</p> <p>Mars rise, transit, set: $almanac.mars.rise $almanac.mars.transit $almanac.mars.set</p> <p>Azimuth, altitude of mars: $almanac.mars.az $almanac.mars.alt</p> <p>Next new, full moon: $almanac.next_new_moon $almanac.next_full_moon</p> <p>Next summer, winter solstice: $almanac.next_summer_solstice $almanac.next_winter_solstice</p> #else <p>Sunrise, sunset: $almanac.sunrise $almanac.sunset</p> #end if </body> </html>
If your installation has pyephem installed this would result in:
Current time is 29-Mar-2011 09:20 Sunrise, transit, sunset: 06:51 13:11 19:30 Moonrise, transit, moonset: 04:33 09:44 15:04 Mars rise, transit, set: 06:35 12:30 18:26 Azimuth, altitude of mars: 124.354959275 26.4808431952 Next new, full moon: 03-Apr-2011 07:32 17-Apr-2011 19:43 Next summer, winter solstice: 21-Jun-2011 10:16 21-Dec-2011 21:29
Otherwise, a fallback position is used, resulting in
Current time is 29-Mar-2011 09:20 Sunrise, sunset: 06:51 19:30
As shown in the example, you can test whether this extended almanac information is available with the value $almanac.hasExtras.
The almanac information falls in two categories:
We will cover each of these separately.
"Calendar events" do not require a heavenly body. They cover things such as "next_solstice", or "next_first_quarter_moon". The syntax here is
$almanac.next_solstice
or
$almanac.next_first_quarter_moon
Here is a table of the information that falls into this category:
previous_equinox | next_equinox | previous_solstice | next_solstice |
previous_autumnal_equinox | next_autumnal_equinox | previous_vernal_equinox | next_vernal_equinox |
previous_winter_solstice | next_winter_solstice | previous_summer_solstice | next_summer_solstice |
previous_new_moon | next_new_moon | previous_first_quarter_moon | next_first_quarter_moon |
previous_full_moon | next_full_moon | previous_last_quarter_moon | next_last_quarter_moon |
The second category does require a heavenly body. This covers queries such as, "When does Jupiter rise?" or, "When does the sun transit?" Examples are
$almanac.jupiter.rise
or
$almanac.sun.transit
To accurately calculate these times, weewx automatically uses the present temperature and pressure to calculate refraction effects. However, you can override these values, which will be necessary if you wish to match the almanac times published by the Naval Observatory as explained in the pyephem documentation. For example, to match the sunrise time as published by the Observatory, instead of
$almanac.sun.rise
use
$almanac(pressure=0, horizon=-34.0/60.0).sun.rise
By setting pressure to zero we are bypassing the refraction calculations and manually setting the horizon to be 34 arcminutes lower than the normal horizon. This is what the Navy uses.
If you wish to calculate the start of civil twilight, you can set the horizon to -6 degrees, and also tell weewx to use the center of the sun (instead of the upper limb, which it normally uses) to do the calcuation:
$almanac(pressure=0, horizon=-6).sun(use_center=1).rise
The general syntax is:
$almanac(pressure=pressure, horizon=horizon, temperature=temperature_C).heavenly_body(use_center=[01]).attribute
As you can see, in addition to the horizon angle, you can also override atmospheric pressure and temperature (degrees Celsius).
PyEphem offers an extensive list of objects that can be used for the heavenly_body tag. All the planets and many stars are in the list.
The possible values for the attribute tag are listed in the following table:
az | alt | a_ra | a_dec |
g_ra | ra | g_dec | dec |
elong | radius | hlong | hlat |
sublat | sublong | next_rising | next_setting |
next_transit | next_antitransit | previous_rising | previous_setting |
previous_transit | previous_antitransit | rise | set |
transit |
Warning!
The programming interface for
search list extensions is still in a state of flux, and likely
to change in Version 3. Be prepared to modify any extensions you
write.
In the section on Customizing templates, we have seen how you can change a template and make use of the various tags available such as $day.outTemp.max for the maximum outside temperature for the day. But, what if you want to introduce some new data for which no tag is available?
If you wish to introduce a static tag, that is, one that will not change with time (such as a Google analytics Tracker ID, or your name), then this is very easy: simply put it in section [Extras] in the skin configuration file. More information on how to do this can be found there.
But, what if you wish to introduce a more dynamic tag, one that requires some calculation, or perhaps uses the database? Simply putting it in the [Extras] section won't do, because then it cannot change.
The answer is to write a search list extension.
Let's start by taking a look at how the Cheetah search list works.
The Cheetah template engine (which weewx uses) finds tags by scanning a search list, a Python list of objects. For example, for a tag $foo, the engine will scan down the list, trying each object in the list in turn. For each object, it will first try using foo as an attribute, that is, it will try evaluating obj.foo. If that raises an AttributeError exception, then it will try foo as a key, that is obj[key]. If that raises a KeyError exception, then it moves on to the next item in the list. The first match that does not raise an exception is used. If no match is found, Cheetah raises a NameMapper.NotFound exception.
Now let's take a look at how the search list interacts with weewx tags. Let's start by looking at a simple example: station altitude, available as the tag
$station.altitude
As we saw in the previous section, Cheetah will run down the search list, looking for an object with a key or attribute station. In the default search list, weewx includes one such object, an instance of the class weewx.cheetahgenerator.Station, which has an attribute station, so it gets a hit on this object.
Cheetah will then try to evaluate the attribute altitude on this object. Class Station has such an attribute, so Cheetah evaluates it.
What this attribute returns is not a raw value, say 700, nor even a string. Instead, it returns an instance of the class ValueHelper, a special class defined in module weewx.units. Internally, it holds the formats, labels, and conversion targets you specified in your configuration file. It's job is to make sure that the final output reflects these preferences. Cheetah doesn't know anything about this class. What it needs, when it has finished evaluating the expression $day.outTemp.max is a string. In order to convert the ValueHelper into a string, it does what every other Python object does when faced with this situation: it calls the special method __str__. Class ValueHelper has a definition for this method. Evaluating this function triggers the final steps in this process. Any necessary unit conversions are done, then formatting occurs and, finally, a label is attached. The result is a string something like
700 feet
which is what Cheetah actually puts in the generated HTML file.
Now let's look at a more complicated example, one that uses lazy evaluation,say the maximum temperature since midnight:
$day.outTemp.max
When this is evaluated by Cheetah, it actually produces a chain of objects. At the top of this chain is class weewx.stats.TaggedStats, an instance of which is included in the default search list. Internally, this instance stores the database to be hit, as well as a few other bookkeeping objects (mostly having to do with formatting and labeling).
This instance is examined by Cheetah to see if it has an attribute day. It does and, when it is evaluated, it returns the next class in the chain, an instance of weewx.stats.TimeSpanStats. This instance stores not only the database, but also the requested time span, namely the time since midnight, internally.
Cheetah then tries to find an attribute outTemp of TimeSpanStats. There is no such hard coded attribute (hard coding all the different observation types would be impractical!). Instead, class TimeSpanStats returns an instance of the next class in the chain, weewx.stats.StatsTypeHelper, which not only remembers what is already known, namely the database to be hit and the time span, but also adds the observation type, outTemp, in this case.
Cheetah then tries to evaluate an attribute max of this class. Now, finally, the chain ends. The attribute max triggers the actual calculation of the value, using all the known parameters: the database to be hit, the time span of interest, the observation type, and the type of aggregation, querying the database as necessary. This is an example of lazy evaluation. That is, the database is not actually hit until the last possible moment, after everything needed to do the evalation is known.
The results of the evaluation is then packaged up in, you guessed it, an instance of ValueHelper, which does the final conversion to the desired units, formats the string, then adds a label. The results, something like
12°C
are put in the generated HTML file. As you can see, a lot of machinery is hidden behind the deceptively simple expression $day.outTemp.max!
As mentioned, weewx comes with a number of objects already in the search list, but you can extend it. To do so, you should have some familiarity with Python, in particular, how to write new classes and member functions for them.
Let's look at an example. The regular version of weewx offers statistical summaries by day, week, month, and year. Suppose we would like to add two more:
This example is included in the distribution as $BIN_ROOT/examples/xsearch.py:
import datetime import time from weewx.cheetahgenerator import SearchList from weewx.stats import TimeSpanStats from weeutil.weeutil import TimeSpan class MyXSearch(SearchList): #1 def __init__(self, generator): #2 SearchList.__init__(self, generator) def get_extension(self, valid_timespan, archivedb, statsdb): #3 # First, get a TimeSpanStats object for all time. This one is easy # because the object valid_timespan already holds all valid times to be # used in the report. all_stats = TimeSpanStats(valid_timespan, statsdb, formatter=self.generator.formatter, converter=self.generator.converter) # 4 # Now get a TimeSpanStats object for the last seven days. This one we # will have to calculate. First, calculate the time at midnight, seven # days ago. The variable week_dt will be an instance of datetime.date. week_dt = datetime.date.fromtimestamp(valid_timespan.stop) - datetime.timedelta(weeks=1) #5 # Now convert it to unix epoch time: week_ts = time.mktime(week_dt.timetuple()) # 6 # Now form a TimeSpanStats object, using the time span we just calculated: seven_day_stats = TimeSpanStats(TimeSpan(week_ts, valid_timespan.stop), statsdb, formatter=self.generator.formatter, converter=self.generator.converter) #7 # Now create a small dictionary with keys 'alltime' and 'seven_day': search_list_extension = {'alltime' : all_stats, 'seven_day' : seven_day_stats} # 8 return search_list_extension
Going through the example, line by line:
That one was relatively easy because we already had an instance of TimeSpan, valid_timespan, that represented the time over which we wanted to do the calculations. Setting up an instance that will work for the last seven days is a bit trickier. Continuing our example...
The last step is to tell the template engine where to find our extension. You do that by going into the skin configuration file, skin.conf, and adding the option search_list_extensions with our new extension. When you're done, it will look something like this:
[CheetahGenerator]
# This section is used by the generator CheetahGenerator, and specifies
# which files are to be generated from which template.
# Possible encodings are 'html_entities', 'utf8', or 'strict_ascii'
encoding = html_entities
search_list_extensions = examples.xsearch.MyXSearch
[[SummaryByMonth]]
...
Our addition has been highlighted. Note that it is in the section [CheetahGenerator]. (This section was called [FileGenerator] in earlier versions of weewx, a name which will still work.)
Now, if the Cheetah engine encounters the tag $alltime, it will scan the search list, looking for an attribute or key that matches alltime. When it gets to the little dictionary we provided, it will find a matching key, allowing it to retrieve the appropriate TimeSpanStats object.
With this approach, you can now include "all time" or "seven day" statistics in your HTML templates:
... <table> <tr> <td>Maximum temperature to date: </td> <td>$alltime.outTemp.max</td> </tr> <tr> <td>Minimum temperature to date: </td> <td>$alltime.outTemp.min </tr> <tr> <td>Rain over the last seven days: </td> <td>$seven_day.rain.sum </tr> ... (more table entries)
If you place a custom generator somewhere other than the $BIN_ROOT hierarchy where weewxd resides, you may have to specify its location in the environment variable PYTHONPATH in the shell where you start weewx:
export PYTHONPATH=/home/me/secret_location
This section is a reference to the options appearing in the Standard skin configuration file, found in $SKIN_ROOT/Standard/skin.conf.
It is worth noting that, like the main configuration file weewx.conf, UTF-8 is used throughout. The most important options are up near the top of the file. The truly important ones, the ones you are likely to have to customize for your station, are highlighted.
This section is available to you to add any static tags that you might want to be available in the templates.
As an example, the Standard skin.conf file includes three options:
Skin option | Template tag |
radar_img | $Extras.radar_img |
radar_url | $Extras.radar_url |
googleAnalyticsId | $Extras.googleAnalyticsId |
If you take a look at the template index.html.tmpl you will see examples of testing for these tags (search the file for the string radar_img to find them).
radar_img
Set to an URL to show a local radar image for you.
radar_url
If the above radar image is clicked, the browser will go to this URL. This is usually used to show a more detailed, close-up, radar picture.
For me in Oregon, setting the above two options to:
[Extras] radar_img = http://radar.weather.gov/ridge/lite/N0R/RTX_loop.gif radar_url = http://radar.weather.gov/ridge/radar.php?product=NCR&rid=RTX&loop=yes
results in a nice image of a radar centered on Portland, Oregon. When you click on it, it gives you a detailed, animated view. Take a look at the NOAA radar website to find a nice one for you in the USA. In other countries, you will have to consult your local weather service.
googleAnalyticsId
If you have a Google Analytics ID, you can set it here. The Google Analytics Javascript code will then be included, enabling analytics of your website usage. If commented out, the code will not be included.
Other tags can be added in a similar manner, including sub-sections. For example, say you have added a video camera and you would like to add a still image with a hyperlink to a page with the video. You want all of these options to be neatly contained in a sub-section.
[Extras] [[video]] still = video_capture.jpg hyperlink = http://www.eatatjoes.com/video.html
Then in your template you could refer to these as:
<a href="$Extras.video.hyperlink"> <img src="$Extras.video.still" alt="Video capture"/> </a>
This section deals with Units and their formatting.
This sub-section lists all the Unit Groups and specifies which unit system is to be used for each one of them.
As there are many different observational measurement types (such as 'outTemp', 'barometer', etc.) used in weewx (more than 50 at last count), it would be tedious, not to say possibly inconsistent, to specify a different measurement system for each one of them. At the other extreme, requiring all of them to be "U.S. Customary" or "Metric" seems overly restrictive. Weewx has taken a middle route and divided all the different observation types into 12 different "unit groups." A unit group is something like "group_temperature." It represents the measurement system to be used by all observation types that are measured in temperature, such as inside temperature (type 'inTemp'), outside temperature ('outTemp'), dewpoint ('dewpoint'), wind chill ('windchill'), and so on. If you decide that you want unit group group_temperature to be measured in "degree_C" then you are saying all members of its group will be reported in degrees Celsius.
Note that the unit system is always specified in the singular. That is, specify "degree_C" or "foot", not "degrees_C" or "feet". See the Appendix Units for more information, including a concise summary of the groups, their members, and which options can be used for each group.
Which measurement unit to be used for altitude. Possible options are 'foot' or 'meter'.
group_direction
Which measurement unit to be used for direction. The only option is "degree_compass".
group_moisture
The measurement unit to be used for soil moisture. The only option is "centibar."
group_percent
The measurement unit to be used for percentages. The only option is "percent".
group_pressure
The measurement unit to be used for pressure. Possible options are one of "inHg" (inches of mercury), "mbar", or "hPa."
group_radiation
The measurement unit to be used for radiation. The only option is "watt_per_meter_squared."
group_rain
The measurement unit to be used for precipitation. Options are "inch", "cm," or "mm."
group_rainrate
The measurement unit to be used for rate of precipitation. Possible options are one of "inch_per_hour", "cm_per_hour", or "mm_per_hour".
group_speed
The measurement unit to be used for wind speeds. Possible options are one of "mile_per_hour", "km_per_hour", "knot", or "meter_per_second."
group_speed2
This group is similar to group_speed, but is used for calculated wind speeds which typically have a slightly higher resolution. Possible options are one "mile_per_hour2", "km_per_hour2", "knot2", or "meter_per_second2".
The measurement unit to be used for temperatures. Options are "degree_F" or "degree_C."
group_volt
The measurement unit to be used for voltages. The only option is "volt."
This sub-section is used to specify what string format is to be used for each unit when a quantity needs to be converted to a string. Typically, this happens with y-axis labeling on plots and for statistics in HTML file generation. For example, the options
degree_C = %.1f inch = %.2f
would specify that the given string formats are to be used when formatting any temperature measured in degrees Celsius or any precipitation amount measured in inches, respectively. The formatting codes are those used by Python, and are very similar to C's sprintf() codes.
You can also specify what string to use for an invalid or unavailable measurement (value 'None'). For example,
NONE = " N/A "
This sub-section specifies what label is to be used for each measurement unit type. For example, the options
degree_F = °F inch = ' in'
would cause all temperatures to have unit labels °F and all precipitation to have labels in. If any special symbols are to be used (such as the degree sign above) they should be encoded in UTF-8. This is generally what most text editors use if you cut-and-paste from a character map.
If the label includes two values, then the first is assumed to be the singular form, the second the plural form. For example,
foot = " foot", " feet" ... day = " day", " days" hour = " hour", " hours" minute = " minute", " minutes" second = " second", " seconds"
This is particularly useful when localizing the weewx and server "uptimes."
This sub-section is used for time labels. It uses strftime() formats. The default looks like this:
[[TimeFormats]] # This section sets the string format to be used for # each time scale. day = %X week = %X (%A) month = %x %X year = %x %X rainyear = %x %X current = %x %X ephem_day = %X ephem_year = %x %X
The specifiers %x, %X, and %A code locale dependent date, time, and weekday names, respectively. Hence, if you set an appropriate environment variable LANG, then the date and times should follow local conventions (see section Environment variable LANG below for details on how to do this). However, they will not look particularly pretty and you may want to change them. For example, I use this in the U.S.:
[[TimeFormats]] # # This section sets the string format to be used # each time scale. # day = %H:%M week = %H:%M on %A month = %d-%b-%Y %H:%M year = %d-%b-%Y %H:%M rainyear = %d-%b-%Y %H:%M current = %d-%b-%Y %H:%M ephem_day = %H:%M ephem_year = %d-%b-%Y %H:%M
The last two formats, ephem_day and ephem_year allow the formatting to be set for almanac times The first, ephem_day, is used for almanac times within the day, such as sunrise or sunset. The second, ephem_year, is used for almanac times within the year, such as the next equinox or full moon.
directions
Set to the abbreviations to be used for ordinal directions. By default, this is N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW, N.
heating_base
cooling_base
Set to the base temperature for calculating heating and cooling degree-days, along with the unit to be used. Examples:
heating_base = 65.0, degree_F cooling_base = 20.0, degree_C
time_delta
Set to the time difference over which you want trends to be calculated. The default is 3 hours.
time_grace
When searching for a previous record to be used in calculating a trend, a record within this amount of time_delta will be accepted. Default is 300 seconds.
This section sets the various labels to use.
hemispheres
Comma separated list for the labels to be used for the four hemispheres. The default is "N, S, E, W".
latlon_formats
Comma separated list for the formatting to be used when converting latitude and longitude to strings. There should be three elements:
This allows you to decide whether or not you want leading zeroes. The default includes leading zeroes and is "%02d", "%03d", "%05.2f"
This sub-section specifies default labels to be used for each observation type. For example, options
inTemp = Temperature inside the house outTemp = Outside Temperature
would cause the given labels to be used for plots of inTemp and outTemp.
This section controls what text to use for the almanac. It consists of only one entry
moon_phases
This option is a comma separated list of labels to be used for the eight phases of the moon. Default is "New, Waxing crescent, First quarter, Waxing gibbous, Full, Waning gibbous, Last quarter, Waning crescent".
This section is used by generator weewx.cheetahgenerator.CheetahGenerator and controls text generation from templates, specifically which files are to be produced from which template.
Before V2.5, this section was called [FileGenerator]. Older versions and names are 100% backwards compatible.
Files are generated from templates, and each template is identified by the template parameter.
Each template file is named something like D/F.E.tmpl, where D is the (optional) directory the template sits in and will also be the directory the results will be put in, and F.E is the generated file name. So, given a template file with name Acme/index.html.tmpl, the results will be put in $HTML_ROOT/Acme/index.html.
The configuration for a group of templates will look something like this:
[CheetahGenerator] [[index]] template = index.html.tmpl [[textfile]] template = filename.txt.tmpl [[xmlfile]] template = filename.xml.tmpl
There can be only one template in each block. In most cases, the block name does not matter - it is used only to isolate each template. However, there are two block names that have special meaning: SummaryByMonth and SummaryByYear. These are described below.
The file generator runs on each new archive record. In a default weewx installation, that would be every 5 minutes.
Cheetah processes each template to generate a file. Cheetah follows any logic defined by directives such as for or if ... else, and it replaces variables such as $Extras.radar_url or $current.outTemp.max.
Variables are defined by objects in weewx. Some variables are static, others are linked to data in databases. The list of variables can be extended.
search_list
This is the list of search list objects that will be scanned by the template engine, looking for tags. See the section Defining new tags and the Cheetah documentation for details on search lists. If no search_list is specified, a default list will be used. The default list is:
search_list = weewx.cheetahgenerator.Almanac, weewx.cheetahgenerator.Station, weewx.cheetahgenerator.Stats, weewx.cheetahgenerator.UnitInfo, weewx.cheetahgenerator.Extras, weewx.cheetahgenerator.Current
search_list_extensions
This defines one or more search list objects that will be appended to the search_list. For example, the following adds alltime and forecast variables to the search list.
search_list_extensions = examples.xsearch.MyXSearch, user.forecast.ForecastVariables
encoding
This option controls which encoding is to be used for the generated output. The encoding can be specified for individual files. There are 3 possible choices:
Encoding | Comments |
html_entities | Non 7-bit characters will be represented as HTML entities (e.g., the degree sign will be represented as °) |
utf8 | Non 7-bit characters will be represented in UTF-8. |
strict_ascii | Non 7-bit characters will be ignored. |
The encoding html_entities is the default.
template
The name of a template file. A template filename must end with .tmpl. Filenames are case-sensitive. If the template filename has the letters YYYY or MM in its name, these will be substituted for the year and month, respectively. So, a template with the name summary-YYYY-MM.html.tmpl would have name summary-2010-03.html for the month of March, 2010.
stale_age
File staleness age, in seconds. If the file is older than this age it will be generated from the template. If no stale_age is specified the file will be generated each time the generator runs.
[[SummaryByMonth]]
The SummaryByMonth section defines some special behavior. Each template in this section will be used multiple times, each time with a different per-month timespan. Be sure to include YYYY and MM in the filename of any template in this section.
[[SummaryByYear]]
The SummaryByYear section defines some special behavior. Each template in this section will be used multiple times, each time with a different per-year timespan. Be sure to include YYYY in the filename of any template in this section.
The best way to customization file generation is to make a copy of a working report/skin, then make incremental changes.
When there is an error during template generation, the error will show up in the log file. Many errors are obvious — Cheetah will display a line number and list the template file in which the error occurred. In some cases the error reporting is rather obscure. So make small changes and tests often. Use the tool wee_reports to test modifications to the generator configuration and/or the template contents.
Here is the [CheetahGenerator] section from the Standard skin.conf
[CheetahGenerator] # This section is used by the generator CheetahGenerator, and specifies # which files are to be generated from which template. encoding = html_entities [[SummaryByMonth]] # Reports that summarize "by month" [[[NOAA_month]]] encoding = strict_ascii template = NOAA/NOAA-YYYY-MM.txt.tmpl [[SummaryByYear]] # Reports that summarize "by year" [[[NOAA_year]]] encoding = strict_ascii template = NOAA/NOAA-YYYY.txt.tmpl [[ToDate]] # Reports that show statistics "to date", such as day-to-date, # week-to-date, month-to-date, etc. [[[day]]] template = index.html.tmpl [[[week]]] template = week.html.tmpl [[[month]]] template = month.html.tmpl [[[year]]] template = year.html.tmpl [[[RSS]]] template = RSS/weewx_rss.xml.tmpl [[[Mobile]]] template = mobile.html.tmpl
The Standard skin contains three different kinds of generated output:
The encoding for text files is strict_ansii, whereas the encoding for html files is html_entities. In the Standard skin this is specified by declaring encoding = html_entities at the top level of [CheetahGenerator] then encoding = strict_ansii for each text file.
Other than SummaryByMonth and SummaryByYear, the section names are arbitrary. ToDate could just as well have been called files_to_date, and the sections day, week, and month could just as well have been called tom, dick, and harry.
This section is used by generator weewx.reportengine.CopyGenerator and controls which files are to be copied over from the skin directory to the destination directory. Think of it as "file generation," except that rather than going through the template engine, the files are simply copied over.
copy_once
This option controls which files get copied over on the first invocation of the report engine service. Typically, this is things such as style sheets or background GIFs. Wildcards can be used.
copy_always
This is a list of files that should be copied on every invocation. Wildcards can be used.
Here is the [CopyGenerator] section from the Standard skin.conf
[CopyGenerator] # This section is used by the generator CopyGenerator # List of files to be copied only the first time the generator runs copy_once = backgrounds/*, weewx.css, mobile.css, favicon.ico # List of files to be copied each time the generator runs # copy_always =
The Standard skin includes some background images, CSS files, and icons that need to be copied once. There are no files that need to be copied each time the generator runs.
This section is used by generator weewx.reportengine.ImageGenerator and controls which images (plots) get generated and with which options. While complicated, it is extremely flexible and powerful.
The section consists of one or more sub-sections, one for each time period (day, week, month, and year). These sub-sections define the nature of aggregation and plot types for the time period. For example, here is a typical set of options for sub-section [[month_images]], controlling how images that cover a month period are generated:
[[month_images]] x_label_format = %d bottom_label_format = %m/%d/%y %H:%M time_length = 2592000 # == 30 days aggregate_type = avg aggregate_interval = 10800 # == 3 hours
The option x_label_format gives a strftime() type format for the x-axis. In this example, it will only show days (format option "%d"). The bottom_label_format is the format used to time stamp the image at the bottom. In this example, it will show the time as 10/25/09 15:35. A plot will cover a nominal 30 days, and all items included in it will use an aggregate type of averaging over 3 hours.
Within each sub-section is another nesting, one for each image to be generated. The title of each sub-sub-section is the filename to be used for the image. Finally, at one additional nesting level (!) are the logical names of all the line types to be drawn in the image. Values specified in the level above can be overridden. For example, here is a typical set of options for sub-sub-section [[[monthrain]]]:
[[[monthrain]]] plot_type = bar yscale = None, None, 0.02 [[[[rain]]]] aggregate_type = sum aggregate_interval = 86400 label = Rain (daily avg)
This will generate an image file with name monthrain.png. It will be a bar plot. Option yscale controls the y-axis scaling — if left out, the scale will automatically be chosen. However, in this example we are choosing to exercise some degree of control by specifying values explicitly. The option takes a 3-way tuple (ylow, yhigh, min_interval), where ylow and yhigh are the minimum and maximum y-axis values, respectively, and min_interval is the minimum tick interval. If set to 'None', the corresponding value will be automatically chosen. So, in this example, the setting
yscale = None, None, 0.02
will cause weewx to pick sensible y minimum and maximum values, but require that the tick increment (min_interval) be at least 0.02.
Continuing on with the example above, there will be only one plot "line" (it will actually be a series of bars) and it will have logical name "rain". Because we have not said otherwise, the SQL data type to be used for this line will be the same as its logical name, that is, rain, but this can be overridden (see below). The aggregation type will be summing (overriding the averaging specified in sub-section [[month_images]]), so you get the total rain over the aggregate period (rather than the average) over an aggregation interval of 86,400 seconds (one day). The plot line will be titled with the indicated label ('Rain (daily avg)')
If there is a time gap in the data, the options line_gap_fraction and bar_gap_fraction control how it will be drawn. The former, line_gap_fraction, is used for line graphs, the latter, bar_gap_fraction, for bar graphs. Here's what the resultant plots look like without and with this option being specified:
No line_gap_fraction specified
With line_gap_fraction=0.01
More than one SQL type can be included in a plot. For example, here is how to generate a plot with the week's outside temperature as well as dewpoint:
[[[monthtempdew]]] [[[[outTemp]]]] [[[[dewpoint]]]]
This would create an image in file monthtempdew.png that includes a line plot of both outside temperature and dewpoint.
Another example. Say you want a plot of the day's temperature, overlaid with hourly averages. Here, you are using the same data type ('outTemp') for both plot lines, the first with averages, the second without. If you do the obvious it won't work:
## WRONG ## [[[daytemp_with_avg]]] [[[[outTemp]]]] aggregate_type = avg aggregate_interval = 3600 [[[[outTemp]]]] # OOPS! The same section name appears more than once!
The option parser does not allow the same section name ('outTemp' in this case) to appear more than once at a given level in the configuration file, so an error will be declared (technical reason: formally, the sections are an unordered dictionary). If you wish for the same SQL type to appear more than once in a plot then there is a trick you must know: use option data_type. This will override the default action that the logical line name is used for the SQL type. So, our example would look like this:
[[[daytemp_with_avg]]] [[[[a_logical_name]]]] data_type = outTemp aggregate_type = avg aggregate_interval = 3600 label = Avg. Temp. [[[[outTemp]]]]
Here, the first logical line has been given the name "a_logical_name" to distinguish it from the second line "outTemp". We have specified that the first line will use data type outTemp and that it will use averaging over a one hour period. The second also uses outTemp, but will not use averaging.
The result is a nice plot of the day's temperature, overlaid with a 3-hour smoothed average:
One more example. This one shows daily high and low temperatures for a year:
[[year_images]] ... [[[yearhilow]]] [[[[hi]]]] data_type = outTemp aggregate_type = max label = High [[[[low]]]] data_type = outTemp aggregate_type = min label = Low Temperature
This results in the plot yearhilow.png:
Weewx can produce progressive vector plots as well as the more conventional x-y plots. To produce these, use plot type 'vector'. You need a vector type to produce this kind of plot. There are two: 'windvec', and 'windgustvec'. While they do not actually appear in the SQL database, weewx understands that they represent special vector-types. The first, 'windvec', represents the average wind in an archive period, the second, 'windgustvec' the max wind in an archive period. Here's how to produce a progressive vector for one week that shows the hourly biggest wind gusts, along with hourly averages:
[[[weekgustoverlay]]] aggregate_interval = 3600 [[[[windvec]]]] label = Hourly Wind plot_type = vector aggregate_type = avg [[[[windgustvec]]]] label = Gust Wind plot_type = vector aggregate_type = max
This will produce an image file with name weekgustoverlay.png. It will consist of two progressive vector plots, both using hourly aggregation (3,600 seconds). For the first set of vectors, the hourly average will be used. In the second, the max of the gusts will be used:
By default, the sticks in the progressive wind plots point towards the wind source. That is, the stick for a wind from the west will point left. If you have a chronic wind direction (as I do), you may want to rotate the default direction so that all the vectors do not line up over the x-axis, overlaying each other. Do this by using option vector_rotate. For example, with my chronic westerlies, I set vector_rotate to 90.0 for the plot above, so winds out of the west point straight up.
If you use this kind of plot (the out-of-the-box version of weewx includes daily, weekly, monthly, and yearly progressive wind plots), a small compass rose will be put in the lower-left corner of the image to show the orientation of North.
Remember that values at any level can override values specified at a higher level. For example, say you want to generate the standard plots, but for a few key observation types such as barometer, you want to also generate some oversized plots to give you extra detail, perhaps for an HTML popup. The standard weewx.conf file specifies plot size of 300x180 pixels, which will be used for all plots unless overridden:
[ImageGenerator] ... image_width = 300 image_height = 180
The standard plot of barometric pressure will appear in daybarometer.png:
[[[daybarometer]]] [[[[barometer]]]]
We now add our special plot of barometric pressure, but specify a larger image size. This image will be put in file daybarometer_big.png.
[[[daybarometer_big]]] image_width = 600 image_height = 360 [[[[barometer]]]]
Here is part of the [ImageGenerator] section from the Standard skin.conf
[ImageGenerator] image_width = 300 image_height = 180 ... plot_type = line # default to line, can be overridden at any level aggregate_type = none # default to none, can be overridden at any level width = 1 # default to skinny lines time_length = 86400 # == 24 hours [[day_images]] x_label_format = %H:%M bottom_label_format = %m/%d/%y %H:%M time_length = 97200 # == 27 hours [[[daybarometer]]] [[[[barometer]]]] [[[dayrain]]] # Make sure the y-axis increment is at least 0.02 for the rain plot yscale = None, None, 0.02 plot_type = bar [[[[rain]]]] aggregate_type = sum aggregate_interval = 3600 label = Rain (hourly total) [[[daywinddir]]] # Hardwire in the y-axis scale for wind direction yscale = 0.0, 360.0, 45.0 [[[[windDir]]]] [[[daywindvec]]] [[[[windvec]]]] plot_type = vector [[[dayradiation]]] [[[[radiation]]]] [[week_images]] x_label_format = %d bottom_label_format = %m/%d/%y %H:%M time_length = 604800 # == 7 days aggregate_type = avg aggregate_interval = 3600 [[[weekbarometer]]] [[[[barometer]]]] [[[weekrain]]] yscale = None, None, 0.02 plot_type = bar [[[[rain]]]] aggregate_type = sum aggregate_interval = 86400 label = Rain (daily total) [[[weekwinddir]]] yscale = 0.0, 360.0, 45.0 [[[[windDir]]]] [[[weekwindvec]]] [[[[windvec]]]] plot_type = vector [[[weekradiation]]] [[[[radiation]]]] ...
The Standard skin defines many different types of plots. Note that each plot will be created only if there are data that match the definition. For example, if a station does not produce radiation data, no radiation images will be generated. This means that a single report can be used for many different stations, even if the stations have different sensors or capabilities.
This section defines the generators that should be run as well as options for specific generators.
generator_list
This option controls which generators get run for this skin. It is a comma separated list. The generators will be run in this order.
Here is the [Generators] section from the Standard skin.conf
[Generators] generator_list = weewx.cheetahgenerator.CheetahGenerator, weewx.imagegenerator.ImageGenerator, weewx.reportengine.CopyGenerator
The Standard skin uses three generators: CheetahGenerator, ImageGenerator, and CopyGenerator.
Weewx has been designed to make localization fairly straightforward. What follows is a guide to localizing to a non-English language and/or locale.
First, you will need to go through the templates and translate to your target language. Obvious text strings such as "Current Weather Conditions" will need to be translated.
Next, you will need to go through skin.conf and modify any labels to follow local conventions.
The hemisphere abbreviations may have to be changed:
hemisphere = N, S, E, W
The wind ordinal directions may have to be changed:
[[Ordinates]] # The ordinal directions. The last one should be for no wind direction directions = N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW, N/A
Don't forget the moon phases:
moon_phases = New, Waxing crescent, First quarter, Waxing gibbous, Full, Waning gibbous, Last quarter, Waning crescent
Most of the unit labels either follow ISO conventions, or are unlikely to be used outside English speaking countries (an example would be "foot"). But, there are a few exceptions, used to label the weewx and server "uptimes":
day = " day", " days" hour = " hour", " hours" minute = " minute", " minutes" second = " second", " seconds"
You will probably want to change the generic labels used for the observation types:
barometer = Barometer dewpoint = Dew Point heatindex = Heat Index inHumidity = Inside Humidity inTemp = Inside Temperature outHumidity = Outside Humidity outTemp = Outside Temperature radiation = Radiation rain = Rain rainRate = Rain Rate rxCheckPercent = ISS Signal Quality windDir = Wind Direction windGust = Gust Speed windGustDir = Gust Direction windSpeed = Wind Speed windchill = Wind Chill windgustvec = Gust Vector windvec = Wind Vector extraTemp1 = Pond Temperature
Finally, you will need to set the environment variable LANG to reflect your locale. For example, assuming you set
$ export LANG=es_ES.UTF-8
before running weewx, then the local Spanish names for days of the week and months of the year will be used. The decimal point for numbers will also be modified appropriately.
This is an advanced topic intended for those who wish to try their hand at extending the internal engine in weewx. You should have a passing familiarity with Python or, at least, be willing to learn it.
Please note that the service engine is likely to change in future versions!
At a high level, weewx consists of an engine that is responsible for managing a set of services. A service consists of a Python class which binds its member functions to various events. The engine arranges to have the bound member function called when a specific event happens, such as a new LOOP packet arriving.
To customize, you can
See the table Default services above for a list of the services that are normally run.
The service weewx.wxengine.StdPrint prints out new LOOP and archive packets to the console when they arrive. By default, it prints out the entire record, which generally includes a lot of possibly distracting information and can be rather messy. Suppose you do not like this, and want it to print out only the time, barometer reading, and the outside temperature whenever a new LOOP packet arrives. This could be done by subclassing the default print service StdPrint and overriding member function new_loop_packet().
Create the file $BIN_ROOT/user/myprint.py:
from weewx.wxengine import StdPrint from weeutil.weeutil import timestamp_to_string class MyPrint(StdPrint): # Override the default new_loop_packet member function: def new_loop_packet(self, event): packet = event.packet print "LOOP: ", timestamp_to_string(packet['dateTime']), "BAR=", packet.get('barometer', 'N/A'), "TEMP=", packet.get('outTemp', 'N/A')
This service substitutes a new implementation for the member function new_loop_packet. This implementation prints out the time, then the barometer reading (or 'N/A' if it is not available) and the outside temperature (or 'N/A').
You then need to specify that your print service class should be loaded instead of the default StdPrint service. This is done by substituting your service name for StdPrint in service_list, located in [Engines][[WxEngine]]:
[Engines]
[[WxEngine]]
service_list = weewx.wxengine.StdConvert, weewx.wxengine.StdCalibrate, weewx.wxengine.StdQC, weewx.wxengine.StdArchive, weewx.wxengine.StdTimeSynch, user.myprint.MyPrint, weewx.wxengine.StdRESTful, weewx.wxengine.StdReport
Note that the service_list must be all on one line. Unfortunately, the parser ConfigObj does not allow options to be continued on to following lines.
Suppose there is no service that can be easily customized for your needs. In this case, a new one can easily be created by subclassing off the abstract base class StdService, and then adding the functionality you need. Here is an example that implements an alarm that sends off an email when an arbitrary expression evaluates True. This example is included in the standard distribution in directory $BIN_ROOT/examples.
File examples/alarm.py:
import time import smtplib from email.mime.text import MIMEText import threading import syslog import weewx from weewx.wxengine import StdService from weeutil.weeutil import timestamp_to_string, option_as_list # Inherit from the base class StdService: class MyAlarm(StdService): """Custom service that sounds an alarm if an arbitrary expression evaluates true""" def __init__(self, engine, config_dict): # Pass the initialization information on to my superclass: super(MyAlarm, self).__init__(engine, config_dict) # This will hold the time when the last alarm message went out: self.last_msg_ts = 0 try: # Dig the needed options out of the configuration dictionary. # If a critical option is missing, an exception will be thrown and # the alarm will not be set. self.expression = config_dict['Alarm']['expression'] self.time_wait = int(config_dict['Alarm'].get('time_wait', 3600)) self.smtp_host = config_dict['Alarm']['smtp_host'] self.smtp_user = config_dict['Alarm'].get('smtp_user') self.smtp_password = config_dict['Alarm'].get('smtp_password') self.SUBJECT = config_dict['Alarm'].get('subject', "Alarm message from weewx") self.FROM = config_dict['Alarm'].get('from', 'alarm@weewx.com') self.TO = option_as_list(config_dict['Alarm']['mailto']) syslog.syslog(syslog.LOG_INFO, "alarm: Alarm set for expression: \"%s\"" % self.expression) # If we got this far, it's ok to start intercepting events: self.bind(weewx.NEW_ARCHIVE_RECORD, self.newArchiveRecord) # NOTE 1 except Exception, e: syslog.syslog(syslog.LOG_INFO, "alarm: No alarm set. %s" % e) def newArchiveRecord(self, event): """Gets called on a new archive record event.""" # To avoid a flood of nearly identical emails, this will do # the check only if we have never sent an email, or if we haven't # sent one in the last self.time_wait seconds: if not self.last_msg_ts or abs(time.time() - self.last_msg_ts) >= self.time_wait : # Get the new archive record: record = event.record # Evaluate the expression in the context of the event archive record. # Sound the alarm if it evaluates true: if eval(self.expression, None, record): # NOTE 2 # Sound the alarm! # Launch in a separate thread so it doesn't block the main LOOP thread: t = threading.Thread(target = MyAlarm.soundTheAlarm, args=(self, record)) t.start() # Record when the message went out: self.last_msg_ts = time.time() def soundTheAlarm(self, rec): """This function is called when the given expression evaluates True.""" # Get the time and convert to a string: t_str = timestamp_to_string(rec['dateTime']) # Log it in the system log: syslog.syslog(syslog.LOG_INFO, "alarm: Alarm expression \"%s\" evaluated True at %s" % (self.expression, t_str)) # Form the message text: msg_text = "Alarm expression \"%s\" evaluated True at %s\nRecord:\n%s" % (self.expression, t_str, str(rec)) # Convert to MIME: msg = MIMEText(msg_text) # Fill in MIME headers: msg['Subject'] = self.SUBJECT msg['From'] = self.FROM msg['To'] = ','.join(self.TO) # Create an instance of class SMTP for the given SMTP host: s = smtplib.SMTP(self.smtp_host) try: # Some servers (eg, gmail) require encrypted transport. # Be prepared to catch an exception if the server # doesn't support it. s.ehlo() s.starttls() s.ehlo() syslog.syslog(syslog.LOG_DEBUG, " **** using encrypted transport") except smtplib.SMTPException: syslog.syslog(syslog.LOG_DEBUG, " **** using unencrypted transport") try: # If a username has been given, assume that login is required for this host: if self.smtp_user: s.login(self.smtp_user, self.smtp_password) syslog.syslog(syslog.LOG_DEBUG, " **** logged in with user name %s" % (self.smtp_user,)) # Send the email: s.sendmail(msg['From'], self.TO, msg.as_string()) # Log out of the server: s.quit() except Exception, e: syslog.syslog(syslog.LOG_ERR, "alarm: SMTP mailer refused message with error %s" % (e,)) raise # Log sending the email: syslog.syslog(syslog.LOG_INFO, " **** email sent to: %s" % self.TO)
This service expects all the information it needs to be in the configuration file weewx.conf in a new section called [Alarm]. So, add the following lines to your configuration file:
[Alarm] expression = "outTemp < 40.0" time_wait = 3600 smtp_host = smtp.mymailserver.com smtp_user = myusername smtp_password = mypassword mailto = auser@adomain.com, anotheruser@someplace.com from = me@mydomain.com subject = "Alarm message from weewx!"
There are two important points to be noted in this example, each marked with a "NOTE" flag in the code.
Another example expression could be:
expression = "outTemp < 32.0 and windSpeed > 10.0"
In this case, the alarm is sounded if the outside temperature drops below freezing and the wind speed is greater than 10.0.
Option time_wait is used to avoid a flood of nearly identical emails. The new service will wait this long before sending another email out.
Email will be sent through the SMTP host specified by option smtp_host. The recipient(s) are specified by the comma separated option mailto.
Many SMTP hosts require user login. If this is the case, the user and password are specified with options smtp_user and smtp_password, respectively.
The last two options, "from" and "subject" are optional. If not supplied, weewx will supply something sensible. Note, however, that some mailers require a valid "from" email address and the one weewx supplies may not satisfy its requirements.
To make this all work, you must tell the engine to load this new service. This is done by adding your service name to the list service_list, located in [Engines][[WxEngine]]:
[Engines]
[[WxEngine]]
service_list = weewx.wxengine.StdConvert, weewx.wxengine.StdCalibrate, weewx.wxengine.StdQC, weewx.wxengine.StdArchive, weewx.wxengine.StdTimeSynch, weewx.wxengine.StdPrint, weewx.wxengine.StdRESTful, weewx.wxengine.StdReport, examples.alarm.MyAlarm
Note that the service_list must be all on one line. Unfortunately, the parser ConfigObj does not allow options to be continued on to following lines.
In addition to the example above, the distribution also includes a low-battery alarm (lowBattery.py), which is similar, except that it intercepts LOOP events (instead of archiving events).
For most users the default database will work just fine. It has the added advantage of being compatible with the wview database. Nevertheless, there may be occasions where you may want to add a SQL type to your database, or change its unit system. This section shows you how to do this, using the utility $BIN_ROOT/wee_config_database.
This utility also has the ability to check a sqlite version of the archive database for embedded strings (where a float is expected).
Before starting, it's worth running the utility with the --help flag to see how it is used:
$BIN_ROOT/wee_config_database --help
This will result in an output that looks something like this:
Usage: wee_config_database: [config_path] [--help] [--create-database] [--create-stats] [--reconfigure] [--backfill-stats] [--string-check] [--fix] Configure the weewx databases. Most of these functions are handled automatically by weewx, but they may be useful as a utility in special cases. In particular, the 'reconfigure' option can be useful if you decide to add or drop data types from the database schema or change unit systems. Options: -h, --help show this help message and exit --config=FILE use configuration file FILE --create-archive Create the archive database. --create-stats Create the statistical database. --reconfigure Create a new archive database using configuration information found in the configuration file. In particular, the new database will use the unit system found in option [StdConvert][target_unit]. It will use the schema found in './bin/user/schemas.py'. The new database will have the same name as the old database, with a '_new' on the end. --backfill-stats Backfill the statistical database using the archive database --string-check Check a sqlite version of the archive database for embedded strings in it. --fix If a string is found, fix it. If you are using the MySQL database it is assumed that you have the appropriate permissions for the requested operation.
Suppose you have installed an electric meter at your house and you wish to correlate electrical usage with the weather. The meter has some sort of connection to your computer, allowing you to download the consumption. At the end of every archive interval you want to sample the meter for the electricity consumed during the interval, then store the results in the archive database, along with the weather data. How would you do this?
First, you would write a custom service that retrieves the electrical consumption data and adds it to the archive record. See the section Customizing the weewx service engine for details on how to write a custom service. However, when you are done it will look something like this:
from weewx.wxengine import StdService class AddElectricity(StdService): def __init__(self, engine, config_dict): # Initialize my superclass first: super(AddElectricity, self).__init__(engine, config_dict) # Bind to any new archive record events: self.bind(weewx.NEW_ARCHIVE_RECORD, self.new_archive_packet) def new_archive_packet(self, event): (code that downloads the consumption data from the connection to the meter) event.record['electricity'] = retrieved_value
This adds a new key 'electricity' to the record dictionary and sets it equal to some value. As an aside, if you do something like this, you would want to make sure that the code to retrieve the current electrical consumption does not delay very long so it does not slow down the main loop. If it's going to cause a delay of more than a couple seconds you might want to put it in a separate thread and feed the results to AddElectricity through a queue.
As usual, you would add your new service to the option service_list in weewx.conf, making sure it appears before StdArchive so your new value is inserted into the record before the data is archived.
So, now you have created a new observation type, 'electricity'. Trouble is, there is no corresponding type in the schema of the SQL database and, therefore, it won't be stored there. How would you add such a type?
Here's our general strategy:
1. Add a new type to the schema. When creating a database the schema is obtained from file $BIN_ROOT/user/schemas.py. Take a look at it now. You will see a list called defaultArchiveSchema that holds all the observation names and their SQL types. It looks something like:
defaultArchiveSchema = [('dateTime', 'INTEGER NOT NULL UNIQUE PRIMARY KEY'), ('usUnits', 'INTEGER NOT NULL'), ('interval', 'INTEGER NOT NULL'), ('barometer', 'REAL'), ('pressure', 'REAL'), ('altimeter', 'REAL'), ('inTemp', 'REAL'), ('outTemp', 'REAL'), ... ('inTempBatteryStatus', 'REAL')]
Let's modify it to add our new type, 'electricity'. Now it looks like this:
defaultArchiveSchema = [('dateTime', 'INTEGER NOT NULL UNIQUE PRIMARY KEY'),
('usUnits', 'INTEGER NOT NULL'),
('interval', 'INTEGER NOT NULL'),
('barometer', 'REAL'),
('pressure', 'REAL'),
('altimeter', 'REAL'),
('inTemp', 'REAL'),
('outTemp', 'REAL'),
...
('electricity', 'REAL'),
('inTempBatteryStatus', 'REAL')]
The new line has been highlighted .
2. Check permissions. The reconfiguration utility will create a new database with the same name as the old, except with the suffix '_new' attached to the end. Make sure you have the necessary permissions to do this. In particular, if you are using MySQL, you will need 'CREATE' privileges.
3. Run wee_config_database. Now run the utility wee_config_database with the --reconfigure option and the path to the configuration file:
$BIN_ROOT/wee_config_database --reconfigure $CONFIG_ROOT/weewx.conf
This will create a new database (nominally, weewx.sdb_new if you are using sqlite, weewx_new if you are using MySQL) using the new schema and populate it with data from the old database.
4. Shuffle the databases. Now arrange things so weewx can find the new database.
Warning!
Make a backup of the data before doing any of the next steps!
You can either shuffle the databases around so the new database has the same name as the old database, or edit weewx.conf to use the new database name. To do the former:
For sqlite:
cd $SQLITE_ROOT
mv weewx.sdb_new weewx.sdb
For MySQL:
mysql -u <username> --password=<mypassword> mysql> DROP DATABASE weewx; # Drops the old database mysql> CREATE DATABASE weewx; # Create a new one with the same name mysql> RENAME TABLE weewx_new.archive TO weewx.archive; # Rename to the nominal name
5. Modify the stats database. At this point, you can use the new observation type in the plots. However, if you wish to use it in the statistical summaries, you will also have to add it to the stats database. To do this, add the type to the Python list stats_types, which can be found in $BIN_ROOT/user/schemas.py, so it reads something like this:
stats_types = ['barometer', 'inTemp', 'outTemp',
'inHumidity', 'outHumidity',
'rainRate', 'rain', 'dewpoint', 'windchill', 'heatindex', 'ET',
'radiation', 'UV', 'extraTemp1', 'rxCheckPercent', 'wind',
'electricity']
Now delete the stats database (nominally stats.sdb for sqlite, stats for MySQL). Weewx will automatically rebuild it, including your new type.
Now you've added a new type. How do you use it?
Pretty much like any other type. For example, to do a plot of the month's electric consumption, totaled by day, add this section to the [[month_images]] section of skin.conf:
[[[monthelectric]]] [[[[electricity]]]] aggregate_type = sum aggregate_interval = 86400 label = Electric consumption (daily total)
This will cause the generation of an image monthelectric.png, showing a plot of each day's consumption for the past month.
If you wish to use the new type in the templates, it will be available using the same syntax as any other type. Here are some other tags that might be useful:
Tag | Meaning |
$day.electricity.sum | Total consumption since midnight |
$year.electricity.sum | Total consumption since the first of the year |
$year.electricity.max | The most consumed during any archive period |
$year.electricity.maxsum | The most consumed during a day |
$year.electricity.maxsumtime | The day it happened. |
$year.electricity.sum_ge(5.0) | The number of days where more than 5.0 kWH of energy was consumed. |
Normally, data is stored in the databases using US Customary units and, normally, you don't care --- data can always be displayed using any units you choose. It's an "implementation detail." Nevertheless, there may be special situations where you wish to store the data in Metric units. For example, you may need to allow direct programmatic access to the databases from another piece of software that expects metric units.
Weewx does not allow you to change the database unit system midstream. You can't start with one unit system then, in the middle of the database, switch to another. See the section [StdConvert] in the Weewx User's Guide. However, you can reconfigure the database by coping it to a new database, performing the unit conversion along the way. You then use this new database.
The steps are pretty much the same as Adding a New Type to the ArchiveDatabase, described above.
1. Modify weewx.conf. Edit the configuration file to change option target_unit in section [StdConvert] to reflect your choice. If you are switching to metric units, the option will look like
[StdConvert] target_unit = METRICWX # Alternative: METRIC
2. Check permissions. The reconfiguration utility will create a new database with the same name as the old, except with the suffix '_new' attached to the end. Make sure you have the necessary permissions to do this. In particular, if you are using MySQL, you will need 'CREATE' privileges.
3. Run wee_config_database. Now run the utility wee_config_database with the --reconfigure option:
$BIN_ROOT/wee_config_database --reconfigure $CONFIG_ROOT/weewx.conf
This will create a new database (nominally, weewx.sdb_new if you are using sqlite, weewx_new if you are using MySQL), using the schema found in $BIN_ROOT/user/schemas.py, and populate it with data from the old database, while performing the unit conversion.
4. Shuffle the databases. This is identical to the description above.
5. Recreate the stats database. Delete the stats database, then let weewx regenerate it. It will use the new unit system.
Naturally, this is an advanced topic but, nevertheless, I'd like to encourage any Python wizards out there to give it a try. Of course, I have selfish reasons for this: I don't want to have to buy every weather station ever invented! It's expensive, and my roof would look like a weather station farm.
Here's the general strategy for doing a port.
Inherit from the abstract base class weewx.abstractstation.AbstractStation. Try to implement as many of its methods as you can. At the very minimum, you must implement the first two methods, hardware_name and genLoopPackets.
Return a string with a short nickname for the hardware, such as "ACME X90"
This should be a generator function that yields loop packets, one after another. Don't worry about stopping it: the engine will do this when an archive record is due. A "loop packet" is a dictionary. At the very minimum it must contain keys for the observation time and for the units used within the packet.
dateTime | The time of the observation in unix epoch time. |
usUnits | The unit system used. weewx.US for US customary, weewx.METRICWX, or weewx.METRIC for metric. See the file units.py, dictionaries USUnits, MetricWXUnits, and MetricUnits for the exact definition of each. |
Then include any observation types you have in the dictionary. Every packet need not contain the same set of observation types. Different packets can use different unit systems, but all observations within a packet must use the same unit system. If your hardware has an error and you don't have a value, you can either leave it out of the dictionary or (preferred) set its value to None.
A couple of observation types are tricky. In particular, rain. Generally, weewx expects to see a packet with the amount of rain that fell in that packet period included as observation 'rain'. It then sums up all the values to get the total rainfall and emits that in the archive record. If your hardware does not provide this value, you might have to infer it from changes in whatever value it provides, for example changes in the daily or monthly rainfall. I know this is not the best solution, but it is the most general solution. Any alternatives are welcome!
Wind is another tricky one. It is actually broken up into four different observations: 'windSpeed', 'windDir', 'windGust', and 'windGustDir'. Supply as many as you can. The directions should be compass directions in degrees (0=North, 90=East, etc.).
Be careful when reporting pressure. There are three observations related to pressure. Some stations report only the station pressure, others calculate and report sea level pressures.
pressure | The Station Pressure (SP), which is the raw, absolute pressure measured by the station. This is the true barometric pressure for the station. |
barometer | The Sea Level Pressure (SLP) obtained by correcting the Station Pressure for altitude and local temperature. This is the pressure reading most commonly used by meteorologist to track weather systems at the surface, and this is the pressure that is uploaded to weather services by weewx. It is the station pressure reduced to mean sea level using local altitude and local temperature. |
altimeter | The Altimeter Setting (AS) obtained by correcting the Station Pressure for altitude. This is the pressure reading most commonly heard in weather reports. It is not the true barometric pressure of a station, but rather the station pressure reduced to mean sea level using altitude and an assumed temperature average. |
If your hardware does not have an archive record logger, then weewx can do the record generation for you. It will automatically collect all the types it sees in your loop packets then emit a record with the averages (in some cases the sum or max value) of all those types. If it doesn't see a type, then it won't appear in the emitted record. If your hardware does have a logger, then you should implement method genArchiveRecords as well. It should be a generator function that returns all the records since a given time.
If your hardware has an onboard clock and supports reading the time from it, then you may want to implement this method. It takes no argument. It should return the time in Unix Epoch Time.
If your hardware has an onboard clock supports setting it, then you may want to implement this method. It takes no argument and does not need to return anything.
If the driver needs to close a serial port, terminate a thread, close a database, or perform any other activity before the application terminates, then you must supply this function. Weewx will call it if it needs to shut down your console (usually in the case of an error).
This is a factory function that returns an instance of your driver. It has two arguments: the configuration dictionary, and a reference to the weewx engine.
You then include a new section in the configuration file weewx.conf that includes any options your driver needs. It should also include an entry 'driver' that points to where your driver can be found. Set option station_type to your new section type and your driver will be loaded.
Before starting, take a look at the simulator code in bin/weewx/drivers/simulator.py. It's dirt simple and you can easily play with it. Many people have successfully used it as a starting point for writing their own custom driver.
The next most complicated is the driver for the WMR100 series, located in wmr100.py. It reads from a USB port and has to do some tricky buffer decoding. Nevertheless, it's general architecture is fairly straightforward.
The driver for the Vantage series is by far the most complicated. It actually multi-inherits from not only AbstractStation, but also StdService. That is, it also participates in the engine as a service.
Naturally, there are a lot of subtleties that I've glossed over in this high-level description. If you're game, give it a try — I'm happy to help you out!
Now that you have made some customizations, you might want to share those changes with other weewx users. Put your customizations into an extension to make installation, removal, and distribution easier.
Customizations typically fall into one of these categories:
Extensions are a way to package one or more customizations so they can be installed and distributed as a functional group.
Here are a few guidelines for creating extensions:
The structure of an extension mirrors that of weewx itself. If the customizations include a skin, the extension will have a skins directory. If the customizations include python code, the extension will have a bin/user directory.
Each extension should also include:
For example, here is the structure of a skin called basic:
basic/ basic/changelog basic/install.py basic/readme.txt basic/skins/ basic/skins/basic/ basic/skins/basic/basic.css basic/skins/basic/current.inc basic/skins/basic/favicon.ico basic/skins/basic/hilo.inc basic/skins/basic/index.html.tmpl basic/skins/basic/skin.conf
Here is the structure of a search list extension called xstats:
xstats/ xstats/changelog xstats/install.py xstats/readme.txt xstats/bin/ xstats/bin/user/ xstats/bin/user/xstats.py
See the extensions directory of the weewx source for examples.
To distribute an extension, simply create a compressed archive of the extension directory.
For example, create the compressed archive for the basic skin like this:
tar cvfz basic.tar.gz basic
The ExtensionInstaller is built in to setup.py. It can install new extensions, uninstall installed extensions, or enumerate installed extensions.
To install an extension:
./setup.py --extension --install extensions/basic ./setup.py --extension --install basic.tar.gz
To uninstall an extension:
./setup.py --extension --uninstall basic
To list installed extensions:
./setup.py --extension --list
Other options:
./setup.py --help --extension Usage: setup.py --extension --install (filename|directory) --uninstall extension_name --list install/remove/list extensions to weewx Options: -h, --help show this help message and exit --extension --install=NAME install extension from file or directory --uninstall=NAME uninstall extension --list list installed extensions --layout=LAYOUT layout is deb, rpm, or py --tmpdir=DIR temporary directory --dryrun print what would happen but do not do it --verbosity=N how much status to spew, 0-3
The ExtensionInstaller will make only the following changes:
The ExtensionInstaller makes a copy of any file or directory that it modifies or replaces.
If weewx was installed using setup.py, the ExtensionInstaller depends upon setup.cfg to locate the weewx installation. If weewx was installed using a DEB or RPM package, setup.cfg is not used.
The ExtensionInstaller creates a directory called installer in the user directory. The contents of the installer directory are used to enumerate and uninstall extensions.
Archive types are weather observations that have come from your instrument and been stored in the archive database, a SQL database. They represent the current conditions as of some time. They are available to be used in two places:
The following table shows all the possible archive types and whether they can be used in tag $current or in a plot. Note that just because a type appears in the table does not necessarily mean that it is available for your station setup. That would depend on whether your instrument supports the type.
Archive Type | SQL Type (appears in archive database) |
Can be used in plots |
Can be used in tag $current |
altimeter | X | X | X |
barometer | X | X | X |
consBatteryVoltage | X | X | X |
dateTime | X | X (represents current time) | |
dewpoint | X | X | X |
ET | X | X | X |
extraHumid1 | X | X | X |
extraHumid2 | X | X | X |
extraTemp1 | X | X | X |
extraTemp2 | X | X | X |
extraTemp3 | X | X | X |
hail | X | X | X |
hailRate | X | X | X |
heatindex | X | X | X |
heatingTemp | X | X | X |
heatingVoltage | X | X | X |
inHumidity | X | X | X |
inTemp | X | X | X |
inTempBatteryStatus | X | X | X |
interval | X | X | X |
leafTemp2 | X | X | X |
leafWet2 | X | X | X |
outHumidity | X | X | X |
outTemp | X | X | X |
outTempBatteryStatus | X | X | X |
pressure | X | X | X |
radiation | X | X | X |
rain | X | X | X |
rainBatteryStatus | X | X | X |
rainRate | X | X | X |
referenceVoltage | X | X | X |
rxCheckPercent | X | X | X |
soilMoist1 | X | X | X |
soilMoist2 | X | X | X |
soilMoist3 | X | X | X |
soilMoist4 | X | X | X |
soilTemp1 | X | X | X |
soilTemp2 | X | X | X |
soilTemp3 | X | X | X |
soilTemp4 | X | X | X |
supplyVoltage | X | X | X |
txBatteryStatus | X | X | X |
usUnits | X | X | X |
UV | X | X | X |
windvec | X (special vector type) | ||
windBatteryStatus | X | X | X |
windDir | X | X | X |
windGust | X | X | X |
windGustDir | X | X | X |
windSpeed | X | X | X |
windchill | X | X | X |
The table below lists all the unit groups, their members, which units are options for the group, and what the defaults are for each standard unit system.
Group | Members | Unit options | US | METRICWX | METRIC |
group_altitude | altitude | foot meter |
foot | meter | meter |
group_degree_day | cooldeg heatdeg |
degree_F_day degree_C_day |
degree_F_day | degree_C_day | degree_C_day |
group_direction | gustdir vecdir windDir windGustDir |
degree_compass | degree_compass | degree_compass | degree_compass |
group_interval | interval | minute | minute | minute | minute |
group_moisture | soilMoist1 soilMoist2 soilMoist3 soilMoist4 |
centibar | centibar | centibar | centibar |
group_percent | extraHumid1 extraHumid2 inHumidity outHumidity rxCheckPercent |
percent | percent | percent | percent |
group_pressure | barometer altimeter pressure |
inHg mbar hPa |
inHg | mbar | mbar |
group_radiation | UV radiation |
watt_per_meter_squared | watt_per_meter_squared | watt_per_meter_squared | watt_per_meter_squared |
group_rain | rain ET hail |
inch cm mm |
inch | mm | cm |
group_rainrate | rainRate hailRate |
inch_per_hour cm_per_hour mm_per_hour |
inch_per_hour | mm_per_hour | cm_per_hour |
group_speed | wind windGust windSpeed windgustvec windvec |
mile_per_hour km_per_hour knot meter_per_second |
mile_per_hour | meter_per_second | km_per_hour |
group_speed2 | rms vecavg |
mile_per_hour2 km_per_hour2 knot2 meter_per_second2 |
mile_per_hour2 | meter_per_second2 | km_per_hour2 |
group_temperature | dewpoint extraTemp1 extraTemp2 extraTemp3 heatindex heatingTemp inTemp leafTemp1 leafTemp2 outTemp soilTemp1 soilTemp2 soilTemp3 soilTemp4 windchill |
degree_F degree_C |
degree_F | degree_C | degree_C |
group_time | dateTime | unix_epoch dublin_jd |
unix_epoch | unix_epoch | unix_epoch |
group_uv | UV | uv_index | uv_index | uv_index | uv_index |
group_volt | consBatteryVoltage heatingVoltage referenceVoltage supplyVoltage |
volt | volt | volt | volt |
group_NONE | NONE | NONE | NONE | NONE | NONE |
Most of the templates are devoted to reporting statistical types, such as temperature, wind, or rainfall, using various aggregates, such as min, max, or sum. These are called aggregations, because they are a summary of lots of underlying data. However, only certain aggregates make sense for certain statistical types. For example, heat degree days is defined on a daily basis, so while the day's average temperature is meaningful, the day's heating degree days do not.
The following table defines which aggregates are available to be used in your template for which statistical types (assuming your station supports them and you have specified that it be stored in your stats database. See section [Stats] in the weewx.conf configuration file).
Type | min | mintime | max | maxtime | avg | sum | rms | vecavg | vecdir |
barometer | X | X | X | X | X | ||||
inTemp | X | X | X | X | X | ||||
outTemp | X | X | X | X | X | ||||
inHumidity | X | X | X | X | X | ||||
outHumidity | X | X | X | X | X | ||||
wind | X | X | X | X | X | X | X | X | |
rain | X | X | X | X | X | X | |||
dewpoint | X | X | X | X | X | ||||
windchill | X | X | X | X | X | ||||
heatindex | X | X | X | X | X | ||||
heatdeg | X | X | |||||||
cooldeg | X | X | |||||||
ET | X | X | X | X | X | ||||
radiation | X | X | X | X | X | ||||
UV | X | X | X | X | X | ||||
extraTemp1 extraTemp2 extraTemp3 |
X | X | X | X | X | ||||
soilTemp1 soilTemp2 soilTemp3 |
X | X | X | X | X | ||||
leafTemp1 leafTemp2 |
X | X | X | X | X | ||||
extraHumid1 extraHumid2 |
X | X | X | X | X | ||||
soilMoist1 soilMoist2 soilMoist3 soilMoist4 |
X | X | X | X | X | ||||
leafWet1 leafWet2 |
X | X | X | X | X | ||||
rxCheckPercent | X | X | X | X | X |
The following tables show which data are provided by the station hardware and which are calculated by weewx.
H indicates data provided by Hardware
D indicates data calculated by the Driver
Observation | Loop | Archive |
barometer | H | H |
pressure | D | D |
altimeter | D | D |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
windGust | H | H |
windGustDir | H | H |
rainRate | H | H |
rain | H | H |
dewpoint | D | D |
windchill | D | D |
heatindex | D | D |
radiation | H | H |
UV | H | H |
extraTemp1 | H | H |
extraTemp2 | H | H |
extraTemp3 | H | H |
extraTemp4 | H | |
extraTemp5 | H | |
extraTemp6 | H | |
extraTemp7 | H | |
soilTemp1 | H | H |
soilTemp2 | H | H |
soilTemp3 | H | H |
soilTemp4 | H | H |
leafTemp1 | H | H |
leafTemp2 | H | H |
leafTemp3 | H | H |
leafTemp4 | H | H |
extraHumid1 | H | H |
extraHumid2 | H | H |
extraHumid3 | H | |
extraHumid4 | H | |
extraHumid5 | H | |
extraHumid6 | H | |
extraHumid7 | H | |
soilMoist1 | H | H |
soilMoist2 | H | H |
soilMoist3 | H | H |
soilMoist4 | H | H |
leafWet1 | H | H |
leafWet2 | H | H |
leafWet3 | H | H |
leafWet4 | H | H |
txBatteryStatus | H | H |
consBatteryVoltage | H | H |
Observation | Loop | Archive |
barometer | D | |
pressure | H | |
altimeter | D | |
inTemp | H | |
outTemp | H | |
inHumidity | H | |
outHumidity | H | |
windSpeed | H | |
windDir | H | |
windGust | H | |
rainRate | H | |
rain | H | |
dewpoint | D | |
windchill | D | |
heatindex | D | |
extraTemp1 | H | |
extraTemp2 | H | |
extraTemp3 | H | |
UV | H | |
inTempBatteryStatus | H | |
outTempBatteryStatus | H | |
rainBatteryStatus | H | |
windBatteryStatus | H | |
uvBatteryStatus | H |
Each packet contains a subset of all possible readings. For example, a temperature packet contains extraTempN and batteryStatusN, a rain packet contains totalRain and rainRate.
Observation | Loop | Archive |
barometer | H | H |
pressure | D | D |
altimeter | H | H |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
windGust | H | H |
rainRate | H | H |
rain | H | H |
dewpoint | D | D |
windchill | H/D | H/D |
heatindex | H/D | H/D |
extraTemp1 | H | H |
extraTemp2 | H | H |
extraTemp3 | H | H |
extraHumid1 | H | H |
extraHumid2 | H | H |
extraHumid3 | H | H |
UV | H | H |
outTempBatteryStatus | H | |
rainBatteryStatus | H | |
windBatteryStatus | H | |
uvBatteryStatus | H |
Each packet contains a subset of all possible readings. For example, a temperature packet contains extraTempN and batteryStatusN, a rain packet contains totalRain and rainRate.
Observation | Loop | Archive |
barometer | H | |
pressure | H | |
altimeter | D | |
inTemp | H | |
outTemp | H | |
inHumidity | H | |
outHumidity | H | |
windSpeed | H | |
windDir | H | |
windGust | H | |
windGustDir | H | |
rainRate | H | |
rain | H | |
dewpoint | D | |
windchill | H | |
heatindex | D | |
extraTemp1 | H | |
extraTemp2 | H | |
extraTemp3 | H | |
extraHumid1 | H | |
extraHumid2 | H | |
extraHumid3 | H | |
UV | H | |
inTempBatteryStatus | H | |
outTempBatteryStatus | H | |
rainBatteryStatus | H | |
windBatteryStatus | H | |
batteryStatusTH1 | H | |
batteryStatusTH2 | H | |
batteryStatusTH3 | H | |
batteryStatusT1 | H | |
batteryStatusT2 | H | |
batteryStatusT3 | H |
Each packet contains a subset of all possible readings. For example, a temperature packet contains extraTempN and batteryStatusN, a rain packet contains totalRain and rainRate.
Observation | Loop | Archive |
barometer | D | D |
pressure | H | H |
altimeter | D | D |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
windGust | H | H |
windGustDir | D | D |
rainRate | D | D |
rain | H | H |
dewpoint | H/D | D |
windchill | H/D | D |
heatindex | H/D | D |
radiation | H | H |
UV | H | H |
The radiation and UV data are available only from 30xx stations.
Observation | Loop | Archive |
barometer | D | D |
pressure | H | H |
altimeter | D | D |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
windGust | D | D |
windGustDir | D | D |
rainRate | H | H |
rain | H | H |
dewpoint | D | D |
windchill | H/D | H/D |
heatindex | H/D | H/D |
Observation | Loop | Archive |
barometer | D | D |
pressure | H | H |
altimeter | D | D |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
windGust | H | H |
windGustDir | D | D |
rainRate | H | |
rain | H | H |
dewpoint | D | D |
windchill | H/D | H/D |
heatindex | H/D | H/D |
rxCheckPercent | H | |
windBatteryStatus | H | |
rainBatteryStatus | H | |
outTempBatteryStatus | H | |
inTempBatteryStatus | H |
Observation | Loop | Archive |
barometer | H | |
pressure | D | |
altimeter | D | |
inTemp | H | |
outTemp | H | |
inHumidity | H | |
outHumidity | H | |
windSpeed | H | |
windDir | H | |
windGust | H | |
windGustDir | D | |
rainRate | D | |
rain | H | |
dewpoint | D | |
windchill | H/D | |
heatindex | D | |
UV | H | |
extraTemp1 | H | |
extraTemp2 | H | |
extraTemp3 | H | |
extraTemp4 | H | |
extraHumid1 | H | |
extraHumid2 | H | |
extraHumid3 | H | |
extraHumid4 | H | |
txBatteryStatus | H | |
windBatteryStatus | H | |
rainBatteryStatus | H | |
outTempBatteryStatus | H | |
extraBatteryStatus1 | H | |
extraBatteryStatus2 | H | |
extraBatteryStatus3 | H | |
extraBatteryStatus4 | H |
Some stations support up to 5 remote temperature/humidity sensors. The UV data are available only with the optional solar radiation sensor.
Observation | Loop | Archive |
barometer | H | |
pressure | D | |
altimeter | D | |
inTemp | H | |
outTemp | H | |
inHumidity | H | |
outHumidity | H | |
windSpeed | H | |
windDir | H | |
rainRate | D | |
rain | H | |
dewpoint | D | |
windchill | D | |
heatindex | D |
The rain and rainRate are available only on stations with the optional rain bucket.
Pressure, inside temperature, and inside humidity data are not available on all types of Ultimeter stations.
Observation | Loop | Archive |
barometer | D | |
pressure | H | |
altimeter | D | |
inTemp | H | |
outTemp | H | |
inHumidity | H | |
outHumidity | H | |
windSpeed | H | |
windDir | H | |
rain | H | |
dewpoint | D | |
windchill | D | |
heatindex | D |
Observation | Loop | Archive |
barometer | D | D |
pressure | H | H |
altimeter | D | D |
inTemp | H | H |
outTemp | H | H |
inHumidity | H | H |
outHumidity | H | H |
windSpeed | H | H |
windDir | H | H |
rain | H | H |
dewpoint | D | D |
windchill | D | D |
heatindex | D | D |
radiation | H | H |
UV | H | H |
The radiation and UV data are available only with the optional solar radiation sensor.
© Copyright Tom Keffer