Air Quality Egg: Firmware/Software Update

Here’s the quickie summary. I reorganized the software on the Air Quality Egg to do the conversion math on the Nanode. Good news is this seems to be giving better results. The downside is it was a fairly broad impact, and I believe every board needs to be reprogrammed to get the system running at the latest baseline. I haven’t yet pushed the updated software and binaries up to Git Hub, but I’m running it and observing this feed, so far so good.

I’ve previously published instructions on how to reprogram the Nanodes using an FTDI-compatible programming cable in videos that are linked on the FAQ. I also plan to make a video that shows how to re-program the Sensor Shield and add-on modules, though that is admittedly not as easy as I’d like it to be for the average user.

I really want the fielded Eggs to get to latest version of the software though, so I’m willing to  reprogram units personally, at no additional cost, if you send them back to us and just pay for the round trip shipping. I know this is a real hassle for our international community members, with customs and all. If there are community members that are willing to be “local experts” for reproramming Eggs that would ease the process. If you are willing to be a volunteer in this capacity please let me know at support AT wickeddevice DOT com. It would be really great to have some local people help support Europe, Asia, and Australia.

Technical Details

Math Code Refactoring

The sensor shield code now responds with a record containing the following information in response to the EGG_BUS_SENSOR_BLOCK_RAW_VALUE request:

   uint32_t adc_value     :: the analog-to-digital conversion (ADC) count
   uint32_t low_side_r    :: the low side resistance associated with the ADC count
   uint32_t sensor_vcc    :: the high side voltage applied to the sensor divider circuit
   uint32_t adc_vcc       :: the reference voltage of the ADC
   uint32_t max_adc_value :: 2^(ADC # of bits), e.g. 2^10 = 1024 for a 10-bit ADC

Given this information, the Nanode computes the measured resistance as follows (using floating point math):

   vx = (adc_value / max_adc_value) * adc_vcc; // this is the voltage divider output voltage
   sensor_resistance = (sensor_vcc - vx) * low_side_r / vx;

And the value of the independent variable used for computing the interpolated response is calculated as sensor_resistance / R0, where the (mutable) calibration value R0 and the piece-wise linear response function table is retrieved from the sensor shield and applied as previously.

A few side note here:

• This refactoring effectively deprecates the EGG_BUS_SENSOR_BLOCK_MEASURED_INDEPENDENT field, as that quantity is now computed on the Nanode as a derived value. This is all obviously “hidden” in the EggBus library, but I did want to make that clear.
• I tried to keep enough information on the endpoint to allow for future flexibility. I could have included less information in the EGG_BUS_SENSOR_BLOCK_RAW_VALUE response, but it would be less generic. With the interface defined as above, a sensor endpoint could for example use a 24-bit ADC, or a different voltage reference, and so forth, without loss of generality.
• The terms may mean different things to different sensor endpoints, but they can rely on the Nanode to use them in downstream calculations as described above. The dust sensor, for example, has “occupancy” in “hundredths of a percent” as it’s raw value. To respect the calculation stream, the dust sensor now reports adc_value = 1, max_adc_value=1, adc_vcc=1, low_side_r=1, sensor_vcc=(occupancy + 1), and r0=100.

Data Refactoring

In the updated software, for each sensor there are three values reported to Cosm on every sample interval:

• Raw Value := the result of the sensor_resistance calculation above. Tagged with ”aqe:data_origin=raw”
• Computed Value := the result of the interpolation calculation derived from the Raw Value, R0, and the piece-wise linear transfer function. Tagged with ”aqe:data_origin=computed”
• R0 := reporting this to Cosm once calibration enters the picture we’ll need to recognize that this value may vary over time, but this should be a mostly constant. Tagged with ”aqe:data_origin=r0″

One thing to note, I’m changing the meaning of the raw value feed here. Where previously it reflected the “independent variable” (i.e. the quantity resistance / r0), it now reflects the sensor resistance.

Library Updates

There have been some updates to the Ethercard library in recent months. I’m not sure, but perhaps the latest library baseline addresses the issue causing some Air Quality Eggs to freeze up in the field, requiring a restart to resume publishing data to the web.

Air Quality Egg: Research Findings

I’ve been looking at a lot of the data being published and trends in questions on the forum, and there is a theme developing. The results are not consistent with expectations. We’re also now getting some reports from laboratory studies that are confirming there’s a problem. The collective information strongly suggests that the data the Air Quality Eggs are reporting is incongruent with the natural response of the sensors.

In light of this growing body of evidence, I built a much simplified reference circuit based on the sensor datasheets and recorded sensor readings along side the Egg for several hours. Sure enough, the reference circuit exhibited a fairly smooth / continuous response over time. The Egg did not. 

I decided the rational thing to do was to conduct a series of controlled experiments in an effort to establish the root cause of the variation between the Sensor Shield and the Reference Circuit behavior. This was a daunting proposition with so many potential variables in play, but I did my best to come up with some viable hypotheses. The fundamental differences between the reference circuit and the Sensor Shield / Air Quality Egg are:

• The Air Quality Egg had a fan (which affects airflow and electrical load), the reference circuit did not
• The Air Quality Egg was in an enclosure, the reference circuit was not
• The Sensor Shield dynamically controlled the heater voltage, the reference circuit did not
• The Sensor Shield dynamically selected the voltage divider, the reference circuit did not

For the first two bullets I did a series of tests that ultimately revealed the enclosure didn’t have much of an impact, but that disconnecting the fan from the Air quality Egg circuit has a very noticeable impact. As a quick aside, all of the following graphs have elapsed time in milliseconds on the x-axis and ADC counts on the y-axis. The following graph compares the response of the reference circuit, to an Air Quality Egg with a fan and enclosure, and to an Air Quality Egg with neither a fan nor an enclosure.

The difference is readily apparent. The data with the fan (red) has a bout 50 ADC counts of noise on it. The data with no fan (blue) in the circuit, however, still had a peculiarity of these randomly appearing spike discontinuities in the data. I made modified builds of the Sensor Shield firmware that took out the Heater Control algorithm, took out the Range Selection algorithm, and took out both.  The result of taking out both looked like this (with and without an enclosure) in comparison to the reference circuit.

Lo and behold, with the fan removed and the software sophistication removed, the Air Quality Egg response has neither noise nor spikes, and it seems to track the reference circuit quite well. I then added just the Heater Control algorithm back in and I got this result:

This confirms, in my view, that the Heater Control algorithm is the source of spiky-ness in the Air Quality Egg data. I’m currently studying the software intensely to come up with an improved version of the Sensor Shield firmware, and will post another update when I know more.

“Air Quality Egg” is now a trademark

We have followed in the footsteps of Arduino™ and received a trademark on the “Air Quality Egg”. This post explains why we did this, and it’s effects.

A quick note in case you are unfamiliar with Trademarks: This has no effect on the open source-ness of anything of the hardware and software. It is still 100% open source.

This blog post has a pretty good explanation why open source projects should trademark. A trademark prevents other people from calling something an “Air Quality Egg”. We did this for two reasons:
- It would be confusing to us if other people created a product with the same name.
- If might also confuse customers, and they might expect us to support all products called “Air Quality Egg”

Here is another good discussion on trademarks and open source.

Air Quality Egg status update

All,

It has been awhile. Here is an update on the Egg.

If you ordered:

• an egg with no add-ons (Just the basic egg),
• a shield only or
• a Kit,

Then we have shipped your order. If you don’t think we have, then please contact support AT wickeddevice.com

If you ordered an Egg with any add-ons (Ozone, VoC, etc), we have not shipped your egg yet, but will be doing so soon (starting next week).

If you ordered a radiation module, as noted in January, Joe Saveedra has been kind enough to design and help with delivering it.

Wicked Device announces strategic partnership with ITT Exelis

ITT Exelis (NYSE: XLS) and Wicked Device LLC have entered into a strategic partnership to explore innovative technology to provide real-time weather information at a hyper-local scale.

Wicked Device is developing an embedded system to integrate with Exelis’s Helios™ platform. The Helios platform enables traffic and other cameras to function as weather sensors, providing more accurate, real-time environmental intelligence for better decision-making and response.

The Helios platform integrates networks of surveillance cameras already in use to watch traffic, facility security and railroad assets for weather monitoring. It aggregates disparate images into a single source and uses algorithms and image processing to reveal actionable real-time environmental intelligence. Providing hyper-local weather data, versus regional data, to national and local agencies, utility companies, transportation services and first responders can help in their decision-making process, saving lives and resources.

AQE Bug: Ridiculously High Readings

Short Story

I’ve been reading reports of really high (some would say ‘ridiculous’) values being reported by Air Quality Eggs in recent weeks as they’ve started to reach their homes. I think I’ve figured it out, and I’ve pushed an update to the Remote Egg software (more specifically, the Egg Bus Arduino Library). Basically there was a bug in the ‘extrapolation’ case of the getSensorValue function. If you want to skip reading the rest and go patch your Egg with the fix, just go download the latest zip from Git-Hub and reload the software as described in the tutorial videos we recently produced <LINK HERE>.

Details

In EggBus.c in the getSensorValue function, the way it works is it searches a table of (x, y) samples from the sensor hosting device (i.e. the Egg Shield) that relates the “raw” value from the sensor to a computed value that is something we are more used to thinking about.  For example, the table might take us from measured resistance in ohms (the raw value of the NO2 sensor) to concentration in ppb (parts per billion). The way the Nanode uses this table is it look for the two rows in this table that the raw value falls between and performs a linear interpolation – this is the most common case we should expect to see. There are of course a few other ‘edge cases’ that need to be handled: namely (1) exact matches to a table row, (2) values that are smaller than the smallest value represented in the table, and (3) values hat are larger than the largest value in the table. Exact matches are trivial, as we just return the mapped value with no further maths applied. The other two cases are slightly less trivial, but the principle is to calculate the slope of the sampled curve at either extreme and then perform a linear extrapolation in the corresponding direction off the sampled curve.

The problem was in the handling of the third case above – an extrapolation above the highest value in the table. Notably:

float EggBus::getSensorValue(uint8_t sensorIndex){

  // some setup code omitted here
  while(getTableRow(sensorIndex, row++, &xval, &yval)){
    real_table_value_x = x_scaler * xval;
    real_table_value_y = y_scaler * yval;

    // bunch of code here for the other cases omitted here...

    // store into the previous values for use in interpolation/extrapolation
    previous_real_table_value_x = real_table_value_x;
    previous_real_table_value_y = real_table_value_y;
  }

  // case 4: the independent variable is must be greater than the largest value in the table, so we need to extrapolate forward
  // if you got through the entire table without an early return that means the independent_variable_value
  // the last values stored should be used for the slope calculation
  slope = (real_table_value_y - previous_real_table_value_y) / (real_table_value_x - previous_real_table_value_x);
  return real_table_value_y + slope * (independent_variable_value - real_table_value_x);
}

I removed all the irrelevant stuff. Do you see the problem? Basically at the slope calculation right before the return is *guaranteed* to result in the calculation 0 / 0 which turns out to be Not-a-Number (nan) in floating-point-speak. So in the case where the “measured value” turns out to be on the high side, things would sort of fly off the handle. There is still the ‘problem’ that the values are coming out high to begin with, but that’s because we don’t have a calibration procedure at this point, and is a topic for a different post. Thanks for reading!

Air Quality Egg: The Making of a Sensor Data Point

I have been wanting to write this blog post for a while. I hope this article will give some insight to the Air Quality Egg community about how the system works end-to-end. I want to provide details about how the Air Quality Egg software and hardware work hand in hand to provide an extensible platform for getting data from the a transducer (sensor) to the internet. I’ll try and structure this in into sections are data-centric, framing it in the context of the “lifespan” of a sensor reading. I chose to write it with a biologically inspired vantage point, hopefully it’s not too lame.

Inception

When you turn on your Base Egg, it broadcasts its MAC address (unique ID) for a period of about 30 seconds (the pairing interval) while illuminating the shell with a yellow pulse. During this time, the Base Egg is offering itself for pairing in order to establish a data link with a Remote Egg. A Remote Egg, in turn, listens patiently for Base Egg advertisement broadcasts for about a 15 second period after power up. A Remote Egg hearing a Base Egg stores away that Base Egg address in persistent memory (EEPROM). Base Eggs are not “monogamous” and willingly accept data from any Remote Egg that is within range and has learned the Base Egg’s advertised address during the pairing interval. Neither do Remote Eggs “mate for life.” They can always be paired to a new Base through the same procedure. They are nevertheless “loyal” in that if they reset, hear no new Base Egg advertisements, and have previously paired with a Base Egg, they will resume communicating with that previously paired Base Egg.

Nurturing

Every sensor hosted by the system has a small microcontroller attached to it. The job of this microcontroller is three-fold. It:

1. Provides the interface to the Egg Bus.
2. Performs any tasks necessary to keep the sensor happy and safe, for example regulating the heater power for a CMOS ozone sensor.
3. Is responsible for actually reading the sensor at the lowest level, be it through analog-to-digital conversion like the CMOS gas sensors, or a time-based digital occupancy measurement like the dust sensor.

This federated sensor management architecture separates the concern of how each sensor is dealt with into bite-sized, compartmentalized fragments, and allows the system to scale well locally. A major upside to the standardized interactions is that it allows you to plug in new sensors into the system without needing to change the software running on your Air Quality Egg. Another big upside is that new sensors can be developed by anyone because of the openness of the interface, software, and hardware.

Discovery

In the Remote Egg, Sensors are attached to the Egg Shield through the Egg Bus. The Remote Egg, at one minute intervals, searches the Egg Bus and finds attached sensors. Upon encountering a sensor, it interrogates it for a variety of information. Using the information gleaned by the Egg Bus, the Remote Egg calculates both a “raw” and “computed” sensor reading.

In a similar theme of discovery, the first time you plug your Base Egg into your network it reaches out to Cosm over the internet and attempt to negotiate the creation a feed where it will ultimately store your Egg’s data. This mechanism is enabled by the fact that each and every Air Quality Egg is labeled with a Unique ID that is also bound to the physical hardware, sort of like your Egg’s fingerprint! When you enter your Egg Serial Number into airqualityegg.com and fill out the form, this process is enabled.

Migration

Having discovered a sensor, and having calculated the raw and computed data points, the Remote Egg ships it wirelessly to its paired Base Egg in a Sensor Packet. The Base Egg, upon receiving, a Sensor Packet addressed to it, illuminates its shell a color (chosen from a predefined sequence) to give you, the proud parent of your Air Quality Egg, feedback that your Remote and Base Eggs are getting along.

Metamorphosis

The Base Egg then restructures the information it receives wirelessly into a format suitable for Cosm’s API. The Base Egg also decorates the datapoint with various tags to help applications search, filter, and group the data more efficiently. The Base Egg transmits the datapoint over the internet where it can live out the rest of its days as part of the world’s first open public Air Quality data set, and doing its part to give us a better understanding of air quality across time and space.

 Evolution

As the data set becomes larger and more dense, and as our understanding of the data grows, the hope is that our community will develop algorithms and procedures to improve the quality of the data through further post-processing. Techniques will also be hopefully be developed to help ‘calibrate’ your Egg’s sensors in a low cost, DIY way. As new sensors become available modules will be developed that will seamlessly plug into your existing Egg through the aforementioned Egg Bus connection, giving us an ever richer set of correlated data.

Kickstarter Eggs Shipping

It has been a long road, and we are very happy to say that we are shipping our first batch of Kickstarter Air Quality Eggs. We are shipping plain eggs first (that is, with no add on boards).

This is what 100 Eggs looks like. The address labels are on the side so that they don’t cover up the packaging. (If the one upside down package is driving you nuts, don’t worry, we turned it around

Once you get your egg, remember to go to airqualityegg.com and enter the serial number, which is printed on the inside of the box. Finally, we will be shipping more soon – if your egg is not in this batch, you will be getting it in a week or three.

Air Quality Egg: Gas Sensor Parameters

I thought I’d write a little bit about how the sensors are “mapped” to software for the technically interested. First some quick background. The NO2, CO, VOC, and Ozone sensors used in the Air Quality Egg have a lot in common. They are all CMOS sensors whose resistance varies in proportion to the amount concentration of gas present. The sensors’ operate based on surface chemistry, and the surfaces react selectively with a their target gas under prescribed surface temperatures. They differ in the following ways:

• Nominal resistance in clean air (R0)
• Response curve relating measured resistance to gas concentration (T)
• The range of resistance values over which the sensor varies (S)
• Required heater power (P)
• Maximum sensor voltage (Vs)

Air Quality Egg: Setting R0 Values

Some people have been trying to update their fielded Egg Shields and running into trouble with really small values being reported (like 0). One reason this could happen is a mistake made here at the “factory” during programming. The sensors rely on a value stored per sensor called R0 – which is to say the nominal resistance of the sensor in ambient conditions. During device programming, we should have set these value to something reasonable. We set R0 for the NO2 sensors to 2200 (ohms) and R0 for the CO sensors to 750000 (ohms).

To do this procedure, you’ll need an FTDI cable or equivalent to program the device using the Arduino IDE. If you download the EggBus library from Git-Hub, it comes with a few useful examples. If you start by loading up the PollEggBus example (File => Examples => EggBus => PollEggBus”), it will show you, for each sensor it finds on the bus, a bunch of useful information. You’re going to want to look closely at what it says for “R0:”  - and if it’s a “really big value” (like 4294967295, which turns out to be 0xffffffff to the binary initiated) we’ll have to run a quick sketch to get things back on track. Load up the PollEggBus_WriteTest example (File => Examples => EggBus => PollEggBus_WriteTest) and run it. After it runs you should see NO2′s R0 value is 2200 and CO’s R0 value is 750000 as advertised.

You can change these values to whatever you want actually, and I’m going to write up a derivative example that lets you perform a targeted calibration procedure in the near future.

Load back your Air Quality Egg sketch back onto your Nanode after you run this program (for standalone eggs this is probably the AQEBaseSensor sketch, and for Egg pairs it’s probable the AQERemote sketch). Hopefully this helps you out if it seems like your Egg is always outputting zero.