Migrating to a not-misspelled-domain…

This blog used to be aperature.org and while I used that domain name for some time, it was a fairly bad misspelling and needed to be corrected.  If that wasn’t enough, the name didn’t really reflect the content very well.  When I got the original domain, I was very much into photography.  I migrated into other interests and didn’t change the name.  Besides all that, finding something in a .com, .org, or .net that was original and memorable was very difficult at the time.

So, this is now going to be stderr.info.  I like it because it’s short and it refers to the standard error file descriptor from the standard C library.  While some may think this refers to a medical condition, that’s not the case.  The great thing about TLA’s is that you can always come up with something different than what was intended.  🙂

Complete code for sun_switch project

outsideLight
Some time ago, I started documenting my project to automatically switch on/off some external lighting on the house. You can read the full rationale behind the project in this post. I’ve been delayed in getting this posted because I’ve had other personal projects that have been a higher priority lately.

Today, I’m posting the (nearly) final code for the entire project. All you need to play with this is a WWVB receiver, an Arduino, the TimeLord library, and an optional LED. The LED would be attached to the ssrPin output (with a current limiting resistor) to give an indication of whether the lights would be on or off. Initially, the output defaults to having the lights on. This is so that a power outage in the middle of the night will default to the on state until the system receives a valid time from the WWVB receiver. The LED that’s built into the Arduino is not used for this because I used it in another part of the code to indicate the stability of the received signal. That LED should turn on/off at a rate of once per second if the received signal is clean and free of errors.

There are probably a few bugs in the code at this point, but my testing has shown it to be fairly reliable – enough so that I’m going to be working on a project enclosure as well as some modifications to the electrical in the garage to put this project to good use. I think this code demonstrates a wide range of the capabilities of the arduino. There is some use of the default libraries as well as some custom programming that utilizes the atmega168 hardware directly.

I don’t think I’ll draw the ire of the Arduino haters on this one, but you never know. Personally, I think the flexibility of the platform is something that most of ‘haters’ don’t understand. You don’t have to use any of the libraries if you don’t want to and you can pretty much just skip forward to writing straight C for the mega168 if that’s what you want to do. Personally, I wouldn’t pass up an opportunity to use a development platform with those capabilities; especially when you can avoid spending extra time and money on creating custom boards.

If you want more details and explanations of the code, please read my prior posts.

Source: sun_switch

Translating WWVB time to local time

We’re almost at a point where I can share the final code. One of the minor hurdles I had to tackle was converting UTC time with a day of year value into a more standard Gregorian local time. It turns out that the TimeLord library doesn’t have the facilities for doing this conversion on it’s own – probably because it was designed to work with a real time clock module instead of a WWVB receiver with my clock code.

Below is the code I developed for doing the conversion. This function is much longer than I would have liked, and I try to keep things small so it’s easier to debug. We start off with a definitition of what time zone we’re in. I’m in GMT-5, so the definition goes as follows:

#define timezone -5

Also, we’ll need our definition of ‘struct time t’ which carries the UTC version of our current time:

struct TIME {
  uint8_t seconds;
  uint8_t minutes;
  uint8_t hours; 
  uint16_t doy; // day of year
  uint16_t year;
  uint8_t leapYear;
};

volatile struct TIME t; // our only global variable

If I recall correctly, those are the only definitions that are missing from the following code block. This code tries to generically calculate local time without the added issue of having to jump forward or back due to DST and it should work for all timezones. Note also that a good chunk of the code is trying to compensate for the local day/month/year changing due to crossing midnight on a particular day. I probably didn’t need to worry about that since the actual sunrise/sunset values will only change by about a minute from one day to the other. I guess I was being a little OCD when I wrote this.

// compute gregorian date from time struct t
void getGregorianDate(byte cdate[])
{
  uint8_t months[] = {31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
  uint16_t daysLeft = t.doy;
  uint8_t curMonth = 1;
  int8_t curHour, curDay, 
         curYear = t.year - 2000;
  
  if (t.leapYear == 0)
    months[1] = 28;
    
  while (daysLeft > months[curMonth-1])
  {
    daysLeft -= months[curMonth-1];
    curMonth++; 
  }
  
  // convert our stored UTC time to local time
  if (((int8_t) t.hours + timeZone) < 0)
  {
    // adjust for negative timeZone constant with rollovers 
    curHour = t.hours + 24 + timeZone;
    curDay = daysLeft - 1;
    
    if (curDay < 1) // roll back the month
    {
      // we're actually on the last day of the previous month
      curMonth--;
      curDay = months[curMonth - 1];
    }
    
    if (curMonth < 1) // roll back to prior year
    {
      curDay = months[11];
      curMonth = 12;
      curYear--;
    }
  }
  else // positive calculation with possible rollovers
  {    // note that even if Timezone is a negative constant, this code still works
    curHour = t.hours + timeZone;
    curDay = daysLeft;
    if (curHour > 23) // day
    {
      curHour -= 24;
      curDay++;
    }
    if (curDay > months[curMonth - 1]) // month
    {
      curDay = 1;
      curMonth++;
    }
    if (curMonth > 12) // year
    {
      curMonth = 1;
      curYear++;
    }
  }
  cdate[tl_second] = t.seconds;
  cdate[tl_minute] = t.minutes;
  cdate[tl_hour] = curHour;
  cdate[tl_day] = curDay;
  cdate[tl_month] = curMonth;
  cdate[tl_year] = curYear;
}

Could this be better? Absolutely. I just haven’t taken the time to go through and simplify it yet. There are too many variables being used, for one. I could eliminate all of the curDay, curMonth, and curYear references and use the cdate[] array in their place. I’m also not sure we really have to worry so much about whether the end result of adding the timezone results in a positive or negative value. There’s something there that makes me think I could possibly cut the code for calculating the day, month, and year in half, but the technique for doing so isn’t clear to me just yet. The one good thing I can say about it is that it works great in it’s current form, so I’m leaving it well enough alone for the moment.

Calculating Sunrise/Sunset

sunset

So, now that we have most of the components of this project posted (including this post), I can finally give you the details of what we’re trying to accomplish. My house is situated in a smallish town without a lot of street lighting and the nearest light is too far away to effectively illuminate the house. Even worse than that, I have a driveway that slopes back towards the house creating a small, dark area just in front of the garage door. The problem with this arrangement is that it’s very easy for someone to spend a lot of time trying to steal gas from the one car I have that doesn’t fit in the garage. (funny tip: If you’re trying to steal gas, don’t go after a 4 cylinder, 35 MPG car like the guy in my neighborhood did. There is, on average, less than 4 gallons of fuel in the tank!) To help with this problem, I do have a pair of recessed lights on the house over the garage, but I don’t have a way to effectively turn them on and off at the right time of day. Currently, I have to leave the lights on all the time due to an unpredictable personal schedule of work, family, and friends which isn’t very efficient.

What about a light sensor? I did think of that solution, but that has its own set of problems. Being in the northeast, we get a fair bit of snow in the winter time. That snow can reflect a lot of light and cause the sensor to turn the lights off at the wrong time. Taking it a step further, if someone’s willing to resort to minor criminal activity, nothing would stop them from figuring out some way to fool the sensor since it would have to be mounted on the outside of the house where it would be accessible and visible.

My solution is to use a combination of a RTC, the WWVB time signal, and the TimeLord library to accurately compute sunrise and sunset. The resulting circuit would automatically turn the lights on at sunset and turn them off at sunrise. Not only would this be an energy savings over the course of each day, but with advancements in LED technology, I could reduce my energy use to just pennies per month and get the benefit of having a properly lighted driveway. Furthermore, an addition of some wireless communications could allow me to plot the energy usage over the course of the year. That last feature isn’t planned for this version but it wouldn’t be difficult to do.

So, how do you compute sunrise and sunset? Not being afraid of a little math, I started looking online at the equations necessary to accomplish this. I found this page that had all of the calculations and put together some proof of concept code to see how accurate it would be on the arduino. Being spoiled from working on PC’s I was a little surprised to find that the answers didn’t make much sense. In particular, the trig functions in the avr-libc package tended to give me the most inaccurate values. Furthermore, the calculation of the number of days since the last epoch was giving me a number that was too large to be represented on the microcontroller.

My next thought was to add a floating point processor to the circuit, but the problem with that is 1) added chips take up space, making the project bulky …and… 2) added chips drive up the overall cost of the project. Sometimes the best path forward is the simplest, so I did a search online to see if anyone else had already solved the problem. As it turns out, the TimeLord library does exactly what I needed it to do and it’s very easy to use. My only suggestion for the authors of this library would be to improve the documentation. Sometimes it’s hard to tell if values should be passed as local time or UTC. Beyond that, I would also add functions so that the user can pass either local or UTC without having to do the conversion. However, it did solve my problem without needing any extra circuitry.

I will be posting the full code to this project at a later date. For the time being, though, I’m posting the pieces in the same way I developed the final code which is to say that I put together several small and easy to debug code modules and then later integrated all of them into one large project. The benefit of doing it this way is that it’s much easier and faster to debug small modules than it is to try and put the whole project together in one attempt. Here is the proof of concept code for this week:

#include <TimeLord.h>

// current date - BTW - don't do this if you can avoid it.  Globals are evil.
int curYear = 2010;
int curMonth = 11;
int curDay = 12;

// west longitude and north lattitude - approximate - 
//      I'm not giving you my address in an online code example anyway... :-)
double Lw = -78;
double Ln = 43;

void setup() 
{
  // put your setup code here, to run once:
  Serial.begin(9600);
  
  TimeLord tardis;
  tardis.TimeZone(-5 * 60);
  byte day[] = { 0, 0, 12, curDay, curMonth, curYear }; // noon
  tardis.Position(Ln, Lw);
  if (tardis.SunRise(day))
  {
    Serial.print("Sunrise: ");
    Serial.print((int) day[tl_hour]);
    Serial.print(":");
    Serial.println((int) day[tl_minute]);
  }
  
  if (tardis.SunSet(day))
  {
    Serial.print("Sunset: ");
    Serial.print((int) day[tl_hour]);
    Serial.print(":");
    Serial.println((int) day[tl_minute]);
  }
}

void loop() {}

The next post in this series will have the fully integrated and tested code. I still have some modifications to make to the electrical in my garage to enable this to work, so it may be a while before I can post pictures of the final project. My goal is to have this done and installed sometime before the end of April.

Arduino as a RTC

So, I’ve been putting together this project where I’m receiving accurate time via a WWVB receiver (see previous post). To really make use of this, I need some way to keep track of the time during those times of the day when the WWVB signal isn’t available.

Initially, I looked at the chronodot from Adafruit industries, but one of the things I didn’t like about it was the fact that the battery mounts to the back side of the breakout board. I know that this was done to reduce the cost of the breakout board, but it makes the end result kinda bulky. Besides that, changing out that battery could be tricky depending on how you mount it in your project enclosure. Also, the fact that it’s round doesn’t help when have to mount it into my project case, either.

In addition to considering the chronodot, I also have been waiting on a similar product from Sparkfun.com which is based on the ds3234 real time clock module. Their version, at the time I started development, was simply just the bare timekeeping chip and the breakout for it was unavailable. Fast forward a few months and now the breakout is available, but yet again, the battery is mounted to the back of the board 😦

Not being too sure of the best way to proceed, it hit me one day that I might be able to program the arduino to perform the real time clock functions. Yeah, the crystal that’s on the board may drift with temperature by a few parts per million, but if I’m receiving a valid WWVB signal at some point of the day, I might not need extreme accuracy at all. In fact, after considering the requirements of my project, it wouldn’t really matter if the time drifted by up to 5 minutes over the course of the day.

Below is the proof of concept code I wrote:

// What do you do when an accurate, temperature compensated real time 
//    clock chip breakout is unavailable from your favorite vendors?  
//    Well, you first consider the less accurate battery backed 
//    alternatives until you later realize that there's not really a 
//    big difference between using the non-temp-compensated rtc and 
//    just programming the arduino to do it directly. 
//
//   So - that's what this is.  A real time clock implemented based on 
//    16bit Timer1 and the cpu's clock frequency.  Since both the 
//    dedicated rtc and the this rtc are both dependent on temperature 
//    variations and since both are using crystals for their time source, 
//    the amount of error should be acceptible.  Also, if you integrate 
//    wwvb receiver code with this code, your accuracy should be more 
//    than close enough...

#include <avr/io.h>
#include <avr/interrupt.h>

#define ISR_TIMER1_COUNT 15625 // 16.0MHz clock / (prescaler = 1024)

struct TIME {
  int seconds;
  int minutes;
  int hours; 
};

volatile TIME t;  // global time struct

void setupTimer1(void)
{
  // Setup Clear Timer on Compare Match mode.  We should interrupt each 
  //    time TCNT1 is equal to the value in OC1A register.  TCNT1 will 
  //    automaticall be reset to 0 each time a compare match happens.  
  //    Because this is done in hardware, the interrupt frequency should 
  //    be very stable (no interrupt service routine overhead to deal 
  //    with).
  
  // Please refer to the atmega168 data sheet for an explanation of the 
  //    registers and the values chosen here.
  
  // WGM13:0 = 4  for CTC on OCF1A value.  Prescaler set to divide by 1024.
  TCCR1A &= ~(1<<COM1A1) &   // Clearing bits
            ~(1<<COM1A0) &
            ~(1<<COM1B1) &
            ~(1<<COM1B0) &
            ~(1<<WGM11) &
            ~(1<<WGM10);
            
  TCCR1B &= ~(1<<ICNC1) &    // Clearing bits
            ~(1<<ICES1) &
            ~(1<<WGM13) &
            ~(1<<CS11); 
            
  TCCR1B |= (1<<WGM12) |    // Setting bits
            (1<<CS12) |
            (1<<CS10);
  
  OCR1A = ISR_TIMER1_COUNT;
  
  
  // OCIE1A interrupt flag set
  TIMSK1 |= (1<<OCIE1A);
  
  // Start counter at 0, not that it would matter much in this case...
  TCNT1 = 0;
}

// This interrupt service routine gets called once per second
ISR(TIMER1_COMPA_vect)
{
  t.seconds++;
  if (t.seconds > 59)
  {
    t.minutes++;
    t.seconds = 0;
  }
  if (t.minutes > 59)
  {
    t.hours++;
    t.minutes = 0;
  }
}

void setup()
{
  Serial.begin(9600);
  
  t.seconds = 0;  // initialize our time struct
  t.minutes = 0;
  t.hours = 0;
  
  setupTimer1();
  sei();    // allow interrupts globally
}

void loop()
{
  int sec = t.seconds;
  while (sec == t.seconds) {delay(100);} // wait for t.seconds to increment
  
  Serial.print("Time: ");
  Serial.print(t.hours);
  Serial.print(":");
  Serial.print(t.minutes);
  Serial.print(":");
  Serial.println(t.seconds);
}

You’ll note here that very little of the actual arduino library is used at all. I decided to use the 16bit Timer1 on the microcontroller directly with an interrupt to keep track of the time. When I started monitoring the program, I wrote down the current time on the computer. After 24 hours, I came back and again compared the time on the computer with the time that was coming out of my program and discovered that the time drift was only 3 seconds over the course of a full day! If it performs that well, I guess I don’t need a separate clock chip.

In the next post, I’ll be writing about the TimeLord library.

WWVB – Accurate time reception via a radio receiver

I’ve been working on a project that integrates a few ideas into one really cool project. The details of the project itself will become apparent in a few more posts as I get closer to completing the project, but I don’t want to put it all out there at once. It would make this post overly long, so please be patient as the details are revealed.

Part of this project depends on using a WWVB receiver to receive an accurate time/date signal from the broadcast station in Colorado, USA. The NIST time signal is a national radio signal that propagates along the ground throughout much of the United States. This signal is just a simple carrier wave that’s modulated to provide a digital stream of time data to anyone with a receiver. Once the signal is fully received and decoded, you have a time reference that’s just as accurate as your GPS. The nice thing about using WWVB is that the signal easily propagates through walls and you don’t need a clear view of the sky.

For this project, I’m using a WWVB receiver module I purchased at Sparkfun.com. This module only requires three connections: power, ground, and signal out. No complicated interfaces here, that’s for sure.  You may note in the comments on the Sparkfun site that if you live on the east coast, it’s recommended to get the longer loopstick antenna from Digikey.  I should note here that I live in western NY and I’m having no problems receiving the signal between the hours of 10pm (22:00 hours) and 12:30pm UTC.  My receiver isn’t even near a window.  The signal, in my case, has to penetrate through two walls of my house to be received and that’s not even considering the fact that there are a lot of other houses in my neighborhood that the signal has to get through.

WWVBReceiver-300x238

So, now that we have our radio receiver, we need a microcontroller and some code to be able to receive and decode the signal.  I chose an arduino pro mini for the microcontroller platform.  It’s simple to program, tiny, and more than capable of doing what I need for this project.  Of course, I also had to add a regulated 5V supply and a few other parts to make the project complete.  More on that later.

As for code, there are many examples online to look at.  I searched and found several examples, but I didn’t like the way they were written.  Very few, if any, showed good programming practices such as code reuse and small, tightly maintained functions that were clearly commented and documented.  Being frustrated with this situation, I wrote my own code from scratch.  It was easier than I expected:

note: wordpress does not make posting C code easy. If you notice any clipped or improperly formatted sections in the code below, let me know so I can fix it.:

#include 

#define wwvb 10  // input pin

#define pulseMarker 0 // all because processing doesn't understand 'enum'...
#define pulseOne 1
#define pulseZero 2
#define pulseInvalid -1

int8_t wwvbLastState;
int8_t ledPin = 13;
int8_t ledState = 0;

struct WWVBFRAME
{
  int8_t minutes;
  int8_t hours;
  int16_t doy;
  int16_t year;
  int8_t leapYear;
};

void setup()
{
  Serial.begin(9600);
  pinMode(wwvb, INPUT);
  pinMode(ledPin, OUTPUT);

  // At the start of our program, we have no idea what state our input is in
  //  or even where we are in the sequence.  Wait for two consecutive
  //  markers...
  wwvbLastState = digitalRead(wwvb);
  while (digitalRead(wwvb) == wwvbLastState) {} // wait for a transition to
                                                //  happen

}

/* note:  0.8S is a marker.  There are two to three consecutive markers at
              the start of the minute.
          0.2S is a zero
          0.5S is a one

          The signal transitions low for the start of each pulse.

          In this program, pulse types are represented as an int value
          according to the following:  0 = marker; 1 = logical one;
          2 = logical zero.  An enum would have been a more elegant solution,
          but processing doesn't have enums as far as I can tell (gives a
          compile error).
*/

void loop()
{

  // Rant:
  // Somethings *really* not right when you can declare this variable *two*
  //  different ways *and* one of those ways is kinda broken.  For instance,
  //  saying "WWVBFRAME wwvbFrame" works if you never have to pass it to a
  //  function.  To get passing to a function to work, you have to change the
  //  declaration to "struct WWVBFRAME wwvbFrame".  This is broken, imho.
  //  Either complain ALL THE TIME about the incorrect declaration or ACCEPT
  //  BOTH.  The last thing I want to do is to waste time debugging code that
  //  fails mysteriously without any explanation!
  struct WWVBFRAME wwvbFrame;
  int8_t dataValid = 0;

  resetWWVBFRAME(&wwvbFrame);

  while(1)
  {
    if (dataValid > 0)
    {
      Serial.print("valid: ");
      showTime(&wwvbFrame);
    }
    resetWWVBFRAME(&wwvbFrame);
    dataValid = 0;

    waitMinuteMarker(); // wait for next frame start...

    wwvbFrame.minutes = wwvbGetMinutes();
    if (wwvbFrame.minutes < 0) continue; // error - go back to the beginning
                                         //  of the while loop  

    wwvbFrame.hours = wwvbGetHours();
    if (wwvbFrame.hours < 0) continue;

    wwvbFrame.doy = wwvbGetDayOfYear();
    if (wwvbFrame.doy < 0)  continue;

    if (wwvbSkipDUT1() < 0) continue;

    wwvbFrame.year = wwvbGetYear();
    if (wwvbFrame.year < 0) continue;

    wwvbFrame.leapYear = wwvbGetLeapYear();
    if (wwvbFrame.leapYear < 0) continue;     // If we made it here, we have
      dataValid = 1;                                       // valid data
  }
} 

void resetWWVBFRAME(struct WWVBFRAME *w)
 {   
  w->minutes = -1;
  w->hours = -1;
  w->doy = -1;
  w->year = -1;
  w->leapYear = -1;
}

void showTime(struct WWVBFRAME *w)
{
  Serial.print("DOY: ");
  Serial.print((int) w->doy);
  Serial.print(" Year: ");
  Serial.print((int) w->year);
  Serial.print(" LeapYear: ");
  Serial.print((int) w->leapYear);
  Serial.print(" Time(UTC): ");
  Serial.print((int) w->hours);
  Serial.print(":");
  Serial.println((int) w->minutes + 1);
}

int8_t wwvbGetMinutes(void)
{
  int8_t minutes = 0; int8_t temp = 0;
  minutes = wwvbGetNumber(3, 10);
  if (minutes < 0) return minutes;

  temp = getPulseByType(); // discard unused pulse

  temp = wwvbGetNumber(4, 1);
  if (temp < 0) return temp;  

  minutes += temp;
  return minutes;
}

int8_t wwvbGetHours(void)
{
  int8_t hours = 0; int8_t temp = 0;

  temp = getPulseByType();
  if (temp != pulseMarker) return -1;

  temp = getPulseByType();  // discard unused pulses
  temp = getPulseByType();

  hours = wwvbGetNumber(2, 10);
  if (hours < 0) return -1;

  temp = getPulseByType(); // discard unused pulse

  temp = wwvbGetNumber(4, 1);
  if (temp < 0) return -1;  

  hours += temp;
  return hours;
}

int16_t wwvbGetDayOfYear(void)
{
  int16_t doy = 0; int8_t temp = 0;

  temp = getPulseByType();
  if (temp != pulseMarker) return -1;

  temp = getPulseByType();  // discard unused pulses
  temp = getPulseByType();

  doy = wwvbGetNumber(2, 100);
  if (doy < 0) return -1;

  temp = getPulseByType(); // discard unused pulse

  temp = wwvbGetNumber(4, 10);
  if (temp < 0) return -1;
  doy += temp;

  temp = getPulseByType();
  if (temp != pulseMarker) return -1;

  temp = wwvbGetNumber(4, 1);
  if (temp < 0) return -1;

  return doy + temp;
}

// DUT1 is not used in this project.
int8_t wwvbSkipDUT1(void)
{
  // note: it may be a good idea to beef up the error checking here
  int8_t i;
  for (i = 0; i < 5; i++) // two unused pulses plus the DUT1 sign bits
    getPulseByType();     // skip over them...

  int8_t temp = getPulseByType(); // verify marker
  if (temp != pulseMarker) return -1;

  for (i = 0; i < 4; i++) // skip the DUT1 value
    getPulseByType();

  return 0;
}

int16_t wwvbGetYear(void)
{
  int16_t year; int8_t temp;
  temp = getPulseByType(); // discard unused bit

  year = 2000 + wwvbGetNumber(4, 10);
  if (year < 2000) return -1;

  temp = getPulseByType();
  if (temp != pulseMarker) return -1;

  temp = wwvbGetNumber(4, 1);
  if (temp < 0) return -1;
    return year + temp;
} 

int8_t wwvbGetLeapYear(void)
{
  uint8_t leapYear= getPulseByType(); // unused pulse

  leapYear = getPulseByType();
  if ((leapYear == pulseInvalid) ||
      (leapYear == pulseMarker))
    return -1;
  else if (leapYear == pulseOne)
    return 1;

  return 0;
}

/* wwvbGetNumber() - this is one of the core functions that makes the
    whole thing work.  This function gets called to retrieve a sequence
    of pulses from the wwvb stream and convert those pulses into an
    integer.  Typically, the numbers are encoded as binary bits and their
    position in the stream determines if they are to be multiplied by 1,
    10, or 100.  In addition, there is some error checking going on to make
    sure we are receiving a valid signal from teh receiver.  The count
    variable is expected to start at 1 instead of 0 and indicates the number
    of pulses to be evaluated.  As is common in the rest of the code,
    we return -1 if we detect an error.
*/
int16_t wwvbGetNumber(int8_t count, int multiplier)
{
  int8_t i; int8_t result = 0; count--;
  for (i = count; i >= 0; i--)
  {
    int8_t pulse = getPulseByType();
    if ((pulse == pulseInvalid) ||
        (pulse == pulseMarker))
      return -1;

  // Note: Lesson learned: never send a floating point function to do an
  //    integer operation.  In converting the numbers, I originally used
  //    a conversion trick I had learned from programming on pc's where
  //    you set up a loop counting down to zero and use the counter variable
  //    with the pow() function from the math library to determine the value.
  //    In our case, a one in the data stream indicates a value of 2^i.
  //    However, in 'arduino-land', this does *not* work. Somehow when you
  //    try to compute anything equal to or above 2^2, a rounding error
  //    occurs and internally you get back something like 3.9 which, when
  //    cast back to an int, gives you 3 instead of 4!  Bit shifting is fool
  //    proof.

 if (pulse == pulseOne)
      result = result + (multiplier * (1 < 900)          // error conditions. Pulse must be between 150 
     return pulseInvalid; //   and 900mS long. 
   else if (pw  650)
     return pulseMarker;  
   else if (pw > 350)
     return pulseOne;  
   else 
     return pulseZero;  
}

The above code has been fully debugged and works great.  Sure, it could use some improvements like better error checking in a few places, but it handily gets the job done.  I’ve left in some comments that I used to help me remember how to decode the signal that I could have removed, but then thought it would probably make the program more readable if I left them in.

In our next post, we’ll be integrating this with a real time clock module so that we can have an updated and accurate source of time 24 hours a day.  Currently, atmospheric and man-made noise make it impossible to depend on a solid signal throughout the day.  However, I can say that I do get at least one valid WWVB frame per day, so that coupled with a RTC module should give me time keeping abilities that are more than close enough for my project.

Forged Emails

Note: this is a re-publishing of a post I had on my blog back in 2007.  Lately I’ve been getting quite a few questions regarding this topic, so I decided it would be good to bring this material back to the forefront.

Every once in a while, I get asked the following question: “I just got an email that says I tried to send an email message to someone I’ve never heard of and it bounced back. Why?” Or it goes something like this: “I got a message that says I tried to send a virus to someone and the email was rejected. How is that possible?” Well, the answer is a little technical, but I’m going to show you how this happened.

The short answer is that someone created a fake email that shows you as the sender. The reasons for doing this are usually related to adware, spyware, or viruses. Basically, they’re trying to install a piece of software onto as many systems as possible without the users knowing about it. The end result can range from something that’s just plain annoying to stealing the contents of your bank account and you can bet someone is getting paid to do it. By naming another valid email account as the sender, they raise the potential to get their virus/adware/spyware installed onto at least one computer. If the recieving email account exists and there isn’t any antivirus software scanning the email, it might get installed on that user’s computer, if the reciever opens the message along with any attachments that are inside it. If the message bounces back to the email account named as the sender, the person named as the sender would naturally be confused by a message he or she didn’t send. Consequently, they investigate the content of the original message and wind up unintentionally installing the nastyware on their computer instead. It’s a win-win situation for the person who’s trying to infect as many machines as possible even though it leads to confusion for the recipients.

That leads us to the question of how it’s possible to create a message and name a false To: and From: address. To start, the malicious user connects to a SMTP server somewhere on the internet and issues the necessary commands to send a message with your email address in the From: field. This is possible because most SMTP servers do not require the user to authenticate themselves before issuing the necessary commands to send an email. Therefore, any anonymous user on the internet can send an email that pretends to come from anyone else in the world! This problem was introduced with RFC821 in the early days of email. If you go ahead and read that document, it will tell you how to manually send an email, but it’s rather long and complicated. Here’s an example of how to open a connection to a mail server, create a fake message, and get the mail server to deliver it: (Note that the ‘>’ characters have been added here to mark where I manually entered commands to the SMTP server. In reality, those characters don’t actually appear on the screen. Names of servers and email addresses have been changed to protect private information and I emphasize that this is only an example – not a real message I may have tried to send):

C:\> telnet smtp.somedomain.com 25

220 smtp.somedomain.com ESMTP Server (Microsoft Exchange Internet Mail S
ervice 5.5.2657.72) ready
> HELO nastyware.com
250 OK
> MAIL FROM: John.Smith@redmail.com
250 OK - mail from <John.Smith@redmail.com>

> RCPT TO: Joe.Johnson@bluemail.com
250 OK - Recipient <Joe.Johnson@bluemail.com>
> DATA
354 Send data.  End with CRLF.CRLF
Subject: Your password has been updated.
Date: September 10, 2005
From: support@somebank.com
To: Joe.Johnson@bluemail.com

Your password was recently updated in our system.  If you did not change your 
password, please use the link below to notify us that someone has illegally 
accessed your banking information.  Thank you.

http://www.S0mebank.com/support/unauthorized_access.asp
.
250 OK
> QUIT
221 closing connection

Connection to host lost.

C:\>

As you can see from the above example, I lied about who I am and why the message was being sent. Furthermore, I named John.Smith@redmail.com as the sender, but the actual body of the email indicates that email was sent by support@somebank.com. This is so the person who eventually recieves this email will think their bank is trying to alert them to some kind of unauthorized access. If this message bounces because the recipient’s email isn’t real or because their spam filter rejects it, John.Smith@redmail.com will recieve the message instead. It won’t bounce back to the support@somebank.com address because the addresses in the body of the message aren’t used for delivering the message. Those addresses are only used by mail client software such as Outlook, Netscape, or Thunderbird for replying to the messages after they are delivered. Digging a little deeper, you’ll also notice that the url at the bottom of the message body has a zero where there should be an ‘o’ so that when the user clicks on the link, they will be directed to a website that looks like the website of their bank. The reality is that the website is most likely run by the person who forged the email and is designed to lure the recipient into revealing their account information such as an account number and password.

Unfortunately, RFC821 allowed this sort of abuse and had no protections in place to prevent it. There were no provisions in the specification that would require end-users to provide any proof of their identity before an email message would be accepted for delivery. Furthermore, most implementations of the protocol didn’t check to see that the addresses given in the MAIL FROM: and RCPT TO: portions of the protocol matched those addresses given later on in the body of the message – the From:, To:, and Reply-To: lines. Consequently, it was very easy to create a message like the one above and nearly every mail server on the internet would happily accept the message for delivery.

Fortunately, many email providers today are starting to require that users authenticate to the server before the server will accept an email message for delivery. In addition to that, some mail servers have restrictions in place to help verify the email addresses specified on the MAIL FROM: line match those used in the body of the message. Another methon being adopted is to have the mail server verify that the MAIL FROM: address belongs to a valid account.

However, even with those restrictions in place, you will still get the occasional fake message because most mail servers don’t have a reliable way of verifying the authenticity of a message when it gets passed from one mail server to the next. The typical example is where Joe sends an email to John. The message goes to bluemail.com’s email server for routing and delivery. That server then passes the message to redmail.com’s mail server for delivery to the end user. Redmail.com’s mail server doesn’t have any way to verify that Joe.Johnson@bluemail.com is a real mailbox. It can’t because only bluemail.com’s servers know what addresses are registered in the system. As a result, redmail.com’s servers accept the message for delivery.

Thankfully, this situation is slowly changing and the holes in the system are being plugged. Unfortunately, until all of the insecurities in the system are eradicated, you’ll still get fake emails and there isn’t much that can be done about it in the short term. In closing, if you avoid opening attachments and refuse to just click on the links you recieve in your email, you will be much better protected from these kinds of problems.

Disclaimer: The above is an example of a malicious message and is not intended to represent an actual email that may have been sent or recieved. The information given is a result of a real email message, but the content has been changed significantly to illustrate the themes mentioned in this posting.

Serial Debugging without a UART

attiny24I sometimes have projects using AVR microcontrollers – specifically, I use the ATTiny24 quite a lot since it has a fair number of IO lines and has most of the basic features you would expect from an AVR.   The only problem I have with this chip is that it doesn’t seem to have a full UART on it.  I’ve looked through the datasheet at the universal serial IO it has, but it doesn’t strike me as being useful for RS-232 communications.  Besides that, if you have the code I’m going to show you in this post, you can do RS-232 output on any unused pin without the need for a UART, which in my opinion, is a more flexible solution.

(Yes, you can use JTAG if you have the tools for it, but it’s not ideal when all you really need is simple feedback to know what/how your program is behaving.  I actually have the JTAG ICE mkII and I mostly just use it as a programmer if I’m writing my program in C.)

To do this, you need to know about two basic approaches to structuring a program for an embedded system.  First, we’ll be implementing an interrupt driven scheduler that uses a timer.  The timer helps to ensure that the serial output line is changing at the correct rate so that the remote system can understand the data being sent.  In this example, we’ll be trying to communicate at 9600 baud, so we’ll need to have the timer interrupt occur every 0.104mS (1/9600 = 0.104mS).  Each time the interrupt occurs, we’ll change the state of the output pin to signal the next bit to be transmitted.

The second concept we’ll need to implement is a state machine.  Basically, we need to keep track of where we are in the current byte to be sent as well as where we are in the output string.  As each bit is sent, we need to determine if the current byte is  just beginning and we need to send a space (start) bit or if the current byte is ended and we need to send a mark (stop) bit.  (See Wikipedia for an in-depth discussion of the RS-232 protocol.)  The state machine is built around the use of a few variables and control logic so that the information sent complies with the protocol.  Here is the code:

/* Headers needed for this module */
#include <stdio.h>
#include <string.h>
#include <inttypes.h>
#include <avr/interrupt.h>
#include <avr/pgmspace.h>
volatile char msg[32]; // message buffer
//Sample message - note how the string is null terminated so we can use strcpy()
const char testmsg PROGMEM = "Testing Serial IO!\r\n";
// This line is our output for serial data
#define SEROUT PB2
// Scheduler interrupt frequency.  This directly affects the baud rate
//  for serial communications.  Do not change this.
// This was approximately calculated as follows (some tweaking was needed):
//    ((0xFFFF) - ((1 / BAUD) / (1 / F_CPU)))
#define TCNT1_CNT  0xFD00
void init_timer(void)
{
 TCCR1A &= ~(1<<COM1A1) & ~(1<<COM1A0) & ~(1<<COM1B1) & ~(1<<COM1B0) &
    ~(1<<WGM11) & ~(1<<WGM10);
 TCCR1B &= ~(1<<ICNC1) & ~(1<<WGM13) & ~(1<<WGM12) &
    ~(1<<CS12) & ~(1<<CS11);
 TCCR1B |= (1<<CS10);
 // WGM13:0 = 0 for normal mode
 TIMSK1 |= (1<<TOIE1);  // interrupt on overflow
 TCNT1 = TCNT1_CNT;
}
ISR(TIM1_OVF_vect)  // TimerCounter1 overflow interrupt handler
{
 /*  Some of the frequency calculations may need to be tweaked to make
  it so that the routines execute with an acceptable amount of error.
  Since we are bit banging serial communication, the amount of error
  we can have before the serial stream turns to garbage is fairly
  small */
 TCNT1 = TCNT1_CNT; // reset counter for next time
 // This must be done first because the timing is critical
 do_serial_comm();
// ... do other things if need be....
  }
}
void do_serial_comm(void)
{
 /* State machine to output serial data.  This function must be
  called with the proper frequency to output serial data at the correct
  rate.  If the rate varies slightly, the baud rate won't be stable
  enough to ensure the data can be understood by the remote host.
 */
 static volatile uint8_t bitPos = 8;
 static volatile uint8_t charPos = 0;
 static volatile uint8_t needStopStart = 0;
 static volatile char curChar;
 /* The order of the following blocks is critical.  For this
  state machine to work correctly, we have to do something
  with the transmit line every time this function is called.
  The operation to be performed depends on where we are in
  current character and the message itself.
 */
 // send serial data
 if (needStopStart > 1)
 {
  PORTB |= (1<<SEROUT);    // logical mark output
  needStopStart--;
 }
 else if (needStopStart > 0)
 {
  PORTB &= ~(1<<SEROUT);   // output a start bit
  needStopStart--;
 }
 else if (bitPos < 8)        // LSB first
 {
  if ((curChar & (1<<bitPos++)) == 0) // bit is 0
   PORTB &= ~(1<<SEROUT);
  else       // bit is 1
   PORTB |= (1<<SEROUT);  
 }
 else if (charPos < strlen(msg))
 {       // move to the next byte
  curChar = msg[charPos++];
  bitPos = 0;
  needStopStart = 3;    // Sends 2 idle bits + 1 start bit
  PORTB |= (1<<SEROUT);   // Idle the line
 }
 else  // only get here if bitpos is 0 and curPos = strlen(msg)
 { // end of string
  strcpy(msg,"");         // blank the message
  bitPos = 8;
  charPos = 0;
  needStopStart = 0;
  PORTB |= (1<<SEROUT);   // Idle the line
 }
}
/* Code */
int main(void)
{
 init_timer();
 strcpy(msg, testmsg); // initialize output message
 sei();    // enable interrupts
 
 for (;;) {}  // loop
}

Some of the above code may look confusing, but it works quite well.  Our cpu is configured to run at 8MHz from the internal oscillator.  Since that timing source is going to drift from chip to chip, some tweaking of the counter overflow value may be needed to get the timing of the serial line changes to work correctly.

In the do_serial_comm() call, the needStopStart value is used to output the necessary stop, idle, and start bits that have to occur in between each byte.  Since the sequence in between each character is always the same – a stop bit, idle bit, and a start bit – we can simply decrement needStopStart each time the function is called to get the correct output.  When we are sending character bits, we simply test the current bit to see if it’s a 0 or a 1 and then send the proper logic level on the output pin.

When the message buffer is exhausted the message buffer is set to a blank string and the remaining state machine variables are reset.  The remaining program code can then test the length of the message buffer to figure out if the current message has been completely sent before placing a new message into the buffer.  Testing for a blank message buffer has to be done each time the program wants to send a new message.  If a new message is put into the buffer before the previous message has been completely sent, the output may be corrupted.

This technique works best if your program has a lot of time in between status updates.  In my case, I used this code in a NiCD battery charger I built and I wanted the charger to be able to log the battery voltage on the computer once per second.  So long as the message is short and the check for the battery voltage isn’t happening too quickly this method is very reliable.  If on the other hand, you’re doing something 1000 times per second, you’ll find this to be very limiting because you simply can’t send messages that fast at this baud rate.  Also the baud rate is also a limiting factor since the amount of acceptable timing error is reduced as you try to communicate at faster speeds.

Anyway – I hope this information was helpful.  If you spot an error, please let me know in the comments.

SQL Server 2005: Logon Error 18456, State 11

First, I would like to make a request of anyone reading this.  If you post in some forum somewhere about a problem you’ve been experiencing, please take a few moments to also post what the solution was after you’ve gotten your problem figured out.  I spent a good amount of time researching why I was getting the error in the title of this post and while I could easily find posts online describing the issue, it seemed like almost no one took the time to post the solution.  Those of you who do post your solutions, thank you.

This week we’ve been working on installing VMWare VDI/View so that we can do an evaluation of the product.  Part of that installation process involves setting up a SQL Server database for the vSphere and View pieces of the software and though we could have used MSDE or whatever MS is calling it these days, we wanted to mirror a production environment as closely as possible.  This means that we wanted to use SQL Server 2005 as the back end database and use an Active Directory service account to grant access to the database which is the best practice standard in our environment.

After configuring the account in the AD and setting group memberships, I proceeded to add the new service account to the database server, create a database for the user, make the service account the database owner, and grant further access to the msdb system database.  All of that seemed to work perfectly until it was time to create a data source so the vSphere could access the database.  Suddenly, the service account couldn’t connect and the above error was appearing in the activity log on our SQL Server.

Of course, the above error means that the server was able to verify the account as valid, but had denied the logon attempt.  (Note to the MS SQL Server Product Team: this error is useless without knowing WHY the logon was denied.  Makes me wonder who did the QA on this product…) I spent some time searching online forums for a solution and I actually found one post that mirrored my situation exactly.  The only problem was the original person who posted it didn’t mention if others had helped solve the problem.  Instead, they came to the forum, posted their problem, got some advice, and disappeared into the ether to never be heard from again.  I didn’t want to follow the advice in the thread because it seemed like I’d be going in circles.  After all, I just created the account, added it to the server, and granted rights to the account only moments prior to this.  How could anything be different?

In an attempt to isolate the issue, I added one of my test accounts to the server and made it the owner of the database.  I then went back to the vSphere server, added the test account to the local administrators group, and logged in under the test account.  When I then tested the connection to the database, it worked as expected. Since I couldn’t figure out what was different between the test account and the service account, I ended up deleting the credentials from the SQL Server management console, removing all of the rights I had granted, and then finally adding all of that information back to the server.  Strangely enough, that worked.

I have no explanation as to why that worked as a solution, but next time I run into it, this is the first thing I’m going to try.   Hope it helps.