Jared Sanson

Tags

As part of a Summer Scholarship at the University of Canterbury, I developed a system for detecting fire hotspots using thermal imaging cameras and remotely piloted aircraft systems (RPAS). The project was part of a group of four fire-related research projects, organized by Tait Communications and SCION.

It was an amazing learning experience as I got to experiment with many different technologies, such as OpenCV, quadcopters, thermal imaging cameras, and mapping algorithms. I also got to talk to some interesting people within the fire industry about what it's really like to be out on the fireground.

Background

Fire hotspots are a real problem when fighting large rural fires, as they can remain undetected long after the fire has been extinguished, potentially re-igniting the fire and endangering homes and property. Currently firefighters have to hire an expensive helicopter and trained crew, who then fly over with a thermal imaging camera and manually drop down markers on top of the hotspots. Unfortunately helicopters are not always available after a fire, in which case they must instead locate the hotspots manually using the back of their hands, which can take a very long time over large firegrounds!

Fire hotspots are not easy to locate with normal vision, since they are usually buried underneath ash or burn inside tree stumps. Thankfully the invention of far-infrared thermal imaging cameras has made this kind of thing very easy to detect:

As you can see from the photos above, the thermal photo quite clearly shows the hotspots which were otherwise completely invisible! Some hotspots can be in excess of 150°C, more than enough to re-ignite the fire if a substantial wind picks up the embers.

Project Overview

The reason for my project was to make it easier, safer, and cheaper to detect fire hotspots, and hopefully with more accuracy than current methods. The basic idea was to combine a Remotely Piloted Aircraft System (RPAS) with a small lightweight thermal imaging camera, and then be able to detect and locate hotspots from the captured imagery and data.

The actual algorithm was split up into two parts: the hotspot detection, and the geo-location of the hotspots. Detecting hotspots is actually fairly easy to do, though I can't go into details about it here. Finding the real-world co-ordinates of the hotspots is a lot trickier to do, as you have to worry about things like the RPAS's rotation, the ground elevation, and the camera parameters. The calculations behind that part are actually quite similar to the algorithms behind raytracers!

My algorithm was able to automatically detect and locate hotspots to within 50 meters or so, with no user intervention. I tested it on a fly-over of a power station, which easily picked out some transformers that were running quite hot:

Hotspots detected in imageHotspots detected in image
Geo-located hotspots on mapGeo-located hotspots on map

Due to some difficulty around the university's quadcopter we weren't able to set up a fully functioning system, but it worked well on the data I had at hand.

Overall, it was a good project and I learnt a lot about both programming and the industry. I am hoping to extend it into my final year research project for my degree, where I'd like to turn it into a fully functioning system.

The content on this page was reproduced with permission of Tait, and may not be used or copied.

View Comments...

After my blog post on e-ink displays, I wanted to go into more detail on how I generated the bitmap fonts I used. In this blog post, I'll show how to generate bitmap fonts from fonts you can get online, and a brief intro on how to use them.

I know there are a few tools out there for generating this kind of thing, but this was a custom thing for my microcontroller and I like doing and learning things myself!

Bitmap Fonts

There are quite a few sites with freely downloadable fonts that make great pixel fonts (FontGal.com is one), although .ttf files can't really be used on microcontrollers, so we first need to rasterize them to a bitmap font. To conserve space, each character is 6x8 pixels in size (WxH), and only common ASCII characters are drawn.

C Header

Before we can use this in a microcontroller, we need to turn the bitmap font into a C header file, so it can be compiled into the firmware. Since the bitmap font is black and white, the most efficient way to store it is an array of 6 bytes, with each byte representing one vertical column of pixels (8 pixels high). The resulting header file looks something like this:

const unsigned char font[96][6] = {
    {0x00,0x00,0x00,0x00,0x00,0x00}, //  
    {0x2f,0x00,0x00,0x00,0x00,0x00}, // !
    {0x03,0x00,0x03,0x00,0x00,0x00}, // "
        ...

This is a fairly standard way of representing fonts, though it does pose a limitation in that characters can only be 8 pixels high, and it makes it somewhat difficult to represent variable-width characters (since each char must have exactly 6 bytes)

For example, the character 'A':

{0x3C,0x12,0x12,0x12,0x3E,0x00}, // A

Drawing Fonts

It's fairly easy to draw fonts stored like this. First we convert the ASCII character to an index by subtracting the offset of the first character in the font. This works because the characters in the font are defined in numerical order ('0'=48, 'A'=65, etc.), so subtracting the space character (' '=32) returns the index in the array. We also replace any invalid characters with space, to prevent the code from trying to draw characters that aren't defined!

#include "font.h"

#define CHAR_WIDTH 6
#define CHAR_HEIGHT 8

void DrawChar(char c, uint8 x, uint8 y, uint8 brightness) {
    uint8 i,j;

    // Convert the character to an index
    c = c & 0x7F;
    if (c < ' ') {
        c = 0;
    } else {
        c -= ' ';
    }

    // 'font' is a multidimensional array of [96][char_width]
    // which is really just a 1D array of size 96*char_width.
    const uint8* chr = font[c*CHAR_WIDTH];

    // Draw pixels
    for (j=0; j<CHAR_WIDTH; j++) {
        for (i=0; i<CHAR_HEIGHT; i++) {

            if (chr[j] & (1<<i)) {
                DrawPixel(x+j, y+i, brightness);
            }
        }
    }
}

This can easily be used to draw a C-string:

void DrawString(const char* str, uint8 x, uint8 y, uint8 brightness) {
    while (*str) {
        DrawChar(*str++, x, y, brightness);
        x += CHAR_WIDTH
    }
}

It is possible to extend this code to support variable-width characters, or dynamically selectable fonts (using pointers), but that's a bit trickier to implement.

Downloads

An archive of my processed fonts is available here: fonts.zip

This includes the processing script, rasterized .png files, and the source .ttf files.

As always, if you want to know more or need some help, feel free to contact me or leave a comment!

View Comments...

Revision 2

As detailed in my last post about this project, I forgot to make sure that my microcontroller would actually fit the footprint! Since the PCB would need to be completely redone to fit in the new package, I took the opportunity to improve on my previous design.

One change I made was in the power supply; I realised that the MCU can't operate above 3.6V, and lithium ion batteries supply around 4.3V when fully charged. So I replaced the original MCP73831 lithium ion charger with a BQ25010 which combines a buck regulator with a lithium ion battery charger into a very small package. It actually uses less PCB space than the original lithium ion charging circuit. An LDO may have been a better choice, but I am only aiming for a week or so of battery life in standby, so this chip should be fine.

Another addition I made was bluetooth 4.0 (Low Energy), using the nRF8001 chip. I've never done RF PCB layout before so I have no idea if it will work, but I wanted to try anyway! There is an open source driver for it already out, and I've seen it in a lot of arduino shields too.

Routing the last tracks...Routing the last tracks...

Ordering the Parts

I got three PCBs from OshPark (purple!), and lots of SMD parts from Element14 (Farnell)

I'm not sure what the total cost of this project is, but I don't care since the main point of it is to learn about building a small low power device with lots of features.

Soldering the PCB

To make sure the design actually worked, I started by soldering just the PIC24F and making sure I could program it, and after that I added the BQ25010 chip. I ran into problems with both of these! I also ended up having to do some patch work since I overlooked some parts of the design, such as the charge enable pin being active low, not active high.

MCU Problems

I spent quite a while pulling my hair out trying to get the PIC to be detected by my PicKit2, but it just wouldn't register! Eventually I discovered that the PicKit2 didn't actually support the chip I was using. See my blog post on adding extra devices to the PicKit on how I fixed that.

Luckily all I need to do is program the bootloader, and the rest can be done over USB (which is much faster anyway)

Regulator Problems

Initially the regulator wouldn't work, it'd produce a very unstable voltage somewhere between 2-4V, so I probed it with an oscilloscope and noticed it was oscillating heavily! Luckily the fix for this was to add a single 100nF capacitor across the battery connection.

Decoupling capacitors are important!!

OLED Problems

I knew I wasn't going to be able to get this working first try. Unfortunately, I made another huge mistake in my PCB layout... the connector was reversed!! Luckily the connector I was using actually works both ways, so I could get the OLED to work by flipping it. Now it's on the wrong side of the board, but at least it works!

Interestingly the pinout of the OLED seems to be designed to not be damaged if connected around the wrong way. The +12V pin matches an unconnected pin on the other side.

Success!Success!
Taking its final formTaking its final form
Advanced graphicsAdvanced graphics

I'm getting some strange ghosting problems which I suspect is because I ommitted the VSL pin's components, but I haven't really looked into it yet and it doesn't bother me too much.

USB Connector Problems

After a few weeks of delicious PIC programming, the USB connector snapped clean off, taking the data traces with it!

I think the problem here was a combination of using chinese parts, not adding structural holes to the PCB, and not glueing it down. Unfortunately that means I'm stuck using a slow ICSP connection, and don't have any way to communicate with the device from my PC.

Next Steps

I haven't been able to work on it lately due to lots of university work (only a few weeks left!), but the next thing I want to do is to shift the components to one of my spare PCBs, so I can have my USB connector back. The OLED connector also started wearing out after flexing it too much, so I might order a new one. Eventually I'd like to 3D print a case for it, but we've run out of PLA filament!

I might try design revision 3 as well, and improve from what I've learnt on this revision.

I hear e-ink is quite attractive now, and it seems to be something missing from every smart-watch I've seen online so far...

Want to know more?

Leave a comment!

View Comments...

PicKit2 Unsupported

For my latest project, I foolishly ordered some PIC24FJ256DA206 chips without checking that my programmer actually supported them! So after much pulling of hair, I finally tracked down the problem and realised the chip wasn't actually supported by my PicKit2.

The PicKit3 can program my chip through MPLABX, but I couldn't find any in my local stores, and buying one online would have taken weeks to arrive. So I decided to try and see if I could manually add the chips to the PicKit2 GUI. 

Why Won't it Work?

When I attempted to program the chip, PicKit2 just gave me a "Invalid Device ID 0xFFFF" which was strange since usually it would at least report a proper device ID and then say it didn't support it.

So I dug deeper into the family programmer's reference manual, then I spotted this buried within:

The address of Special Function Register, TBLPAG, has changed from 0x32 to 0x54 in PIC24FJXXXDA1/DA2/GB2/GA3/GC0 family devices. In those cases where legacy programming specification code from other device families is used as a basis to implement the PIC24FJXXXDA1/DA2/GB2/GA3/GC0 families programming specification, special care must be taken to ensure all references to TBLPAG, in any existing code, are updated with the correct opcode hex data for the mnemonic and operands

This gave me a clue of what I needed to fix, since the PicKit2 actually has a scripting language that appears to actually do all the device-specific work. If they've changed that register, then perhaps all the current PIC24 scripts are no longer valid?

Before I investigated that, I decided to figure out how to find the proper parameters when adding a new chip.

The PicKit2 Device File

I have previously used the Device File Editor in this blog post

By comparing a supported PIC24 family by using the PicKit2 Device File Editor and the family's programming manual, I was able to reverse engineer enough information to be able to add my PIC24F devices. Listed below are the necessary parameters, and how to find them.

Device ID

This is easily found in both the programmer's manual (section 6) and the datasheet.

ProgramMem

This is also easily found in both the programmer's manual () and the datasheet, however the value found there is actually in bytes, and needs to be converted to instruction words by using the following formula:

ProgramMem = DatasheetValue / 2 + 1

PIC24FJ256DAxxx: ProgramMem = 0x02ABFE / 2 + 1 = 0x15600 words

ConfigAddr

This was a bit trickier, but can be found in the same section as the ProgramMem value, and needs to be converted using the following formula:

ConfigAddr = CONFIGN * 2

Where N is the highest config register (eg. CONFIG4 for my chip)

PIC24FJ256DAxxx: ConfigAddr = 0x2ABF8 * 2 = 0x557F0

ConfigWords

Equal to the number of configuration words!

PIC24FJ256DAxxx: ConfigWords = 4

ConfigMasks

This was trickier to find, and seems to be quite crucial.

Basically find the configuration registers in the device datasheet, set all the reserved bits to 1, and all the other bits to 0. Also duble check the programmer's reference manual to see if there are any special reserved bits there.

ConfigBlanks

I'm not 100% sure on this, but definitely check the programmer's reference manual for what values the reserved bits must have. For example, the PIC I'm using requires bit CW1<15> to be set to 0. Note that this doesn't have to match the mask. If you're unsure, set it to 0xFFFF and see if that works.

CPConfig

This is for the PicKit2 GUI, it tells it which config register the code-protect bits are located at. It is 1-based, and in reverse order. For my PIC, the code-protect bits are located in CONFIG1, so CPConfig = 4.

CPMask

Related to CPConfig above. Set the code-protect bits to 1 in this mask, so if they are set to 0, the GUI will show a warning.

The PicKit2 Scripting Language

Luckily the PicKit2 GUI source code is available, so I downloaded it and had a shot at making my own script viewer to see if the problem was easily fixable. Since it was written in C#, it was fairly easy to make my own GUI that could read the PicKit2 Device File.

I started by parsing the script against the constants defined in the source code, and found out that some of the scripts were actually executing PIC24 instructions on the device! Looking at the programmer's manual, I could match some of the byte-codes with the values I was seeing there.

I then added a simple decompiler that supports a few of the opcodes.

PicKit2 Script Editor & Patcher

The full source code for this is available on github

Now that I had everything sorted out and working, I improved the GUI to add a basic device file viewer,and also a patcher that adds support for an entire family of devices! It supports the following chips:

PIC24FJ128DA106, PIC24FJ256DA106, PIC24FJ128DA110, PIC24FJ256DA110, PIC24FJ128DA206, PIC24FJ256DA206, PIC24FJ128DA210, PIC24FJ256DA210, PIC24FJ64GA306, PIC24FJ64GA308, PIC24FJ64GA310, PIC24FJ128GA306, PIC24FJ128GA308, PIC24FJ128GA310, PIC24FJ128GB206, PIC24FJ256GB206, PIC24FJ128GB210, PIC24FJ256GB210, PIC24FJ64GC006, PIC24FJ64GC008, PIC24FJ64GC010, PIC24FJ128GC006, PIC24FJ128GC008, PIC24FJ128GC010

I also added a basic part creator wizard:

And for convenience, you can download my modified PK2DeviceFile.dat which supports all the above chips.

Did it Work?

Yes! My PIC programs fine and verifies! 

However, I seem to be having a few problems when the programming executive is enabled; the program memory repeats at address 0x1000 and 0x2000. Disabling the programming executive fixed that though. (Tip: It can be disabled by adding a PICkit2.ini in the PicKit2 GUI folder, running PicKit2 once, and setting PE24: N)

If anyone finds this useful, leave a comment below! If there's enough interest, I can add more chips to my patcher.

Update: This will also work fine with the PicKit3 GUI! However I can't seem to find a way to update the MPLABX side of things.

View Comments...