A bit sidelined by Xmas, but still:
Much work on the DrivenData Cloud Cover Competition. I got the
benchmark solution implemented, then looked around for a better
approach. I decided to try implementing Weakly-Supervised Cloud
Detection with Fixed-Point GANs by Joachim Nyborg and Ira
Assent, which is a paper with code
(https://github.com/jnyborg/fcd). I’ve been slowly plugging away at
it, and recently finished a first full training run on a very small
subset of training data, though I still need to get a working
There were a lot of things I had to change (because the code was
designed for the author’s problem, not mine) and fix (because the
code did not always work as described); I plan on submitting the
fixes back to the author, or at least letting them know about them.
This was done on two different machines. The first is a Fedora
server I have at home, an Intel NUC with 16 GB of memory. It
doesn’t have a GPU or a GUI (all text Emacs on this one), but it
works pretty well. It’s got more memory free than my laptop (which
also has 16GB but also runs browsers), and enough disk space that I
don’t need to think about it too much. This worked well for…let’s
call it minute-to-minute development: working in my editor, banging
on bugs and adding the features I wanted, committing to git and
pushing to the remote repo.
The training was all done on Microsoft’s Planetary Computer
platform, which gives me access to JupyterLab and a GPU. It has a
lot less disk space (~40GB or so, compared to ~500GB at home), no
Emacs 🤯 and no
make, but it does have
workflow was usually to commit at home, push to git, then pull in
Jupyterlab. This was almost all in the shell, btw; the code I’m
working with was all regular Python modules, not Jupyter notebooks.
There were some times when a notebook would have been handy, but so
far I’m finding it hard to switch easily between the two – figuring
out easy ways to duplicate the parts of
main() I want seems
non-trivial. Overall, this workflow worked pretty well for me.
As for Planetary Computer: it’s had some hiccups, but overall I’ve
been quite satisfied. This is a free trial, given because they’re
sponsoring this competition. Disk space is a constraint, but for a
free account I can’t really complain. And it continues to be
wonderful – and shocking – to see Microsoft embrace Free Software
in this way. No, they haven’t open-sourced Windows, but this still
feels like pigs flying.
With Xmas and family being the focus of the month, this took up most
of my hobby time.
- Upgrade servers to Debian 12. A bit tedious, but painless. Bless
Debian and all who work on it.
- The endoscopes I ordered last month came in, so I played with
those a bit. I think they should do for the birdhouse project,
though low light may be a problem. Set up a burner Android phone to
get the streaming application working, which was fine if still
sketchy-looking AF. It also works well enough over USB, though with
an interesting lag that kicks in if there’s too much change in the
picture – for example, if you wave it around a lot, rather than
just keeping it focused on a smaller area.
This month, I actually got to meet with Jennifer Whiteside, my MLA
and BC’s Minister of Education, about natural gas
fracking in BC. I was there with two other folks from New
Westminster, and I think it went fairly well. She agreed to meet
with us again in a few months.
Lots of letters.
- First graph for the birbcam! We’re posting data to InfluxDB, so I
get to play with it in Grafana (see below). Lots more to do, but this is a good start.
Ordered a couple wifi endoscopes for the next project: a birdhouse camera.
The tipping bucket precipitation meter that’s part of the weather
station will sometimes record a lot of tips in a very short
time…like, a ridiculous number. I’ve tried various things to
filter out spurious signals, but we still see them from time to
time. This time, I tried setting a threshold in the graph –
exclude measurements with more than 5 tips in a 30 second period.
This brings things down to a sane level. Thing is, after
experimentation it is possible to have about a tip per second or
so – but that requires literally pouring water into the funnel
constantly to keep it full, and even in the midst of this I
don’t think we were getting that much rain.
Machine learning/data science:
Worked on the Kaggle Time Series course. Left some
feedback. Still not done.
Worked on visualizing data for Russet, a project I’ve let languish
for a while. The goal was to take pictures out my office window,
which faces a lot of trees (mix of deciduous & conifers), and try to
see if I could track changes in the average colour over time. The
result – just a first pass – is a half-hour animated bar
chart. It’s definitely interesting to see the change over the
course of hours, days and months. But it is also a half hour bar
chart. Lots of room for improvement.
- Observations & IDs for iNaturalist, observations for Nature’s
Notebook. I’m still managing to find new things to look at.
That was October. Of 2021. Time is weird, yo.
More letters each week, but now with a reply from the BC environment
minister now that the update to BC’s climate plan has come out.
Lots of reading through that and figuring out how I feel about it.
More asking my MLA (who’s the education minister) for a meeting to
talk about fracking. No response.
Attended a protest for the first time since university. Felt
awkward…but if that’s the worst, I’ll keep doing this.
Playing with a TM1638 module I ordered.
BIRBCAM! Set up cheap binoculars so that they’re focused on a bird
feeder at my in-laws' house; set up a webcam behind them; get the
Coral dev board connected to their wifi; get Motion running on it;
and get Motion taking pictures, then getting an example script to
analyze the pictures post-hoc. The identification is a little all
over the map, but as a bird detector it works great. Pic can be
I ordered a handheld anemometer a while back; it arrived, and I
realized it did Bluetooth. That led me down the path of trying to
decode the packets, rather than install the dodgy-looking app that
I’m sure is totally fine, not even a problem Still not figured out
My wife and I were out at the Coquitlam River Park today, walking
along the trails. We hadn’t been there before, and it was amazing to
see the river flow. It was fast, and it was easy to imagine the bad
things that would happen if it flooded.
On our way back, we crossed Patricia Bridge, a small footbridge
that took us back to where we’d parked. For fun, I decided to try
guestimating the river flow, and then see if I could find actual data
on it once we got home. I figured I probably wouldn’t be terribly
accurate, but it would be interesting to see how wrong I was – and
By dropping a stick in the river & watching it go, I estimated that
the water speed was something like 2 metres/second (at least, near the
bank). By pacing out the footbridge, I estimated the width to be 30
metres. I had no clue about the depth of the water, so I decided to
call it 5 metres; similarly, to simplify things I decided to assume an
oblique triangular profile for the river bottom. That gave me a
surface area of 75 m^2 (that is, of the face of water from the bottom of
the river to the top); assuming 2m/s, that gave me a volume of 150
Is there real data on this? You damn betcha! Station 08MH002 is
maybe a kilometer from Patricia Bridge, and even has a graph:
The damn tooltip doesn’t show up in the screenshot, but it read 62.3
m^3/s at 2:15 PM PDT – right about when I was pacing off the breadth of
the river. That puts me off by about 2.5X. I was hoping to be closer
So what did I have wrong? Well, the depth – which the same graph
shows as 8.871 metres at the same time. That doesn’t help my model
any…that would make my volume about 266 m^3/s.
What about flow? My model assumes that water flow is the same from
top to bottom; is that correct? Probably not! (Incidentally, I
didn’t know that ResearchGate had its own StackExchange-like Q&A
feature…) The linked paper has this graph:
which came from this USGS publication.
At this point, I fell down another rabbit hole around modelling
velocity distribution in rivers, other papers giving empirical
results on velocity distribution, and so on. This
article gave Manning’s Equation, which I was excited about
until I realized this doesn’t really apply here. But it was
interesting reading about Manning himself.
I was hoping to get a rough-and-ready formula to figure out the
average velocity distribution, but that didn’t come up in my very
quick, not terribly attentive reading. I wonder if I could just use a
multiplier of 0.4 (my estimate vs what the readings were) as an
Also…as far as the big surge in data goes, my assumption is that the
dam that’s upstream released a bunch of water, given the very sharp
rise that occurred right at midnight. Still digging into that.
What did I do this month? Let’s see.
Machine learning/data science
More work on MLHub’s Earth observation & machine learning bootcamp
Participated in Kaggle’s 30 Days of Machine Learning
course/contest. Some of it was stuff I already knew, but it was a
good prod to do a bit of ML (nearly) every day.
Automated importing walking data from my phone. Since I’ve got an
iPhone (sigh) and Linux, this means exporting data from the Health
app, emailing it to myself, then processing it with Python to add it
to InfluxDB with the help of this repo. Since I got my phone
in 2018, I’ve walked about 7,650 km – here to St John’s,
Newfoundland is only 6,800 km.
Also automated importing air quality data downloaded from the BC
As I mentioned last month, I began some experiments to track the
running time of some battery-powered fans – some with batteries
built in, some that rely on external batteries through USB
connections. I got one of the ubiquitous USB voltage meters,
and it’s perfect for this.
Began monitoring sound levels in my office with a Raspberry Pi and
the Seeed ReSpeaker 2-mic hat. It would have been really good
to get this working before the pandemic hit, because I think it
would have demonstrated the change in traffic noise due to the
pandemic…but better late than never.
Finally added soil temperature probes to my father-in-law’s garden.
We’ve got three at different depths: 1 foot down, 2 feet, and 3
feet. The trends so far have been pretty cool:
Note the dual scales – air temp (green shaded line) on the left,
soil temp (yellow/blue/orange) on the right. Fascinating to see how
the change in temperature is buffered at different depths.
More GIS podcasts/courses.
I made a dirt-simple Arduino GPS logger that used a small GPS module
I got as a gift from my father, and managed to map the results.
I fired up a long-dormant account on OpenStreetMap.org and
added a bunch of little free libraries (“public bookshelves” is the
tag OSM uses). Fun to do.
Letters every Sunday to government – mostly provincial, because of
the Federal election in progress right now.
Submitted a letter against the Tilbury LNG port expansion,
which is right near me.
Joined Follow This, an organization dedicated to shareholder
activism in energy companies – Shell, BP, Chevron and Total.
So late! Let’s catch up.
More work on the weather station to accommodate the one-wire sensors
intended for the ground, and to prepare for the anemometer that’s
been built. I think we’re going to skip calibrating the anemometer,
and just record RPM.
Set up (finally!) a sound card hat for the Pi to catch ambient noise
levels in my home office; we’re right by a major road, so my hope is
that this will let me track traffic levels by proxy. I’d thought
about this when COVID hit, but didn’t get on this for a long time.
Helping out Ayush Bansal, our Google Summer of Code student; his
final report will be coming out any moment, and I’ll mention that in
(checks watch) 7 days.
ESA turned down our proposal to run code on OPS-SAT :-(, but another
Libre Space Foundation project got accepted. :-)
More listening to GIS podcasts and courses.
More work on the tree map, including trying to get Bootstrap
working. I think this was a bit ambitious for me, though – I need
a much better foundation in the basics of web development.
Machine learning/data science
- First road trip, to see what it’s like to drive longer distances.
This was only about 270km round trip, but it was illuminating.
Props to the fast charging station in Chilliwack at City Hall.
- We went through the big heat wave; outside temperatures hit
45.1 C in the sun at my inlaws' pace, and 32.8 C indoors at my
place; as for the max overnight temperature, it was 24.9 C at my
inlaws, and 29.1 C at my place:
This scared the shit out of me. A number of things have come out of
For a start, my wife & I have begun talking about emergency
cooling. We’re in a townhouse and have no AC; we coped by staying
indoors, and taking us all out to malls & other places with
cooling. If there had been a widespread power outage, we would have
been in serous trouble. We’ve decided to start trying to prepare
for that, much as we try to prepare for an earthquake.
I’ve signed up for an energy efficiency assessment for our house,
which is something I’ve been meaning to do for the longest time.
Hopefully we can find some cooling options that are energy-efficient.
I’ve started with some battery-powered fans, and am running some
experiments to see how long they can run on simple battery banks
(like for charging phones). This is partly to get a bit of
experience, partly to make sure I don’t run out and spend a bunch of
money on something useless, and partly to – honestly – give myself
a sense of control by having some experiments to do. It’s not the
only thing we’ll do, but it’s a start. I’ll write this up later.
Another thing that came out of this is a commitment to writing my
local, provincial and federal governments every week for a year on
the climate emergency. This month: writing my provincial
government to end old-growth logging in BC.
Patrick Johnstone, one of my city’s councillors, wrote a
heartbreaking blog post about the heat wave and how New
It was a cascading failure, a demonstration we were simply not
ready, as a City and as a Province. People died, leaving behind
families and neighbours traumatized by the lack of response. I am
afraid first responders were equally traumatized, as they had to
operate in a broken and failing system that didn’t allow them to
do the work they are trained for and dedicated to doing – protect
and comfort the residents they serve. Instead, they spent three
days in the stifling heat surrounded by the suffering and death of
people they wanted to help. I cannot imagine, but once again, they
deserve not just our recognition and gratitude, but a response – a
way to fix this so they don’t have to go through it again.
June: sick or whack? Let’s crunch the numbers.
Lots of work mentoring our GSOC student, who’s doing amazing work.
Work on a presentation about analysis of QUBIK data with Polaris.
A lot of work with another Polaris member digging into dependency
problems; written up here, MR merged in July.
Submitted our proposal for running code on OPS-SAT. I’m
incredibly excited about this. 🤞
- More Aleasat meetings and helping them out as I can.
Start graphing EV efficiency data for our Kia Soul: cookiecutter
repo, import into InfluxDB, graph in Grafana.
More work on New West Trees. Signed up for a free account on Carto,
thanks to this tutorial; I’ll look at hosting this on PostGIS
locally, but for now this gets me started. Current state:
- Able to search for 5 nearest trees
- Able to display just a particular species of tree
- Able to mark all the unknown trees with a separate icon
- Able to display this on my phone without crashing, thanks to Leaflet.markercluster
Still lots to do, but I’m happy.
Got ethernet breakout boards for the weather station, which allow me
to use cat6 cable to take readings from Dallas 1-wire temperature
sensors. These will get buried in the soil at my inlaws' garden. A
lot of soldering work to get this done, and then rebuilding the
Arduino software for the first time in years. Oh, and setting up
udev rules to create static rules for
/dev/sds011. …which I haven’t mentioned yet!
Bought a couple SDS011 particulate matter sensors; I’ve added one to
the weather station, and one at home. Interesting to see how
- Outing to local park; one QSO, truly awful signal reports from RBN.
Not sure what’s going on.
What happened in May? Let’s see.
- Initial analyses for OPS-SAT, BOBCAT-1, and QUBIK
- PRs to resolve a few small issues
- Help review abstracts for conference presentations
- Lead preparation of a proposal to run code on OPS-SAT. I’m
super excited about this.
- Played a bit with the nanosat-mo-framework in preparation for
- We’ve got a Google Summer of Code student, Ayush Bansal! 🎉 Very
much looking forward to working with him.
- I’ve been asked to be an advisor for ALEASAT, a cubesat project
being built by UBC and SFU students. I’m incredibly thrilled about
Replaced rain sensor on weather station at my in-laws
Tested running 3 Dallas 1-wire sensors over a 25 foot / 7.5 metre
ethernet cable: one twisted pair element each for positive, ground
and signal. Worked a treat! These are going to be buried in the
garden there to get soil temperatures at different depths
First POTA activation: Ve-3300, Cariboo Hill Park. 21 contacts,
including 2 park-to-park. Closest I’ve come yet to a pileup.
Power went out at my house for a few hours, so I used the time to
make contacts on my homebrew magloop on 20m while it was dead quiet.
Made England, plus one with KD6JUI/MM, who was kayaking (!) with a
homebrew magloop (!!).
CQ WPX contest: 55 contacts over 3 days. I’ll be honest, it was a
bit of a chore by the end. But I managed to make New Zealand on 5W,
and Australia on 5W on 40m (!).
A little late (hah!), but still trying to keep the habit.
A lot of work getting ready for Google Summer of Code – our third
Initial analyses for a couple different satellites: QUBIK-1 and -2
(using data from integration testing), OPSSAT (see below for why).
Documentation improvements, always important.
Begin working (with a crapton of other people!) on a proposal to
run our software on OPSSAT. This has been a lot of fun.
Machine learning / data science
More work on the dishwasher loading critic; not as much as I would
have liked, though. But I did pay my son to annotate ~ 100 images. 🤘
Got my tree map page put up on this website.
- Replace failing hard drive for Zombie, the home server that does it
- More work on the anemometer. My father-in-law built a shelter for
this to keep the rain off, and we’ve now got the sensors/magnets
permanently (*with crazy glue) mounted on the arms.
- First attempt at POTA, at a local park. Unfortunately, I only got
four QSOs, so no good. I think part of that is probably due to the
location: it was in a lower part of the park, and it seemed to