European Space Agency ETV-5 approaches the ISS va7unx.space

Hugh Brown, VA7UNX

What Happened in January 2022 calendar Apr 3, 2022

Welp…this took a long time to write up. In my defense, I got COVID in January and that sort of threw me off for a couple months. But it’s also just taken me a while to get back to it. Anyhow, onward!

Machine learning/data science

  • I submitted my entry to the Data Driven Cloud Cover Competition! Aaaaand…my score was terrible. However, I got a lot of practice out of this, and it was valuable for that. I intended to go back and figure out exactly why my scores were so abysmal, but got derailed (see first paragraph). But I think that for whatever reason, my GAN was just not working at all. I need to get more practice with this technique.

  • Some volunteer work for a local environmental society to demonstrate how to use Pandas for graphing.

Climate emergency

  • More letters to politicians. Did not make every week, but I’m cutting myself some slack here.
What Happened in 2021: Year End Review Edition calendar Apr 3, 2022

What happened in 2021? Time to look back.

  • We got an EV! It’s a Kia Soul, and I love it.

  • Started writing in here semi-regularly. 💪

  • Expanded the weather station: precipitation meter, anemometer, soil temp, particulate matter sensors.

  • Recorded a talk for PyCascades!

  • Much work on machine learning and data science: the dishwasher loading critic, some Kaggle courses, mapping New Westminster trees.

  • A lot of work on Polaris: telemetry analyses, supervising our third co-op student, and a proposal to run code on an ESA satellite (sadly, denied).

  • Radio took a bit of a backseat by the end of the year.

  • I was asked to be an advisor for ALEASAT, and that was wonderful.

  • The heat wave scared me, and I turned that into a focus on climate activism. Small steps, but I’m taking them. Met with my MLA in November to discuss climate change. I think of this as being the start of about 30 years of work.

  • Total distance walked since getting my latest phone in January 2018: 8,664 km. Daily average distance in 2021 was 7.2 km, up from last year (6.4 km).

  • Got more into birdwatching, phenology and natural history. Lots of data taking, which I enjoy.

  • Bird feeder camera with ML to recognize the birds. Does a fairly crappy job of picking out species but a good job of detecting birds.

  • Entered a couple of ML contests – no wins, but that’s expected; it’s the practice I’m after.

  • At work: joined a new team which has a definite data science focus. Learning a lot.

I’m leaving out all the incredibly important time with my family; this isn’t the venue I choose to record that in.

What Happened in December 2021 calendar Jan 18, 2022

A bit sidelined by Xmas, but still:

Machine learning

  • Much work on the DrivenData Cloud Cover Competition. I got the benchmark solution implemented, then looked around for a better approach. I decided to try implementing Weakly-Supervised Cloud Detection with Fixed-Point GANs by Joachim Nyborg and Ira Assent, which is a paper with code (https://github.com/jnyborg/fcd). I’ve been slowly plugging away at it, and recently finished a first full training run on a very small subset of training data, though I still need to get a working submission going.

    There were a lot of things I had to change (because the code was designed for the author’s problem, not mine) and fix (because the code did not always work as described); I plan on submitting the fixes back to the author, or at least letting them know about them.

    This was done on two different machines. The first is a Fedora server I have at home, an Intel NUC with 16 GB of memory. It doesn’t have a GPU or a GUI (all text Emacs on this one), but it works pretty well. It’s got more memory free than my laptop (which also has 16GB but also runs browsers), and enough disk space that I don’t need to think about it too much. This worked well for…let’s call it minute-to-minute development: working in my editor, banging on bugs and adding the features I wanted, committing to git and pushing to the remote repo.

    The training was all done on Microsoft’s Planetary Computer platform, which gives me access to JupyterLab and a GPU. It has a lot less disk space (~40GB or so, compared to ~500GB at home), no Emacs 🤯 and no make, but it does have git and vim. My workflow was usually to commit at home, push to git, then pull in Jupyterlab. This was almost all in the shell, btw; the code I’m working with was all regular Python modules, not Jupyter notebooks. There were some times when a notebook would have been handy, but so far I’m finding it hard to switch easily between the two – figuring out easy ways to duplicate the parts of main() I want seems non-trivial. Overall, this workflow worked pretty well for me.

    As for Planetary Computer: it’s had some hiccups, but overall I’ve been quite satisfied. This is a free trial, given because they’re sponsoring this competition. Disk space is a constraint, but for a free account I can’t really complain. And it continues to be wonderful – and shocking – to see Microsoft embrace Free Software in this way. No, they haven’t open-sourced Windows, but this still feels like pigs flying.

    With Xmas and family being the focus of the month, this took up most of my hobby time.

Home sysadmin

  • Upgrade servers to Debian 12. A bit tedious, but painless. Bless Debian and all who work on it.

Hardware hacking

  • The endoscopes I ordered last month came in, so I played with those a bit. I think they should do for the birdhouse project, though low light may be a problem. Set up a burner Android phone to get the streaming application working, which was fine if still sketchy-looking AF. It also works well enough over USB, though with an interesting lag that kicks in if there’s too much change in the picture – for example, if you wave it around a lot, rather than just keeping it focused on a smaller area.

Climate emergency

  • More climate letters.
What Happened in November 2021 calendar Dec 18, 2021

Climate emergency:

Hardware hacking:

  • First graph for the birbcam! We’re posting data to InfluxDB, so I get to play with it in Grafana (see below). Lots more to do, but this is a good start.

1

  • Ordered a couple wifi endoscopes for the next project: a birdhouse camera.

  • The tipping bucket precipitation meter that’s part of the weather station will sometimes record a lot of tips in a very short time…like, a ridiculous number. I’ve tried various things to filter out spurious signals, but we still see them from time to time. This time, I tried setting a threshold in the graph – exclude measurements with more than 5 tips in a 30 second period. This brings things down to a sane level. Thing is, after experimentation it is possible to have about a tip per second or so – but that requires literally pouring water into the funnel constantly to keep it full, and even in the midst of this I don’t think we were getting that much rain.

Machine learning/data science:

  • Worked on the Kaggle Time Series course. Left some feedback. Still not done.

  • Worked on visualizing data for Russet, a project I’ve let languish for a while. The goal was to take pictures out my office window, which faces a lot of trees (mix of deciduous & conifers), and try to see if I could track changes in the average colour over time. The result – just a first pass – is a half-hour animated bar chart. It’s definitely interesting to see the change over the course of hours, days and months. But it is also a half hour bar chart. Lots of room for improvement.

Natural History

  • Observations & IDs for iNaturalist, observations for Nature’s Notebook. I’m still managing to find new things to look at.
What Happened in October 2021 calendar Nov 6, 2021

That was October. Of 2021. Time is weird, yo.

Climate Emergency

  • More letters each week, but now with a reply from the BC environment minister now that the update to BC’s climate plan has come out. Lots of reading through that and figuring out how I feel about it.

  • More asking my MLA (who’s the education minister) for a meeting to talk about fracking. No response.

  • Attended a protest for the first time since university. Felt awkward…but if that’s the worst, I’ll keep doing this.

Hardware hacking

  • Playing with a TM1638 module I ordered.

  • BIRBCAM! Set up cheap binoculars so that they’re focused on a bird feeder at my in-laws’ house; set up a webcam behind them; get the Coral dev board connected to their wifi; get Motion running on it; and get Motion taking pictures, then getting an example script to analyze the pictures post-hoc. The identification is a little all over the map, but as a bird detector it works great. Pic can be found here.

  • I ordered a handheld anemometer a while back; it arrived, and I realized it did Bluetooth. That led me down the path of trying to decode the packets, rather than install the dodgy-looking app that I’m sure is totally fine, not even a problem Still not figured out

Data science/ML/GIS

River Flow Guestimates and Hard Data calendar Oct 23, 2021

My wife and I were out at the Coquitlam River Park today, walking along the trails. We hadn’t been there before, and it was amazing to see the river flow. It was fast, and it was easy to imagine the bad things that would happen if it flooded.

On our way back, we crossed Patricia Bridge, a small footbridge that took us back to where we’d parked. For fun, I decided to try guestimating the river flow, and then see if I could find actual data on it once we got home. I figured I probably wouldn’t be terribly accurate, but it would be interesting to see how wrong I was – and maybe why.

By dropping a stick in the river & watching it go, I estimated that the water speed was something like 2 metres/second (at least, near the bank). By pacing out the footbridge, I estimated the width to be 30 metres. I had no clue about the depth of the water, so I decided to call it 5 metres; similarly, to simplify things I decided to assume an oblique triangular profile for the river bottom. That gave me a surface area of 75 m^2 (that is, of the face of water from the bottom of the river to the top); assuming 2m/s, that gave me a volume of 150 m^3/s.

Is there real data on this? You damn betcha! Station 08MH002 is maybe a kilometer from Patricia Bridge, and even has a graph:

Station 08MH002

The damn tooltip doesn’t show up in the screenshot, but it read 62.3 m^3/s at 2:15 PM PDT – right about when I was pacing off the breadth of the river. That puts me off by about 2.5X. I was hoping to be closer than that!

So what did I have wrong? Well, the depth – which the same graph shows as 8.871 metres at the same time. That doesn’t help my model any…that would make my volume about 266 m^3/s.

What about flow? My model assumes that water flow is the same from top to bottom; is that correct? Probably not! (Incidentally, I didn’t know that ResearchGate had its own StackExchange-like Q&A feature…) The linked paper has this graph:

Velocity River Profile

which came from this USGS publication.

At this point, I fell down another rabbit hole around modelling velocity distribution in rivers, other papers giving empirical results on velocity distribution, and so on. This article gave Manning’s Equation, which I was excited about until I realized this doesn’t really apply here. But it was interesting reading about Manning himself.

I was hoping to get a rough-and-ready formula to figure out the average velocity distribution, but that didn’t come up in my very quick, not terribly attentive reading. I wonder if I could just use a multiplier of 0.4 (my estimate vs what the readings were) as an empirical heuristic?

Also…as far as the big surge in data goes, my assumption is that the dam that’s upstream released a bunch of water, given the very sharp rise that occurred right at midnight. Still digging into that.

What Happened in September 2021 calendar Oct 17, 2021

Machine learning:

  • Entered the STAC Overflow (get it?) contest. I managed to go through three iterations of my model, and improve a bit over the baseline model they had as a tutorial. I finished in 71st place out of 664 – not bad! However, my final score (0.5314) was nowhere near the winners; the top four were all over 0.8. Still, this was a good exercise.

Mapping/GIS:

Hardware hacking:

  • More fan runtime experiments. It’s interesting to see the different battery behaviours.

  • Got the anemometer hooked up to the weather station and working at last! 🎉 The one thing it’s not is calibrated – so I’ve got RPM, but I don’t know what that translates to in wind speed. Yes, you can calculate the circumference of one rotation & figure it out from there, but…well, it’s complicated.

  • This also required rejigging the cable on the tipping bucket rain meter to use the same cat6 cable I used for the anemometer. Twisted pair cabling, people, it’s the bomb.

  • Big refactoring of the Arduino code for the weather station; it’s a lot more readable now. And I’m reasonably confident that my floating point math is probably okay.

  • Begin plans for a bird feeder camera. I’ve got some Coral dev boards, and it turns out the example code for it includes a bird species recognition model based on iNaturalist data. Bought a cheap pair of binoculars to try using as a telephoto lens for a webcam.

Nature/science:

  • Continuing phenology measurements for Nature’s Notebook.

  • Lots of observations for iNaturalist and eBird, including submitting some historical bird counts from the past year.

  • After a lot of looking around, I bought two big hardcover sketch books to use as phenology/nature journals: one page per day, and observations from each year on that page.

Climate emergency:

What Happened in August 2021 calendar Sep 11, 2021

What did I do this month? Let’s see.

Machine learning/data science

  • More work on MLHub’s Earth observation & machine learning bootcamp

  • Participated in Kaggle’s 30 Days of Machine Learning course/contest. Some of it was stuff I already knew, but it was a good prod to do a bit of ML (nearly) every day.

  • Automated importing walking data from my phone. Since I’ve got an iPhone (sigh) and Linux, this means exporting data from the Health app, emailing it to myself, then processing it with Python to add it to InfluxDB with the help of this repo. Since I got my phone in 2018, I’ve walked about 7,650 km – here to St John’s, Newfoundland is only 6,800 km.

  • Also automated importing air quality data downloaded from the BC government.

Polaris

Hardware hacking

  • As I mentioned last month, I began some experiments to track the running time of some battery-powered fans – some with batteries built in, some that rely on external batteries through USB connections. I got one of the ubiquitous USB voltage meters, and it’s perfect for this.

  • Began monitoring sound levels in my office with a Raspberry Pi and the Seeed ReSpeaker 2-mic hat. It would have been really good to get this working before the pandemic hit, because I think it would have demonstrated the change in traffic noise due to the pandemic…but better late than never.

  • Finally added soil temperature probes to my father-in-law’s garden. We’ve got three at different depths: 1 foot down, 2 feet, and 3 feet. The trends so far have been pretty cool:

Grafana temperature graph for August 2021

Note the dual scales – air temp (green shaded line) on the left, soil temp (yellow/blue/orange) on the right. Fascinating to see how the change in temperature is buffered at different depths.

Mapping/GIS

  • More GIS podcasts/courses.

  • I made a dirt-simple Arduino GPS logger that used a small GPS module I got as a gift from my father, and managed to map the results.

  • I fired up a long-dormant account on OpenStreetMap.org and added a bunch of little free libraries (“public bookshelves” is the tag OSM uses). Fun to do.

Nature/science

Climate emergency

  • Letters every Sunday to government – mostly provincial, because of the Federal election in progress right now.

  • Submitted a letter against the Tilbury LNG port expansion, which is right near me.

  • Joined Follow This, an organization dedicated to shareholder activism in energy companies – Shell, BP, Chevron and Total.

What Happened in July calendar Aug 24, 2021

So late! Let’s catch up.

Hardware hacking

  • More work on the weather station to accommodate the one-wire sensors intended for the ground, and to prepare for the anemometer that’s been built. I think we’re going to skip calibrating the anemometer, and just record RPM.

  • Set up (finally!) a sound card hat for the Pi to catch ambient noise levels in my home office; we’re right by a major road, so my hope is that this will let me track traffic levels by proxy. I’d thought about this when COVID hit, but didn’t get on this for a long time.

Polaris

  • Helping out Ayush Bansal, our Google Summer of Code student; his final report will be coming out any moment, and I’ll mention that in (checks watch) 7 days.

  • ESA turned down our proposal to run code on OPS-SAT :-(, but another Libre Space Foundation project got accepted. :-)

Mapping

  • More listening to GIS podcasts and courses.

  • More work on the tree map, including trying to get Bootstrap working. I think this was a bit ambitious for me, though – I need a much better foundation in the basics of web development.

Machine learning/data science

  • Start going through MLHub’s Earth observation & machine learning bootcamp

  • First work on the dishwasher loading critic project in a while

  • Upgrade Paperspace to a paid account (which I still haven’t used very much 😬)

EV

  • First road trip, to see what it’s like to drive longer distances. This was only about 270km round trip, but it was illuminating. Props to the fast charging station in Chilliwack at City Hall.

Climate Emergency

  • We went through the big heat wave; outside temperatures hit 45.1 C in the sun at my inlaws’ pace, and 32.8 C indoors at my place; as for the max overnight temperature, it was 24.9 C at my inlaws, and 29.1 C at my place:

Grafana temperature graph for August 2021 heat wave

This scared the shit out of me. A number of things have come out of that.

  • For a start, my wife & I have begun talking about emergency cooling. We’re in a townhouse and have no AC; we coped by staying indoors, and taking us all out to malls & other places with cooling. If there had been a widespread power outage, we would have been in serous trouble. We’ve decided to start trying to prepare for that, much as we try to prepare for an earthquake.

  • I’ve signed up for an energy efficiency assessment for our house, which is something I’ve been meaning to do for the longest time. Hopefully we can find some cooling options that are energy-efficient.

  • I’ve started with some battery-powered fans, and am running some experiments to see how long they can run on simple battery banks (like for charging phones). This is partly to get a bit of experience, partly to make sure I don’t run out and spend a bunch of money on something useless, and partly to – honestly – give myself a sense of control by having some experiments to do. It’s not the only thing we’ll do, but it’s a start. I’ll write this up later.

  • Another thing that came out of this is a commitment to writing my local, provincial and federal governments every week for a year on the climate emergency. This month: writing my provincial government to end old-growth logging in BC.

  • Patrick Johnstone, one of my city’s councillors, wrote a heartbreaking blog post about the heat wave and how New Westminster responded:

    It was a cascading failure, a demonstration we were simply not ready, as a City and as a Province. People died, leaving behind families and neighbours traumatized by the lack of response. I am afraid first responders were equally traumatized, as they had to operate in a broken and failing system that didn’t allow them to do the work they are trained for and dedicated to doing – protect and comfort the residents they serve. Instead, they spent three days in the stifling heat surrounded by the suffering and death of people they wanted to help. I cannot imagine, but once again, they deserve not just our recognition and gratitude, but a response – a way to fix this so they don’t have to go through it again.

What Happened in June calendar Jul 5, 2021

June: sick or whack? Let’s crunch the numbers.

Polaris

  • Lots of work mentoring our GSOC student, who’s doing amazing work.

  • Work on a presentation about analysis of QUBIK data with Polaris.

  • A lot of work with another Polaris member digging into dependency problems; written up here, MR merged in July.

  • Submitted our proposal for running code on OPS-SAT. I’m incredibly excited about this. 🤞

Space

  • More Aleasat meetings and helping them out as I can.

Data science

  • Start graphing EV efficiency data for our Kia Soul: cookiecutter repo, import into InfluxDB, graph in Grafana.

  • More work on New West Trees. Signed up for a free account on Carto, thanks to this tutorial; I’ll look at hosting this on PostGIS locally, but for now this gets me started. Current state:

    • Able to search for 5 nearest trees
    • Able to display just a particular species of tree
    • Able to mark all the unknown trees with a separate icon
    • Able to display this on my phone without crashing, thanks to Leaflet.markercluster

    Still lots to do, but I’m happy.

Hardware hacking

  • Got ethernet breakout boards for the weather station, which allow me to use cat6 cable to take readings from Dallas 1-wire temperature sensors. These will get buried in the soil at my inlaws’ garden. A lot of soldering work to get this done, and then rebuilding the Arduino software for the first time in years. Oh, and setting up udev rules to create static rules for /dev/weatherstation and /dev/sds011. …which I haven’t mentioned yet!

  • Bought a couple SDS011 particulate matter sensors; I’ve added one to the weather station, and one at home. Interesting to see how they’re doing.

Radio

  • Outing to local park; one QSO, truly awful signal reports from RBN. Not sure what’s going on.