A while back I started exploring data from the Reverse Beacon Network. My initial goal had been to come up with an ML model to predict how many DX stations the local skimmer would receive – but there was a lot of exploration of the data as well. I captured that exploration in a series of notebooks, and set aside the project after a while.

One of the things I never accomplished was a satisfying display of where stations were being received from. I was aiming for something that would show changes over time, as well as location. Yesterday I was browsing through this Kaggle notebook for the BirdCLEF 2021 competition when I saw a cool map being generated from something called a shape file. A bit of browsing through the Internet found some great tutorials, and I think I have a better sense of what I can do.

Animation

First off, a choropleth map seems like a good first step – not exactly what I want, but with Plotly it seems like the initial animated view should be pretty simple. It can be exported as a gif, or even as an MP4.

This tutorial gets into the weeds with matplotlib to do the animation.

Maps

This tutorial also shows using matplotlib to draw the map, which is another way to get that done.

There’s jupyter-gmaps, a library for displaying Google Maps in a notebook.

For OpenStreetMap, there’s this tutorial from ArcGIS and IPyLeaflet. (God, I wish I’d known about that…) IPyLeaflet also has an amazing series of notebooks for experimenting. And this article has a lot of great demos.

Github supports rendering GeoJSON.

This article goes over timestamped GeoJSON files – brilliant! This article is probably closest to what I had in mind.

Libraries

• Cartopy
• Plotly
• Jupyter-Gmaps
• IPyLeaflet

Hello world. March felt busy.

Polaris

• The Libre Space Foundation (and thus Polaris) was accepted for the Google Summer of Code, and we had bunch of awesome students show up in our chat room. A lot of work came out of that: coaching students, evaluating their MRs, giving early feedback on proposals, and helping them find their way through the codebase and the problems. But these are definitely good problems to have!

• I prepared an initial analysis of data from the QUBIK satellites; the data was from integration testing, and we’re hoping to compare it with what we receive afterward. You can see the graphs for QUBIK-1 and QUBIK-2. Next up will be adding info to our documentationto show how we did this.

• A short blurb about Polaris will be going out in the IAF newsletter, which is cool!

Machine learning

• Finished up tracking down a bug in Detecto, a wrapper around PyTorch for object detection.

• Dig into more options for image augmentation, including Albumentation

• Came up with a rough prototype for the Dishwasher Loading Critic: a (poorly) trained model, sitting behind an API written in Fast, with a copied bootstrap template. I was able to post pictures to it from my phone & get some (poor) bounding boxes around things. Progress!

• Still trying to figure out where I want to go with this project: stick with Detecto, or move to PyTorch? I’d like to do the latter, but I have a lot of learning to do there.

• Got LSP-mode enabled for Emacs. Interesting, and I suspect this will be a way forward for Emacs.

• Tried Paperspace again after their upgrade, and WOW: it’s blazingly fast to start up. I’m going to re-open my account with them again.

Sysadmin

• Finally got Fedora 33 installed on an Intel NUC. The problem had been that wifi did not work after installation, even though it worked during installation. Turns out there’s a bug where wpa-supplicant is not installed during installation; installing it afterward by hand did the trick.

• Learned about nftables…huh.

Hardware hacking

• First prototype of anemometer working – I’m now able to get RPM read and displayed in Grafana. Apparently, the best option open to me for calibrating this thing is to use a car: hold it out the window, go at a set speed, and take measurements.

(Drafted with the help of x-hugh-blog-what-happened-last-month!)

Memo to myself: to set projectile’s project type to golang, create a .dir-locals.el file that looks like this:

((nil . ((projectile-project-type . go)))

Shortcut for editing a projects .dir-locals.el file: C-c p E and select projectile-project-type.

Here’s what I got up to in February 2021:

Polaris

• Recorded and presented my talk on Polaris for PyCascades. It went pretty well, I think.

Machine learning/data science

• Began Chapter 9 of the FastAI book. This is on tabular learning, which is really interesting; I think this is the sort of approach I’d want to take for loostmap, my attempt to predict HF propagation by looking at data from the Reverse Beacon Network (I picked that project name from a random name generator…I really need something that makes more sense.)

• Began playing with the New Westminster tree inventory, an open data file from my city. I’ve tried mapping that https://va7unx.space/trees, and the code can be found here.

• Played with Roboflow, an online service that augments image data for machine learning. Also came across imgaug, a Python library that covers much the same ground.

• Some work on the dishwasher loading critic, including beginning to work with PyTorch directly rather than using Detecto.

• Dig into what may (or may not) be a bug in Detecto with bounding boxes.

• Began feature engineering course on Kaggle.

• Talked to my manager about the possibility of looking for DS/ML projects at work. Apparently there’s one team he knows of that’s looking into a project in this area, and the possibility exists to work with them for a bit. 🤞

Hardware hacking

• My father-in-law finished a prototype of our anemometer; he’s a retired millwright, so he actually knows what he’s doing. (puts popsicle sticks and yarn away)

Radio

• A few contests entered. Closer to getting my WAS – only missing Maine and Nebraska, and state contests for those are coming up in the next few months.

• Reached Japan (7550 km) via CW on one watt!

• Sysadmin work for the club.

Home sysadmin

• Finally figured out an Apache and UTF-8 thing that had been bugging me for years

Birding

• Backyard bird count, plus started doing counts in local parks on weekend; submitted through Audobon app, which goes to ebird.org.

Gardening

• Began growing wildflower seedlings at home under a grow lamp and promptly got mildew. There are a couple that have survived; I plan on transplanting those & trying again.

A while back, I started having problems with the output of Venus, a planet-like aggregator I use to read a bunch of things. The symptoms were broken characters for things like apostrophes, quotes and so on – which rendered the output nearly unusable. I dug into it, but couldn’t resolve the problem…so I resorted to a bletcherous hack (cron job to copy the file to my laptop, and view it with file:///...) and blamed Python 2.

Today I came across the same problem but manifested in another set of files. This time I managed to find the answer:

AddCharset UTF-8           .htm .html .js .css

To be clear, I already:

• had made sure that the headers for the file included Content-Type: text/html;charset=utf-8
• had made sure the html file had <meta charset="utf=8">

Weirdly enough, changing that meta tag to:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" >

worked…the apostrophes and such were displayed correctly. But they never showed up in the output when I ran a curl on the URL. Does Apache filter this stuff on the fly?

Anyhow…that’s enough encoding debugging for one day. Or possibly a year.

Here’s a quick list, for my own reference, of what I got up to in January. It’s heartening to see everything laid out, and realize that I’ve actually managed to get a fair bit done!

Hardware hacking

• My father-in-law and I worked on getting the precipitation meter going for our weather station. It took a while, but we finally got it working. 🎉

• Some one-wire temperature sensors came in, and I was able to whip up a quick demo to make sure they worked.

• Talked to my father-in-law about building a Lehmann seismograph. Early days, but I think he’s in.

Polaris

• Worked on my upcoming talk on Polaris for PyCascades 2021. (Final version recorded just yesterday.)

• Got a hugo site whipped together for polarisml.space, which now has links to our demo, code, chat and documentation.

• Cut my first release for Polaris after getting our page on PyPi sorted out.

Machine learning

• Some progress, though slow, on going through the FastAI book.

• Tripped over Roboflow, which generates synthetic data for ML; very interesting, and I may give this a try for the dishwasher loading critic.

• Some initial experiments with detecto, a simple wrapper for PyTorch object detection.

Radio

• Not a whole lot of trips out, but some…and managing to reach D4Z Cape Verde on 10W. 9,155 km!

• Totalled up my contacts toward SKCC Centurion…42/100. Normally I’m not big on this sort of thing, but it’s a number to reach for, and that’s no bad thing right now.

Last year, my father-in-law got a trail cam at my suggestion – mainly to get pictures of the rats that were eating his compost. It worked:

I borrowed it a while back, and finally set it up today under our bird feeder to see what we could get. Not a bad haul! We got:

• Spotted towhees:

• Dark-eyed juncos:

• Black and grey squirrels:

• A chickadee:

Not bad!

Out for radio as well: 12 QSOs from the North American QSO contest, including D4Z from Cape Verde – about 9150km on 10W. Nice!

My project: critiquing your dishwasher loading technique using machine learning. A work in progress. You can find the repo here.

I’ve been interested in machine learning for a while now. Like a lot of things, my approach has been a bit scattered. I’m slowly learning how to get better at that, but I still tend to veer around.

A couple of months ago, I decided to take the Fast.ai course again. I had done a couple of lessons a year ago, but had not followed it up. This time around, I saw that they not only had a new version of the course, but a book as well. I ordered the book (and another book as well), and got started on the Jupyter notebooks that the book is based on.

Let’s see where this goes!