Category: Other nerd pursuits

Tonight we venture into the world of Charlieplexing. Don’t be late.

Some of us may slide over into the main Monday channel after class, if we see anyone there. So if you wish to interact, hang out in the Monday voice channel around 7:30ish.

So here’s what happens when you give a data junkie an API, MySQL, a few hours of Python import coding, and 15 minutes to install and learn Grafana. All of this data is directly imported from the Oura Ring API.

I swore off of fitness data acquisition a while back. I hated the way big tech invasively mines your data, and I hated the Black Mirror-level machinations they seem to have for it.

But when I caught Covid at Def Con last year, a friend mentioned in passing that NIH was doing a Covid study and needed volunteers. I looked into it, and it seemed interesting, so I signed up. The study provided a BioStrap Evo wristband and an underarm temperature sensor. The study was gathering metrics to explore the relationships between bio data and long covid, I guess to try to create better predictions of who is at risk for long covid, etc.

It was a six-month study. The BioStrap collects interesting data, especially on sleep cycles, and I grew rather accustomed to looking at that data. I found it helpful, and started looking into the best options for collecting fitness and workout data without necessarily providing it to big tech.

Also, most of the services are subscription services, and it doesn’t make sense to me to subscribe to multiple services that essentially provide the same thing, with maybe small variations on how the data is presented.

I’m a data junkie at heart, I love having access to raw data to create my own presentations, interpretations, and correlations. I’m also a lazy engineer, so I prefer automating things up front and avoiding manual or repetitive work. So I looked forward to my options.

I wasn’t particularly fond of the BioStrap form factor, to be honest. It’s a wristband without a display, so you’re relying on the app. The bluetooth comms between the wristband and the app can be sketchy at times, requiring bluetooth resets, phone reboots, and sometimes deleting and re-adding the device profile to re-establish comms. Data is lost when that happens.

I started looking at competing products. I read a lot of good things about the Oura Ring, which I had looked at a year or so earlier, but rejected because I was still in the anti-tech-data mode. I discovered they have an API, so you have access to the data collected. So I bought one, and have been mucking with it for a couple of weeks now. The stats seem to be pretty close to what the BioStrap provided, but I like the ring form factor much better than the wristband (which makes me look like I’m wearing two watches), and the charge lasts for several days, where the BioStrap pretty much needs a charge daily.

This morning, I woke up with the intent of exploring that API a bit.

So far I’ve been able to pull in workouts for the past couple of weeks, including date, type of workout, intensity, length of workout, and calories burned (based on Oura’s calculations, of course). I’m also able to pull in the “tags,” which is sort of a custom comment field where you can add a mood, or an activity, for correlation. Some examples include noting when you drink alcohol, or getting a massage, and looking at how those activities affect your heart rate, sleep, and “recovery” level.

I’m also able to log workouts into my self-hosted Gitlab instance, which is not the best way to track workouts, but at least it accumulates time and comments. I’ve also installed a local instance of WGER, which is some odd combination of workout scheduling and “gym management.” It seems like WGER has a bit of an identity crisis going on. I tell you what, if WGER decided to go all in on allowing users to import fitness data from other apps with APIs, such as Oura, Google Fit, Strava, Apple, etc., I feel like the user base would explode.

It’ll take more time to pull in heart rate data, which seems to take averages every five minutes. That really seems to lend itself to something like MRTG, RRDTool, Grafana, etc. And the other data is in there as well. Looking at heart rate variation over time could be interesting, correlating it to workouts, sleep data, and other activities. I guarantee there are some DIY folks whose imaginations are bigger than BFT (Big Fitness Tech) when it comes to chewing on and presenting data.

But the really cool part is the automation. I’ve got it pulling the data in daily via cron job. I simply “record workout” when I start a workout, and end it when I’m done (via the phone app) and the rest of the collection and distribution of my data happens behind the scenes.

Look, I realize my lifestyle is a bit of an anachronism. I prefer keeping things working vs buying replacements when possible. I don’t mind getting my hands dirty and doing a deep-dive if it means my tech will last longer.

Here are two examples from recent weeks.

I scan all of my documents into a Mayan EDMS installation in my home network. Incoming mail, contracts, documentation, you name it. In the early days, I used an all-in-one scanner, but the feeder was less than optimal. I shopped for network scanners, but so, so pricy. So I picked up a USB scanner with a decent feeder, the Fujitsu ScanSnap S1500. I posted about this project. It scans in color, and scans both sides. It’s a relatively simple device, and until last week, I considered it super reliable. Last week I rearranged my office and moved it to a different table. Suddenly it would no longer feed. It emitted a horrific grinding sound. Research revealed that the rubber feed rollers had turned to goo, and would no longer spin. I was at a loss on how this happened, until I read the manual. You know, that pesky thing a lot of us tech folks don’t bother to read? Apparently the expectation is that we clean the rollers FREQUENTLY with F1 cleaner. I don’t know what freaking ink I’ve been scanning that eats rollers, but for reference, here is what rollers in good condition look like:

And here is what my rollers looked like after I used a razor and tweezers to peel them off of the shaft:

I have found feed roller shaft replacements on the ubiquitous web shopping site, but I have yet to find any document explaining how to disassemble and replace that particular part. It’s not obvious, anyway. The main parts Fujitsu consider consumable are the pick roller and pad assembly. I’ll dig down and figure it out at some point. For now, I just bought another scanner (super cheap) with rollers in good condition. There’s also mention somewhere of buying just the rollers, slicing one side of them and slipping them onto the existing shafts.

Considering the cost of network scanners, even buying two of them I got a deal (made them networked via Raspberry Pi, see prior post).

PART 2: THE QNAP

I have a QNAP with 8TB of disk space on it. This equates to 5.4TB of usable space in the RAID. I want to upgrade but I’m terrified of disrupting the gobs and gobs of media I’ve got stored on it. So I went shopping for a new one. They’re ridiculously expensive, some of them. But I found one “for parts or repair” for just $67. I did some research, and was convinced I could restore it to service. It’s a TS-453 Pro, which suffers from the famous J1900 LPC bug, which causes it to degrade and become unbootable. I got the sense from the listing that the “fix” (which may be temporary) was never attempted on this unit, and I was correct. The fix involves soldering a 100 ohm resistor from the clock pin to ground. The headers appeared unmolested on the board, so I grabbed a resistor from the basement and quickly soldered it into place. Sure enough, it booted right up. I slapped four 1TB hard drives from my surplus rack into it, and got the unit fully upgraded and functional. Then I ordered four 12TB drives to truly populate it. When they come, I’ll go through the recommended exercise of replacing each drive in the array one by one, and waiting for RAID to complete rebuilding itself, until all of the drives are replaced, and then expanding the storage pool and volume. I looked into a current equivalent of what I’m assembling — the Synology with 48TB of space is $1359 at B&H. I put this together for under $600. If it fails (nobody seems to know if the “fix” is permanent or just prolonging the inevitable — it’s a real scandal) I still have the other QNAP and should be able to just import the drives into that unit. Or maybe I’ll buy a brand-new 6-bay or 8-bay when that time comes. Who knows?

Starting around 1830 eastern in our Discord. Some of us will be at Shmoocon, so if you’re planning to be there as well, pop in and say hey!

So maybe you’re in Nova and have been craving one of these meetups, but maybe you’ve been busy, or maybe you’ve been battling your social anxiety, or maybe you’ve been having pressing issues at home, or maybe you’re wearing an ankle monitor.

Whoever you are, and whatever brings you to our meetup, you are invited, and welcome. We’ll be at Social House in Chantilly. Look for the group of tired, jaded tech folks, probably mostly dressed in black, and probably mostly drinking. Maybe I’ll bring some swag to reward anyone brave enough to join us.

There’s a Dairy Queen across the parking lot if that sways you. Official meetup time is 1830 eastern, but some show up earlier, and some show up later. Come out. You deserve this. We deserve each other.

We’re giving away a couple of badges and a few other soldery trinkets for new year’s.

Enter by following our YouTube. We’ll choose at random on New Year’s Eve (hell, maybe even live!) from the email connected to your youtube follower account and notify you there.

Content is becoming more frequent. PCB design classes, modular synth explorations, and who knows what we’ll get into in 2023? Follow and stay tuned!

https://www.youtube.com/@dc5402

The basement will be open to regulars & trusted friends on Monday 12/26 for our regular Monday meeting. I’ll open up the Discord channel for those who can’t make it. I’m feeling a bit like celebrating.

We’re part of the FIOS outage, apparently.

So I managed to find a satisfactory and effective resolution to my problem with serial number 0123456789ABCDEF. If you recall, the problem is that certain consumer-grade USB devices ship with non-unique serial numbers, making it difficult to use them with USB pass-through under VirtualBox.

• Serial number on multiple devices = 0123456789ABCDEF
• VirtualBox USB filtering (permanently mapping a device to a specific VM) relies a unique field. With diverse devices, this can be the vendor/product ID. With normal devices with unique serial numbers, the serial number can be used in the filters.
• With identical product, vendor and serial numbers, you’re stuck with “Port,” which at least in CentOS/Rocky’s USB subsystem, is dynamically assigned when the device is plugged in to the host. Example: Plug in a device, it shows up on Bus 001, Device 005. Unplug it, plug it back in, and it’s now on Bus 001, Device 006.
• Allegedly, udev rules can be set to assign a specific port (or a port “nickname”) using some other unique identifier, like the actual physical port the device is connected to. Still less than ideal, since that would require adherence to a strict protocol of unplugging and plugging into the same port. Anyway, I couldn’t get that to work.

So I dug around a bit, and found that the devices DO have a unique component — the device’s MAC address. It’s visible when using lsusb in verbose mode. But VirtualBox can’t see it, and it’s not an option in the VirtualBox filtering.

So what I ended up doing was abandoning VirtualBox’s USB filtering entirely.

I created a text file mapping those MAC addresses to VMs. Then I created a script which iterates through LSUSB, and checks the MAC addresses of every USB device it finds through that mapping file. When it finds one, it needs two things to make the usb attachment happen via command line. It needs either the UUID or machine name of the VM, and it needs the UUID or “address” of the USB device. My mapping file stores the MAC address of the device (discovered via lsusb) and the UUID of the VM, which is available via vboxmanage list vms.

The UUID of the USB device can be found by comparing the Bus and Port values found in lsusb to the output of vboxmanage list usbhost. With creative use of bash and awk, I was able to iterate through the devices, determine which ones are assigned to VMs, check to see if they’re currently attached, and if not, issue the vboxmanage controlvm vm-uuid usbattach usb-uuid command to attach the device to its home VM. So in case of a host reboot or someone rejiggering ports, a single command is enough to get things back where they belong.

I realize that there are probably only three people on the planet who have run into this particular problem, but I thought the solution was an interesting exercise in information-gathering on a VirtualBox host on a Rocky Linux platform, so I felt i was worth sharing.