#10yearchallenge – Twitter edition

I am seeing a lot of people doing the”10 year challenge”: posting two pictures side by side – one from today and the other from ten years ago.

Some say it might be a planted meme to train AIs the effect of human aging… What an interesting hypothesis!

Funny enough I did not take part. And sure enough I am getting all these nudges by services like Twitter… So. Ten years ago this blog existed and I started using Twitter. Apparently I will still use this website but changes a bit my general approach to services like Twitter.

And the next time I’ll explain why I am always awake quite early 🙂

small and cheap multi-sensor nodes for home automation

I had reported on my efforts to develop an indoor location tracking system previously. Back in 2017 when I started to work on this I only planned to utilize inexpensive EspressIf ESP32 SoCs to look for bluetooth beacons.

In the time between I figured that I could, and should, also utilize the multiple digital and analog input/output pins this specific SoC offers. And what better to utilize it with then a range of sensors that also now could feed their measurements into an MQTT feed along with the bluetooth details.

And there is a whole lot of sensors that I’ve added. On a breadboard it looks like this:

So what do we have here:

  • Motion sensor
  • Temperature sensor
  • Humidity sensor
  • Light sensor
  • Barometric pressure sensor
  • and of course an RGB LED to show a status

The software I’ve done already and after 3 weeks of extensive testing it seems that it’s stable. I will release this eventually later in the process.

I’ve also found plastic cases that fill fit this amount of sensory over the sensor cases I had already bought for the ESP32 alone. For now I’ll close this article with some pictures.

The MQTT feed one of these nodes produces…

…and the Grafana dashboard I am using for this specific prototype device.

progressive web applications

These days even heise online is writing up about the wonders of PWA (progressive web applications).

PWA simply put is a standardized way to add some context to websites and package them up so they behave as much like a native mobile application. A mobile application that you are used to install onto your phone or tablet most likely using an app store of some sort.

The aim of PWA is to provide a framework and tooling so that the website is able to provide features like push notifications, background updates, offline modes and so on.

Very neat. I’ve just today have enabled the PWA mode of this website, so you’re now free to add it to your home screen. But fear not: You won’t be pestered with push notifications or any background stuff taking place. It’s merely a more convenient optional shortcut.

Instagram – until now

I’ve had already added a couple of pictures to my instagram account – mainly while abroad. Pictures that I consider nice enough to be shared.

Of course my latest switch away from those public silos will include having those pictures posted mainly on this website and maybe as a side-note on those services as well.

To begin with I will have a separate page created that will host those pictures I consider nice enough to be shared.

blog maintenance – status

A bit of feedback is in on the plan to revitalize this blog. Thanks for that!

I have spent some more time this weekend on getting everything a bit tidied up.

There is the archive of >3.000 posts that I plan to review and re-categorize.

There is the big number of comments that had been made in the past and that I need to come up with a plan on how to allow/disallow/deal with comments and discussions in general on this website.

There is also the design and template aspects of this website. I switched to a different template and started to adjust it so that it shall make access to the stream of posts as easy as possible. Until then you need to wait or contact me through other means. But contacting is another post for another time.

resilvering …

The last Ubuntu kernel update seemingly kicked two hard disks out of a ZFS raidz – sigh. With ZFS on Linux this poses an issue:

Two hard drives that previously where in this ZFS pool named “storagepool” where reassigned a completely different device-id by Linux. So /dev/sdd became /dev/sdf and so on.

ZFS uses a specific metadata structure to encode information about that hard drive and it’s relationship to storage pools. When Linux reassigned a name to the hard drive apparently some things got shaken up in ZFS’ internal structures and mappings.

The solution was these steps

  • export the ZFS storage pool (=taking it offline for access/turning it off)
  • use the zpool functionality “labelclear” to clear off the data partition table of the hard drives that got “unavailable” to the storage pool
  • import the ZFS storage pool back in (=taking it online for access)
  • using the replace functionality of zpool to replace the old drive name with the new drive name.

After poking around for about 2 hours the above strategy made the storage pool to start rebuilding (resilvering in ZFS speak). Well – learning something every day.

4+ hours to go.

Bonus: I was not immediately informed of the DEGRADED state of the storage pool. That needs to change. A simple command now is run by cron-tab every hour.

zpool status -x | grep state: | tr –delete state: |mosquitto_pub -t house/stappenbach/server/poppyseeds/zpool -l

This pushes the ZFS storage pool state to MQTT and get’s worked on by a small NodeRed flow.

taking the social stream back to this blog

I am currently in the process of reducing my presence on the usual social networks. Here is my reasoning and how I will do it.

Facebook, Twitter, Instagram and alike are seemingly at the peak of their popularity and more and more users get more and more concerned about how their data and privacy is handled by those social networks. So am I.

Now my main concern is not so much on the privacy side. I never published anything on a social network – private or public – that I would not be published or freely distributed/leak. But:

I have published content with the intention that it would be accessible to everyone now and in the future. The increasing risk is that those publishing platforms are going to fade away and thus will render the content I had published there inaccessible.

My preferred way of publishing content and making sure that it stays accessible is this website – my personal blog.

I am doing this since 2004. The exact year that Facebook was founded. And apparently this website and it’s content has a good chance of being available longer than the biggest social network at the present time.

So what does this mean? 3 basic implications:

  1. I will become a “lurker” on the social networks. Now and in the future.
  2. All comments and reactions I will make will be either directly in private or through my personal website publicly available and linkable.
  3. I will minimize my footprint on the social networks as much as possible. This for example means: If I use Twitter, all tweets will live 7 days and automatically be removed after this time. Deleting your tweets automatically is something others do as well.

As you can see: This is not about a cut or abstinence. I get information out of social networks, tweet message flows. But I do not put any trust in the longevity of both the platforms and the content published there.

The next steps for me will be a complete overaul of this website. Get everything up to current standards to streamline my publishing process.

Expect a lot of content and change – and: welcome to my blog!

Converting ひらがな to “hiragana” and カタカナ to “katakana” – Romaji command line tool

I had this strange problem that my car was not able to display japanese characters when confronted with them. Oh the marvels of inserting a USB stick into a car from 2009.

stupid BMW media player without proper font

Now there’s no real option I know of without risking to brick the car / entertainment system of the car to get it to display the characters right.

Needless so say that my wifes car does the trick easily – of course it’s an asian car!

Anyway. I wrote a command line tool using some awesome pre-made libraries to convert Hiragana-Katakana characters to their romaji counterpart. 

You can find it on github: https://github.com/bietiekay/romaji 

Join me implementing a neural network to improve accuracy of an OpenSource indoor location tracking system

To all techies reading this:

GIST: I am looking for interested hackers who want to help me implement a neural network that improves the accuracy of bluetooth low energy based indoor location tracking.

Longer version:

I am currently applying the last finishing touched to a house wide bluetooth low energy based location tracking system. (All of which will be opensourced)

The system consists of 10+ ESP-32 Arduino compatible WiFi/Bluetooth system-on-a-chip. At least one per room of a house.

These modules are very low powered and have one task: They scan for BLE advertisements and send the mac and manufacturer data + the RSSI (signal strength) over WiFi into specific MQTT topics.

There is currently a server component that takes this data and calculates a probable location of a seen bluetooth low energy device (like the apple watch I am wearing…). It currently is using a calibration phase to level in on a minimum accuracy. And then simple calculation matrices to identify the most probable location.

This all is very nice but since I got interested in neural networks and KI development – and I think many others might as well – I am asking here for also interested parties to join the effort.

I do have an existing set-up as well as gigabytes of log data.

I know about previous works like „Indoor location tracking system using neural network based on bluetooth

Now I am totally new to the overal concepts and tooling and I start playing with TensorFlow right now.

If you want to join, let me know by commenting!

Source: http://ieeexplore.ieee.org/document/7754772/

making your home smarter use case #14 – prevent fires while charging LiPo batteries

Did you know how dangerous Lithium-Polymer batteries can be? Well, if not treated well they literally burst in flames spontaneously.

So it’s quite important to follow a couple of guidelines to not burn down the house.

Since I am just about to start getting into the hobby of FPV quadcopter racing I’ve tried follow those guidelines and found that the smart house can help me tracking things.

Unfortunately there are not a lot of LiPo chargers available at reasonable price with computer interfaces to be monitored while charging/discharging the batteries. But there are a couple of workarounds I’ve found useful.

  1. a proper case. I’ve got myself one of those “Bat-Safe” boxes that fit a couple of battery packs and help me store them safely. Even if one or many burst into flames the case is going to contain any heat and fire as good as possible and with the air / pressure filter it’ll hopefully get rid of most of the very nasty smoke (I hear). Cables go into it, so the actual charging process takes place with everything closed and latched.
  2. the obvious smoke detector which is on it’s own connected to the overall fire alarm is mounted on top, like literally on top. It’ll send out the alert to all other smoke alarms in the house making them go beep as well as sending out high priority push notifications to everyone.
  3. an actual camera is monitoring the box all the time calling on alerts if something is fishy (like making sound, smoke, movement of any sort). When charging is done the charger will beep – this is being caught by the cameras microphones and alerts are sent out.
  4. the temperature inside the case is monitored all the time. The surrounding temperatures are usually pretty stable as this case is stored in my basement and as the charging goes on the temperatures inside the case will climb up and eventually level out and fall when charging / discharging is done. Now the system basically will look at the temperatures, decide wether it’s rising of falling and alerting appropriately.

There’s a couple more things to it, like keeping track of charging processes in a calendar as you can see in the flowchart behind all the above.

making your home smarter use case #13 – correlations happen

There are a lot of things that happen in the smart house that are connected somehow.

And the smart house knows about those events happening and might suggest, or even act upon the knowledge of them.

A simple example:

In our living room we’ve got a nice big aquarium which, depending on the time of the day and season, it is simulating it’s very own little dusk-till-dawn lightshow for the pleasure of the inhabitants.
Additionally the waterquality is improved by an air-pump generating nice bubbles and enriching the water with oxygen. But that comes a cost: When you are in the room those bubbles and the hissing sound of the inverter for the “sun” produces sounds that are distracting and disturbing to the otherwise quiet room.

Now the smart home comes to the rescue:

It detects that whenever someone is entering the room and staying for longer, or powering up the TV or listening to music. Also it will log that regularly when these things happen also the aquarium air and maybe lights are turned off. Moreso they are turned back on again when the person leaves.

These correlations are what the smart house is using to identify groups of switches, events and actions that are somehow tied together. It’ll prepare a report and will recommend actions which at the push of a button can become a routine task always being executed when certain characteristics are lining up.

And since the smart house is a machine, it can look for correlations in a lot more dimensions a human could: Date, Time, Location, Duration, Sensor and Actor values (power up TV, Temperature in room < 22, Calendar = November, Windows closed => turn on the heating).