offset \ˈȯf-ˌset\ noun

a force or influence that makes an opposing force ineffective or less effective

Home Automation

Continuing onto the previous post, after we got the apartment, I invested some time into home automation. At first I rolled out my own set of small scripts, but eventually figured out that there is a web project called Home Assistant that hosts most of my half-baked solutions with an already nice UI so I switched to that.

Sometimes it can be a pain if you're not a power user, but I don't have a problem with it. For example, there are still some things missing that can't be solved with their DSL, but one can always go down to the implementation in the configuration and change it there. It would also be great if things failed with a bang instead of letting them run anyway, but you live with it.

I bought some devices from IKEA to get the ball rolling. The whole system is called TRÅDFRI and underneath it all lies the Zigbee protocol.

I got some ambient lights, got some motion sensors to power the lights on so I don't need to look for the switch and it also helps when I go away so I don't have to worry about forgetting to turn the lights off.

I set up the smart socket switch for the dehumidifier to run when we're away so we avoid the noise. Ireland can be damp.

I also connected two public API endpoints to the system. Open data can be very useful. One API call checks the weather and the other checks the public transportation. I made it change the color of the hallway light-bulb depending on the weather forecast in the morning so I know if I should bring a waterproof jacket, and I have it recite the upcoming buses for work that are due on the nearby bus stop via speakers. Unfortunately, due to the COVID-19 circumstances, those automations are turned off because we're not leaving the apartment. Nothing is working anyway and we're doing fine at home.

I also employed a Telegram connection so it notifies me on my smartphone (Fairphone 3 now) as well when needed.

There is also a smart thermostat in place so our heating is precisely controlled depending on if we're at home, but it doesn't activate that often since the apartment is south-oriented with a glass wall in the living room so it gets enough warmth during the day.

As a bonus, I've set up my phone to use the Automate app and solve some of the most common things there. Like when I plug in my headphones, Spotify activates so I don't need to turn it on manually.

There are small things that one can do to make their lives a bit easier with automation. If it doesn't exist natively, see if you can find a way around it.

Buying an Apartment in Dublin

Like I hinted in the Trello to Coggle post, and like I mentioned in the 2019 recap, let's start at the beginning.

In January we got a notice that we needed to move out and our notice period started. Nothing out of the ordinary there. We had been in the apartment for four years so the notice period was quite long and our landlord was patient, but we didn't want to leave things to chance. We were already thinking of looking for something to buy so the notice just kicked us into a higher gear. Not to be lazy, the first week we immediately went looking and saw three places. This proved to be enough to establish the current market situation. We didn't bid on any of the places then. In total, we saw close to forty places.

We established a Trello board to keep track of the apartments we had seen and kept notes in the cards there. Eventually I visualized the data with the Coggle mindmap so Mellow was born out of necessity. It helped keep our thoughts in order.

We were looking for a small place for a couple without kids, pets or a car. Preferably close to the city center so we could walk. Preferably new or well-kept. Preferably cheap and well-insulated. Looking back, it's impossible to get such a thing in Dublin. You always have to compromise on something and the closest you can get to what you want is to build a place. However, even then, it likely won’t be in the city center.

We looked mostly at one-bedrooms. Most of the places we saw on the market in Dublin were old builds, overpriced due to their central location. The layouts were often odd, with long entry hallways and walls joining at odd angles. Almost all of them had the central fireplace that was usually decorative and took up valuable space in already small rooms. The bathrooms usually had two taps: cold and hot water separately. Dublin buildings have issues with water pressure so electric water pumps in bathrooms were not unusual. Then there were the damp related issues.

It was a far cry from what we were expecting of our own place, but that's just the state of affairs in Dublin. The messed up thing is the disproportion between the quality and price. I couldn’t help myself but compare the places we saw with the offer in Croatia, where the same amount of money would’ve given us more space and higher quality. Then again, if we had stayed, we would have had less money at our disposal and would’ve had to compromise as well. Still, some of the things we saw cannot be unseen.

There are things to check out when looking at the place and the ads won't be forthcoming. Some agencies and sellers will go so far as not to disclose significant damage in the ad photos before you are actually right there viewing it. From the floors to the ceilings, you need to check everything. Water pressure, mold damage, windows, walls, electricity, sunken floors. It would be better to take someone with you, but we didn't have anyone. A second pair of eyes works wonders, though. Sleep it over as well, and if possible go see the place more than once, which is possible when there are more public viewings organized. When you're satisfied, you need to place a bid and then the whole bidding thing starts, which is an exercise in frustration.

Bid lower than the asking price and go from there because the price can and will go up. There are other people bidding as well and if you suspect they are made up by the agency you are entitled to ask for proof. However, if the seller won’t sell at your bidding price, there’s nothing you can do about it. They will sometimes put up an ad just to test the market or lower the initial price to create hype for the place, and then just pull out if the bids don’t meet their expectations. It's a cutthroat disingenuous market where the buyer is always more exposed than the seller.

We got close to buying one duplex, got it to be sale agreed, but the vendor simply pulled out after ghosting us for weeks. Looking back, it was a good thing they did. We were exhausted from the search, this was the first place we both actually liked, and we weren’t thinking straight. Losing it saved us years of being in debt because the place was more expensive than what we planned to give. It's just that you can either pay less and buy a dump or pay way more for a decent, but not perfect, apartment. We never got an explanation either. It’s likely they advertised it to get the place appraised so they could get a loan from the bank. Who knows what. You just have to be ready for that.

Almost six months later we got lucky to find a place that looked very cozy to us. It was In the same complex as the one that we previously lost, but smaller. The complex was interesting to us since it was relatively new. Turned out the owners of the apartment were geeks as well. Could’ve just left everything in place, books, games and cats, since it would have saved us the trouble of buying and moving things.

The long wait was finally over, but landed us 5 km further than the place we were renting. That was our compromise in the end. We were still not settled in though, as we mostly had naked walls to work with.

In the beginning we had a roof over our heads and that was the most important bit. We slept on the floor while waiting for our couch to come and started buying all the things you usually take for granted when you move into a rented apartment. From the small things like the cutlery organizer for the kitchen drawer to the major necessities like the fridge. To this day, we're still buying stuff like book-ends and we still haven’t decided on the posters for the empty frames. Mostly because we're lazy or indecisive. It is shaping up, however.

Buying stuff now takes an additional toll on our finances since we have to pay off the mortgage. We have to calculate what and when to buy since our savings are limited now. We could’ve taken a bigger loan to get equipped, but that would’ve meant even more to repay. We’ll manage, but things are easier for us because we don’t have kids, a car or pets. It’s a sad fact, but those things we might take for granted are a significant financial burden.

The crux of the matter is that the system makes it difficult to buy an apartment and have kids in the first place. Instead of being able to independently afford it, we rely on extended family as a band-aid to help financially or with child rearing. The ‘it takes a village’ approach may seem romantic, but it hides a harsh truth that raising a family at a normal living standard comes prepackaged with a big mortgage and wage slavery for life, and that might not even be enough. Who knows, maybe the future has in store for us alternative marriage arrangements where people would pool resources just to bring up children. It’s OK when you give up on some things because they are just not for you or you are ambivalent about them, like we are about children. It’s sad when you have to give up on them because you don’t have the resources.

But back to our apartment. We opted for the variable mortgage rate so we could divert all the extra money into repayment. Every three months an interest gets charged to our account so we have to expect that to happen and have enough money to cover it. Hopefully we won't see another recession hit us so the plan to pay it off earlier succeeds.

We didn’t have much trouble getting the mortgage. We took it out at the bank that held our accounts already so that expedited things. The bank will check your financial history for the last six months and it has to be stable. No overdue payments, no gambling addictions, no health issues, a trend of steady savings or rent payments. They will check your transactions for that period and the credit history in the country and abroad (if you have defaulted on a debt somewhere else). That part was easy: we had already closed all our accounts in Croatia, we were both employed, we had been paying rent and saving on top of that for four years, had no kids and our major expenses were the occasional trips.

Even though the mortgage wasn't a problem, the closing date was. Our notice was nearing the end. The vendors were buying a place themselves, but it wasn’t ready so they weren’t moving out. Our landlord had new tenants waiting for us to leave. It was all a huge chain of people waiting. In the end we had to move out so we transferred all of our stuff into storage and some sensitive electronics to our friends’ place. We can't thank them enough for all the trouble with our moving out and in. They drove us back and forth as many times as needed and were really there for us.

So with our stuff in the storage, we moved to temporary student accommodation waiting for the closing date to happen, which was the date we would get our keys from the agency. We had to extend our stay there twice, and then got our keys at the last second, but at least it was summer and not cold and wet. It was OK. It took us two months after we got in to get a bed installed and another week to get the mattress, but now we are just a desk away from having it all.

I mentioned already that I started doing some work that is becoming a series of hobbies now. Home automation, balcony gardening. I'll write more about that in the following posts. We're both keeping busy now. Figuring out how to approach the life-work balance. The commute, the lunch breaks... It's forming up. We'll get there.

2019 Recap

Our best laid plans didn't come to fruition last year, mostly because a conundrum about future accommodation, which I'll write about later. Whatever we had in store for longer travels was postponed so in the end it was more of Europe this time:

  • Croatia, again to see family and friends
  • Sweden, to rendezvous with friends and see Stockholm, that was encased in ice at the time
  • Spain, but really Tenerife, a small archipelago off of Africa where the highlights were the huge volcano caldera of Mount Teide and an isolated mountain village of Masca
  • Czechia, Prague and Říp mountain, to meet up with my parents who haven't had much chance to go anywhere during their lifetime and my dad fares from that country
  • Cyprus, Paphos and its surrounding areas
  • Northern Ireland, Belfast, just to see some of the city properly
  • Bosnia and Herzegovina, to take my mother to a village nearby Bosanski Brod where she's from and where she hasn't been for years
  • Netherlands, Amsterdam, Haarlem, Zaanse Schans, Zandvoort, to meet up with friends and go museum hopping
  • Germany, Munich and Frankfurt, used the layovers to see the cities a bit again
  • Slovenia, Ljubljana, to see the capital

Across Ireland itself we planned a number of sites to see, but it ended up being:

  • Dalkey, for a day getaway
  • Kerry, Killarney, likewise, to see some of the national park itself, Adare village, Ross castle, Muckross house, Dingle Bay and Inch Beach
  • Tayto park again

Concerts:

  • Radical Face, which was cozy and awesome and (a good kind of) sad
  • Dido, which was a standard pop concert you'd expect it to be

So, yeah, the accommodation conundrum I mentioned - we bought an apartment. It tossed a wrench in our well-oiled machinery running the plans, but it was a huge achievement in itself. I immediately started some automation for it tying everything together with Home Assistant.

  1. Lighting (LED lights) and some motion sensors to trigger the lights
  2. Heating
  3. Weather notifications in the form of a Telegram text and changing the bulb light color at the exit before leaving for work
  4. Speech notifications for the public transportation
  5. Other textual notifications

I am working on getting the presence detection tied into the whole story so it's not relying on timestamps.

After finding the apartment, we had to get the papers sorted and buy most of the stuff like furniture and electronics, so that significantly affected our free time and routine. I did start some projects: for a hydroponics greenhouse balcony garden, and near the end of the year I also started work on a large scale geographic analysis, related to food security. I'll see how that goes. There are some personal projects in the pipeline but we'll get them off the ground eventually.

Regarding the projects that were actually finished, they were:

Vesna finished a scrum master course and got more comfortable with the JavaScript programming up to the point where I can say she's a proper developer.

We attended the Worldcon in Dublin. An SF convention.

We also started to play pen and paper physically in Dublin. A nice change of pace from the Skype sessions we have occasionally.

I didn't draw much in 2019, which was a bit disheartening. I never found the proper conditions, but, unfortunately for me, found all the excuses.

I managed to fulfill the conditions for the Irish citizenship, but I'll need to get the paperwork sorted this year for that.

Lastly, but far from the least, I was working on the TZM poll that was featured on the main site over there, so if you're part of that demographic, do answer.

Hopefully, this year will see us recover our pace a bit, but I am not setting ourselves for a failure by putting a big goal in front of us. It will most likely be wrapping up whatever we have left and doing small things.

Shamrock - A Trefle API Library for Plant Data

Shamrock is a Trefle API library written in Python. Trefle is a botany API with a number of plants in their database. It can be used for research, gardening software, automation, etc.


/media/images/shamrock.jpeg

I made the Shamrock library using the existing requests library to interact with the Trefle API because I needed some structure in my upcoming project. I wanted to do things properly; testing, documentation, CI/CD pipeline, ensuring that the repository conforms to certain standards and publishing everything mentioned. It took me around a month and a half of casual work in free time to complete the first release. I also learned some new things along the way.

Without getting further into how the library is used, you can check it at these three links:

Now onto the technical side of things. The library acts as a shallow API layer between Python and the Trefle service. You need to get the token from them and use it when instantiating the Shamrock class.

The library relies on lists for navigation and endpoints. When you call a method on the Shamrock instance, it checks the __getattr__ magic method and filters according to the name called. That way I avoid code duplication and the code is routed to either the navigation or an API endpoint.

The navigation is largely solved by the dependent requests library.

While making requests, the library will raise any errors if the service is down in any way.

When a request is made, the response is stored in a variable on the instance itself so the subsequent request to the same endpoint is checked, and if it's the same, it's not repeated. Instead, the stored response is returned. I don't want the users to make an unnecessary hit on the API and to burn their traffic and need to wait on the response itself.

What I learned is using the PyPI publishing with Twine, using Sphinx to generate the documentation from docstrings and to use type hinting in Python 3. I also learned a bit about CircleCI because so far I was using other pipelines to run things. I also learned a bit about VCR.py library that I used with tests. The coverage is 100% because the surface of the library is not that big.

Now what can you use the library with? Well any kind of project that involves Python and botany. For instance, if you write a web application in Python and use the library on the back-end, you can make a comprehensive gardening software to keep track of your garden. Off the top of my head, you can have the number of plants planted on some surface, expected yield and such, and with the Shamrock have a rich display of what kind of plants you planted. I understand that their database is not complete, but they are working on it. The full example they have is the plant Allium acuminatum (tapertip onion) so you can query the plant with the ID 103505 and poke around with the data in the response. For any issues with their service, you can refer to the Trefle issue tracker on GitHub.

I hope you find the Shamrock library useful and that you might use it in one of your projects or even find some time to contribute to the library itself.

Mellow - Trello to Coggle Converter

I wanted to figure out how to set up serverless things so I opted for a simple application, although it turned out the architecture of it all had a lot of moving parts. The application in question is taking the Trello board JSON export and converting it to an MM file for Coggle.

Let's say you are primarily using Trello to organize your to-do list. It's a great tool with a lot of features, why not use it. But you are more of a visual type, and seeing it in a color-coded diagram could help you prioritize and group up the tasks more efficiently. If that's the case, this small app would competently meet that very specific need.

It's as simple as going to a Trello board settings and exporting it as JSON.


/media/images/export.png

You can either save a file to disk from your browser or copy the JSON content to a clipboard. With it, you can go to the service I made and put either of those there and press the "Convert" button.


/media/images/convert.png

It will give you an MM file for download that you can save and when you later open up Coggle, you can create a new diagram and drag and drop the file there.


/media/images/diagram.png

The result is your Trello board in a new, mind map, view.


/media/images/diagram2.png

Simple as that, without any hassle of giving permissions and any info to the app. I called the converter from Trello board to MM for Coggle, Mellow.

Why I actually went for it is that we've suddenly found ourselves in the process of looking to buy an apartment, and moving things on the screen and seeing them presented in different ways actually helps a lot with establishing previously unseen patterns and connections. Both Trello and Coggle came in handy, and as I am a tinkering IT guy, Mellow came to be as well.

That was really the main reason. The other reason is taking my mind off from the fruitless pursuit of finding a space we would actually pay to own, rather than tolerate to rent.

But back to Mellow. I made a primitive SDK and CLI for converting the Trello export to Coggle so it was just a matter of wrapping it in a web service.

Now on to the boring part about the architecture. The front-end part is a simple static app that just features plain old HTML and some new JavaScript for modern browsers. It is jQuery, though, since jQuery comes as a requirement for the Bootstrap components that the front-end is made with. It made sense to keep it minimal, but I had to include a bigger version of jQuery to compensate for the CORS calls to a different service. I put the Trello to Coggle front-end repository on GitHub in the end since it allows for the deployment previews in Netlify as opposed to Bitbucket that I'm usually working with. Vesna helped around a lot with the parts I didn't know how to make, but had a rough idea. Netlify is an option that came recommended from a friend since you can specify the security HTTP headers properly with it.

Having the front-end set up, I had to have something that it could communicate with. I made a small Flask web application that takes a text string as an argument via form submit and gives back the JSON in response. If successful, it will have the base64 string that is used to construct an MM file in the browser. I didn't want to deal with file management in the transfer. The application imports the SDK and CLI script as a dependency and it worked properly with the front-end communicating the data it needed. A small set of tests were added to it to make sure it responds correctly.

Since it's Python all the way, I opted for Zappa to create a serverless AWS Lambda function out of the Flask application. The resulting URL of the function is added as the form action attribute in the front-end of the service. Of course, I've set up the custom subdomain for it on the existing one I already own. The FaaS repository for Trello to Coggle converter is hosted on the Bitbucket and I'm deploying it with their Pipelines making sure that the tests it has pass before deployment.

Deploying to Netlify is just a push away with the settings in the netlify.toml file. It doesn't have a specific build since I didn't want to overengineer it. Deploying to AWS is described in the bitbucket-pipelines.yml file and is pretty straightforward.

To recap what I had problems with, it was definitely debugging the security headers. I thought that I've cleared everything up so it's not inline, but apparently extensions can throw you off. You'd need to run your browser in the safe mode to see the proper output. The other thing is pointing the custom domains from your DNS (Digital Ocean in my case) to Netlify and AWS. While that didn't present a problem because you just follow the instructions for adding a custom domain and issue a CNAME redirect on your DNS, getting certification, so you can use HTTPS, in AWS only works in the us-east-1 region, the Virginia center. I couldn't register a certificate in my preferred region. I had to resort to that other one for the moment.

Regarding costs, the service falls in the free tier in every part since it's not like it's going to be overly used. I also disabled keep_warm in Zappa because it's not like the service needs to be up all the time. I didn't want to add the auth mechanism because that would require me to dabble with the user info and the GDPR compliance. It could be useful to connect the apps and just negotiate the content, but it's not necessary for now. I just wanted to try out the FaaS part. The domain I already own so it was just a question of adding the subdomain to the service. The service could benefit from additional tests and analytics as well as monitoring, however, it would be an overkill at this stage.

I hope some of you will find Mellow, the Trello to Coggle converter useful. If you do, but find it can be improved, speak up.