Trying out a new form of organizer…  doing it a bit more simplistically.   Instead of hauling around a DayTimer where each day’s page needed to be the same size (and for which I ended up lugging around a bunch of sheets of mostly empty paper), I’m trying out a technique called bullet journaling.  Each day, I keep notes of appointments to keep, things to do, and any notes worth mentioning.  I keep a monthly calendar page, sort of like a calendar on the wall in my kitchen.  Just holds enough to remind me of an event.  I keep a future plans page – listing any events that don’t fit on the current month’s calendar.  And there’s an index page at the front, where I log any significant entries I think I’ll want to get back to.

For some folks, their bullet journals are as much works of creative expression as organization.  That’s not me.  My writing is still scribbles, I don’t doodle or calligraphy in it, and if I’ve changed ink colors, it’s because that was the pen at hand.  Today I did upgrade a bit, and started pasting in a calendar printable, rather than writing the days out on the page.  I’m also trying out a monthly habit tracker printable to see if it helps keep me accountable for reading my Bible and exercising.  Housecleaning might make the list, too, I guess…

I’ve been testing this out since November and am very happy with how it’s worked out for me.  I’ve generally been the type to lug around notebooks anyway to jot things down in – this just extends that with a bit of structure.

 

Some stuff for work is running on a Kubernetes 1.3 cluster.  It nicely restarted when something went wrong in its environment.  That’s good.  Our pods were logging – that’s good.  But when the pod restarted, its logs were lost: that’s bad.  No idea why the pod restarted to help determine if it’s something we should either prevent or add a health check for.  So, now I’m evaluating logging solutions appropriate for our Kubernetes environment.  And because I’m a geek, I’m blogging about it here in my pjs on my day off: you know you’re in the right field when you’re enjoying researching something for it on your day off.

Scope of problem: not trying to log all of the kubernetes events themselves: that’s the job of the infrastructure.  Which also means then I’m not worried about filtering out messages I shouldn’t see from pods that aren’t my own.  I want to collect log messages in my namespace, have them tagged (at least) by which application source they came from, and I want to have them be archived _off_ of the cluster.  The guys running the infrastructure are great, but if they reconfigure the cluster (as happens occasionally) or have a hiccup with their persistent volumes, still want my logs.

A bit of survey suggests there are two primary solutions out there: Fluentd or Logstash, one of the components of the ELK stack.  Both are open-source.  I’ve long heard of ELK, though not yet myself futzed with it in a system.  I’ve seen many stickers for Fluentd in open-source conferences…  :-)  I’ve also noticed Fluentd as part of the Cloud Native Computing Foundation‘s list of supported projects.  Hmm..  that last bit gives me a bit of bias towards Fluentd, sight unseen.

So now to look for other folks comparisons of the two to pick the first one to try out..  My strategy is to start with one and see if it’s ‘good enough’ to meet our needs.  But I’ll often rely on what the ecosystem has already discovered to make my choice between.

Fluentd vs. Logstash: A Comparison of Log Collectors: logz.io uses both.  My takeaway – leans towards Fluentd in its recommendation, even though it’s built on ELK

Fluentd vs. Logstash: Panda Strike’s tried both..  They’ve ended up with EFK (ElasticSearch, Fluentd, Kibana) as their platform for reasons of stability and performance.  Side note: describes logs forwarders vs. log shippers.  Found another post on their site that describes EFK in action for JMeter load testing result visualization.    Another side note: found a good bit of useful content in their blog overall..  adding ’em to the rotation of stuff to keep an eye on…

Fluentd vs Logstash: Jason Wilder walks through both.  Older post (Nov 2013) but assuming all is still relevant, good technical info.

Outcome: looks like I’ll be starting with Fluentd.

 

 

 

 

I got a cold-call from a recruiter today, seeing if I’d be interested in a Technical Manager job.  That’s right in my bailiwick: I tend to be more technical than most technical managers, but she had me pegged in the right general zone of job, anyway.  What surprised me was her saying she had a resume of mine from 2006.  Wow.

Luckily for me, 2006 rang a bell.  That’s the year my son was born, so that’s the year I traded a commute around the Beltway to a job I loved for a new series of adventures with a company closer to home.

In 2006..  I was building an e-commerce system using Struts.  I _think_ we were debating a rewrite in Struts 2.  Somewhere in that time frame, I became a certified Scrum practitioner.  (I’ve since let the certified part of that lapse, though I’d still consider myself a Scrum practitioner.  Just one who doesn’t pay certification fees to prove her worth.)  We’d do production deployments once a week, beginning at 11pm and working through the night until we succeeded or rolled back.

Since then, I’ve done all sorts of things…  OSGi in its relatively early days.  Grails web applications that I helped take GOSS (government open-source) and then fully open-sourced.  D3 visualizations for SBIRs.  Actually, D3 visualizations for two different SBIRs.  Many hours realizing I needed to better grok D3.  Lots of business development.  Winning multiple proposals and then staffing teams to do the technical delivery.  Where those teams typically included me, hands-on coding.  Ramping up on AngularJS and Node to lead a “thin-client” team.  That’s as opposed to the program’s thick-client team building on top of Java Swing – thankfully, although I managed the overall UI team, didn’t have to wade too far into the Swing code.  Docker containers containing stub implementations of enterprise system interfaces we wanted vendors to be able to execute against.  Now Kubernetes: figuring out how to deliver and sustain systems in a cloud.  I’ve delivered two OSCON briefings, one using an Arduino and C and one using Go and a Raspberry Pi.

It’s been a fun 10 years.  If she calls again, though, she’d better go find my updated resume.  For the record, LinkedIn’s got a pretty good idea of what I’ve been up to…

Needed a pick-me-up, so went to the tea shop. Ooh, bubble tea. I’ve heard good things, let me give it a try…

Reaction: bleah. Tea should not have floaters. And then I looked up calories and carbs. Double-bleah. Straight tea and/or espresso for me next time.

A few months ago, I joined a software team that’s making use of Kubernetes as a platform to provide elastically scalable software services.  It’s Docker + more magic.

One of the tenets espoused by some folks building container deployments is that a container should only have the software on it required to run a single process / microservice.  The particular software system I’m deploying is a private Docker registry, but to deliver the service, there are currently 4 Docker containers in play.  (I actually think that to solve a problem I’m having, I’ll need to add two more.)

One of the containers wasn’t behaving nicely for me today.  The container provides a web user interface and needs to communicate with a database provided by another container to store and retrieve its data.  But the logs were saying that the user interface container couldn’t log into the database.

The first two options that came to mind:

– the hostname wasn’t accessible to the ui container

– the port wasn’t open on the database container

Turns out, it was neither of those: it was a badly shared database password.  But in trying to prove or disprove those options, I ran into a challenging problem.  See, the UI container _only_ had the binary for the UI process on it.  Which meant: it didn’t have ssh, telnet, curl, wget, netstat, or any of the various other options I came up with to verify network connectivity between the containers.  How to get my answer?  Google + Stack Exchange to the rescue!

In recent versions of bash (which thankfully the golang image did provide), bash will let you access TCP and UDP ports directly, ala

cat < /dev/tcp/databasehost/3306

Running the above from the ui container got me a garbled prompt received from the database container, which proved that the hostname resolved AND the port was open, all in one command.

Kudos to Peter Mortenson and his answer on superuser.com  for helping me crack the bug.  Tuesday’s bug (courtesy of the Monday holiday) will be to see if Beego.Run() doesn’t crash if I manage to map in the app.conf file not otherwise being provided through my Kubernetes replication controller descriptor.  Not good to have applications just stop without an error message!!

 

 

We’re hiring at my company, Hayden Software, and I’ve been asked to help interview for my contract, so I’ve had a few different opportunities to ask candidates how they track down programming problem solutions.  The usual answer?  Either Google or Stack Overflow.  Both are reasonable answers, though are a bit broad brush in terms of their coverage.  If I find something mentioned in documentation, what I’d _really_ like is to see a working example of it.  Particularly since most programmers seem to write awful documentation.

I’ve taken, of late, to trying to use GitHub as my compendium of source code examples.  I’ll assume that the folks who cared about this post already know what GitHub is…  if you’re hear to read a post about clowning, you’ll have to wait for the next one…  GitHub announced in 2013 that it had reached over 10 million code repositories.  It’s not quite yet the infinite monkey theorem, but there’s definitely enough source code there to make me think that any library or framework feature that I’m interested in using already has a bit of sample code for it in someone else’s repository.

But how to find the examples I’m seeking?

Recently I was seeking examples of fabric8’s docker-maven-plugin in use.  It’s one of a couple of plugins that are supposed to help with managing Docker builds and Docker containers.  I don’t personally find Docker build files particularly challenging, but perhaps the managing the containers part would somehow be made easier with the plugin.  (TL;DR: finally didn’t find it useful, at least for our uses).  But in my hunt, I wanted to find examples of folks using this thing…

That plugin would be used in a pom file.  So, I want to find pom files that make use of the plugin.  In the case I originally went looking for, I wanted to use something called inline plugin configuration.  Basically, if I was going to use a Maven file to do the docker build, I wanted to keep all of the build information in the single pom file.  The docs described it as possible, but it just wasn’t coming together for me.  So, the query against Github:

(inline assembly docker-maven-plugin fabric8 dockerfile)  filename:pom.xml fabric8

Look across all repositories for a file named pom.xml which had certain keywords in it, and which were part of a repository or owned by a user which included the word fabric8. Found a few examples (and, and), but nothing which blew me away on the usefulness of that plugin and approach.

What about when I wanted to see examples of a particular routing package used in Go code?

language:go beego router

Wow, that gave me back 7K+ results.  Maybe narrow that down a bit?  I’d like to narrow that down by cross-checking the stars on the owning repository: theoretically, something with more stars would have more useful examples.  Interestingly, I cannot seem to get that to work as I’d like…  Although I can hunt for repositories with the phrase beego in their name or referenced in their description, I cannot get it to further check files.  The query below ignores the ‘stars’ descriptor and gives the same 7K+ results.  (I say ignores because I pulled up one of the repositories and it showed zero stars.)

language:go stars:>10 beego router

So, GitHub filtering has its challenges and quirks.  That said, I can also filter on looking for code within a particular repository or by  particular user, both of which help in filtering out the wheat from the chaff.

I can’t learn everything in software on my own, but if I can learn from someone else’s example (good or bad, assuming I recognize the difference), I can get the thing done and build upon my knowledge very efficiently.  Would love to hear if other folks have code search tricks and techniques, on GitHub or otherwise.

 

In last month’s Nerd Nightmare, I described the interruption of flow of a broken arm when one works at a keyboard.  So, here’s the update since then…  I did get a plate and some eight screws to hold everything to the bone.  I spent two weeks in a splint and a sling, and then got my stitches out and was put in a exoskeleton sort of removable hard splint.  Turns out, I get light-headed when I see fresh scars on me: nearly hit the floor the day the doc took the stitches out.  Since that appointment, I’ve been doing a twice a week physical therapy visit to work to get the mobility back in my wrist and arm.  Two days ago, the doc even told me to take off the exoskeleton..  Scary times.

The bones are apparently healing nicely, based on what the doc said this week after seeing new x-rays.  The marathon part refers to what it takes to get mobility and strength back.  Right now, my right (non-broken) wrist can pivot 90 degrees from thumb up…  I can go palm down or palm up, no issues.  My left wrist can go mostly flat if I go clock-wise.  And I maybe get 10 degrees counter-clockwise.  My right wrist can get close to 90 if I bend it up, and something much less than that bending it down.  My left wrist gets about half down what the right does, and barely makes it out of flat when trying to pull up.  Needless to say, push-ups are out of the question the right now.

My physical therapist tells me not to worry, that it’s a marathon.  I told her today, I’ve run marathons.  There are those who run sub 5 minute miles and finish in 2 1/2 hours (not me) and there are those who run 5 1/2 hours or more (me).  She laughed and didn’t comment further…

I got a turtle tattoo on my leg after my first marathon, as a marker of a dream accomplished and work put in.  What’s my wrist marathon tattoo, I wonder?  Got a handy 6 inch scar I guess I can work in…

I’m typing this out one-handed, as my left arm is in a splint with a broken arm and wrist.  Nerd nightmare.  Complete interruption of “flow”.  No longer do my thoughts just appear in code..  everything’s slowed down enough to make even the ideas disjoint, as well as my arm.  Pain coupled with frustration makes every day a slog…  and the guy who wanted to hold a design discussion in a chat room?  He held a special label in my heart that day.

Now, a few advantages just to help focus my mind in a better direction…

– I’m getting better at taking advantage of bash shortcuts, aliases, pulls from command history, etc

– More in-person chats to get things done mean a chance to get to know my co-workers better

– The more junior members of my team are getting more focused time from me.  Their hands on a keyboard solving a problem are much more efficient than mine.  My personal productivity is down, but our team’s going to look great at next week’s sprint review.

Tuesday I’ll get to become a bit bionic with a plate and screws, and some two weeks after that I might get the ok to do more than just open and close my hand.  So, there’s hope in sight. But only a week into this nightmare, I’m really not looking forward to several more…