I’m talking with a group of young ladies this week about software development. They’re part of a HowGirlsCode group, which seeks to “provide[..] computer science and engineering education designed to inspire young girls in computer and engineering sciences”.   Women apparently only earn about 20% of the computer science undergraduate degrees and then often leave the field, so that only some 13% of folks in the field are women.[1]  I wish that didn’t ring true but it does.  It’s rare and exciting when there’s another woman on my team.  I make a point of trying to recruit women in particular, just so we can amass a core group of gals to show the world how it’s done.

The talk this week focuses on the fun in software development.  I love it – tried to leave the field in my mid twenties and finally realized this is where I was meant to be.  It’s provided well for me and my family and given me lots of opportunities.  In what other field could I build out conference talks about Furbies (twice?!)  I’ve gotten to travel, both in the US and around the world.  I’ve _never_ been without an opportunity to learn something new.

When I first dreamed of doing software development, I thought I’d go into artificial intelligence.  When I graduated college in the mid 90’s, AI seemed far away, something that only PhDs were thinking about.  While we’re still not where I thought we’d be when I was in high school (and earlier) contemplating a career, researchers in the UK recently announced the world’s largest computer simulation of the brain.  There’s software now in my phone.  There’s software in my car.  I bought Christmas presents this evening through a web browser hooked up through the Internet to an e-commerce infrastructure, undoubtedly hosted in a cloud infrastructure somewhere.  Heck, if I cared to, there could be software in my refrigerator!

I’m really looking forward to talking with the HowGirlsCode young ladies on Thursday.  Computers and software are ubiquitous as far as they’re concerned: they may not even realize how software infuses almost everything they touch.  But I’m looking forward to showing them the opportunities that opens up!

 

 

Question from one of my teammates on my new work team:

“Did you just learn techA & techB in the 3 days since you’ve joined the project?”

Well, enough to make things work for this portion of the project, anyway.  And thank you for noticing!

My teammate and I have been “pair-programming” remotely, by which I mean: we talk over the phone about the approach and occasionally screen-share / present to show what we mean.  Oh, and of course, commit at regular intervals into a shared git branch.  When I joined the team, the story (uh, work unit, I guess, for those of you not well-versed in software) was written such that the work would have 4 subtasks.  I proposed doing it differently, based on some prior experience I had with techC, which is the end-result of our efforts with techA and techB.  The team bought in, and off we went!

Challenges:

  • neither my teammate nor I had much experience with techA or techB
  • my teammate doesn’t have much experience with techC
  • my teammate and the rest of the team are in Minnesota, which means: no whiteboard drawings, an offset of an hour in schedule, we haven’t met each other in person, …
  • I’m brand-spanking new to the team, so am still navigating getting all of my accounts, figuring out how not to break other folks’ work, figuring out how to prove things _do_ work, …

It looks like by not too much longer today, I’ll be putting in my first merge request for a significant feature for the new project.  Woot!  Good first (real) week.

For those techies interested in the secret decoder ring for the technologies:

  • techA = Ansible
  • techB = Salt
  • techC = Kubernetes

 

 

“NOTE”:ed in documentation I was looking at today…


NOTE: “Default” is not the default DNS policy. If dnsPolicy is not explicitly specified, then “ClusterFirst” is used.

This is the sort of thing that should _not_ pass muster for code-review. Kudos to whoever recognized the issue and at least put it in documentation. But there oughta be a kubernetes GitHub issue out there somewhere to fix the above. And no, telling me that it’s been released this way and thus must be maintained is not an acceptable argument. Deprecate the word ‘Default’, if you must. If the default is not actually that, then the impact is likely small. ‘Default’ could become ‘InheritFromNode’ or ‘Inherited’ or …

A little bit of Google digging found a related, but not quite what I mean GitHub issue. Grumble, grumble, growl….

A friend / previous co-worker of mine sent out an intriguing tweet:

George is an agile coach, among many other things.  (I believe him also to be a sailor and a grandfather, and someone whose tweets I enjoy.)  I assume his book proposal has something to with agile development or coaching of agile teams.  I’ll be looking to see when he announces that someone’s accepted it!  In the meantime, I’ve offered my services for review. 

George’s book would make #3 of book reviews for me.  He’d be in the esteemed company of Steve McConnell (‘Software Estimation’) and Karl Fogel (‘Producing Open Source Software’, 2nd edition).  I’d forgotten the McConnell book until George mentioned it recently, and Karl’s finished up his second edition fairly recently (November).  In Mr. Fogel’s case, I was spear-heading an interesting project in an interesting space and so had some experiences to offer; in Mr. McConnell’s case, I believe my pitch for reviewing was my relative lack of experience at the time – could his material speak to a neophyte software project manager?  (This was years and years ago – I’m now much older and much less neophyte.)  Both spectrums were useful for the authors: in the one case, could I offer a new insight?  In the second case, does the insights the author shares come through to the audience they intend?

I find it interesting that both ranges of experience are useful.  I find that to be the case in my projects, as well, both software and otherwise.  Seek to contribute whether you’re the expert or the newbie.  The value you offer is different, but valuable on both ends!

I’m working on a tech talk for my company on Kubernetes. I use Kubernetes on a customer system every day. I wrote and delivered a half day training class on it, help customers understand how to use it and then help them work through any hiccups they run into in deploying their own systems to it.

So, you’d think I’d be in good shape for a 45 minute tech talk which gives an overview and shows a little bit of stuff running.  If I were just talking, I’d be fine: I can talk about why k8s, where it came from, key concepts and benefits in it, how to deploy and monitor things within it, how to figure out what’s wrong with your system running on it… What I hadn’t had to do was deploy my own Kubernetes installation: there are a few to work with in our customer environment, which each have their own quirks.  If a particular quirk is getting in my way, I jump to a different cluster assuming there’s no other constraint preventing me from doing so.  That helps me and my team keep abreast of things our clients will run into, and we share that guidance with them.

Last night I started to set up my company laptop, the one I’ll use for the presentation next week, to have a single node Kubernetes implementation via minikube. I’d looked through the tutorials, everything seemed straightforward. Very straightforward: either install it via curl or via a brew cask install.  Minikube downloaded, I started my cluster, and pointed kubectl at that cluster. Very neat and tidy.  But I spent a few hours last night trying to figure out why it would neither finalize my deployments (and thus give me running pods) nor let me delete an existing deployment. Nor why the minikube dashboard wasn’t available. The dashboard used to be, when I’d done a fabric8 installation that itself bundled a minikube… When I tore that down to install from scratch to give tested instructions for the tutorial, though, the dashboard stopped working. (I’ll take a guess that it’s because the pod used to satisfy the service that provides the dashboard endpoints isn’t deploying – seems logical given other evidence.)

Tonight I’ll spin up a Google Container Engine small cluster to let me have an alternate path for my talk, rather than continue to beat my head. That’ll also let me demonstrate multi-node interactions and seeing pods jump from one node to another. Critical path is the talk, not the talk running off an environment on my laptop.  But having a hard time taking my brain away from debugging why minikube isn’t working.  When I figure it out (after the tech talk’s written), I’ll come back and see if I can leave a tracer here for some other stuck soul.

Realized my last post was almost two months ago! It’s been a bit of a busy period.

Clementine shared balloons and fun with kids at a community festival, marched in a parade, and served as a entertainment on the midway. Oh, and got herself a new pair of polka dotted oversize Converse sneakers!

I bought, built, and directed the decorations for vacation bible school, led games for vacation bible school, and went with a group of youth (including both of my daughters) on a week long mission trip. I even got to use one of my clown magic tricks for the VBS kids… that was my excuse to buy the requisite parts, so good to use it at least once or twice.

In between, I’ve also been working my way through a Coursera multi-class specialization on strategic leadership, and gotten to help bring a few candidates into our company. A bit more learning on Kubernetes, Gitlab, and ELK as part of the day job, too.

Oh, and we got a kitten! Strategy suggests I should keep up with my daily allergy pills. Miso (kitten’s name) is cute, but he makes ‘me so’ sneezy

I need to be at work earlier this week than normal. We’re giving training on Docker and Kubernetes, and the first class session starts at 8. The classroom needs to be open earlier than that, and I want to be in and settled before the first student arrives.

That’s meant that I’ve set up a whole system of wakeup alarms, between my phone and my alarm clock.   The phone has multiple staggered times, and the clock has another one.  That’s to handle the challenge of sleeping through or turning off an alarm and remaining in bed.  But that means that once I’m up and moving, there can be multiple alarms that still go off. Did I mention that my husband _doesn’t_ need to get up earlier this week?

I’m now thinking through my optimum user interface to let me say, I’m up, keep the rest of the alarms from going off.  Is it: no more alarms at all today?  Is it: no more alarms within the next two hours?  Is it: show me a table of the alarms yet to go off this morning and let me turn them all off in one quick move?  Is it: here are my wakeup alarms – turn them all off, yet leave reminder alarms for later in the day?

Oh, and I’m hunting for the source code for the existing Alarm clock app and widget.  The app has an option to let me see its open-source licenses (Apache, MIT).  That bodes good things for getting to try something out..  I’ve only done minimal Android development before, just enough to make a UI to control a Furby.  I like the idea of solving a personal concern and getting to play in geekery.  Now we’ll see if liking the idea actually translates into hours spent sometime.

Some stuff for work is running on a Kubernetes 1.3 cluster.  It nicely restarted when something went wrong in its environment.  That’s good.  Our pods were logging – that’s good.  But when the pod restarted, its logs were lost: that’s bad.  No idea why the pod restarted to help determine if it’s something we should either prevent or add a health check for.  So, now I’m evaluating logging solutions appropriate for our Kubernetes environment.  And because I’m a geek, I’m blogging about it here in my pjs on my day off: you know you’re in the right field when you’re enjoying researching something for it on your day off.

Scope of problem: not trying to log all of the kubernetes events themselves: that’s the job of the infrastructure.  Which also means then I’m not worried about filtering out messages I shouldn’t see from pods that aren’t my own.  I want to collect log messages in my namespace, have them tagged (at least) by which application source they came from, and I want to have them be archived _off_ of the cluster.  The guys running the infrastructure are great, but if they reconfigure the cluster (as happens occasionally) or have a hiccup with their persistent volumes, still want my logs.

A bit of survey suggests there are two primary solutions out there: Fluentd or Logstash, one of the components of the ELK stack.  Both are open-source.  I’ve long heard of ELK, though not yet myself futzed with it in a system.  I’ve seen many stickers for Fluentd in open-source conferences…  🙂  I’ve also noticed Fluentd as part of the Cloud Native Computing Foundation‘s list of supported projects.  Hmm..  that last bit gives me a bit of bias towards Fluentd, sight unseen.

So now to look for other folks comparisons of the two to pick the first one to try out..  My strategy is to start with one and see if it’s ‘good enough’ to meet our needs.  But I’ll often rely on what the ecosystem has already discovered to make my choice between.

Fluentd vs. Logstash: A Comparison of Log Collectors: logz.io uses both.  My takeaway – leans towards Fluentd in its recommendation, even though it’s built on ELK

Fluentd vs. Logstash: Panda Strike’s tried both..  They’ve ended up with EFK (ElasticSearch, Fluentd, Kibana) as their platform for reasons of stability and performance.  Side note: describes logs forwarders vs. log shippers.  Found another post on their site that describes EFK in action for JMeter load testing result visualization.    Another side note: found a good bit of useful content in their blog overall..  adding ’em to the rotation of stuff to keep an eye on…

Fluentd vs Logstash: Jason Wilder walks through both.  Older post (Nov 2013) but assuming all is still relevant, good technical info.

Outcome: looks like I’ll be starting with Fluentd.

 

 

 

 

I got a cold-call from a recruiter today, seeing if I’d be interested in a Technical Manager job.  That’s right in my bailiwick: I tend to be more technical than most technical managers, but she had me pegged in the right general zone of job, anyway.  What surprised me was her saying she had a resume of mine from 2006.  Wow.

Luckily for me, 2006 rang a bell.  That’s the year my son was born, so that’s the year I traded a commute around the Beltway to a job I loved for a new series of adventures with a company closer to home.

In 2006..  I was building an e-commerce system using Struts.  I _think_ we were debating a rewrite in Struts 2.  Somewhere in that time frame, I became a certified Scrum practitioner.  (I’ve since let the certified part of that lapse, though I’d still consider myself a Scrum practitioner.  Just one who doesn’t pay certification fees to prove her worth.)  We’d do production deployments once a week, beginning at 11pm and working through the night until we succeeded or rolled back.

Since then, I’ve done all sorts of things…  OSGi in its relatively early days.  Grails web applications that I helped take GOSS (government open-source) and then fully open-sourced.  D3 visualizations for SBIRs.  Actually, D3 visualizations for two different SBIRs.  Many hours realizing I needed to better grok D3.  Lots of business development.  Winning multiple proposals and then staffing teams to do the technical delivery.  Where those teams typically included me, hands-on coding.  Ramping up on AngularJS and Node to lead a “thin-client” team.  That’s as opposed to the program’s thick-client team building on top of Java Swing – thankfully, although I managed the overall UI team, didn’t have to wade too far into the Swing code.  Docker containers containing stub implementations of enterprise system interfaces we wanted vendors to be able to execute against.  Now Kubernetes: figuring out how to deliver and sustain systems in a cloud.  I’ve delivered two OSCON briefings, one using an Arduino and C and one using Go and a Raspberry Pi.

It’s been a fun 10 years.  If she calls again, though, she’d better go find my updated resume.  For the record, LinkedIn’s got a pretty good idea of what I’ve been up to…

A few months ago, I joined a software team that’s making use of Kubernetes as a platform to provide elastically scalable software services.  It’s Docker + more magic.

One of the tenets espoused by some folks building container deployments is that a container should only have the software on it required to run a single process / microservice.  The particular software system I’m deploying is a private Docker registry, but to deliver the service, there are currently 4 Docker containers in play.  (I actually think that to solve a problem I’m having, I’ll need to add two more.)

One of the containers wasn’t behaving nicely for me today.  The container provides a web user interface and needs to communicate with a database provided by another container to store and retrieve its data.  But the logs were saying that the user interface container couldn’t log into the database.

The first two options that came to mind:

– the hostname wasn’t accessible to the ui container

– the port wasn’t open on the database container

Turns out, it was neither of those: it was a badly shared database password.  But in trying to prove or disprove those options, I ran into a challenging problem.  See, the UI container _only_ had the binary for the UI process on it.  Which meant: it didn’t have ssh, telnet, curl, wget, netstat, or any of the various other options I came up with to verify network connectivity between the containers.  How to get my answer?  Google + Stack Exchange to the rescue!

In recent versions of bash (which thankfully the golang image did provide), bash will let you access TCP and UDP ports directly, ala

cat < /dev/tcp/databasehost/3306

Running the above from the ui container got me a garbled prompt received from the database container, which proved that the hostname resolved AND the port was open, all in one command.

Kudos to Peter Mortenson and his answer on superuser.com  for helping me crack the bug.  Tuesday’s bug (courtesy of the Monday holiday) will be to see if Beego.Run() doesn’t crash if I manage to map in the app.conf file not otherwise being provided through my Kubernetes replication controller descriptor.  Not good to have applications just stop without an error message!!