Folks who are paying closer attention to this blog than it warrants may have caught notice of a link in the left navigation to a ‘Kubernetes 101‘ presentation. That link came about when I was asked a year or two ago to give a presentation at work on Kubernetes. I built the presentation deck based on a presentation I’d put together at a previous company which they were kind enough to give me access to again, and THAT presentation was a recap of some training materials I’d built out for a customer. So, I’ve gotten to present on Kubernetes a few times.

I’m now on my third project making use of Kubernetes, or k8s for short. The first go-round, I helped developers understand how to deploy things to it and someone else stood up and maintained the cluster. The second project, I built tools (“operators”) to run within k8s, as well as built scripts that automated the deployment of our clusters. This go-round, we’re using a new k8s distribution, with its own tooling for deployment and administration, and part of my role is to figure out whether our team found all the bits I’d been able to turn on in previous installations. (Auditing, for the record, is a good thing…). With each new project, k8s has matured and my angle for working with it has changed, so I get to learn and try new things.

That’s generally how software and systems development works… no one (or at least, vanishingly few) ever really knows a tool or language inside and out completely, particularly in connection with its full ecosystem. I’ve gotten to write Golang, Ansible, and Java (via k8s’s client-sdk). I’ve used REST APIs invoked via curl or hit the same endpoints using kubectl and its command syntax to interrogate k8s internal state. I’ve figured out how to query Prometheus using PromQL, and then how to interact with a time series database to which we’d exported the Prometheus data. Oh, and with each new release of k8s (they’re about to release 1.18), the capabilities and APIs change.

I got to interview an internship candidate today, and she (yay!) asked me what sorts of things you have to know to be a good candidate for our company. I told her a few of the technologies our current interns are using, but tried to make clear that the biggest thing about a career in technology is that you have to keep learning. That you have to keep humbly realizing you don’t (and can’t!) know it all. That you keep plugging away at deepening and widening your experience. That sometimes your experience tells you to bring in someone whose breadth and depth hits the problem from a different angle than your own.

Today was a fun day. Can’t wait to see what projects 3, 4, … and n, in k8s or other things, bring my way.

I spent most of last weekend at my alma mater, UMBC. Friday night, I met some new mentees through the CWIT mentoring program, and Saturday and Sunday were spent at HackUMBC. So, lots of opportunities to observe undergraduates in action and answer questions about what sorts of things my company does and who we hire.

The hackathon was a very interesting experience for me. Participants got started after lunch on Saturday and turned in their projects Sunday at 1. There was no guidance on what to build or who to build it with, other than that teams could consist of 1-4 participants. There were a few prizes offered by sponsors such as ourselves for which a team could go after – ours was for best data visualization but others sought best hack using Docker, best use of public financial data, or best use of Google Cloud Platform, just to name a few. There was nothing stopping a project from applying for multiple categories: I know we saw a project for our data visualization judging that used financial data and Docker containers – not sure if they hosted anything on Google Cloud Platform.

The goal of a hackathon isn’t only to win prizes, of course. It’s also supposed to give teams a chance to learn and apply new skills. The team that won our prize used Unity, a gaming engine. Other teams used d3.js or plot.ly or Google Maps + some HTML or even Minecraft (linking directly to that project – innovative idea). Some teams got farther than others: one team had a great concept and a locally installed Jupyter notebook (via Docker, if I remember correctly: check off a potential prize category) with a well-built out machine learning model that they could reason about and defend. But they just hadn’t gotten to hooking up their prototype UI to their data. Another team had a drop-down list to trigger a visualization, but could only as yet talk to their concept of the visualization. That didn’t win them our prize, but still gave those teams a good bit of interesting experiences to talk to us about.

Remember, these students had 24 hours to bring together a team, put together a project concept, and then execute on their concept. Now, I know practically that some of these folks team regularly together. And at least one team indicated they’d been scraping Twitter data ahead of the event to give them a leg up on building out their display that needed geo-located tweets. Still, though: I saw team formation happening in the hackathon Slack channel and at the tables in front of our sponsor area.

What was more amazing to me was that a few teams came up to our table and asked my guidance on what tools to use. Some of that happened late in the afternoon on Saturday. Meaning, they were picking their toolkits on the fly, and then building out their app without prior experience in at least portions of the stack. For a project that had a hard timeline, though admittedly loose requirements. Wow – the very thought gives me personally the shudders, were I in their shoes. Uh, I’d want to form my team knowing that folks had complementary skills that could come together to solve a generic set of problems. One team told me they didn’t know how to interact with databases and knew they wanted one, so they coded up a flat file database on the fly. I have to believe I’d have taken a different route, but kudos to them for pulling something off with it.

I’m trying to imagine how to use that hackathon idea for an event at my company or through BWIC. I’d have a hard time personally carving out a full weekend: attending the event during the day was a big enough lift, but many of the students stayed overnight. One indicated to me she’d had a great idea and burst of energy after her 20 minute power nap. Ugh. Been there, done that, don’t wanna go back! But maybe spreading it out over a week would work. Or constraining it to a day. It just looked like so much fun!

Handy command for k8s pod status:

kubectl get pods –all-namespaces –field-selector=status.phase!=Running -o wide

May your list be empty.

(Decoding: find me all pods in all namespaces in my kubernetes cluster that are not running.  Note not running can mean: Completed.  Completed can be good.  Others are usually not.)

I’m cheating and cross-posting my writeup of my KubeCon visit – hit my writeup on LinkedIn. Oh, and shoot me a message if you’d like to work for a company that sends folks to conferences – I was one of a group of 5 of us out there from ClearEdge. Never hurts to be able to compare notes with other folks from your company as to which sessions to catch the recording of and which ones to bypass.

I’m talking with a group of young ladies this week about software development. They’re part of a HowGirlsCode group, which seeks to “provide[..] computer science and engineering education designed to inspire young girls in computer and engineering sciences”.   Women apparently only earn about 20% of the computer science undergraduate degrees and then often leave the field, so that only some 13% of folks in the field are women.[1]  I wish that didn’t ring true but it does.  It’s rare and exciting when there’s another woman on my team.  I make a point of trying to recruit women in particular, just so we can amass a core group of gals to show the world how it’s done.

The talk this week focuses on the fun in software development.  I love it – tried to leave the field in my mid twenties and finally realized this is where I was meant to be.  It’s provided well for me and my family and given me lots of opportunities.  In what other field could I build out conference talks about Furbies (twice?!)  I’ve gotten to travel, both in the US and around the world.  I’ve _never_ been without an opportunity to learn something new.

When I first dreamed of doing software development, I thought I’d go into artificial intelligence.  When I graduated college in the mid 90’s, AI seemed far away, something that only PhDs were thinking about.  While we’re still not where I thought we’d be when I was in high school (and earlier) contemplating a career, researchers in the UK recently announced the world’s largest computer simulation of the brain.  There’s software now in my phone.  There’s software in my car.  I bought Christmas presents this evening through a web browser hooked up through the Internet to an e-commerce infrastructure, undoubtedly hosted in a cloud infrastructure somewhere.  Heck, if I cared to, there could be software in my refrigerator!

I’m really looking forward to talking with the HowGirlsCode young ladies on Thursday.  Computers and software are ubiquitous as far as they’re concerned: they may not even realize how software infuses almost everything they touch.  But I’m looking forward to showing them the opportunities that opens up!

 

 

Question from one of my teammates on my new work team:

“Did you just learn techA & techB in the 3 days since you’ve joined the project?”

Well, enough to make things work for this portion of the project, anyway.  And thank you for noticing!

My teammate and I have been “pair-programming” remotely, by which I mean: we talk over the phone about the approach and occasionally screen-share / present to show what we mean.  Oh, and of course, commit at regular intervals into a shared git branch.  When I joined the team, the story (uh, work unit, I guess, for those of you not well-versed in software) was written such that the work would have 4 subtasks.  I proposed doing it differently, based on some prior experience I had with techC, which is the end-result of our efforts with techA and techB.  The team bought in, and off we went!

Challenges:

  • neither my teammate nor I had much experience with techA or techB
  • my teammate doesn’t have much experience with techC
  • my teammate and the rest of the team are in Minnesota, which means: no whiteboard drawings, an offset of an hour in schedule, we haven’t met each other in person, …
  • I’m brand-spanking new to the team, so am still navigating getting all of my accounts, figuring out how not to break other folks’ work, figuring out how to prove things _do_ work, …

It looks like by not too much longer today, I’ll be putting in my first merge request for a significant feature for the new project.  Woot!  Good first (real) week.

For those techies interested in the secret decoder ring for the technologies:

  • techA = Ansible
  • techB = Salt
  • techC = Kubernetes

 

 

“NOTE”:ed in documentation I was looking at today…


NOTE: “Default” is not the default DNS policy. If dnsPolicy is not explicitly specified, then “ClusterFirst” is used.

This is the sort of thing that should _not_ pass muster for code-review. Kudos to whoever recognized the issue and at least put it in documentation. But there oughta be a kubernetes GitHub issue out there somewhere to fix the above. And no, telling me that it’s been released this way and thus must be maintained is not an acceptable argument. Deprecate the word ‘Default’, if you must. If the default is not actually that, then the impact is likely small. ‘Default’ could become ‘InheritFromNode’ or ‘Inherited’ or …

A little bit of Google digging found a related, but not quite what I mean GitHub issue. Grumble, grumble, growl….

A friend / previous co-worker of mine sent out an intriguing tweet:

George is an agile coach, among many other things.  (I believe him also to be a sailor and a grandfather, and someone whose tweets I enjoy.)  I assume his book proposal has something to with agile development or coaching of agile teams.  I’ll be looking to see when he announces that someone’s accepted it!  In the meantime, I’ve offered my services for review. 

George’s book would make #3 of book reviews for me.  He’d be in the esteemed company of Steve McConnell (‘Software Estimation’) and Karl Fogel (‘Producing Open Source Software’, 2nd edition).  I’d forgotten the McConnell book until George mentioned it recently, and Karl’s finished up his second edition fairly recently (November).  In Mr. Fogel’s case, I was spear-heading an interesting project in an interesting space and so had some experiences to offer; in Mr. McConnell’s case, I believe my pitch for reviewing was my relative lack of experience at the time – could his material speak to a neophyte software project manager?  (This was years and years ago – I’m now much older and much less neophyte.)  Both spectrums were useful for the authors: in the one case, could I offer a new insight?  In the second case, does the insights the author shares come through to the audience they intend?

I find it interesting that both ranges of experience are useful.  I find that to be the case in my projects, as well, both software and otherwise.  Seek to contribute whether you’re the expert or the newbie.  The value you offer is different, but valuable on both ends!

I’m working on a tech talk for my company on Kubernetes. I use Kubernetes on a customer system every day. I wrote and delivered a half day training class on it, help customers understand how to use it and then help them work through any hiccups they run into in deploying their own systems to it.

So, you’d think I’d be in good shape for a 45 minute tech talk which gives an overview and shows a little bit of stuff running.  If I were just talking, I’d be fine: I can talk about why k8s, where it came from, key concepts and benefits in it, how to deploy and monitor things within it, how to figure out what’s wrong with your system running on it… What I hadn’t had to do was deploy my own Kubernetes installation: there are a few to work with in our customer environment, which each have their own quirks.  If a particular quirk is getting in my way, I jump to a different cluster assuming there’s no other constraint preventing me from doing so.  That helps me and my team keep abreast of things our clients will run into, and we share that guidance with them.

Last night I started to set up my company laptop, the one I’ll use for the presentation next week, to have a single node Kubernetes implementation via minikube. I’d looked through the tutorials, everything seemed straightforward. Very straightforward: either install it via curl or via a brew cask install.  Minikube downloaded, I started my cluster, and pointed kubectl at that cluster. Very neat and tidy.  But I spent a few hours last night trying to figure out why it would neither finalize my deployments (and thus give me running pods) nor let me delete an existing deployment. Nor why the minikube dashboard wasn’t available. The dashboard used to be, when I’d done a fabric8 installation that itself bundled a minikube… When I tore that down to install from scratch to give tested instructions for the tutorial, though, the dashboard stopped working. (I’ll take a guess that it’s because the pod used to satisfy the service that provides the dashboard endpoints isn’t deploying – seems logical given other evidence.)

Tonight I’ll spin up a Google Container Engine small cluster to let me have an alternate path for my talk, rather than continue to beat my head. That’ll also let me demonstrate multi-node interactions and seeing pods jump from one node to another. Critical path is the talk, not the talk running off an environment on my laptop.  But having a hard time taking my brain away from debugging why minikube isn’t working.  When I figure it out (after the tech talk’s written), I’ll come back and see if I can leave a tracer here for some other stuck soul.