Seeing Red

Can you imagine having this level of objective feedback within five seconds of making your change? I can, because despite my being lazy about it in recent years, my teams and I have enjoyed the benefits of investments in test.

Okay, sooooooo it's possible that wasn't a good change. pic.twitter.com/04ZUjW1Rnq

— Nick Craver (@Nick_Craver) October 26, 2021

Can you imagine merging this type of change and only finding out how bad you’ve broken things after it’s in production and impacting your business? I can, because due to my being lazy about it in recent years, my teams and I have suffered the pains of throwing things over the wall.

Why Test

I’ve given conference talks on all of this, but the React Native docs say it better than I ever could. Check out the Why Test section.

Assumptions

Here are a few things that I’ve been guilty of saying.

Works on one platform, no need to test the other.

And from browser compatibility to PhoneGap plugins to React Native components, I’ve been wrong countless times.

The change I made to this shared component looks great when I verify the bug I was trying to fix. Surely it’ll be fine for all other consumers.

Of course, even when I think I’ve added nothing but new functionality driven by optional parameters with sensible back-compatible defaults, I’ve been wrong countless times.

There’s no way we could get here in null or empty set conditions, so we don’t need to guard against those conditions.

LOLOLOLOLOLOL

I’ve done a custom sort function enough times to know the difference between returning -1 and 1, and I’d certainly never forget to return 0 when the items should be treated as equal.

I have become the physical embodiment of shame.

Two Birds, One Stone

I recently shared my silly little side project, Phasekeeper. As we’ve continued using this at home, I’ve become frustrated that the end screen didn’t sort the players in descending order.

I’ve also wanted to create a reduced, streamlined example to demonstrate the return on investment with automated testing.

Let’s go!

Exploding components

Before I tackled the sorting, I had something else on my mind.

There’s no way we could get here in null or empty set conditions, so we don’t need to guard against those conditions.

I cast no stones, as I’d need a pitchback to direct them at myself, but I’ve lost count of how many times a component explodes while accessing foo.bar while foo is falsey.

In Phasekeeper terms, I’ll admit this is contrived. The flow of Phasekeeper does ensure that we wouldn’t get to the End Screen without players. Future me looks back at this and says, “That’s what you thought at the time.” Present me says, “Why not take this opportunity to demonstrate how we’d test and fix it anyway?” Past me says, “Listen to them.”

Now, we have a few options to dodge the investment in automated tests:

• Could I have checked this manually?
• Spotted it during code review?
• Just been a better @^#*! developer who guards against every possible condition, no matter how unlikely?

Probably at least two of those three, sure. But in less than twenty minutes I fixed the bug with an objective data point. The collection of that data point is repeatable to ensure we don’t regress with future changes:

• At 7:49AM I tweeted about the Why Test docs
• I drank some coffee while deciding whether to do the thing
• At 8:04AM I had a failing test that proved the bug
• At 8:07AM I had committed a fix to make the test pass

Getting things sorted

Next, I committed a change to test and fix the sorting of players. Note that this test arranges the state so that all players had completed Phase 10. More on that later.

So far, so good. We’ll render the End Screen with a test state on our context, and now we know

• The Congratulations text is presented
• It uses the correct name
• Players are listed in the order of their scores (remember, in Phase 10, a lower score is better)

Whoops

Do you remember when I said

I’ve done a custom sort function enough times to know the difference between returning -1 and 1…

I promise you, I did not do this on purpose for the sake of a contrived demonstration. I actually, with two decades of experience, got the sort return values backwards. Again.

Here’s the commit where I fix the sorting to account for the condition where not all players complete Phase 10. For those who don’t play this game often… this is not an edge case!

It’s not as clear in this commit as in the Exploding Component commits because I didn’t commit the failing state first, but I did

• write the new tests first
• run the tests, fully expecting to see everything green
• watch the new tests fail
• facepalm myself on the reversed -1 / 1 return values
• swap the -1 and 1 returns
• watch the new tests pass

It’s important to note that I’m not just getting an objective data point that I’ve fixed the condition I was concerned about. What did I get as a bonus?

• I didn’t have to click through my UI to manually check this result
• Which means I got the answer in less than three seconds, not minutes
• AND ALL PREVIOUS TESTS ALSO STILL PASS WITHOUT ME MANUALLY RETESTING THEM

That’s right! I don’t have to go do a manual regression pass to make sure that all those previous scenarios still behave as intended.

Mileage Varies

As always, everyone and every team is different. You may get different returns on this investment.

For me, these tests have become executable specifications that provide a more efficient and more objective evaluation of the code. It took me less time to write and run them than it would have taken to run through the UI for each combination.

We now have a permanent record of which combinations we’ve checked, and which ones we haven’t.

And that’s just the initial investment. Now that the harness exists, it will be even more efficient to assert other interesting combinations if the need arises… all the while avoiding the need to repeat manual regression passes against the previously tested combinations.

All in under three seconds.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

 PASS  src/components/EndScreen.test.tsx
  EndScreen
    with no players
      √ it renders an error view (25 ms)
    with all players completing Phase 10
      √ has congratulations text (7 ms)
      √ congratulates the player with the lowest score (4 ms)
      √ lists all players (4 ms)
      √ lists the winner first (3 ms)
      √ sorts the other players with the lowest scores first (3 ms)
    with only some players completing Phase 10
      √ has congratulations text (4 ms)
      √ congratulates the player that completed Phase 10 with the lowest score (4 ms)
      √ lists all players (7 ms)
      √ lists the winner first (5 ms)
      √ sorts the other players in order of completed phase, then lowest score (4 ms)

Test Suites: 1 passed, 1 total
Tests:       11 passed, 11 total
Snapshots:   0 total
Time:        2.596 s, estimated 3 s
Ran all test suites.

If you love to play Phase 10, but you’re tired of finding pen and paper or doing the math, here’s a free scorekeeper I threw together as a side hack. We’ve been using this for weeks in my house, and I figured, why not share it with the world.

Navigate to https://phasekeeper.davidruttka.com and have fun! Since this is a Progressive Web App, you can also “Add to Home Screen” to make it feel like an installed application on your phone.

The source is available on GitHub. While there are some open issues around visuals, touch targets, and experience optimizations, I hope you find it as useful as we have!

I’ve gotten some interesting questions about the airline and grocer metaphor and my thoughts on delivering value rather than features.

Here I’ll try to fill in some of the gaps in how I wrote those posts. All along the way, in case I haven’t said this clearly before, remember that all teams are different. I’ve curated these thoughts across several teams and almost two decades, but even for me, it didn’t fit everywhere.

I’ll also update this post if additional questions come in, and I’ll put a log of updates in this spot whenever that happens.

Isn’t That More Work?

I can see how keeping the next right thing for the business on the top of the board optimizes for devs to self-serve when they need more work. Isn’t it a lot more work for project managers, though?

I’d ask two questions in return:

1. Have you ever finished a bowl of oatmeal and left it next to the sink all day?
2. Have you ever finished a bowl of oatmeal and rinsed it immediately?

One feels like less work in the moment, but you’re going to have a much harder time later. The other invests 10 seconds now to save a lot of friction (both figurative and literal scraping) later.

I am not proposing this methodology only from the perspective of a software engineer. I’ve also been in the tech lead and project manager seats, and this took roughly five minutes a day, often a no-op if the business needs had not changed.

Knowing exactly what work sits ahead of the team and letting devs self-serve by pulling from the top proved dramatically more efficient than pre-assigning days or weeks worth of work to individuals. It also gave us an immediate, realistic signal when communicating with customers about what’s Now, Next, or Later, and allowed us to effortlessly change the order if the customer provided feedback that a Later should actually be Next.

Let’s also get real about the time you’re already spending planning, replanning, re-replanning, doing math to balance work across the team, doing it again when reality didn’t fit your assumptions, shuffling work items between people, running bespoke queries to determine what to give a dev who says they’re out of work…

How or When Would We Even Start?

I can see how this works if we’ve already got the order established and we’re simply maintaining it, but we’ve currently got a real mess and don’t have time to clean it up. The oatmeal is already caked on, and we’re already struggling to keep up!

You’re struggling to carry a couch up ten flights of stairs by yourself. Someone is offering to join you with a dolly, and they’re pointing to the freight elevator. You’re replying, “Ugh, thanks. That looks amazing. Maybe I’ll try that someday, but for now I just really gotta get this done!”

You’re being given a deal through which you can invest $5/month for a $10/day return, and you’re refusing to skip one morning coffee.

What I’m hearing is that you’re too busy to consider improvements that have been proven, if only anecdotally, to yield exponentially greater returns.

And I’ve done that myself. Then I learned that change will always feel hard, often much harder than it truly is. The human brain perceives change as a threat, and will go to great lengths to avoid it. Let’s recognize what our brains are doing and make our best efforts to stay objective about what’s really involved here:

1. We’re taking the time we spend calculating and pre-assigning today
2. We’re acknowledging how often those predictions and assumptions are incorrect
3. We’re taking the time we spend re-calculating and re-assigning each time business needs or team bandwidth changes
4. We’re exchanging all of the above for simple, continuous adjustments that send clear signals to the entire team re: the truth of Now and Next.

We ultimately expend far less time and far less energy, and we eliminate the stress and urgency of a synchronous wait loop to figure things out in the moment of greatest need.

Too Many Work Items To Put In Order

Are you seriously asking me to constantly re-rank hundreds or thousands of work items?!

No. Sorry if this wasn’t clear, but there’s no need to rank all of the Laters… too much will change by the time we get to them anyway.

The methodology I’m proposing focuses on keeping the team executing on the most immediate needs of the business, and there’s a limit to how much the team can take on at any given time. Once we have enough items ranked to ensure that no one is idly waiting for their next assignment, we’ve gone far enough. Indeed, we’ve saved ourselves the churn of ordering all the things that are destined to be reordered next week, anyway.

There’s magic built into this, too. Work that does not carry sufficient impact automatically disappears from scope! If you don’t care enough to pull it into the ranked list, you don’t care enough to do it. Years ago, a conference speaker told a story in which the customer cut 30% of work from scope because ordering the stories made them realize how much they didn’t really want. I’ve later observed this myself. Order eats priority for breakfast.

Perfection

My team should do this, but we need to wait until we’re sure we can do it right.

You will never hear me propose a wholesale transition to perfection. Continuous improvement, starting from the now, will always win against the deferred perfection that never comes. With that, each step can be as small or large as you do have time for today. The gains from even the smallest investment will snowball rapidly.

My most successful teams made time for regular retrospectives that went beyond a 15 minute “Mad, Sad, Glad” exercise. This helped us target the right process improvements to make relative to current conditions and committments.

We’ve also found it helpful to take the following approach:

1. spec out the goal state as bullet points
2. highlight the bullet points the team already does well in green
3. highlight the ones the team has started but not perfected in yellow
4. highlight the ones the team hasn’t touched in red
5. decide as a team which yellow to turn green, and/or which red to turn yellow
6. hold each other accountable until that happens
7. repeat

Save me, SME

Telling the team to pull the next right thing assumes that everyone has the same strengths, context, and experience. In the airline analogy, how you do handle oversized baggage that not all agents can handle?

Fantastic question. On a similar note, Eric Brechner recently posted an insightful take on delegation that made me rethink a lot of my airline versus grocer metaphor.

There is absolutely a place for subject matter expertise, and we do need to consider whether work may be performed more efficiently by a person who has the deepest understanding and context.

Backups and shadows make a lot of sense. Even one backup has doubled your team’s resilience in that subject area, avoiding a disaster if the One True Hero gets sick or leaves the team. But even then, what happens when there’s more urgent work in that area than those two can reasonably take on in a sprint? Is this the bottleneck you signed up for when you chose silos over knowledge share and cross-training?

Here’s how we handled it in the past. If a person doesn’t feel capable of handling the next top item, trust their judgment to take the second instead. Maybe even the third. If they’d need to go to the fourth, heed the smoke alarm! The process screams at you, “You have not enabled your team to carry out the needs of the business!”

It’s at this moment that we intentionally decide to invest in our team’s success to accelerate business impact. Whether that’s pairing, a ramp-up session, or (you tell me some other options), the business slows down until you clear this roadblock. Spoiler: regular pairing and internal tech talks give your team a tremendous head start on these fire drills… but that’s another post.

The other option, for teams or organizations where there truly is deep specialization, is to apply the same “next right thing” approach to each subject area. SMEs pull from the ordered list of work that fits their specialized domain, reverting to the generalists’ queue if that runs dry.

After receiving questions and comments on this post, I’ve started a FAQ to clarify points where I wasn’t clear enough, and rethink points where I didn’t consider all the angles. Please let me know if you have additional feedback on this post.

Activity Versus Impact

…driving toward the true goals of the business

Several years ago, I heard the term Feature Factory, and it changed the way I think about delivering value. I’ve also heard managers encourage me to understand the difference between activity and impact. Picture wheels spinning in the mud. Lots of activity. No forward motion.

Some teams will add features because someone had an idea, and everyone needs to stay busy. There’s a lot of activity in this model, but the team may not be driving toward the true goals of the business.

Other teams will add value, focusing on desired outcomes (objectives, key results, key performance indicators, and any number of other gauges). There’s a lot of impact in this model, and teams will almost certainly deliver what the business needs.

What Do We Want To Do Today

…driving value for the business’s sake, not chasing features for busyness’s sake

When determining what to deliver, I find it helpful to start with why. Determining why we are doing the thing ensures that we’re driving value for the business’s sake, not chasing features for busyness’s sake.

Importantly, we must remain aware that if we cannot understand and communicate why we’d do the work, then going further may not have any value. It may be another wasteful output of an activity-ridden Feature Factory.

Order Versus Priority

…if it’s not a priority, why are we doing it at all

If we only have one idea with clear business value, we can get straight to work. In reality, we’ll end up with a long list of ideas from brainstorming the Why, What, and How. We’ll need to determine the relative impact against these opportunities. Business development is a little outside of my wheelhouse, but in terms of executing on requirements, I can say that order has been much more useful than priority.

I typically see four or five Priority buckets, but if it’s not a priority, then why are we doing it at all? So, we end up with a lot of items marked as a top priority, and we still have to determine the relative value within that bucket. We define Severity or add Milestone tags to help, but we still have many items with the same markers.

Allow me to present the deep magic of determining Order instead of Priority. Imagine that we’re shipping tommorrow, like it or not.

• Which of the outstanding stories do we wish had gotten done last sprint?
• Which bug is most damaging to the user, or embarassing to the team?
• What contractual or legal ramifications come into play?
• Is there tech debt that, if we had paid it down last month, we’d be better set to execute on all of the above?
• Are there items we want to put in front of testers or customers sooner, for earlier feedback or to build good faith?

By asking ourselves these questions, we can determine whether each deliverable should come Now, Next, or Later. Better yet, there’s no need to rank all of the Laters… too much will change by the time we get to them anyway.

Staying Objective

What is the next thing the business needs, which is not already in progress?

Everything above helps us determine what is impactful to the business. What follows has helped my teams to execute efficiently to continuously and consistently deliver that value.

We’ve touched on Why, What, How, and When. Regarding Who, assigning work, we can save a lot of time and energy if we embrace uncertainty and start with what we can prove:

1. We can prove that work items exist
2. We can prove that we established a desired order of delivery relative to business impact
3. We can prove whether a work item is done, in progress, or not started
4. We can prove when someone is free to take an item

Everything else changes daily under our feet. If a team encounters a lot of randomization, found work, or other uncertainty, then deciding in advance who will do what is an exercise in futility. We’d do well to choose a queueing system that avoids reshuffling or rebalancing. So let’s think again about airline check-in versus grocer check-out.

If a person checking in for their flight argues for 10 minutes about their seat upgrade, anyone other than that person and their agent hardly notices. The other agents continue to process all other customers in order, always calling for the next in line. Compare this to what happens when someone at the grocery store argues over a price check.

By deferring assignment until work begins, we also capture a clear signal on our board. What is the next thing the business needs, which is not already in progress? This should be provable, but if work is assigned before it begins, we’ve introduced the cost of ambiguity with no benefit in return. That ambiguity begets additional wasteful communication as we attempt to unwind the stack of inaccurate predictions.

Deferring assigment also allows us to go with the flow of dynamic business demands.. Much like a frequent flier with elite status, we may find new work or encounter an emergency that must be taken ASAP. If we haven’t pre-assigned future work, then the only adjustment we need to make is to put that work item on the top of our ordered list. On the other hand, if we have pre-assigned work, we need to recalculate what can be taken off of whom.

A very similar principle applies to team dynamics. When people join the team, get sick, or move on, we’ll only need to reshuffle if we chose to play a guessing game that we know we’re destined to lose. If we stay grounded in the reality of what we can prove, the system becomes a self-working magic trick.

Wibbly-Wobbly, Timey-Wimey Stuff

The further out, the more time for something to disrupt it, the lower the confidence.

“But, Dave!, I need to tell people when to expect these deliverables!”

Great! As above, let’s start with what we can prove.

1. We can prove how much value the team added in the previous sprint(s)

And that’s it. That’s all we can prove. Anything that isn’t done yet is subject to someone getting sick, to a dependency not coming through, to a bug being more mysterious than anyone could have known, or to found work and emergencies preempting it. Making guesses toward which individual will pick up a work item doesn’t help us avoid any of that.

So, are story points useful for predicting when work might be done? Yes! But only as a prediction, and only as a relative measure of complexity and velocity compared against historical evidence of team throughput. If we’re using story points to determine how much to pre-assign to a given individual, we’re going to have a bad time rebalancing things halfway through the sprint.

When we do have concrete data that the team trends toward 50 story points per sprint, then we can give stakeholders a rough feel for when work will come in. Since we’re looking at the team, not at each individual, disruptions and variations tend to even out. If there are 128 story points ahead of Item X, then Item X is likely to come in after 128/50 = the third sprint from now.

But note, in my opinion, it’s still best to answer in terms of “Now, Next, or Later” for as long as possible. The further out, the more time for something to disrupt it, the lower the confidence. The good news is, when the disruption comes, it’s a blip on the board that most people won’t even notice. Everyone keeps working the same:

Free Dev: What’s the next thing the business needs that isn’t in progress yet?

The Board: It’s right there on the top, unassigned.

Free Dev: Great, I’ll get right on it.

Those who know me know that I really love gaming, and in particular, game design. A desire to build new worlds and deliver the feelings of wonder that I’ve experienced through play are why I got into tech in the first place! I’m very enthusiastic about creating or remixing mechanics, defining core engagement loops, and balancing systems to force players to make choices that matter.

While I’ve constructed a few tabletop campaigns, designed some “escape” rooms for the family, started my secret (LOL) portfolio of designs, and dabbled in Unity, I have not yet channelled my enthusiasm into really “doing the thing.” I’ve recently decided that I’d get further if I put some more structure and concrete goals around my study. In the spring, I intend to take a formal game design certification course through the University of Washington. In the meantime, I’ve been doing some reading.

The Art of Game Design: A Book of Lenses

I’m very much enjoying The Art of Game Design: A Book of Lenses. The introduction alone struck me. Schell mentions working as an entertainer and as a software engineer at IBM before circling back around to game design. Given that I also pursued entertainment first and software engineering second, this certainly reminded me that it’s never too late to chase one’s dreams =)

I’ve been capturing copious notes and my own reflections in a personal OneNote, and now that I have the blog back up, I intend to play catch-up on sharing some of that very soon.

Reality is Broken

I’m also reading Reality Is Broken: Why Games Make Us Better and How They Can Change the World. This has renewed my thinking around serious games and meaningful play. I plan on making some time to look more closely at Gamers for Good, Games for Change, and MSU’s Meaningful Play.

Level Up

To round out a set of three, I’ve also got my hands on Level Up! The Guide to Great Video Game Design. I find that where the Art of Game Design is guiding the way I look at concepts and elements of my ideas, Level Up! is full of tips for more hands-on, practical execution. I’m looking forward to making more concrete use of that guidance.

Putting it all together

Everything I’m highly skilled at today, I was absolutely terrible at when I started. Please, (don’t) look at my Game of Life or Lemonade Stand from CS101! Ramping up and adapting quickly to new challenges has been a superpower, and in fact, I believe the gaming I did while growing up helped me develop it.

It’s perhaps a bit silly that I’ll spontaneously leverage this ability when it comes to ever-changing tech stacks but forget about it when it comes to what I really want to do. Maybe I’ve let myself believe that game design is too far outside the box, but no more. There’s only one way to find out how far I can go and how much I’ll really love it once I’m more seriously into the weeds. I feel that the resources above are going to help me focus my energy toward that end.

Are you a more seasoned game designer? Are there other resources or exercises you’d recommend? What are the “new player” levels and achievements to get the ball rolling? Let me know in the comments or on LinkedIn!

After receiving questions and comments on this post, I’ve started a FAQ to clarify points where I wasn’t clear enough, and rethink points where I didn’t consider all the angles. Please let me know if you have additional feedback on this post.

Cashier to aisle three

Raise your hand if you’ve played the chaotic game of choosing a grocery store checkout line. See yourself and your competition carefully assessing the number of customers, the number of items, the speed at which each item is being scanned and bagged. Revel in the memory of arriving just in time to join a line with only one customer ahead of you. Recall your despair when that customer experienced an issue that took a long time to resolve. See your rival from a much longer line get out the door before your first item is scanned. All the while, other customers keep jumping between lines, and store management has difficulty tracking pressure on the overall system.

Next in line, please

Now think about your experiences at an airport or the DMV. While it’s rare to find someone who enjoys waiting in these lines, take a moment to consider the flow of work through the system. There’s the set of people waiting, and there’s the set of people ready to serve them. When an associate finishes with one visitor, they call for the next in line. Visitors don’t need to guess or monitor which line moves more efficiently. There’s just the one, and they’ll be called when it’s their turn. If the system is moving too slowly, management can put more people at the counter without shuffling or rebalancing the queue. When the pressure ebbs, people can move back to other work, again without shuffling or rebalancing the queue.

VIPs

When a frequent flier with the most elite status shows up and an airline wants to prioritize that person’s service, this causes little disruption to how work flows through the system (although it may disappoint those who are not so prioritized). Associates have been trained to call upon people from the prioritized queue first, and when it’s empty, the standard flow picks up right where it left off.

Continuous value streams

In my experience, when a product or service will ship in a continuous and iterative model, the business is best served by establishing the desired order of deliverables. Order, not priority. If the work is not high priority, then why are we doing it at all? (Thanks, Phil Japikse).

Having accomplished the ordering, the next step is to defer assigment until the last responsible moment, which is very often the moment when someone is actually available to execute it. Can you imagine being told, immediately upon arrival at the airport, which member of airline staff will check you in? What if that associate has a personal emergency and has to leave suddenly? Who serves you now, and more importantly, where do you fit into that person’s existing commitments? Instead, let teams continuously execute on the next value we wish to add to the system. Order the backlog such that the handful of “next right things” to deliver are right there on top, and let each team member pull the top item¹ as soon as they’re free to take on new work.

Across multiple development teams, I’ve found a number of advantages to this model. Aside from simplifying assignment and facilitating dynamic adjustment to those “next right things,” it inherently dismantles the thinking that Person 1 always does X and Person 2 always does Y. While there’s certainly a place for subject matter expertise, the darker side of that coin manifests as knowledge silos and single points of failure. If the five top value-adds for the business are all in the domain of X, and we only have one person doing X, then our system is screaming at us: “It’s time for the team to crosstrain!” Better, in my opinion, to get ahead of that fire drill by thinking in terms of collective ownership all along the way.

What do you think? Where has this model worked for you, and in what cases does it fall apart? Let me know in the comments or LinkedIn!

Lately, I’ve found myself looking into UWP development. Tonight I want to share a problem I ran into and a workaround that I found. I’ve created a repo that shows the step by step progression, and this blog post is meant to provide additional context around that.

Pre-emptive nitpickers note: for the sake of problem reduction, the code in this repo is not overly concerned with patterns, best practices, error checking, and so on

Some Background

In XAML, there’s the concept of a DataTemplate. This allows you to define what UI components or user controls are used when displaying certain objects inside of other content controls. A canonical example is a ListView.

Without Templates

By default, a ListView will just ToString() the object and put it on the screen. You can run this commit, or look at the screenshot below.

Notice that the ListView says “DataTemplates.RedThing” and “DataTemplates.BlueThing” (i.e., the System.Type of each item) instead of any meaningful representation of the instance properties.

DataTemplate, Binding

In this commit we create a ResourceDictionary¹ with a couple of DataTemplate definitions in it. We also merge that ResourceDictionary into the app resources in App.xaml, and create a ThingTemplateSelector to use one or the other based on the type of underlying Thing.

I’m going to be very careful not to say this looks good, but it proves the technique. We can now let our designers have all the fun polishing the DataTemplate itself, and we’re good to go! Right?

What Are Tests? Baby, Don’t Hurt Me. Don’t Hurt Me. No More!

I have to say that whatever positives there might be with UWP, the unit testing story is not one of them. The vast majority of issues we’ve seen so far have boiled down to testability.

The next commit highlights one such example where our app runs run fine, but our unit test application crashes at startup. The error isn’t what I’d call helpful.

… test discovery output, a la deployment of application, etc., omitted …

A user callback threw an exception. Check the exception stack and inner exception to determine the callback that failed.

========== Run test finished: 0 run (0:00:04.9484062) ==========

If you debug the test to check the stack and inner exception, both are null. You may now flip your tables.

Update 6/30: After some discussion about this post, I realized it could be even clearer. In the original commits, the unit test was a UITestMethod that tested the template selector itself, making it seem like the problem was limited to testing UI concerns. Now, our unit test does nothing other than Assert.IsTrue(true) in a TestMethod. Let us all agree that one should always be able to Assert.IsTrue(true) without one’s unit tests crashing.

The Workaround

Of course, I neither force quit nor flipped tables in real life. I took a breath and decided to push through the pain. What I found was that our test project would not crash if I removed our equivalent of <local:ThingTemplates /> from the App.xaml.

In the final commit, you can see how our custom DataTemplateSelector takes care of instantiating the dictionary it needs itself. One might argue this is actually cleaner anyway. I’m not currently going to argue that point either way. All I’m going to say is, “Now we can put DataTemplates in a ResourceDictionary and still keep our unit tests, too!” That makes me a much happier camper.

¹Note: The ResourceDictionary needs a code behind because we’re using x:Bind. This is a new alternative to Binding that lets you take advantage of static types. Leaving aside any debates on the merit of static types themselves, I feel that you might as well use ‘em if you got ‘em. I have not confirmed if the problems in the unit test project would also exist when using {Binding} and eliminating the code behind.

Today I realized I’ve overlooked a great feature of Hapi.js, so I figured I’d write a quick post about it.

What I Was Doing

For a server rendered view, I was doing this.

var jade = require('jade');
var fs = require('fs');
var template = jade.compile(fs.readFileSync(pathToTemplate));

// later on
return template(context);

This was fine for what I was doing at the time, since the rendered output was really part of a larger response, and was pretty much the only area in the system where such a thing was being done.

What I Was Doing Today

Instead of having some rendered output as part of a larger API response, I was creating a route that should actually return an HTML view. The above still works fine, but I could see a bit of friction coming in if this project has more views.

• I would want to set the Content-Type, either through Hapi’s reply.header or reply.type. This could get repetitive.
• The readFileSync and compile dance could get a bit repetitive or tedious.

What I Do Now

// During the server setup 
server.views({
  engines: { 
    html: require('jade'),
    path: __dirname + '/templates'
  }
});

// Later, in handlers
return reply.view('index', context);

Hapi takes care of reading, compiling, and applying the template as long as it’s in the configured path.

More information about what engines are supported and how to use other options can be found by reading up on Hapi’s vision plugin.

This is part of a series on good commits.

In this post, we’ll discuss atomicity and frequency. Remember that this series and the talk from which it came is a description of what has worked well for me, not a prescription of what will work well for you. It’s ok to do things differently.

Atomic Commits

By an atomic commit, I mean there’s just one reason for change included in the commit.

This Commit Is Not Atomic

Fix 4, 5, and 99

4 Adjust font sizes and colors
5 Changed the splash screen timeout from 1 sec to 5 sec
99 Implemented the new ruleset for discounts

4 and 5 are pretty small, so why not include them all in one commit? Here are a few reasons to think about.

1. The commit message is not concise. The actual change description comes in the extended details, not the summary.
2. None of these can be reverted without reverting the others¹.
3. None of these changes can be cherry-picked without the others².
4. This works against making frequent commits, which we’ll discuss shortly.

We can see some parallels with the debugging and troubleshooting concept of only turning one knob at a time. If you change three things and it gets better, you don’t know if it was one of them, a combination of two of them (which two?!), or all three working together. If all three are needed, then of course commit them as one cohesive unit. If not, consider keeping them separate³.

Why Not?

I’ve only heard two common arguments against this, and one is the same “habits” reason we’ve covered in almost every topic in the series.

The other is that the log gets much longer, and that’s certainly true. Whether this is a problem is another matter. I find that it’s often helpful to see a greater number of more granular commits in the log than to see a few gigantic ones. It’s more clear to me what changed when and why, especially since the message are more concise and specific. By the same token, it’s easier to search for a given commit.

There is one more option to keep in mind as you consider what works best for you. With some version control systems and branching strategies, you can have the best of both worlds. When the full history becomes irrelevant as a new feature reaches completion (i.e., in the future, we’ll only care that the feature was implemented, not about each step we took along the way), then you can squash the Work In Progress commits into a smaller set just before you merge them in.

Frequency

If you’re making atomic commits, you’re probably also committing frequently.

It’s like undo/redo, but
– with named states
– across files
– without loss of the undo stack when the IDE or system restarts
– and you can jump back and forth multiple points at a time

It’s like saving your game right before the boss fight

I’ll usually commit each time I make forward progress toward my goal, or whenever I’m about to make a significant change across multiple files. This might mean I get one more test passing, or it might mean I’ve created something that “works” but needs to be refactored. The value in the commit is that no matter how bad I break things during the next step, there is zero effort to put things right. If I find myself down a terrible path, I just reset to HEAD, and I’m safe at home.

Depending on your branching strategy, you might even push your commits frequently so that you can get early feedback (from your peers and/or a build server). Contrast this with waiting until work is done, when feedback will often be withheld or ignored “to avoid the cost of rework”⁴.

So how frequently?

It depends ;) I certainly don’t think of it in terms of time. I think of it more in terms of progress versus risk. Do you have more value than you had before? Do you want to protect it as a known good state to which you can time travel later? Are you about to experiment with a wild idea or undergo a large refactoring? Consider whether there’s value in giving yourself a checkpoint, especially if your VCS supports squashing it later if you realize later that you didn’t need it.

Why not?

A lot of people worry that this will cause destabilization.

This is a very valid concern if committing means that it affects the whole team instead of just you⁵. If your commits only live locally until you push them to the server, or if your pushed commits will be isolated in a topic branch, then you’re only affecting yourself.

If the commits (or check-ins) are into a shared mainline, then yes, frequently adding your half-baked work in progress can indeed break builds and impact your team. However, there are ways around breaking your team and benefits to earlier integration. You will have to integrate at some point, and delaying that will only make it harder at the end. Do a check-in dance. Find out earlier what’s going to break, and it’ll be easier to correct your course before getting too far down the wrong path⁶.

Thoughts?

Again, this is what has worked for me and my teams. It might not be best for everyone. It might not even be best for me…just the best I’ve found yet. Please do share the pros and cons of any alternatives that have worked well for you.

Me: Have you checked the closet?
5yo: No, I know it’s not there.
Me: But have you checked the closet?
5yo: Nooooo. I know it isn’t there.
Me: I hear you, but I want to know if you’ve actually looked.
(wife comes back with the missing shirt)
Me: Was it in the closet?
Wife: Yep.

As a result, we had a good discussion about the differences between knowing and thinking. It boiled down to these points.

1. You didn’t know. You thought. There’s a difference.
2. Not knowing is OK. It’s just one more thing you can learn.
3. Indeed, smart people don’t know everything. They know there’s more to learn.
4. Smart people consider helpful suggestions.
5. Being adamant that you know when you don’t really know can make problems harder to solve.
6. Just because someone says they know doesn’t mean that they know. Trust, but verify.
7. Smart people test what they think until they know.
8. Again, always, always remember that it’s ok to be wrong.

Verbosity Ahead. You could stop reading here and be fine.

Smart Doesn’t Mean You Know Everything

My 5yo is incredibly smart, and there’s no parental bias coloring that statement =) The problem is when her confidence transitions into a need to be the smartest person in the room. She often joins an ongoing conversation with, “Well, actually, it’s (fill in the blank).” Sometimes, she’s technically correct, but is missing the point. Other times, she has drawn an incorrect conclusion, but it’s amusingly logical based on the data that she has. Then, there are the times when she’s just making noise, trying to avoid looking like a person who doesn’t know something. As the old saying goes, it’s usually better to remain silent and appear ignorant than to speak up and remove all doubt.

This isn’t just my five year old. Many professionals (not excluding myself) do this because they confuse being vocal with looking smart, and they conflate being silent with appearing confused or uninformed. Sometimes we even chime in with something that isn’t even relevant, but it’s a data point that proves I Know Things^tm!

The other danger in thinking you know everything is that you lose the appetite to learn and improve. Knowing that you can’t even know how much you don’t know keeps you reading, keeps you experimenting, and keeps you pushing for even better ways to do things. Accepting that you’ve learned all there is to learn keeps you stuck doing whatever you’re doing. You may be the #1 ranked expert at frobbing the wizbangs, while everyone else has realized that wizbangs don’t even need to be frobbed anymore.

Smart People Accept Help

Just as my 5yo wouldn’t check the closet, many professionals refuse to accept a suggestion because they are sure that it’s wrong. As time passes and it becomes the last possible thing to try, it turns out to be the solution.

Consider that your team is probably trying to be helpful, so they probably have reason to believe that the idea is worth a shot. Why not consider what they’re offering? In the worst case, you spend ten seconds opening a closet. In the best case, you have your shirt.

My discussion with a 5yo put it a little something a little like this

Remember that we’re trying to be helpful, not trying to waste your time or patronize¹. Listen.

Do Not Present Guesses As Facts

We hear a lot about how interrupting a dev is the worst thing you can do. Let’s consider that there might be something worse. You could send them on a wild goose chase by presenting a guess as if it was a vetted root cause analysis.

Guess liberally, but be clear that you’re guessing. Do not present your guesses as data. – @druttka

What if we took my kid’s confident statement of knowing the shirt wasn’t there at face value? The shirt would still be “lost”. We professionals often present our own guesses as facts. I can think of many times that a bug report came with a full analysis of when it started happening, why it was happening now, and what we needed to change to fix it. After a few hours of tunnel vision², it was discovered that everything we “knew” was wrong. The timeline was off, the root cause didn’t even exist, and the proposed change was impossible because the code to change didn’t exist either.

The tunnel vision could have been avoided, and time saved, if the briefing had included a simple, humble, “I think”. Then the ideas would have been considered as ideas instead of taken as facts.

Test Assumptions

This brings us to the fundamental truth that we should always test assumptions. We all know the old saying that goes with this one, but we often forget to put it into practice. I’ve been trying to incorporate the following ideas into my thought process.

First, I try to remember to say “I think” if I just think³. If I do have evidence or reasoning, I try to remember to say “because (evidence or reasoning).” This helps the other person take what I’m saying for what it’s worth, not as hard truth. I also ask them to confirm what I’m saying, since I could be wrong.

Second, I add a silent “I think” to what others tell me unless or until they say “because (evidence or reasoning).” Even when they present evidence or reasoning, I try to hold off on accepting it as truth until I can confirm it in the same way I’d expect them to confirm anything I told them.

Third, I’m getting more liberal with asking “How do we know that?” It works well in combination with the point above because it prompts a person to voice their evidence or reasoning. Then we can vet it together. For example, if I sit down to pair with someone who has been investigating a problem, they might say, “We know this is being called.” I will usually be the kind of person who asks how we know it’s called⁴. If the response is “Because we see this text on a button changing,” I will be the kind of person who asks “Is that the only thing that changes the button text?” I’m aware that this could come across as annoying, but it’s often followed by “Oh…”. The realization is not always directly related, but sending the train of thought down the right track is still a positive outcome.

I guess the tl;dr on all of this is that problems can be more easily solved if we are careful to separate what we know from what we think. All the rest would flow naturally in a healthy, functioning team.