Software Engineering Wisdom
Thinking The Impact Of Technical Decision
The Options. Image Credits: Pixabay
Software Engineers nowadays are faced with decisions. Sometimes, it becomes exhausting. There are a lot of small technical decisions piled up during the project’s course. The fact that new software and frameworks can be born anytime does not help. Last year it was React, and this year is VueJS. Last 5 years it was Java EE Server Container, today it is lightweight docker container.
The Magpie Syndrome
How many times, you choose a new tools before you fully understand it because you just went back from conference or meetup? We seem to crave shiny new things even if it does not have any business value. This is called Magpie Syndrome. An irrational affinity to shiny objects, just like a Magpie bird.
As a software developer, we’re often oblivious that every technical decisions will have a business, development, or operational impact as time goes on. Developer tools advocates and marketers seems to know that. MongoDB adoption in past few years is one of example.
Competence Drives Decision Quality
Hubert Dreyfus had developed a skill acquisition model. Developers’ skill progression can be mapped to 5 levels: Novice, Advanced Beginner, Competent, Proficient, and Expert.
Hubert’s Model of Skill Acquisition
During developers’ career advancement, you’ll be at least on one of the level. The level goes with the experience. An expert should have been exposed to many domains to be able develop intuition. In term of developers, these are the people that has been using enough tools and programming languages on many problem domains to have opinion and foresee the impact of using said tools to the business. Due to the endemic of the magpie syndrome, less-experienced devs made careless and sloppy decisions when using technology.
Ever meet senior developers who you think so “backward” on insisting on some using ‘old’ tool?. There are two probabilities either they are really backward, which most of the case, not. Or because they have ‘intuition’ that using the tools you’re proposing will have major operational burden, for example. A senior developer will not just use whatever shiny out there. They’ll assess the the risk and impact of said tools to the business. This is what I meant with engineering wisdom.
Left Picture: How Junior Solves Problem, Right Picture: How Senior Solves Problem
Senior solves problem by exploring options and weight the pros and cons of that particular choice. Beginner tries thing and learns from errors.
Developing Wisdom
There are three stages of career when you’re working of developers: junior, mid, and senior. These are common carreer path for a developer.
During your first years, you’re given title somewhat similar to junior. You’re given a function or class to be solved and ticket to close. That’s it. You’re given the rules of always writing tests, and no test meant no code merge. At this stage you’re working as solution implementor. The autonomy is limited but you can be care less as long as you follow the rules, you’ll be fine. In Dreyfus model, this is the Novice portion of your skill acquisition phase. The wisdom you have here is very limited as you’re just following the rules.
After you’re working for some time you may still be junior, but you have been working enough time to give a little bit of judgement for general situations and sometimes you want to test the rule. This is the advanced beginner phase. Your curiousity drives you to try new things. The engineering wisdom on you is started to be shaped by learning from errors and failures.
The Myth of Lone Wolf 10× Developer
And then you got promoted because your bosses and peers says now you have very good conceptual understanding of some of the programming problems. You can also give valuable input on the solution made when new problems arise. However, you’re still does not understand the big picture, so they said. Don’t be disheartened. You’re good as you’re now competent. You’re able to apply the wisdom to actively decide the solutions of new problems by drawing line from the experiences.
You moved to a new job, as you feel that you’re stuck on your old job. You think that you know the big picture now. You know what is the purpose of using patterns and best practises, but you feel that they ignore you when you have opinion of what matters most. On the new job you’re given title senior. Congratulations, you are proficient. You’re now able to explore, weigh options, and decide which one works for your particular problem. You may be assigned to mentor one or two developers.
Suddenly you got a pay rise and new title. Your CTO told you that you’re now an expert. An asset to the company as it seems that you can automatically and quickly decide the architecture and pinpoint the future potential problem. You have intuition on which direction you’ll take without too much exploring and trying. This is where your wisdom is the tacit knowledge you’ve been accumulated. The development and executive team will ask you for some advice.
Mentoring And Pairing
The most effective way of developing your skill and wisdom is to find a good mentor with matching skill level. This is what Dan North calls Dreyfus Squared. We take the four level to create a table of pairs.
Dreyfus Squared
The table above is an example of the matrix. Some pairs work well, some does not. An advanced beginner mentoring another advanced beginner will result on mayhem as they constantly testing the rules. However, a compentent with a novice will result of guidance to the beginner as well as lesson learned. An proficient mentored by expert will build confidence. The rule of the thumb is 1 or 2 levels distance gap is good.
Decision, Decision
You’ll see the main theme of the career progression or skill acqusition above. It’s all about autonomy on making decision. In software engineering, we have a lot of thing to decide. Some examples are:
- Are we going to do automated or manual build?
- Do we test manually or automated?
- Monolith or microservices architecture?
- Synchronous or asynchronous I/O model?
- PostgreSQL, MongoDB, or Cassandra?
And many more. We need to be careful when deciding here as sometimes the marketing of some frameworks or practises is very strongly influence our decision. Some practise that is better from some situations but not the other.
The first question you need to ask is WHY. This question is so simple powerful, it’s easy to overlook. When you heard someone preaching new practises, always ask the why.
Jonah Lehrer on his book : How We Decide comes with the rubric of decision making:
- If the problem is novel (to you), instinct alone will not suffice. Our instinct can often help on solving the subproblems but our rational mind should integrate them.
- If the problem is routine and can be fully characterized with a few well defined variables (or simplified in this way because the decision’s consequence matters little), let reason carefully assess and analyze the options.
- If the problem is routine, but cannot be simplified to a few well defined variables, use your rationale mind to identify what information is and is not available, but let the emotional part of your mind process and analyze it. The instinct resulting from this is your mind’s expert judgement.
In connection with the Dreyfus model, we acquire instinct when reaching expertise. That’s why usually the most senior developer or architect is tasked to explore new problem. Or, in my word, to do software reconnaissance. This is due to their ability to decide quickly and effectively the choices of solving novel problems. They are more prepared like a special force.
Conclusion
Engineering wisdom is the ability to weigh and decide the best methodologies and solutions for particular problem. The more wisdom you gather the likelihood of finding effective solution is higher. This growth can only be executed in an organisation with growth mindset and learning culture.
