Rampant CodeCraft

I used to be Projects over People. The work was all that mattered. Woe to those who came between me and delivering the highest value to the user via the cleverest user interface and highest quality of code I knew how to produce. Reminds me of a Robert E. Howard quote, but we will get to that in a bit.
Why? Considering the amount of time I’ve thought about it, you would think I would have a better answer. But alas all I have are these: The illusion of a meritocracy maybe? A meritocracy that, because of it’s existence in our schooling, I thought existed everywhere. If I wrote the best software, everyone would (somehow) know and I would be respected. I know you just chuckled. I didn’t hear it, but I still know you did. That’s ok, it was naive of me and warrants a good laugh.
Maybe it was a solution to a flaw I saw in using clients to measure project success: it’s not very scientific. The phrase “It’s not what you know but who you know” applies here and reeks of corruption that I despise. Which brings up another Howard quote:

“Civilized men are more discourteous than savages because they know they can be impolite without having their skulls split, as a general thing.”
               –Robert E. Howard (The Tower of the Elephant)

I saw clients evaluating the success of a project as a fail because at the time I thought it mattered that they understand the internal workings, you know, all the crap I had to go through and all the ‘brilliance’ I showed on the inside of the app. They can’t understand it, and it doesn’t matter if they do. I also didn’t see them as capable of giving honest feedback because of the naturally adversarial relationship that occurs between one paying for work and one delivering the work. But the relationship between these two need not be adversarial.
Maybe the true reason: it is in my nature, it’s in your nature, it’s in our nature to strive to be the best and people are competition, or just get in the way. I need a machine, an IDE and a compiler to make users lives better. Go away and let me get to saving the world. ;)
This Projects-over-People philosophy almost cost me one of the most valued relationships I have now. It was my manager at the time that made me realize Projects will come and go, but the relationships you make with the People you meet may last a lifetime. He is a gentle man and his gentleness is something I appreciated while working with him and began to emulate. So after the longest and most challenging project I’ve ever had in my short career, one that I alone followed to it’s full end, sacrificing everything else in my life, protecting it and defending it, I flipped. I became People over Projects. On my next project I nurtured the relationships with the client-users, managers and developers first and foremost, to the detriment of the codebase.
That was some time ago and the codebase that resulted has still not shipped. For my sake, the sake of the people that would have to use it, and the sake of the developers that would have to maintain it, I hope it never does. My change in philosophy was not the entire reason the project failed nor was it the determining factor, but I would be lying to myself to say it didn’t contribute.
What I see now is that one or the other is an insufficient solution. If you deliver a great app but all you made were enemies, you have failed because very quickly you will find there is no one that wants to work with you or wants you working for them. If you make a bunch of new friends but didn’t enforce coding practices you know are necessary to ensure the quality of the application and therefore develop a non-maintainable application, you have also failed because you didn’t do your job which was to deliver a functional application that can afford change. Both are needed.
A successful career is built by delivering a great application to the users, while leading your team, not by instruction or condescension but by example.  Prove to the team why the methodologies you use are worth the extra work. If you can’t prove that your methodologies are worth it, then you need to reconsider your chosen methodologies or practice to increase your skill in using them. This builds trust between you and the developers you work with in a positive way. An adversarial relationship with the client can be avoided with an agile project approach that delivers an application in pieces that is a good fit for the user as the target moves into focus. The client gets a front row seat to the addition and modification of functionality and trust and confidence is built as they watch you encounter and overcome obstacles along the way (especially the obstacles the client creates themselves as they refine the design).
Being Projects over People ignores the fact that no large software system can be built by one person in a short amount of time, which is not practical. Being People over Projects requires one to tame their individual desires of conquest for the perceived betterment of the individual egos in the group, which is not natural and benefits no one. 

“Let me live deep while I live; let me know the rich juices of red meat and stinging wine my palate, the hot embrace of arms, the mad exultation of battle when the blue blades flame and crimson, and I am content.

Let teachers and philosophers brood over questions of reality and illusion. I know this: if life is illusion, then I am no less an illusion, and being thus, the illusion is real to me. I live, I burn with life, I love, I slay, and am content.”
               –Robert E. Howard (Queen of the Black Coast)

The above quote, I thought, supported my initial Projects-over-People philosophy but I’ve found in practice that living your professional life to the fullest cannot be sustained without a balance of conquest and cooperation. The quote “It’s not what you know but who you know” is emphasizing the reality that without people, one cannot become successful. For the first part you must be great at what you do. But being only this will leave you alone to appreciate yourself. For the second part you must include, cooperate, and treat kindly the people you work and with and for. But being only this will provide you with the illusion of success that will soon evaporate once it is seen that you are not as good at your job as the illusion has led people to believe. Done together you practice and hone your craft to deliver the greatest applications you can while cultivating relationships with people that will spread your software’s success, and appreciation for your skill as well what a pleasure that it was to work with you.
Be good at what you do, and be good to the people around you so that you may have more opportunities to be successful.

I was sent this this morning:
http://zedshaw.com/essays/master_and_expert.html

It’s been chewing at me all day. I don’t know that I agree with this.

I think the analogy is wrong in part. In the analogy, the master is in control of what he knows and practices. But I still say that he needs to check his form as he practices to make sure bad form doesn’t become habit. This is analogous to ‘unit-testing’ himself, not the extraneous movement during performance of a young learner. His code, will always and forever be his own. He need not protect himself from other’s modifications of his form, only his own degradation. But even here unit testing is required.

Comparing the young learners unit test writing to the final performance is not a valid pairing. Unit tests are written when practicing and rehearsing. (Note: practicing is writing code at home, and rehearsing is writing code at work) The final performance, the operation of the code in production, doesn’t involve unit tests. The creation of code involves unit tests. You can’t compare the creation of unit tests during the writing of code to the extraneous operations in a young learner’s final performance. Those aren’t the same thing.

I fear his point is muddled in his own head. The practice of a martial art is analogous to the programmer writing code. The performance of the martial-art in competition (the result of practice, refinement and quality checking) is analogous to the performance of the code in production (the result of writing code, refactoring and unit testing).

He is comparing unequal things, and therefore invalidates his argument. His point is, I assume, that a master accomplishes a task (victory in combat, or automation of a business task) with the least effort possible, but the details are mismatched.

In software we make analogies for –everything-. Most times because we have to. People we are explaining things to won’t understand the context of the deeply technical or specialized things we are describing so we create an analogy to something they do have context in. But we need to be careful that our analogies are valid and we are comparing two equal things.

Was talking to a colleague, friend and mentor Troy Bourgeois and in the course of the conversation he said something that struck me. He said writing code comments is just another form of repeating yourself.

I’ve been thinking about this all night. Just like copied code is a symptom of not getting the abstraction right, commenting is a symptom of not getting the code right. Code is supposed to communicate its intent. If it doesn't, then you comment in order to do so.

The real question here is why I feel like I have to add a disclaimer every time I say commenting is a smell? Whether it’s in front of college students, professionals or colleagues. I truly believe that the desire to write a comment is a sign that your code doesn’t communicate effectively, but I guess I fear that people will not strive to write code that doesn’t need comments and rather just omit them.

As Uncle Bob said:
It's not about _omitting_ comments, it's about making them unnecessary.

[…]

Engineering vs. Science

With only about 40 percent of software developers holding computer science degrees and practically none holding degrees in software engineering, we shouldn’t be surprised to find people confused about the difference between software engineering and computer science. […] Scientists learn what is true, how to test hypotheses, and how to extend knowledge in their field. Engineers learn what is true, what is useful, and how to apply well-understood knowledge to solve practical problems. Scientists must keep up to date with the latest research. Engineers must be familiar with knowledge that has already proven to be reliable and effective. […] An undergraduate science education prepares students to continue their studies. An undergraduate engineering education prepares students to enter the workforce immediately after completing their studies.

[…]

This puts computer science students into a technological no-man’s land. They are called scientists, but they are performing job functions that are traditionally performed by engineers, without the benefit of engineering training. The effect is roughly the same as it would be if you assigned a physics Ph.D. to design electrical equipment for commercial sale. […] We would expect the equipment designed by the physics Ph.D. to work, but perhaps to lack some of the robustness that would make it usable or safe outside a laboratory.

[…]

[workers educated as computer scientists] focus narrowly and deeply on minor considerations to the exclusion of other factors that are more important. They might spend two days hand-tuning a sorting algorithm instead of two hours using a code library or copying a suitable algorithm from a book.

[…]

The lack of professional development isn’t solely the software developer’s failure. The software world has become a victim of its own success. The software job market has been growing faster than the educational infrastructure needed to support it, and so more than half the people holding software development jobs have been educated in subjects other than software.

[…]

When a building is designed, the construction materials must suit the building’s purpose. I can build a large equipment shed to store farming vehicles from thin, uninsulated sheet metal. I wouldn’t build a house the same way. But even though the house is sturdier and warmer, we wouldn’t refer to the shed as being inferior to the house in any way. The shed has been designed appropriately for its intended purpose. If it had been built the same way as a house, we might even criticize it for being “over-engineered”—a judgment that the designers wasted resources in building it and that it actually isn’t well engineered.

[…]

Today’s pervasive reliance on code-and-fix development—and the cost and schedule overruns that go with it—is not the result of a software engineering calculation, but of too little education and training in software engineering practices.

“Developing the user interface of a professional software application is not easy. It can be a murky blend of data, interaction design, visual design, connectivity, multithreading, security, internationalization, validation, unit testing, and a touch of voodoo. Considering that a user interface exposes the underlying system and must satisfy the unpredictable stylistic requirements of its users, it can be the most volatile area of many applications.”

The focus of the design iteration is the document. The product of the design iteration is this document. The software is defined by this document. The estimation is created from this document. The sale is secured with this document. Client trust is established by this document. Test cases are made from this document. Who we are as a company is represented by this document.

All wrong.

The focus of the design iteration is the design. The product of the design iteration is the design. The software is defined by this design. The estimation is created from this design. The sale is secured with this design. Client trust is established by this design. Test cases are made from this design. Who we are as a company is represented by this design.

How do we communicate this design to the client? What does the client need? Let's see, the client needs to be able consume this design in its entirety and, if complex, the order in which components of the design are conveyed may need to be structured so as to build ontop of each other to communicate a larger concept, implementation or idea. A document sounds like a good candidate for this. It can be read from beginning to end so consumption of the entirety is straight forward. The order of consumption can be predicted and so the order of component description can be arranged as desired. The document is not the design. It is a representation of the design; an expression of the design selected for it's suitability to the intended audience.

How do we communicate this design to the development team? A document? Turns out that's not working out so well. In this case a document is not really the most suitable form for the audience. We have found in the past that if the design's primary medium is a document it is difficult to maintain, to modify, to update, to change, to track changes, to work on concurrently etc.

So we need to be able to express the design in document form for the client. But this form isn't suitable for the development staff. We need a primary medium for this design to exist that it gives the management, development and QA staff what they need while also being exportable to a document for the client.

We are going to try a Wiki. This should give the development, QA, and management staff the flexible, easily editable format they need while we can programmatically flatten the Wiki into a specification document for the client. Hopefully later we can get out of this upfront design and design things in the wiki right before we need to develop it. Our first step to getting there is to move from a big upfront design (BUFD) to a medium upfront design. The document format is only required for upfront designs. If the client is working with us hand in hand, then they don't ever have to read the design from end to end, they only need to focus on a single feature implementation: one wiki page.

Some advantages of the wiki are:

• When defining the design, the developers aren't in a document, they aren't evaluating the size of the document, they aren't comparing document sections before or after it for page length or content. Their sense of done is soley based on the detail they have inscribed within the wiki page they are focused on.
  
• Wiki pages, images and other resources are versioned, and are versioned better than tracking changes in a document provides.
  
• There is no sense of beginning and end in the wiki. Wiki pages that are related to the page the developer is currently either designing, or constructing software from, is linked to from that page. This puts all the information the developer needs within easy reach.
  
• During development the development, QA and management staff are focused on portions of the document and not the entirety, when the client reads the design in document format they will be able to consume the entirety from beginning to end because the document version provides this.
  
• Wikis are easy for multiple people to edit at the same time. no passing around a document or merging document sections together.
  
• In digital format they are very easy to navigate.
  
• In document format the embeded links to other document sections is inherent and doesn't require one manually entering in the links.

Ability to QA a system
This facilitates the developer being able to catch and resolve issues before they become problems.

Write Effective Unit Tests
There is nothing I know of that enforces good Object Oriented design more potently than Unit Tests. It's not about a specific unit testing program because being able to create unit tests is a fundamental skill. It is a skill that translates across all programs and methodologies. Creating good unit tests is a science, and like any science can be perfected with enough experience and practice. The first thing people find when starting to write unit tests for the first time is that their code is not unit testable. They thought they were object oriented design masters and find that they’re doing it wrong. Unit tests flush out this misconception and force good object oriented design. Raw experience writing unit tests is required before quality code coverage is achieved. I would like to see students write unit tests in the major software project courses and give them this experience, as well as force the code into good OO design.

For more information:
http://en.wikipedia.org/wiki/Unit_test
http://en.wikipedia.org/wiki/Test-driven_development (TDD)

Refactoring
If you work your entire career creating new software for clients that don’t change requirements then you are the one, because no one else gets to. You will have to maintain a code base or adopt a code base or have to modify a code base. Refactoring skill is required; otherwise you’re just fumbling around hoping your changes work out, relying solely on your QA department to catch problems before the client does.
Refactoring is key to the new agile implementations, but again, more importantly the ability to transform existing code into unit testable, object oriented, patterned code is key to a successful career as well as being a chief tenant of any agile implementation. Being able to architect a great pattern-rich, mature solution and code it from the ground up is great. It’s still not simple to do, but when done right is a beautiful thing. More likely than not though you aren’t given that opportunity. You are given an existing code base and are asked to make changes. This can be in the form of a rotten code base written years ago, to an existing code base that is part of an active scrum sprint whose requirements were changed at the end of the last sprint. Either way being a code alchemist, being able to transform existing good or rotten code into the appropriate code is a huge skill. It’s also a fundamental skill that permeates through languages, frameworks and platforms. If this skill was focused on you would see the code that people adopt improve in quality, not just being retro fit in any way possible for the immediate need.

For more information:
Refactoring, Martin Fower, 1999.

Design Patterns
So many fields have achieved greatness today because they build upon the lessons learned in the past. For the vast majority of the Computer Science and software development field we restart from scratch every time. There is little growth. We have many of the same problems today that people in the 70s and 80s had. Design patterns facilitate mature initial design and simple/reliable extendibility later among so many other things. This is a key part to all the new agile implementations, but more importantly this is key to growth of the field, and growth in the field. For those looking for shoulders to stand on, here they are. These patterns reflect over 30 years of design refinement and extension. Software developers are foolish not to implement them. That is assuming they know Design Patterns for software development exist. That’s one of the problems, many are not aware of design patterns. Identifying patterns in use in code they are to maintain, identifying when to apply patterns to a new design, and identifying which pattern to refactor to are all difficult skills to master.

For more information:
Head First Design Patterns, Freeman, Sierra, Bates, 2004.
http://www.dofactory.com/Patterns/Patterns.aspx

Performance Testing
If there is a bottle neck, it has to be found before it can be fixed. There is a skill involved in finding the problem and a whole different skill set involved in improving performance.