Duct Tape Developers? I'm done with this guy
Posted by Phillip Jackson in Coding, Craftsmanship, Duct Tape Developer, Joel Spolsky on 23 October 2009
“…but it seems to me like a lot of the Object Oriented Design principles you're hearing lately from people like Robert Martin and Kent Beck and so forth have gone off the deep end into architecture for architecture's sake. It doesn't seem like you could actually get any code written if you're spending all your time writing 8,000,000 unit tests, and every single dinky little class that you need to split a URL into four parts becomes an engineering project worthy of making a bridge, where you spend six months defining 1000 little interfaces. They've just gone off the deep end, and I don't think these people write very much code if they're coming up with these principles, to be honest, it doesn't even make sense.”
“One of the SOLID principles, and I'm totally butchering this, but…”
Interaction Design
“Developing the user interface of a professional software application is not easy. It can be a murky blend of data, interaction design, visual design, connectivity, multithreading, security, internationalization, validation, unit testing, and a touch of voodoo. Considering that a user interface exposes the underlying system and must satisfy the unpredictable stylistic requirements of its users, it can be the most volatile area of many applications.”
Divergence
March 06, 2007 Phillip Atkinson sent this link:
http://www.joelonsoftware.com/articles/APIWar.html
From the article:
I first heard about this from one of the developers of the hit game SimCity, who told me that there was a critical bug in his application: it used memory right after freeing it, a major no-no that happened to work OK on DOS but would not work under Windows where memory that is freed is likely to be snatched up by another running application right away. The testers on the Windows team were going through various popular applications, testing them to make sure they worked OK, but SimCity kept crashing. They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it.
Wow.
As for the rest. I’m all about delivering products to the user as fast as possible. Period. Even in managed langs it’s not fast enough for me. My dream is to envision a UI and then once the UI is constructed be able to wire up the functionality to it without effort. I’m a visually oriented guy and am not as concerned with how cool it would be to create my own X, Y, Z objects to handle A-F data as I am concerned with how cool it is to provide tools to the client(often myself) that can make their life better in a near immediate timeframe.
If the world’s core development population views managed languages like I view VB6, then I’m going to have to just suck it up. While they are flexing their muscle writing that app in C, C++ I will have whipped out several and learned a significant amount about the interface between Developers and Users while the C guys learned a little about the interface between Computers and Developers but not much more than they already knew. I see the world changing and it will very soon be accelerating even more. Once that last person, that didn’t grow up around computers, retires then the demand for software that does everything needed and does it right now will grow even stronger than it already is.
For years I’ve balked at business application developers calling themselves Software Engineers. I’ve done so because as this industry matures there will be a divergence in what is now a single group of professionals. I see Software Engineers emerging as those that are focused solely on writing software, but on the software that exists close to the metal with knowledge of: compilers, linkers, machine lang, memory management, binary arithmetic, semaphores, dining philosophers, red-black trees, Turing machines, non-deterministic finite state machines, framework creation. The diverging vector are the Software Developers (or Application Developers) who are focused on delivering functionality to the masses. Yeah yeah, it’s a computer… so what… watch what I can make it do FOR YOU. They exist in an area removed from the metal; in frameworks and with tools that facilitate rapid construction and solid functionality. They are about fulfilling the user’s needs and transforming a computer into a useful device. The ultimate device.
Currently there are those that can live in both realms. They fully understand compiling, linking, NAND gates and voltages and write assembly code regularly while also being able to construct a J2EE app with EJB, and do it well. There are not many. The university view is teach them to be Software Engineers and they can become Software Developers if they so choose because the learning curve on becoming a Software Developer is flattened by the ease of use of the quality of tools and frameworks that facilitate rapid development. I can’t argue with that now, how else are we going to create Software Engineers? But who will take on the responsibility of training Software Developers? The employers? How is that working for you?
I’d like to declare that I’m proud to be a Software Developer. But there is a part of me that looks down my nose that those developers that don’t know the fundamental Computer Science stuff. As if that makes me more of a man. I learned C at LSU, C++ at SELU and have coded a fair amount in them. Do I miss it? Not even a little bit. No one misses STDLIB, do they? And for me they are in direct opposition to my dream of rapid development.
I’d characterize this stage of the software industry as the Wild West. It’s after the pioneers have settled, but before cities have emerged. Though with my employer choice I was careful not to be the Town Mayor shaking his fist as the cowboys ride through town, I’m also not the cowboy that does whatever they please because they can. I’d like to think of myself in between. No home to speak of, but has a strong sense of what is best for the current town, country and most importantly the people involved, with the will to enforce that belief.
DOS to Windows was like the pioneer’s log cabins and mud huts to wood plank construction. The new shift from Windows XP and WinForms to Vista and WPF is less dramatic on the outside, but is a required part of the inevitable future growth. As the divergence becomes greater, things like the Vista Bridge will be standard fair as insulation from the realm of the Software Engineer will become an accepted reality.
Back to the point. So MS is dumping backwards compatibility in favor of a shift forward? So what. Since when is it new that we have to learn something new. Take something far enough and it will stagnate, other industries would stay stable instead. Do we crave chaos? Maybe so, or maybe it’s because of the phase of growth our industry is in. This shifting foundation is the world we live in, we are Software Developers, we are beholden to the Software Engineers on the small scale. In the grand scheme though, Software Engineers are reacting to the needs we express.
Wagging the Dog
Posted by Phillip Jackson in Documentation, System Design, TFS, Wiki on 11 December 2008
The focus of the design iteration is the document. The product of the design iteration is this document. The software is defined by this document. The estimation is created from this document. The sale is secured with this document. Client trust is established by this document. Test cases are made from this document. Who we are as a company is represented by this document.
All wrong.
The focus of the design iteration is the design. The product of the design iteration is the design. The software is defined by this design. The estimation is created from this design. The sale is secured with this design. Client trust is established by this design. Test cases are made from this design. Who we are as a company is represented by this design.
How do we communicate this design to the client? What does the client need? Let's see, the client needs to be able consume this design in its entirety and, if complex, the order in which components of the design are conveyed may need to be structured so as to build ontop of each other to communicate a larger concept, implementation or idea. A document sounds like a good candidate for this. It can be read from beginning to end so consumption of the entirety is straight forward. The order of consumption can be predicted and so the order of component description can be arranged as desired. The document is not the design. It is a representation of the design; an expression of the design selected for it's suitability to the intended audience.
How do we communicate this design to the development team? A document? Turns out that's not working out so well. In this case a document is not really the most suitable form for the audience. We have found in the past that if the design's primary medium is a document it is difficult to maintain, to modify, to update, to change, to track changes, to work on concurrently etc.
So we need to be able to express the design in document form for the client. But this form isn't suitable for the development staff. We need a primary medium for this design to exist that it gives the management, development and QA staff what they need while also being exportable to a document for the client.
We are going to try a Wiki. This should give the development, QA, and management staff the flexible, easily editable format they need while we can programmatically flatten the Wiki into a specification document for the client. Hopefully later we can get out of this upfront design and design things in the wiki right before we need to develop it. Our first step to getting there is to move from a big upfront design (BUFD) to a medium upfront design. The document format is only required for upfront designs. If the client is working with us hand in hand, then they don't ever have to read the design from end to end, they only need to focus on a single feature implementation: one wiki page.
Some advantages of the wiki are:
TFS wants to be the source for the design, but the UI is really difficult to interact with, and the navigation is worse. It is not ready to be used this way yet, so we will use the wiki until TFS is ready.
Event: Requirements Documentation
My company produces software that doesn't break. Our entire reputation is based on that. But behind that software is code that even the original authors have real trouble modifying. Why is that? All software produced everywhere is assigned a version number. Wikipedia has 4000 words on the subject of Software Versioning. Versioning is inherent to software. We know that software will be updated, modified, changed. So this begs the question: If we know that software will need to be modified, then why have we been setting ourselves up to fail by not writing code that can be modified? We've relied on something characteristic of software production for small business: there is no version 2. Problem now is, this is no longer our niche in the market. We have grown and produce software for large corporations now.
We are in the early stages of an event here. Parts of this are changes in the fundamental way my company authors code. Gone are the days of "Git-er-Done however you need cuz the client can't afford a version 2". Now is the time of "Why did this maintenance modification cost more than the original construction?". I could go on, but I mention this only to frame a point. Our move to code creation that is able to afford change is not this singularly great idea for a problem that exists in the development phase. The new concepts and skills can be applied to everything we do here.
In the development phase we produce what the client requests: software that gets a job done and doesn't break. For the past however long, that is all that was required. Our search for efficiency, combined with a track record of software not needing an update because it doesn't break and clients not requesting a second version has resulted in us gaining efficiency by cutting out all real considerations for future extension or modification. Our ability to do this well has resulted in larger clients with larger needs. And now when their large software solutions need an update; when version 2 is requested, we find ourselves in real trouble. This situation not only exists in our development phases of projects. It also exists in the requirements and specification phases.
In the design phase we produce what the client requests: a document that defines the intent, appearance and functionality of the software to be produced. But after development based on that document is over; when the client requests a version 2 and we need to refer back to the document to understand version 1, we find ourselves in real trouble. Why? Because we don't document the changes made to the product during development. We can't keep up, and we have never had a real need to, so we don't. Our specification documents have just enough detail to satisfy the client. The document leaves so much of the technical detail up to the developer coding, that within a matter of weeks what is delivered to QA has diverged from the specification so far that the document is useless and the QA assigned needs the new specification communicated to them directly from the developer before they can begin testing.
What is needed is method of documenting requirements that can be so easily modified that the developers actually do update the documentation as the application is refined, or requirements change or the application design diverges from the original for any reason.
They're Doing it Wrong
Firefighters – Are called in when a project spins into a runaway or fires rage out of control in Production. They are highly skilled, very driven and goal oriented. They are there to fix a problem and do so in short order. They don't answer to you they answer to the stakeholder or your boss's boss's boss, and therefore they often care little about the resulting code. Much like a real firefighter will destroy your house with the fire hose to put out the fire in the kitchen. But hey, it’s not on fire anymore, right.
Pitbulls – Can range from non-technical to very technical. The determining criterion is their incessant need to be an ass. They almost never admit when they are wrong, especially in the face of overwhelming evidence. They are agitators used to force a stakeholder’s desires on the project. They are passive aggressive and Roosters. If a pitbull is the product owner the project will fail. The only way to handle a pit is to not step in the ring, but rather deal directly with the stakeholder holding the leash. They are unhappy about something or feel powerless, that is why they aquired a pitbull. Resolve that problem and the pitbull problem will resolve itself.
Architects – Talk a lot. No really A LOT. The rule is if you can get an architect to stop talking, then they aren’t really an architect. Be wary of the ones with charisma and technical skill. They are very dangerous. Where architects tread, ivory grows: this is their job. Regardless of any other consideration get to know your architect. You can learn a lot from just sitting and listening, just don't make them 100% of your information base. It is when an Architect stops practicing the craft of code writing that they become dangerous.
Skillz
Ability to QA a system
This facilitates the developer being able to catch and resolve issues before they become problems.
Write Effective Unit Tests
There is nothing I know of that enforces good Object Oriented design more potently than Unit Tests. It's not about a specific unit testing program because being able to create unit tests is a fundamental skill. It is a skill that translates across all programs and methodologies. Creating good unit tests is a science, and like any science can be perfected with enough experience and practice. The first thing people find when starting to write unit tests for the first time is that their code is not unit testable. They thought they were object oriented design masters and find that they’re doing it wrong. Unit tests flush out this misconception and force good object oriented design. Raw experience writing unit tests is required before quality code coverage is achieved. I would like to see students write unit tests in the major software project courses and give them this experience, as well as force the code into good OO design.
For more information:
http://en.wikipedia.org/wiki/Unit_test
http://en.wikipedia.org/wiki/Test-driven_development (TDD)
Refactoring
If you work your entire career creating new software for clients that don’t change requirements then you are the one, because no one else gets to. You will have to maintain a code base or adopt a code base or have to modify a code base. Refactoring skill is required; otherwise you’re just fumbling around hoping your changes work out, relying solely on your QA department to catch problems before the client does.
Refactoring is key to the new agile implementations, but again, more importantly the ability to transform existing code into unit testable, object oriented, patterned code is key to a successful career as well as being a chief tenant of any agile implementation. Being able to architect a great pattern-rich, mature solution and code it from the ground up is great. It’s still not simple to do, but when done right is a beautiful thing. More likely than not though you aren’t given that opportunity. You are given an existing code base and are asked to make changes. This can be in the form of a rotten code base written years ago, to an existing code base that is part of an active scrum sprint whose requirements were changed at the end of the last sprint. Either way being a code alchemist, being able to transform existing good or rotten code into the appropriate code is a huge skill. It’s also a fundamental skill that permeates through languages, frameworks and platforms. If this skill was focused on you would see the code that people adopt improve in quality, not just being retro fit in any way possible for the immediate need.
For more information:
Refactoring, Martin Fower, 1999.
Design Patterns
So many fields have achieved greatness today because they build upon the lessons learned in the past. For the vast majority of the Computer Science and software development field we restart from scratch every time. There is little growth. We have many of the same problems today that people in the 70s and 80s had. Design patterns facilitate mature initial design and simple/reliable extendibility later among so many other things. This is a key part to all the new agile implementations, but more importantly this is key to growth of the field, and growth in the field. For those looking for shoulders to stand on, here they are. These patterns reflect over 30 years of design refinement and extension. Software developers are foolish not to implement them. That is assuming they know Design Patterns for software development exist. That’s one of the problems, many are not aware of design patterns. Identifying patterns in use in code they are to maintain, identifying when to apply patterns to a new design, and identifying which pattern to refactor to are all difficult skills to master.
For more information:
Head First Design Patterns, Freeman, Sierra, Bates, 2004.
http://www.dofactory.com/Patterns/Patterns.aspx
Performance Testing
If there is a bottle neck, it has to be found before it can be fixed. There is a skill involved in finding the problem and a whole different skill set involved in improving performance.