How To Be A Deep Thinker

Programming over the years

Yes, it still is the most popular square peg round hole on the block. The most interesting element is that – yes agile programming is both evolutionary and emergent. Evolution and emergence are processes and properties where the results are arrived at in a way that straight foresight never could have anticipated. But ironically the most adept agile developers are those who were “forced” to learn non-agile product development techniques. This should make sense at the end of the day, evolution and emergence only produce good results when they are building on other good practices. Unless you carefully analyze what those practices are you will never realize them.How many agile/extreme programmers have you come in contact with that have absolutely no concept about their problem domain. I’m not talking about the problem domain realized by “domain experts” in other fields. I mean the problem domain they are supposed to be experts in. We’ve all encountered it. People who are truly bright but have a little understanding of programming technique. Abstraction and simplification are lost on them. But convenient extreme mantras are held close to their heart. They love the built-in protection of extreme tenements.

Without access to a customer 24-7 how could they know what a customer wants? Anticipating needs is unimportant – code for todays needs. Long term scheduling is unimportant and a waste of time. All of these concepts work if you’re an experienced developer. Not because they are correct, but because they have become automatic to an almost sub-conscious degree. Their code auto magically is written to the tune of design patterns, simplicity, flexibility, and automatic recognition of common long-term needs. But the new-comers fail miserably. They end up building for today and generating a spaghetti mass that is impossible to maintain.

Many old-timers had a hard time cutting their teeth on design. And with design as with anything else it is important not to “think too much”. The point of diminishing returns. When the level of detail involved to analyze the problem eclipses the level of detail of actually implementing the correct solution you have gone too far. The correct solution is the key here. If a function is supposed to test for whether or not it is prime within a certain amount of time simply implementing it and verifying it can be the best course of action. However, when “correct” can’t be verified easily more attention should be given and is warranted.

I have been programming since I was 12. For those who are not in the know that takes us back to 1982. 6502 machine language and ms basic reminds me of the loose weirdness present in a lot of scripting languages today. PHP on a shared server only performs so well for intensive tasks. It is appropriate although interesting that in the interest of compatible languages and common abstractions performance is┬áso severely hindered. The idea that you have the same hardware, say an intel box with all the bell’s and whistles running the same processor at the same speed and the same memory, with a multitude of abstraction layers built on top of it only to have the final one be PHP or whatever in order to make these systems virtually the same. I’m not picking at the situation here only poking at the irony. You start with “the same exact system” and force divergent abstraction layers only to push them back together at the top. There are good reasons for this of course.

This overwhelming array of options is part of what confuses programmers with little background. They start to lose sight of rhyme and reason. If a poorly written program at abstraction layer 1 can run 1000 times faster than a well written one at layer 10 they get truly confused what as to what appropriate optimization is. How many times have I seen a programmer try to optimize performance in the most ridiculous ways because of this complexity.

Other levels of complexity that confuse the newer developer involve things that seem to be apparent contradictions. A number of zealots cry for API simplicity while others taught the advantages of using the loosely grouped API’s that have the kitchen sink thrown in. Kitchen sink API’s might work fine – when the code is say open source and popular and tested a million times over. It’s all about trust. Being someone who’s always loved graphics programming I have a strong trust for the OpenGL API. It’s not object-oriented, and it is big in some respects. But it has been tested and proven. That being said I would never design a new API like it. Too many points of potential failure. This is my fault. Or other coders faults. Or human nature. But the fewer points of failure you have the better off you are. OpenGL is an awesome rock now. But in its day it had plenty of bugs to be fixed I’m sure. Now to be fair to OpenGL it does it’s job well. But at the same time the API is very large. Of course compared to many object-oriented API’s it is quite small. This should be seen as a truly terrible evil.

The choice of using an underlying library that has vast capabilities confuses the newcomer because they’ve been told that the simplest API is the one to implement. They have been told the gospel truth. But it is a great idea to rest on strong well-tested bug free API’s. Even if they aren’t the most streamlined. Better an evolved API that is big and tested than a small one that is full of bugs. But when you start out bugs are guaranteed. So small is key. There are many other reasons small can be key. But ironically small API’s often require you to understand the effects of the inner combinations of calls better. You need to be able to build clever solutions from a few parts. Whereas bigger API’s have convenience functions for everything – even if it seems painfully obvious. Some kinds of programmers are more rote – they function better with a large variety of calls that each do specific things. Others are more concept oriented and have an easier time putting together permutations themselves from as few existing parts.

But what does this have to do with agile programming? The concepts I mention above are those concepts discusses when not being “extreme” about programming. Design causes you to take a step back and look at the big picture. Conversations like those above will rarely be held in purely “extreme” shops. They don’t immediately solve a useful problem. They can be safely ignored as topics when you already understand them. But when you don’t understand them they will keep biting you until you come back to them.

In 20 plus years of coding I have seen the full spectrum. I started coding by the seat of my pants, I was 12. When I started coding with others I learned the ins and outs of design. After a few years I learned about absolute simplicity, and the bang for your buck paradigm. Less is more. And then of course came the extreme and agile movement. They are all parts of the equation. The industry is too young to simply teach first principles in school and let people stick to a completely agile mechanism in the real world. First principles, design patterns and other essentials are still being uncovered even as we speak. Design is essential to understand the big picture of what you are doing. Agile development is essential to discover it. And you need both. Use the mind you have AND understand it’s limitations.

Design is good use of learned techniques, evolution and emergence are good ways to discover new possibilities. If you’ve ever built a house using stories you know how shortsighted extreme programming can be. However if you’ve ever worked on a strictly waterfall development project you know how shortsighted a project that doesn’t accept the limitations of planning can be.

Do both. And makes sure those who work for you do as well.

Leave a Reply