The optimal programming language is the one the programmers who will be using it are best suited to.

This doesn’t mean that it’s the one they already know. This means it’s the one that matches their skill level and the way they think. It’s more important to match up the language with the programmer than the problem that is being solved or any other criteria. There has been a lot of talk lately about “bondage and discipline” languages, dynamic typing and expressive power in programming languages. People think in different ways for different reasons. People have different skillsets. And others can be productive in an environment with rules and can create spaghetti in one without.

If I had a small team of top-notch programmers I would love to fully embrace and use a language like lisp or a purely functional language. That being said I have worked with a large number of programmers who can’t understand the basics of object oriented programming. They know the syntax but klduge the code into something that grinds to a halt and is unmanageable very quickly. The thought of the meta-languages those same programmers would create in lisp send shivers up my spine.

I hear a lot of crap about “the heat and the kitchen” and “if you don’t have the highest quality programmers you are doomed to failure”. Frankly that is pure crap. A certain percentage of programmers are “beyond repair”. But a large number of programmers are extremely solid when in the right environment. Frankly they need more structure and less wiggle room. And then maybe 1 in 10 programmers is the type that can be given full range to “be trusted” to use the full set of programming paradigms in any way they see fit. It may seem a little heavy handed, but show me any experienced team lead who disagrees.

Personally I find “bondage and discipline” languages to be best suited to the real world. We’ve all heard the statement “Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.” That statement is almost correct. I would redact that statement to the following “Any sufficiently complicated and well engineered program contains an implementation of Lisp.” The first statement simply isn’t true by itself because,if nothing else, many implementations of Lisp have been written in C. “ad hoc, informally-specified, bug-ridden, slow” - these added bits certainly don’t have to be the case.

I’ve done a lot of game programming in the past. A very common technique is to have the core in C++/C and have a scripting language sit on top of it. Often times it’s LUA or a Javascript-like language. The programming teams working on the scripting languages vs. the core C++/C code are different groups.

Design patterns are another way of embedding these principles in the language that’s being used. You often hear, “Design patterns! - those are bad smells! The programming language is insufficient - this is the only reason the design patterns are important”. I disagree. Design patterns may involve “wordier” constructs than a language that supports those concepts natively.  This involves less expressive languages having more lines of code to express the same concepts.  This part always makes me laugh. I don’t care what the programming language is - the number one problem I have seen again and again is TOO MUCH CODE and not enough THINKING. I’m not talking about lines of code - I’m talking about code control points or cyclomatic complexity.  Your XP junkie would cry foul but some programmers should learn using a punch card type mentality. Once they have mastered their craft only then should they be able to rebuild and modify “on instinct”. Carpal tunnel syndrome should never be a programmers fear.

I have worked with C code that was object oriented. Yes, I can hear you coughing. It is wordier than any language that embraces some form of OO but it can be done. And once it’s done it isn’t that terrible. Encapsulation using static inside compilation units, function pointer tables and a system to manage it, the whole 9 yards. Turing complete is turing complete.

Sometimes creating a barrier to entry to implementing more advanced concepts can be a good thing.

How many languages could be an optimal number to have at our disposal? With all considerations taken - hiring new coders, etc., etc. Some contenders based on popularity, power and potential include ones from several camps:

1. Java/C# - These guys are today’s C++. And they pretty much shine for the same reasons. Any good programmers you hire will be able to code in them with reasonable proficiency.

2. Lisp and it’s spawns - These guys seem great for that core of coders that will be building domain specific languages and need the expressive power. It’s hard to find programmers that will do it justice (as opposed to kludge up something truly clumsy), but when you do you will have some terse truly elegant code that is both flexible and powerful. But throw the code into a normal pool of programmers and they will beg to rewrite it in Java/C#. So definitely don’t do that.

3. Javascript - Sure it’s Lispy. But it is stands apart because of its ubiquity. You can find plenty of coders who think they can do it justice. And under constrained environments they may be right.

4. Truly functional languages - Haskell, etc. - I’ll be frank. I really can’t say I fully grok the trade-offs with these languages. I definitely don’t see these languages ever going mainstream simply because of the difficulty in finding people to use them appropriately. In certain domains they will rule, in others they will simply be overkill.

Those languages seem to have strong enough support and be sufficiently different to justify their utility.

That being said a lot of the conclusions I have come to are based on “justified” instinct based on my background. The languages I’ve coded in extensively since 1982 include (chronologically) 6502 machine language, “Bill Gates” Basic, Borland’s Turbo Pascal, 80×86 assembler, C, C++ and C#. I have also done some commercial projects in Visual Basic, PHP, Java, and Javascript . I have written code for a large number of languages but my career has involved those listed above.

I feel that in today’s world specialization is in order. Programmers should be able to choose different tracks and learn different languages in school and for their careers. We draw a distinction between digital logic design and programming languages. I think we should also break programming into specialties.

The programming paradigm should fit the programming audience (those who code to it). Like math or any science, programming is becoming a broad enough discipline that specialization is needed. After all, would you want your brain surgery performed by any doctor? I wouldn’t think so.

Please read Programming Paradigms and Programmer Audiences for a more through explanation of the metaphor below.

Driving:

Freedom…..

1. I know how fast I can safely drive and don’t need a speed limit to govern this.
2. I should be able to make a u-turn on the highway if I need to.
3. If there are no oncoming cars at a red light I should be able to drive through.
4. When there is space to park right in front of my destination and its not marked as a parking space, parking lot spaces are truly inconvenient.
5. Turn signals aren’t useful for someone who is paying attention.
6. A driver’s license doesn’t actually help me drive any better.
7. I’m aware of my car and don’t need to have it inspected constantly in order to know it running properly.
8. Having tags on my car isn’t useful to me - if I commit a driving violation pull me over and I’ll give you that information.

I am clever enough and good enough to master all of these things.

Constraints…….

People are inconvenienced by them.

However,

1. I feel safer knowing others have to obey the speed limit.
2. I don’t want people making u-turns on the highway.
3. People driving through red lights are accidents waiting to happen.
4. Parking spaces don’t impose perfect order by any means, but the order they do impose is good enough to prevent parking congestions.
5. Turn signals are reassuring because it seems like no one is paying attention.
6. Having a drivers license makes certain things more uniform and less arbitrary.
7. If people weren’t required to get their cars inspected the road would be a dangerous place.
8. People feel accountable with tags on their cars, they are more likely to play by the rules and abide by their contracts.

I don’t feel people are clever enough and good enough to master all of these things without rules.
Programming:

Freedom………

1. Rapid code generation and prototyping gets things up and running faster.
2. Lambda functions make it possible to elegantly solve some otherwise difficult problems.
3. If I know it is safe I should be able to do it …. period.
4. I should be able to organize data and code any way I’d like, flexibility and true macros are a good thing.
5. If I call a function it will see what I’m passing when it gets there - signature signaling is gratuitous.
6. I don’t need formal class definitions….. I’m not sure what the point is.
7. Compile time checking is pointless - run time checking does it all.
8. Static typing enforces rules that would be enforced anyways.

I feel I am clever enough and good enough to master all of these things.

However,

Constraints…….

1. So people create some truly terrifying prototype code.
2. Some people find lambda functions difficult to grok.
3. A lot of people will shoot themselves in the foot if handed a gun
4. Programmers can create some truly obscure metalanguages, and given the flexibility create masterpieces in obfuscation
5. When functions have explicit contracts they are more likely to at least try and follow them
6. Formal class definitions make it easier for most people to understand what is going on.
7. No matter how much you test you probably haven’t tested enough.
8. Static typing makes people more aware of what the rules and contracts actually are.

How do I think I drive:

I’m streamlined, clever and carefree. I bend the rules when I can to get the most flexibility I can.

How do I really drive:

Sometimes those rules and regulations really save me. And I am extremely glad they are there for others. There are some idiots on the road.

SOLUTION:

Why won’t these idiots just get off the road (not going to happen).

If programmers aren’t up to the tricky stuff they should be coding (not going to happen).

REALITY:

Everyone obeys the rules but bends them when they can to get the flexibility they need.

Languages need to support multiple paradigms to truly be safe and useful.

So I was working with a company that did 3D development for games! It was 1998 and things were still fresh and new in that area. The three API’s of mention were Glide, Direct 3D and OpenGL. I was working with some amazing artists and programmers and was glad to be learning more about the ins and outs of 3D development and tools. This was my first taste of a scene graph using Paradigm Entertainment’s Viskit. Scene Graphs are amazing things - and I was learning everything I could. Viskit was based on SGI’s Performer and like OpenGL - they had done something really, really right when crafting this API. Viskit sat effortlessly on top of OpenGL or Direct3D. I ended up working quite a bit on our Multigen loader to get all the bells and whistles supported. We used Max for our prerendered scenes but at this point Multi-Gen creator was huge in developing real-time models. It was a great app that made both programmers and artists feel immediately at home. Scene Graph nodes could be mapped very easily to Multi-Gen nodes. It was a perfect match.

As usual my dedication ran high. I was able to entice a truly stellar programmer from my time in NJ to live with me a significant portion of the month so he could code on the team as well. I worked with two artists that two this day I have to say are second to none. We created an amazingly beautiful world with truly stunning environmental effects. It was enthralling. In the end though small companies with only one project are fraught with peril. In 2000 Aeon closed it’s doors and my love for Viskit took me to my next job at console developer Paradigm Entertainment. They had built a version of Viskit that ran on all the next-gen consoles including the X-Box, Game Cube, PS/2 and even the dreamcast. It could still be baselined on the PC as well. This was my first time working with consoles - a fixed format machine, finally!
It was now 2000 and I found myself in Dallas, Texas, In typical game industry fashion several weeks after joining Paradigm they sold themselves to Infogrammes. However, they still were able to call their own shots for my time at the company. It was awesome having 3 or 4 teams working with 3D technology for multiple consoles. This technology was developed by a dedicated core technology group - I hadn’t been lucky enough to work with a company big enough to have this luxury before. They were the think tank of the company always getting all the heavy pressures but coming up with some truly stunning innovations. All of our artists were using Maya - which worked well, it supported a group hierarchy for scenes like Multi-Gen creator had previously.

Consoles are truly outstanding beasts to develop for. In all the years of working on the PC I had always dreamed of being able to develop for fixed platforms. You could know that once your testing was done it was good to go - no surprises based on machine configurations later in the pipeline. C++ was still the defining factor in game programming, although plenty of PS/2 code was written in Microcode as well. Common acceptance of scripting languages for high level prototyping was still a few years off. The best part about working at Paradigm was access to resources. When teams ramped up they could have 8-10 artists and almost as many coders. Things could really come together fast.
2003 found me back in Virginia where I started working for ITSpatial - a company that did 3D representations of cities a la Google Earth … before Google Earth had been around. In fairness there was Keyhole, but what ITSpatial offered was a different kind of thing all together. ITSpatial built a product that provided data integration and fusion in an environment that supported 2D and 3D mapping. This was perfect for command and control applications, situational awareness and emergency training. It was really one big serious game and had many of the elements of a full fledged game development effort and content production pipeline. The only thing difference was the price point of the product and the number of shipping units.

ITSpatial was strong and knew a lot about sales and business development. They were phenomenal sales people. I will always be amazed at how effortlessly they were able to get potential clients in the building, looking at demos - 3 or 4 times a week some weeks. They really were devoted to finding and wooing clients. This is a skill I’ve still got a long way to go on. I guess in some ways I’m still 100% a programmer at heart:)

In 2005 I broke away and this time started my own company Eureka 3D, Inc. I’m more of a hired gun/contractor with the luck to have access to a lot of other proven hired gun contractors. Eureka3D has worked on entertainment 3D, GIS and I’ve really boned up on my web programming skills in the last two years. Deep down inside I’m still wanting to get back “into the game” and sink my teeth into that big 2 year development project. Right now my pipeline is very full and I really am enjoying what I’m doing - so no complaints!

Here we are in 2007. What have I learned from the game and serious game industry after all these years? So much… I’ve worked with top notch programmers, others who couldn’t program their way out of a paper bag; I’ve worked with great artists, and a young up and coming project manager who was one of the sharpest guys I’ve ever met. I’ve worked with some truly genius biz-dev types that knew how to sell and pitch to clients. In 24 years every theory on project management for software/hardware/games has been debated, turned over and debated again much to my chagrin. I’ve seen every element of the business from pitching the contracts, building proof of concepts to final testing and delivery/deployment/distribution. I’ve learned a thing or two about politics and how important it is to avoid if at al possible. I’ve literally seen programmers in drop-down, drag out fights (visiting programmers from another company no less!). Religious wars over version control systems, graphics api’s, and taking on the latest new technology - been there, seen that.
The main thing I’ve learned however is that I thrive on the high-intensity, creative, ingenius environment that games and serious games involve. Finding a team that works well together, or a few truly brilliant individuals makes it all so worth while. I know I want to keep raising the bar and never lose the adventure. Because in the end it’s about the product, the team, and the experience. What an amazing 24 years it’s been!

1. Being a great problem solver.
2. Being driven and lazy at the same time.
3. Ability to understand other people’s code
4. Having a passion for programming
5. Loving learning for the sake of learning
6. Being good at math
7. Having good communications skills
8. Strong debating skills
9. Extreme optimism
10. Extreme pessimism

After putting together this list some aspects surprised me, and I was the one who put together the list. So let me explain each in detail.These attributes describe those I’ve found in pretty much every great programmer I’ve come across. There were a number that fell through the cracks and I’ll explain those later.
1. Being a great problem solver - Hopefully everyone recognizes this one. Most good programming is all about being able to find solutions where others can’t see them. If you don’t have this skill the others matter far less.
2. Being driven and lazy at the same time - This one surprises some people. Programmers question things and are often “too lazy” to take the long route. The will spend countless cycles trying to simplify the problem and simplify their task. That said they having a burning need to get the job done, they just want to do it as efficiently as possible.
3. Ability to understand other people’s code - This point is essential but cuts some good programmers off from being great programmers. It doesn’t matter how well you can rewrite everything yourself - you need to be able to work with other people’s code on existing projects, lean on opensource in new projects, and learn good techniques from the code base that is already out there.
4. Having a passion for programming - on some level you have to love programming for programming’s sake. I suppose to be truly great at anything you have to love it in most cases.
5. Loving learning for the sake of learning - Programming is a moving target. Unless you love the art of edification you will sink fast. There are no laurels to rest on and no one cares what you did yesterday. Unless you are aware of the techniques on the horizon, you won’t be ready to embrace them when they become relevant.
6. Being good at math - Different people will have different opinions here - but at the very least having a strong grip on pre-Calculus math. I’ve never seen a great programmer without a solid grasp of at the very least algebra and trig.
7. Having good communications skills - This doesn’t mean that they can communicate with anyone and everyone. Specifically this means that they are able to clearly express their thoughts on their own terms. I’ve met plenty of great programmers who couldn’t communicate well with the world at large. However, given someone to talk to who understands the problem domain, they were all able to clearly state the problem and the solutions proposed.
8. Strong debating skills - This follows the same logic as #7.
9. Extreme optimism - Great programmers I have encountered have an insane certainty they can get the job done once they have chewed on it a bit.
10. Extreme pessimism - Great programmers I have encountered have an insane insistence that when they lack the information needed to make a good judgment that they won’t be able to make one at all.

Some of the things I instinctively wanted to put on the list but couldn’t say were true of at least 95% of great programmers include the following:

1. Being extremely organized - Understanding when and where organization is important, yes. But anal attention to detail is something present in great programmers as often as it is people in other disciplines.
2. Being good at managing other people and or programming projects - Somehow these skill sets are wonderfully synergistic when they sit side by side, but management and programming are often completely different disciplines.
3. Being able to write good design documents - Same as #2. This skill may make some people better programmers and I am in favor of learning it. However, plenty of great programmers I have encountered couldn’t write a coherent design doc if their life depended on it. This will know doubt be debated by heavily by some.
4. Having an ability to estimate time frames - Once again like #2. This is an acquired skill and a very useful one. However, I have seen absolutely zero correlation between great programmers and estimation skills.
5. Prolific reading of tech books - I do this all the time myself, but many great programmers don’t. Let me be clear though - most programmers who aren’t all that hot could definitely benefit from bootstrapping their skills with some good reading.
6. Ability to transfer their programming skills to any programming domain - Although many can, some great programmers can’t, or refuse to, grok other programming technologies. I like to think that this is a “refuse to” situation.
7. Write code that is correct the first time around - Many great programmers commonly have syntactic issues flagged by compilers or at runtime interpretation. Some are zealots about the details the first time out, others are much more “extreme” in this area.
8. Having other areas of great skills - some great programmers are good at only one thing - programming.
9. Social or antisocial - Great programmers come in both forms.
10. Are someone you’d want on you team - Unfortunately some of them just can’t work with others well.
]]> http://programmingmatters.com/2007/02/05/the-top-10-attributes-of-a-great-programmer/feed/ http://programmingmatters.com/2007/01/26/wonderful-blogs-evil-splogs-and-now-entertaining-clogs/ http://programmingmatters.com/2007/01/26/wonderful-blogs-evil-splogs-and-now-entertaining-clogs/#comments Sat, 27 Jan 2007 00:44:50 +0000 Steve Riley http://eureka3d.com/blog/2007/wonderful-blogs-evil-splogs-and-now-entertaining-clogs/ Wonderful (web)blogs, evil sp(am)logs and now entertaining c(orporate)logs. We’ve come a long way since the .plan files that internet users made publilc to keep people up to date back in the day. Each popular thing evolves - this is the nature of technology and the net. And .plan files evolved with the power of html to the force of nature they are today.

Blogs, splogs and clogs. Technology has a way of coining cute terms and sticking to the theme. The latest in the series, the clog, a new breed of corporate advertising log. What’s a clog and how did they get here? Let’s look from the start.

When blogs came on the scene they changed the landscape over night it seemed. Who would have thought that such a simple concept could have such impact.? I remember years ago when online diary sites were a minor hit. And I do mean minor. It was inconceivable that in the future many, many people would be busy posting their ideas and feelings about topics on the web on daily basis. But that time came. Networks and communities sprung up overnight. Social networking sites like myspace and livejournal became 10 pound gorillas. At some point it seemed like everyone had a blog. It didn’t take for that crafty breed of annoying innovator , the spammer, to take notice.

Enter splogs. Spammers scraping sites to generate the appearance of personal, fresh content. They usually smell bad like all spam, but they proliferated all the same. Spammers were sprinkling bot dust on real blogs and even creating deep scraped blogs of their own. They did this of course to fool people into clicking on and buying products. Or to generate enough raw pages and links to get pagerank and get their affiliate sites high up in search engine rankings. Google and blogging software were finding themselves fully occupied to combat this. At around the point where everyone had a blog and was complaining about the prevalence of splogs….. clogs start to come on the scene.

These not so distant relatives of splogs and blogs are unique in their own right. Companies paying money to increase their search engine rank and presence on the web by creating high quality corporate blogs. Typically the polish of their content was higher than blogs, however, their motives were oddly similar to sploggers. They wanted to add real quality content and value to increase search engine placement, create more interest and generate exposure. Sometimes the lines get blurred. Everybody remembers the Blair Witch online hype from before that movie. This was the grandmother of cloglike thinking, studios themselves creating interest using efforts that appeared to be grassroots . UTube has been hit by the same kinds of things in their arena. Nobody likes deception. But oddly enough many of these high quality paid or encouraged clogs are starting to take hold. Many companies let their employees blog about their jobs online at work. The line is blurred in a good way.
Today there are quite a few companies clogging and adding real value to the web. They are entertaining and providing the information using the informal setup of the
blogs. In many cases they are visually appealing and contain great content. The web has grown up like all industries do. At first pioneers break new ground, prove their concept and gain notoriety. Then the corporate world catches on and takes the concept, polishes it and runs with it. This is certainly the case with clogs.

Think about Joel Spolsky or Matt Cutts… personable bloggers who mention their companies in each passing breath. But this is all good. Content is still king. And everyone has come to the table willing to share. Blogs, splogs and clogs….

The overriding question is where will it end and what will be next in this evolution?

]]> http://programmingmatters.com/2007/01/26/wonderful-blogs-evil-splogs-and-now-entertaining-clogs/feed/ http://programmingmatters.com/2007/01/19/how-seo-will-kill-google-or-the-problem-with-backlinks/ http://programmingmatters.com/2007/01/19/how-seo-will-kill-google-or-the-problem-with-backlinks/#comments Sat, 20 Jan 2007 05:09:31 +0000 Steve Riley http://eureka3d.com/blog/2007/how-seo-will-kill-google-or-the-problem-with-backlinks/ When the web was young, backlinks were the perfect way to measure a sites popularity. The sites with backlinks were more popular and the sites with backlinks from more popular sites were even more popular as you would expect. And all was well.

But in today’s world no matter how much bleach you apply to your white hat, you can’t get around that little bit of knowledge you’ve acquired. Backlinks = success on Google. And by knowing this and “exploiting” this you will knock out those websites that don’t attempt to do any “SEO”. Your white hat may look white, but really it’s gray. You will do all the right things to generate backlinks. Invariably as much as Google would hate to admit it, this breeds a competitive landscape. “You can’t win if you don’t play.” It has now become essential to “ethically” build backlinks in order to get organic Google search results. And this is very bad for the quality of links on Google’s search results.

No matter how innocent you try to be, the cat is out of the bag. It is far to easy to get a large number of “encouraged” backlinks to a given site. When you search on Google the really big players come up first. And this is good. They have such a huge number of backlinks that they can pretty much be seen as genuine. It’s the middle ground. The “long tail”. These are the problems. If a smart developer/designer releases a Wordpress theme with a backlink and that theme is used by several hundred people they get lots of page rank. These wordpress themes can be used on popular blogs with plenty of pagerank themselves. This is “white hat” activity. But it’s really intelligently gaming the system. The SEO gurus know plenty of tricks to get backlinks and PR.
Because of this many Google “long tail” searches that are more specific are filled with SEO’d sites both white hat and black hat spam. I know quite a few people who actually are switching to other search engines because of the “backlink” spam. “Quality backlinks” are just too easy to get, especially if you are willing to pay money for them. And that goes on everyday.

Backlinks were a good idea at some point. Now they are only showing that you are trying really hard. The people that talk the loudest and the most often aren’t always the best. And proliferating backlinks to your site with all the right keywords to game the system doesn’t mean your content is the best either. It just means that you are good at generating backlinks.

Until spiders can have a clue what they are parsing it will be hard to impossible to solve this problem and still use backlinks.
Social networks like digg.com, stumbleupon.com and reddit.com are “the next new thing”. By having people vote on pages… the crap filters to the bottom. This of course will have it’s own set of problems - but invariably will be far better and more accurate than backlinks. At the end of the day some content will be brilliant and good - and still be undiscovered. However more data will yield better results.

How do you really judge the worth of a website? Is it how many people like it? The demographics of those people linking to worth for specific topics? How long people spend on the site itself? These are fascinating questions that search engines will be forced to answer in upcoming year - or they will be inconsequential.

]]> http://programmingmatters.com/2007/01/19/how-seo-will-kill-google-or-the-problem-with-backlinks/feed/ http://programmingmatters.com/2007/01/19/why-silver-bullets-tarnish-over-time-new-paradigms-getting-thrashed-in-the-field/ http://programmingmatters.com/2007/01/19/why-silver-bullets-tarnish-over-time-new-paradigms-getting-thrashed-in-the-field/#comments Fri, 19 Jan 2007 07:30:29 +0000 Steve Riley http://eureka3d.com/blog/2007/why-silver-bullets-tarnish-over-time-new-paradigms-getting-thrashed-in-the-field/ This concept really applies to much of life, however, computer science in particular.

Why is is that new ways of doing things can have remarkable success when they are at “the discovery stage” and end up being useful but disappointing in the field? Object oriented programming turns into spaghetti class interdependencies and deep hierarchies in day to day use. A giant leap forward and two steps back. Software patterns turn into misuse and abuse of the Singleton pattern and inappropriate shoehorning. Extreme programming becomes an excuse to write unmaintainable code that passes each and every unit test. Simplicity becomes an excuse to build things so bare bone as to become unusable. It seems like each new “promising technique” becomes troublesome once it goes mainstream. Still worth the trouble, but troublesome all the same.

You may find yourself scratching your head about why such issues aren’t discovered before they gain mass acceptance. Why when things are in “the R&D” phase they perform so well. Even when these same things are tested in limited circles on “real world problems” they continue to seem like the next silver bullet.

The answer unfortunately is as hard to swallow as it is simple. Early on the people developing and working with these new technologies are quite often some of the best in the field and the smartest guys out there. By the time the concept migrates to the other 90% of the coding world it finds itself slammed against the middle and trailing ugly end of the bell curve. These portions of the curve will often times misunderstand and misuse the new tools and techniques presented to them. It’s during these phases that the backlash usually kicks in. Guidelines and rules must be created to keep the average programmer from hanging himself with too much rope.

Often times new paradigms go through a few separate stages of development and acceptance. In stage 1 the really smart R&D types hit it. These people are actually great at creating new solutions that are both groundbreaking and elegant. Once these “conceptual entrepreneurs” have firmed things up it is ready for stage 2. In stage 2 the brightest of the “real world group” is brave enough to tackle integrating these concepts into production code. They become evangelists of the techniques they have had so much success with. Stage 3 involves more risk takers embracing the techniques advocated by stage 2. They pour over the concepts involved, grok them and integrate them into their code bases. It’s at stage 4 that the tide starts to turn. At stage four the middle of the bell curve is beginning to be pierced. The new mantra is gaining success. By stage 5 you have everyone accepting the new paradigm as “the way to go” based more on appeal to authority and peer pressure. At this stage a significant number may not understand the new techniques at all. They will advocate them but only graft them on top of their existing ways of doing things.

I like to think of myself as a stage 3 integrator. At this point things have hit my comfort zone and they are worth a try. I will do everything I can to understand a new technique, but by no means have created it or been the true dare devil integrating it at stage 2 into production code. Later down the pipe when I see these concepts misused and abused - I avidly follow the “shoring up” techniques to keep people from blowing body parts off my misusing a new technique.

No matter what the latest silver bullet appears to be, it is merely getting it justly deserved 15 minutes of fame during its integration period. These techniques will live on but the focus will move on. It’s important not to throw the baby out with the bath water. New techniques may disappoint after several years of growth - but in the end they often stay part of our process. Object orientation, design patterns, components, agile programming, the list goes on. Don’t forget that silver bullets are still bullets even when they tarnish. Just because a concept hasn’t completely lived up to its hype doesn’t make it a great and useful part of your process.

]]> http://programmingmatters.com/2007/01/19/why-silver-bullets-tarnish-over-time-new-paradigms-getting-thrashed-in-the-field/feed/ http://programmingmatters.com/2007/01/10/24-years-of-game-programming-thrills-chills-and-spills-part-2/ http://programmingmatters.com/2007/01/10/24-years-of-game-programming-thrills-chills-and-spills-part-2/#comments Wed, 10 Jan 2007 07:36:21 +0000 Steve Riley http://eureka3d.com/blog/?p=24 If you haven’t read the first part of this article, you’ll probably want to check it out here.

1995 - Legend Entertainment - My First Industry Years

I was finally working full-time in the game industry at Legend Entertainment and couldn’t be more thrilled! No more sandwiching game coding in between contracting gigs. I was with a company that has produced a large number of award winning titles, gotten them published and distributed - all with a company of under 20 people!

I started out coding in Watcom C on a daily basis and working in crunch time on a game called Star Control 3. I was thrilled working with industry veterans from the Zork era! We even had an inhouse testing team. I can honestly say I’ve never worked at a company that ran as smoothly and without contention as the time I worked at Legend. There were blips on the radar, but over-all development fell into place extremely well during my years there. This is a testament to team dynamics. Get a good team that works well together and keep them together.

I learned so much while at Legend. These guys were experts at what they did. It was amazing being able to sit in on the entire development process - creative, technical and other. A lot of my best practices I learned while I was at Legend and over the years this feeling never left me. We’re talking about game development done the way everyone dreams of it being done. Projects would often times have just a few coders on them for most of their development cycle. I can say that I never saw a game canceled while I was at Legend - and had never heard of one being canceled before I joined them. They ran lean and mean and couldn’t afford that kind of wasteful slack. This had an extremely positive effect on morale needless to say.

At the time we were writing DOS based games that used SVGA libraries to build 256 color 640×480 games. The WATCOM compiler let us overcome DOS’ one meg memory barriers and access all of extended memory. We had to support a dizzying array of low tech video cards and collection of VESA video modes. But boy was it fun.

From a philosophy standpoint I finally started to get behind the understanding that old rules change. Optimization was still king, but design came first. Things I had previously considered wasteful were trivial on 486 systems with 4 meg of available memory. I began to realize that when working with a group of programmers design and communication were of paramount importance. These things had to be worked at, they just didn’t fall out naturally.

By 1996 I was coding in C++ in Microsoft (Visual) C++ and we were working on a game library using Direct X 5 for Windows 95. DirectX was all the rage. The access to hardware sprites and features was a major boon. OpenGL was at the time still too high level(slow) and lacked ubiquitous support. Of course we weren’t using DirectX for Direct3D at the time. Direct3D was a few revs a way from being usable. But uniform driver support for 2D sprites and pages was a major plus in itself. However 3D was on the horizon and I was very curious.

While I was at Legend I worked on 3 games and saw 5 games ship - all with a group of under 20 people; including the testing department, customer support and marketing! I owe them a huge debt of gratitude for everything they taught me and breaking me into the game industry.

By 1997 I would find myself somewhere new. I had a hunger for cutting edge 3D and a company in NJ was doing that and more. They were actually producing their own OpenGL compliant hardware for the military using an off the shelf Glint chip (from 3D Labs) and a proprietary in house ASIC they had built for voxelized 3D terrain rendering. I was now at ASPI!

1997 - ASPI


ASPI was another opportunity to work with a small group of brilliant people. The engineers that worked on digital logic design considered machine language high level. They used custom software to produce their own ASIC’s, soldering them to the boards in-house. I had already become very sold on object oriented development at the time. I had definitely drank the cool aid. I picked up a copy of Design Patterns by the Gang of Four and fell in love with it. Years later I would spend plenty of time examining the ways that novice coders abused OOP and design patterns. But in 1997 I was still loving learning all the ins and outs. I picked the few patterns that fit the API we were developing and got to work. It took a bit but I got everyone on board with tightly optimized C++ (which at the time many saw as an oxymoron). I had realized that C++ could give you the low level control to maintain C like speed. And I liked it - a lot.

I was now using OpenGL and MESA(an opensource OpenGL) and accepting the fact that on modern machines - OpenGL definitely was worth the minor performance loss. Back in those days there were still camps that wanted much lower level access to graphics hardware to eek out every last bit of power. We even got to write our own custom drivers for MESA under SCO Unix.

The cards were awesome and we ended up calling them the “True Terrain 3D”. The military had a contract to buy them up and deploy them. They were able to ingest DTED data and use LODed voxel planes to create amazing looking terrain. We interleaved access to the frame and depth buffers with the Glint chip and OpenGL/Mesa to add polygonal features. This was in 1997 and at the time polygonal 3D cards couldn’t come close to generating the terrain that the custom cardset could. Not in the under $20,000 price range at least.
I loved everyone I was working with but invariably even cutting edge high tech couldn’t keep me from wanting to go back into the games industry. Somehow high tech and “serious games” were exciting, but games were still in my blood.
By 1999 I was back in the industry working on a submarine sim at Aeon Entertainment in Baltimore.

But we’ll cover that in part 3….

]]> http://programmingmatters.com/2007/01/10/24-years-of-game-programming-thrills-chills-and-spills-part-2/feed/ http://programmingmatters.com/2006/12/31/how-null-breaks-polymorphism-or-the-problem-with-null-part-2/ http://programmingmatters.com/2006/12/31/how-null-breaks-polymorphism-or-the-problem-with-null-part-2/#comments Mon, 01 Jan 2007 06:05:42 +0000 Steve Riley http://eureka3d.com/blog/?p=23 If you haven’t read part one you really need to do that first. It’s here.

I got a lot of interesting feedback on part 1 of this topic and found I needed to further explain myself in certain areas.


Two initial responses to issues brought to me from part 1:

1. Typed languages

In languages that aren’t typed at all, null is no more a problem than any other reassignments as they never care about types. They all can result in similar issues. Which in my mind is in fact a problem. I am definitely a huge proponent compile time type checking. This is why perhaps I seem so hard on it in part 1. Compile time checks should be able to help out.


2. When functions might not return a value

There are plenty of times when a function may or may not return a value. If this is the case the return type should reflect that. Having null used as a “magic number” is not the ideal solution in my mind. I’d much rather see a return type that forces the issue to be very clear. It may seem non-standard but a type masquerading as a collection that potentially has one item seems ideal. Most any programmer will realize he needs to check the size of a collection before trying to access its first element. This is no more cumbersome than checking for null - but seems logically intuitive. This can be easily optimized so that there is no performance hit in a language like C++ using templates. A container is intuitive as it can be potentially empty - this very straightforward.

Making the problem even more clear:

I often find it easier to express programming concepts in real world terms. This helps to reduce them to the absurd when appropriate. This doesn’t always work but a lot of times can help look at the problem from a different perspective.

Let’s take the concept of typical null check behavior and attempt to map it to a real world procedure.

We are going to explore John teaching Norm to drive a car. The following may sound a little familiar.

John: “First you are going to need to make sure you have a car. Do you have a car? If you don’t have a car just return to what you were doing. I won’t be able to teach you to drive a car.

“Norm: “I have a car.”

John: “Let’s call that car normsCar. Check to see if your car has a door. We’ll call it normsCarDoor. Does normsCar have a normsCarDoor?”

Norm: “Yes”

John: “Great, but if you don’t just skip the rest of this - I won’t be able to tell you how to drive a car.”

Norm: “I have a car door.”

John: “Once you open the door check and see if it has a seat we’ll call normsComfySeat. If you don’t have a seat skip the rest of this - I won’t be able to tell you how to drive a car.”

Norm: “I have a seat”

John: “Things have changed scope a bit - can you check whether you have a door again for me - we called it normsCarDoor?”………..

I think you can see where this is going. Classes have contracts. It should be reasonable to talk about a car that always has a door and a seat without having to neurotically check at all times.

Unlike the real world, when we code we make new things up all the time. So even though seats and doors can be reasonably inferred on cars the things that a UserAgentControl may or may not have probably won’t be obvious. Does the UserAgentControl have a ThrottleManager all the time? If I have a non-nullable class type I can check by looking at the class. If I get it wrong for some reason maybe I can have the compiler issue a warning. Maybe I can be forced to “conjugate” it with a special syntax every time I use it to help me remember (”.” vs. “->” in C++). Or a naming convention.

Why is this such an overarching problem?

It may seem like griping over a simple check for a things existence. Except this check can conceivably apply to each and every object I can have (or don’t have). This makes the issue enormous. A lot of the time programmers put in checks for null when it seems appropriate or when unit testing throws an exception at run-time. This sounds a whole lot like something statically typed languages are supposed to help us protect ourselves against. The unchecked null is by far the most common runtime error and the “overchecked” null one of the most prevalent unintentional code obfuscation techniques.

Solutions that don’t involve changing existing languages:

1. If your language supports the concept out of the box (C++, Spec#, F#, OCaml, Nice, etc.) or it can be built with templates or other mechanisms use not-nullable types. Use these types whenever possible. If you’re language doesn’t support it then use number 2 alone.
2. Create a simple naming convention (don’t go hungarian on me) to discern what is nullable and what isn’t. This is a fundamental concept and it should be obvious everytime a value, instance or object is accessed. Use this to compensate for your languages deficiency in a similar manner to prefixing private variables in languages that don’t have mechanisms for hiding access to variables and member functions.
3. Check for null as early as possible in the food chain and prefer to call methods that are granular and don’t have to check themselves. This means make every parameter passed to a routine get used. If a parameter is optional write a new routine. Make the types passed to these routines be not-nullable if the language supports it.

4. If it makes sense prefer empty collections to collections containing nothing but null for obvious reasons.

Wrap-up:

There is no perfect solution. But so many times in code I see null being checked or not checked and am left wondering. Is the check gratuitous? Is this a runtime error waiting in the wings? And unless its commented it or I check it’s usage - I get that typically uneasy feeling. Bound to catch it at runtime with unit testing. This is not the answer I want to hear.

I think a few simple practices and conventions could get this off our plate so we can get back to the problem at hand. Solving actual problems.

Afterthoughts:

Some other articles that beat me to the punch on some of these concepts:

1. Let’s Reconsider That Offensive Coding from Michael Feathers.
2. Null:The Runtime Error Generator by Blaine Buxton

Empty containers solve this problem nicely for return types. Also as many readers have mentioned, monads are extremely useful in solving this problem as well. But once again, stop propagating unneeded nulls and similar “safer null-like” structures as soon as possible. As Michael Feathers states - it’s just offensive code.

]]> http://programmingmatters.com/2006/12/31/how-null-breaks-polymorphism-or-the-problem-with-null-part-2/feed/ http://programmingmatters.com/2006/12/31/24-years-of-game-programming-thrills-chills-and-spills-part-1/ http://programmingmatters.com/2006/12/31/24-years-of-game-programming-thrills-chills-and-spills-part-1/#comments Sun, 31 Dec 2006 18:07:21 +0000 Steve Riley http://eureka3d.com/blog/?p=22 Originally I was thinking about calling this article 24 years of game programming: observations and lessons learned. Somehow the title seemed much too stuffy for an entry about something I’ve loved so much over the years.

In this three part series I trace my 24 years of game programming. From machine language in 1982, Pascal in 1987, C in 1992, C++ in 1995, using “high level API’s” like OpenGL in 97, catching the pattern craze in ‘98, the component mantra in 99, coding for 3 consoles at once in 2000, moving into Java, C#, and javascript on the top of C++ in the 21st century. And more!

Let’s go back to 1982:

This is where it all began for me. I had received my first computer a Vic-20 and dreamed of everything I could do with it. William Shatner had advertised it as a gaming system and a computer too (and we all know he has integrity from the whole priceline fiasco). I remember writing my first basic program for it from the thin manual that came with it. An animated bird that used character graphics to flap it’s wings and fly around the screen. I was mesmerized!

I quickly learned the machine inside and out, bought the Vic-20 Programmer’s Reference Guide, hung the schematic of the machine on my wall and went to work. Very quickly I realized that a 1 mhz machine with 5k of ram was going to take some heavy tricks to create games for. So I learned 6502 machine language to really pull the stops out. An 8 bit processor with an X, Y and Accumulator that could add and subtract any 8 bit value! X and Y could only increment and decrement. And of course when the numbers were larger than 255 a little juggling was required.

Very soon the Commodore 64 came out and I immediately upgraded. Programmable sprites, a SID chip for sound, 64k of ram (with 16k bank switched under rom that held Bill’s basic and the “OS”). Now we were playing with power. I wrote more than a few games for the Commodore 64 as a hobbiest in straight machine code. An Olympic games type clone, a chess game that worked with the 300 baud modem, a utility to dump on screen graphics to the printer, some arcade variations, a sprite editor.
These were the truly brass tacks years. 6502 machine language mixed with little bits of bootstrap basic was hardcore by any standards. Jumping to subroutines was something you couldn’t take lightly and the “hardware” stack was only 512 bytes long. It was strictly for keeping the return address from subroutine calls. I was firmly convinced games required maximum power and could only be written in machine language. I was pretty convinced this fact would never change.

I was of course wrong.

Fast forward to 1987:

The IBM PC/XT was starting to show promise and a friend introduced me to Turbo Pascal on it. 4.77 megahertz! 640k of ram. Sure my machine was an XT and only capable of CGA graphics (and character mode graphics - not even programmable at that). But Turbo Pascal was my first true compiler. It was actually possible to write a significant chunk of many games in high level code. Sure low-level interrupt stuff was often in assembly. We spent many, many all nighters coding games and often ignoring our girlfriends. But they couldn’t code:) I slowly got used to the fact that passing variables on the stack was a useful enough concept that it was worth the performance hit it sometimes had on games. We are talking about functions passing parameters on the stack! But this was “non-trivial” and took cycles. Things were still so tight that every cycle mattered.

Fast forward to 1992:

I got my first 386 20 mhz clone with 4 meg of memory and an 80 meg hard drive. I quickly bought a soundblaster card for it. Before long I was coding games in MCGA 320×200 256 color graphics mode. I learned everything I could about palette manipulation and palette color theory. I was coding in C by this time as well 8086 assembly of course. You could actually inline your assembly right there with the C code. It was awesome. I wrote a sprite compiler that would literally turn all the data into straight machine code so it didn’t have to loop and check for key colors. It used a triple buffer, dirty rect system to update the screen as efficiently as possible.

I learned Autodesk animator inside and out and spent hundreds of hours working on a sprite editor with bucket fills, circles, lines and palette manipulation functions as well as animation capability. I was in heaven. I recruited some good friends to do graphics for me and created a number of demos to show off the library.

I still hand-optimized everything and didn’t trust the compilers at all. A C compiler may not use a bit shift for an integer multiplication. The horror!

I made sure and learned all I could about fixed point numbers. At this point floats were still done in software and slower than hell. Since C didn’t support operator overloading the code looked pretty clunky. But it didn’t matter. 320×200 256 color mode was fast - really fast. And I was learning all I could from Abrash books about mode 13 hex that allowed the full 256k of graphics memory to be utilized. At the time the PC was still back switching 64k into it’s memory map. And don’t even get me started on 80×86 segment offset addressing!

I ended up releasing all my code as shareware and learning the ins and outs of shareware distribution mechanisms. I remember printing labels for 5 1/4 and 3 1/2 disks and mailing them off to shareware distribution hubs. This was before the internet took off of course and everything was BBS based.

1995 was right around the corner and soon I would find myself working at Legend Entertainment on Star Control 3 - my first commercial game product.

Part 2 cracks the lid open on my game industry years.

]]> http://programmingmatters.com/2006/12/31/24-years-of-game-programming-thrills-chills-and-spills-part-1/feed/