How null breaks polymorphism: or the problem with null: part 2

If you haven’t read part one you really need to do that first. It’s here.

I got a lot of interesting feedback on part 1 of this topic and found I needed to further explain myself in certain areas.

Two initial responses to issues brought to me from part 1:

1. Typed languages

In languages that aren’t typed at all, null is no more a problem than any other reassignment as they never care about types. They all can result in similar issues. Which in my mind is in fact a problem. I am definitely a huge proponent compile time type checking. This is why perhaps I seem so hard on it in part 1. Compile time checks should be able to help out.

2. When functions might not return a value

There are plenty of times when a function may or may not return a value. If this is the case the return type should reflect that. Having null used as a “magic number” is not the ideal solution in my mind. I’d much rather see a return type that forces the issue to be very clear. It may seem non-standard but a type masquerading as a collection that potentially has one item seems ideal. Most any programmer will realize he needs to check the size of a collection before trying to access its first element. This is no more cumbersome than checking for null – but seems logically intuitive. This can be easily optimized so that there is no performance hit in a language like C++ using templates. A container is intuitive as it can be potentially empty – this very straightforward.

Making the problem even more clear:

I often find it easier to express programming concepts in real world terms. This helps to reduce them to the absurd when appropriate. This doesn’t always work but a lot of times can help look at the problem from a different perspective.

Let’s take the concept of typical null check behavior and attempt to map it to a real world procedure.

We are going to explore John teaching Norm to drive a car. The following may sound a little familiar.

John: “First you are going to need to make sure you have a car. Do you have a car? If you don’t have a car just return to what you were doing. I won’t be able to teach you to drive a car.

“Norm: “I have a car.”

John: “Let’s call that car normsCar. Check to see if your car has a door. We’ll call it normsCarDoor. Does normsCar have a normsCarDoor?”

Norm: “Yes”

John: “Great, but if you don’t just skip the rest of this – I won’t be able to tell you how to drive a car.”

Norm: “I have a car door.”

John: “Once you open the door check and see if it has a seat we’ll call normsComfySeat. If you don’t have a seat skip the rest of this – I won’t be able to tell you how to drive a car.”

Norm: “I have a seat”

John: “Things have changed scope a bit – can you check whether you have a door again for me – we called it normsCarDoor?”………..

I think you can see where this is going. Classes have contracts. It should be reasonable to talk about a car that always has a door and a seat without having to neurotically check at all times.

Unlike the real world, when we code we make new things up all the time. So even though seats and doors can be reasonably inferred on cars the things that a UserAgentControl may or may not have probably won’t be obvious. Does the UserAgentControl have a ThrottleManager all the time? If I have a non-nullable class type I can check by looking at the class. If I get it wrong for some reason maybe I can have the compiler issue a warning. Maybe I can be forced to “conjugate” it with a special syntax every time I use it to help me remember (“.” vs. “->” in C++). Or a naming convention.

Why is this such an overarching problem?

It may seem like griping over a simple check for a things existence. Except this check can conceivably apply to each and every object I can have (or don’t have). This makes the issue enormous. A lot of the time programmers put in checks for null when it seems appropriate or when unit testing throws an exception at run-time. This sounds a whole lot like something statically typed languages are supposed to help us protect ourselves against. The unchecked null is by far the most common runtime error and the “overchecked” null one of the most prevalent unintentional code obfuscation techniques.

Solutions that don’t involve changing existing languages:

  1. If your language supports the concept out of the box (C++, Spec#, F#, OCaml, Nice, etc.) or it can be built with templates or other mechanisms use not-nullable types. Use these types whenever possible. If you’re language doesn’t support it then use number 2 alone.
  2. Create a simple naming convention (don’t go hungarian on me) to discern what is nullable and what isn’t. This is a fundamental concept and it should be obvious every time a value, instance or object is accessed. Use this to compensate for your languages deficiency in a similar manner to prefixing private variables in languages that don’t have mechanisms for hiding access to variables and member functions.
  3. Check for null as early as possible in the food chain and prefer to call methods that are granular and don’t have to check themselves. This means make every parameter passed to a routine get used. If a parameter is optional write a new routine. Make the types passed to these routines be not-nullable if the language supports it.
  4. If it makes sense prefer empty collections to collections containing nothing but null for obvious reasons.

<

p>Wrap-up:

There is no perfect solution. But so many times in code I see null being checked or not checked and am left wondering. Is the check gratuitous? Is this a runtime error waiting in the wings? And unless it’s commented it or I check it’s usage – I get that typically uneasy feeling. Bound to catch it at runtime with unit testing. This is not the answer I want to hear.

I think a few simple practices and conventions could get this off our plate so we can get back to the problem at hand. Solving actual problems.

Afterthoughts:

Some other articles that beat me to the punch on some of these concepts:

  1. Let’s Reconsider That Offensive Coding from Michael Feathers.
  2. Null:The Runtime Error Generator by Blaine Buxton

Empty containers solve this problem nicely for return types. Also as many readers have mentioned, monads are extremely useful in solving this problem as well. But once again, stop propagating unneeded nulls and similar “safer null-like” structures as soon as possible. As Michael Feathers states – it’s just offensive code.

16 thoughts on “How null breaks polymorphism: or the problem with null: part 2

  1. Mike
    #

    Thanks for article. The car example really drives the point (ok, bad pun). I’ve been avoiding null for years without ever really understanding why. It just didn’t feel right for some reason. Returning null feels a bit like an error return code. Exceptions are a better mechanism for reporting errors in my opinion.

    Reply
  2. Nate Kohari
    #

    I agree with the general premise of your articles, in that null checking is irritating and causes ugly code. However, I believe it’s a natural aspect of programming, because there has to be a way to specify a “lack” of something. I disagree with your previous assertion that checking for null is equivalent to a type check. While a null instance doesn’t satisfy the contract of the type, it does act as a placeholder for a value that does.

    As you alluded to in the previous article, what you’re really suggesting is implementing the null object pattern for every type that you define. However, that would result in MUCH less elegant code than using the null value. There are many methods (particularly ones with side-effects, that do more than simply calculate a value) that simply cannot be reduced to their null object equivalents. So, the implementation of these methods on the null object would likely be to fault (throw an exception, etc.) each time a method was accessed.

    You also aren’t considering the fact that using null is not always an error. The null value is often used to improve performance, for delaying resolution of dependencies when it isn’t certain that client code will require them.

    Consider this (simple) example in C#:

    public class MyType
    {
    private MyDependency _dependency;

    public MyDependency Dependency
    {
    get
    {
    if (_dependency == null)
    _dependency = ResolveDependency();
    return _dependency;
    }
    }

    private MyDependency ResolveDependency()
    { … }
    }

    It would be much more difficult to implement this type of “lazy resolution” without null. If we were using the null object pattern instead, for example, it would require an actual type check (using the ‘is’ operator, for example), which would be much more expensive than the value check.

    All in all, like I said, I agree that null can be bad if used incorrectly, but when used correctly and safely, it can be a useful tool.

    Reply
  3. Stephen Colebourne
    #

    I posted one possible syntax for enhancing Java with NPE-avoidance about a year ago:
    http://www.jroller.com/scolebourne/entry/adding_auto_null_checks_to

    public isFromBritain(Profile profile) {
    return profile#getPerson()#getAddress()#getCountry()#equals(“GB”);
    }

    The problem with Java is not that is has nulls, that’s fine by me. Instead, the problem is that Java doesn’t have all the language syntax tools to be able to control and manipulate null.

    Reply
  4. Steve Riley Post author
    #

    Stephen,

    I agree – nulls are a part of languages and will continue to be. Having a separate syntax for this fundamental non-nullable distinction seems like a perfect solution and one I would love to see in C#, Java and C++. If C++ allowed reassignment of references then its . vs -> would already fit quite nicely.

    I’ll take a closer look at your link. In the last few days I have found quite a few sites that in the last year have explored this issue in depth. I’m looking forward to looking at everyone’s thoughts on the matter.

    Reply
  5. Steve Riley Post author
    #

    Nate,

    I think the null object pattern can fit the bill in a small number of cases. In other cases I would love to see empty container syntax or some other mechanism to make it clear that a check is in fact needed.

    I agree – when objects don’t exist the dependency shouldn’t exist either. I’d just like to be able to see an instance and know whether or not I should check it for null based on some immediately recognizable mechanism. Without this looking at code you will always ave the tendency to want to double check when working in team environments.

    Too many coders don’t check when they should, or do check when the class will never in fact allow it to be null.

    Reply
  6. Max Bolingbroke
    #

    Spec# (a C# dialect) lets you have non-nullity like so:

    void MyFunctionTakingSomethingThatIsNotNull(object! myObject)
    {
    myObject.GetType(); // No NRE!
    }

    As well as letting you do a lot of other cool stuff with class invariants. Incidentall, the “type?” syntax (for nullable value types, e.g. int? myInteger = null) that was introduced in C# 2.0 actually originated from Spec#, so I’m hoping that this feature makes the leap too :-). By the way, this issue (and some other interesting correctness-related ones) was addressed by Tim Sweeney in his excellent “The Next Mainstream Programming Language” talk, which you can view here if you have not already seem it.

    Reply
  7. Steve Riley Post author
    #

    Max,

    That Tim Sweeney talk was an excellent one. I need to look at it again – I remember him making some truly excellent points.

    I would love to see the Spec# syntax make it into C#.

    I know it’s a very minor gripe but I’d still love to it expressed in the dereference as well (. syntax for nullable types in C# and -> perhaps for non-nullable – even though this reverses C++ handling of the same:) ).

    Reply
  8. George
    #

    I don’t see how the concept of null is so bad. I think it represents the same thing as the number Zero. I bet lots of people’s heads started to hurt when the idea of Zero came along. Just means the absence of any number.

    Thus, Null is the absence of any data. It means that instead of getting a reference that points to a data location, we’ve been given a reference that points to nothing. That is called null.

    I do agree that sometimes coding with checking nulls is a bit tiring but its quite simple to do. Sometimes it is useful. Sometimes you want the user to send a null and it will reset something. Or if someone gives you a string of values and the result must be an integer, but you can’t send any integer back as it may represent something (even Integer.MAX_VALUE and Integer.MIN_VALUE) you send a null, meaning the values given didn’t produce any result, a pointer to Nothingness :)

    Reply
  9. Steve Riley Post author
    #

    George,

    But zero is in fact interchangeable with other numbers. All of the “member functions” of real numbers have support. Zero can be used where other real numbers can and produce meaningful results.

    If we throw out type checking all together, at all levels anything type could be assigned to any reference. By your logic that would merely require checking the type before calling the function to make sure it is supported as a minor inconvenience. Unfortunately this takes away the entire point of polymorphism and dynamic dispatch.

    If a value assigned to a pointer or reference has to be checked to make sure its member functions can be called it defeats the purpose. The pointer is no longer typesafe because it can hold values not of that type.

    Type safety exists to abstract out the blocks of if else statements that were so prevalent in structural methodologies to strongly prefer virtual functions and inheritance techniques.

    Reply
  10. Tony Morris
    #

    “There are plenty of times when a function may or may not return a value.”

    Here is your crucial mistake. I think that once you realise how devastating this mistake is to the remainder of your article, your higher understanding will compel you to shift your position quite dramatically.

    There is not a function that may not return a value. Not even in the most imperative of imperative programs. Try reasoning about such a function and you will (inadvertently or not) introduce a return value. The fact that it is not expressed in the code is a different (and unfortunate) matter.

    Reply
  11. Steve Riley Post author
    #

    Tony,

    When I say doesn\’t return a value – I am speaking of null.  A monad should be used instead of null in these cases as others have pointed out.  These can be easily created with templates in C++.
    The main point I\’m trying to make – subtler points aside – is that null is grossly overused and abused because programmers don\’t take the time to make separate conditional logic into dynamic dispatch – one of the crown gems of OOP. In OOP 101 you learn to prefer dynamic dispatch to conditional checks and endless if…then…else constructs. Refactor out your nulls into more sensible points of dynamic dispatch.

    Checking every single pointer or reference for null on an ongoing basis is an obvious sign of this sickness. In many languages it is possible to create references and pointers that can\’t be null. There is a reason for this. And it\’s a very good one.

    Reply
  12. Reinier Zwitserloot
    #

    Here’s what I want:

    A type-system level distinction between values which may be null, and values which never should be. In java you see tentative steps (With e.g. the IDEA IDE compile-time checking and enforcing a @NonNull annotation) but to do it right, I propose the following:

    Adding a ! as a suffix to *ANY* type enforces it to never ever be null, compile-time checked. e.g:

    public String! toString();

    You can safely pass either String! or String into something that requires a “String” or an “Object”, but you can only pass a String!, not a String, into “String!” or “Object!”.

    You can cast to ! types, obviously.
    This cast won’t actually be used nearly as often as one might think, but on the ‘border’, e.g. code programmed with this using a library that does not, it’s useful.

    The type of literals is obvious (e.g. “hello” has a type of String!). Solves this entire problem quite simply.

    The only annoyance is the need to have a sane default on declares. In java, a variable is valid only inside its scope. Thus:

    if ( something ) {
    int x = 5;
    } else {
    int x = 10;
    }

    Doesn’t do what you think – it’s valid but x is not accessible anywhere except inside the if. Fortunately java has a mechanism already in place for this for local variables – it just gets a bit annoying for fields and constructors. e.g, you’d have to go:

    public class Something {
    private String! bla = “”; //note the = “” – this is required
    public Something(String! name) { bla = name; }
    }

    There’s a mechamism here, again, fortunately – namely that of the ‘final’ check. You can leave a ‘final’ attribute blank as long as all constructors in the class give it a value exactly once in all statically analysed possible code paths. You can pull the same trick with ! – it must get a value (which obviously must be compile-time verifiable to never be null) at least once in all possible constructor code paths, and until that time, you aren’t allowed to access the thing.

    Reply
  13. Steve Riley Post author
    #

    Reinier,

    I agree completely. Having a simple notation to specify non-null to the compiler would be very clear and exactly what we need.

    Reply
  14. Stephen Colebourne
    #

    As well as the Null-ignore blog – I also wrote about Null-safe types.

    This would allow a type to be defined as ‘not allowed to be null’ using the # modifier:

    public #Person lookupPerson(#String personId) { … }

    This is the most powerful way to attack the villian that is NPE.

    Reply
  15. Zubair
    #

    I tend to write functions with arguments that are all required (and try to make that obvious) and then assume that the args are not null. In this case, if there is a null, an exception will be thrown and caught later but this should be a rare event.

    Find this is better than the alternative of ‘ugly’ code.

    Reply

Leave a Reply