How null breaks polymorphism: or the problem with null: part 1

Filed under: Development Philosophy, Programming — by Steve Riley at 12:19 am on Sunday, December 31, 2006

Preface: After talking to a number of people I realize that somehow I managed to misrepresent myself with respect to type systems in this article. This article is an attack on null, and to point out that null is still problematic in many (if not all) strongly typed languages. Many who prefer strong types and static types feel they are more immune to certain runtime type related inappropriate behaviors. I feel the ability to use null in most languages in place of an object breaks polymorphism in the most extreme possible way. I am in no way implying that dynamic or weak type systems are better at handling these issues. As for me - I prefer languages with stronger compile time type checking.

This is a difficult concept for a lot of died in the wool strong staticly typed OO programmers to fully digest and accept. There is an immense sense of pride in the strong staticly typed community about the fact that unlike untyped languages, strong staticly typed languages protect them from run-time errors related to type mismatches and unavailable methods. Unless you do a dynamic type cast (frowned upon heavily) you should be safe from at least this broad class of error. But they are wrong. Type mismatches and unavailable methods occur all the time in strong staticly typed languages. And it is a common form on runtime surprise. What causes this common problem: the null which can be used with any type yet breaks polymorphism with every single one.

Unlike types in loosely typed languages the null is guaranteed not work polymorphicly thus requiring a specific type check. Did I say type check? But I have no dynamic casts, I’m following all the rules. Why should I have any type checks? Checking for null is a type check. It’s the mother of all type checks. Instead of having code littered with conditional checks for types and branches based on those types (an OO worst practice) you have code littered with conditional checks for null having branches based on whether it is null or not.

Now granted, life in a world without nulls isn’t easy and I use null often myself. It’s too tempting to use this magic value instead of writing code more appropriately. Some will mention the null object design pattern that “does nothing” with pride as a solution to this problem. These are in fact polymorphic. The only issue is that null objects only work in special circumstances. If you really don’t have a thing you shouldn’t be pretending you do and having it do nothing. You should have a separate chain of logic that doesn’t use the thing you don’t have.

I have talked to a number of coders that think that removing null from a majority of their code would be difficult to impossible. An difficult to grok kind of problem perhaps but intractable, no. Consider the following function:

int DoSomethingSpecific( int x, int y, int z);

Now I will asked the magic question. Do you check z for null in case you don’t have it? (or x or y for that matter). In C++ that isn’t even possible because it’s passed by value. If an appropriate default exists for z you may set z to that default before it is called. However plenty of times that concept isn’t the one you are looking for. What do you do? You simply write another function that doesn’t take z.

int DoSomethingSpecific(int x, int y);

Now let’s use generic objects:

int DoSomethingSpecific(object x, object y, object z);

int DoSomethingSpecific(object x, object y);

Using this approach doesn’t break polymorphism. You only call the appropriate function when you actual have the parameters in question.

Of course this brings us back to a more fundamental problem. The concept of null is so burned in to most OO languages that visual inspection of code reveals that most any object should be nullable and thus checked for null. C++ has a way around this with references that can not be null or checked for null (yes I know many compilers will let you assign null but you make clear your intent in using a reference:it should not be null). The C++ reference being used this way is at best an afterthought in the language. These references can’t be reassigned and thus are limited to incoming parameters on function calls in many cases.

Even if you create a class which prohibits non-null assignment casual readers of your code in many languages will miss this fact and do gratuitous checks for null anyways; defeating much of the purpose. The key is supporting syntax that makes clear the fact that an object can not be null. But that discussion is for another day.

In part two of this article I will explain many of the misconceptions and supposedly intractable issues related to removing null. It’s not as hard as you might at first think. I will also further explore the syntax issue, or without language support at least a possible naming convention.

20 Comments »

101

Comment by Oliver

December 31, 2006 @ 12:42 am

I agree with you, null is a plague upon all our houses :-/

Personally I would like a language in which null has to be explicitly allowed (Haskell basically does this — the Maybe data type basically allows nullable values). I dare say if anyone eventually creates a such a language it will probably default to allowing null though

–Oliver

102

Comment by eulerbernays

December 31, 2006 @ 1:20 am

Agreed but you can do things to limit the problems. Its a good practice as a team rule to return empty objects where null might come into play. For instance collections, return an empty set if there are no records. Then, on the consumer side the code checks for a count. Same with other types. Strings, return empty string “” not null, or int return 0 or -1. Rules setup front limit the wacky ways of dealing with it.

106

Comment by Ricky Clarkson

December 31, 2006 @ 5:31 am

A commenter mentioned the Maybe type from Haskell. I cover its implementation and use in Java on my blog.

I find it very useful to disallow null, minimise the places where I think it is needed, then convert those from ‘might be nulls’ to Maybes. There still are some cases where null is useful, simply because Java doesn’t have tail-call optimisation, but only as intermediate values inside a method, not as fields of an object.

108

Comment by Reg Braithwaite

December 31, 2006 @ 7:58 am

Null is an artefact of Java’s “backward compatibility” with C++. C++ ‘inherited’ null from C, and if you look at C and C++ programs, mistakenly dereferencing null pointers is a significant problem.

Compare and contrast null to the Undefined Object in Smalltalk or nil in Ruby. I think you’ll agree that it was one of those “let’s win over the C++ programmers” decisions that bites Java where it sits down.

109

Comment by Jonathan Allen

December 31, 2006 @ 11:16 am

I dream of a day where reference variables, parameters, and return types can be tagged as nullable or not nullable like in SQL. I think 75+% of runtime errors I see in large projects (20+ developers) are NullReferenceException.

111

Comment by Isaac Gouy

December 31, 2006 @ 12:48 pm

Please don’t use vague terms like loosely typed, or confuse dynamically type checked languages with untyped languages, or conflate what might be true of Java or C++ with what’s true of the very different type systems available in other statically type checked languages.

The difficult concept is that support for dynamic type checking by-itself is not what classifies a language as dynamically type checked.
We classify a language as dynamically type checked when it does not support static type checking and does support dynamic type checking.
We classify a language as statically type checked when it does support static type checking - the language might also support dynamic type checking.

Checking for null is a type check
Is checking for 467 a type check?

Two of the issues you brought up are
- separate non-null and null types
- dispatch on null at runtime

Here’s a statically type checked language code snippet using a List type that excludes nulls and a ?List type that includes nulls, and using dispatch on null at runtime:
<T> int lazySize(?List<T> list); lazySize(List list) = list.size(); lazySize(null) = 0;


void main(String[] args){

   let a = new LinkedList();

   a.add(500);

   println(a.lazySize());
   let b = new LinkedList();

   b.add("Five Hundred");

   println(b.lazySize());
   let c = null;

   println(c.lazySize());

}
$ /opt/nice-0.9.12/bin/nicec --sourcepath .. nicenull -a t.jar

nice.lang: parsing

nicenull: parsing

nicenull: typechecking

nicenull: generating code

nicenull: linking

nicenull: writing in archive

nice.lang: writing in archive

$ java -jar t.jar 1 1 0

The Nice Programming Language

112

Comment by Steve Riley

December 31, 2006 @ 1:25 pm

Isaac,

You are absolutely right. For purposes of this writeup I realize I should have avoided talking about strong vs. loose and static vs. dynamic at all. It really doesn’t make a difference.

I think null is an important concept though - one thing I would like is clear language syntax when a type allows it or doesn’t. When an object is dereferenced syntax revealing whether it is nullable would be a definite plus, IMHO. At the very least the compiler should complain when a null check is inserted and “shouldn’t be allowed” by a non-null type.

C++ supports separate null and non-null types out of the box - not as robustly as I’d like perhaps. Through templates it certainly does. F# also supports non-null types out of the box as do others.

This article was more about clearing up misconceptions. In my mind null shouldn’t be able to do anything - it is null and there can be no appropriate behavior for it by definition. Having a null “fall through” behavior that happens to be benign seems wrong to me.

Null is not polymorphic because it can’t reasonably by expected to fill the contract required of the class it supposedly represents. If it does it should be called something other than null. Creating dynamic dispatch on null seems as flawed to me as the null object pattern. Although at least the null object pattern is a genuine class and easily discernible from null itself.

113

Comment by Steve Riley

December 31, 2006 @ 1:26 pm

Oliver,

Haskell certainly seems interesting to me - I’ll have to give it a look. It’s popped on my radar over the years and I’ve always been curious about all the excitement it has generated.

114

Comment by Steve Riley

December 31, 2006 @ 1:29 pm

Eulerbernays,

I especially like the idea of returning empty sets where appropriate. This makes it very clear what needs to be done. An empty set can be appropriately handed.

I’m generally against magic numbers (in theory) for the same reason I dislike null. But when I’m prototyping I certainly have dropped a -1 of two into code.

115

Comment by Steve Riley

December 31, 2006 @ 1:32 pm

Ricky,

I agree when null is appropriate it is sometimes very useful as other methods may prove too cumbersome. Having the choice of using non-null types is useful across the board. I really do like C++’s . vs -> syntax in making things very obvious. When you use a dot operator you know that you shouldn’t be checking for null. It may be little more than syntactic sugar - but it is nice.

116

Comment by Steve Riley

December 31, 2006 @ 1:35 pm

Johnathan,

I agree and this is one of the main points I’m pushing. When a huge number of runtime errors are because of not handling nulls properly it seems that the language could afford to help out in every way it can to allow you to prevent it through explicit type definition and syntactic clues.

120

Comment by Isaac Gouy

December 31, 2006 @ 5:15 pm

Steve Riley replied …one thing I would like is clear language syntax when a type allows it or doesn’t
Aren’t ?List and List clear enough?

Steve Riley replied At the very least the compiler should complain when a null check is inserted and “shouldn’t be allowed” by a non-null type.
void main(String[] args){ ?List d = new LinkedList(); if (d == null) println("Should not be null");List e = new LinkedList(); if (e == null) println("Should not be null"); // line 6 }
$ /opt/nice-0.9.12/bin/nicec --sourcepath .. nicenull -a t.jar nice.lang: parsing nicenull: parsing nicenull: typechecking

~/tmp/nicenull/test.nice: line 6, column 8: warning: Comparing a non-null value with null nicenull: generating code nicenull: linking nicenull: writing in archive nice.lang: writing in archive compilation completed with 1 warning

Steve Riley replied In my mind null shouldn’t be able to do anything - it is null and there can be no appropriate behavior for it by definition.
By the definition you have in mind, other people may have different definitions in mind
We still need some argument that the definition you have in mind is in a way more useful than other possibilities.

121

Comment by Jonathan Feinberg

December 31, 2006 @ 6:21 pm

See also F#/OCaml 'a option, a type that accepts the values None and Some(foo), where foo is of type ‘a.

122

Comment by Steve Riley

December 31, 2006 @ 9:28 pm

Issac,

Your example does show the compiler catching the null and giving a warning. I think null is such a fundamentally used paradigm that I’d love to see a different syntax imposed at each reference as well though. Similar to C++.

With respect to null I stand by my argument. When you don’t have something, having it do something (even if it only benignly pretends to do nothing) seems logically inconsistent.

Great comments though - and you’ve turned me on to another language I need to take a look at.

132

Comment by Jude

January 2, 2007 @ 10:44 pm

I agree that null can cause a lot of problems , and an inherent NULL Pattern in the language subsystem would help a lot . It would be so nice , if all objects could have their NULL counter parts pre defined . But the Time being Null Pattern still comes to the rescue .

133

Comment by Steve Riley

January 2, 2007 @ 11:07 pm

Jude,

The NULL pattern can be useful in some cases - unfortunately not all though. The null pattern is only useful in those few cases where the existence of an object is truly decorative - meaning the block of code doesn’t logically break when the object doesn’t exist. Otherwise it just quietly breaks the intent of a code block without providing run-time or compile time errors. This can lead to even more difficult to track down problems later.

134

Comment by Sam

January 3, 2007 @ 10:59 am

I think this problem has been approached by languages in at least two ways. One is to make null an object like Ruby’s nil as stated by the earlier commenter. Null checks are replaced by respond_to? checks and even nil can have methods so as to fufill some interface if it needs to.

The other solution is like Haskell’s apprach where the type information is very strict. If a function can return null the common practice is to use a Maybe monad for the return type. Maybe is a container for either Nothing or Just a, where a is any other type.

On one hand, the Ruby type system keeps things very flat (non treelike) for the programmer. A type is essentially an object with methods or the lack of methods. Haskell, on the other hand, forces the programmer to think in terms of a type tree all the time.

In Ruby I get lots of nil errors. In Haskell I still have trouble getting anything done, I admit, but the compiler catches ALL type errors.

148

Comment by Enrique

January 12, 2007 @ 8:19 am

Yes, null must be forbidden.

A variable must have a defined value or not exist at all.

Unfortunatelly languages are not perfect. For example the try/catch forces to define variables outside the try block (probably with null value) just in case we want to manipulate them in the catch/finally blocks. Asigning visibility scope for try/catch/finally blocks is a big mistake, since error control ought to be orthogonal to the “normal flow”, but is a common mistake in many languages like C++, Java or C#.

Notice next pseudo code:
01 Connection con = null; ResultSet rs = null; 02 try{ 03 Connection con = getConnection(....) 04 rs = con.execute.... 05 }catch(SQLException ...){ 06 ... 07 }finally{ 08 if (con!=null) { 09 try{ 10 con.close() 11 }catch(....){ 12 logger.warn(....) 13 } 14 }
Without the nonsense visibility scope it would be:

01 try{ 02 Connection con = getConnection(....) 03 ResultSet rs = con.execute.... 04 }catch(SQLException ...){ 05 ... 06 }finally{ 07 try{ 08 con.close(); 09 }catch(....){ 10 logger.warn(....) 11 }

3 lines shorter, no need for “if (rs!=null)”/”if (con!=null)” checking, and no null/undefined variables around !!!

154

Comment by Steve Riley

January 13, 2007 @ 1:15 am

Enrique,

Try, finally and scope….. - yep the scoping is stupid - you are wrong there. Take a look at the D programming language - this page explains exactly the argument you mention and the need for a \’dummy object\’ (or nulled reference in your case). Try and finally should share more than just scope but be colocated as well.

// snippet of D

Transaction abc()
{
Foo f;
Bar b;
f = dofoo();
scope(failure) dofoo_undo(f);
b = dobar();
return Transaction(f, b);
}
The dofoo_undo(f) only is executed if the scope is exited via an exception. The unwinding code is minimal and kept aesthetically where it belongs. It scales up in a natural way to more complex transactions.

That is much cleaner.

158

Comment by Wolfie

January 15, 2007 @ 3:36 am

I agree with Jon (above commentator) that a simple solution for Java is an SQL like declaratory such as null/not null to fix the leak, but as has been pointed out above that would have to be implemented in conjunction with a fix to the scoping of the try/catch/finally syntax [I hate that one].

As for method parameter redundancy its simply sloppy not to provide alternative forms of the method to avoid the scenario entirely.

I agree that there is some improvement required with null but its still a valid and useful part of the language and I don’t really see that it needs to be “type-safe” exactly. Sometimes you need to say in code somehow : “I don’t know this just yet” or “scrub that its not relevant” and null fits the bill just fine.

There is only so far a language can go to prevent sloppy/lazy development and I’m not keen on more core language bloat than is already present.

RSS feed for comments on this post. TrackBack URI

Programming Matters

My thoughts on all things technical, programming and otherwise

How null breaks polymorphism: or the problem with null: part 1

20 Comments »

Leave a comment

Recent Posts

Archives

Categories

Eureka 3D, Inc.

Meta