Developing Large Projects in F#

After trying to build a slightly above-toy-size project in F#, I came to the conclusion that with current tools it would be quite difficult to maintain a project of even moderate complexity. Unfortunately, I don’t believe this situation will dramatically improve any time soon.

Maintaining F# projects is hindered by two main obstacles:

* Necessity to maintain a proper source file order
* Awkwardness of assembly references

File Order

In F# the order of files in a project is significant. It seems that F# completely lacks the concept of a linker. The F# compiler takes all source files in the project and compiles them in order, as if they were a single large file. Of course, this may not be how it exactly works internally, but this is the way it looks from the outside.

A type (variable, module, class) may be used only after its definition.

#light

let foo = new Foo() // does not work - type Foo is not defined

type Foo =
 class
   new() = {}
 end

let bar = new Foo()

There are two very bad consequences that arise from this:

* You always need to worry about the file order
* Since no other major language cares about it, reordering files is not properly supported by Visual Studio

You Always Need to Worry about the File Order

Whenever something in file B uses something in file A, file A must precede file B in the project. This means that a developer must always keep a mental picture of the entire project dependency graph. Needless to say, this graph grows exponentially with project size, and it is not explicitly stored anywhere. Since each file may (and normally does) define multiple entities, all this becomes even more complicated. If you inherit a big project you don’t know much about, I don’t want to be in your shoes. Let say you want to add this little class or function. Between which of the 133 source files do you insert it?

Maintaining File Order Is Hard

Visual studio does not have built-in support for reordering files, even with F# plugin. If you add a file, it goes to the end of the project. If you rename a file, it goes to the end of the project. Current recommendations for changing the file order are outright ridiculous, not to mention time consuming and error prone:

To change the order of the files in the project tree you must unload the project (right click on the project in the solution explorer), choose “edit ProjectName.fsharpp” on the closed project node (again in the solution explorer), make your edits then reload the project.

Assembly References

You cannot normally reference an F# project from C#, or C# project from F#. By “normally” I mean going to the “references” folder of your project and choosing a project from the list.

Assembly references are added via compiler options, including the path information, or in the source files themselves (the #r directive). This is not as bad as reordering files, but it still is an impediment.

Summary

The file order problem makes writing large F# projects impractical. The assembly reference problem only slightly adds to it. After all, you usually have many more files than assemblies.

I believe that the technical issue of nicely reordering files in Visual Studio may eventually be resolved. It may be not easy though. Since none of the major languages require file reordering, Visual Studio may “resist” that kind of changes. However, even if Visual Studio cooperates, the dependency of F# compiler on the exact file order will remain, and this makes dealing with large projects difficult. I am afraid that this dependency is baked into the language: I saw some references that it is required for type inference. It seems unlikely that the dependency on the file order could go away completely.

So, the prospects of F# becoming big-project-friendly are not very good.

Posted in

7 Comments


    1. I am very intersted on how you are managing this.

      For your 250kLOC codebase, howmany files do you have then? Howmany lines per file? Are they equal size?

      Reply

      1. Michiel, this is an interesting question, but that comment you are replying to was made 9 years ago…

        Reply

  1. I am glad to hear that. How do you solve the problems outlined above?

    Reply

  2. We ensure that the file order is kept correct according to the dependencies by hand. That requires very little work because the dependencies rarely change.

    There are many advantages to this approach as well. With a well-defined evaluation order there is no chance of erroneous accesses to not-yet-initialized objects that would otherwise produce run-time errors.

    There is also the problem that definitions split between compilation units cannot be mutually recursive. That is easily solved using a parameterization-based technique known as “untying the recursive knot”. This was described in the last OCaml Journal article.

    Although we have both C# and F# projects we have not needed to mix them so we have not encountered the referencing issue that you refer to. We also winding down our C# developments because F# is already more lucrative.

    Reply

  3. The file order and the fact that a type may only be used after its definition is one of my favourite F# features and I hope it never goes away. It forces you to architect in a way that avoids circular dependencies. When coming to a new F# project it is a pleasure to open it up and immediately see all the layers of code involved, starting from the core code at the top working your way down to the program boundaries at the bottom.

    More info on why this is one of the best F# features:
    https://fsharpforfunandprofit.com/posts/cyclic-dependencies/
    http://blog.ploeh.dk/2015/04/15/c-will-eventually-get-all-f-features-right/

    From Mark Seemann:

    The F# compiler doesn’t allow circular dependencies. You can’t use a type or a function before you’ve defined it. This may seem like a restriction, but is perhaps the most important quality of F#. Cyclic dependencies are closely correlated with coupling, and coupling is the deadliest maintainability killer of code bases.

    In C# and most other languages, you can define dependency cycles, and the compiler makes it easy for you. In F#, the compiler makes it impossible.

    Studies show that F# projects have fewer and smaller cycles, and that there are types of cycles (motifs) you don’t see at all in F# code bases.

    Reply

Leave a Reply to Jon Harrop Cancel reply

Your email address will not be published. Required fields are marked *