F# type providers – as if by magic…

F# 3 will introduce a new feature called type providers. A type provider is a gadget that creates new types for you. The F# team, reflecting their focus on data access, have shipped type providers for databases, DBML and EDMX files, and WSDL and OData services. If you use one of these type providers in your F# program, it will give you strong-typed access to the database or service. So from this point of view it’s a lot like the code generator that runs when you create a DBML or EDMX file, or do an Add Service Reference to a Web service or OData service.

But F# type providers can create any kind of type, and they don’t need to consult an external resource to do it. This means a type provider can be used to automagically replace repetitive boilerplate code with a simple declaration.

Consider a vector maths library. Users will want to be able to create vectors with various numbers of dimensions — 2D vectors, 3D vectors, etc. And they might even want to be able to create vectors with the same number of dimensions but different names (for example, Horizontal/Vertical instead of X/Y). In C#, you’d have to write out each vector type by hand. In F# 3, you can ship a Vector type provider, and users can create their desired vector types from that:

type Vector2D = Vector<"X", "Y">
type Vector3D = Vector<"X", "Y", "Z">
type VectorHV = Vector<"Horizontal", "Vertical">
 
let v1 = Vector2D()
v1.X <- 100.0
v1.Y <- 200.0
let v2 = VectorHV()
v2.Horizontal <- 300.0
v2.Vertical <- 400.0

Each of the created types is a real compile-time type, with type safety enforced by the F# compiler. For example, if the type provider supplies a dot product operation with vectors of the same kind (only), the F# compiler will check that the user doesn’t take dot products between vectors of different kinds:

let v1 = Vector2D()
let v2 = Vector2D()
let vh = VectorHV()
 
let v1v2 = v1.DotProduct(v2)  // okay
let v1vh = v1.DotProduct(vh)  // type checks

The created vector types work exactly as if you had written them longhand, but without all the effort, and without the limitation of having to guess what types your users will need.

Implementing a type provider

The F# team have indicated that they’ll be shipping the source for some sample type providers soon, which will include some handy helpers which should rather simplify the process of writing type providers, but for now it’s still not too difficult to implement type providers directly against the interfaces. Here’s an outline, albeit with plenty of rough edges.

The first thing you need to do is create a F# 3.0 library project. At the moment, this requires the Visual Studio 11 preview (the full version, not the Metro apps version). And you’ll need to apply TypeProviderAssemblyAttribute to that library:

[<assembly: TypeProviderAssembly>]
do ()

Next, you’ll need to implement the type provider class itself. This is a normal F# class which implements the ITypeProvider and IProvidedNamespace interfaces, and has the TypeProviderAttribute:

[<TypeProvider>]
type VectorProvider() =
  interface ITypeProvider with // ...
  interface IProvidedNamespace with // ...

A type provider can provide multiple kinds of types, but for this example I’m going to assume that the provider only does Vector types. This keeps my implementation simple because I don’t need to keep dispatching on ‘what kind of type is the user creating.’ This makes it pretty easy to implement IProvidedNamespace:

// Cheating for simplicity
type Vector() = inherit obj()
 
[<TypeProvider>]
type VectorProvider() =
  interface IProvidedNamespace with
    member this.ResolveTypeName(typeName) = typeof<Vector>
    member this.NamespaceName with get() = "Mindscape.Vectorama"
    member this.GetNestedNamespaces() = [| |]
    member this.GetTypes() = [| typeof<Vector> |]

What’s that do-nothing Vector type for? Well, when the user of the type provider writes Vector<…>, F# wants to interpret “Vector” as a type name. The real type providers included with F# 3.0 create synthetic types to carry the desired provider names, but I’m too lazy to do that, so I’ve just created a dummy type to carry the name “Vector” for me.

For a simple demo, much of the ITypeProvider implementation can also be kept trivial:

[<TypeProvider>]
type VectorProvider() =
  let invalidation = new Event<EventHandler, EventArgs>()
  interface ITypeProvider with
    member this.GetNamespaces() = [| this |]
    member this.Dispose() = ()
    [<CLIEvent>]
    member this.Invalidate = invalidation.Publish

The meat of the type provider is in the remaining ITypeProvider methods: GetStaticParameters, ApplyStaticArguments and GetInvokerExpression.

What can the user specify about the type?

GetStaticParameters specifies what parameters the user can pass to the type provider when they create a type. In our case, we want the user to be able to pass the names of the axes of their vector type — that is, the parameters are an arbitrary number of strings. However, GetStaticParameters doesn’t have a way to indicate “any number of parameters,” so we have to settle for, for example, “you can pass 7 parameters, but they’re all optional.” (In reality you’d make the first one mandatory, but I want to keep things simple.) If we later need to support 8-dimensional vectors, we can simply update the limit in the type provider. The parameters are going to become property names, so they need to be strings. We’ll create a helper method to produce ParameterInfo objects with the right name, data type and optionality.

module Helpers =
  let stringParameter index defaultVal =
    { new ParameterInfo() with
      override this.Name with get() = sprintf "axis%d" index
      override this.ParameterType with get() = typeof<string>
      override this.Position with get() = 0
      override this.RawDefaultValue with get() = defaultVal
      override this.DefaultValue with get() = defaultVal
      override this.Attributes with get() = ParameterAttributes.Optional
    }
 
open Helpers

and then use this to specify what parameters the user can provide to the Vector type provider:

member this.GetStaticParameters(typeWithoutArguments) =
  [1..7] |> List.map (fun i -> stringParameter i "")
         |> List.toArray

This is enough to enable the user of our library to type Vector<"X", "Y"> with intellisense on the parameters. Now we need to actually create a type to realise the user’s specification.

Building the concrete type

Building the type from the parameters provided by the user is the job of the ApplyStaticArguments method. For the Vector demo, I’ll just create a class with properties named for each of the user parameters, and a DotProduct method. (A better provider would create an immutable value type, but I want to keep it simple.)

At the moment, it seems that F# requires ApplyStaticArguments to create a physical assembly on disk. If you build the type in memory, you get strange path errors. I’m not sure if there’s a way around this, but for now I’ll just live with it and litter my temp directory with temporary assemblies.

So the code generator for vector types looks something like this:

module Helpers =
  let makeClass body name =
    let code = "namespace Mindscape.Vectorama { public class " + name + " {" + Environment.NewLine + body + Environment.NewLine + "} }"
    let dllFile = System.IO.Path.GetTempFileName()
    let csc = new Microsoft.CSharp.CSharpCodeProvider()
    let parameters = new System.CodeDom.Compiler.CompilerParameters()
    parameters.OutputAssembly <- dllFile
    parameters.CompilerOptions <- "/t:library"
    // Ignoring error checking
    let compilerResults = csc.CompileAssemblyFromSource(parameters, [| code |])
    let asm = compilerResults.CompiledAssembly
    asm.GetType("Mindscape.Vectorama." + name)
  let makeVector name argnames =
    let propNames = argnames |> Seq.filter (fun arg -> arg <> null && not (String.IsNullOrWhiteSpace(arg.ToString())))
                             |> Seq.map (fun arg -> arg.ToString())
                             |> Seq.toList
    let props = propNames |> List.map (fun arg -> "public double " + arg + " { get; set; }")
                          |> String.concat Environment.NewLine
    let dotProductBody = propNames |> List.map (fun arg -> sprintf "this.%s * other.%s" arg arg)
                                   |> String.concat " + "
    let dotProduct = sprintf "public double DotProduct(%s other) { return %s; }" name dotProductBody
    let body = props + Environment.NewLine + dotProduct
    makeClass body name

This isn’t as intimidating as it looks! The makeClass function wraps a code string in a namespace and class declaration, compiles it and returns the compiled type. The makeVector function removes empty parameters (remember we had to permit lots of parameters but defaulted unused ones to be blank), then spits out a property for each surviving parameter, and a DotProduct method generated by iterating over the parameters.

It turns out that makeVector is pretty much all we need to implement ApplyStaticArguments. The F# compiler tells us the name of the type the user wants us to provide, and the list of parameters supplied by the user, and all we need to do is pass them on to makeVector:

member this.ApplyStaticArguments(typeWithoutArguments, typeNameWithArguments, staticArguments) =
  makeVector typeNameWithArguments staticArguments

Compiling property and method calls to the created type

We are nearly there! Our type provider now provides types to the user on demand, but the user will still get errors if they try to instantiate or invoke those types. GetInvokerExpression is the final piece of the jigsaw, the piece which tells the F# compiler how to generate code for method invocations on the generated type. It’s not yet clear to me how comprehensive GetInvokerExpression has to be, but for the simple Vector example, we just need to handle a couple of cases — construction and method invocation:

member this.GetInvokerExpression(syntheticMethodBase, parameters) =
  match syntheticMethodBase with
  | :? ConstructorInfo as ctor -> Expression.New(ctor) :> Expression
  | :? MethodInfo as mi -> let args = parameters |> Seq.skip 1 |> Seq.cast<Expression>
                           Expression.Call(parameters.[0], mi, args) :> Expression
  | _ -> let pnames = parameters |> Seq.map (fun p -> p.Name) |> String.concat ", "
         failwith (sprintf "ERROR: Don't know what to do in GetInvokerExpression - %s(%s)" syntheticMethodBase.Name pnames)

The most alarming bit about this code is probably the strange F# type operators. We’re just seeing whether F# wants to compile a constructor or a method call, packaging up the appropriate expression tree, then upcasting to the Expression base type to stop F# whining about type mismatches.

And we’re done!

Well, okay, there’s a lot of tidying up to do. But hopefully the samples promised by the F# team will help with that. And at any rate we now have a Vector library that users can use to define Vector types of arbitrary complexity (well, up to 7 dimensions anyway) with a single line of code:

[<Generate>]
type Vector3D = Vector<"X", "Y", "Z">
 
let v3 = Vector3D()
v3.X <- 1.0
v3.Y <- 2.0
v3.Z <- 3.0

(If you count two lines of code here, well, fair enough, but the GenerateAttribute will be removed before the final release.)

As if by magic…

Obviously, the Vector example is pretty trivial, and for most applications it would be easier to write the boring Vector code longhand rather than writing a whole type provider. But the possibilities for this seem endless. Type providers don’t just have to be about data access. In the same way as dynamic programming already enables users to create their own APIs without having to write their own implementations, type providers allow users to create their own APIs without having to write their own implementations, but this time with all the benefits of type checking, intellisense and so on. And of course type providers can be as sophisticated as you like, far more sophisticated than my simple example.

Oh, and one more thing. Although only the F# compiler can invoke type providers, the types generated by the provider are real, compiled-in types. That means you can use those types from C# or Visual Basic code just by referencing the F# project that declares them.

Yep, this is going to be pretty cool.

Tagged as F#

11 Responses to “F# type providers – as if by magic…”

  • Mindscape Blog » Blog Archive » F# type providers – as if by magic……

    Thank you for submitting this cool story – Trackback from DotNetShoutout…

  • Hi,

    thanks for this really cool article.

    What do you think about adding this as the first working type provider to FSharpx.TypeProviders (http://github.com/fsharp/fsharpx)? I think it would be cool to have such a vector type provider in there and we could improve it further.

    If you are interested then please contact me by mail or http://twitter.com/sforkmann.

    Cheers,
    Steffen

  • thank you very much, this post brings much light on this esoteric subject.

  • ¿Are you sure that F# provided types can be referenced in C#?

    I Tried a F# library referenced by a C# project, and C# ignores the provided type.

    Also you can read Don answers in this post:

    http://geekswithblogs.net/akraus1/archive/2011/06/23/145958.aspx

    Thanks

  • Hi Santiago,

    Thanks for the pointer. C# can’t use type providers directly, but I was able to create a type from the Vector type provider in a F# library, and use that vector type in a C# program:

    // F#
    []
    type Vector3D = Vector< "X", "Y", "Z">

    // C#
    static void Main(string[] args)
    {
    Vectorama.Vector3D v1 = new Vectorama.Vector3D { X = 1, Y = 2, Z = 3 };
    Vectorama.Vector3D v2 = new Vectorama.Vector3D { X = 6, Y = 5, Z = 4 };
    Console.WriteLine(v1.DotProduct(v2));
    Console.ReadKey();
    }

    Maybe I just got lucky and this won’t work in the more general case, but I’m not sure why not. Reflector shows the generated types as real honest-to-goodness classes built into the F# assembly, so there’s no reason they shouldn’t be usable from C# or any other .NET language. (I did notice that a lot of Don’s samples do depend on F#’s ability to have spaces in type or member names, though, which would NOT work in C#.)

  • Hi Steffen, thanks for the feedback — I’ve emailed you.

  • Hi Ivan,

    Thanks for the feedback.

    My provided type is an “erased provided type” or “no-generated” one. Perhaps that’s why C# isn’t aware of it. Trying to solve the problem, I have created a tricky F# function:

    type FType=ProvidedType //no parameters yet

    let getFType=FType()

    C# intellisense sees the tricky function and shows the inner ProvidedType returned (but not FType). It also shows an error saying that the Provider assembly must be referenced. If the Provider assembly is referenced, C# is ok with ProvidedType (and ignores FType).

  • Hi Ivan,

    Now I have a working Generated Provided Type with parameters. And C# can use from the intermediate F# library.

    Note: my provider is coded in C#

    Thanks

  • […] This post contains the best example of a type provider implementation that I have seen so far, showing how vector types could be provided for n-dimensional vectors where the n is specified in the source code. LD_AddCustomAttr("AdOpt", "1"); LD_AddCustomAttr("Origin", "other"); LD_AddCustomAttr("theme_bg", "ffffff"); LD_AddCustomAttr("theme_text", "333333"); LD_AddCustomAttr("theme_link", "0066cc"); LD_AddCustomAttr("theme_border", "f2f7fc"); LD_AddCustomAttr("theme_url", "ff4b33"); LD_AddCustomAttr("LangId", "1"); LD_AddCustomAttr("Tag", "computers-and-internet"); LD_AddSlot("LD_ROS_300-WEB"); LD_GetBids(); Like this:LikeBe the first to like this post. This entry was posted in Computers and Internet. Bookmark the permalink. ← COM is coming home […]

  • I update this code to work with the final bits of F# 3.0 and Visual Studio 2012. Thanks for the post!
    http://blog.ctaggart.com/2012/08/how-do-i-create-f-type-provider-that.html

  • […] FSharpx.TypeProviders.Math which contains a type provider for vector data structures. [Read more] […]

  • Leave a Reply

Archives

Join our mailer

You should join our newsletter! Sent monthly:

Back to Top