The complete code can be found on GitHub.
I’m currently having fun playing with the development of a virtual console / simple game engine in my free time. The basic idea is that it will pick up and run a “cartridge” - a cartridge being a collection of assets within a folder: images, maps, scripts etc.
Originally I’d thought about using Lua for the scripting language but I thought it might be fun to keep things fully functional and in F# and started to explore what that might look like. It didn’t take long until I came across the F# Compiler Services and, to skip to the obvious end given the title of this blog post, that worked out quite well.
First lets define what I mean by compile and run F# at runtime - in my case I want to load a script from a folder, compile it, and call it from my virtual console engine using a common set of models. For example to create a scene, or to respond to an event like a gamepad button.
To accomplish this their are two basic steps to the process:
- Compile the F# code
- Extract the functions you want to call
- Call the functions
Lets start their and break down what compilation looks like. First we need the compiler services themselves which can be found in the FSharp.Compiler.Services NuGet package.
Compiling an assembly
With the NuGet package added to a project we need to load the script into memory and compile it as shown below:
let compileScript name =
let script = $"./Scripts/{name}.fsx"
let outputAssemblyName = Path.ChangeExtension(Path.GetFileName(script),".dll")
let checker = FSharpChecker.Create()
let errors, exitCode, dynamicAssemblyOption =
checker.CompileToDynamicAssembly([|
"-o" ; outputAssemblyName
"-a" ; script
"--targetprofile:netcore"
|], Some (stdout, stderr))
|> Async.RunSynchronouslyThe options provided mirror those of the command line compiler (fsc). If we’re targetting .NET Core / 5.0 / 6.0 then the “–targetProfile:netCore” option is important as the default is to target .NET Framework. On successful output dynamicAssemblyOption will be an option type with an Ok value of type System.Reflection.Assembly - our compiled assembly ready to be used.
In the final code I tail this method with some basic error handling, converting the output to a Result type. Depending on your context you may be better served by an exception - in my scenario I need to carefully handle the failure states and give the user meaningful information in the virtual console.
Extracting our functions
Lets assume our script (ExampleScript.fsx) contains a simple add function (types added here for clarity):
let add (value1:int) (value2:int) = value1 + value2How do we go about calling this? Really we just want our code to be “normal” - we’d like to be able to write:
printf $"{add 2 3}\n"To do this we’re going to need to jump through a few hoops beginning with using reflection to find the MemberInfo that describes our function. Something I learned while doing this is that the default module name for a fsx script like the ones we’re using here is the name of the script. So in our case the fully qualified name, which we will need to locate the function, is ExampleScript.add. The following code (abridged from this useful blog post) will look for our MemberInfo given a fully qualified name and assembly:
let getMemberInfo (name:string) (assembly:Assembly) =
let fqTypeName, memberName =
let splitIndex = name.LastIndexOf(".")
name.[0..splitIndex - 1], name.[splitIndex + 1..]
let candidates = assembly.GetTypes() |> Seq.where (fun t -> t.FullName = fqTypeName) |> Seq.toList
match candidates with
| [t] ->
match t.GetMethod(memberName, BindingFlags.Static ||| BindingFlags.Public) with
| null -> Error "Member not found"
| memberInfo -> Ok memberInfo
| [] -> Error "Parent type not found"
| _ -> Error "Multiple candidate parent types found"Having located the MemberInfo we now need to turn it into a callable F# method signature. My first stab at this was to use Delegate.CreateDelegate which can be passed a MemberInfo as in the example below:
getMemberInfo name assembly
|> Result.map(fun memberInfo ->
let createdDelegate = Delegate.CreateDelegate(typeof<Func<int,unit>>, null, memberInfo)
)However underneath all this is the .NET type system and although it works for some cases (such as our add function) it fails for functions that return unit as if we look at the CLR type model for that we’ll find void as the return type for the MemberInfo and recieve a cast exception when we attempt to create the delegate.
That being the case my next step was to use the System.Linq.Expression namespace to compile calling wrapper functions that dealt with the type discrepancy. My first pass, and perhaps the easiest to explain, compiled a function for a specific signature. Here’s this approach for our add function:
let getAddFunction (assembly:Assembly) : Result<int -> int -> int,string> =
match getMemberInfo "ExampleScript.add" assembly with
| Ok memberInfo ->
let lambda =
let valueOneParameter = Expression.Parameter(typeof<int>)
let valueTwoParameter = Expression.Parameter(typeof<int>)
Expression.Lambda<Func<int,int,int>>(
Expression.Convert(
Expression.Call(
memberInfo,
valueOneParameter :> Expression,
valueTwoParameter :> Expression
),
typeof<int>
),
valueOneParameter,
valueTwoParameter
)
let systemFunc = lambda.Compile()
FuncConvert.FromFunc systemFunc |> Ok
| Error error -> Error errorHere we build a syntax tree that calls our add function, identified by the member info, and passes the two parameters in. It then converts the result to an int (as it will be an obj). Finally we compile it in (lambda.Compile()) which results in a Func<int,int,int> and then convert it to an F# signature.
If we want to deal with a unit return type then we need something slightly different:
let getAddFunction (assembly:Assembly) : Result<ExampleRecord -> unit,string> =
match getMemberInfo "ExampleScript.doSomething" assembly with
| Ok memberInfo ->
let lambda =
let valueParameter = Expression.Parameter(typeof<ExampleRecord>)
Expression.Lambda<Func<ExampleRecord,unit>>(
Expression.Convert(
Expression.Block(
Expression.Call(
memberInfo,
valueOneParameter :> Expression
),
Expression.Constant((), typeof<unit>) :> Expression
),
valueOneParameter
)
let systemFunc = lambda.Compile()
FuncConvert.FromFunc systemFunc |> Ok
| Error error -> Error errorThis time we use a block to first call our MemberInfo (which if you recall returns void) and then after it has run return a unit using Expression.Constant.
This works but I wanted to generalise it so I could support functions with arbitary signatures - different parameter types and different numbers of parameters. This needed some reworking and the introduction of generic types. I needed functions with different number of generic parameters so I’ve landed on an approach as shown below - extractFunction1 has 1 generic parameter and 1 generic return type, extractFunction2 has 2 generic parameters and 1 generic return type etc.:
let extractFunction1<'p1,'r> name (assembly:Assembly) : Result<'p1 -> 'r,string> =
let parameters = [| Expression.Parameter(typeof<'p1>) |]
let systemFuncResult = extractor<'r> name assembly parameters
systemFuncResult |> Result.map(fun systemFunc -> systemFunc :?> Func<'p1,'r> |> FuncConvert.FromFunc )
let extractFunction2<'p1,'p2,'r> name (assembly:Assembly) : Result<'p1 -> 'p2 -> 'r,string> =
let parameters = [|
Expression.Parameter(typeof<'p1>)
Expression.Parameter(typeof<'p2>)
|]
let systemFuncResult = extractor<'r> name assembly parameters
systemFuncResult |> Result.map(fun systemFunc -> systemFunc :?> Func<'p1,'p2,'r> |> FuncConvert.FromFunc )These functions are used to frame the parameter set from the generic parameters and then finally to cast to an appropraite Func as we’ll lose some static type information during the process of constructing our Lambda.
Ths construction takes place in the extractor<‘r> method:
let extractor<'r> name assembly parameters =
match getMemberInfo name assembly with
| Ok memberInfo ->
try
let lambda =
let expression =
if (typeof<'r> = typeof<unit>) then
Expression.Block(
Expression.Call(
memberInfo,
parameters |> Array.map(fun param -> param :> Expression)
),
Expression.Constant((), typeof<'r>)
) :> Expression
else
Expression.Convert(
Expression.Call(
memberInfo,
parameters |> Array.map(fun param -> param :> Expression)
),
typeof<'r>
) :> Expression
Expression.Lambda(expression, parameters)
let systemFunc = lambda.Compile()
systemFunc |> Ok
with
| ex -> Error $"{ex.GetType().Name}: {ex.Message}"
| Error error -> Error errorThis is really a composite of our previous static approach but checks whether or not the return type is of type unit and branches accordingly. The other main difference is that we are using Expression.Lambda without generic parameters. This means that a delegate is returned rather than a Func<’t> and is why our final step in the extractFunction1(2,3 etc.) functions needs to cast this back to the appropriate type. (Its worth noting I attempted a purely generic version of this but I’ve not yet found a way to avoid an escaping generic type).
Before moving on its worth highlighting another technique for extracting functions that I came across while researching this and that’s to assign them to a variable in your script. In my case I discounted this approach as it required the end user to remember to undertake a special and non-obvious step.
Finally - I’m likely to refine some of these techniques as I spend more time with them, its been an iterative process getting here, and I’ve learned something on each pass. A couple more passes and hopefully I’ll have deleted some code :)
Using the compiled functions
With all that out the way we can now call our compiled functions:
match compileScript "ExampleScript" with
| Ok assembly ->
assembly
|> extractFunction1<ExampleRecord, unit> "ExampleScript.scriptWithUnitResult"
|> (function
| Ok extractedFunction -> extractedFunction { Surname = "Smith" ; Forename = "Joe" }
| Error error -> error |> logError
)
assembly
|> extractFunction1<ExampleRecord, int> "ExampleScript.scriptWithIntResult"
|> (function
| Ok extractedFunction -> extractedFunction { Surname = "Smith" ; Forename = "Joe" } |> ignore
| Error error -> error |> logError
)
assembly
|> extractFunction2<int, int, int> "ExampleScript.add"
|> (function
| Ok extractedFunction -> printf $"Add: {extractedFunction 1 2}"
| Error error -> error |> logError
)
| Error error -> error |> logError There is some error handling required to ensure we handle functions that fail to compile (and there is a lot of scope for error!) but once you’ve got hold of a function you can see that their is nothing special about their usage. As I noted earlier depending on your use case you could rely on exceptions rather than a Result type - very much context based!
Conclusions
Their is certainly a degree of complexity in making this work but once its up and running this opens up some powerful scenarios for runtime configuration and meta-programming. In my case I’m allowing users to execute scripts containing their game logic but you can easily imagine using this in, for example, an ETL engine or workflow system. And taking it a step further you could generate code from, for example, data discovered at runtime that is then compiled (the Expression subsystem of .NET should be your first port of call here - but their are things it doesn’t support). My old open source project Function Monkey did that with C#, Roslyn and Azure Functions.
In any case you can save yourself some of the complexity by reusing the code in the sample project! If you take a look in the Scripting.fs file you’ll see the reusable helper functions.
Happy compiling!