C# Delegates strike back

My last article (The dark side of C# Delegates) detailled how delegates are declared, instanciated and invoked. We have seen the role of the compiler, through analysis of dis-assembled code. But we haven't found the implementation of either the .ctor or the Invoke methods on the compiler-generated delegate classes.
In this follow-up we'll go through new evidence of the inner workings that we analyzed and we'll look at the missing details.

First, I wanted to confirm the role of the compiler in all the expressions involving delegates.
I checked out the ECMA specs for C# and the CLI. You can find links to these specs on this MSDN page
Here is a copy of the parts related to delegates:

From the ECMA spec (CLI partition I)
8.9.3 Delegates
Delegates are the object-oriented equivalent of function pointers. Unlike function pointers, delegates are object-oriented, type-safe, and secure. Delegates are created by defining a class that derives from the base type System.Delegate (see Partition IV). Each delegate type shall provide a method named Invoke with appropriate parameters, and each instance of a delegate forwards calls to its Invoke method to a compatible static or instance method on a particular object. The object and method to which it delegates are chosen when the delegate instance is created.
In addition to an instance constructor and an Invoke method, delegates may optionally have two additional methods: BeginInvoke and EndInvoke. These are used for asynchronous calls. While, for the most part, delegates appear to be simply another kind of user defined class, they are tightly controlled. The implementations of the methods are provided by the VES, not user code. The only additional members that may be defined on delegate types are static or instance methods.

From ECMA CLI specs (partition II)
13.6 Delegates
Delegates (see Partition I) are the object-oriented equivalent of function pointers. Unlike function pointers, delegates are object-oriented, type-safe, and secure. Delegates are reference types, and are declared in the form of Classes. Delegates shall have an immediate base type of System.MulticastDelegate, which in turns has an immediate base type of System.Delegate (see Partition IV).
Delegates shall be declared sealed, and the only members a Delegate shall have are either two or four methods as specified here. These methods shall be declared runtime and managed (see clause 14.4.3). They shall not have a body, since it shall be automatically created by the VES. Other methods available on delegates are inherited from the classes System.Delegate and System.MulticastDelegate in the Base Class Library (see Partition IV).
Rationale: A better design would be to simply have delegate classes derive directly from
System.Delegate. Unfortunately, backward compatibility with an existing CLI does not permit this design.
The instance constructor (named .ctor and marked specialname and rtspecialname, see clause 9.5.1) shall take exactly two parameters. The first parameter shall be of type System.Object and the second parameter shall be of type System.IntPtr. When actually called (via a newobj instruction, see Partition III), the first argument shall be an instance of the class (or one of its subclasses) that defines the target method and the second argument shall be a method pointer to the method to be called.

Looking in the mono:: compiler (mcs), I found the classes that implements the delegate compilation: Delegate, NewDelegate and DelegateInvocation, in the Mono.CSharp namespace.

You'll notice the Delegate.Define() method which registers some extra methods on the represented delegate class, like the constructor and the Invoke methods (and the asynchronous versions of Invoke as well). This Define method details how these compiler-added methods get the correct signature. But although the methods are registered, no implementation appears to be attached at this point. That's because they are marked with the MethodImplAttributes.Runtime attribute (see the MSDN reference here).

In the NewDelegate.Emit(EmitContext ec), you can see that the parameters for the instanciation is not a method, like the C# suggests, but in fact an object (that may be null) and a method pointer (via the OpCodes.Ldftn operation code). They are placed on the IL stack before the constructor is called.

In the DelegateInvocation.Emit(EmitContext ec), IL code generated for the delegate invocation is actually a regular method invocation on the delegate object, on the Invoke method. In the DoResolve method, the method member variable is set to reference the Invoke method on the delegate. Then the Emit simply calls Invocation.EmitCall (implemented in expression.cs) to generate the IL for a call to this method method.

runtime managed method attributes
The only parts missing in the puzzle is how are the delegate constructor and the Invoke method implemented.

As the ECMA specs and the MSDN references explain, the "runtime managed" methods are supposed to be implemented by the runtime or VES (virtual execution system).
Searching in the mono:: runtime (called mono), I found the definition of the "runtime" attribute: METHOD_IMPL_ATTRIBUTE_RUNTIME. This attribute is used in a couple locations in the VES implementation and the one that appeared relevant is ves_exec_method in mono/interpreter/interpret.c. This function apparently handles these runtime-implemented methods by calling ves_runtime_method in the same file.

In ves_runtime_method, you can see that the method name is used to switch between the various method implementations. The two first ones are .ctor and Invoke. In the case of .ctor, mono_delegate_ctor is called, and in the case of Invoke, the method pointer helds by the delegate (delegate.method_ptr) is used to do a method call (with ves_exec_method).

All the pieces now come together, we have seen how the compiler treats a delegate like a special class (that it generates), which has some compiler-added methods that are implemented by the runtime.
The delegate declaration causes the compiler to generate a class with specific .ctor and Invoke methods. The instanciation gets mapped by the compiler to a .ctor(object, int) call that is implemented by the runtime. And the invocation gets mapped by the compiler to a Invoke call that is also implemented by the runtime.

The only issues that I am still curious about are:
- What about the JIT? We have seen the mono:: interpreter's implementation of ves_runtime_method, but how does the JIT-compiled version work?
- Is is possible to create, instanciate or invoke a delegate a runtime using reflection? In this case how is the type-safety enforced and how is the conversion from MethodInfo to "native int" method pointer (and back) done?
- What is the "event" type and how much more compiler and runtime support does it need on top of the delegates support?

Some delegate topics we didn't dig into, but I believe can be explained easily using what we have found so far: the asynchronous Invoke methods (BeginInvoke and EndInvoke) and the multicast delegates. Looking through the code that we looked at so far, you will find the details for these features.

The three main CodeDom classes references

MSDN reference for CodeTypeDelegate Class (representing a delegate declaration)

MSDN reference for CodeDelegateCreateExpression Class (representing a delegate instanciation)
MSDN reference for CodeDelegateInvokeExpression Class (representing a delegate invocation)

Mono:: implementation of the System.CodeDom namespace
Mono:: implementation of the Microsoft.CSharp namespace (includes CSharpCodeProvider and CSharpCodeGenerator)

Note: as Miguel nicely pointed out to me, the CodeDOM classes aren't used to represent the code tree during compilation, because they aren't powerful enough. It is meant to generate textual source code (that can then be compiled). I just included the reference links above to provide another angle on delegates.
Update: I just found some more details on the limitations of CodeDOM in the C# CodeDOM parser article.

Posted by Julien on April 25, 2003. Permalink
comments powered by Disqus

I still have the nagging feeling that reflection is involved within the invoke method of the delegate, and still seem to feel that delegates and therefore events are late-bound and potentially not performant...

Posted by: Dave Hiers at March 16, 2004 09:36 AM

Here is a benchmark:

There is a big difference between direct method invocation and delegates. But apparently this has been optimized for the next release (Whidbey) to bring delegates up to par with interfaces method calls.
I don't know the explanation for these differences, but delegates are faster than InvokeMember (reflection) calls.

The article also shows how to create delegates at runtime, via Reflection, which I was wondering about when I wrote this post.

Posted by: Dumky at March 16, 2004 03:16 PM