Duck Typing in a Static Core

I was reading an interesting blog post that described duck typing and how it helps dynamic languages, and how it works in theory. I'm thinking of a few ways on how this might be feasible to implement, and I think I have one. I'm going to do a few whiteboards on the idea and post back when I get a chance.

The basic idea involves use of generics. The full idea will take some time to just map out, and to implement it might take a few months, and that's only after I get the code generator going full steam.

Since the entire project is about code generation, the idea involves dynamic mutability descriptors (classes) that help define the viability of a call to a method given the types for a call to that said method. There would also be a type-change lookup, with a running deviation gauge on the signature provided, if a type changed, structurally, since the last time the method was called, it would run the check again for that signature, and adopt a new method in place for the old one.

The issue with this lies in the fact that the CLI doesn't allow (generated) code to be disposed, except for entire app domains (to be unloaded). While I could encase all changes into their own app domain; management, and communication between them would be nightmarish at best, and not time, or performance worthy. A dynamic language that continually changed a type would yield gradually more and more junk in memory if it were to restructure itself on need. The solution to this would be to, obviously, try to reduce the need to continually change the type structure.

One thing that would allow this to go smoothly is enabling the system to look for the dynamic method descriptors first, thus ensuring that it locks onto the dynamic descriptor, which would take precedence (except for exact signature matches). The dynamic descriptors would be entirely generic, and would thus take any signature without the need to worry about deviations. Any change to a type through dynamic means would utilize a dynamic descriptor, and thus would not cause lookups to occur on types that changed, where the signature deviation on a specific call is greater than one. In plain English, this means that: if method A calls method on type X, method A has a dynamic descriptor, and the current signature on the call to method A did not deviate greater than 0 during deviation (dynamic descriptors don't deviate, since they accept any signature), then the method doesn't need to be regenerated, even if the type has changed since the last call to that method.

The complexity of each mutability descriptor will vary based upon how complex the method itself is. It will then utilize an IL generator to build dynamic variants of the method based upon the signature of the values provided. This will, in turn, cache the result. The reason for the descriptor's complexity is the need to go deeper than one level. It'll also utilize a secondary cache to help relate to certain kinds of linker-based relationships (does type X have method A with this exact signature, if not, look it up (full look-up), if that fails, find the next best alternative from cache, then the next best from the type (full lookup)). The worth of such a system is also debatable, and would likely only benefit dynamic languages, it'd (probably?) be faster than a fully dynamic system that handles the same thing.

The other reason for the concept above is I didn't really like the methods employed in Iron Python, using confusing, near undecipherable means to handle method calls, inheritance, and so forth. Though I'll admit, it works. I might find, in my research into this, that their method is favorable. Another thing I'll admit: the proposed solution would be incredibly irksome to build; however, I'm doing this because I find languages interesting.

Light Refactoring turns into Major Refactoring

Well,

As much as I'd like to keep the project simple, it hasn't been simple from day one. Due to the features I'm planning on, I figured it best if I restructured the project, the Scripting Language Foundation, into a series of projects. There's the primary Type System, the CLI Type system, and the OIL (Intermediate) Type System. These three systems are best kept separate to help simplify project management. I also just got tired of scrolling through over a thousand files in one project.

The advantage of the larger scale refactoring is quicker build times (yay) so long as I don't modify the core type system (which I'm still doing, so 'boo').

Another thing I'm hoping this will result in is the ability to allow for the system to be lightly retargetable in the future. So long as there's an alternative for the CLI type system, and the CIL in general. The abstract type system, as it is, really just defines things in general sense; the primary reasons it relies on the CLI type system are for the base-types of arrays, byref types, enumerations, structures, and so forth (a simple retarget, if another target exists). Granted there are probably very few targets that are as high level as the abstract type system specifies.

Here's hoping things go well. The refactoring presently copmiles. Instead of one quite sizable library, it's now broken down into concepts:

1. Abstract Syntax Tree
   1. Abstract
   2. Common Language Infrastructure
   3. Objectified Intermediate Language
2. Compilers
3. Languages
   1. Abstract
   2. C♯
   3. Common Intermediate Language (CIL)
   4. Visual Basic.NET
4. Linkers
5. Transformation
6. Translation

The Abstract Syntax Tree (AST) for the first two targets, Abstract and CLI, are more type-systems than ASTs. The third, OIL, is a type-system and malleable infrastructure that injects more code-principles than the first two.

The transformation stage will utilize information about the language as stipulated by contextual data, which is defined in the Language Abstract (a given language specifies what's supported, this information is piped through the appropriate channels that use the transformers, such as compilers and code translators). The transformation framework will utilize the limitations of the language to determine what parts of the intermediate structure need transformations applied and which ones to apply.

It will also verify that the code as presented is even valid for that language, given its limitations. Certain features of the CLI don't have workarounds, such as 'base', 'MyClass', while others do, a la Lambdas. If it can't already explicitly call a virtual method non-virtually in an appropriate context (base, current scope, et cetera), there's no amount of trickery that can allow for it.

You could, in theory, write an adapter class for such instances that would emit the proper CIL to handle the task at hand, but the usability and maintenance of such code would be... questionable at best. It's best to flag the region as invalid for that particular language. The areas marked as 'invalid' for a language are typically why two languages are not 100% interoperable between each other. Visual Basic code can't always be translated to C♯ code, same applies to C♯.

Light Refactoring

After a short delay, due to work, I'm back working on this.

I'm doing a little clean up work on the core type system and rewriting the assist methods that are associated to the functionality, that's presently broken. This will greatly assist in streamlining the code and reduce the dependency on internal components that were used before. I also hope to be able to open up the type system a little more to allow for extensibility; though I'm not sure to what this will result in, overall.

Adjusted the disambiguation and type-parameter verification aspects to simplify things in the end.

So far things are looking OK; however, I'll know more when it compiles (280 compile errors due to restructure, slowly eliminating them one by one.)

   1:  /// <summary>
   2:  /// Verifies the type-parameters of a generic method 
   3:  /// <paramref name="signature"/> with the <paramref name="typeReplacements"/>
   4:  /// provided.
   5:  /// </summary>
   6:  /// <typeparam name="TSignatureParameter">The type of parameter used in the <typeparamref name="TSignature"/>.</typeparam>
   7:  /// <typeparam name="TGenericParameter">The type of generic type-parameter used in the <typeparamref name="TSignature"/></typeparam>
   8:  /// <typeparam name="TGenericParameterConstructor">The type used to signify the constructors used on the
   9:  /// generic parameter.</typeparam>
  10:  /// <typeparam name="TGenericParameterConstructorParameter">The type used to signify
  11:  /// the parameters on the constructors of the generic parameters.</typeparam>
  12:  /// <typeparam name="TSignature">The type of signature used as a parent of <typeparamref name="TSignatureParameter"/> and
  13:  /// <typeparamref name="TGenericParameter"/> instances.</typeparam>
  14:  /// <typeparam name="TSignatureParent">The parent that contains the <typeparamref name="TSignature"/> 
  15:  /// instances.</typeparam>
  16:  /// <param name="signature">The <see cref="IMethodSignatureMember{TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent}"/>
  17:  /// to verify the <paramref name="typeReplacements"/> against.</param>
  18:  /// <param name="typeReplacements">The <see cref="ITypeCollection"/> that defines
  19:  /// the replacement types to verify.</param>
  20:  /// <remarks>TGeneric* type-parameters are primarily used to complete the hierarchy
  21:  /// chain to allow for type strict reverse-traversal.</remarks>
  22:  public static void VerifyTypeParameters<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>(this IMethodSignatureMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent> signature, ITypeCollection typeReplacements)
  23:      where TSignatureParameter :
  24:          IMethodSignatureParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
  25:      where TGenericParameter :
  26:          IMethodSignatureGenericTypeParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
  27:      where TGenericParameterConstructor :
  28:          IMethodSignatureGenericCtorMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
  29:      where TGenericParameterConstructorParameter :
  30:          IMethodSignatureGenericCtorParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
  31:      where TSignature :
  32:          IMethodSignatureMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
  33:      where TSignatureParent :
  34:          ISignatureParent<TSignature, TSignatureParameter, TSignatureParent>
  35:  {
  36:      if (signature == null)
  37:          throw new ArgumentNullException("signature");
  38:      if (typeReplacements == null)
  39:          throw new ArgumentNullException("typeReplacements");
  40:      /* *
  41:       * Setup the test case parameter logic.
  42:       * *
  43:       * If the method belongs to a generic type, mine the 
  44:       * generic type-parameters from it to help in 
  45:       * properly disambiguating the method's type-parameter
  46:       * constraints.
  47:       * */
  48:      ITypeCollection parentTypeReplacements = TypeCollection.Empty;
  49:      TypeParameterSource source = TypeParameterSource.Method;
  50:      if (signature.Parent is IGenericType && ((IGenericType)(signature.Parent)).IsGenericType)
  51:      {
  52:          IGenericType parent = ((IGenericType)(signature.Parent));
  53:          /* *
  54:           * Special case on when the parent is a generic definition.
  55:           * It's still a valid call if the type-replacements passed
  56:           * contain enough replacements to fill the parent's
  57:           * and the method's type-parameters.  Least likely case.
  58:           * *
  59:           * Obtain the replacements 
  60:           * */
  61:          if (parent.IsGenericTypeDefinition)
  62:              if (typeReplacements.Count == parent.GenericParameters.Count + signature.GenericParameters.Count)
  63:              {
  64:                  parentTypeReplacements = typeReplacements.Take(parent.GenericParameters.Count).ToCollection();
  65:                  typeReplacements = typeReplacements.Skip(itc.Count).ToCollection();
  66:                  source = TypeParameterSource.Both;
  67:              }
  68:              else
  69:                  throw new ArgumentException("typeReplacements");
  70:          else
  71:          {
  72:              parentTypeReplacements = parent.GenericParameters;
  73:              source = TypeParameterSource.Both;
  74:          }
  75:      }
  76:      else if (signature.TypeParameters.Count != typeReplacements.Count)
  77:          throw new ArgumentException("typeReplacements");
  78:      VerifyTypeParameters_VerifyReplacements(typeReplacements);
  79:      /* *
  80:       * Generate test-case generic parameters
  81:       * to do the verification with.
  82:       * *
  83:       * Logic to handle the parentTypeReplacements is 
  84:       * defined above.
  85:       * */
  86:      ITypeCollection testCases =
  87:          from IGenericTypeParameter t in signature.GenericParameters
  88:          select new GenericVerificationParameter(
  89:              t.Constraints.OnAll(
  90:                  k => k.Disambiguify(
  91:                      parentTypeReplacements,
  92:                      typeReplacements,
  93:                      source)).ToCollection(), t);
  94:      VerifyTypeParametersInternal(typeReplacements, testCases);
  95:  }
  96:   
  97:  /// <summary>
  98:  /// Verifies a set of <paramref name="typeReplacement"/> <see cref="IType"/> instances
  99:  /// against the type-parameters defined on the <paramref name="genericType"/>.
 100:  /// </summary>
 101:  /// <param name="genericType">The <see cref="IGenericType"/>
 102:  /// which contains the parameters to verify against.</param>
 103:  /// <param name="typeReplacements">The <see cref="ITypeCollection"/> that defines
 104:  /// the replacement types to verify.</param>
 105:  public static void VerifyTypeParameters(this IGenericType genericType, ITypeCollection typeReplacements)
 106:  {
 107:      if (genericType == null)
 108:          throw new ArgumentNullException("genericType");
 109:      if (typeReplacements == null)
 110:          throw new ArgumentNullException("typeReplacements");
 111:      if (genericType.TypeParameters.Count != typeReplacements.Count)
 112:          throw new ArgumentException("typeReplacements");
 113:      VerifyTypeParameters_VerifyReplacements(typeReplacements);
 114:   
 115:      /* *
 116:       * Obtain a series of generic verifiers that act as dummy
 117:       * generic parameters for the checks to be performed.
 118:       * *
 119:       * On method type-parameter verification, 
 120:       * the 'typeReplacements' parameter will be 
 121:       * used on the methodReplacements parameter
 122:       * in k.Disambiguify; whereas the 
 123:       * method's parent type-parameters will only
 124:       * be included if it is: An IGenericType
 125:       * and that generic type is actually IType.IsGenericType.
 126:       * */
 127:      ITypeCollection testCases =
 128:          from IGenericTypeParameter t in genericType.GenericParameters
 129:          select new GenericVerificationParameter(
 130:              t.Constraints.OnAll(
 131:                  k => k.Disambiguify(
 132:                      typeReplacements,
 133:                      TypeCollection.Empty,
 134:                      TypeParameterSource.Type)).ToCollection(), t);
 135:      VerifyTypeParametersInternal(typeReplacements, testCases);
 136:  }
 137:   
 138:  private static void VerifyTypeParameters_VerifyReplacements(ITypeCollection typeReplacements)
 139:  {
 140:      for (int i = 0; i < typeReplacements.Count; i++)
 141:          if (typeReplacements[i] == null)
 142:              throw new ArgumentNullException(string.Format("typeReplacements[{0}]", i));
 143:          //Pointer types, by-reference types, and the void type cannot be generic type parameters.
 144:          else if (typeReplacements[i].ElementClassification == TypeElementClassification.Pointer ||
 145:                   typeReplacements[i].ElementClassification == TypeElementClassification.Reference ||
 146:                   typeReplacements[i] is ICompiledType && ((ICompiledType)typeReplacements[i]).UnderlyingSystemType == typeof(void))
 147:              throw new ArgumentException(string.Format(Resources.TypeConstraintFailure_InvalidType, typeReplacements[i].ToString()));
 148:  }
 149:   
 150:  private static void VerifyTypeParametersInternal(ITypeCollection typeReplacements, ITypeCollection testCases)
 151:  {
 152:      /* *
 153:       * Iterate through the replacements and compare them against 
 154:       * the test cases.
 155:       * */
 156:      for (int i = 0; i < testCases.Count; i++)
 157:      {
 158:          IGenericParameter param = (IGenericParameter)(testCases.Values[i]);
 159:          IType replacement = typeReplacements[i];
 160:          /* *
 161:           * Generic parameters require special processing.
 162:           * */
 163:          if (replacement.IsGenericTypeParameter)
 164:          {
 165:              IGenericParameter replacementParam = ((IGenericParameter)(replacement));
 166:              if (param.RequiresNewConstructor && !replacementParam.RequiresNewConstructor)
 167:                  throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 168:                      /*{0}*/ replacementParam.Name,
 169:                      /*{1}*/ param.Name,
 170:                      /*{2}*/ Resources.TypeConstraintFailure_NewConstraint));
 171:              if (!(param.SpecialConstraint == GenericTypeParameterSpecialConstraint.Class &&
 172:                    (replacementParam.SpecialConstraint == GenericTypeParameterSpecialConstraint.Class ||
 173:                     (replacementParam.Constraints.Count > 0 &&
 174:                      replacementParam.Constraints[0] is IReferenceType))))
 175:                  throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 176:                      /*{0}*/ replacementParam.Name,
 177:                      /*{1}*/ param.Name,
 178:                      /*{2}*/ Resources.TypeConstraintFailure_ReferenceType));
 179:              else if (param.SpecialConstraint == GenericTypeParameterSpecialConstraint.Struct &&
 180:                       replacementParam.SpecialConstraint != GenericTypeParameterSpecialConstraint.Struct)
 181:                  throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 182:                      /*{0}*/ replacementParam.Name,
 183:                      /*{1}*/ param.Name,
 184:                      /*{2}*/ Resources.TypeConstraintFailure_ValueType));
 185:              /* *
 186:               * Verify each constraint, they were translated from their
 187:               * earlier form into a type-parameter resolved form.
 188:               * */
 189:              foreach (IType constraint in param.Constraints)
 190:                  if (!replacementParam.Constraints.Any(replacementConstraint => constraint.IsAssignableFrom(replacementConstraint)))
 191:                      throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 192:                          /*{0}*/ replacementParam.Name,
 193:                          /*{1}*/ param.Name,
 194:                          /*{2}*/ string.Format(Resources.TypeConstraintFailure_ParamConstraint, constraint.IsGenericTypeParameter ? constraint.Name : constraint.FullName)));
 195:   
 196:          }
 197:          else
 198:          {
 199:              /* *
 200:               * Structs and enumerations are both value types and
 201:               * automatically contain a default constructor.
 202:               * */
 203:              if (param.RequiresNewConstructor && !(replacement.Type == TypeKind.Struct || replacement.Type == TypeKind.Enumerator))
 204:              {
 205:                  if (replacement.Type == TypeKind.Interface || (!(replacement is ICreatableType)))
 206:                      throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 207:                          /*{0}*/ replacement.Name,
 208:                          /*{1}*/ param.Name,
 209:                          /*{2}*/ Resources.TypeConstraintFailure_NewInterfaceDelegateOther));
 210:                  ICreatableType creatableReplacement = (ICreatableType)(replacement);
 211:                  if (creatableReplacement.Constructors.Find().Count == 0)
 212:                      throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 213:                          /*{0}*/ replacement.Name,
 214:                          /*{1}*/ param.Name,
 215:                          /*{2}*/ Resources.TypeConstraintFailure_NewStandard));
 216:              }
 217:   
 218:              /* *
 219:               * SpecialConstraint check.
 220:               * */
 221:              switch (param.SpecialConstraint)
 222:              {
 223:                  case GenericTypeParameterSpecialConstraint.Struct:
 224:                      if (replacement.Type != TypeKind.Enumerator ||
 225:                          replacement.Type != TypeKind.Struct)
 226:                          throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 227:                              /*{0}*/ replacement.Name,
 228:                              /*{1}*/ param.Name,
 229:                              /*{2}*/ Resources.TypeConstraintFailure_ValueType));
 230:                      break;
 231:                  case GenericTypeParameterSpecialConstraint.Class:
 232:                      /* *
 233:                       * Since I can't predict the future, I used an
 234:                       * interface to replace specific type checking.
 235:                       * If someone wants to expand the framework
 236:                       * for whatever reason, this makes things easier.
 237:                       * */
 238:                      if (!(replacement is IReferenceType))
 239:                          throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 240:                              /*{0}*/ replacement.Name,
 241:                              /*{1}*/ param.Name,
 242:                              /*{2}*/ Resources.TypeConstraintFailure_ReferenceType));
 243:              }
 244:              /* *
 245:               * Every constraint needs to be assignable 
 246:               * by the replacement.
 247:               * */
 248:              foreach (IType constraint in param.Constraints)
 249:                  if (!constraint.IsAssignableFrom(replacement))
 250:                      throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
 251:                          /*{0}*/ replacement.Name,
 252:                          /*{1}*/ param.Name,
 253:                          /*{2}*/ string.Format(Resources.TypeConstraintFailure_Constraint, constraint.IsGenericTypeParameter ? constraint.Name : constraint.FullName)));
 254:   
 255:          }
 256:          /* *
 257:           * Dispose the type-parameter test case, it's no longer needed
 258:           * */
 259:          param.Dispose();
 260:      }
 261:  }
 262:   
 263:  /// <summary>
 264:  /// Resolves generic type-parameters contained within the
 265:  /// <paramref name="target"/> <see cref="IType"/> by 
 266:  /// rebuilding the <paramref name="target"/> with
 267:  /// the <paramref name="typeReplacements"/> used in place of
 268:  /// the <see cref="IGenericTypeParameter"/> instances.
 269:  /// </summary>
 270:  /// <param name="target">The target <see cref="IType"/>
 271:  /// to disambiguify.</param>
 272:  /// <param name="typeReplacements">The <see cref="IType"/>
 273:  /// series which contains a series of types that is index-relative
 274:  /// to the <paramref name="target"/>'s type-parameters.</param>
 275:  /// <param name="parameterSource">The point to source
 276:  /// the replacements from.</param>
 277:  /// <returns></returns>
 278:  public static IType Disambiguify(this IType target, ITypeCollection typeReplacements, ITypeCollection methodReplacements, TypeParameterSource parameterSource)
 279:  {
 280:      /* *
 281:       * Assumes a great deal in that: Types provided are used in scope.
 282:       * Beyond scope this does not work as intended since the positions
 283:       * of out-of-scope type-parameters might not be relatively 
 284:       * equivalent to the scope they are used in (typeReplacements).
 285:       * */
 286:      if (target.IsGenericTypeParameter)
 287:      {
 288:          if (target is IGenericParameter)
 289:          {
 290:              /* *
 291:               * If the declaring type is null, this method assumes 
 292:               * it's a method.  Primarily to aid in possible other 
 293:               * uses.
 294:               * */
 295:              if ((parameterSource & TypeParameterSource.Method) == TypeParameterSource.Method && 
 296:                  ((IGenericParameter)(target)).DeclaringType == null &&
 297:                  ((IGenericParameter)(target)).Position < methodReplacements.Count)
 298:                      return methodReplacements[((IGenericParameter)(target)).Position];
 299:              if ((parameterSource & TypeParameterSource.Type) == TypeParameterSource.Type && 
 300:                  ((IGenericParameter)(target)).DeclaringType != null &&
 301:                  ((IGenericParameter)(target)).Position < typeReplacements.Count)
 302:                      return typeReplacements[((IGenericParameter)(target)).Position];
 303:          }
 304:      }
 305:      else
 306:      {
 307:          switch (target.ElementClassification)
 308:          {
 309:              case TypeElementClassification.Array:
 310:                  if (target is IArrayType)
 311:                      return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeArray(((IArrayType)target).ArrayRank);
 312:                  break;
 313:              case TypeElementClassification.Nullable:
 314:                  return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeNullable();
 315:              case TypeElementClassification.Pointer:
 316:                  return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakePointer();
 317:              case TypeElementClassification.Reference:
 318:                  return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeByReference();
 319:              case TypeElementClassification.GenericTypeDefinition:
 320:                  if (target.ElementType is IGenericType)
 321:                      return ((IGenericType)target.ElementType).MakeGenericType(((IGenericType)target).GenericParameters.OnAll(gP => gP.Disambiguify(typeReplacements, parameterSource)));
 322:                  break;
 323:              case TypeElementClassification.None:
 324:                  if (target is IGenericType &&
 325:                     (parameterSource == TypeParameterSource.Type && ((IGenericType)(target)).GenericParameters.Count == typeReplacements.Count))
 326:                      return ((IGenericType)(target)).MakeVerifiedGenericType(typeReplacements);
 327:                  break;
 328:          }
 329:      }
 330:      return target;
 331:  }

Constructor Arguments on Generic Type Parameters

This is something I've wanted on Generics since they first introduced them. It didn't come in version 3.0 or 3.5; however because I'm writing my own Objectified Intermediate Language with the goal of translating the objects into Common Intermediate Language, I can add features similarly to how C#'s compiler adds support for lambda expressions, anonymous methods, closures, and so on.

I decided to write this post after talking to Peli about this and confusing the hell out of him by not defining the problem/solution in full. I even provided a bad example of how such functionality would benefit a coder.

Let's start out with what this would mean. The primary reason you would even want this is if you wanted to create instances of some arbitrary type not explicitly defined in your code. That's what the original new() came in for. The problem with this is not all code elements are properly initializable through a zero-parameter constructor additionally they don't expose a zero-parameter constructor for this reason.

I've written a small usage example, with background code, to illustrate how the functionality will work. Using the standard 'Widget' sample, it illustrates using a list that contains multiple widget providers. Basically allowing you to define a list that is restricted to a base type, also allowing you to arbitrarily instantiate a widget by using its System.Type using much quicker instantiation methods than using a ConstructorInfo instance's Invoke method.

This would be useful for cases where you have an extensibility model that handles things through similar providers and has a need for type-parameter constructors with arguments. If, for instance, you had a project infrastructure and you allowed add-ins to define projects, and the elements that define them, through attributes or other means. You'd need a model that would enable construction of the project and its elements.

Sure you could argue that you could hard-code the specifics based upon the type requested on the provider, but that misses the point entirely.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace WidgetWorks
{
    class Program
    {
        internal static void Main()
        {
            Type[] widgetTypes = new Type[] { typeof(MyWidget), typeof(YourWidget), typeof(TheirWidget) };
            WidgetList k = new WidgetList(new WidgetProvider<MyWidget>(), new WidgetProvider<YourWidget>(), new WidgetProvider<TheirWidget>());
            Random u = new Random();
            for (int i = 0; i < 6; i++)
            {
                int l = u.Next(1, 4);
                k.Add(string.Format("Test Widget {0}", i + 1), widgetTypes[l - 1]);
            }
            foreach (var w in k)
            {
                Console.Write("{0}\n\t", w.Name);
                w.DoSomething();
                Console.WriteLine();
            }

            Console.ReadKey(true);
        }
    }
}

As you can see it's pretty straightforward; however, as before it doesn't utilize the instance method ConstructorInfo.Invoke. The idea is the expansion of new constructors with parameters would be like so:

    public class WidgetProvider<[GenericParamCtorSignatures(typeof(WidgetProvider<>.TCtorData))] T> :
        IWidgetProvider
        where T :
            Widget/*,
            new(string)
            */
    {

        #region Generic Type Constructor Code

        #region T

        private delegate T CreateWidget1(string name);
        private static RuntimeMethodHandle _widgetCtor1Ref;
        private static CreateWidget1 _widgetCtor1;

        private static CreateWidget1 WidgetCtor1
        {
            get
            {
                if (_widgetCtor1 == null)
                {
                    _widgetCtor1 = ((ConstructorInfo)(MethodBase.GetMethodFromHandle(_widgetCtor1Ref, typeof(T).TypeHandle))).BuildOptimizedConstructorDelegateEx<CreateWidget1>(typeof(CreateWidget1).Module);
                    _widgetCtor1Ref = default(RuntimeMethodHandle);
                }
                return _widgetCtor1;
            }
        }

        [StructLayout(LayoutKind.Sequential, Size = 0)]
        private struct TCtorData
        {
            public TCtorData(string name) { }
        }

        #endregion

        private static void VerifyGenericParamConstructors()
        {
            Type t = typeof(T);
            ConstructorInfo[] tCtors = t.GetConstructors();
            ConstructorInfo widgetCtor1 = tCtors.FindConstructor(typeof(string));
            if (widgetCtor1 == null)
                throw new GenericParamCtorFailureException("T", t, typeof(string));
            _widgetCtor1Ref = widgetCtor1.MethodHandle;
        }
        #endregion

        static WidgetProvider()
        {
            VerifyGenericParamConstructors();
        }

        #region IWidgetProvider Members

        public Widget GetWidget(string name)
        {
            return WidgetCtor1(name);
        }

        public Type WidgetType
        {
            get { return typeof(T); }
        }

        #endregion

    }

If there is no static method it will add a simplistic one that invokes a VerifyGenericParamConstructors. Since it's not a CLR implementation, I'll have to make libraries expressly wanting this functionality rely on a small library that emits quick constructor invoke methods. It handles the quick invoke through the following code:

public static partial class LanguageMetaHelper
{
    #region Type Conversion Info

    private static Dictionary<TypeCode, Dictionary<TypeCode, bool>> conversionInfo = GetConversionInfo();
    private static Dictionary<TypeCode, Dictionary<TypeCode, bool>> GetConversionInfo()
    {
        Dictionary<TypeCode, Dictionary<TypeCode, bool>> conversionInfo = new Dictionary<TypeCode, Dictionary<TypeCode, bool>>();
        TypeCode[] supportedTypeCodes = new TypeCode[] { TypeCode.Byte, TypeCode.SByte, TypeCode.Single, TypeCode.Double, TypeCode.Char, TypeCode.UInt16, TypeCode.UInt32, TypeCode.UInt64, TypeCode.Int16, TypeCode.Int32, TypeCode.Int64 };
        foreach (TypeCode tc in supportedTypeCodes)
        {
            Dictionary<TypeCode, bool> current = new Dictionary<TypeCode, bool>();
            switch (tc)
            {
                case TypeCode.Char:
                    current[TypeCode.UInt16] = true;
                    current[TypeCode.UInt32] = true;
                    current[TypeCode.UInt64] = true;
                    current[TypeCode.Int32] = true;
                    current[TypeCode.Int64] = true;
                    current[TypeCode.Single] = true;
                    current[TypeCode.Double] = true;
                    break;
                case TypeCode.Byte:
                    current[TypeCode.Char] = true;
                    current[TypeCode.UInt16] = true;
                    current[TypeCode.UInt32] = true;
                    current[TypeCode.UInt64] = true;
                    goto case TypeCode.SByte;
                case TypeCode.SByte:
                    current[TypeCode.Int16] = true;
                    current[TypeCode.Int32] = true;
                    current[TypeCode.Int64] = true;
                    current[TypeCode.Single] = true;
                    current[TypeCode.Double] = true;
                    break;
                case TypeCode.UInt16:
                    current[TypeCode.UInt32] = true;
                    current[TypeCode.UInt64] = true;
                    goto case TypeCode.Int16;
                case TypeCode.Int16:
                    current[TypeCode.Int32] = true;
                    current[TypeCode.Int64] = true;
                    current[TypeCode.Single] = true;
                    current[TypeCode.Double] = true;
                    break;
                case TypeCode.UInt32:
                    current[TypeCode.UInt64] = true;
                    goto case TypeCode.Int32;
                case TypeCode.Int32:
                    current[TypeCode.Int64] = true;
                    current[TypeCode.Single] = true;
                    current[TypeCode.Double] = true;
                    break;
                case TypeCode.UInt64:
                case TypeCode.Int64:
                    current[TypeCode.Single] = true;
                    current[TypeCode.Double] = true;
                    break;
                case TypeCode.Single:
                    current[TypeCode.Double] = true;
                    break;
            }
            conversionInfo[tc] = current;
        }
        return conversionInfo;
    }
    /// <summary>
    /// Checks to see if you can go <paramref name="from"/> one type <paramref name="to"/> another.
    /// </summary>
    /// <param name="from">The type to check conversion of.</param>
    /// <param name="to">The type to see if <paramref name="from"/> can go to.</param>
    /// <returns>True if <paramref name="from"/> can be cast/converted <paramref name="to"/>; otherwise false.</returns>
    public static bool CanConvertFrom(this Type from, Type to)
    {
        TypeCode fromTC = Type.GetTypeCode(from);
        TypeCode toTC = Type.GetTypeCode(to);
        try
        {
            if (fromTC != toTC)
                return conversionInfo[fromTC][toTC];
            else if (fromTC == TypeCode.Object)
                return (to.IsAssignableFrom(from));
            else
                return true;
        }
        catch
        {
            return false;
        }
    }

    #endregion

    #region CtorBinding
    public static ConstructorInfo FindConstructor(this ConstructorInfo[] list, params Type[] binding)
    {
        if (list == null)
            throw new ArgumentNullException("list");
        if (binding == null)
            throw new ArgumentNullException("binding");
        var match = new bool[list.Length];
        var deviations = new Dictionary<ConstructorInfo, int>();
        for (int i = 0; i < list.Length; i++)
        {
            var current = list[i];
            var paramsInfo = current.GetParameters();
            if (paramsInfo.Length != binding.Length)
                continue;
            match[i] = true;
            for (int j = 0; j < paramsInfo.Length; j++)
            {
                if (paramsInfo[j].ParameterType != binding[j])
                    if (paramsInfo[j].ParameterType.CanConvertFrom(binding[j]))
                        if (deviations.ContainsKey(list[i]))
                            deviations[list[i]]++;
                        else
                            deviations.Add(list[i], 1);
                    else
                    {
                        match[i] = false;
                        break;
                    }
            }
        }
        int index1 = 0;
        try
        {
            return (from constructor in list
                    where match[index1++]
                    orderby deviations.ContainsKey(constructor) ? deviations[constructor] : 0
                    select constructor).First();
        }
        catch (InvalidOperationException)
        {
            return null;
        }
    }
    #endregion

    #region BuildOptimizedConstructorDelegate
    #region Helpers
    private delegate void FuncV<T>(T arg);

    private static IEnumerable<TCallResult> OnAll<TItem, TCallResult>(this IEnumerable<TItem> e, Func<TItem, TCallResult> f)
    {
        foreach (TItem t in e)
            yield return f(t);
        yield break;
    }

    private static void OnAll<TItem>(this IEnumerable<TItem> e, FuncV<TItem> f)
    {
        foreach (TItem t in e)
            f(t);
    }

    private static string GetStringFormSignature(ParameterInfo[] parameters)
    {
        bool firstMember = true;
        StringBuilder sb = new StringBuilder();
        foreach (ParameterInfo paramInfo in parameters)
        {
            if (firstMember)
                firstMember = false;
            else
                sb.Append(", ");
            sb.Append(paramInfo.ParameterType.FullName == null ? paramInfo.ParameterType.Name : paramInfo.ParameterType.FullName);
            sb.Append(" ");
            sb.Append(paramInfo.Name);
        }
        return sb.ToString();
    }
    #endregion

    /// <summary>
    /// Returns an optimized delegate which invokes a constructor described through <paramref name="ctor"/>.
    /// </summary>
    /// <typeparam name="T">The type of object created by the constructor.</typeparam>
    /// <param name="ctor">The constructor to build the dynamic delegate off of.</param>
    /// <returns>A new <see cref="CreateObjectInvoke{T}"/> which wraps around the 
    /// <paramref name="ctor"/> provided.</returns>
    public static T BuildOptimizedConstructorDelegateEx<T>(this ConstructorInfo ctor, Module m)
    {
        Type u = typeof(T);
        if (!u.IsSubclassOf(typeof(Delegate)))
            throw new ArgumentException("T");
        MethodInfo delegateInvoke = u.GetMethod("Invoke");
        Type[] delegateTypes = delegateInvoke.GetParameters().OnAll(param => param.ParameterType).ToArray();
        ParameterInfo[] ctorParameters = ctor.GetParameters();
        ILGenerator interLangGenerator = null;
        if (delegateTypes.Length != ctorParameters.Length)
            throw new ArgumentException("ctor");
        DynamicMethod optimizedCtor = new DynamicMethod(string.Format(".ctor@{0}({1})", ctor.DeclaringType.Name, GetStringFormSignature(ctorParameters)), delegateInvoke.ReturnType, delegateTypes, m);
        interLangGenerator = optimizedCtor.GetILGenerator();
        List<LocalBuilder> paramLocals = new List<LocalBuilder>();

        int argIndex = 0;
        ctorParameters.OnAll(parameter =>
        {
            if (parameter.IsOut || parameter.ParameterType.IsByRef)
            {
                if (argIndex < 128)
                {
                    interLangGenerator.Emit(OpCodes.Ldarga_S, argIndex);
                }
                else
                    interLangGenerator.Emit(OpCodes.Ldarga, argIndex);
            }
            else
            {
                switch (argIndex)
                {
                    case 0:
                        interLangGenerator.Emit(OpCodes.Ldarg_0, argIndex);
                        break;
                    case 1:
                        interLangGenerator.Emit(OpCodes.Ldarg_1, argIndex);
                        break;
                    case 2:
                        interLangGenerator.Emit(OpCodes.Ldarg_2, argIndex);
                        break;
                    case 3:
                        interLangGenerator.Emit(OpCodes.Ldarg_3, argIndex);
                        break;
                    default:
                        if (argIndex < 128)
                            interLangGenerator.Emit(OpCodes.Ldarg_S, argIndex);
                        else
                            interLangGenerator.Emit(OpCodes.Ldarg, argIndex);
                        break;
                }
            }
            argIndex++;
        });

        interLangGenerator.Emit(OpCodes.Newobj, ctor);
        if (ctor.DeclaringType.IsValueType)
            interLangGenerator.Emit(OpCodes.Box, ctor.DeclaringType);

        interLangGenerator.Emit(OpCodes.Ret);
        return (T)(object)optimizedCtor.CreateDelegate(typeof(T));
    }
    #endregion

    internal static void Init()
    {
    }
}

Simply speaking, it merely alters the static constructor. This works because each closed generic type is a new type and thus calls the static constructor for each unique set of type parameters. Later on I'll further the functionality for obtaining the constraint information, for parametered constructors, on type arguments. That's what the placeholder struct TCtorData is for, and why there's a generic attribute pointing to it for. Basically it'll be used as a data source for the constraint information, since there is no built in functionality for it.

Download the WidgetWorks code

Type-Inference - Part 1

Well,

I finally got to a point where I can start the fun challenging aspects involved in the new languages. Smart type inference. I'm just posting a few notes about it for others to either laugh about or reply to. First I'll start with Lambda expressions and how to properly discern a good match from a not-so-good match.

Figure 1

Basically a lambda signature type, if encountered as a parameter of a call, shall yield explicit or implicit T_n elements and a T_r element for the result. Each signature will only match with a relative RoughSignatureMatch (see: Fig. 1) series if the delegate type assigned has the same parameter count, some or all of the type-paramters will be filled in initially based upon other known data, such as the source of the call (if referring to an extension method), and the data-types associated to the non-lambda call parameters. In cases where a match cannot be found this is when you come to the conclusion: "Type parameters cannot be inferred, try defining them explicitly."

I'll try to make the constructors of the lambda expressions as straight forward as possible. The specifics towards the type inference here will require a certain bit of analysis of the statement/expression associated to the lambda expression, based upon the yielded result.

There's a lot of work to be done, this might actually require a later revisit once the statement/expression model is further. I still have to rewrite a large portion of the internal auto-linking system to enable a more realistic view of how it should be done. Especially since the contextual information needed to do some of the inference won't be immediately available.

The other reason for the rewrite of the linking system is with a compiler context provided, I can actually emit error messages/warnings as needed, presently it throws errors, halting the current linker state and generally causing difficulty handling things, especially since I didn't add the appropriate code to handle the errors when they are thrown.

A good example of this is how it relates to the namespace includes (using) of the current context. Since this project is aimed at hobbyist lingual theorists, they'll need something more than a single using reference, so the linking methods will need to provide a compiler context that can give them this contextual information.

A backwards look of the current expression tree might also provide more information that might not be immediately obvious, but this is something I'll investigate as I go further into the type inference. The extra data mostly applies to closures and lifting them into appropriate generated display classes on translation into CIL.

C♯ - Fast Array Initialization

I've been researching the .NET Common Intermediate Language (CIL) lately, and I noticed a few funny things in how C♯ exports array initializers.

If you use a primitive type, such as Int32, Int64, Byte, SByte, and so on, instead of creating a new array and initializing each element individually, it merely creates a new array and pushes a RuntimeFieldHandle of a static field stored privately.

Now to help illustrate, lets use an example, starting with a sample C♯ Array:

byte[, ,] o0 = new byte[,,] {
   {
       { 0x01, 0x02, 0x03, 0x1C },
       { 0x04, 0x05, 0x06, 0x1D },
       { 0x07, 0x08, 0x09, 0x1E }
   },
   {
       { 0x0A, 0x0B, 0x0C, 0x1F },
       { 0x0D, 0x0E, 0x0F, 0x20 },
       { 0x10, 0x11, 0x12, 0x21 }
   },
   {
       { 0x13, 0x14, 0x15, 0x22 },
       { 0x16, 0x17, 0x18, 0x23 },
       { 0x19, 0x1A, 0x1B, 0x24 }
   }
};

The C# Compiler translates this into a class that contains two things:

1. A struct with a StructLayout attribute applied. The size is the length of the overall set of data; the packing = 1, and the LayoutKind is Explicit. This instructs the CLR to manage the data of the struct at that size, specifically. No members are defined in this 'filler' type. Since it's merely used as a placeholder for data.
2. A field with the Field Type as the struct from above. Instead of initializing like most C# fields do, it uses a CIL feature that C# doesn't explicitly allow, data 'at' a named location.

From the example above, it would define the field like so:

.field assembly static valuetype  '<PrivateImplementationDetails>{17D2CA44-BFFB-4117-B3DF-49EC5806703D}'/'__StaticArrayInitTypeSize=36' '$$method0x6000001-1' at I_000020D0

The data would be defined:

.data cil I_000020D0 = bytearray (01 02 03 1C 04 05 06 1D 07 08 09 1E 0A 0B 0C 1F 0D 0E 0F 20 10 11 12 21 13 14 15 22 16 17 18 23 19 1A 1B 24)

When you want to load the information and initialize the array, you would do something like so:

.method public static hidebysig void main() cil managed
{
   .entrypoint
   .maxstack 3
   .locals init (
       [0] uint8[0...] ja)

   //Load the constant 4-byte integer '36' onto the stack
   ldc.i4.s 36
   //Create a new instance of an array and push it onto the stack.
   newobj instance void uint8[0...]::.ctor(int32)
   //Duplicate the reference on the stack, since we'll be passing
   //it to a method that doesn't return a value.
   dup
   //Load the RuntimeFieldHandle of the data blob onto the stack.
   ldtoken field valuetype '<PrivateImplementationDetails>{17D2CA44-BFFB-4117-B3DF-49EC5806703D}'/'__StaticArrayInitTypeSize=36' '<PrivateImplementationDetails>{17D2CA44-BFFB-4117-B3DF-49EC5806703D}'::'$$method0x6000001-1'
   //Initialize the array.
   call void [mscorlib]System.Runtime.CompilerServices.RuntimeHelpers::InitializeArray(class [mscorlib]System.Array, valuetype [mscorlib]System.RuntimeFieldHandle)
   //Store the array
   stloc.0
  
}

So effectively to do the same in the Abstraction Project on translation into CIL, all that would need done is:

1. A check of the data-type for the array, once confirmed...
   
2. Enumerate the elements of the array using enumeration methods, regardless of the number of dimensions.
3. Encoding the data relative to bit-size per element and making the relationships accordingly.

Fairly simple setup, but it took a while to figure out, .NET Reflector doesn't expose all the details, a few were left out. I contacted Lutz Roeder and he said he was busy at the moment and to re-send the message in two weeks.

Compiled Types - Part II

Finished the limited load aspect of the Compiled types. Now even if you access a property like so:
console.Members["WindowHeight"] <-- the 'Members' will only encapsulate the WindowHeight property. The difference there is the indexer on the Members returns a MasterDictionaryEntry<IMember> instance instead of a property. The reason for this is in order to make the members dictionary work, I used a master/subordinate system where the Members Dictionary is the master dictionary and the sub-types (method, property, indexer, field, constructor, et al.) are subordinates of the master. When the Members property is requested, it is propagated with the reflection instances (along with a pair of the subordinate that contains that instance, so it can properly link back to the kind of element, i.e. property or field or so on). When a specific instance is requested, it connects with the appropriate subordinate and requests the fully qualified IMember, giving the subordinate the appropriate reflection object.

The appropriate dictionary helper properties, Keys and Values respectively, are also in on the game and appropriately only request the necessary information when needed. This ensures that the least load is placed at any given time. There are only two drawbacks to this, one is subsequent calls are minutely slower, but it's a decent trade-off as opposed to creating a wrapper object for every reflection element just by requesting the .Members of a type; the other is it really means you can't use the Debugger Visualizers to drop down and view the properties of a type and expect to be given a list of the members on that type. All you'll get is a series of reflection objects and the Keys/Values will present a series of translated nulls, except for the areas where you've actually requested a specific member.

Expressions - Primitive Reduction

Just thought I'd post on the latest changes to the OIL framework, part of the Abstraction framework.

Recently I wrote a mini evaluator for the primitive expressions inside the system. This way if you had a statement that, for whatever reason, utilized literal mathematical expressions, it would properly reduce them in such a way that the resulted code would contain the least possible execution to get the same result.

I'll probably have to go and rewrite it, but here's an example:

-(((float)8) / 9 * (33 + 4) * 9 / 99 * -(3 + 9))

The code above is pretty trivial, naturally someone would be really silly to do that in code, and even the C# Compiler reduces that to the following: 35.87879F

The process to reduce the expression is simple, since I've structured OIL to mimic the order operations found in most languages. To handle the operations, it merely goes through and checks if each expression and sub-expression can be reduced. If it can, it reduces.

There's a loop to handle multiple passes that pegs out at 100 runs, in case there's a bug in the code. Once I've verified that it works solidly, I'll remove the cap.

Look familiar? 35.87879.

Compiled Type Optimization

I was going through the Compiled Type system and was irritated because the system auto-loaded every member, always, whenever you merely accessed the property associated to them.

For example, if I have a reference to Console, if I accessed its 'Properties'; before, it would load all the properties (which is fine) and then encapsulate every member with the framework's equivalent. While that's fine, it might make more sense to load the MemberInfo relative to the properties, and only instantiate the wrapper classes when they're called for. So that's exactly what I did.

The example below:

ICompiledClassType console = typeof(Console).GetTypeReference<ICompiledClassType>();
ITypeReferenceExpression itre = console.GetTypeExpression();
IMethodInvokeExpression imie = itre.GetMethod("WriteLine").Invoke((ExpressionBase)"Console Dimensions: ({0}, {1})", itre.GetProperty("WindowWidth"), itre.GetProperty("WindowHeight"));
IType readKeyResult = imie.ForwardType;
IMethodInvokeExpression imie2 = itre.GetMethod("ReadKey").Invoke((ExpressionBase)true);
IType readKeyResult2 = imie2.ForwardType;

When 'ForwardType' on a given expression is accessed, it auto-links the expression and any sub-expressions that are dependent upon it.

In the case above, IMethodInvokeExpression imie = . . . properly resolves to 'String' and 'Int32', 'Int32', and accordingly links to the associated member:

void Console.WriteLine(System.String, System.Object, System.Object)

The reason such indirect and lengthy optimizations are important, is if you're dealing in very large scale code generation, if it instantiated a property instance for all of the Console's properties to access its WindowWidth property, then things would suddenly start to take a long time.

As for the actual linking of methods, there wasn't really that much that could be done. In order to properly perform the lookups I went ahead and used the framework I put in place. It still does the on-access instantiation, but the iterative nature of the filtering methods ends up loading all of the elements anyway, to ensure the most accurate match is provided.