Saturday, July 26, 2008

Light Refactoring turns into Major Refactoring

Well,

As much as I'd like to keep the project simple, it hasn't been simple from day one. Due to the features I'm planning on, I figured it best if I restructured the project, the Scripting Language Foundation, into a series of projects. There's the primary Type System, the CLI Type system, and the OIL (Intermediate) Type System. These three systems are best kept separate to help simplify project management. I also just got tired of scrolling through over a thousand files in one project.

The advantage of the larger scale refactoring is quicker build times (yay) so long as I don't modify the core type system (which I'm still doing, so 'boo').

Another thing I'm hoping this will result in is the ability to allow for the system to be lightly retargetable in the future. So long as there's an alternative for the CLI type system, and the CIL in general. The abstract type system, as it is, really just defines things in general sense; the primary reasons it relies on the CLI type system are for the base-types of arrays, byref types, enumerations, structures, and so forth (a simple retarget, if another target exists). Granted there are probably very few targets that are as high level as the abstract type system specifies.

Here's hoping things go well. The refactoring presently copmiles. Instead of one quite sizable library, it's now broken down into concepts:

  1. Abstract Syntax Tree
    1. Abstract
    2. Common Language Infrastructure
    3. Objectified Intermediate Language
  2. Compilers
  3. Languages
    1. Abstract
    2. C♯
    3. Common Intermediate Language (CIL)
    4. Visual Basic.NET
  4. Linkers
  5. Transformation
  6. Translation

The Abstract Syntax Tree (AST) for the first two targets, Abstract and CLI, are more type-systems than ASTs. The third, OIL, is a type-system and malleable infrastructure that injects more code-principles than the first two.

The transformation stage will utilize information about the language as stipulated by contextual data, which is defined in the Language Abstract (a given language specifies what's supported, this information is piped through the appropriate channels that use the transformers, such as compilers and code translators). The transformation framework will utilize the limitations of the language to determine what parts of the intermediate structure need transformations applied and which ones to apply.

It will also verify that the code as presented is even valid for that language, given its limitations. Certain features of the CLI don't have workarounds, such as 'base', 'MyClass', while others do, a la Lambdas. If it can't already explicitly call a virtual method non-virtually in an appropriate context (base, current scope, et cetera), there's no amount of trickery that can allow for it.

You could, in theory, write an adapter class for such instances that would emit the proper CIL to handle the task at hand, but the usability and maintenance of such code would be... questionable at best. It's best to flag the region as invalid for that particular language. The areas marked as 'invalid' for a language are typically why two languages are not 100% interoperable between each other. Visual Basic code can't always be translated to C♯ code, same applies to C♯.

Thursday, July 3, 2008

Light Refactoring

After a short delay, due to work, I'm back working on this.

I'm doing a little clean up work on the core type system and rewriting the assist methods that are associated to the functionality, that's presently broken. This will greatly assist in streamlining the code and reduce the dependency on internal components that were used before. I also hope to be able to open up the type system a little more to allow for extensibility; though I'm not sure to what this will result in, overall.

Adjusted the disambiguation and type-parameter verification aspects to simplify things in the end.

So far things are looking OK; however, I'll know more when it compiles (280 compile errors due to restructure, slowly eliminating them one by one.)

   1:  /// <summary>
2: /// Verifies the type-parameters of a generic method
3: /// <paramref name="signature"/> with the <paramref name="typeReplacements"/>
4: /// provided.
5: /// </summary>
6: /// <typeparam name="TSignatureParameter">The type of parameter used in the <typeparamref name="TSignature"/>.</typeparam>
7: /// <typeparam name="TGenericParameter">The type of generic type-parameter used in the <typeparamref name="TSignature"/></typeparam>
8: /// <typeparam name="TGenericParameterConstructor">The type used to signify the constructors used on the
9: /// generic parameter.</typeparam>
10: /// <typeparam name="TGenericParameterConstructorParameter">The type used to signify
11: /// the parameters on the constructors of the generic parameters.</typeparam>
12: /// <typeparam name="TSignature">The type of signature used as a parent of <typeparamref name="TSignatureParameter"/> and
13: /// <typeparamref name="TGenericParameter"/> instances.</typeparam>
14: /// <typeparam name="TSignatureParent">The parent that contains the <typeparamref name="TSignature"/>
15: /// instances.</typeparam>
16: /// <param name="signature">The <see cref="IMethodSignatureMember{TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent}"/>
17: /// to verify the <paramref name="typeReplacements"/> against.</param>
18: /// <param name="typeReplacements">The <see cref="ITypeCollection"/> that defines
19: /// the replacement types to verify.</param>
20: /// <remarks>TGeneric* type-parameters are primarily used to complete the hierarchy
21: /// chain to allow for type strict reverse-traversal.</remarks>
22: public static void VerifyTypeParameters<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>(this IMethodSignatureMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent> signature, ITypeCollection typeReplacements)
23: where TSignatureParameter :
24: IMethodSignatureParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
25: where TGenericParameter :
26: IMethodSignatureGenericTypeParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
27: where TGenericParameterConstructor :
28: IMethodSignatureGenericCtorMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
29: where TGenericParameterConstructorParameter :
30: IMethodSignatureGenericCtorParamMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
31: where TSignature :
32: IMethodSignatureMember<TSignatureParameter, TGenericParameter, TGenericParameterConstructor, TGenericParameterConstructorParameter, TSignature, TSignatureParent>
33: where TSignatureParent :
34: ISignatureParent<TSignature, TSignatureParameter, TSignatureParent>
35: {
36: if (signature == null)
37: throw new ArgumentNullException("signature");
38: if (typeReplacements == null)
39: throw new ArgumentNullException("typeReplacements");
40: /* *
41: * Setup the test case parameter logic.
42: * *
43: * If the method belongs to a generic type, mine the
44: * generic type-parameters from it to help in
45: * properly disambiguating the method's type-parameter
46: * constraints.
47: * */
48: ITypeCollection parentTypeReplacements = TypeCollection.Empty;
49: TypeParameterSource source = TypeParameterSource.Method;
50: if (signature.Parent is IGenericType && ((IGenericType)(signature.Parent)).IsGenericType)
51: {
52: IGenericType parent = ((IGenericType)(signature.Parent));
53: /* *
54: * Special case on when the parent is a generic definition.
55: * It's still a valid call if the type-replacements passed
56: * contain enough replacements to fill the parent's
57: * and the method's type-parameters. Least likely case.
58: * *
59: * Obtain the replacements
60: * */
61: if (parent.IsGenericTypeDefinition)
62: if (typeReplacements.Count == parent.GenericParameters.Count + signature.GenericParameters.Count)
63: {
64: parentTypeReplacements = typeReplacements.Take(parent.GenericParameters.Count).ToCollection();
65: typeReplacements = typeReplacements.Skip(itc.Count).ToCollection();
66: source = TypeParameterSource.Both;
67: }
68: else
69: throw new ArgumentException("typeReplacements");
70: else
71: {
72: parentTypeReplacements = parent.GenericParameters;
73: source = TypeParameterSource.Both;
74: }
75: }
76: else if (signature.TypeParameters.Count != typeReplacements.Count)
77: throw new ArgumentException("typeReplacements");
78: VerifyTypeParameters_VerifyReplacements(typeReplacements);
79: /* *
80: * Generate test-case generic parameters
81: * to do the verification with.
82: * *
83: * Logic to handle the parentTypeReplacements is
84: * defined above.
85: * */
86: ITypeCollection testCases =
87: from IGenericTypeParameter t in signature.GenericParameters
88: select new GenericVerificationParameter(
89: t.Constraints.OnAll(
90: k => k.Disambiguify(
91: parentTypeReplacements,
92: typeReplacements,
93: source)).ToCollection(), t);
94: VerifyTypeParametersInternal(typeReplacements, testCases);
95: }
96:  
97: /// <summary>
98: /// Verifies a set of <paramref name="typeReplacement"/> <see cref="IType"/> instances
99: /// against the type-parameters defined on the <paramref name="genericType"/>.
100: /// </summary>
101: /// <param name="genericType">The <see cref="IGenericType"/>
102: /// which contains the parameters to verify against.</param>
103: /// <param name="typeReplacements">The <see cref="ITypeCollection"/> that defines
104: /// the replacement types to verify.</param>
105: public static void VerifyTypeParameters(this IGenericType genericType, ITypeCollection typeReplacements)
106: {
107: if (genericType == null)
108: throw new ArgumentNullException("genericType");
109: if (typeReplacements == null)
110: throw new ArgumentNullException("typeReplacements");
111: if (genericType.TypeParameters.Count != typeReplacements.Count)
112: throw new ArgumentException("typeReplacements");
113: VerifyTypeParameters_VerifyReplacements(typeReplacements);
114:  
115: /* *
116: * Obtain a series of generic verifiers that act as dummy
117: * generic parameters for the checks to be performed.
118: * *
119: * On method type-parameter verification,
120: * the 'typeReplacements' parameter will be
121: * used on the methodReplacements parameter
122: * in k.Disambiguify; whereas the
123: * method's parent type-parameters will only
124: * be included if it is: An IGenericType
125: * and that generic type is actually IType.IsGenericType.
126: * */
127: ITypeCollection testCases =
128: from IGenericTypeParameter t in genericType.GenericParameters
129: select new GenericVerificationParameter(
130: t.Constraints.OnAll(
131: k => k.Disambiguify(
132: typeReplacements,
133: TypeCollection.Empty,
134: TypeParameterSource.Type)).ToCollection(), t);
135: VerifyTypeParametersInternal(typeReplacements, testCases);
136: }
137:  
138: private static void VerifyTypeParameters_VerifyReplacements(ITypeCollection typeReplacements)
139: {
140: for (int i = 0; i < typeReplacements.Count; i++)
141: if (typeReplacements[i] == null)
142: throw new ArgumentNullException(string.Format("typeReplacements[{0}]", i));
143: //Pointer types, by-reference types, and the void type cannot be generic type parameters.
144: else if (typeReplacements[i].ElementClassification == TypeElementClassification.Pointer ||
145: typeReplacements[i].ElementClassification == TypeElementClassification.Reference ||
146: typeReplacements[i] is ICompiledType && ((ICompiledType)typeReplacements[i]).UnderlyingSystemType == typeof(void))
147: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure_InvalidType, typeReplacements[i].ToString()));
148: }
149:  
150: private static void VerifyTypeParametersInternal(ITypeCollection typeReplacements, ITypeCollection testCases)
151: {
152: /* *
153: * Iterate through the replacements and compare them against
154: * the test cases.
155: * */
156: for (int i = 0; i < testCases.Count; i++)
157: {
158: IGenericParameter param = (IGenericParameter)(testCases.Values[i]);
159: IType replacement = typeReplacements[i];
160: /* *
161: * Generic parameters require special processing.
162: * */
163: if (replacement.IsGenericTypeParameter)
164: {
165: IGenericParameter replacementParam = ((IGenericParameter)(replacement));
166: if (param.RequiresNewConstructor && !replacementParam.RequiresNewConstructor)
167: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
168: /*{0}*/ replacementParam.Name,
169: /*{1}*/ param.Name,
170: /*{2}*/ Resources.TypeConstraintFailure_NewConstraint));
171: if (!(param.SpecialConstraint == GenericTypeParameterSpecialConstraint.Class &&
172: (replacementParam.SpecialConstraint == GenericTypeParameterSpecialConstraint.Class ||
173: (replacementParam.Constraints.Count > 0 &&
174: replacementParam.Constraints[0] is IReferenceType))))
175: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
176: /*{0}*/ replacementParam.Name,
177: /*{1}*/ param.Name,
178: /*{2}*/ Resources.TypeConstraintFailure_ReferenceType));
179: else if (param.SpecialConstraint == GenericTypeParameterSpecialConstraint.Struct &&
180: replacementParam.SpecialConstraint != GenericTypeParameterSpecialConstraint.Struct)
181: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
182: /*{0}*/ replacementParam.Name,
183: /*{1}*/ param.Name,
184: /*{2}*/ Resources.TypeConstraintFailure_ValueType));
185: /* *
186: * Verify each constraint, they were translated from their
187: * earlier form into a type-parameter resolved form.
188: * */
189: foreach (IType constraint in param.Constraints)
190: if (!replacementParam.Constraints.Any(replacementConstraint => constraint.IsAssignableFrom(replacementConstraint)))
191: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
192: /*{0}*/ replacementParam.Name,
193: /*{1}*/ param.Name,
194: /*{2}*/ string.Format(Resources.TypeConstraintFailure_ParamConstraint, constraint.IsGenericTypeParameter ? constraint.Name : constraint.FullName)));
195:  
196: }
197: else
198: {
199: /* *
200: * Structs and enumerations are both value types and
201: * automatically contain a default constructor.
202: * */
203: if (param.RequiresNewConstructor && !(replacement.Type == TypeKind.Struct || replacement.Type == TypeKind.Enumerator))
204: {
205: if (replacement.Type == TypeKind.Interface || (!(replacement is ICreatableType)))
206: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
207: /*{0}*/ replacement.Name,
208: /*{1}*/ param.Name,
209: /*{2}*/ Resources.TypeConstraintFailure_NewInterfaceDelegateOther));
210: ICreatableType creatableReplacement = (ICreatableType)(replacement);
211: if (creatableReplacement.Constructors.Find().Count == 0)
212: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
213: /*{0}*/ replacement.Name,
214: /*{1}*/ param.Name,
215: /*{2}*/ Resources.TypeConstraintFailure_NewStandard));
216: }
217:  
218: /* *
219: * SpecialConstraint check.
220: * */
221: switch (param.SpecialConstraint)
222: {
223: case GenericTypeParameterSpecialConstraint.Struct:
224: if (replacement.Type != TypeKind.Enumerator ||
225: replacement.Type != TypeKind.Struct)
226: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
227: /*{0}*/ replacement.Name,
228: /*{1}*/ param.Name,
229: /*{2}*/ Resources.TypeConstraintFailure_ValueType));
230: break;
231: case GenericTypeParameterSpecialConstraint.Class:
232: /* *
233: * Since I can't predict the future, I used an
234: * interface to replace specific type checking.
235: * If someone wants to expand the framework
236: * for whatever reason, this makes things easier.
237: * */
238: if (!(replacement is IReferenceType))
239: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
240: /*{0}*/ replacement.Name,
241: /*{1}*/ param.Name,
242: /*{2}*/ Resources.TypeConstraintFailure_ReferenceType));
243: }
244: /* *
245: * Every constraint needs to be assignable
246: * by the replacement.
247: * */
248: foreach (IType constraint in param.Constraints)
249: if (!constraint.IsAssignableFrom(replacement))
250: throw new ArgumentException(string.Format(Resources.TypeConstraintFailure,
251: /*{0}*/ replacement.Name,
252: /*{1}*/ param.Name,
253: /*{2}*/ string.Format(Resources.TypeConstraintFailure_Constraint, constraint.IsGenericTypeParameter ? constraint.Name : constraint.FullName)));
254:  
255: }
256: /* *
257: * Dispose the type-parameter test case, it's no longer needed
258: * */
259: param.Dispose();
260: }
261: }
262:  
263: /// <summary>
264: /// Resolves generic type-parameters contained within the
265: /// <paramref name="target"/> <see cref="IType"/> by
266: /// rebuilding the <paramref name="target"/> with
267: /// the <paramref name="typeReplacements"/> used in place of
268: /// the <see cref="IGenericTypeParameter"/> instances.
269: /// </summary>
270: /// <param name="target">The target <see cref="IType"/>
271: /// to disambiguify.</param>
272: /// <param name="typeReplacements">The <see cref="IType"/>
273: /// series which contains a series of types that is index-relative
274: /// to the <paramref name="target"/>'s type-parameters.</param>
275: /// <param name="parameterSource">The point to source
276: /// the replacements from.</param>
277: /// <returns></returns>
278: public static IType Disambiguify(this IType target, ITypeCollection typeReplacements, ITypeCollection methodReplacements, TypeParameterSource parameterSource)
279: {
280: /* *
281: * Assumes a great deal in that: Types provided are used in scope.
282: * Beyond scope this does not work as intended since the positions
283: * of out-of-scope type-parameters might not be relatively
284: * equivalent to the scope they are used in (typeReplacements).
285: * */
286: if (target.IsGenericTypeParameter)
287: {
288: if (target is IGenericParameter)
289: {
290: /* *
291: * If the declaring type is null, this method assumes
292: * it's a method. Primarily to aid in possible other
293: * uses.
294: * */
295: if ((parameterSource & TypeParameterSource.Method) == TypeParameterSource.Method &&
296: ((IGenericParameter)(target)).DeclaringType == null &&
297: ((IGenericParameter)(target)).Position < methodReplacements.Count)
298: return methodReplacements[((IGenericParameter)(target)).Position];
299: if ((parameterSource & TypeParameterSource.Type) == TypeParameterSource.Type &&
300: ((IGenericParameter)(target)).DeclaringType != null &&
301: ((IGenericParameter)(target)).Position < typeReplacements.Count)
302: return typeReplacements[((IGenericParameter)(target)).Position];
303: }
304: }
305: else
306: {
307: switch (target.ElementClassification)
308: {
309: case TypeElementClassification.Array:
310: if (target is IArrayType)
311: return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeArray(((IArrayType)target).ArrayRank);
312: break;
313: case TypeElementClassification.Nullable:
314: return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeNullable();
315: case TypeElementClassification.Pointer:
316: return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakePointer();
317: case TypeElementClassification.Reference:
318: return target.ElementType.Disambiguify(typeReplacements, parameterSource).MakeByReference();
319: case TypeElementClassification.GenericTypeDefinition:
320: if (target.ElementType is IGenericType)
321: return ((IGenericType)target.ElementType).MakeGenericType(((IGenericType)target).GenericParameters.OnAll(gP => gP.Disambiguify(typeReplacements, parameterSource)));
322: break;
323: case TypeElementClassification.None:
324: if (target is IGenericType &&
325: (parameterSource == TypeParameterSource.Type && ((IGenericType)(target)).GenericParameters.Count == typeReplacements.Count))
326: return ((IGenericType)(target)).MakeVerifiedGenericType(typeReplacements);
327: break;
328: }
329: }
330: return target;
331: }