Improvement for building library on parallelism

[kekyo]

The current runtime library build method (#79) follows a simple conflict avoidance method similar to the ConcurrentDictionary<K,V>.GetOrAdd(key, func) we are all familiar with, where the generator (in this case the runtime library compilation) is run and finally It follows a simple conflict avoidance method, leaving only one task to save the result and discarding all the rest.

The disadvantage of this approach is that it incurs a very large computational cost the first time you do a build from git clean -xfd. We believe that for the majority of users, this has almost no impact, except that it takes longer the first time, but the CI process pays this cost every time.

I have adopted a series of conflict avoidance techniques because of the simplicity of this method (another thing is that MSBuild does not have such flexible control over parallel execution), but we may be able to tighten it up a bit more:

• If there is no directory with the corresponding hash in libcache, a build is required:
  • Wait around 0...hundred ms and then check that the file to be marked exists. This is touch with a file name that can be uniquely identified by the hash value.
  • If the touched file does not exist, start the build. This may still cause slight conflicts, but in most cases they can be avoided.
    (Or, totally could avoid when touch with readonly marked ...?)
  • If the touched file exists, wait for the library generation to complete in a spin loop with sleep.

A series of methods may require writing a few C# inline custom tasks.

We don't see much need to make this improvement just for runtime libraries, but as we showed in #121, it is likely to have an impact on build time if libcache is made available for any nuget library or reference assembly.