Summary: Parallel FX library
The Parallel Extensions team has a technical talk on Channel 9, "Parallel Extensions: Inside the Task Parallel Library".
Here are my notes, if you don't have time to watch the hour long video.
You can find more details about the Parallel FX library and other work by Microsoft in that area from the Parallel Computing Developer Center.
It has download links for the CTP and reference documentation, as well as pointers to the team's blogs.
Parallel FX is a .Net 3.5 library which makes it easier to write parallel code, which takes advantage of multi-core machines.
Parallel.For, Parallel.While, ...
It presents some constructs such as Parallel.For and Parallel.Do. These can run a function or set of functions in parallel, and handle the join synchronization.The parallelism declared by the code is not guaranteed at runtime, the library manages the tasks which are queues and their allocation on the different cores is dynamic, depending on availability.
Much of the value of the library is in its low overhead and load balancing smarts (work stealing). Also, it helps handling parallel exceptions.
In any case, the amount of performance gain you should expect from parallelizing the execution onto multiple cores is limited by the portion of your application which remains sequential (as stated by Amdhal's law), plus the overhead of parallel execution.
Task, Future
The primitives above are built on top of the lightweight Task abstraction, which encapsulates a piece of independent computation (lambda function) and later wait for the result.The Future primitive is another way to lazily request the result of a function.
Either way, the computation will start in parallel if some cores are idle (no more than one OS thread per core, for best results).
Otherwise, it will be performed sequentially when your code requests the result (Task.Wait, Future.Value).
PLINQ
A third way to access the functionality of this library is by using the PLINQ classes and LINQ query syntax.I don't know much specifics about PLINQ. But I think that it centers around IParallelEnumerable, which is an implementation of the .NET Standard Query Operators, and handles some of the primitives (such as Filter) in a parallel way. Any IEnumerable can be converted to an IParallelEnumerable thru the .AsParallel() extension method.
By the way, the C# Version 3.0 Specification gives a good sense of how the LINQ query syntax gets translated into applications of the standard query operators.
Conclusion
The PFX library still requires a good level of developer awareness to parallelism issues.For example, it does not prevent bad concurrent code from being written.
Also, it pushes a number of trade-offs onto the developer.
That said, the library does make it easier to experiment and express different parallelization approaches.
And for those who need finer grain control of how the Tasks are executed can create different TaskManagers to implement different policies.
Stephen Toub illustrates the library on a concrete problem (parallel tree traversal), iterating on different manual implementations to finally converge on the Parallel FX model.
The resulting code is definitely very clean, and it efficiently handles a number of problems for you.