The Microsoft 365 Community is introducing a public preview of a new project called “Microsoft 365 PnP Transformation Framework.” The goal of this new project is to help customers and partners to transform existing portals into Microsoft SharePoint Online modern portals.

Architectural and Functional Overview

The project is the evolution of an already existing project called “PnP Modernization Framework,” introduced a couple of years ago that targets modernization of single pages from SharePoint Online and SharePoint on-premises (2010, 2013, 2016, 2019) toward “modern” SharePoint Online sites. However, the new Transformation Framework is based on a generic model that can be used to transform any web-based/content-based solution into a modern SharePoint Online portal.

In fact, the new framework is built with an open and abstract architecture, supporting modern development techniques like dependency injection and asynchronous development. Internally, the new framework is built on top of the PnP Core SDK, meaning that it targets .NET Standard 2.0 and .NET 5.0, and you can use it on any platform (Windows, Linux, macOS) and in any cloud-based service. For example, it will be really easy to create an Azure Function to transform “on-the-fly” a page of content into a modern page of SharePoint Online:

As you can see in Figure 1, the framework is based on some pillars. First, there’s the concept of Data Source, which is “something” that you can use to read content that you want to transform into modern pages. Data sources are pluggable and based on external libraries that can be created by anyone in the community.

Then, there is a “Transformation Engine,” which is the actual engine that takes care of transforming content retrieved from any external Data Source. At a high level, the Transformation Engine relies on a set of pluggable mappers, which can be easily configured and customized using dependency injection in .NET Core/.NET 5.0. Last but not least, a transformed page will be saved into a modern site of SharePoint Online, which is the only and unique output supported by the framework.

Initially, the framework supports SharePoint Online and SharePoint on-premises (2013, 2016, and 2019) as the available data sources. However, in the future, we expect to see many more data sources to be released by partners or by community projects. For example, there could be a data source for WordPress, or any other content management system available in the market.

From a functional point of view, you can simply transform a single page with an in-process model, or you can transform sites or batches of pages with a distributed architecture hosted, for example, on Microsoft Azure. The PnP Transformation Framework natively provides all the architectural components and customization endpoints to allow a developer to create a distributed architecture.

For example, there are components useful to decouple and scale the processing of transformations using a back-end Azure Function and an Azure Blob Storage Queue to process the transformations asynchronously. You will be able to use the framework as a reference in a custom solution of your own, or you will simply use it via PnP PowerShell to trigger the transformation of single pages via a PowerShell script. Regardless of what your goal is, the new PnP Transformation Framework provides you with the right tools to work with.

In Figure 2 you can see the high-level architecture of the Transformation Engine:

As you can see, there is a Transformation Executor component that manages a Transformation Process. The Transformation Process is made of Transformation Tasks, which can be executed live and in-process, or which can be transferred to a back-end system via a Transformation State Manager like an Azure Blob Storage Queue or an Azure Service Bus Queue, just to make few examples.

The Transformation Tasks are generated by a Transformation Distiller, which is nothing more than an object that analyzes the Data Source and defines a list of pages to transform. In the Transformation Executor, which by default runs in-process but can also be in a back-end system, happens the actual transformation, relying on a set of mapping providers.

Using the PnP Transformation Framework

From a developer point of view, you can easily use the framework simply by plugging into your projects one or more of the following NuGet packages:

• PnP.Core.Transformation: to simply get a reference to the transformation engine and to the abstract types and interfaces. You can use it, whenever you want to create a custom data source and you need to implement all the abstract classes and interfaces in your own custom library.
• PnP.Core.Transformation.SharePoint: to reference the transformation engine that targets SharePoint Online and/or SharePoint on-premises as the supported data sources.

To use the new PnP Transformation Framework you simply need to configure a .NET project to support dependency injection, and then you can easily set up the services you need. In the following code excerpt, you can see how to trigger the transformation of a single page in a .NET Console application created with .NET Core 3.1 and configured to use dependency injection through the Microsoft.Extensions.Hosting NuGet package, with interactive authentication:

var host = Host.CreateDefaultBuilder()
// Configure services with Dependency Injection
.ConfigureServices((hostingContext, services) =>
{
    // Add the PnP Core SDK library services
    services.AddPnPCore();

    // Add the PnP Core SDK library services configuration from the appsettings.json file
    services.Configure<PnPCoreOptions>(hostingContext
        .Configuration.GetSection("PnPCore"));

    // Add the PnP Core SDK Authentication Providers
    services.AddPnPCoreAuthentication();

    // Add the PnP Core SDK Authentication Providers 
    // configuration from the appsettings.json file
    services.Configure<PnPCoreAuthenticationOptions>(hostingContext
        .Configuration.GetSection("PnPCore"));

    // Add the PnP Transformation Framework services with some custom settings
    services.AddPnPSharePointTransformation(
        pnpOptions => 
        {
            pnpOptions.PersistenceProviderConnectionString = @"c:\temp"
        },
        pageOptions =>
        {
            pageOptions.CopyPageMetadata = true;
            pageOptions.KeepPageSpecificPermissions = true;
            spOptions.TargetPagePrefix = "Migrated_";
        }
        spOptions =>
        {
            spOptions.RemoveEmptySectionsAndColumns = true;
            spOptions.ShouldMapUsers = true;
        }
    );
})

// Let the builder know we're running in a console
.UseConsoleLifetime()
// Add services to the container
.Build();


// Start console host
await host.StartAsync();

// Optionally create a DI scope
using (var scope = host.Services.CreateScope())
{
    // Create a PnP Core SDK Context Factory
    var pnpContextFactory = scope.ServiceProvider
        .GetRequiredService<IPnPContextFactory>();

    // Create a CSOM ClientContext using whatever technique you like, to read the source
    ClientContext sourceContext = ...; // Up to you how to create the ClientContext

    // Create a PnPContext of PnP Core SDK to write the target
    var targetContext = await pnpContextFactory.CreateAsync(TestCommon.TargetTestSite);

    // Define the URI of the source page
    var sourceUri = new
        Uri("https://<tenant>.sharepoint.com/sites/Classic/SitePages/sourcePage.aspx");

    // Transform the page
    var result = await pageTransformator.TransformSharePointAsync(
        sourceContext, targetContext, sourceUri);

}

// Cleanup console host
host.Dispose();

As you can see, there is some plumbing to configure the dependency injection context and for creating the source and target contexts, but at the very end the transformation itself is just a method call: TransformSharePointPageAsync.

When defining and configuring the services for dependency injection, you can always register your own customizations of the mappers and services, in order to plug in your own custom logic. For example, if you want to provide a custom user mapping logic, you can simply override the default behavior configuring your own service.

By default, the AddPnPSharePointTransformation method of PnP.Core.Transformation.SharePoint configures a set of pre-defined mapping services with basic functionalities.

Another option that you have, if you like to use the new PnP Transformation Framework via PowerShell, is to rely on the new Invoke-PnPTransformation cmdlet, which is available under preview in the PnP PowerShell library. Here you can see a sample excerpt of PowerShell code invoking the new cmdlet:

# Connect to the target site
$targetConnection = Connect-PnPOnline https://<tenant>.sharepoint.com/sites/Modern -ReturnConnection

# Connect to the source site
Connect-PnPOnline https://<tenant>.sharepoint.com/sites/Classic 

# Trigger transformation
Invoke-PnPTransformation -Identity sourcePage.aspx -TargetConnection $targetConnection

Aside from connecting to the target and source sites, you simply need to invoke the cmdlet providing the name of the page to transform. Of course, you can provide plenty of options to customize the transformation, but the very basic syntax is simple like what you see in the above sample.

Inside the PnP Transformation Framework

Internally the transformation engine relies on a set of customizable mappers. Here is the list:

• Metadata Mapping Provider: handles page metadata and converts metadata of the source content into field values in the target SharePoint Online page item.
• Page Layout Mapping Provider: manages the definition of the Page Layout and it is used only when transforming from SharePoint publishing pages.
• Taxonomy Mapping Provider: takes care of transforming taxonomies into values supported by the Managed Metadata Service of the target SharePoint Online environment.
• Url Mapping Provider: converts URLs referenced inside the source document into actual URLs in the target environment.
• User Mapping Provider: maps users in the source platform to Microsoft 365 users in the target SharePoint Online environment.
• Web Part Mapping Provider: this is the core of the transformation engine and converts content widgets, components, or web parts of the data source into client-side web parts of modern SharePoint Online.

When you execute a transformation the engine goes through some of the above mappers in order to create an abstract definition of the content page and then relies on a Page Generator to create the actual modern page in SharePoint Online. As such, the data source can be any platform, as long as the specialized mappers can generate the abstract definition of the content needed by the Page Generator.

Wrap up

The current plan is to release the GA version of the library by the end of 2021. Right now, you can play with the public preview. Give it an eye, play with it, test it and let us know what you think of this new project. Last but not least, remember that all the projects and libraries described in this article are open source.

As such, you can contribute in many different ways. For example, you can simply provide feedback using the issue list of the corresponding repositories. But you can also submit proposals for code changes, creating Pull Requests on GitHub.

Moreover, you can actively contribute to community life by attending the weekly community calls. If you don’t know how to start, open your browser, navigate to http://aka.ms/m365pnp and look for the “Sharing is caring” initiative. There you will find other community members willing to help you start your journey in the amazing Microsoft 365 Community! See you there, then!