Sentiment analysis for linguistics is one of the most studied aspects of Artificial Intelligence (AI). If you search online, you’ll find hundreds of articles about the different algorithms available, their advantages, disadvantages and how to implement them. Here, you’ll come across a lot of information on Sentiment Analysis, Intent Analysis, Contextual Semantics and more.
Sentiment analysis is best defined as the process of identifying the emotion associated with a piece of text. It relies on applying machine learning techniques to classify information based on features extracted from the text.
In the era of social networks, Sentiment Analysis can be an important and highly transparent tool to manage processes: it indicates how the users are emotionally reacting to content and enables you to steer the context into one direction or another.
I might be biased, but Office 365 is by far the best place to store enterprise information. If we start by looking at the extensive options available within the cloud platform. Such as SharePoint, the repository for structured and non-structured information, to Yammer and Teams, which allow users to exchange data directly in chats and meetings. We can see here there are several attractive choices for implementing and applying sentiment analysis to enrich data systems.
On the other hand, Azure offers a ready to use Sentiment Analysis service, based on its Text Analytics API which you can find within the Cognitive Services family. Microsoft has based the Project Cortex, the initiative to make Office 365 and SharePoint Online more ‘intelligent’, on the Azure Cognitive Services family. At the time of writing this article, there is limited information on the technical implementation of Cortex in Office, and if Sentiment Analysis will be a part of it.
The AI models used by the Azure Text Analytics API are provided and trained by Microsoft and ready to use. Sentiment Analysis is staged on the entire offered text, instead of words in the it, and it produces a more refined result when its evaluating smaller pieces of text. Ideally, text size must be under 5,120 characters.
There are two versions of the Text Analytics API. In Version 2, the sentiment analyzer classifies text as predominantly positive or negative and assigns a score between 0 and 1. In Version 3 (in preview now), the text is analyzed with sentiment labels (positive, negative, mixed, or neutral) and a score is given reflecting the model’s confidence.
The API can be used in two ways: an Object Model and a REST API. The REST API sends the information as a JSON string in the body of the call using the format ID, text, and language (all properties obligatory), together with the endpoint URL to the service, and the access key in the request header.
The following example shows how to use the API Object Model to analyze the text in a field from a custom List in SharePoint Online. This is just a proof-of-concept that can be applied to more real-world cases: for example, to apply the analysis to the entry paragraph of Word documents saved in a SharePoint Library, or using another kind of Lists in the classic version of SharePoint (a Survey List, for instance).
Create the Service in Azure
1. Open the Azure portal using an administrators account and create a new Resource Group
2. When the Resource Group is ready, use the +Add button and search for text analytics. Click on the Create button of the service.
3. Fill the information for the service (Name, Subscription, Location and Resource Group). There are seven pricing tiers to choose, from free (including 5000 transactions per month), through paid tiers to 10 million transactions per month. Use the Create button.
4. When the service is created, go to its configuration window and copy the Key1 and Endpoint. This information is always available in the Keys section, if necessary.
5. Because we will use a SharePoint WebHook to connect the SharePoint List with the Azure service, create in the Resource Group another service of the type Function App. Go back to the Resource Group, click on the +Add button, search for function app and create the service.
6. In the configuration window for the Function App fill in the required data (Subscription, Name, Region). Select Code in the Publish box and .NET Core in the Runtime stack
By default, the creation process will make a new Storage account, Application Insights, and Plan (of the type Consumption) services. If you want to select the names for these services, you can use the Next button and do so. Otherwise, just hit the Create button and use the default configuration.
7. The Function App is created using the .NET Core framework. Unfortunately, when this article was created, the SharePoint CSOM (Client Side Object Model) was not yet able to work with .NET Core, only with the .NET Framework. To change it, when the Function App is ready, go to its configuration window, click on Platform features – Function app settings, and change the Runtime version from -2 to -1.
The SharePoint List
8. Create a Custom List in SharePoint and add two fields: one called Comments (Multiple lines of text), and a second called Sentiment (Single line of text)
Creating the code
Use Visual Studio to create the source code. We are using Visual Studio 2019 Enterprise, but you can use any other version (including 2016/2017) or Visual Studio Code.
9. In Visual Studio, create a new Solution of the type Azure Functions. In the Create a new Azure Function Application window, select Azure Functions v1 (.NET Framework) and Http trigger. Then, create the Solution.
10. Add the NuGet packages “Newtonsoft.Json” (version 10.0.3), “Microsoft.Azure.CognitiveServices.Language.TextAnalytics” and “AppForSharePointOnlineWebToolkit”.
11. Use the source code that you can find in the GitHub repository to replace the original code created by the Visual Studio template. The code implementation is a standard SharePoint WebHook that first reviews the hook subscription and then recall and analyze the changes in the List. The relevant parts for the call to the Azure service are:
// Get what is changed
ListItem changedListItem = changedList.GetItemById((oneChange as ChangeItem).ItemId);
SPClientContext.Load(changedListItem);
SPClientContext.ExecuteQuery();
// Create a Text Analytics client
ApiKeyServiceClientCredentials myCredentials = new ApiKeyServiceClientCredentials(serviceKey);
TextAnalyticsClient myClient = new TextAnalyticsClient(myCredentials)
{
Endpoint = serviceEndpoint
};
// Call the service
SentimentResult myScore = myClient.Sentiment(changedListItem["Comments"].ToString(), "en");
// Insert the values back in the Item
changedListItem["Sentiment"] = myScore.Score.ToString();
changedListItem.Update();
SPClientContext.ExecuteQuery();
- When a change in the List is detected, the changed item is recovered from SharePoint
- A client for the Text Analytics is created passing the key and endpoint of the service
- The string from the Comments field is passed to the Azure service, and the response from the service is saved in the myScore variable
- The value of the score is transmitted back to the List item
The Text Analytics client is created using the following class:
class ApiKeyServiceClientCredentials : ServiceClientCredentials
{
private readonly string apiKey;
public ApiKeyServiceClientCredentials(string apiKey)
{
this.apiKey = apiKey;
}
public override Task ProcessHttpRequestAsync(HttpRequestMessage request,
CancellationToken cancellationToken)
{
if (request == null)
{
throw new ArgumentNullException("request");
}
request.Headers.Add("Ocp-Apim-Subscription-Key", this.apiKey);
return base.ProcessHttpRequestAsync(request, cancellationToken);
}
}
12. Upload the code to the Azure Function using Visual Studio.
13. The Azure Function must be subscribed to the SharePoint List. There are several ways to do this, but probably the easiest is using the PowerShell SharePoint PnP cmdlets. See further instructions in the Microsoft documentation here.
Run the example
14. In the SharePoint List, create a new item. The Function will be triggered, calling the Azure service with the text of the Comments field, getting the sentiment score and setting it back in the List item:
As you can see, the sentiment analyzer of the Azure Cognitive Services can be implemented with just a couple of code lines. It could be used in several ways not only to increase the value of the information in SharePoint, but it has the potential to be used in many ways inside other communication hubs in Office 365.
