Property promotion and Programatically Copying documents with only required fields in SharePoint 2013.

Recently I was working on a requirement to copy documents between two site collections where the source and target document library had the same content type applied to them. The approach I took was to
- Get the source document library item
- Get the target folder where the document was to be copied
- Use the SPFolder.Files.Add() to copy the document.

Fairly simple ! The only thing I had to take care of was to copy only a specified set of field values (not all fields from content type) to the target file. (This was a part of the requirement !)

There is an overload of the ADD method that allows us to pass a Hashtable which will contain the fields that we want to copy over to the target file. So for example if I want to copy "Document Id" field of a document, I'll pass the property like this

Hashtable itemProperties = new Hashtable();
itemProperties.Add ("Document Id", "<VALUE>");

Now, though I passed only the required properties through a hashtable, all the field values of the content type were being copied. After a bit of head-scratching, I figured out that this scenario was not reproducible for a ".TXT" file but only for Microsoft Office files. This was because of SharePoint property promotion. What this basically does is when a document is added to a document library, the values of the fields in the list item for the document are added to the metadata properties of the document file and since the stream of the file was used to copy the document, the properties were getting transferred with it.

The property "ParserEnabled" of Web object allows SharePoint to know whether to retain all the properties when a file is copied/downloaded-uploaded.

https://msdn.microsoft.com/EN-US/library/microsoft.sharepoint.spweb.parserenabled.aspx

So to copy only the required properties, you can disable ParserEnabled property of the web, copy the document and then set it back again







But you would not want to set the ParserEnabled property to false permanently so of course you would set it back to true in the FINALLY block of your code :)

Hope this helps !

get_label() or .Label : Retrieve taxonomy column value using JSOM

This is going to be a short post about something I observed recently while working with Taxonomy field and JavaScript object model. While retrieving value of the Taxonomy field of a list item, we generally use the "get_label()" method.

This is the case if you have already referenced SP.Taxonomy.js in your code before fetching the list item. We generally do something like this

It is not necessary to have SP.Taxonomy.js referenced in your code in order to retrieve the Taxonomy field value. If you do not reference SP.Taxonomy.js, you need to use the ".Label" property which is returned with the list item to retrieve the label of the term.

So the next time you are tired of scratching your head trying to figure out why you are not able to retrieve the label using get_label(), check if you have referenced Taxonomy JS and use the appropriate method.

Cheers !

Export a term set to 'Importable' CSV using JavaScript in SharePoint online

It is a well known fact that we can import term set in SharePoint using a CSV file. This is particularly helpful when you are working in multiple on-premise environments/multi-tenant environment and you want to replicate a deep and complex term hierarchy everywhere. This can be very tedious if done manually. Also, you may not always have the liberty to use server side code with SharePoint online to export the term set. Here, we'll explore a way to export a term set in CSV file using JavaScript only.

Note, the implementation does not work in IE (I have tested this in IE 9) because of the way we export data to CSV from client-side. But it works very well in Chrome. I did not have much time to explore the reason for this but I plan to do so in the near future :)

The format of CSV which is acceptable during import is provided to us as a sample file in SharePoint. To see the format, navigate to Term store management and click on the Taxonomy node.

The columns in the CSV are in the following order

"Term Set Name", "Term Set Description", "LCID", "Available for Tagging", "Term Description", "Level 1 Term", "Level 2 Term"......"Level 'n'  Term".

You can see from the test file to write each term, a proper hierarchy has to be maintained.

For testing purpose, I have created two term groups, one to Export a term set and one to import from CSV. The term set contains 'Regions'

get_pathOfTerm() : 

Before going into the code, I want to mention this method which has made writing the hierarchy of term set to CSV so easy !! When we use the "getAllTerms()" method on a term set, the terms are returned without its hierarchy details. So if we have to write the terms according to the hierarchy, we might have to take the longer route and iterate through each term and its children. This is where "get_pathOfTerm()" comes to our rescue. This method returns the hierarchy of a particular term, with each level separated by a semi-colon ';'. What this means is, referring to the above screenshot (term set hierarchy) if I get the path of the term "New York", the value returned is

North America;USA;New York

Now, we anyway need the values as comma separated since we are writing it to a CSV. Just replace the semi-colon with comma and you have the string for the entire hierarchy of a term you want to write.

I have developed a simple SharePoint hosted app to demo this. We will provide the GUID of a term set to our app which then exports that particular term set. To get GUID of a term set, click on the term set that you want to export in Term Management and in the right pane you'll find the unique identifier at the bottom.

Import this into a term set.

Here is the full code of the functionality



I found the method to export data to CSV using JavaScript on Raymond Camden's blog.

Hope this helps !

Index web property bag using JavaScript object model-AngularJS in SharePoint online

SharePoint 2013 added a new capability to index the property bags. Indexing property bag through the server side object model code is pretty easy. The WEB object has a property called "IndexedPropertyKey" which is a collection of all the web properties that should be indexed. Here is a reference article . However, doing this through client object model is tricky because we do not have access to IndexedPropertyKey through CSOM. Vesa Juvonen has made our life easy by documenting a work-around to do this through CSOM.

Basically the way this works is - all the properties that need to be indexed are encoded and stored in a "vti_indexedpropertykeys" property. In a scenario where multiple properties need to be indexed, the encoded values are separated by a PIPE "|" character. In this post we are going to index the web properties using JavaScript object model on a SharePoint online site. We will develop a SharePoint hosted app to add and index a property. Here is how our application will look like.

Lets go ahead and add a couple of new properties to be indexed.

The checkbox facilitates the user to choose whether the new property should be indexed or not. As we have chosen to index the properties , the vti_indexedpropertykeys is created in the web properties with the encoded values of property keys separated by PIPE character

Once the incremental crawl completes, the web properties that we just added are indexed.

Lets go through the code


The AngularJS code has a controller "SPWebPropertiesController" and an AngularJS service "WebPropertiesService" which contains function to fetch and add new web properties. The function $scope.GetWebProperties uses AngularJS defer and on success, it populates the keys and values of the properties. It also checks whether the key "vti_indexedpropertykeys" exists and sets the flag $scope.vtiIndexedPropertyKeysExists accordingly.

All the AngularJS and Javascript code below is a part of single JavaScript file. It is separated here in this post and GIT for the ease of understanding.

  

FetchWebProperties function:

This function gets all the properties from the property bag and returns it to the controller where the key and values are populated in $scope.WebProperties=[];

AddNewProperty function: 

- This function adds a new property to the property bag using the set_item(Property_Key, Property_Value) function. If you want to modify the value of a property, use "webProps.set_item(Existing_Property_Key, New_Value)".
- If we have chosen to index the property using the checkbox, we must get the encoded value of the property key. This is done using the EncodePropertyKey function.
- Checks whether the key "vti_indexedpropertykeys" exists, using the flag that we have set in GetWebProperties function of the controller. If it exists, it adds the encoded property key.
- If the "vti_indexedpropertykeys" is not present, it creates this new key and then adds the encoded property key that has to be indexed.

EncodePropertyKey function : 

This function converts the property key to an encoded string. Initially I observed the bytes array in C# console application and Javascript conversion and noticed that the '0's from JS conversion were missing. So I added those (line 5 in the below function)

WebPropertiesService code :

SharePoint online - Scenario to Re-index a site or list

Recently I was working with Search on a custom list in SharePoint online where I observed that not all the list items that fulfill a criteria are being returned in search results. Initially, I had created a custom list named "News" with the following fields

- Title
- Teaser
- Description.

I created 2 list items which were picked up by the continuous crawl in SP online and were being fetched properly in search results. Then I realized that I do not have a field that indicates when this news should be published to the user. So I added a new column "Publish Date" to the list, edited the 2 existing items to assign the Publish date and also created 4 new items.

In order to use the publish date field in search, I mapped it to an existing managed property and tried to query these items on a specific criteria for the PUBLISH DATE field using content search webpart. Though all the 6 items were satisfying the criteria, only the 4 NEW items created were being returned by the search. I gave it some time hoping that the SP online full crawl might index the two items, but it did not !

I then remembered the new feature in SharePoint 2013 which allows us to Re-Index a list, library or a site. Thought I would give it a shot. To re-index a list, go to the List Settings -> Advanced Settings  there is an option to re-index the list.

This will fully re-index the entire list. I also came across this article later which explains that re-index is required after changing the managed properties.

MIME type mismatch issue in SharePoint hosted app

Recently while working on a SharePoint hosted app, I encountered an issue where the App page was not loading correctly. The SharePoint hosted app was hosted on a site collection under the managed path "/sites" as in the following:

http://host-header.com/sites/my_site_coll

The reason I mention the site collection URL is because the issue is specifically related to it.This SP hosted app was a custom ribbon action made available on custom lists and on click of this action, the app page was being displayed in a dialog box. I observed that the UI of the app page was broken and it was apparent that some JavaScript/CSS references were missing.

Error displayed in the IE developer toolbar is shown below.

The SharePoint JavaScript files SP.runtime.JS and init.js were failing to load with the error "MIME 
type mismatch". Using the "Network" tab in the IE developer toolbar, I captured the page load network trace.

Shoot up the IE developer toolbar. Select the Network tab, click Start capturing and Trigger the app from ribbon. Then, let the page load and click "Stop capturing". 

Observe the type of sp.runtime.js. It was returned as "text/html".  My first guess was to check the MIME type values in the IIS website of the SharePoint web application. However, the type configured for JS files was "application/javascript". Also, the type returned for my custom JS files was "application/javascript", so only the JS files from "_layouts" failing to load properly!

The same app was working perfectly in another web application! Notice the URL from where the JS is loaded:

http://apprefix-ID.domain.com/_layouts/15/sp.runtime.js

I tried to hit this URL in the browser. For the web application where the app was working, I got a prompt to save the file. For the web application where the app was not working, I did not get any response. So the path from where the JS was being loaded was incorrect or did not exist. !!

The issue was that the web application, where app was not working, did not have a site collection at the "/"  managed path. Creating this root site collection at the "/" path resolved the issue.

Hope this helps.

SharePoint 2013 Work Management Service with Client API

The Work Management Service application was one of the new service applications introduced in SharePoint 2013. This service provides functionality to aggregate tasks from several locations into a central location in SharePoint. With the Work Management Service, we can aggregate tasks from SharePoint sites, an Exchange Server and Project Server into the user's My Site so that the user can see all the tasks assigned in one place. It also provides an API for developers to write custom components.

Consider you are working on an Intranet portal site. This service can be very helpful to write a component that sits on the home page of the portal site and displays all the tasks assigned to the logged in user.

Note:
We cannot use the Work Management Service API from a SharePoint APPS (at least now, as I am writing this article). I tried to use it in a provider hosted app but got an error that specified the Work Management Service assembly does not support app authentication.

Let us start by setting up all the required things for the Work Management Service.

From Central Administration select Application Management -> Manage Service Application then create a new "Work Management Service Application".

Make sure that the "Work Management Service" is started.

I have created two site collections with Test data.

The following are task lists in the two sites respectively.

Now navigate to your Profile page (Person.aspx) by clicking "OneDrive" of your SharePoint site and you will see a "Tasks" link in the left navigation.

Alternatively, navigate to the following URL: "/personal/USERID/AllTasks.aspx". In the tasks list, you'll see the combined list of tasks from both the SharePoint sites we created for testing.

Working with the API

We will the use of the client API of the Work Management Service to retrieve information related to our tasks. In work management terminology, "LOCATIONS" are the places where the tasks are stored. Let us try to retrieve the locations using the client-side object model. For demo purposes, I am writing the code in a console application.

Make sure you have all the DLL references in place.

The WorkManagement client DLL can be found in the "15/ISAPI" folder.Then we create a ClientContext and an instance of UserSettingsManager to retrieve the locations information.

Output

Retrieve single-select Managed Metadata field Using REST API

A while ago I came across this very interesting piece of information of properly retrieving a single-select managed metadata column value.

The following is the reference of the article:Taxonomy Columns & REST API

Let us first try to fetch the value of a managed metadata column that allows adding multiple values to it. I have created a simple taxonomy term set called "Countries".

A custom list that contains a column "Country" that is a Managed Metadata column and allows multiple values to be added

The list contains the following values for testing:

Now we try to fetch the list items from this list using the REST API from the provider-hosted app. To use the REST API from the provider hosted app, we need to send the access token in the REST request. This will be used by SharePoint to authorize the call.

We have written the JSON response on our page and we can see that the label of the taxonomy field is retrieved properly.

Now If I change the managed metadata setting to single-select, we see the following difference:

 To retrieve this value properly, we need to make use of the GetItems method and SPQuery in the REST request.

And the value will be properly retrieved.

References:

• Use CAML queries with REST API
• Taxonomy Columns and REST API

SharePoint 2013 High trust provider hosted app : Upload large files

Consider a situation where you want to allow a user to upload a large document in your SharePoint application, however the user does not have contribute permission to the Document Library. This can be for several reasons. You don't want the user to see/modify other content and only provide an interface to submit documents. 

We will use the app-only policy of provider hosted apps to upload the documents to the HOST web. This article assumes you understand the basics of the procedure to setup the environment and develop Provider-Hosted apps. If not, here is a technet article to follow.

I have created a provider hosted app with a client web part. The client webpart will be added to a page in the host web that will show a File upload control. The page that the client webpart shows will be deployed to the remote web of the provider hosted app. In this case, it is FileUpload.aspx.

Now, we need to tell the app to use the app-only policy when calling into SharePoint. With the app-only policy, irrespective of what level of permission the user has on the SharePoint site, only the app's permissions will be considered for authorization of the call. So, we set the value in the AppManifest.xml file of the provider hosted app. 

Also, since we are uploading a file in Document Library, the app needs to have the correct permissions on the SharePoint site. This is also set in the AppManifest.xml file.

 

In the FileUpload.aspx page, I have designed a simple UI with a file upload control and a button to upload the document. We write the code to upload the file to a SharePoint Document Library on the event handler of the button click event.

We will use the SharePointContextProvider to get the SharePoint context and theCreateAppOnlyClientContextForSPHost since we are using the app-only policy and uploading the file to the Host web. 

 

Using the "ContentStream" property, we can upload files of large size in the SP Document Library. Refer to:

https://msdn.microsoft.com/en-us/library/office/dn904536.aspx 

Now I have packaged and deployed my remote web in IIS and uploaded the app to my app catalog. I have also added the app to my SharePoint site. Edit the page and insert the client web part to a page in the Host Web.



And the client webpart shows the upload control and button. Please excuse the alignment of controls. :) 



Try to upload a large document (say about 80 MB, which I tried), and you may run into the below error

"The request filtering module is configured to deny a request that exceeds the request content length" 

This is because the IIS is configured to handle requests of only a specific size and since we are uploading a file of about 80 MB, it fails. We can change that using the web.config file. In the web.config of our remote web application, set the following values:

There is one more setting required. Open IIS then navigate to your remote web site. Click on "Request Filtering" . Click on "Edit Feature Settings" and change the value of "Maximum allowed content length".

Now we are all set. Select a document in the file upload control and hit Upload. Our file is uploaded and is modified by the APP because we are using the App-only policy.