PowerShell and CSOM Bulk Delete List Items

From my desktop, I wanted to delete all the items in a list. The list had 10,000+ item and loop through the items one at a time is slow and not efficient. This is where batch process comes in and will help to quickly delete all the items in a large list.

In this example, I commented out the SharePoint Online Credentials because I’m working with on on-prem environment.
The batch size is set to 1,000

#Load SharePoint CSOM Assemblies (you will need to grab these from a SharePoint server)
Add-Type -Path "C:\SharePoint DLL\Microsoft.SharePoint.Client.dll"
Add-Type -Path "C:\SharePoint DLL\Microsoft.SharePoint.Client.Runtime.dll"

$SiteURL = "https://mylocalsite/sites/taco/"
$ListName = "GiantList"
$BatchSize = "1000"

Try {
#$Cred= Get-Credential
#$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Cred.Username, $Cred.Password)

#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
#$Ctx.Credentials = $Credentials

#Get the List
$List = $Ctx.Web.Lists.GetByTitle($ListName)

#Define Query to get List Items in batch
$Query = New-Object Microsoft.SharePoint.Client.CamlQuery
$Query.ViewXml = @"



#Get List Items in Batch
$ListItems = $List.GetItems($Query)

if ($ListItems.Count -gt 0)
for ($i = $ListItems.Count-1; $i -ge 0; $i--)
Write-Host "." -foregroundcolor black -backgroundcolor yellow
$continue = $false;

$Query.ListItemCollectionPosition = $ListItems.ListItemCollectionPosition
While($Query.ListItemCollectionPosition -ne $null)

Catch {
write-host -f Red "Error Adding Items to List!" $_.Exception.Message

Limiting the number of rows returned in each query will help to avoid this error:

Exception calling "ExecuteQuery" with "0" argument(s): "The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator."

Some of the scripts was borrowed from:

I plan to revisit this and batch processing where you combine all the delete statements into a large XML string and pass it to SharePoint for processing.

Copy or Move Folder Structure in SharePoint Using CSOM

This post is intended for SharePoint Online and 2016+ using CSOM.

In previous version of SharePoint, moving a folder structure to a different library or a different library in another site collection required a good amount of code and effort. With the introduction of Microsoft.SharePoint.Client.MoveCopyUtil we now have the ability to use methods like MoveFolder to perform a cut and paste of a folder structure. There are other methods that are worth exploring like FolderCopy that is a copy and paste of a structure.

More methods can be found here: MoveCopyUtil members

Name Description
Public method Static member CopyFile
Public method Static member CopyFileByPath
Public method Static member CopyFolder
Public method Static member CopyFolderByPath
Public method Static member MoveFile
Public method Static member MoveFileByPath
Public method Static member MoveFolder
Public method Static member MoveFolderByPath

Example of moving a folder between site collections.

string dstUrl = "https://sharepoint.com/sites/B/";

using (ClientContext srcContext = new ClientContext(dstUrl))
  string sourceFolder = "https://sharepoint.com/sites/A/libraryname/foldername";
  string destFolder = dstUrl + "libraryname/foldername";

  MoveCopyUtil.MoveFolder(srcContext, sourceFolder, destFolder);

Again, this is using CSOM with the latest version of the NuGet package (16.0.4351.1000).

Move OneNote Notebooks to New SharePoint Library or Server

My company recently completed a SharePoint 2010 to 2016 migration. With the migration came the use HTTPS security, so my OneNote notebooks stored in SharePoint would no longer sync. All of my notebooks displayed an error of Not syncing.

Here is how I fixed the issue:
First, make sure all of your notebooks are in SharePoint.
In OneNote, click on the File tab.
Locate the first notebook you want to update.
Next to the notebook name click Settings, then Properties.
In the Properties window click on Change Location…
Copy the URL of your SharePoint document library.
Paste the URL into the OneNote Chose a sync location… window.
Select the folder you want to sync OneNote with.
Click OK, a message box should appear saying the item is syncing.
Give it a minute and you should be set.

SharePoint SPFx PNP gulp trust-dev-cert Error

Trying to setup my dev environment to play around with SPFx and ran into this error:

[08:33:19] Error - [trust-cert] Error: root "Trusted Root Certification Authorities"
Signature matches Public Key
CertUtil: -addstore command FAILED: 0x80070005 (WIN32: 5 ERROR_ACCESS_DENIED)
CertUtil: Access is denied.

[08:33:19] Error – [trust-cert] Certificate trust failed with an unknown error.
[08:33:19] Error – ‘trust-cert’ sub task errored after 5.61 s
“Error trusting development certificate.”
[08:33:19] ‘trust-dev-cert’ errored after 5.62 s
[08:33:19] ==================[ Finished ]==================
Error – [trust-cert] Error: root “Trusted Root Certification Authorities”
Signature matches Public Key
CertUtil: -addstore command FAILED: 0x80070005 (WIN32: 5 ERROR_ACCESS_DENIED)
CertUtil: Access is denied.

Error – [trust-cert] Certificate trust failed with an unknown error.
Error – ‘trust-cert’ sub task errored after 5.61 s
“Error trusting development certificate.”

The only way I could manage to get around it was to manually create a cert using the OOB Windows tools.

Creating your .cer file

  1. Start by locating makecert.exe. You can typically find it in and C:\Program Files (x86)\Windows Kits\8.1\bin\x64 on Windows Server 2012 R2.
  2. Browse to that location with Command Prompt with Administrative privileges.
  3. You can actually create a certificate to run against your machine name and you can also have it work against localhost. To create a certificate for your machine name, use the following command:

makecert -n “CN=%ComputerName%, CN=localhost” -ss MY -sr LocalMachine -b 08/09/2016 -e 09/09/2028 -a sha256 -sky exchange -r -pe myCert.cer

Return to where you tried to run trust-dev-cert and move to the next step.  We now have a dev cert so there is no use in creating one using gulp.

Thanks to Santi Murtagh for the instructions on setting up the cert.

Creating a Self-Signed SSL Certificate for Developing and Testing Against IIS

The Web application at X could not be found.

Error: The Web application at https://sharepoint.sharepointed.com could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application.

I created a .net console app to update some stuff in SharePoint.  When executing the .exe file with a new service account, I received the above error.

First I tried granting Shell access to the content db that I was working with but that didn’t solve the problem.

$cDb = Get-SPContentDatabase -site "https://taco.sharepointed.com/"
Add-SPShellAdmin -UserName "domain\userAccount -database $cDb

Running the same command without the database switch fixed my problem.

Add-SPShellAdmin -UserName "domain\userAccount"

SharePoint CSOM Upload File Access Denied Error

Simple process, using a .Net Web App, create a local file and upload it to SharePoint. I thought the service account I was using had ampel permissions to the Site, but for it didnt… For testing, I granted the seveice account Full Control at the Web App level (user policy), site collection admin, and more. Nothing worked.

Sample of the code I was using:

            ClientContext ctxH = new ClientContext(hURL);
            Web siteH = ctxH.Web;

            List _library = siteH.Lists.GetByTitle("Drop Off Library");

            Folder _oFolder = siteH.GetFolderByServerRelativeUrl(siteH.ServerRelativeUrl.TrimEnd('/') + "/" + "DropOffLibrary");

            FileStream fileStream = System.IO.File.OpenRead(fileName);
            FileCreationInformation fileInfo = new FileCreationInformation();
            fileInfo.ContentStream = fileStream;
            fileInfo.Overwrite = true;
            fileInfo.Url = destFileName;

            Microsoft.SharePoint.Client.File _oFile = _oFolder.Files.Add(fileInfo);

Quick and simple fix:
Grant your account / service account Design permissions to the Site/Web where you are uploading files.

Error at the web level:
C:\Windows\TEMP\tmp28E2.tmp: Access denied. You do not have permission to perform this action or access this resource.

ULS errors:
Permission check failed. Asking for 0x00040002, have 0x1B00C0310EF
Access denied.
System.UnauthorizedAccessException: Access denied., StackTrace:
Exception : System.UnauthorizedAccessException: Access denied.
Exception occured in scope Microsoft.SharePoint.SPFileCollection.Add. Exception=System.UnauthorizedAccessException: Access denied.
Original error: System.UnauthorizedAccessException: Access denied.
SocialRESTExceptionProcessingHandler.DoServerExceptionProcessing – SharePoint Server Exception [System.UnauthorizedAccessException: Access denied.
Throw UnauthorizedAccessException instead of SPUtilityInternal.Send401 for client.svc request.

Use PowerShell to Execute SharePoint Search Queries

In this example, I’m narrowing my search to one library and a search term.

At a high level, the script is searching the FoodSite for the word GoodTaco.


function Query-SPSearch {
		[Parameter()][Int32]$Count = 10

	$QueryXml = @"

<QueryPacket xmlns="urn:Microsoft.Search.Query" >
            <QueryText type="STRING">$KeywordQuery</QueryText>
	$ServicePath = "/_vti_bin/search.asmx"
	$SearchWS = New-WebServiceProxy -Uri ($WebApplicationPath + $ServicePath) -UseDefaultCredential
	$Results = $SearchWS.QueryEx( $QueryXml )
	# we excluded all other result sets, but just in case get the one we want:

Query-SPSearch -WebApplicationPath "https://sharepointed.com/sites/foodsite" -KeywordQuery "GoodTaco AND path:https://sharepointed.com/sites/foodsite/tacos" -Count 20 | Format-Table Title, Author, Path

Create a console app to query userprofileservice.asmx

In Visual Studio, create a console application.
Create a web reference using the following inputs:
URL: https://yourSite.url/_vti_bin/userprofileservice.asmx?WSDL
Name: UPS1

In the program.cs class, replace Main method with the following:

static void Main(string[] args)
UPS1.UserProfileService ups1 = new UPS1.UserProfileService();

ups1.UseDefaultCredentials = true;

UPS1.PropertyData[] info = ups1.GetUserProfileByName(@"domain\username");

for (int i = 0; i < info.Length; i++) { Console.WriteLine(info[i].Name); for (int j = 0; j < info[i].Values.Length; j++) { Console.WriteLine(info[i].Values[j].Value); } } Console.ReadLine(); }

Run the app.

Searching SharePoint Using PowerShell

In this example, I needed to search a farm for every site under a managed path. BUT, the sites I’m searching for were built using a 3rd part tool and would not correctly appear in the search results.  The problem was related to having Trim Duplicates enabled by default.  Easy fix… Set your search property trim duplicates = false.

$site = Get-SPSite "https://sharepointed.com"

$keywordQuery = New-Object Microsoft.Office.Server.Search.Query.KeywordQuery($site)
$queryText = "ContentClass:STS_Site AND Path:https://sharepointed.com/TACOS/*"
$keywordQuery.QueryText = $queryText
$keywordQuery.TrimDuplicates = $false
$searchExec = New-Object Microsoft.Office.Server.Search.Query.SearchExecutor
$searchResults = $searchExec.ExecuteQuery($keywordQuery)

Write-Host "'r'n"
$table = $searchResults.Table
Write-Host $table.Length" Results Found" -BackgroundColor "Green" -ForegroundColor "Black"
$table | select Title, Path, IsDocument

The search results will display all sites that have Taco as its managed path. If you are not retrieving the results you expect, try switching TrimDuplicates = $false .

Remove Upload Button and Drag files here to upload text

There might be a better way to get this done, but for now, this works for me. Keep in mind, this update works on a per view basis. If I find a way to correctly update the masterpage, I will update this post.

At a high level, you will need to modify the page, add two content editor web parts, and save.

Normal page displaying the Upload button and Drag files here to upload text.

Start by editing the page.  Top right of the screen, click the gear icon, then select Edit Page.  Once in edit make, add two Content Editor web parts to the page.

Place your cursor in the top web part, select Edit Source in the Format Text ribbon window.

In the Source window, enter the following text: link to script

Update the second web part inserting the following text: link to script

Once the web parts have been updated, click on Stop Editing.

All done.

You can also upload the attached web parts to your page:  zip file of web parts