Connect to SharePoint Online Using PowerShell

In this example, I’m connecting to a Site Collection on my tenant.

Assumptions:
1) You have created a token in your o365 site
1.1) https://portal.office.com/account/
1.2) On the left site of the page click Security & privacy, then click Create and manage app passwords
1.3) In the app password page click the create button and give it a name.
1.4) Save the password to a secure location.
1.5) There is a better way of doing this that I will cover in a future post.
2) You have downloaded to CSOM DLL(s) from Nuget

Clear-Host

$userName = "ihayse@sharepointed.com"
$pw = "abc123taco"
$siteCollectionUrl = "https://sharepointed.sharepoint.com/sites/taco"

#Secure the password
$securePassword = ConvertTo-SecureString $pw -AsPlainText -Force

Add-Type -Path "C:\Code\DLL\Microsoft.SharePoint.Client.dll"
Add-Type –Path "C:\Code\DLL\Microsoft.SharePoint.Client.Runtime.dll"

#Create a Context
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteCollectionUrl)

#Authorise
$ctx.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($userName, $securePassword)

$web = $ctx.Web
$properties = $web.AllProperties
$ctx.Load($web)
$ctx.Load($properties)
$ctx.ExecuteQuery()

Write-Host " Site Collectione URL: $($web.Url)"
Write-Host "Properties are "

foreach ($prop in $properties) {
    $prop.FieldValues
}

SharePoint CSOM Upload File Access Denied Error

Simple process, using a .Net Web App, create a local file and upload it to SharePoint. I thought the service account I was using had ampel permissions to the Site, but for it didnt… For testing, I granted the seveice account Full Control at the Web App level (user policy), site collection admin, and more. Nothing worked.

Sample of the code I was using:

            ClientContext ctxH = new ClientContext(hURL);
            Web siteH = ctxH.Web;
            ctxH.Load(siteH);
            ctxH.ExecuteQuery();

            List _library = siteH.Lists.GetByTitle("Drop Off Library");

            Folder _oFolder = siteH.GetFolderByServerRelativeUrl(siteH.ServerRelativeUrl.TrimEnd('/') + "/" + "DropOffLibrary");
            ctxH.Load(_oFolder);
            ctxH.ExecuteQuery();

            FileStream fileStream = System.IO.File.OpenRead(fileName);
            FileCreationInformation fileInfo = new FileCreationInformation();
            fileInfo.ContentStream = fileStream;
            fileInfo.Overwrite = true;
            fileInfo.Url = destFileName;

            Microsoft.SharePoint.Client.File _oFile = _oFolder.Files.Add(fileInfo);
            ctxHub.Load(_oFile);
            ctxHub.ExecuteQuery();

Quick and simple fix:
Grant your account / service account Design permissions to the Site/Web where you are uploading files.

Error at the web level:
C:\Windows\TEMP\tmp28E2.tmp: Access denied. You do not have permission to perform this action or access this resource.

ULS errors:
Permission check failed. Asking for 0x00040002, have 0x1B00C0310EF
Access denied.
System.UnauthorizedAccessException: Access denied., StackTrace:
Exception : System.UnauthorizedAccessException: Access denied.
Exception occured in scope Microsoft.SharePoint.SPFileCollection.Add. Exception=System.UnauthorizedAccessException: Access denied.
Original error: System.UnauthorizedAccessException: Access denied.
SocialRESTExceptionProcessingHandler.DoServerExceptionProcessing – SharePoint Server Exception [System.UnauthorizedAccessException: Access denied.
Throw UnauthorizedAccessException instead of SPUtilityInternal.Send401 for client.svc request.

Download all files from a document library using client object model

For this to work, you will need to obtain a copy of the SharePoint Client DLL (microsoft.sharepoint.client.dll).

On a server with SharePoint intalled:
SharePoint 2010: 14 hive
SharePoint 2013: 15 hive
SharePoint 2016: 16 hive
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\

In Visual Studio, create a console application and name it Download_All_Files. Copy all the code below and paste it into your code window. Update site URLs and list name (Taco Storage).

Using this process, I was able to download 10,000 documents from a library. Took around 30 minutes to complete.

using System;
using System.IO;
using Microsoft.SharePoint.Client;

namespace Download_All_Files
{
    class Program
    {
        static void Main(string[] args)
        {
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            // CHANGE THIS to your library name, get this from the Library Settings page
            List list = web.Lists.GetByTitle("Taco Storage");
            site.Load(list);
            site.ExecuteQuery();
            site.Load(list.RootFolder);
            site.ExecuteQuery();
            site.Load(list.RootFolder.Folders);
            site.ExecuteQuery();
            processFolderClientobj(list.RootFolder.ServerRelativeUrl);
            foreach (Folder folder in list.RootFolder.Folders)
            {
                processFolderClientobj(folder.ServerRelativeUrl);
            }
        }
        public static void processFolderClientobj(string folderURL)
        {
            // folder on your computer where all the files will be downloaded to
            string Destination = @"C:\\yourFolder";
            
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            Folder folder = web.GetFolderByServerRelativeUrl(folderURL);
            site.Load(folder);
            site.ExecuteQuery();
            site.Load(folder.Files);
            site.ExecuteQuery();
            foreach (Microsoft.SharePoint.Client.File file in folder.Files)
            {
                string destinationfolder = Destination + "/" + folder.ServerRelativeUrl;
                Stream fs = Microsoft.SharePoint.Client.File.OpenBinaryDirect(site, file.ServerRelativeUrl).Stream;
                byte[] binary = ReadFully(fs);
                if (!Directory.Exists(destinationfolder))
                {
                    Directory.CreateDirectory(destinationfolder);
                }
                FileStream stream = new FileStream(destinationfolder + "/" + file.Name, FileMode.Create);
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(binary);
                writer.Close();
            }
        }
        public static byte[] ReadFully(Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }
    }
}