Uploading Files to SharePoint 2016 Using ListData.svc

Ran into a small issue when testing some code for a SharePoint 2010 to SharePoint 2016 migration.

With SharePoint 2010, the following code sample would work to upload files to a library.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            } 

When trying to use this same code with SharePoint 2016, I was receiving the following Errors:
Output error: An error occurred while processing this request.

Errors from Fiddler:
Auth:
No Proxy-Authenticate Header is present.

No WWW-Authenticate Header is present.

Caching:

Under RFC2616, HTTP/500 responses will not be cached regardless of what caching headers may be present. HTTP/1.1 Cache-Control Header is present: no-cache

This response does not specify explicit HTTP Cache Lifetime information and does not specify a Last-Modified date. Heuristic expiration is typically based on Last-Modified date. Lacking Last-Modified, this response may be revalidated on every use or once per browsing session, depending on the browser configuration.

This response contains neither an ETAG nor a Last-Modified time. This will prevent a Conditional Revalidation of this response.

FIX
For SharePoint 2016, the upload process required that the Path property be populated.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    Path = path,
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            } 

Make Your PowerShell Script Environment Aware

In place of hard coding URLs for each environment, I decided to make a single script that is environment aware. Why? Cuts down on the number of scripts that have to be supported for a single development cycle. To make this more dynamic, you could move this to a function script and reference it from all your scripts.

if ((Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null)
{
	Add-PsSnapin Microsoft.SharePoint.PowerShell
}

#get config database server
$ConfigDB = Get-SPDatabase | where-Object{$_.Type -eq "Configuration Database"}
$serverName = $ConfigDB.Server.Displayname

#replace this with the web app you want to target.  taco, burrito, nacho...
$webApp = "taco"

#set variable equal to the environment url
$siteURL = switch ($serverName.ToLower())
{
	"dev_db" {"http://$webApp.sharepointed.com/"}
	"test_db" {"http://test$webApp.sharepointed.fcbt/"}
	"build_db" {"http://build$webApp.sharepointed.com/"}
	"prod_db" {"http://$webApp.sharepointed.com/"}
}

Use PowerShell to get all your SQL databases and their size

Quick script to get all the databases on a server, then output to a CSV file.

In this example, all I needed was the database name and its size.

[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | out-null
$s = New-Object ('Microsoft.SqlServer.Management.Smo.Server') "YourSQLServer"
$dbs=$s.Databases
$dbs | SELECT Name, Size | Export-Csv c:\test.txt

Other properties:

$dbs | SELECT Name, Collation, CompatibilityLevel, AutoShrink, RecoveryModel, Size, SpaceAvailable

Credit for the base script:
Edwin M Sarmiento

Update Password in Remote Desktop Manager RDCMAN

I can never manage to remember how to do this. Simple enough, how do you update your login credentials in Remote Desktop Manager ( RDCMAN )?

Right click on the root node and select Properties.

Select the Security Settings tab, then click on Source: default settings group.
A new window will open, in it, select the Profile Management tab.
Select the account you want to update, click the Edit button on right, then click Ok on the bottom.

All done.

Server was unable to process request Access is denied. Connecting to SharePoint

Errors:
Server was unable to process request. —> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Object moved
Object moved to

In your project, navigate to the Web Reference folder.
One at a time, open each of your reference folders.
Edit the Reference.cs file (notepad++ or Visual Studio).
Directly after this line: public partial class Lists : System.Web.Services.Protocols.SoapHttpClientProtocol {
Insert the following code:

protected override System.Net.WebRequest GetWebRequest(Uri uri)
{
System.Net.WebRequest wr = base.GetWebRequest(uri);
wr.Headers.Add(“X-FORMS_BASED_AUTH_ACCEPTED”, “f”);
return wr;
}

Save the file.
Try connecting to SharePoint again.
If it works, complete the same update on your other Reference.cs files.

Hyper-V Client Window is Black

New Dell XPS 15
Enabled Hyper-V and tried to create a Server 2012 VM.
Client window was black but the preview window was displaying the correct view of the client OS.
The other fixes that appear on Google did not apply because I was in the process of setting up a new VM.

Fix:
With Hyper-V running and the client window open.
Open the NVIDIA control panel
Click on Manage 3D settings
1. select a program to customize
Select c:\windows\system32\vmconnect.exe
IF you are not able to select this, click the Add button and locate it in the displayed list of programs.
2. Select the preferred…..
Select Integrated graphics
Click Apply

Now, close the Hyper-V client window and reopen it.

Use PowerShell to add Holidays to Outlook Calendar

Geeking around with PowerShell today trying to add all company holidays to my Outlook calendar. In the script, I’m creating all day appointments and setting the Show As to out of office. Simple enough!

function get-mailfolders { 
	$outlookfolders = @() 
	$outlook = New-Object -ComObject Outlook.Application 
	foreach ($folder in $outlook.Session.Folders){ 

		foreach($mailfolder in $folder.Folders ) { 
			$olkf = New-Object PSObject -Property @{ 
				Path = $($mailfolder.FullFolderPath) 
				EntryID = $($mailfolder.EntryID) 
				StoreID = $($mailfolder.StoreID) 
			} 

			$outlookfolders += $olkf 
		} 
	} 
	$outlookfolders 
}

$outlook = new-object -com Outlook.Application
$folder = get-mailfolders | where {$_.Path -like "*calendar*" -and $_.Path -and $_.Path -like "*$mailbox*"} 
$calendar = $outlook.Session.GetFolderFromID($folder.EntryID, $folder.StoreID) 

$holidays = @{"01/01/2017"="New Year’s Day"; "01/16/2017"="Martin Luther King Day"; "02/02/2017"="Presidents Day"; "05/29/2017"="Memorial Day"; `
	"07/04/2017"="Independence Day"; "11/4/2017"="Labor Day"; "11/23/2017"="Thanksgiving Break"; "11/24/2017"="Thanksgiving Break"; "12/25/2017"="Christmas Day"}

foreach($holiday in $holidays.GetEnumerator() | Sort Key)
{
	[string]$hName = $holiday.Value
	$hDate = Get-Date $holiday.Key
	
	$appt = $calendar.Items.Add(1) 
	$appt.Start = $holiday.key.ToString()
	$appt.AllDayEvent = $true
	$appt.Subject = $hName
	$appt.Body = $hName

	<#
	Show As / Status
	0 = Free 
	1 = Tentative 
	2 = Busy 
	3 = Out of Office 
	#>
	$appt.BusyStatus = 3

	$appt.Save()
}

This post helped guide me in the right direction:
Create a calendar item

Get all Groups and Users in a Site Collection or Web

Recently a user contacted me asking how to get all the groups and users in a site and subsites.

Both example will output to the C:\ drive of the server.

This script will output all the groups and user from the root of a site collection and all the subsites.
$site = Get-SPSite "http://sharepointed.com"

$userOutput = @()

foreach($subWebs in $site.AllWebs)
{
Write-Host $subWebs.Title

$groups = $subWebs.sitegroups

foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name ‘Account’ -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
}

$userOutput | export-csv c:\site_collection_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

 

This script will output all the groups and users from a single web.
$web = Get-SPWeb "http://sharepointed.com/sites/taco/SubSite"

$userOutput = @()

$groups = $web.Groups
$users = $web.Users

#get groups and users in the groups
foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
#get users not in groups
foreach($user in $users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value ""
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $user.Name

$userOutput += $object
}

$userOutput | export-csv c:\web_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

The Web server at does not appear to have Microsoft SharePoint Foundation installed

When you try to open a SharePoint site from SharePoint Designer and you receive this error:
The Web server at does not appear to have Microsoft SharePoint Foundation installed

sharepoint-not-installed

Microsoft suggests doing the follow (did not work in my case)

To resolve this problem, enable the client integration. To do this, follow these steps:
Start SharePoint 2010 Central Administration, and then click Security.
Click Specify authentication providers.
If the Web application in the Web Application list is not the same application that you want to open in SharePoint Designer 2010, click Web Application on the Change Web Application list.
On the Select Web Application page, click the name of the Web application that you want to open.
Under Zone, Click Default to open the Edit Authentication page.
Under Enable Client Integration, click Yes, and then click Save.

In my case, it turned out that my web.config file was out of wack.
Only one of my web apps would fail to load in SharePoint Designer, all the other Web Apps / sites would load.
Opened a working and the non-working sites web.configs in NotePad++ and did a file compare.
Located the difference in the file and removed it.
For good housekeeping, i then fired off an IISreset.
*things to note, make sure you have a backup copy of your web.config file and this work was done in my Develop environment.)

Download all files from a document library using client object model

For this to work, you will need to obtain a copy of the SharePoint Client DLL (microsoft.sharepoint.client.dll).

On a server with SharePoint intalled:
SharePoint 2010: 14 hive
SharePoint 2013: 15 hive
SharePoint 2016: 16 hive
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\

In Visual Studio, create a console application and name it Download_All_Files. Copy all the code below and paste it into your code window. Update site URLs and list name (Taco Storage).

Using this process, I was able to download 10,000 documents from a library. Took around 30 minutes to complete.

using System;
using System.IO;
using Microsoft.SharePoint.Client;

namespace Download_All_Files
{
    class Program
    {
        static void Main(string[] args)
        {
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            // CHANGE THIS to your library name, get this from the Library Settings page
            List list = web.Lists.GetByTitle("Taco Storage");
            site.Load(list);
            site.ExecuteQuery();
            site.Load(list.RootFolder);
            site.ExecuteQuery();
            site.Load(list.RootFolder.Folders);
            site.ExecuteQuery();
            processFolderClientobj(list.RootFolder.ServerRelativeUrl);
            foreach (Folder folder in list.RootFolder.Folders)
            {
                processFolderClientobj(folder.ServerRelativeUrl);
            }
        }
        public static void processFolderClientobj(string folderURL)
        {
            // folder on your computer where all the files will be downloaded to
            string Destination = @"C:\\yourFolder";
            
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            Folder folder = web.GetFolderByServerRelativeUrl(folderURL);
            site.Load(folder);
            site.ExecuteQuery();
            site.Load(folder.Files);
            site.ExecuteQuery();
            foreach (Microsoft.SharePoint.Client.File file in folder.Files)
            {
                string destinationfolder = Destination + "/" + folder.ServerRelativeUrl;
                Stream fs = Microsoft.SharePoint.Client.File.OpenBinaryDirect(site, file.ServerRelativeUrl).Stream;
                byte[] binary = ReadFully(fs);
                if (!Directory.Exists(destinationfolder))
                {
                    Directory.CreateDirectory(destinationfolder);
                }
                FileStream stream = new FileStream(destinationfolder + "/" + file.Name, FileMode.Create);
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(binary);
                writer.Close();
            }
        }
        public static byte[] ReadFully(Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }
    }
}