SharePoint listdata.svc Returns Error – FIXED

With SharePoint 2016 and 2013:
If you try to access listdata.svc you receive an error This page can’t be displayed or Sorry, something went wrong.
SharePoint Designer, you try to open Lists and Libraries and receive a message of There are no items to show in this view.

The root problem is that the Farm is missing a feature. In SPD, if you click on All Files, Lists, then click on each list and click the Preview in Browser button (ribbon). You will sooner or later find the problem list. From there, you can remove the list or find the problem feature and unhook it.

Basic script to find the problem list in SharePoint 2013 and 2016:

function Get-WebPage([string]$url)
{
	$pageContents = ""

	try
	{
		$wc = new-object net.webclient;
		$wc.credentials = [System.Net.CredentialCache]::DefaultCredentials;
		$pageContents = $wc.DownloadString($url);
		$wc.Dispose();
	}
	catch{}
    return $pageContents;
}

$webX = Get-SPWeb "https://yourSpWebUrl"

foreach($list in $webX.Lists)
{
	$listUrl = $list.ParentWeb.Url + "/" + $list.RootFolder.Url

	$xo = Get-WebPage -url $listUrl 

	if($xo -like "*Sorry, something went wrong*")
	{
		Write-Host $listUrl
	}
}

Officialfile.asmx SharePoint 2013 and 2016

As part of a SharePoint 2010 to SharePoint 2016 migration I needed to test the Officialfile.asmx service. When testing the service I encountered this error:
NotFound

string sFilePath = @"c\taco.txt";
byte[] fileContent = File.ReadAllBytes(sFilePath );

List<My_Web_Ref.RecordsRepositoryProperty> props = new List<My_Web_Ref.RecordsRepositoryProperty>();

//populate the properties for the document
props.Add(new My_Web_Ref.RecordsRepositoryProperty { Name = "Title", Type = "Text", Value = "Taco" });

result = repo.SubmitFile(fileContent, props.ToArray(), null, sFilePath , "domain\\userName");

Adding my account or the account that is submitting files to this group resolved the issue.
Records Center Web Service Submitters

Uploading Files to SharePoint 2016 Using ListData.svc

Update
Another issue we ran into was related to client machines having an outdated cert. Once the updated cert was published to SharePoint, the client machines downloaded the new cert and were able to upload it to SharePoint.

Ran into a small issue when testing some code for a SharePoint 2010 to SharePoint 2016 migration.

With SharePoint 2010, the following code sample would work to upload files to a library.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            }

When trying to use this same code with SharePoint 2016, I was receiving the following Errors:
Output error: An error occurred while processing this request.

Errors from Fiddler:
Auth:
No Proxy-Authenticate Header is present.

No WWW-Authenticate Header is present.

Caching:

Under RFC2616, HTTP/500 responses will not be cached regardless of what caching headers may be present. HTTP/1.1 Cache-Control Header is present: no-cache

This response does not specify explicit HTTP Cache Lifetime information and does not specify a Last-Modified date. Heuristic expiration is typically based on Last-Modified date. Lacking Last-Modified, this response may be revalidated on every use or once per browsing session, depending on the browser configuration.

This response contains neither an ETAG nor a Last-Modified time. This will prevent a Conditional Revalidation of this response.

FIX
For SharePoint 2016, the upload process required that the Path property be populated.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    Path = path,
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            }

Make Your PowerShell Script Environment Aware

In place of hard-coding URLs for each environment, I decided to make a single script that is environmentally aware. Why? Cuts down on the number of scripts that have to be supported for a single development cycle. To make this more dynamic, you could move this to a function script and reference it from all your scripts.

if ((Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null)
{
	Add-PsSnapin Microsoft.SharePoint.PowerShell
}

#get config database server
$ConfigDB = Get-SPDatabase | where-Object{$_.Type -eq "Configuration Database"}
$serverName = $ConfigDB.Server.Displayname

#replace this with the web app you want to target.  taco, burrito, nacho...
$webApp = "taco"

#set variable equal to the environment url
$siteURL = switch ($serverName.ToLower())
{
	"dev_db" {"http://$webApp.sharepointed.com/"}
	"test_db" {"http://test$webApp.sharepointed.com/"}
	"build_db" {"http://build$webApp.sharepointed.com/"}
	"prod_db" {"http://$webApp.sharepointed.com/"}
}

Same as above, but using a wildcard in the switch statement.

$siteURL = switch -Wildcard ($serverName.ToLower())
{
	"*dev*" {"http://$webApp.sharepointed.com/"}
	"*test*" {"http://test$webApp.sharepointed.com/"}
	"*build*" {"http://build$webApp.sharepointed.com/"}
	"*prod*" {"http://$webApp.sharepointed.com/"}
}

Make sure to check $serverName = $ConfigDB.Server.Displayname
This might need to be replaced with $ConfigDB.Displayname

Server was unable to process request Access is denied. Connecting to SharePoint

Errors:
Server was unable to process request. —> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Object moved
Object moved to

In your project, navigate to the Web Reference folder.
One at a time, open each of your reference folders.
Edit the Reference.cs file (notepad++ or Visual Studio).
Directly after this line: public partial class Lists : System.Web.Services.Protocols.SoapHttpClientProtocol {
Insert the following code:

protected override System.Net.WebRequest GetWebRequest(Uri uri)
{
System.Net.WebRequest wr = base.GetWebRequest(uri);
wr.Headers.Add(“X-FORMS_BASED_AUTH_ACCEPTED”, “f”);
return wr;
}

Save the file.
Try connecting to SharePoint again.
If it works, complete the same update on your other Reference.cs files.

Get all Groups and Users in a Site Collection or Web

Recently a user contacted me asking how to get all the groups and users in a site and subsites.

Both example will output to the C:\ drive of the server.

This script will output all the groups and user from the root of a site collection and all the subsites.

$site = Get-SPSite "http://sharepointed.com"

$userOutput = @()

foreach($subWebs in $site.AllWebs)
{
Write-Host $subWebs.Title

$groups = $subWebs.sitegroups

foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name ‘Account’ -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
}

$userOutput | export-csv c:\site_collection_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

 

This script will output all the groups and users from a single web.

$web = Get-SPWeb "http://sharepointed.com/sites/taco/SubSite"

$userOutput = @()

$groups = $web.Groups
$users = $web.Users

#get groups and users in the groups
foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
#get users not in groups
foreach($user in $users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value ""
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $user.Name

$userOutput += $object
}

$userOutput | export-csv c:\web_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

The Web server at does not appear to have Microsoft SharePoint Foundation installed

When you try to open a SharePoint site from SharePoint Designer and you receive this error:
The Web server at does not appear to have Microsoft SharePoint Foundation installed

sharepoint-not-installed

Microsoft suggests doing the follow (did not work in my case)

To resolve this problem, enable the client integration. To do this, follow these steps:
Start SharePoint 2010 Central Administration, and then click Security.
Click Specify authentication providers.
If the Web application in the Web Application list is not the same application that you want to open in SharePoint Designer 2010, click Web Application on the Change Web Application list.
On the Select Web Application page, click the name of the Web application that you want to open.
Under Zone, Click Default to open the Edit Authentication page.
Under Enable Client Integration, click Yes, and then click Save.

In my case, it turned out that my web.config file was out of wack.
Only one of my web apps would fail to load in SharePoint Designer, all the other Web Apps / sites would load.
Opened a working and the non-working sites web.configs in NotePad++ and did a file compare.
Located the difference in the file and removed it.
For good housekeeping, i then fired off an IISreset.
*things to note, make sure you have a backup copy of your web.config file and this work was done in my Develop environment.)

Download all files from a document library using client object model

For this to work, you will need to obtain a copy of the SharePoint Client DLL (microsoft.sharepoint.client.dll).

On a server with SharePoint intalled:
SharePoint 2010: 14 hive
SharePoint 2013: 15 hive
SharePoint 2016: 16 hive
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\

In Visual Studio, create a console application and name it Download_All_Files. Copy all the code below and paste it into your code window. Update site URLs and list name (Taco Storage).

Using this process, I was able to download 10,000 documents from a library. Took around 30 minutes to complete.

using System;
using System.IO;
using Microsoft.SharePoint.Client;

namespace Download_All_Files
{
    class Program
    {
        static void Main(string[] args)
        {
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            // CHANGE THIS to your library name, get this from the Library Settings page
            List list = web.Lists.GetByTitle("Taco Storage");
            site.Load(list);
            site.ExecuteQuery();
            site.Load(list.RootFolder);
            site.ExecuteQuery();
            site.Load(list.RootFolder.Folders);
            site.ExecuteQuery();
            processFolderClientobj(list.RootFolder.ServerRelativeUrl);
            foreach (Folder folder in list.RootFolder.Folders)
            {
                processFolderClientobj(folder.ServerRelativeUrl);
            }
        }
        public static void processFolderClientobj(string folderURL)
        {
            // folder on your computer where all the files will be downloaded to
            string Destination = @"C:\\yourFolder";
            
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            Folder folder = web.GetFolderByServerRelativeUrl(folderURL);
            site.Load(folder);
            site.ExecuteQuery();
            site.Load(folder.Files);
            site.ExecuteQuery();
            foreach (Microsoft.SharePoint.Client.File file in folder.Files)
            {
                string destinationfolder = Destination + "/" + folder.ServerRelativeUrl;
                Stream fs = Microsoft.SharePoint.Client.File.OpenBinaryDirect(site, file.ServerRelativeUrl).Stream;
                byte[] binary = ReadFully(fs);
                if (!Directory.Exists(destinationfolder))
                {
                    Directory.CreateDirectory(destinationfolder);
                }
                FileStream stream = new FileStream(destinationfolder + "/" + file.Name, FileMode.Create);
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(binary);
                writer.Close();
            }
        }
        public static byte[] ReadFully(Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }
    }
}

Add and Remove Shell Access in SharePoint Using PowerShell

How do you add or remove shell access to a web apps content databases?

This script will grant shell access to a user on all the content databases associated with a content database.

$contentDbs = Get-SPContentDatabase -WebApplication "http://yourSharePointWebApp.com/"

foreach($db in $contentDbs)
{
	Add-SPShellAdmin -UserName "domain\user"  -database $db
}

Remove shell access

$contentDbs = Get-SPContentDatabase -WebApplication "http://yourSharePointWebApp.com/"

foreach($db in $contentDbs)
{
	Remove-SPShellAdmin -UserName "domain\user"  -database $db
}

After running the remove script, make sure you check the WSS_Admin_WPG and WSS_WPG groups on the servers in your farm.

Error Microsoft.Office.RecordsManagement.RecordsRepository.Record

One one my testers was receiving this error when testing a PowerShell script that was doing records management in SharePoint.

System.Management.Automation.RuntimeException: Unable to find type [Microsoft.Office.RecordsManagement.RecordsRepository.Records]: make sure that the assembly containing this type is loaded.
at System.Management.Automation.TypeLiteral.resolveType()
at System.Management.Automation.TypeNode.ResolveType()
at System.Management.Automation.TypeNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.AssignmentStatementNode.Execute(Array input, Pipe outputPipe, ExecutionContext context)
at System.Management.Automation.StatementListNode.ExecuteStatement(ParseTreeNode statement, Array input, Pipe outputPipe, ArrayList& resultList, ExecutionContext context)

User was remoted into a SharePoint server, but did not have the right permissions on the server. I tried adding the user to Power Users group, but that didn’t help. Ended up adding the users to the local server Administrators group and the error went away.