Searching SharePoint Using PowerShell

In this example, I needed to search a farm for every site under a managed path. BUT, the sites I’m searching for were built using a 3rd part tool and would not correctly appear in the search results.  The problem was related to having Trim Duplicates enabled by default.  Easy fix… Set your search property trim duplicates = false.

$site = Get-SPSite "https://sharepointed.com"

$keywordQuery = New-Object Microsoft.Office.Server.Search.Query.KeywordQuery($site)<br>$queryText = "ContentClass:STS_Site AND Path:https://sharepointed.com/TACOS/*"

$keywordQuery.QueryText = $queryText<br>$keywordQuery.TrimDuplicates = $false 
$searchExec = New-Object Microsoft.Office.Server.Search.Query.SearchExecutor<br>$searchResults = $searchExec.ExecuteQuery($keywordQuery)

Write-Host "'r'n"
$table = $searchResults.Table
Write-Host $table.Length" Results Found" -BackgroundColor "Green" -ForegroundColor "Black"
$table | select Title, Path, IsDocument

The search results will display all sites that have Taco as its managed path. If you are not retrieving the results you expect, try switching TrimDuplicates = $false .

Remove Upload Button and Drag files here to upload text

There might be a better way to get this done, but for now, this works for me. Keep in mind this update works on a per view basis. If I find a way to correctly update the masterpage, I will update this post.

At a high level, you will need to modify the page, add two content editor web parts, and save.

A normal page displaying the Upload button and Drag files here to upload text.

Start by editing the page.  Top right of the screen, click the gear icon, then select Edit Page.  Once in edit make, add two Content Editor web parts to the page.

Place your cursor in the top web part, and select Edit Source in the Format Text ribbon window.

In the Source window, enter the following text: link to script

Update the second web part by inserting the following text: link to script

Once the web parts have been updated, click on Stop Editing.

All done.

You can also upload the attached web parts to your page:  zip file of web parts

Email address is incorrect for user in SharePoint

In the process of migrating from SharePoint 2010 to 2016 and ran into a small problem.

When trying to get the email property from the SPUser class, it returned a value of domain\userName. Clearly, this is not correct and caused some other issues.

Sample code

$web = Get-SPWeb "https://webapp.taco/toppings/cheese"
$userEnsure = $web.EnsureUser("domain\yourNameHere")
write-host $userEnsure.Email

Running this returned domain\yourNameHere, when it should have returned yourname@domain.com.

Navigate to Central Admin, then cruise over to your User Profile Service. Once there, run a full synchronization.
Profile Service –> Synchronization –> Start Profile Synchronization –> Start Full Synchronization

Run the PowerShell script again, and it will return the correct data.

Same idea as above but using the SharePoint ClientContext.

            using (ClientContext clientContext = new ClientContext("https://webapp.taco/toppings/cheese"))
            {
                Web web = clientContext.Web;

                clientContext.Load(web);
                clientContext.Load(web.CurrentUser);
                clientContext.ExecuteQuery();

                var userEmail = web.CurrentUser.Email;
           }

SharePoint listdata.svc Returns Error – FIXED

With SharePoint 2016 and 2013:
If you try to access listdata.svc you receive an error This page can’t be displayed or Sorry, something went wrong.
SharePoint Designer, you try to open Lists and Libraries and receive a message of There are no items to show in this view.

The root problem is that the Farm is missing a feature. In SPD, if you click on All Files, Lists, then click on each list and click the Preview in Browser button (ribbon). You will sooner or later find the problem list. From there, you can remove the list or find the problem feature and unhook it.

Basic script to find the problem list in SharePoint 2013 and 2016:

function Get-WebPage([string]$url)
{
	$pageContents = ""

	try
	{
		$wc = new-object net.webclient;
		$wc.credentials = [System.Net.CredentialCache]::DefaultCredentials;
		$pageContents = $wc.DownloadString($url);
		$wc.Dispose();
	}
	catch{}
    return $pageContents;
}

$webX = Get-SPWeb "https://yourSpWebUrl"

foreach($list in $webX.Lists)
{
	$listUrl = $list.ParentWeb.Url + "/" + $list.RootFolder.Url

	$xo = Get-WebPage -url $listUrl 

	if($xo -like "*Sorry, something went wrong*")
	{
		Write-Host $listUrl
	}
}

Uploading Files to SharePoint 2016 Using ListData.svc

Update
Another issue we ran into was related to client machines having an outdated cert. Once the updated cert was published to SharePoint, the client machines downloaded the new cert and were able to upload it to SharePoint.

Ran into a small issue when testing some code for a SharePoint 2010 to SharePoint 2016 migration.

With SharePoint 2010, the following code sample would work to upload files to a library.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            }

When trying to use this same code with SharePoint 2016, I was receiving the following Errors:
Output error: An error occurred while processing this request.

Errors from Fiddler:
Auth:
No Proxy-Authenticate Header is present.

No WWW-Authenticate Header is present.

Caching:

Under RFC2616, HTTP/500 responses will not be cached regardless of what caching headers may be present. HTTP/1.1 Cache-Control Header is present: no-cache

This response does not specify explicit HTTP Cache Lifetime information and does not specify a Last-Modified date. Heuristic expiration is typically based on Last-Modified date. Lacking Last-Modified, this response may be revalidated on every use or once per browsing session, depending on the browser configuration.

This response contains neither an ETAG nor a Last-Modified time. This will prevent a Conditional Revalidation of this response.

FIX
For SharePoint 2016, the upload process required that the Path property be populated.

//ServRef is a Service Reference to _vti_bin/ListData.svc
string sharePointSvc = "https://sp2016.some.url/sites/random/_vti_bin/ListData.svc";

            using (FileStream file = File.Open(@"C:\test1.docx", FileMode.Open))
            {
                ServRef.RandomDataContext ctx = new ServRef.RandomDataContext(new Uri(sharePointSvc));

                ctx.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;

                string filename = Path.GetFileNameWithoutExtension(file.Name);
                string path = "/sites/random/dropofflibrary/" + Path.GetFileName(file.Name);
                string contentType = "Interest Summary";

                ServRef.DropOffLibraryItem documentItem = new ServRef.DropOffLibraryItem()
                {
                    Path = path,
                    ContentType = contentType,
                    Name = filename,
                    Title = filename
                };

                ctx.AddToDropOffLibrary(documentItem);
                ctx.SetSaveStream(documentItem, file, false, contentType, path);

                try
                {
                    ctx.SaveChanges();
                }
                catch (Exception ex)
                {
                    var err = ex.Message;
                    throw;
                }
            }

Make Your PowerShell Script Environment Aware

In place of hard-coding URLs for each environment, I decided to make a single script that is environmentally aware. Why? Cuts down on the number of scripts that have to be supported for a single development cycle. To make this more dynamic, you could move this to a function script and reference it from all your scripts.

if ((Get-PSSnapin -Name Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue) -eq $null)
{
	Add-PsSnapin Microsoft.SharePoint.PowerShell
}

#get config database server
$ConfigDB = Get-SPDatabase | where-Object{$_.Type -eq "Configuration Database"}
$serverName = $ConfigDB.Server.Displayname

#replace this with the web app you want to target.  taco, burrito, nacho...
$webApp = "taco"

#set variable equal to the environment url
$siteURL = switch ($serverName.ToLower())
{
	"dev_db" {"http://$webApp.sharepointed.com/"}
	"test_db" {"http://test$webApp.sharepointed.com/"}
	"build_db" {"http://build$webApp.sharepointed.com/"}
	"prod_db" {"http://$webApp.sharepointed.com/"}
}

Same as above, but using a wildcard in the switch statement.

$siteURL = switch -Wildcard ($serverName.ToLower())
{
	"*dev*" {"http://$webApp.sharepointed.com/"}
	"*test*" {"http://test$webApp.sharepointed.com/"}
	"*build*" {"http://build$webApp.sharepointed.com/"}
	"*prod*" {"http://$webApp.sharepointed.com/"}
}

Make sure to check $serverName = $ConfigDB.Server.Displayname
This might need to be replaced with $ConfigDB.Displayname

Server was unable to process request Access is denied. Connecting to SharePoint

Errors:
Server was unable to process request. —> Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

Object moved
Object moved to

In your project, navigate to the Web Reference folder.
One at a time, open each of your reference folders.
Edit the Reference.cs file (notepad++ or Visual Studio).
Directly after this line: public partial class Lists : System.Web.Services.Protocols.SoapHttpClientProtocol {
Insert the following code:

protected override System.Net.WebRequest GetWebRequest(Uri uri)
{
System.Net.WebRequest wr = base.GetWebRequest(uri);
wr.Headers.Add(“X-FORMS_BASED_AUTH_ACCEPTED”, “f”);
return wr;
}

Save the file.
Try connecting to SharePoint again.
If it works, complete the same update on your other Reference.cs files.

Get all Groups and Users in a Site Collection or Web

Recently a user contacted me asking how to get all the groups and users in a site and subsites.

Both example will output to the C:\ drive of the server.

This script will output all the groups and user from the root of a site collection and all the subsites.

$site = Get-SPSite "http://sharepointed.com"

$userOutput = @()

foreach($subWebs in $site.AllWebs)
{
Write-Host $subWebs.Title

$groups = $subWebs.sitegroups

foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name ‘Web URL’ -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name ‘Web Title’ -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name ‘Group’ -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name ‘Account’ -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
}

$userOutput | export-csv c:\site_collection_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

 

This script will output all the groups and users from a single web.

$web = Get-SPWeb "http://sharepointed.com/sites/taco/SubSite"

$userOutput = @()

$groups = $web.Groups
$users = $web.Users

#get groups and users in the groups
foreach($group in $groups)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$userOutput += $object

foreach($userG in $group.users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value $group.Name
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $userG.Name

$userOutput += $object
}
}
#get users not in groups
foreach($user in $users)
{
$object = New-Object PSobject
$object | Add-Member -Name 'Web URL' -MemberType Noteproperty -Value $web.URL
$object | Add-Member -Name 'Web Title' -MemberType Noteproperty -Value $web.Title
$object | Add-Member -Name 'Group' -MemberType Noteproperty -Value ""
$object | Add-Member -Name 'Account' -MemberType Noteproperty -Value $user.Name

$userOutput += $object
}

$userOutput | export-csv c:\web_$(get-date -f yyyy-MM-dd-hhmmss).csv -notypeinformation

The Web server at does not appear to have Microsoft SharePoint Foundation installed

When you try to open a SharePoint site from SharePoint Designer and you receive this error:
The Web server at does not appear to have Microsoft SharePoint Foundation installed

sharepoint-not-installed

Microsoft suggests doing the follow (did not work in my case)

To resolve this problem, enable the client integration. To do this, follow these steps:
Start SharePoint 2010 Central Administration, and then click Security.
Click Specify authentication providers.
If the Web application in the Web Application list is not the same application that you want to open in SharePoint Designer 2010, click Web Application on the Change Web Application list.
On the Select Web Application page, click the name of the Web application that you want to open.
Under Zone, Click Default to open the Edit Authentication page.
Under Enable Client Integration, click Yes, and then click Save.

In my case, it turned out that my web.config file was out of wack.
Only one of my web apps would fail to load in SharePoint Designer, all the other Web Apps / sites would load.
Opened a working and the non-working sites web.configs in NotePad++ and did a file compare.
Located the difference in the file and removed it.
For good housekeeping, i then fired off an IISreset.
*things to note, make sure you have a backup copy of your web.config file and this work was done in my Develop environment.)

Download all files from a document library using client object model

For this to work, you will need to obtain a copy of the SharePoint Client DLL (microsoft.sharepoint.client.dll).

On a server with SharePoint intalled:
SharePoint 2010: 14 hive
SharePoint 2013: 15 hive
SharePoint 2016: 16 hive
C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\

In Visual Studio, create a console application and name it Download_All_Files. Copy all the code below and paste it into your code window. Update site URLs and list name (Taco Storage).

Using this process, I was able to download 10,000 documents from a library. Took around 30 minutes to complete.

using System;
using System.IO;
using Microsoft.SharePoint.Client;

namespace Download_All_Files
{
    class Program
    {
        static void Main(string[] args)
        {
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            // CHANGE THIS to your library name, get this from the Library Settings page
            List list = web.Lists.GetByTitle("Taco Storage");
            site.Load(list);
            site.ExecuteQuery();
            site.Load(list.RootFolder);
            site.ExecuteQuery();
            site.Load(list.RootFolder.Folders);
            site.ExecuteQuery();
            processFolderClientobj(list.RootFolder.ServerRelativeUrl);
            foreach (Folder folder in list.RootFolder.Folders)
            {
                processFolderClientobj(folder.ServerRelativeUrl);
            }
        }
        public static void processFolderClientobj(string folderURL)
        {
            // folder on your computer where all the files will be downloaded to
            string Destination = @"C:\\yourFolder";
            
            // input YOUR URL below
            var site = new ClientContext("http://sharepoint/sites/somesite");
            var web = site.Web;
            site.Load(web);
            site.ExecuteQuery();
            Folder folder = web.GetFolderByServerRelativeUrl(folderURL);
            site.Load(folder);
            site.ExecuteQuery();
            site.Load(folder.Files);
            site.ExecuteQuery();
            foreach (Microsoft.SharePoint.Client.File file in folder.Files)
            {
                string destinationfolder = Destination + "/" + folder.ServerRelativeUrl;
                Stream fs = Microsoft.SharePoint.Client.File.OpenBinaryDirect(site, file.ServerRelativeUrl).Stream;
                byte[] binary = ReadFully(fs);
                if (!Directory.Exists(destinationfolder))
                {
                    Directory.CreateDirectory(destinationfolder);
                }
                FileStream stream = new FileStream(destinationfolder + "/" + file.Name, FileMode.Create);
                BinaryWriter writer = new BinaryWriter(stream);
                writer.Write(binary);
                writer.Close();
            }
        }
        public static byte[] ReadFully(Stream input)
        {
            byte[] buffer = new byte[16 * 1024];
            using (MemoryStream ms = new MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
        }
    }
}