Using Flow to get or check for files in a large SharePoint library can be a little tricky. If you are sure your library will always stay under 5,000 items the Get Files (properties only) Flow action is a quick n’ simple approach to use. When your library crosses over the mythical 5k mark or somewhere in that neighborhood, the Get Files action will fail to return results without warning. What I’m outlining below are other options when working with large libraries.
Option 1: Get Files using an Odata filter query downside: only use this in small libraries
Option 2: use the SharePoint API downside: the lack of transparency from Microsoft related to how often data is crawled.
Option 3: use the SharePoint API along with a filter action on the library. This option does require that you have metadata set up on the library to filter on and there is not a wildcard / contains option. The filtered value needs to be exact. downside: you will need to set up your metadata ahead of time or create it after the fact then backfill.
The more I learn about Flow and SharePoint Online, the more I’m starting to like Option 3 when doing a lookup type of search. SharePoint Search is an extremely powerful tool if the search index is fresh.
Out of the box, if you upload a document to SharePoint and tag metadata to it, that data is attached to the document properties. This is the case for most all column types you create in a library, and on random occasions, Document IDs are copied.
If you are working with SharePoint on-prem you can disable this functionality at the SPWeb level, but that’s not the case with SharePoint Online.
A couple of notes about the script. You can download the SharePoint DLLs from Microsoft: link In my tenant, using Get-Credential requires that I use my company email address and an App Password. This may/not be the case with your tenant. App Password info: link
And… after running the script, upload another copy of the Excel file to the library.
The TacoFiller property is no longer being extracted from the Excel file!
As of today, there is not a Logic App trigger for Azure File Storage, so I went with a schedule-based approach. Yes, this example leaves out a lot of fine-tuning, but it will get you headed in the right direction.
Create a blank Logic app Trigger: Schedule Action: Azure File Storage – List files Action: SharePoint – Create file
After you add the SharePoint action, the Logic App should automatically add a For Each action and place the SharePoint Create File action inside of it.
In the last screenshot, I tested the Logic App by uploading a couple of documents in Azure Storage Explorer, then I manually ran the Logic App (click the Run button).
Again, this is a simple example. The example does not account for processing the same files over and over…
I was trying to use a SQL Insert Row action to insert a new row in a SQL Server table and received a Bad Gateway error. First, I thought it was a permissions issue, then I thought my Flow stopped working…
Turned out to be an issue with the amount of data being inserted into a field. One SQL column was set to varchar(X) and the Flow was trying to insert more characters than X.
Error I was receiving: System.MissingMethodException: Method not found: ‘System.Runtime.Remoting.ObjectHandle System.Activator.CreateInstance(System.String, System.String)’.
I tried uninstalling VScode, removed all traces of SharePoint from my laptop, and cleared the GAC. Nothing worked.
Here is what did work: In VScode:
Open the Command Palette on Windows or Linux with Ctrl+Shift+P. On macOS, use Cmd+Shift+P.
Search for Session.
Click on PowerShell: Show Session Menu.
Choose one of the ___ (x86) options
Not sure how, but I was using an x64 session and SharePoint PNP clearly didn’t like that.
Edit: Updated VScode to the latest version and it managed to reset my session settings. When this happened, it caused my CSOM scripts to report a The remote server returned an error: (400) Bad Request error. The fix above will resolve the error.
If you encounter this error: This website doesn’t support embedding using just the address ….
You will need to update the HTML Field Security settings in the Site Settings area of your site. In my case, I simply added sharepointed.com to the allow iframes from this domain list, then updated the web part again.
Update and a much better way to approach this: Use a SharePoint App Only Client Id and Secret to access the site, list, or library.
Microsoft documentation: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/security-apponly-azureacs You can create an app principle that is limited to a single site, list, library, or a combination of them: https://piyushksingh.com/2018/12/26/register-app-in-sharepoint/
In this example, I’m connecting to a Site Collection on my tenant.
1) You have created a token in your o365 site
1.2) On the left site of the page click Security & privacy, then click Create and manage app passwords
1.3) In the app password page click the create button and give it a name.
1.4) Save the password to a secure location.
1.5) There is a better way of doing this that I will cover in a future post.
2) You have downloaded to CSOM DLL(s) from Nuget