Part of a project I was working on required mashing up some data from SharePoint with data stored in datalake. We settled on creating a Databricks notebook to read an input file, query data lake using the input file, and then export an enriched file.
Here’s a high-level overview of what’s going to be created: Call the notebook, parse the JSON response, loop until the notebook has finished, then respond to the notebook’s output.
In my case, triggering the notebook will require knowing its URL, bearer token, job id, and input parameters.
Parse the response from the HTTP call:
The notebook will take a little time to spin up, then process the input file. The best way to handle this is to leverage a basic do-until loop to check the status of the notebook job. I opted to use a one-minute delay, call the API to get the job status, parse the response, then evaluate if it’s finished.
One thing to note about the do until action, you don’t want it to run for eternity, and to avoid adding complexity to it, you don’t want to add extra evaluations like: if looped X times, stop If you expand the Change limits option, you can set how many times it loops or change the duration. Here I’ve set the action to stop looping after 20 tries. For more info on this, please check SPGuides for a detailed overview. The last step in the flow is to process the response from the notebook. If the job is success(full), get the file from blob storage and load it to SharePoint; otherwise, create a Slack alert.
That’s it; using the example above, you can trigger a Databricks notebook using a Flow.
I’m working on a Power Pages project that requires a Jira service desk task to be created for each portal submission. Out of the box, Jira provides a simple connector to create tasks and requests, but the connect falls short of handling field types other than simple text. This means choice, checkbox, and dropdown fields are not available. This only leaves a couple of options, and I opted to use a simple HTTP action to create the tasks.
Basic overview of what I’ll be creating: Flow that’s triggered by a dataverse row creation Create a Jira task and populate metadata Attach a file to the Jira task
Jira fields and types: Issue Type – Choice Request Type – Choice Tortilla – Choice Meat – Choice Veggies – Checkbox multi-select Number of Tacos – Number Pickup Date Time – Date and Time Summary – Text Attachment – Attachment
Interfacing with the Jira API requires knowing a little about the fields you’ll be updating and the project and issue type you want to use. If you haven’t created one already, you need a Jira API token to work with the API.
Request type: Go to Project Settings, then look at the URL and copy the value after pid= https://taco.atlassian.net/secure/project/EditProject!default.jspa?pid=10001
With the ID, you can query the service desk request-types endpoint https://taco.atlassian.net/rest/servicedesk/1/servicedesk/request/10001/request-types In the returned payload, note the portal key and key values; combine the two, and you have the request type value tr/9f7c4029-6d23-4cb1-bb8a-02d0050d944b
Project key: The project key is available on the project settings page, it’s listed under the name field. Example: TACOS
Issue type: For simplicity, I’m only dealing with one issue type, and I captured the issueType value using the request-types endpoint noted above. Example: “issueType”: 10015
For the remaining field values, you can get them in one of two ways. Create a new issue in the browser, then use the browser developer tools (F12) to inspect each field’s HTML value.
The other option is to click the gear icon (top right), select Issues, click on Custom Fields, search for a field, click on it, click edit detail, then grab the id value from the URL. Once the id is captured, join it with customfield_, resulting in customfield_10073, this is the fields internal value.
In this example, the summary issue type fields are the only ones that does not have a customfield_X naming convention. It might be possible that some system-generated fields have a diffident naming convention, but I’ll dig into that another day.
Column Display Name
Column Internal Name
Column Type
Issue Type
issuetype
system
Request Type
customfield_10010
system
Tortilla
customfield_10073
Select List (single)
Meat
customfield_10074
Select List (single)
Veggies
customfield_10075
Checkboxes
Number of Tacos
customfield_10076
Number Field
Pickup Date Time
customfield_10077
Date Time Picker
Summary
summary
system
Endpoint URL: https://taco.atlassian.net/rest/api/3/issue/ Headers: {“Content-Type”: “application/json”} Authentication: Raw Key: Basic aWhddsfadfafa..NOT…A…REAL…KEY..dafdfdafd= Example payload:
For the attachment, I’m getting a file from SharePoint and passing its contents to the API call. The same thing works with Azure blob storage or grabbing file from dataverse. If you want to attachment more than one file, create additional HTTP attachment calls.
Over the years, some updates to Flow have been better than others, and others, not so much. If memory serves, the send an email action would use dynamic hyperlinks without much work, but something went sideways with one of the updates causing dynamic hyperlinks not to work as you’d want.
Here is a basic example of including a hyperlink in an outgoing email; further down the page, I’ll provide a more realistitc example.
In this example, I setup the Flow to trigger when an item is added to a SharePoint library. The key thing to note in this example is the value of the varHyerplink variable; note the double quotes around the link to item.
Using the SharePoint HTTP flow action to update a person or group field, I kept getting this error:
A 'PrimitiveValue' node with non-null value was found when trying to read the value of a navigation property; however, a 'StartArray' node, a 'StartObject' node, or a 'PrimitiveValue' node with null value was expected.
The field I was attempting to update is named Submitted By, with an internal name of Submitted_x0020_By. Each time I tried to update the field I was seeing the error noted above. It wasn’t until I looked at one of my previous flow runs did I notice what the issue was. It turns out, that the field name I should be using is Submitted_x0020_ById.
Update flow:
How do you update a Person field if the field allows for multiple selections? The example below will update the field with two different user values, but clearly, this could be extended to be more dynamic.
I was in the middle of streamlining an old flow and needed to compare two dates and times. To simplify things, I opted to use the Convert time zone flow action, but it gave me an error that seemed a little odd.
Unable to process template language expressions for action ‘Convert_time_zone’ at line ‘0’ and column ‘0’: ‘The template language function ‘convert timezone’ expects its fourth parameter to be a string that contains a date time format. The provided value is of type ‘Null’. Please see https://aka.ms/logicexpressions#ConvertTimeZone for usage details.’.
Notice that I did not select an option for the Format string field, this is required, but there’s not a red * next to the field.
Select an option for that field, and the universe will be back in alignment.
Below are all the steps required to add your company holidays to your Outlook calendar. This is a simple approach that can easily be updated to pull the holidays and dates from other data sources like Exel, SharePoint, Dataverse…
Overview of the flow that will be created:
All of the holidays (events) will be stored in an array using a compose action.
After that, a parse JSON event will be used to make the events available in the apply to each action. Note: after the parse JSON action is added to the flow, click the generate from sample button and paste in the holidays JSON from the step above.
Next, add an apply to each action to the flow. In the action, the output box will hold the body of the parse JSON action.
The convert time zone action is used to standardize the date format. For the base time use the Date field from the parse JSON action.
The last item that needs to be added is the outlook create event action. Note: for the End time property a formula is used: addDays(body(‘Convert_time_zone’),1)
Unable to process template language expressions for action 'Apply_to_each_sftp_file' at line '1' and column '30517': 'The template language expression 'body('List_files_in_folder')?['body']' cannot be evaluated because property 'body' cannot be selected. Array elements can only be selected using an integer index. Please see https://aka.ms/logicexpressions for usage details.'.
From one day to the next one of my flows stopped working with the error above. The flow is super simple: Get files from an SFTP folder Loop through the files Copy file to another location Delete file
Looking at the apply to each step I noticed body element was referenced. This might have been related to how I set up the loop or the autogenerated action.
Action the flow was unhappy with: Updated the flow and it started working: Note how the ?[‘body’] element is missing from the second picture.
Fix? Created a new Apply to each loop by first setting a variable to the name of the file I’m looping on, then added my other actions. There might be a better way to fix this, but for now, this is what works.
Using Flow to get or check for files in a large SharePoint library can be a little tricky. If you are sure your library will always stay under 5,000 items the Get Files (properties only) Flow action is a quick n’ simple approach to use. When your library crosses over the mythical 5k mark or somewhere in that neighborhood, the Get Files action will fail to return results without warning. What I’m outlining below are other options when working with large libraries.
Setup:
Option 1: Get Files using an Odata filter query downside: only use this in small libraries
Result:
Option 2: use the SharePoint API downside: the lack of transparency from Microsoft related to how often data is crawled.
Result:
Option 3: use the SharePoint API along with a filter action on the library. This option does require that you have metadata set up on the library to filter on and there is not a wildcard / contains option. The filtered value needs to be exact. downside: you will need to set up your metadata ahead of time or create it after the fact then backfill.
Result:
The more I learn about Flow and SharePoint Online, the more I’m starting to like Option 3 when doing a lookup type of search. SharePoint Search is an extremely powerful tool if the search index is fresh.
Scenario: Each day I have a couple of Azure Runbooks export SharePoint list items and upload them to a SharePoint library. If one of the Runbooks fails, I needed to send an email alert that something went wrong.
Basic logic: If files created today in SharePoint <> X, send an email.
The easy solution would have been to loop through the files, check their created date, increment a variable, then make a condition statement.
More-better way: Run flow daily at 6:00 PM Send an HTTP request to SharePoint to get files Parse the response Condition statement — if true, send an email
