Form Processing Workflow with AI Builder- Part 2

As I mentioned in the last post, I want to create a workflow that I can easily scan evaluations forms, extract the values in the forms and then able to put them in another folder, so I can access them via Power BI to analyze the evaluations.

In the first post, I create my form processing model that can analyze and detect the fields in the form.

 

Second Step: Create Workflow that using form Processing Model

 

Log into Microsoft Flow with the same account you created the form processing model. After sign in, make sure you are in the sane environment and you can see the AI Builder Icon in the left pannel

 

Note! you need to create a solution to use AI Builder model here!

Click on the Solutions option, create a new Solution and name it you can specify the Display Name, Name, Publisher ( That I select the CDS default Publisher).

I create a new solution with the name of AI Builder.

Navigate to the new environment, in the AI Builder –> solution click on the New option.

By clicking on the Flow option, you will be navigated to the Flow creating the page. In the new page, for the trigger, you need to choose the Onedrive

and choose the Trigger When a file is created.

For Onedrive, you need to specify the path of the folder, for when a file is created. My aim is to put the scan files ( in jpg format) there, so when a file is created there, the Microsoft Flow activated.

——->

Add a new step to call the AI model created in Form Processing model. you need to search for Predict action.

Note! you do not have access to Predict option in normal Flow creating, that why you need to create flow in the solution first. 

after selecting the Predict action, you need to select two fields:

Model and Request Payload.

In the Model, select the model you created in the last steps.
{

“base64Encoded”: EXPRESSION,

“mimeType”: image/jpeg

}

 

for pdf, you need to change the code as below: the type change to application and pdf

{

“base64Encoded”: EXPRESSION,

“mimeType”: application/pdf

}

In this example, I am using the image/jpeg file.

however, I need to pass the file name from ONedrive to the code and replace it with Expression

string(triggerBody()?[‘$content’])

just make sure to put the code in Expression area first.

Before going to next step for analysing the result, Test and Run the code to make sure you access to Onedrive and also the Predict code works well, and also copy the JSON output to create a schema.

so Save (put proper name also), Then click on the Test to see the first run. Make sure to have some data in the Onedrive folder ( form). If there is no error, you should see the green message as your flow run successfully. 

Now click on the Predict, then Output, and copy the JSON code.

and back to Edit Mode again. We need to pars the result, the output of the Predict component is a JSON code, so we need the JSOn Parser component. Click on the New Step and search for Parse JSON. In Parse JSON, you need to put the Content fields, click and from Dynamic Content select the Response Payload.

Now, click on the Schema section, and use the option Use Sample payload to generates the schema.

Now in the new editor past the code, you already copied and then click on the Done.

The schema will be generated.

We need to write back the result to another folder in Onedrive as result. In the next step, choose other action, create a file in one drive.

3 things we need to consider:

1- Folder Path in One Drive: need to create the folder beforehand and put the link in the filed

2- File Name: should be Unique name: I want to create a file with : current date and time+filename and the file type as txt

concat(utcNow(),triggerOutputs()?[‘headers/x-ms-file-name-encoded’],’.txt’)
the Concat function, help me to create a name with time date+ file name+.txt

3- File Content: The data to be written in the output

I use function concat to create a text for the out put

concat()

then the first comment was Evaluation Comment

so my code change to

concat(‘Evaluation comment :’,)

the other elemnts I want to add was the json part for the field What you Learned, to extract that part I use below code

body(‘Parse_JSON’)?[‘predictionOutput’]?[‘labels’]?[‘What you learned’]?[‘value’]

Now, I concat it again

concat(‘Evaluation comment :’,body(‘Parse_JSON’)?[‘predictionOutput’]?[‘labels’]?[‘What you learned’]?[‘value’])

I did the same for the Session title and Speaker Name so finally the code is:

concat(‘Evaluation comment :’,body(‘Parse_JSON’)?[‘predictionOutput’]?[‘labels’]?[‘What you learned’]?[‘value’] , ‘ -Session Title: ‘,body(‘Parse_JSON’)?[‘predictionOutput’]?[‘labels’]?[ ‘Session Title’]?[‘value’], ‘ -Speaker Name : ‘,body(‘Parse_JSON’)?[‘predictionOutput’]?[‘labels’]?[‘Speaker Name’]?[‘value’])

 

Just Save again the whole flow and test it with the Current One drive test.

Now I check the OneDrive, a new file has been created

 

so the information in the one drive has been read from folder scan, then form processing model has been applied, now the text is accessible from result folder.

After test now you can upload the files to Scan folder and see the result in Result folder.

Next, you can analyse the result using Power BI.

Some note from my demo: for Trigger the Form processing using creating a file in Onedrive, Upload and copy a file will be considered as creating a file and the flow trigger. But Move to option does not consider as creating a new file.

https://docs.microsoft.com/en-us/ai-builder/form-processing-model-in-flow

Leila Etaati on LinkedinLeila Etaati on TwitterLeila Etaati on Youtube
Leila Etaati
Trainer, Consultant, Mentor
Leila is the first Microsoft AI MVP in New Zealand and Australia, She has Ph.D. in Information System from the University Of Auckland. She is the Co-director and data scientist in RADACAD Company with more than 100 clients in around the world. She is the co-organizer of Microsoft Business Intelligence and Power BI Use group (meetup) in Auckland with more than 1200 members, She is the co-organizer of three main conferences in Auckland: SQL Saturday Auckland (2015 till now) with more than 400 registrations, Difinity (2017 till now) with more than 200 registrations and Global AI Bootcamp 2018. She is a Data Scientist, BI Consultant, Trainer, and Speaker. She is a well-known International Speakers to many conferences such as Microsoft ignite, SQL pass, Data Platform Summit, SQL Saturday, Power BI world Tour and so forth in Europe, USA, Asia, Australia, and New Zealand. She has over ten years’ experience working with databases and software systems. She was involved in many large-scale projects for big-sized companies. She also AI and Data Platform Microsoft MVP. Leila is an active Technical Microsoft AI blogger for RADACAD.

Leave a Reply