1 00:00:00,270 --> 00:00:07,920 In this video, we are going to create infrastructure for Amazon S3, Lambda and A.B. to perform our 2 00:00:07,920 --> 00:00:08,760 hands on loop. 3 00:00:08,940 --> 00:00:15,120 So we will use a management console to create a lambda function and configure a trigger for Amazon S3 4 00:00:15,360 --> 00:00:21,570 The trigger invokes our function every time when we add an object into our Amazon S3 bucket. 5 00:00:21,570 --> 00:00:23,220 So let's get started. 6 00:00:23,930 --> 00:00:26,900 In order to do that, I'm going to start with creating a bucket. 7 00:00:26,930 --> 00:00:28,730 Please open the engine console. 8 00:00:28,760 --> 00:00:30,170 Go to Amazon S3. 9 00:00:30,380 --> 00:00:32,000 We will create a bucket. 10 00:00:32,450 --> 00:00:36,480 Create Amazon S3 bucket and upload a test file to our new bucket. 11 00:00:36,500 --> 00:00:39,950 So go to buckets and click the create bucket. 12 00:00:40,590 --> 00:00:42,720 And give the name these buckets. 13 00:00:42,720 --> 00:00:49,000 I'm going to give a bucket and give the random name to make it unique and select the region. 14 00:00:49,020 --> 00:00:50,340 If you scroll down. 15 00:00:50,430 --> 00:00:51,450 I'm not leaving. 16 00:00:51,450 --> 00:00:53,130 I'm not changing any conversation. 17 00:00:53,130 --> 00:00:57,480 Leave as it is all this configuration and click to create bucket button. 18 00:00:59,760 --> 00:01:03,710 Look, as you can see that we have success to create a steady bucket. 19 00:01:03,720 --> 00:01:06,580 And if you open the bucket, we don't have any object. 20 00:01:06,600 --> 00:01:08,360 I'm not going to upload any object. 21 00:01:08,370 --> 00:01:13,700 No, because we will create a lambda function that triggered when uploading S3 bucket. 22 00:01:13,710 --> 00:01:19,980 So that's why let's go to the lambda functions, open the lambda function page and I'm going to create 23 00:01:19,980 --> 00:01:21,060 a new function. 24 00:01:22,110 --> 00:01:26,730 And author from The Secret and they give the function name is a function. 25 00:01:27,030 --> 00:01:32,370 If you continue to configure our lambda function, you can see the runtime is node.js and architecture 26 00:01:32,370 --> 00:01:38,310 is leave this for the permissions are important we should change the permission because if you expand 27 00:01:38,310 --> 00:01:43,980 these change default execution role, the default execution rule settings is creating any role with 28 00:01:43,980 --> 00:01:45,570 the basic lambda permission. 29 00:01:45,660 --> 00:01:49,920 But these lambda function will be interact with the S3 and Dynamo DB. 30 00:01:49,950 --> 00:01:56,430 If you come back to our picture, you can come here and see in here or lambda function will be triggered 31 00:01:56,430 --> 00:02:02,520 from the Amazon S3 and interact with the Dynamo DB to insert file metadata in Dynamo DB. 32 00:02:02,520 --> 00:02:09,960 So Lambda function should require permissions to invoke S3 and also access dynamic DB to do that in 33 00:02:09,960 --> 00:02:10,350 here. 34 00:02:10,350 --> 00:02:15,720 When we creating Lambda function, we create a new rule from the policy template. 35 00:02:15,750 --> 00:02:21,750 If will remember that a place has some predefined policy templates that we can give this policy template. 36 00:02:21,750 --> 00:02:30,450 So for example C give the row name as a three function rule and see the policy templates. 37 00:02:30,450 --> 00:02:36,420 When we interact any dynamic DB resource from the lambda function, we should select the simple microservice 38 00:02:36,420 --> 00:02:36,930 permission. 39 00:02:36,930 --> 00:02:40,950 This will be part from crud operations from the lambda function. 40 00:02:40,950 --> 00:02:48,390 This is our Dynamo DB integration, but we have also required S3, S3 related permissions. 41 00:02:48,390 --> 00:02:51,930 You can see in here Amazon S3 Object Read only permission. 42 00:02:51,930 --> 00:02:58,290 So we can give these S3 written permission because also in the Lambda function, we can reach the S3 43 00:02:58,290 --> 00:03:02,240 and read some bucket and objects from the S3 in the Lambda function code. 44 00:03:02,280 --> 00:03:09,290 That's why it is required these policies also OC after edit these two policy and we are going to create 45 00:03:09,300 --> 00:03:12,060 new role for our newly created function. 46 00:03:12,060 --> 00:03:13,710 Which name is S3 function? 47 00:03:13,710 --> 00:03:16,170 So make sure that you have selected these. 48 00:03:16,170 --> 00:03:18,150 Create a new role from a policy. 49 00:03:18,180 --> 00:03:21,030 Give the role name and after that select the policy. 50 00:03:21,030 --> 00:03:26,610 Simple Microsoft's permission for Dynamic DB Amazon S3 Object Read only permission for the Amazon S3 51 00:03:26,610 --> 00:03:30,690 interaction in Lambda Function Code and after that, click the create function. 52 00:03:34,600 --> 00:03:40,600 So this will take some time to creating role with these policies and also this lambda function. 53 00:03:41,230 --> 00:03:43,120 It is successfully created. 54 00:03:43,360 --> 00:03:49,750 If you scroll down and come to the configuration and permissions, you can see that we have successfully 55 00:03:49,750 --> 00:03:51,420 create a three functional. 56 00:03:51,430 --> 00:03:57,950 And if you select the function resource summary, we can see the Amazon Dynamo DB comes in here which 57 00:03:57,970 --> 00:04:05,410 performing all operations and also we can see Amazon S3 for allowing to get object actions. 58 00:04:05,500 --> 00:04:06,480 Okay, very good. 59 00:04:06,490 --> 00:04:10,660 So if you come back to our code path now, we can develop our function. 60 00:04:10,660 --> 00:04:17,530 But before that I'm going to only adding the log to see incoming log from the Amazon S3. 61 00:04:17,560 --> 00:04:22,870 Please add this line of code at the beginning of the Lambda function and I am not going to implement 62 00:04:22,870 --> 00:04:23,930 the function code itself. 63 00:04:23,980 --> 00:04:29,350 Now before that, it is good to understand which incoming event comes from the one the asynchronous 64 00:04:29,350 --> 00:04:30,940 invoke from the Amazon S3. 65 00:04:30,940 --> 00:04:34,960 So click to deploy and update the function code. 66 00:04:34,960 --> 00:04:39,850 We have added log to see incoming event from S3 one asynchronous invocation. 67 00:04:39,850 --> 00:04:44,110 So now it is time to very important configuration, which is the trigger. 68 00:04:44,110 --> 00:04:50,110 When you open the S3 function, you can see the trigger button comes in here, click this trigger and 69 00:04:50,110 --> 00:04:51,580 we can select a source. 70 00:04:51,610 --> 00:04:57,370 In our case, this lambda function is triggered from the Amazon S3 so that's why we are going to select 71 00:04:57,370 --> 00:04:58,240 the S3. 72 00:04:59,680 --> 00:05:03,900 And after that it is requesting us the HTC bucket name. 73 00:05:03,910 --> 00:05:08,260 If you come here you can see that this is the name of the edit created bucket name. 74 00:05:08,260 --> 00:05:11,140 Select this bucket name and here you can see the event type. 75 00:05:11,140 --> 00:05:18,160 This is the important because we would like to trigger our lambda function when any object uploaded 76 00:05:18,160 --> 00:05:19,900 to this S3 bucket. 77 00:05:19,900 --> 00:05:23,230 So run the upload, perform in the bucket. 78 00:05:23,260 --> 00:05:29,920 This will be the put operation maybe you remember put object command is using when uploading S3 bucket 79 00:05:29,920 --> 00:05:37,570 so that's why select even time as a put this is very important so I don't add any optional prefix or 80 00:05:37,570 --> 00:05:41,320 suffix and we don't any recursive invocations. 81 00:05:41,320 --> 00:05:43,150 So it is also important. 82 00:05:43,150 --> 00:05:50,440 So that's why we should check this statement and make sure that we don't have any recursive invocations 83 00:05:50,440 --> 00:05:55,570 and we have give the acknowledgement about this checkbox and click add button. 84 00:05:57,400 --> 00:06:04,210 So as you can see that the trigger is a bucket was successfully added to the function which is our function 85 00:06:04,210 --> 00:06:07,320 now to trigger from the S3 bucket. 86 00:06:07,330 --> 00:06:11,580 And the function is now receiving events from the trigger, which is the bucket. 87 00:06:11,590 --> 00:06:16,390 So if you come here and click the configuration and triggers, you can also see the details in here. 88 00:06:16,420 --> 00:06:22,690 The trigger bucket is S3 bucket dash these number and if you click the details, you can see the old 89 00:06:22,690 --> 00:06:24,670 details about the bucket name. 90 00:06:24,940 --> 00:06:28,900 That means we have done with the lambda create operation. 91 00:06:28,900 --> 00:06:32,590 These is the permissions and triggers are adding successfully. 92 00:06:32,620 --> 00:06:39,010 Now I'm going to create Dynamo DB as you know that we are continuing with the creating infrastructure 93 00:06:39,010 --> 00:06:40,270 on this environment. 94 00:06:40,270 --> 00:06:41,320 This is the first step. 95 00:06:41,320 --> 00:06:46,300 So that's why I'm going to create resources first collected IMDB and click the table. 96 00:06:47,490 --> 00:06:50,460 So table name should be the bucket objects. 97 00:06:50,460 --> 00:06:53,670 We will store the bucket objects metadata in this table. 98 00:06:53,670 --> 00:06:55,770 So that's why I'm giving this table name. 99 00:06:55,800 --> 00:06:58,800 After that, we can set the partition key. 100 00:06:58,800 --> 00:07:00,870 Partition key should be the key. 101 00:07:00,870 --> 00:07:02,220 So this is so important. 102 00:07:02,220 --> 00:07:09,840 Please set the key as a partition key because we will pass the incoming bucket objects from the Amazon 103 00:07:09,840 --> 00:07:12,780 S3 to Dynamo DB table for that purpose. 104 00:07:12,780 --> 00:07:21,750 If you look at the S3 JSON format, you can see that this is the object JSON object for the bucket objects. 105 00:07:21,750 --> 00:07:24,930 That means it is including key and other attributes. 106 00:07:24,930 --> 00:07:28,760 We will got this key as a passing key and string value. 107 00:07:28,800 --> 00:07:32,460 That means we will pass this item into the Dynamo DB table. 108 00:07:32,460 --> 00:07:33,960 Which name is bucket object? 109 00:07:33,960 --> 00:07:37,350 So after set the partition key is a key. 110 00:07:37,350 --> 00:07:45,600 I'm not going to set cert key and we can set the customize settings in order to make copies more on 111 00:07:45,600 --> 00:07:46,290 demand. 112 00:07:46,290 --> 00:07:52,410 And I'm not going to change any other configuration leave as it is and click the create table button. 113 00:07:53,520 --> 00:07:59,970 You can see that we have created the infrastructure for Amazon S3, a Lambda and Dynamo DB. 114 00:08:00,000 --> 00:08:04,890 Now it is time to develop our Lambda function for asynchronous invocation from Amazon S3. 115 00:08:05,010 --> 00:08:09,000 Read data from Amazon S3 and persist on Dynamo DB table.