1
00:00:00,120 --> 00:00:07,260
In this video, we are going to test kinesis data streams for using a lambda to process kinesis data

2
00:00:07,260 --> 00:00:07,980
streams.

3
00:00:08,160 --> 00:00:14,960
So that means we will send item to Kinesis data streams and these will be pulled from the database.

4
00:00:15,000 --> 00:00:18,390
Lambda and invocation will be happen from the lambda function.

5
00:00:19,090 --> 00:00:20,180
Okay, let's get started.

6
00:00:20,190 --> 00:00:22,320
Please open the widgets to the code.

7
00:00:22,350 --> 00:00:28,170
As you remember that we have developed our lambda function code for iterating incoming records and we

8
00:00:28,170 --> 00:00:34,950
will decode the incoming kinase theta and log the payload of the data.

9
00:00:35,190 --> 00:00:41,550
Okay, so if you look at the event JSON object, you can see that the whole data is transferred with

10
00:00:41,550 --> 00:00:46,290
the base 64 encode way in the data of the Kinesis section.

11
00:00:46,290 --> 00:00:53,220
So we will try to see which data goes to with this example record and even JSON.

12
00:00:53,220 --> 00:01:01,650
In order to do that, please open the console, go to Kinesis function and go to test section and create

13
00:01:01,650 --> 00:01:05,280
a new test event name, which is the test kinesis.

14
00:01:05,430 --> 00:01:08,550
And if you select the kinesis.

15
00:01:09,500 --> 00:01:15,200
Data streams we can select here and see the example event JSON file.

16
00:01:15,320 --> 00:01:22,970
And if you scroll down, I'm going to save this event template and click the test button and this will

17
00:01:22,970 --> 00:01:27,920
be in our lambda function and you can see the execution performed successfully.

18
00:01:27,950 --> 00:01:37,880
If you go to monitor and we've included looks, we should see the decode value of the these data streams

19
00:01:37,970 --> 00:01:44,840
and we will log these value into our cloud watch logs, open the latest cloud watch logs here you can

20
00:01:44,840 --> 00:01:51,070
see the incoming event, JSON from the kinesis and these are the basics for encoded value.

21
00:01:51,080 --> 00:01:54,620
And if you scroll down, you can see the decoded payload in here.

22
00:01:54,650 --> 00:01:57,320
This is basically a text message which is the halo.

23
00:01:57,350 --> 00:01:58,940
This is test one, two, three.

24
00:01:59,300 --> 00:02:05,240
So you can see the decoded way and we look this decoded way in here and we can perform any business

25
00:02:05,240 --> 00:02:08,660
logic after decode this payload comes from the kinesis.

26
00:02:08,870 --> 00:02:09,260
Okay.

27
00:02:09,290 --> 00:02:16,580
This is one way to test our kinesis data streams and another way is using cloud commands.

28
00:02:16,580 --> 00:02:23,750
If you come here and get the kinesis name, as you remember that we have Amazon Kinesis and if you go

29
00:02:23,750 --> 00:02:25,880
to data systems, we have a test stream.

30
00:02:25,880 --> 00:02:30,950
If you open the Visual Studio code, I have create separate lecture and section folder.

31
00:02:30,950 --> 00:02:37,340
And if you go to command text file, you can see the example CLI command that is using the kinesis,

32
00:02:37,340 --> 00:02:40,850
put record API and we are placing the city name.

33
00:02:40,850 --> 00:02:46,030
Our city name is disk kinesis and we have give you the partition key.

34
00:02:46,040 --> 00:02:47,510
Also provide the data.

35
00:02:47,540 --> 00:02:50,480
Data is a text message as you can see in here.

36
00:02:50,510 --> 00:02:53,390
Let me run this command and see what's happened.

37
00:02:53,630 --> 00:02:57,230
We are trying to put record in the place kinesis.

38
00:02:57,260 --> 00:03:05,030
Please go to lecture folder and hit the enter for running this command which is the kinesis put record.

39
00:03:06,030 --> 00:03:10,180
So as you can see that it is getting error because it is in base64.

40
00:03:10,200 --> 00:03:17,700
So that means we will decode these message before we put into Amazon Kinesis.

41
00:03:17,700 --> 00:03:21,000
In order to do that, I'm going to use a web application.

42
00:03:21,000 --> 00:03:25,510
You can see in here we basically convert base 64 and controller.

43
00:03:25,530 --> 00:03:29,220
So let me copy this text message and come here.

44
00:03:29,250 --> 00:03:36,960
You can see the base64 encode value and after that you can paste these value as a data.

45
00:03:37,230 --> 00:03:39,180
And let me see, it is the same.

46
00:03:39,180 --> 00:03:41,280
Yes, it's totally the same data.

47
00:03:41,280 --> 00:03:48,570
So basically you can paste the data into here with providing the base64 encode value.

48
00:03:48,570 --> 00:03:54,450
And after that, if you copy and paste this command, this command will be input item to kinesis data

49
00:03:54,450 --> 00:04:01,290
streams and this stream will be pulled from the lambda and invocation happens from the lambda function.

50
00:04:01,710 --> 00:04:08,640
OC As you can see that we have successfully test kinase streams using a lambda to process kinesis streams.

51
00:04:08,700 --> 00:04:14,580
At this point, I would like to give you an assignment that you can develop with your environment here.

52
00:04:14,580 --> 00:04:17,580
You can find the article that we mentioned before.

53
00:04:17,580 --> 00:04:24,240
So basically this article explains how to perform order data replication into Amazon dynamic DB streams.

54
00:04:24,240 --> 00:04:29,520
But if you scroll down, there is three way to use Stream's panel pattern.

55
00:04:29,520 --> 00:04:35,580
You can see that you use finite patterns to process the A.B.C. stream and there is three way to handle

56
00:04:35,580 --> 00:04:36,690
this operation.

57
00:04:36,690 --> 00:04:41,580
So this time I would like to give Kinesis data stream panel pattern.

58
00:04:41,580 --> 00:04:47,460
If you scroll down and see the architecture, you will see that we will conduct a kinesis stream to

59
00:04:47,460 --> 00:04:49,170
to provide the final pattern.

60
00:04:49,260 --> 00:04:55,440
So this is an assignment that you can follow and that you can do on your environment.

61
00:04:55,440 --> 00:04:57,450
So we have finished this section.

62
00:04:57,450 --> 00:05:04,140
That means we have finished the streams, both Dynamo DB and Kinesis Stream and using to add some of

63
00:05:04,140 --> 00:05:05,010
the functions.