AWS Discovery Day: Fundamentals of a Modern Data Strategy on AWS

IT Training and Development Resources
Webinars
AWS Discovery Day: Fundamentals of a Modern Data Strategy on AWS [Webinar]

Organizations need a data strategy to succeed, but there’s no one-size fits all approach. Every organization will have unique objectives and obstacles. Watch this webinar to learn how to modernize, unify, and innovate your way to a modern data strategy with AWS.

Skills Gained

How AWS services help you modernize, unify, and innovate your data infrastructure
How to use AWS services to provide secure and well-governed access to data
How to innovate with AI/ML by harnessing your data with built-in ML

WEBVTT

1
00:00:18.510 --> 00:00:28.190
Myles Brown: Good afternoon. It's going to take a few minutes for all the attendees to come in when I first open up the webinar, so we're going to give people a minute or 2 to find their way in.

2
00:00:59.000 --> 00:01:00.140
Myles Brown: I

3
00:01:00.480 --> 00:01:11.530
Myles Brown: so it looks like people are finding the chat already, which is great the way we run these sort of webinars is that you know

4
00:01:11.530 --> 00:01:36.620
Myles Brown: we've got over 100 people registered. I don't know how many people will actually be here live, but because we want to open it up to as many people as possible. We don't run these like a regular class in regular class. We maybe have a dozen people. We encourage everybody to turn on their cameras and and and their microphones in these larger events you really just have to chat. And so, you know, let me just put in

5
00:01:36.950 --> 00:01:38.320
New York City.

6
00:01:38.360 --> 00:01:50.709
Myles Brown: That's where I'm coming from. If you want to throw in where you're coming from that could help people see where you're and in the chat you can chat to everyone so everybody can see it. There's also a way to add a question just to the panelists.

7
00:01:51.160 --> 00:02:07.110
Myles Brown: Really me? But The this is an aws discovery day aws has built these series of webinars that authorized training partners like exit certified are allowed to deliver

8
00:02:07.230 --> 00:02:31.480
Myles Brown: and there's a series of them. There's one just sort of intro to cloud. There's one on migration. There's one coming up next week on machine learning the the ones that we've already done. There's one on security. Those are up on our website so you can go and find those. I'll show you how to get to those a little later. This this discovery day will eventually be recorded. You'll get the an email with the link to it.

9
00:02:31.550 --> 00:02:41.150
Myles Brown: so you can watch it again after I'm not allowed to share the slides. Aws! Says, no, you can't share the slides, but you are allowed to record it. So there'll be a recording there.

10
00:02:41.450 --> 00:02:56.899
Myles Brown: and my name is Miles. I work for a company called Exit Certified. We've been an authorized training partner of Aws for a long time. I think 2014 is when we first started teaching classes when I first started teaching Aws classes.

11
00:02:57.240 --> 00:03:14.930
Myles Brown: But we do a lot more than just aws, we're partnered with, you know, all the major cloud vendors. So Microsoft, Google Oracle, and a lot of other vendors like Ibm and Vmware. And so where there's a vendor of record, we we partner with them and we deliver their authorized training that they've created.

12
00:03:15.030 --> 00:03:32.629
Myles Brown: we also do other, you know, sort of training on open source stuff as well. So I'll maybe talk a little bit more about what exit certified does. near the end of the session. I'll also talk a little bit about a promo code. We have a summer promotion going on where you can get up to $100 per day off of a

13
00:03:32.670 --> 00:03:42.949
Myles Brown: and a, you know, a technical training class. So we'll come back and talk about that. But that that's the company that I work for, and you'll see that logo down in the bottom right of all these slides.

14
00:03:43.580 --> 00:03:45.310
Myles Brown: And so I think.

15
00:03:45.430 --> 00:03:53.869
Myles Brown: and the number of attendees is leveling out a little bit. It's a little lower than a lot of people sign up for these

16
00:03:54.090 --> 00:03:58.160
seminars, and then they don't. You know. They just wait and watch the recording, I guess.

17
00:03:58.490 --> 00:04:11.390
Myles Brown: But you get the option of asking questions in the chat. Live so please feel free to do that whenever whenever you want. I I've got one eye on the chat. I may not see it right away, but I'll I'll get there eventually.

18
00:04:12.230 --> 00:04:26.160
Myles Brown: So this session is all about and what data means? Within an organization. And and really the idea of a modern data strategy which is often driven by the cloud.

19
00:04:26.160 --> 00:04:42.790
Myles Brown: And so Aws has got a couple of slides up here up front, just talking about some stats that they've gotten from from a Forbes article and a few other places. But this idea that the next wave of reinvention will be driven by data, I think, is a sentiment that a lot of people understand

20
00:04:43.130 --> 00:05:02.649
Myles Brown: And so there was a nice article in Forbes where they talk, and and they've quoted a couple of things here where? Where you know a typical fortune, 1,000 company if they make even 10% more of their data accessible to decision makers. You know, they they figure that there can be quite an increase in in their income.

21
00:05:02.850 --> 00:05:19.609
Myles Brown: And so you know those numbers. They've got some things to back it up. But that idea is that they're sort of kind of 3 main things that the business gets out of increasing the amount of data that we give to individual practitioners within our company.

22
00:05:19.760 --> 00:05:22.990
Myles Brown: One is that operationally.

23
00:05:23.070 --> 00:05:33.219
Myles Brown: looking at how you know, there's a lot of data that you collect about your organization and how it works by, by.

24
00:05:33.270 --> 00:05:40.200
Myles Brown: you know, sharing that data around, we can certainly run our business more efficiently looking for you know how best to do things.

25
00:05:40.290 --> 00:05:52.700
Myles Brown: but probably the biggest one is this making more informed decisions. You know, every decision maker I know these days wants to be seen as making data-driven decisions. Because if you're not making a decision based on data.

26
00:05:52.850 --> 00:06:05.799
Myles Brown: Was it gut instinct, you know, like, it's, it's hard to justify your decisions. If you can't point to some data and so getting all the data you need to be able to make those decisions is important.

27
00:06:06.450 --> 00:06:22.979
Myles Brown: And then, once we've collected a lot of data, now, we're seeing that making decisions going forward. You know, we've got machine learning and AI to really help with that. So we can unlock some opportunities that used to be too difficult or even impossible by automating these things with AI and Ml.

28
00:06:24.210 --> 00:06:25.990
Myles Brown: now,

29
00:06:26.930 --> 00:06:37.049
Myles Brown: you know, to be a data-driven decision. Data has to be an organizational asset. It can't be just. You know, this area of the company owns its own data and nobody else gets to see it.

30
00:06:37.080 --> 00:06:44.660
Myles Brown: The data has to be accessible by anybody who's allowed to look at it. And we need to put it to work, which really means

31
00:06:44.710 --> 00:06:46.800
Myles Brown: doing some

32
00:06:46.810 --> 00:06:50.180
Myles Brown: engineering with that data and maybe doing machine learning.

33
00:06:50.770 --> 00:07:04.230
Myles Brown: There's a lot of challenges that go into it. Because first off, we have more data than ever before. I think about my first job out of college in the mid 90 s. I worked as a database developer.

34
00:07:04.610 --> 00:07:10.210
Myles Brown: Every piece of information in our database was typed by a human at some point.

35
00:07:10.250 --> 00:07:17.379
Myles Brown: That's the way it used to be. Now, there's so much machine-generated code that that's just the explosion of data.

36
00:07:18.530 --> 00:07:37.689
Myles Brown: And so, you know, we've got all kinds of data. It might be right now, stored in different silos. So that can be a problem. we want to adapt concepts of of adopt ideas of machine learning, but that is kind of a skill that you know is eludes a lot of individual developers.

37
00:07:38.020 --> 00:07:56.850
Myles Brown: but also, increasingly, this data that we have is being governed by all kinds of, you know. maybe compliance organizations. Maybe the government themselves have rules on what you can and can't do with it. And so just holding that data becomes something that you have to really, you know, manage and govern.

38
00:07:58.780 --> 00:08:19.580
Myles Brown: There's also things we learned even from the pandemic over the last few years to survive. We have to make better and faster decisions. Data isn't critical to that, to thrive. We want to be able to put our data to work, you know, figure out new customer experiences. So some of the use cases that we use this data for I mentioned one of them is just operationally within my company.

39
00:08:19.620 --> 00:08:21.310
Myles Brown: right?

40
00:08:21.700 --> 00:08:32.809
Myles Brown: trying to automate processes and optimize those you a lot of times. You have to gather a lot of metrics to see, you know, where are the bottlenecks in this process so that we can figure out how to get around them

41
00:08:32.929 --> 00:08:38.990
Myles Brown: things like supply chain optimization detecting fraud right? There's a lot of that sort of operational stuff that we can do.

42
00:08:39.250 --> 00:08:52.789
Myles Brown: Then there's more on the marketing side, you know. A lot of times you hear about improved targeting to, you know, ads to the right people. Churn analysis getting a 360 degree view of your customer.

43
00:08:52.810 --> 00:09:07.909
Myles Brown: Right? Figuring out. Well, not just what have they bought from us? But what if they said in chats to our customer service representatives? What if they tweeted about us, you know, get all that information about that customer so that we can, you know, really understand them better.

44
00:09:08.300 --> 00:09:20.169
Myles Brown: And then maybe, you know, when they go to use our website. Let's try and improve that customer experience. So personalize it. Maybe if they do contact somebody in our contact center

45
00:09:20.450 --> 00:09:26.589
Myles Brown: that customer service rep has a history of everything they've told us right there at their fingertips, right?

46
00:09:27.080 --> 00:09:30.410
Myles Brown: So there's a lot of things we can do on that customer experience side.

47
00:09:30.560 --> 00:09:41.679
Myles Brown: And then, finally, you know, if you're company that develops software, you know, let's build some that incorporates all this data. And what any an analysis of that data.

48
00:09:42.470 --> 00:09:52.809
Myles Brown: Now, one of the big problems we have is that some of our traditional approaches to data warehousing. If if I think back to my first job back in the 90 S.

49
00:09:52.850 --> 00:10:01.710
Myles Brown: You know. My my company was at a crossroads. They were moving from a mainframe to more client server, and we brought in oracle databases.

50
00:10:01.970 --> 00:10:13.729
Myles Brown: But that was mostly for the transactional thing. And then we decided, let's go make a data warehouse, right? So we had all these different transactional databases, and we were grabbing data from those. Bring them into

51
00:10:13.840 --> 00:10:24.070
Myles Brown: a central data warehouse, and then we would attach business intelligence tools to, you know. Go and make nice dashboards and reports and things.

52
00:10:24.180 --> 00:10:40.520
Myles Brown: Well, the problem is, you know, if you get a large enough organization, you're not going to have just one data warehouse. You can have a few, you know, each group has their own data. So you end up with those silos. And so the big thing, probably about 10 years ago that we started to see was the concept of data lakes

53
00:10:41.330 --> 00:10:47.399
Myles Brown: where we have, you know, a central place where we can drop all of our data in its natural format.

54
00:10:47.440 --> 00:11:11.120
Myles Brown: So if you've not seen this term before, and I imagine most people have heard of it, you know that analogy is, you know, a data lake holds all the data in its natural state. It might be a little dirty, right? versus a data warehouse which is more like kind of like bottled water. Where you take some water, you clean it up, you package it up for a particular kind of use.

55
00:11:11.350 --> 00:11:21.880
Myles Brown: but it doesn't mean that the lake isn't still very valuable, right? By not processing it. I have that data in its natural state. I haven't lost any context yet.

56
00:11:22.320 --> 00:11:29.459
Myles Brown: So there's, you know, different versions of of where data can live to give us different options.

57
00:11:29.670 --> 00:11:49.299
Myles Brown: And there's not just those, you know. If you look at any kind of modern cloud based application. You've not just got, you know, relational databases, right? You've got all kinds of node sequel databases. And so here's here's a a typical kind of web page where you know, we might have the, you know

58
00:11:50.060 --> 00:12:07.230
Myles Brown: the the basic, you know, shopping card, or whatever that might be a relational database. But then we've got like just a list of all our products. you know, and there's not a lot of movement in that. You add new products. Once in a while you might change the price of a product or whatever. So you might use something like a key value store for that

59
00:12:07.480 --> 00:12:19.249
Myles Brown: When it comes to the customer views, you might put those in some sort of a oh, no! This one's the frequent accessories your friends brought together. That might be something that you put in a graph database

60
00:12:19.470 --> 00:12:25.249
Myles Brown: and then up the top. Here, you've got some sort of data in in a nice format for search.

61
00:12:25.260 --> 00:12:37.259
Myles Brown: So we have all kinds of in aws what we call purpose-built databases, which are nice managed options for doing things like graph databases, key value databases search data.

62
00:12:37.590 --> 00:12:38.850
Myles Brown: And so.

63
00:12:38.950 --> 00:12:46.730
Myles Brown: you know, aws, has kind of put forward this idea that a modern data strategy doesn't ignore any of these things.

64
00:12:46.810 --> 00:13:06.419
Myles Brown: Yes, you've got a data lake, maybe, at the heart of it. But you've also got your data warehouse. You've got all kinds of relational databases. You have non-relational databases you have, you know, log analytic stuff. You've got big data processing. You got machine learning. All of these things are part of your modern data strategy.

65
00:13:07.350 --> 00:13:10.389
Myles Brown: And so we want to be able to hold data at any scale.

66
00:13:10.610 --> 00:13:18.450
Myles Brown: we want to pick the options that are going to give us the best. price to performance ratio and depending what we're doing.

67
00:13:18.720 --> 00:13:32.640
Myles Brown: And ideally, no matter where this data lives, we should be able to get seamless data access to it. And you know, through this idea of the data catalog and governance around this data.

68
00:13:32.650 --> 00:13:52.439
Myles Brown: I think we'd like a unified governance. So where we say something like, okay, any kind of personally identifiable information. Maybe if you've got social security numbers, you know, those need to be obfuscated wherever we store them. And so we need that across the board, no matter what kind of data store this data lives in.

69
00:13:53.430 --> 00:14:00.609
Myles Brown: And then we also want to add in AI and Ml, you know, to to solve those business challenges going forward.

70
00:14:01.480 --> 00:14:16.249
Myles Brown: So this diagram is just sort of, you know, descriptive relational databases, non-relational databases. If we go in and add some names to it. You know, if you're just looking for a relational database a lot of times these days, we're talking about Amazon Aurora.

71
00:14:16.450 --> 00:14:22.340
Myles Brown: one of the most popular. you know. No SQL databases in the Aws world is dynamo. dB,

72
00:14:22.560 --> 00:14:27.810
Myles Brown: when it comes to machine learning, we've got sage maker. Redshift is our data warehouse.

73
00:14:28.050 --> 00:14:45.279
Myles Brown: When it comes to search data, we've got open search service, which you probably heard of elasticsearch that that's what that is built on. There was a bit of a fight between the open source source, you know, kind of group that was doing elastic search and the company elastic. They were trying to

74
00:14:46.060 --> 00:14:55.469
Myles Brown: really clamp on too much and hold too much about that open source. So aws spearheaded this idea to split off open search from elastic.

75
00:14:56.390 --> 00:15:03.340
Myles Brown: And then, Emr, that is our hadoop. And and you know, spark kind of environment.

76
00:15:04.400 --> 00:15:05.660
Myles Brown: So

77
00:15:05.970 --> 00:15:30.779
Myles Brown: you know what we're really looking for in a modern data, strategy is, you know, kind of threefold. So we want to modernize our existing data infrastructure. So if you've got, you know, a bunch of oracle databases running on Ec, 2 instances, we we can get a lot of nice things out of modernizing that and saying, let's take some of the basic Dba duties off of our shoulders and let Amazon do it.

78
00:15:30.780 --> 00:15:43.670
Myles Brown: So modernize moving to more scalable, trusted, secure cloud provider options innovate, which is where we build new experiences and reimagine old processes using AI and Ml.

79
00:15:43.960 --> 00:15:53.259
Myles Brown: and then this idea of unifying, you know, let's let's look at the best of both data lakes and those purpose-built data stores like like our data warehouse.

80
00:15:53.450 --> 00:15:56.649
Myles Brown: And let's put those all together. So let's start with the modernized part.

81
00:15:57.900 --> 00:16:12.930
Myles Brown: there's There's a few things that customers are trying to do, and they've been doing this for probably about 6 or 7 years in aws, this has been a a big trend that I see. You know, I see a lot of students over the years

82
00:16:12.980 --> 00:16:16.660
and talk to a lot of companies that are making a cloud transition.

83
00:16:17.370 --> 00:16:20.979
Myles Brown: and a lot of them are trying to break free from legacy databases.

84
00:16:21.580 --> 00:16:39.289
Myles Brown: and a lot of this was sort of spearheaded by Amazon themselves. Right? They a long time ago came out with the relational database service or Amazon Rds. And the idea was, you're already letting your cloud vendor take care of the physical

85
00:16:39.440 --> 00:16:50.459
Myles Brown: data warehouse. You know the the the sort of data center, right? So you know, Power and Hvac and and racks and servers and physical maintenance of them.

86
00:16:50.690 --> 00:17:04.909
Myles Brown: Why don't we just let the cloud vendor do a little bit more. They can not only install the operating system, but patch it over time. They can install my oracle, database and patch it over time. I can set it up to take nightly backups for me.

87
00:17:04.910 --> 00:17:20.929
Myles Brown: I can set it up so that that one oracle database isn't a single point of failure by launching a second one and setting up synchronous replication between it right? So so that idea of of of helping manage the database moving to fully managed databases

88
00:17:21.060 --> 00:17:25.170
Myles Brown: is it's been an interesting one for a while. Then

89
00:17:25.280 --> 00:17:39.170
Myles Brown: Amazon looked and said, Well, you know they they picked the 2 most popular commercial databases. So that's Oracle and Microsoft, SQL. Server. And then they did the same for the 2 most popular open source databases Mysql and postgres.

90
00:17:39.410 --> 00:17:41.619
Myles Brown: and once they started looking at it, they said.

91
00:17:42.680 --> 00:17:54.320
Myles Brown: you know, with something like oracle, you could either bring your own license if you already had an oracle database license, or you could buy one, you know, so that you just pay your monthly bill to Amazon, and then they turn around and pay oracle.

92
00:17:54.570 --> 00:18:00.469
Myles Brown: So they really start to get an insight into what people are paying for oracle and SQL. Server. And it's a lot.

93
00:18:00.860 --> 00:18:02.350
And so

94
00:18:02.370 --> 00:18:07.629
Myles Brown: what Amazon said was, maybe we can build a cloud power database

95
00:18:08.260 --> 00:18:12.870
Myles Brown: that that is maybe a tenth the cost of those.

96
00:18:12.900 --> 00:18:17.180
Myles Brown: but still gives you the same sort of throughput and scale.

97
00:18:17.460 --> 00:18:18.190
Myles Brown: And

98
00:18:18.500 --> 00:18:25.209
Myles Brown: and so that's where Aurora came in. And so that's sort of the idea we want to break free from these legacy database

99
00:18:25.230 --> 00:18:28.080
Myles Brown: providers, and we might get some nice benefits out of it.

100
00:18:28.160 --> 00:18:34.279
Myles Brown: We're going to fully go to fully manage database services. We're going to do the same thing for our data warehouse.

101
00:18:34.480 --> 00:18:48.819
Myles Brown: and, you know, start building our applications with modern sort of purpose-built databases. So let's start with that first. The Old Old Guard databases. They're very expensive. They're very proprietary. Now you end up with vendor lock in

102
00:18:49.780 --> 00:19:01.050
Myles Brown: oracle, for example, has really started some punitive licensing where, if you're going to run oracle in the cloud unless you're running it in Oracle's Cloud, it's going to cost you quite a bit.

103
00:19:01.250 --> 00:19:12.039
Myles Brown: Okay, and so this is sort of the idea. And and you're still stuck with these database providers who say, Okay, well, that version is no longer supported. You gotta move to a new version and a more, you know.

104
00:19:12.070 --> 00:19:17.920
Myles Brown: So we're seeing a lot of people move away from that and go to like open source databases.

105
00:19:18.360 --> 00:19:38.170
Myles Brown: But the problem with the open source databases is, it takes a bit of expertise to make them run, you know, with the same scale and throughput as those commercial vendors. So what we want is, you know, I wanted to to look and feel like one of these open source, but have that commercial, great performance, and that's exactly what Aurora does.

106
00:19:38.180 --> 00:19:47.750
Myles Brown: So when when Amazon built Aurora. they decided, you know. database administrators. They've seen this

107
00:19:47.780 --> 00:19:58.850
Myles Brown: story before. Oh, there's a brand new database. Great. Well, now, I'm just going to use that I'm going to be beholden to Amazon instead of oracle, and at their whim, when they decide to change things

108
00:19:58.980 --> 00:20:06.130
Myles Brown: and and pay whatever price they want. And so when Amazon built Aurora, they said, We don't want you to force into a vendor lock in.

109
00:20:06.420 --> 00:20:11.620
Myles Brown: so what we'll do is we'll make Aurora look like either Mysql or Postgres.

110
00:20:11.710 --> 00:20:25.640
Myles Brown: What does it mean to look like one of those? It means that for all intents and purposes, the code that you write that talks to the database, thinks it's talking to Mysql. So it's using the Mysql drivers. It's using the Mysql SQL. Dialect.

111
00:20:25.810 --> 00:20:32.139
Myles Brown: So just because I'm running an or aurora. I get that scale and throughput and everything I like about it.

112
00:20:32.760 --> 00:20:59.099
Myles Brown: But if I decided to move this app outside of Amazon to another cloud, or even to my own private cloud, or something. I could go and launch a bunch of my SQL. Databases and and do the back end. Dba work to make them scale the way I need to. And so that's sort of a you know, the big benefit of Aurora. I get the nice performance and scalability much faster than if I just use the open source options.

113
00:20:59.540 --> 00:21:12.159
Myles Brown: I get the high availability and durability. It's very fault tolerant if the database dies, there's a failover right away, it's highly secure. It's fully managed. I get all that. But I don't really get the vendor lock in. Part.

114
00:21:12.650 --> 00:21:19.910
Myles Brown: and so as soon as Aurora came out, he became wildly popular.

115
00:21:20.190 --> 00:21:22.600
Myles Brown: Database is migrating to Aurora.

116
00:21:23.750 --> 00:21:40.090
Myles Brown: and it's not just a roar, if you look, it's you know, not just relational databases if I'm using. I don't know Mongodb or Cassandra, or, you know, some sort of no SQL database that I have to manage myself and deal with. Maybe licensing. A lot of them are open source.

117
00:21:40.210 --> 00:21:44.390
Myles Brown: you know. It turns out across the stack.

118
00:21:44.460 --> 00:22:05.040
Myles Brown: If it comes to databases like Oracle, or you know, Microsoft, SQL. Server, or something. We'll go to Aurora or Rds, maybe. when it comes to Mongodb and Cassandra and Redis, and then some of these different, no sequel databases. Well, we've got dynamo. dB, we've got a Amazon document. dB, which has Mongodb compatibility

119
00:22:05.080 --> 00:22:08.810
Myles Brown: at key spaces which looks a lot. Looks like Cassandra.

120
00:22:09.000 --> 00:22:19.999
Myles Brown: We got a last of cash, which is sort of an in-memory, you know. Looks like either Redis or Memcache d redshift in lieu of, you know, a traditional

121
00:22:20.100 --> 00:22:30.939
Myles Brown: data warehouse appliance, like oracles used to be called exadata. I think it became called data warehouse. Ibm Nitiza, you know green plum. You know all those kind of

122
00:22:30.970 --> 00:22:32.400
Myles Brown: database vendors

123
00:22:32.590 --> 00:22:45.750
Myles Brown: when it comes to Hudoop we've got emr when it comes to elastic search, and things like that. The elk stack you sometimes here called elastic search log, stash and cabana.

124
00:22:45.870 --> 00:22:47.500
Myles Brown: You know we got open search.

125
00:22:47.790 --> 00:23:00.910
Myles Brown: Msk is managed streams for Kafka. and then sage maker lets us use, you know, a whole bunch of different machine learning frameworks like pi towards your Mx net.

126
00:23:01.700 --> 00:23:03.410
Myles Brown: Tensorflow is big one.

127
00:23:03.780 --> 00:23:17.490
Myles Brown: And so that's the big idea of, you know, modernizing to these sort of managed services. We talked about managed services. If I was to run, say, an oracle database in my own data center on-prem.

128
00:23:17.720 --> 00:23:21.429
Myles Brown: You know, this is all the things I have to think about. Right?

129
00:23:21.790 --> 00:23:36.360
Myles Brown: you can imagine the low-level stuff, you know, the hardware maintenance and and of that just running on a virtual machine in the cloud. Amazon does some of that work. But if I move to a fully managed service.

130
00:23:36.360 --> 00:24:03.689
Myles Brown: Basically, they do everything up to, you know, maybe even providing some machine learning through SQL and Apis. Then I'm really just thinking about my data. And you know what's the structure? If I'm using a relational database, I think, what are my tables? What are the columns called? Where the indexes, where it's the partitioning, you know, and beyond that I'm not the Dba a brain. This algorithm in Amazon is. So they're taking care of all the rest of that stack.

131
00:24:04.590 --> 00:24:11.969
Myles Brown: Now, that might sound scary to some people, they said, well, what if I don't like the way that brainless algorithm does the administration?

132
00:24:12.390 --> 00:24:15.810
Myles Brown: Well, you're not stuck using it. You can still use.

133
00:24:15.820 --> 00:24:20.700
Myles Brown: you know, what we some people call infrastructure as a service versus platform as a service.

134
00:24:20.770 --> 00:24:31.370
Myles Brown: That's the beauty of the cloud. There's a lot of options, so you can run things the way you want. But a lot of organizations these days are looking going. hey? I I never enjoyed paying.

135
00:24:31.410 --> 00:24:39.919
Myles Brown: you know a bunch of oracle, Dbas, just to keep an oracle database running, which is the same work I have to do. Your company has to do. Every company has to do

136
00:24:40.270 --> 00:24:53.740
Myles Brown: at at at Aws, the chief technical officer. This guy named Werner vogels. He coined this term. He called it undifferentiated, heavy lifting. It's the same work I have to do. You have to do just to keep this thing going

137
00:24:55.630 --> 00:24:56.390
Myles Brown: to that

138
00:24:58.960 --> 00:25:01.160
Myles Brown: and how to stump that. Okay.

139
00:25:01.320 --> 00:25:05.509
Myles Brown: couple of things left here.

140
00:25:05.770 --> 00:25:22.729
Myles Brown: you know. Aws, obviously is, you know, the focus of this. there's lots of cloud providers there. They're all started getting close to the same set of services over the years. You know, somebody innovates, comes up with something new. A couple of years later all the other cloud painters have it.

141
00:25:22.850 --> 00:25:28.940
Myles Brown: but what we find is that A. Ws. Is is definitely the most comprehensive and broadly adopted cloud platform

142
00:25:29.250 --> 00:25:34.480
Myles Brown: and largest customer base. probably the easiest to get somebody who has

143
00:25:34.520 --> 00:25:36.309
Myles Brown: experience on it.

144
00:25:36.780 --> 00:25:43.720
Myles Brown: But the big thing to understand about building your data stack is that it's not one size fits all.

145
00:25:43.830 --> 00:25:54.609
Myles Brown: We don't just use a relational database for everything anymore. Modern applications require more performance, scale availability. And so you know

146
00:25:55.390 --> 00:25:58.430
Myles Brown: where you used to. Maybe only have

147
00:25:58.970 --> 00:26:03.850
Myles Brown: you know the customers inside your company now you could have millions of users for apps.

148
00:26:03.990 --> 00:26:08.060
and and your data volume might be in the Petabytes. Even exabytes over time.

149
00:26:08.250 --> 00:26:17.490
Myles Brown: and you know more and more often a lot of these apps require, you know, microsecond latency. Forget millisecond, you know. So

150
00:26:17.740 --> 00:26:27.559
Myles Brown: what we sometimes call web scale applications. And so for that reason, we have a lot of different purpose-built data services

151
00:26:28.510 --> 00:26:35.909
Myles Brown: at some very low level. You got the idea of a data lake mostly just uses our object storage. S, 3, right?

152
00:26:35.970 --> 00:26:48.110
Myles Brown: and then we've got a service on top of that called lake formation. That helps us actually set up the data lake, you know, in in in minutes or and hours, rather than weeks, you know.

153
00:26:48.140 --> 00:26:58.850
Myles Brown: And then we've got glue which helps us with Etl. And and in building a data catalog things like that, then we've got all those databases we just mentioned.

154
00:26:59.010 --> 00:27:04.479
Myles Brown: then we've got the analytics side. So so if you want to do a dupe

155
00:27:04.510 --> 00:27:13.279
Myles Brown: or search stuff or messaging, you know, we've got a bunch of specific services for that. Then on top of that. We've got all these business intelligence

156
00:27:13.540 --> 00:27:33.949
Myles Brown: quicksite, you know. That's just the Aws services. Then there's many, many third-party companies doing the same thing. you know. So if you like tableau, you can use tableau. If you want more of a managed service, you could go to aws and use quicksite, you know. It looks at some level, a lot like tableau. Maybe it doesn't have all the bells and whistles.

157
00:27:34.000 --> 00:27:41.450
Myles Brown: but it might have what you need. And so, you know, we've got a lot of stuff when we talk about machine learning.

158
00:27:41.470 --> 00:27:49.460
Myles Brown: you know, here they mentioned, there's some really high level machine learning stuff. Things like.

159
00:27:50.270 --> 00:28:07.080
Myles Brown: That's a good example. Transcribe, comprehend, translate. You know, there's a lot of these machine learning services that were just something that Amazon, you know, had built and trained some models internally, and been using for years, and then they turn around and said, Oh.

160
00:28:07.170 --> 00:28:16.659
Myles Brown: I bet our customers would like this right? So all the components that make up the Alexa where you say, hey, Alexa, you know, do something. It's got to figure out, you know.

161
00:28:17.270 --> 00:28:34.459
Myles Brown: speech to text, and then do some sort of, you know, understanding natural language processing on that text. Figure out what they're looking for, and then figure out a response and then turn that back into text into speech. Right? So all those elements are individual services that we can use.

162
00:28:34.650 --> 00:28:46.659
Myles Brown: and we can use them without really knowing much about machine learning at all right. I could just have an S. 3 bucket load thousands of images into it, and then say, I'm going to use Amazon recognition to go in

163
00:28:46.950 --> 00:28:55.129
Myles Brown: index all those images, and then come back and say, Show me all the images that are of a woman wearing sunglasses.

164
00:28:55.150 --> 00:28:58.179
Myles Brown: and I want you to be 95% sure

165
00:28:58.530 --> 00:29:04.629
Myles Brown: and says, Okay, here's the ones I think are a woman wearing sunglasses. I don't need to know machine learning to do that.

166
00:29:05.160 --> 00:29:13.000
Myles Brown: But if I am a machine learning practitioner and I have very specific things I want to build using using

167
00:29:14.520 --> 00:29:19.449
Myles Brown: maybe tensorflow or something like that. Well, then, I've got Sage Maker to help me with that.

168
00:29:20.410 --> 00:29:27.430
Myles Brown: So we're going to come back to that innovation side of the machine learning. But let's talk a little bit about the the unifying part.

169
00:29:27.640 --> 00:29:44.110
Myles Brown: We want to put our work our data to work. So we need secure and well-governed access to our data, using both data lakes and those purpose-built data stores. So let's come back to that concept of a data lake. We said, it's a place where you just store all your data

170
00:29:44.150 --> 00:29:45.990
Myles Brown: in its natural format.

171
00:29:46.220 --> 00:29:51.809
Myles Brown: So what do I need? I need some sort of storage that can hold any kind of file

172
00:29:53.180 --> 00:29:58.909
Myles Brown: and ideally not lose it and be a place to store things.

173
00:29:59.150 --> 00:30:10.440
Myles Brown: And so, you know, in the Aws world the most obvious candidate for where to store a data lake is Amazon, s. 3, because it's cheap storage. That's very durable.

174
00:30:10.560 --> 00:30:12.369
Myles Brown: And it can hold any kind of data.

175
00:30:13.240 --> 00:30:19.800
Myles Brown: Now, if you think you're just going to make one s. 3 bucket and say, there we go. I've got my data. Link. Just start dropping files in.

176
00:30:19.960 --> 00:30:23.330
Myles Brown: You know. You can get in trouble and hurry. Because.

177
00:30:23.850 --> 00:30:46.999
Myles Brown: you know, sometimes we call that the data swamp. Right? You just have unregulated data. Anybody can drop data in there. You don't know where it came from. You don't know who's allowed to look at it. There's no way to search it right? So a lot of times to be really a a data lake, you might need not just storage. But you need some sort of a data catalog to keep track of that data where it came from. Who's allowed to access it. Things like that.

178
00:30:47.810 --> 00:30:55.529
Myles Brown: But S. 3 is the ideal place for storage, because it's very highly durable. We like to say 11 nines of durability.

179
00:30:56.000 --> 00:31:06.230
Myles Brown: What that means is, if I upload a file to S. 3 today, the chances of Amazon losing it in the next year are point (000) 000-0001.

180
00:31:07.320 --> 00:31:22.539
Myles Brown: How do they come up with those numbers? It's hard to say. But you know, the idea is, when you upload a file to S. 3, it's going to copy it to 3 different physical locations that are geographically separated across multiple Azs or availability zones.

181
00:31:22.740 --> 00:31:37.369
Myles Brown: Each availability zone is a cluster of data centers. So when aws builds infrastructure, they build a cluster of data centers and then, miles away, they build another cluster, and then, miles away, they build another cluster, and those 3 A. Z's together make up a region.

182
00:31:37.380 --> 00:31:39.009
Myles Brown: Some of the regions have more.

183
00:31:39.370 --> 00:31:56.489
Myles Brown: then 3. But that's the big idea. And so, you know, they've built these A Z's close enough together that they can, you know, replicate that data pretty quickly between them. But far enough apart that if there's some local disaster that takes out this cluster of data centers, it shouldn't affect this one over here.

184
00:31:57.030 --> 00:32:09.929
Myles Brown: So very good durability and availability scalability. Well, when I create an S. 3 bucket, that's what the top level namespace is called. I don't tell it how big to make it. It can hold an unlimited amount of objects.

185
00:32:10.220 --> 00:32:12.199
Myles Brown: and there's no size restriction.

186
00:32:12.290 --> 00:32:31.879
Myles Brown: Now. There are some physical size restrictions, but individual companies really don't run into them, you know. because Amazon is just growing all the time right there. They're always adding more capacity to existing data centers having more data centers, adding more availability zones. Everything's growing non-stop.

187
00:32:33.060 --> 00:32:48.389
Myles Brown: there's something called intelligent tearing, because we actually have within S 3 different tearing. If you say, well, this data, I might need access to it quickly. So I'm going to pay a little bit more than if I stored in more of an archival form where I say

188
00:32:48.460 --> 00:32:56.960
Myles Brown: this data. I'm not going to access it very often. so save money on storage. I'll pay a little bit more every time you access it.

189
00:32:57.020 --> 00:33:07.629
Myles Brown: So we have sort of infrequent access. We have the the extreme of that is something called Glacier, which you can imagine. The name indicates. It's cold storage, very slow, moving.

190
00:33:08.180 --> 00:33:19.479
Myles Brown: but intelligenteering says, well, if I don't know what the usage patterns are, I'll throw it in there. It'll start in standard, and if you don't use it, it'll go to infrequent access, and then, if I do use it, it'll move it back for me.

191
00:33:20.420 --> 00:33:41.840
Myles Brown: And with S. 3, you know, because it's not regular block level storage attached to some virtual machine. It's just Internet storage. That means the way I access it is through Http or Https gets inputs very simple Api. Anybody with the right permissions and put data into S 3, or get data out. And

192
00:33:42.020 --> 00:33:49.690
Myles Brown: it doesn't matter what the the nature of the data is to me it's just some sort of binary object.

193
00:33:51.180 --> 00:33:58.229
Myles Brown: So that's some of the things that make it. And it's very cheap, you know. It starts at about 2.3 cents per gig per month

194
00:33:58.280 --> 00:34:02.410
Myles Brown: for standard storage, and then it goes down. The more you store. There's some, you know.

195
00:34:02.570 --> 00:34:05.340
Myles Brown: Tiered pricing.

196
00:34:06.290 --> 00:34:21.340
Myles Brown: So a data lake is very often the root of a modern application. And so some people will start like inside out. They'll start by just dropping all the data into a data lake and then pull it from there.

197
00:34:21.400 --> 00:34:32.449
Myles Brown: And, you know, pull a subset of that data, put it into our data warehouse because I know that there's people with bi tools that are looking for a subset of that data to do really, really fast queries all the time.

198
00:34:33.500 --> 00:34:47.089
Myles Brown: the other option is, you know, naturally, your applications are generating data in different relational and non-relational databases. And I'm pulling that data from the outside into

199
00:34:47.130 --> 00:34:48.319
Myles Brown: the data link.

200
00:34:48.639 --> 00:34:54.429
Myles Brown: And then there's sort of like around the perimeter where you know, the data can live anywhere.

201
00:34:54.469 --> 00:34:55.909
And so

202
00:34:56.330 --> 00:34:59.650
Myles Brown: the

203
00:34:59.750 --> 00:35:09.499
Myles Brown: the terminology that you see use now, and some people will put it as one word. Sometimes it's written as 2 words. Here is this kind of concept of a Lakehouse architecture.

204
00:35:10.350 --> 00:35:18.129
Myles Brown: where we say, I don't really care whether my data lives in the data lake or in the data warehouse, or even one of these other.

205
00:35:18.210 --> 00:35:20.540
Myles Brown: you know, purpose-built databases.

206
00:35:20.970 --> 00:35:27.189
Myles Brown: I'm going to have the same unified data access and unified governance

207
00:35:27.270 --> 00:35:36.870
Myles Brown: to that data, no matter where it lives. And so as long as I have a way to query that data wherever it lives. So that might be through Amazon. Athena.

208
00:35:37.730 --> 00:35:39.800
you talk a little bit about that coming up.

209
00:35:39.950 --> 00:35:50.210
Myles Brown: then I'm not so worried about where the data lives and where it lives depends on really that price to performance, ratio

210
00:35:50.360 --> 00:35:53.460
Myles Brown: now,

211
00:35:53.770 --> 00:36:07.590
Myles Brown: we've sort of gone over the purpose-built data services. But we didn't talk too much about Athena or Emr. So it's probably worth talking about those now. So the the big idea with Athena is

212
00:36:08.730 --> 00:36:13.129
Myles Brown: just this way of querying data

213
00:36:13.730 --> 00:36:26.009
Myles Brown: wherever it lives, whether it's well-structured files in s. 3, or if it's in a relational database or in a no SQL. Database. I want to have this sort of SQL. Front end to get that data, no matter where it lives.

214
00:36:26.890 --> 00:36:38.249
Myles Brown: Now, it's a bit of a different trade-off. If it's just data sitting in S. 3, you know, it's probably not going to query as fast as if it's in a well-structured relational database, especially in redshift.

215
00:36:38.350 --> 00:36:45.089
Myles Brown: You know where, where they've got all kinds of sort of acceleration for how we access the data.

216
00:36:45.450 --> 00:36:52.850
Myles Brown: and with Athena, what you're doing is you're paying every time you run a query, you pay for how much data gets scanned.

217
00:36:53.570 --> 00:36:59.440
Myles Brown: But you're not paying for anything when the data is just sitting there. You're paying for storage of the data. But you're not paying for

218
00:36:59.610 --> 00:37:04.939
Myles Brown: query engine when we're not using it. So it's really good for interactive type. Query.

219
00:37:05.740 --> 00:37:13.719
Myles Brown: there's also something in redshift the data warehouse, which is a little bit like Athena running inside redshift called redshift spectrum.

220
00:37:13.990 --> 00:37:18.349
Myles Brown: which says you can come in and use your regular redshift querying

221
00:37:18.420 --> 00:37:28.900
Myles Brown: to query tables in redshift or external tables. Maybe you know well structured files at S. 3, and for those part of them you're going to pay per query.

222
00:37:29.060 --> 00:37:32.179
Myles Brown: You know how many gigs get scanned.

223
00:37:32.810 --> 00:37:52.739
Myles Brown: So it's a very similar to Athena. So that's the 2 ways that you see people do that unified access to that Lake House architecture. It's either using Athena, or if there are more redshifts centric, then everything goes to redshift spectrum to query whether the data is in redshift or somewhere else.

224
00:37:53.990 --> 00:38:05.930
Myles Brown: The other ones we mentioned Emr Emer originally stood for elastic mapreduce mapreduce you was the original data processing framework inside hadoop.

225
00:38:06.200 --> 00:38:15.040
Myles Brown: But that's been supplanted by things like spark and flink and all kinds of stuff. But Emr just means, hey, I need a cluster of machines

226
00:38:15.080 --> 00:38:20.410
Myles Brown: that are going to do some sort of hadoop processing. and no matter what framework we use on top of it.

227
00:38:20.890 --> 00:38:26.469
Myles Brown: Open search, we said was, for you know, holding data in a way that it's optimized for search.

228
00:38:26.740 --> 00:38:49.269
Myles Brown: So you can imagine when you go to amazon.com. And you say, Hey, I want to find golf clubs or something, you know, down the side you have that sort of faceted search where it says, Oh, surely only things that have, you know, two-star reviews, three-star reviews, four-star reviews. You also have a you know, different price. between this price and that price, or between this price and that price.

229
00:38:49.490 --> 00:38:56.310
Myles Brown: And when you click one of those, it quickly sifts through that list of thousands of results and and whittles it down quickly.

230
00:38:56.550 --> 00:39:07.060
Myles Brown: That's what we call faceted search. And if you search the the data in the right format to begin with, you can make it very easy to then filter it by those different facets.

231
00:39:07.250 --> 00:39:10.969
Myles Brown: That's the kind of thing that open search service really helps with

232
00:39:12.260 --> 00:39:17.899
Myles Brown: Kinesis and Ms. K. Are both ways of dealing with streaming data.

233
00:39:18.810 --> 00:39:23.210
So a lot of times when we talk about like a data warehouse, you say, well, how did the data get in there?

234
00:39:23.560 --> 00:39:36.269
Myles Brown: It was either batch process where we had a bunch of log files sitting around it, and then nightly, we do some sort of big hoop job running spark or something to grab all that data, and you know, massage it and put it into the redshift database.

235
00:39:36.860 --> 00:39:39.830
Myles Brown: or if I've got maybe

236
00:39:39.860 --> 00:39:49.370
Myles Brown: thousands of little sensors firing every 100 ms. We have a steady stream of little bits of Json coming in. I need to process that. That's string processing.

237
00:39:49.720 --> 00:40:00.969
Myles Brown: Well, there's A. A out there called the Patchy Kafka, which is what wildly popular for doing this Amazon built their own service called kinesis. That also helps with that.

238
00:40:01.290 --> 00:40:17.090
Myles Brown: And so now we have 2 different services depending, you know, if you were kind of working before kinesis or working outside of Aws and you're using Kafka. We've got Ms K. If you're just going with Kinesis. Well, then, that makes it easier inside aws.

239
00:40:17.210 --> 00:40:25.560
Myles Brown: So we've got a few different services there. So these are some of our data services that aren't necessarily holding the data, although redshift is definitely that

240
00:40:26.180 --> 00:40:40.029
Myles Brown: redshift these days has about a a 3 time better price performance ratio than other cloud data warehouses, you know. They for a little while I would say they were getting their.

241
00:40:41.610 --> 00:40:50.789
Myles Brown: You know they were really getting beat badly by by new vendors like Snowflake. getting some weird

242
00:40:53.820 --> 00:40:55.569
Myles Brown: sun coming in on me.

243
00:40:57.860 --> 00:40:59.480
Myles Brown: It looks really weird, doesn't it?

244
00:41:00.240 --> 00:41:08.369
Myles Brown: But but they've tweaked how they do things, and they've embraced some modern ways of doing things. So it it's certainly gotten better.

245
00:41:10.760 --> 00:41:11.630
Myles Brown: Sorry I

246
00:41:14.560 --> 00:41:16.350
Myles Brown: I'm going to fix this.

247
00:41:17.760 --> 00:41:20.360
Myles Brown: There need a better angle.

248
00:41:22.140 --> 00:41:22.820
Myles Brown: Okay?

249
00:41:23.090 --> 00:41:36.629
Myles Brown: but yeah. So redshifts come a long way, and it's certainly a lot cheaper than most of the other cloud data warehouses, and certainly a lot cheaper than the traditional. You know. Physical data warehouse appliances.

250
00:41:37.060 --> 00:41:45.380
Myles Brown: Now, a little bit about that 3 time. Better price performance ratio. you know they've got some benchmarks that they've done.

251
00:41:45.690 --> 00:41:48.120
Problem with these benchmarks is, you know.

252
00:41:48.150 --> 00:41:51.939
Myles Brown: did they tune the other ones properly? You know. I I think

253
00:41:52.110 --> 00:41:56.239
Myles Brown: what you can gain from a boast like this is that

254
00:41:57.670 --> 00:42:03.679
Myles Brown: they made the changes so that they're not getting left behind by by some of the newer vendors out there.

255
00:42:04.610 --> 00:42:09.280
Myles Brown: And so you're getting at least as good performance as the other cloud data warehouses.

256
00:42:10.110 --> 00:42:23.510
Myles Brown: But the big challenge with a data warehouse like like redshift is just getting the data in there, right? Where does the data come from? You've got all kinds of data stores that you need to take that data.

257
00:42:23.700 --> 00:42:38.229
Myles Brown: you know, grab that data and massage the data, clean it up and then load it into my my data warehouse. So this is what we traditionally called Etl extract transform and load.

258
00:42:38.240 --> 00:42:41.249
Myles Brown: The big challenges are, what if my

259
00:42:41.470 --> 00:42:53.220
Myles Brown: source data is changing? What if my my target data warehouse changes? Right? What if something's not available when I go to grab it? I need to have some sort of retry logic.

260
00:42:53.350 --> 00:43:01.669
Myles Brown: and there might be some specialized Etl skills right. If I'm trying to do this in parallel across a bunch of machines, I might have to learn spark.

261
00:43:02.560 --> 00:43:05.659
Myles Brown: So there's a lot that goes into the Etl side.

262
00:43:06.840 --> 00:43:13.689
Myles Brown: And so if I want to break down these silos and get that data into one place. and I need to do the Etl

263
00:43:13.860 --> 00:43:32.030
Myles Brown: I might want to make it easy for the simple stuff, like maybe maybe do some sort of visual data preparation, so that I'm not writing code to do very simple things. Right? I'm going to have to replicate data to between different places. I may have to move data to or from the data lake.

264
00:43:32.450 --> 00:43:42.030
Myles Brown: And again, I want that federated query so that I can create the data whether it's in the data lake or in the data warehouse. So when it comes to breaking down silos.

265
00:43:42.110 --> 00:43:46.090
Myles Brown: you know, using that S 3 data lake

266
00:43:46.540 --> 00:43:58.220
Myles Brown: and and and kind of thinking it as well, we've got both places to store data. The data might be stored in both, or it might be in one or the other. I kind of want to treat it all the same.

267
00:43:59.060 --> 00:44:11.550
Myles Brown: And so these days with redshift, I can access the data that's in the managed storage. Of course, that's very easy, or I can use spectrum to go and query the well-structured data in S. 3.

268
00:44:11.770 --> 00:44:20.240
Myles Brown: By well-structured, we mean, you know, if we're going to want the same kind of data warehousing queries, we want it to be, you know, compressed.

269
00:44:20.780 --> 00:44:32.990
Myles Brown: We want it to be in a nice format that we can read quickly, maybe call them nurse storage like like something like park a format. And so redshift is really good at doing this kind of thing.

270
00:44:33.850 --> 00:44:35.820
Myles Brown: but

271
00:44:37.010 --> 00:44:45.510
Myles Brown: you know, at some point we're going to need maybe to standardize on some things so that S. 3 data lake.

272
00:44:45.580 --> 00:44:50.910
Myles Brown: it might be good if we can standardize on part a format so that it's

273
00:44:50.930 --> 00:45:00.689
Myles Brown: column or storage. It's a nice open data format, supported by, you know, Hadoop and Emr. Athena Redshift.

274
00:45:01.020 --> 00:45:07.659
Myles Brown: I can export parquet from Redshift into S. 3.

275
00:45:07.690 --> 00:45:12.499
Myles Brown: I can use SQL with red shifts unload command to export data in parking format.

276
00:45:12.610 --> 00:45:23.179
Myles Brown: And the unloaded data is automatically registered in Aws data catalog. That's the governance part. We need a central catalog where, I say.

277
00:45:23.250 --> 00:45:24.699
Myles Brown: here's a table.

278
00:45:24.790 --> 00:45:30.209
Myles Brown: Is it a table in redshift, or is it a table in s. 3 in my data lake.

279
00:45:31.230 --> 00:45:36.710
Myles Brown: but keeping track of okay, here's what the columns are called and their data types and everything. So that

280
00:45:36.760 --> 00:45:40.520
Myles Brown: you know, no matter where I am. I can query that data wherever it lives.

281
00:45:40.680 --> 00:45:48.749
Myles Brown: So that's where Redshift and Athena come in using that catalog. To grab the data wherever it lives.

282
00:45:50.550 --> 00:45:51.590
Myles Brown: All right.

283
00:45:51.950 --> 00:46:10.240
Myles Brown: Now, over time. Glue has become a few different things right? At first Glue had that data catalog, but it was more about building Etl jobs, using spark, but not having to go and manage the whole loop cluster myself. It was more of a serverless kind of Etl

284
00:46:10.850 --> 00:46:16.650
Myles Brown: over time. They've added more and more to glue. And now they've got something called glue, it elastic views.

285
00:46:17.430 --> 00:46:22.379
Myles Brown: And what this is is, it's, you know, hopefully, you've heard of this term materialized views.

286
00:46:22.500 --> 00:46:28.730
Myles Brown: you know, in a relational database. If you think way back, when you first learned about relational databases, you have tables and you have views.

287
00:46:29.170 --> 00:46:37.240
Myles Brown: and a view is basically a select statement that's like a window through which you look at the underlying data in the tables. The data lives in tables.

288
00:46:37.480 --> 00:46:55.730
Myles Brown: A view just says, Hey, I want you to grab these columns from this table and join them to these columns from this table. And now the view looks like a table like I can say, Hey, describe it. I can select star from it. But the view itself doesn't generally hold data. It's just a way of of simplifying my query

289
00:46:55.790 --> 00:47:00.890
Myles Brown: so that it looks like I'm query in one table when actually, the data is spread around different places.

290
00:47:01.370 --> 00:47:10.720
Myles Brown: Well, the concept of a materialized view is where we actually run the query and grab the data and and store it in this object.

291
00:47:11.670 --> 00:47:27.110
Myles Brown: Well, those are possible to do all over the place. Well, glue allows us to do that, so it can combine and replicate data from wherever to make these very simple tables. To make my, you know, business intelligence tools, or whatever that are querying the data much faster.

292
00:47:27.610 --> 00:47:28.999
And it's

293
00:47:29.040 --> 00:47:35.509
Myles Brown: So it's continually monitoring the source database for changes and updating that materialized view.

294
00:47:35.760 --> 00:47:42.809
Myles Brown: and it should do it all in a serverless way to handle the heavy lifting of copying and combining the data. Or I don't have to do anything.

295
00:47:43.920 --> 00:47:52.219
Myles Brown: And so you know that that glue catalog and something called lake formation help with the unified governance around the whole thing.

296
00:47:52.850 --> 00:47:55.839
Myles Brown: Lake formation is really about setting up

297
00:47:56.450 --> 00:48:06.779
Myles Brown: in creating a data like I said it could be as simple as create an S. 3 bucket. Say, there we go, but that's not great. I need to be able to keep track of who's got access to what?

298
00:48:06.920 --> 00:48:19.610
Myles Brown: And then you've got problems of like, if I make an Athena table on top of S. 3 data. You might have permissions on the table, but somebody might have also permissions to the underlying S. 3 bucket.

299
00:48:20.040 --> 00:48:30.680
Myles Brown: So trying to set things up in a way where we have sort of a consistent security across all that data. That's where lake formation really comes in

300
00:48:30.910 --> 00:48:34.410
Myles Brown: so centrally defining security governance and auditing

301
00:48:34.660 --> 00:48:39.529
Myles Brown: consistently enforcing those policies, no matter where the data lives.

302
00:48:39.690 --> 00:48:55.689
Myles Brown: and integrating with the various services to make sure that happens. And we could do table level, database level or even column novel permissions. So this is what's really making data lakes wildly popular. Obviously, aws, we've got

303
00:48:55.700 --> 00:49:03.180
Myles Brown: a lot of data lakes in there. You know, this is just a smattering of some of the kinds of companies that are doing data lakes.

304
00:49:04.380 --> 00:49:10.020
Myles Brown: not seeing a lot of questions in the chat. I don't know if I've not stopped long enough talking to

305
00:49:10.150 --> 00:49:18.820
Myles Brown: to see any, but I'm keeping an eye on there. I will break at the end. Maybe maybe you're more comfortable at the end, asking all the questions, so

306
00:49:18.960 --> 00:49:21.099
Myles Brown: we'll we'll see how we're doing then.

307
00:49:21.410 --> 00:49:36.139
Myles Brown: Well, we're talking about this 3 prongs we talked about modernized, we talk about unify. The last piece is the innovate part. And this is where we're building new experiences using AI and Ml, and this is a hot topic these days.

308
00:49:36.860 --> 00:49:39.099
Myles Brown: I mentioned that Aws has

309
00:49:39.200 --> 00:49:49.000
Myles Brown: sort of different levels of machine learning services, you know, at at the very bottom. They've got things like, you know.

310
00:49:49.110 --> 00:50:10.510
Myles Brown: I can launch an Ec 2 instance, a virtual machine that has chips that are really good for doing certain things versus other things. You know, I could have the graphical processing units. They have some that are better for doing inference, some that are doing better for training models. Right? So so at the physical level, they have, you know.

311
00:50:10.600 --> 00:50:12.720
Myles Brown: different chips and things like that.

312
00:50:13.010 --> 00:50:22.549
Myles Brown: then you've got the actual frameworks that are, you know, not really built by Amazon, but things like tensorflow and mx, net and pike torch, and all those.

313
00:50:22.830 --> 00:50:28.270
Myles Brown: Well, sage maker is the main tool that a a real heavy duty

314
00:50:28.520 --> 00:50:36.470
Myles Brown: machine learning practitioner will use, because when you look at all the steps in building a machine learning workload.

315
00:50:36.550 --> 00:50:52.239
Myles Brown: it usually starts with, well, I've collected all kinds of data. Maybe the data is sitting there in my data lake. But I may have to label right? There's certain certain machine learning algorithms that require the data to be sort of labeled in some way.

316
00:50:52.430 --> 00:50:57.029
Myles Brown: So I might have to do that. Then I have to start collecting the data from different places.

317
00:50:57.060 --> 00:50:59.880
Myles Brown: storing it, and then setting up what are the features on it.

318
00:51:00.140 --> 00:51:12.019
Myles Brown: then detecting bias and and and you know, visualizing things, maybe in notebooks, picking an algorithm from a very, very big list of algorithms, training the models with that

319
00:51:12.040 --> 00:51:27.549
Myles Brown: tuning, those parameters, deploying it into production, managing that thing in production. So there's a lot to that. And sage maker studio is a nice, integrated development environment that will help us do most of these things and to end

320
00:51:27.730 --> 00:51:32.599
Myles Brown: and and allow us to productize like, when I think about

321
00:51:33.420 --> 00:51:48.460
Myles Brown: 10 years ago, when the term data scientists started to get thrown around a lot. And you know, I worked in a place where we had a data scientist. And he was just this sort of cowboy working on his own looking at data doing experimental stuff

322
00:51:48.630 --> 00:51:54.630
Myles Brown: nowadays. You know that job of the data scientists and the results that they produce

323
00:51:54.770 --> 00:52:01.340
Myles Brown: is no longer just some little experimental thing on the side for a lot of companies that is mission critical.

324
00:52:01.550 --> 00:52:05.909
Myles Brown: And so we have to productize this. It can't be just some little experiments we're running here and there.

325
00:52:05.930 --> 00:52:14.819
Myles Brown: We have to build a repeatable kind of data engineering flow. And so stage maker studio can certainly help us with that.

326
00:52:15.260 --> 00:52:25.509
Myles Brown: And then, you know, the vast majority of developers are not machine learning experts. But we have all these top-level services. I mentioned Amazon recognition

327
00:52:25.750 --> 00:52:31.969
Myles Brown: with A with a K, not a C make it easier, I guess, to Google or something.

328
00:52:31.980 --> 00:52:45.859
so we have all these different services where you can use them without really understanding the machine learning underneath because they are things that Amazon themselves built and train and used for a long time.

329
00:52:46.690 --> 00:52:54.030
Myles Brown: So they've got some to do with speech and something to do with text. They even have some that are specific to certain industries.

330
00:52:54.450 --> 00:53:02.499
Myles Brown: So we might come back and talk about some of those. So if we look at what kinds of business problems we're using machine learning on.

331
00:53:03.310 --> 00:53:06.980
Myles Brown: We mentioned at the top of the

332
00:53:06.990 --> 00:53:19.459
Myles Brown: the discussion that you know enhancing your customer experience is a big part of it. Personalization. you know. Maybe contact lens, which is that you know contact center, keeping track of everything.

333
00:53:19.650 --> 00:53:28.650
Myles Brown: optimizing the business. So gathering those business metrics, fraud, detection, things like that.

334
00:53:28.760 --> 00:53:31.739
Myles Brown: So here's a few of the services that have to do with that

335
00:53:32.080 --> 00:53:40.219
Myles Brown: accelerating software development. Well, now, we've got Code Guru devops, Guru, and probably the one that should be in the list. It's fairly new

336
00:53:40.480 --> 00:53:42.890
Myles Brown: is Amazon code whisperer.

337
00:53:43.750 --> 00:53:56.589
Myles Brown: They don't mention it in this presentation. They built this presentation last year, and so they're missing sort of that new thing. So maybe I'll just pop up so you can see what Amazon code whisper is all about.

338
00:53:57.870 --> 00:54:00.089
Myles Brown: That's sort of the newest thing.

339
00:54:02.420 --> 00:54:03.130
Myles Brown: hey?

340
00:54:03.520 --> 00:54:05.540
Myles Brown: It's really being a pain.

341
00:54:05.790 --> 00:54:06.450
Okay.

342
00:54:13.160 --> 00:54:16.369
Myles Brown: it's Beller in good whisper. Okay, here it is.

343
00:54:17.170 --> 00:54:30.769
Myles Brown: build applications faster and more securely with your AI coding companion. And so you may have heard of things like this, like, I mean, everybody knows. Chat Gpp these days. Right you go. You sign up. You gotta create an account. But I can go in and say

344
00:54:31.320 --> 00:54:47.389
Myles Brown: prompt chat, Gpt, and say, I want you to create a python program that uploads a file called this into an S. 3 bucket called that. And you look at the code that it writes, you know, when I look at that, go. And I say, Oh, that's a pretty good code. That's what I would have written right. And so

345
00:54:47.450 --> 00:54:53.689
Myles Brown: you know, a lot of people are worried. Oh, hey, I was going to take our jobs, you know, as a developer. It's just going to make my job

346
00:54:53.820 --> 00:54:57.820
Myles Brown: a lot easier because I don't have to do a lot of that boiler prey

347
00:54:57.910 --> 00:55:10.390
Myles Brown: kind of coding that the harness around the actual stuff that I want to do in github. They have something called Code pilot. Again, that's using chat gpt under the covers, because Microsoft, you know.

348
00:55:10.580 --> 00:55:19.000
Myles Brown: But Code whisper is what Amazon calls it, and they've now integrated out all over the place. So if you go to make

349
00:55:19.020 --> 00:55:44.509
Myles Brown: a lambda function, say, I'm writing a python lambda function. I put like a little comment that says, I need a function that does Xyz, and it'll suggest some code for me. Right? So that's the big idea of code whisper. They've got plugins for all the major Ids that you can imagine like eclipse, and all the intelligence and all those ones. But also cloud 9, which is the main sort of cloud based ide that Amazon pushes.

350
00:55:44.910 --> 00:55:49.830
Myles Brown: And so so it's free for individual use. Unlimited code suggestions.

351
00:55:50.050 --> 00:56:07.120
Myles Brown: you can have it. Look at your code and do a security scan, and it'll look and say, you know, it looks like you've hard coded your aws access key and secret key, you know. That's a bad idea, you know. And so there's, you know, a few things that it's using AI to do inside there.

352
00:56:07.150 --> 00:56:11.039
Myles Brown: And so that's kind of the newest thing I would say about the

353
00:56:12.550 --> 00:56:23.930
Myles Brown: You know the AI kind of companion on the application development side. And so you know whether you use code, whisper or something else. You know that

354
00:56:23.960 --> 00:56:30.610
Myles Brown: that idea of. you know getting your code written quicker because you're letting the computer code it.

355
00:56:30.950 --> 00:56:40.030
Myles Brown: You know, there's a lot of these sort of low code. No code kind of initiatives. And the code whisper is the one Amazon has.

356
00:56:40.950 --> 00:56:43.060
Myles Brown: I mentioned that there's

357
00:56:43.280 --> 00:56:54.489
Myles Brown: there's a lot of these specific services that are really geared around a particular industry. And so here's some of those

358
00:56:54.590 --> 00:57:01.779
Myles Brown: Amazon monoton. Look out for equipment. Look out for vision. So these

359
00:57:01.850 --> 00:57:24.010
Myles Brown: you know a lot of times in manufacturing you might have all kinds of equipment. All this equipment, these days is spitting out some sort of metrics, usually in the form of Internet of things. So there's there's hundreds, thousands of sensors within your manufacturing plant that are just generating little bits of Json, saying.

360
00:57:24.260 --> 00:57:27.940
Myles Brown: This is the idea of my sensor. Here's my readings right now.

361
00:57:28.820 --> 00:57:35.120
Myles Brown: and it's overwhelming, right? Like, what do you do with all that data? I can just throw it somewhere. But

362
00:57:35.130 --> 00:57:46.099
Myles Brown: I want to proactively look at that and look for patterns that maybe the human eye can't see. I remember the first project I ever saw. It was doing something like this. This was

363
00:57:46.420 --> 00:57:51.530
Myles Brown: not even cloud. This was a long time ago. I I did a lot of training

364
00:57:51.760 --> 00:57:54.859
Myles Brown: with a company called Cloud Era. That was a big data company.

365
00:57:55.060 --> 00:58:00.610
Myles Brown: and and there was this electrical grid. We got quite a large company.

366
00:58:00.810 --> 00:58:06.080
Myles Brown: and what they were doing was they were. They had all these sensors on different

367
00:58:06.310 --> 00:58:08.920
Myles Brown: transformers.

368
00:58:09.150 --> 00:58:11.640
Myles Brown: and so what they were trying to do was figure out.

369
00:58:12.070 --> 00:58:21.699
Myles Brown: what are the What's the scenario, you know? Can we figure out just before one of these transformer blows

370
00:58:21.770 --> 00:58:25.939
Myles Brown: so that we can kind of rewrite the power and avoid that thing happening.

371
00:58:26.530 --> 00:58:44.609
Myles Brown: And it wasn't things that the human eye could find. And so they were just grabbing loads and loads of data and doing some analysis on it and building these machine learning models so that they could say, Hey, we're seeing some conditions that we've seen in the past right before. Something blows. So let's

372
00:58:45.030 --> 00:58:47.859
Myles Brown: you know, reroute some of the power to avoid that.

373
00:58:48.260 --> 00:59:02.970
Myles Brown: And so it's that same sort of thing. But you know, building those things from scratch is a bit of a pain. So that's what some of these do. So look out for equipment. We'll actually do some sort of, you know. Look at for anomalies in those readings.

374
00:59:03.410 --> 00:59:05.570
Myles Brown: Look out for vision says.

375
00:59:05.840 --> 00:59:21.300
Myles Brown: let's brought product defects using computer vision. So you got cameras on the assembly line, and it looks. And it says that looks different than the others. So let's flag it, you know? So there's all sorts of things like that.

376
00:59:21.790 --> 00:59:29.540
Myles Brown: Panoramas, computer vision from existing cameras. Yeah, there's all sorts of things in there. There's even

377
00:59:29.800 --> 00:59:34.770
Myles Brown: a more specific kind of data lake

378
00:59:34.840 --> 00:59:43.740
Myles Brown: for storing health care data. so store transform and analyze health and life sciences, data in the cloud at Petabyte scale.

379
00:59:43.890 --> 00:59:56.910
Myles Brown: And so you know what they're really doing is saying, hey. Amazon says, creating a data lake could be as simple as just making an S 3 bucket. That's usually not enough. You're going to have to have a good data catalog.

380
00:59:57.020 --> 01:00:22.300
Myles Brown: But we know that in the health care industry you've got certain types of data that you're going to see over and over again. So like, whatever Hl, 7 data things like that, we've got certain compliance like hipaa compliance that we know we're going to run into. So we know that there's some governance we need to do in a certain way. And so let's just make a very opinionated way of building a data leg. so that you don't have to, you know.

381
01:00:22.650 --> 01:00:24.320
Myles Brown: do everything from scratch.

382
01:00:25.700 --> 01:00:28.630
So that's the that's the big idea of that one

383
01:00:28.980 --> 01:00:32.989
Myles Brown: I and you know, I think

384
01:00:33.130 --> 01:00:39.369
Myles Brown: I think the thing that you should really get the sense of, you know, going forward with machine learning is that

385
01:00:39.930 --> 01:00:52.720
Myles Brown: development can be complex and expensive. You know, the the the practitioners are famously hard to find, because, you know, colleges and university weren't churning out data scientists as quickly as people needed them.

386
01:00:52.990 --> 01:00:57.539
Myles Brown: And so what we're finding is that anything we can do.

387
01:00:57.860 --> 01:01:08.430
Myles Brown: you know, to make that machine learning more accessible. So if all these steps are required, you know. Maybe. Have sage Maker help us with all those things.

388
01:01:08.750 --> 01:01:18.559
Myles Brown: or alternatively, instead of having people dealing with the low-level machine learning. Let's build the machine learning into some of our existing products.

389
01:01:18.900 --> 01:01:31.089
Myles Brown: So whether we're storing data in databases and querying it there, or a data warehouse, or or even using business intelligence tools. Let's add some Ml. Capabilities to those services directly.

390
01:01:31.450 --> 01:01:36.209
Myles Brown: So in Amazon, Aurora, we now have Aurora, Ml, just like redshift, Ml.

391
01:01:36.230 --> 01:01:38.820
Myles Brown: I, even Neptune for graph databases.

392
01:01:39.060 --> 01:01:46.990
Myles Brown: And what this is doing is it's saying, Hey, I've got, you know, I've got some data. There's definitely some data points missing here.

393
01:01:47.050 --> 01:01:48.650
Myles Brown: So if you want.

394
01:01:48.740 --> 01:01:54.419
Myles Brown: I could build recommendations that would suggest what the missing data would be based on.

395
01:01:54.530 --> 01:01:56.719
Myles Brown: you know, patterns that we found.

396
01:01:56.940 --> 01:02:05.129
Myles Brown: And so we can do that kind of thing even in quicksite. That's the business intelligence tool that Amazon has, that it looks a lot like tableau.

397
01:02:05.270 --> 01:02:14.810
Myles Brown: when you're when you're building your dashboards, you know, you can add some machine learning capabilities in there without knowing a lot about machine learning.

398
01:02:14.860 --> 01:02:25.799
Myles Brown: you know, for the most common type things that you would want to do. Even Athena has an Ml. Piece to it. so that you know, in those queries I can. I can ask for more.

399
01:02:27.740 --> 01:02:36.709
Myles Brown: So this is the big idea. This is the modern data strategy on aws so modernized. We talked about that being moved to

400
01:02:36.810 --> 01:02:38.740
Myles Brown: these.

401
01:02:40.810 --> 01:02:46.829
Myles Brown: you know, manage databases, maybe go towards open source stuff as much as possible.

402
01:02:46.960 --> 01:02:50.899
Myles Brown: Then the unify was really, let's let's

403
01:02:50.960 --> 01:02:57.719
Myles Brown: use the best of both data warehouses and data lakes and make the data available wherever it lives.

404
01:02:57.740 --> 01:03:10.339
Myles Brown: And then the innovate says, now that we've got a data lake with all this data, or wherever it lives. Let's let's add Ml. Into the mix so that we can, you know, make better decisions about the future.

405
01:03:11.420 --> 01:03:31.070
Myles Brown: So that's the sort of three-tone tiered approach that aws has to to modern data strategy. Now, I I think originally we thought this might be a 2 h session. I I think it's going to be more like an hour and a half, depending on how many questions you got. But let's talk a little bit about next steps, you know this is a

406
01:03:31.160 --> 01:03:53.779
Myles Brown: short presentation. You're not going to walk out of here and be an expert in the Aws analytic stack right? Unless you already were. So How can you learn about this? Well, one is to learn at your own pace, right? There's a lot of stuff out there. There is aws has skill builder, which is sort of self-paced.

407
01:03:53.840 --> 01:03:59.040
Myles Brown: If you want lab access. You know, there's many, many videos that you can use just with the free

408
01:03:59.460 --> 01:04:12.110
Myles Brown: version of skill builder. But if you want to do some self-paced labs, you can pay for a subscription, and and you know they give you access to a bunch of labs that you can do. You don't use your own aws account. They give you timed access to one.

409
01:04:12.540 --> 01:04:24.850
Myles Brown: Then you've got, you know actual classes. Instructor led. That's where exit certificate comes in. Because we, you know, that's what we excel. Yeah, lots of good classes. I'm going to show you some of those classes coming up.

410
01:04:25.320 --> 01:04:38.849
Myles Brown: there's some ramp up guides eventually, you know, there's some certifications. The one main certification in this area is called a specialty certification called aw, certified data analytics.

411
01:04:39.040 --> 01:04:44.129
Myles Brown: Now, the problem with those specialty certifications. They're very highly advanced. They they expect you to know

412
01:04:44.140 --> 01:04:54.949
Myles Brown: a lot about aws in general. But you have to have a very good breadth of knowledge on all the aws analytic stack. And there's a lot to that analytic stack.

413
01:04:54.960 --> 01:05:01.630
Myles Brown: So just to show you a little bit of that, let me bring up my console here.

414
01:05:08.620 --> 01:05:15.559
Myles Brown: So this is an aws account I use just for Demos in class. And so if I just go to the list of services

415
01:05:16.690 --> 01:05:19.249
Myles Brown: here to all services.

416
01:05:26.420 --> 01:05:36.910
Myles Brown: I know it's really small to read, but I just want to give you a sense of this is how many services there are in aws in the management console. There's a few more that aren't in the console.

417
01:05:37.160 --> 01:05:45.249
Myles Brown: and so if your new aws, this has got to be very daunting. Okay, I can only imagine you're like, do I have to learn 200 plus services. No?

418
01:05:45.650 --> 01:06:01.389
Myles Brown: Oh, well, not nobody. But I'm sure very few people know all of those services. Quantum compute. You're never going to use Amazon bracket if you're not doing anything with satellites. You're never going to use ground station.

419
01:06:01.740 --> 01:06:05.630
Myles Brown: But if you look at the analytics part well, we've got a lot in there.

420
01:06:05.770 --> 01:06:13.259
Myles Brown: So we? We mentioned most of these along the way, some of them we didn't talk really about, because they're a little older like cloud Search.

421
01:06:13.620 --> 01:06:20.520
Myles Brown: You wouldn't use that because these days you'd use open search instead. It's really just around because it was there for

422
01:06:20.980 --> 01:06:21.730
Myles Brown: and

423
01:06:21.890 --> 01:06:30.269
Myles Brown: you might use. Ms K. You might use kinesis, you probably don't use both right? There's sort of some competing things.

424
01:06:30.300 --> 01:06:39.830
Myles Brown: So that's some of the analytics stack. Then you've got all the machine learning services that we talked about. And then all those different database services they go

425
01:06:40.280 --> 01:06:41.280
Myles Brown: here they are.

426
01:06:42.310 --> 01:06:46.840
Myles Brown: So that's a lot of services that make up the sort of data stack right?

427
01:06:47.070 --> 01:06:49.810
and learning all of them is a real.

428
01:06:49.990 --> 01:06:57.809
Myles Brown: you know, challenge. And so one of the things I wanted to show I can get it up and running here.

429
01:06:59.860 --> 01:07:00.590
Myles Brown: Just

430
01:07:02.870 --> 01:07:03.710
Myles Brown: second.

431
01:07:07.320 --> 01:07:09.079
Myles Brown: let me show you our

432
01:07:12.000 --> 01:07:24.519
Myles Brown: our learning and certification path document. So this is a document we made just at exit certified to kind of hit home. You know what classes you should take depending on your roles.

433
01:07:24.680 --> 01:07:32.730
Myles Brown: you know. So so most of the technical tracks start with the one day tech essentials. Then go on to like a 3 day associate level class.

434
01:07:33.010 --> 01:07:37.729
Myles Brown: But if you're interested in the data analytics side, you know, here's here it is. Let me blow it up a bit.

435
01:07:38.340 --> 01:07:59.140
Myles Brown: So again, you would start with the one day. Tech essentials, then do doesn't really matter whether you do architecting or sysops, or even developing on aws. One of those 3 day associate level classes will will get you the basics of how aws works. Then we have a 4 day class called building modern data analytics, solutions on aws.

436
01:07:59.220 --> 01:08:15.060
Myles Brown: And that really gets you ready for that specialty. Cert, right? So we would say, you first have to know a bit about aws. So take a 3 day class, then take this 4 day class. But if you look at this 4 day class what it really is. It's 4 one day classes.

437
01:08:16.080 --> 01:08:25.990
Myles Brown: So it's a 4 day class. But and and we do run it as a 4 day class fairly often. So August 20, s September twelfth.

438
01:08:26.010 --> 01:08:32.830
Myles Brown: So some of them aren't marked as guaranteed to run. That's that. Gtr. that means if you sign up for it. It's definitely running.

439
01:08:32.990 --> 01:08:53.440
Myles Brown: But if I go back out a level, you're going to see the data analytics classes. There's really for one day class. So typically people start with the building data lakes on Aws, that teaches them, how do we use S. 3 to build a data lake. And then all about aws glue for the catalog and for Etl type purposes.

440
01:08:53.990 --> 01:08:58.999
Myles Brown: then there's a one day class on Redshift.

441
01:09:00.020 --> 01:09:11.120
Myles Brown: There's a one day class on. or is it batch solutions? So this is really about Emr and and spark to a certain degree.

442
01:09:11.260 --> 01:09:15.590
Myles Brown: and streaming which is most well, it's about half and half.

443
01:09:15.609 --> 01:09:22.170
Myles Brown: It's a little more than half on Kinesis, say 60% kines is 40%. Ms, K. So Kafka.

444
01:09:23.670 --> 01:09:34.979
Myles Brown: And you know. So that's the idea. It's those 4 one-day classes. So we we do offer them as one day classes. So if you just want the data lake and the redshift, you can take those 2 days

445
01:09:35.260 --> 01:09:46.700
Myles Brown: it'll probably be the Monday and the Thursday, and then the 2 in between you would skip, and or you can do all 4. But if your goal is to get the certification, you better take all 4 because you're going to need that breadth of knowledge.

446
01:09:48.359 --> 01:09:50.360
Myles Brown: And so that's

447
01:09:50.680 --> 01:10:01.230
Myles Brown: you know. That's what I wanted to say about the classes that we offer. Those classes are are designed and developed by Amazon. They give you a lab environment for 3 months to do

448
01:10:01.320 --> 01:10:13.669
Myles Brown: Are there any free classes that include some free labs to play in? That's a good question, Edward. I would say that you know we do these kind of hour hour and a half sessions.

449
01:10:14.820 --> 01:10:34.919
Myles Brown: if you're looking for free labs, there's really not much out there. If you create your own aws account. You know there is some stuff with the free tier, but as soon as you get into something like redshift, forget the read free tier. You're going to be spinning up a lot of money, so you're better off to go and get the skill builder, you know, paid for the $29 for a month.

450
01:10:34.950 --> 01:10:40.179
Myles Brown: and then run through as many labs you want. You could do a couple months. Get through quite a bit.

451
01:10:40.390 --> 01:10:46.809
Myles Brown: But that's you know. That's that's the lowest cost to get official aws labs. There's some

452
01:10:46.820 --> 01:10:49.919
Myles Brown: third party like gray market stuff as well.

453
01:10:50.020 --> 01:10:55.149
Myles Brown: but you never know how current it is, and things like that.

454
01:10:55.990 --> 01:11:12.239
Myles Brown: Where can we get the Pdf, oh, you mean this? Pdf, oh, yeah, I can drop that into the chat. Maybe let's see if that works. depending how your chat works. This may or may not work. Let's let me drop that learning path document in there.

455
01:11:13.990 --> 01:11:17.730
Myles Brown: Some of you may see that now in the chat, and you can download it from there.

456
01:11:18.520 --> 01:11:24.590
Myles Brown: Maybe not. It depends how your company works with it. I'm gonna drop another file in here as well.

457
01:11:24.810 --> 01:11:29.790
Myles Brown: because I'm going to talk a little bit about what else exit certified does.

458
01:11:29.860 --> 01:11:43.549
Myles Brown: So I mentioned at at the top. You know, we are primarily a Cloud training company, although we do other things that are not necessarily cloud the bulk of our business is based around, you know.

459
01:11:44.290 --> 01:11:50.739
Myles Brown: Well, aws, is definitely our biggest product line. But the other, you know, 3 kind of public cloud vendors.

460
01:11:50.810 --> 01:12:15.550
Myles Brown: and then all the stuff that goes around it right? So you know, containers and cloud native. So docker kubernetes, all that kind of stuff. We do all that training here on the analytics stack. We're partnered with a bunch of other companies like tableau and data bricks and cloud era, you know. So if you want to take some of those frameworks. We've got them Some of our newest ones are starburst. And Nvidia has a lot of

461
01:12:16.310 --> 01:12:17.010
no

462
01:12:17.140 --> 01:12:18.980
Myles Brown: machine learning type staff.

463
01:12:19.120 --> 01:12:41.879
Myles Brown: we have a couple of good class on Snowflake now. but you know, whatever it is, this is just indicative of what we've got. We've got machine learning classes, you know every one of these cloud vendors has their own machine learning flavors. But we also have just some basic hey, I need to learn python. I need to learn the python libraries for data science. And then I want to learn some basics and machine learning

464
01:12:42.070 --> 01:12:46.460
Myles Brown: agnostic from vendors and tools. You know, we've got all that

465
01:12:46.770 --> 01:13:01.130
Myles Brown: And so that's we call this our cloud centrics diagram it. It sort of shows the suite of you know what we do from a training perspective. So I just wanted to kind of hit that home and these things if you click on them, you know, that takes you to

466
01:13:01.170 --> 01:13:10.169
Myles Brown: part of our website on analytics. And then, you know, you can click on a particular vendor. Say, click on aws, it takes me straight to the data. Analytics courses

467
01:13:11.590 --> 01:13:17.639
Myles Brown: click on some of these like data breaks, it takes me to our database training page

468
01:13:19.320 --> 01:13:22.570
Myles Brown: and so on. So that's the

469
01:13:23.380 --> 01:13:30.170
Myles Brown: that's the cloud centrics diagram. I think I threw that in there. I just got a couple slides left to go here.

470
01:13:32.220 --> 01:13:47.130
Myles Brown: So we talked a little bit about the certification. We talked about cloud centrics. we have a promo right now. If you were thinking about taking a course so exit certified.com slash upskill, let me go there.

471
01:13:48.080 --> 01:13:50.450
Myles Brown: and then I'll put that into the

472
01:13:59.400 --> 01:14:00.309
Myles Brown: was it

473
01:14:01.540 --> 01:14:03.710
Myles Brown: upscale? 500

474
01:14:04.130 --> 01:14:08.970
Myles Brown: if that takes me there? Yeah, here it is. So I'll put this link into the chat as well.

475
01:14:10.760 --> 01:14:22.399
It looks like you have to register by August 30 first. So that's the end of our summer program, promo. But you don't have to take the course until it has to be completed by November thirtieth.

476
01:14:22.950 --> 01:14:33.269
Myles Brown: So this promo code, depending on the length of the class, most of our Aws classes are 3 days. Some of them are 4 days. You get basically $100 per day off

477
01:14:33.880 --> 01:14:46.470
Myles Brown: if you do private training. So our public classes are running fairly often, like we have the largest guaranteed to run schedule in North America of all the training authorized training partners.

478
01:14:46.620 --> 01:14:57.419
Myles Brown: so if you have one or 2 people right, you know, you can go into a public course. Some of those analytics ones probably run every least once a month for sure.

479
01:14:57.830 --> 01:15:16.610
Myles Brown: but the you know something like a very basic architecting on aws, we run that every week guaranteed right? So we we have lots of public classes if you want. If you've got a group of people well, then, we can do private training just for you, either on site. We'll send an instructor to you if you want that.

480
01:15:17.080 --> 01:15:27.010
Myles Brown: There's some of that happening now. We could do it at one of our training centers. That's a little less popular these days. People don't want to travel just for training. Most of the training is virtual.

481
01:15:27.120 --> 01:15:49.080
Myles Brown: The nice thing about doing a private class is, you can decide. You know. Normally it's 3, 8 h days. You could cut it up over, you know, 5 half days, or something like that. you decide the hours, and if you don't want all the content you could say, well, this 3 day class, can we do it in 2 because we're not using these services right? No problem. So we can kind of tailor things.

482
01:15:49.180 --> 01:16:00.999
Myles Brown: But you know, obviously, if you're trying to get the certification, you better learn all the material. But that's a trade off. Yeah. So we do a lot of private training. I would say, if you have like

483
01:16:01.690 --> 01:16:07.159
Myles Brown: 5 people, maybe 6 people. Then all of a sudden it becomes much more economical to do group training.

484
01:16:07.330 --> 01:16:13.320
Myles Brown: because then you get a lot of economies of scale. The tenth eleventh person you put into the class. It's much, much cheaper.

485
01:16:14.440 --> 01:16:18.619
Myles Brown: And so that's you know, that's our main business.

486
01:16:18.890 --> 01:16:27.940
Myles Brown: And so that's the summer promo. I guess the last thing I wanted to show was those webinars. I mentioned that this is

487
01:16:28.350 --> 01:16:35.919
Myles Brown: this is the second last in this series of webinars. I think if I go here I can find

488
01:16:37.400 --> 01:16:39.910
Myles Brown: resources webinars.

489
01:16:42.500 --> 01:16:48.790
Myles Brown: So these are our most recent webinars. So this is today's so if you click on that that

490
01:16:48.920 --> 01:17:04.820
Myles Brown: I mean, it'll still get you to sign up next week. We have one specifically on machine learning. So my My colleague, Chris Littlefield, will be delivering that. I'm gonna probably introduce them and help them with the chat for that one, because we expect a lot of people.

491
01:17:04.840 --> 01:17:16.010
Myles Brown: Last week my colleague Pete did one called securing your aws, Cloud, and the week before that my colleague Craig, did one on performing large-scale migration. So those are all there.

492
01:17:16.060 --> 01:17:17.250
Myles Brown: let me

493
01:17:17.670 --> 01:17:21.809
Myles Brown: throw that in here. And so this is where today's

494
01:17:21.960 --> 01:17:33.869
Myles Brown: video will be. But because you registered, you're going to get an email specifically with the link. So you, if you came in late, not a problem. You can watch the recording. If I talk too slow

495
01:17:33.970 --> 01:17:36.310
Myles Brown: you can speed it up. You know all that good stuff.

496
01:17:36.690 --> 01:18:02.679
Myles Brown: but there's some other stuff we have in here. I I did this early this year. A public cloud comparison sort of talking about the 4 main clouds and what they each do. Well, things like that. My colleague, who does a lot more of our azure stuff. He did this to our a webinar on helping get ready for the Az 104 exam. That's the the most popular azure certification is the azure admin.

497
01:18:03.220 --> 01:18:13.030
Myles Brown: I think we've done some in the past on, you know, passing the cloud practitioner and things like that. So if you just keep poking around in there. We've got some good free webinars in there.

498
01:18:13.470 --> 01:18:16.339
Myles Brown: like, I said. You know.

499
01:18:16.370 --> 01:18:30.870
Myles Brown: The big idea of this is to get you an idea of what the aws stack looks like, and their big ideas of you know how to build a modern strategy. I'm going to stick around for questions.

500
01:18:30.870 --> 01:18:48.129
Myles Brown: What you'll realize is when you go to leave it will pop up a little survey. It's only going to take you 2 min. It'll just ask you like, Hey, what did you think of this? And what other topics are you interested in? So we're just trying to, you know. Put our finger on the pulse and figure out what? What kind of the free stuff should we be doing

501
01:18:48.170 --> 01:18:58.329
Myles Brown: so if you can answer any of those questions that would be great. And like, I said, you should get an email. I don't know if it's today or tomorrow, some point where you'll get the link to the recording

502
01:18:58.450 --> 01:19:04.020
Myles Brown: probably be tomorrow or the next day, I would imagine. And

503
01:19:04.150 --> 01:19:14.020
Myles Brown: like I said, I'm here for a while. If you want to throw questions in the chat. I'm here to answer questions as as long as you want. I don't want you to leave with any questions about

504
01:19:14.170 --> 01:19:19.910
Myles Brown: aws, specifically, and that that stack of analytics

505
01:19:20.300 --> 01:19:24.920
Myles Brown: options we have. I think we've lost a couple people. But like I said.

506
01:19:25.470 --> 01:19:39.189
Myles Brown: good luck. oh, I should put my email address in here. Let me do that. Files, Don brown@exitcertified.com. If you have any questions after the course, that's a good place to send them.

507
01:19:45.090 --> 01:19:45.760
Myles Brown: Okay

508
01:19:49.440 --> 01:20:00.619
Myles Brown: again. I don't know some some companies the way they lock down. Zoom, you might not be able to copy and paste out of the chat. Most of them you can copy and paste. You may not be able to get the documents that I sent through.

509
01:20:01.120 --> 01:20:04.209
Myles Brown: but those are all on our website. If you poke around a little bit

510
01:21:14.720 --> 01:21:17.469
Myles Brown: I'm not seeing any questions, so

511
01:21:18.500 --> 01:21:21.510
Myles Brown: I'll wait another couple of minutes, and then I'll shut down the call.

AWS Discovery Day: Fundamentals of a Modern Data Strategy on AWS [Webinar]

To watch enter your email address

Alert!

Modal Title

Error!

Default Title

Prompt

Confirm

Login