Description: This passage talks aboutgetmyoffers-capital-one, which tells us how to test your AWS resources in Capital one and introduces a couple of other approaches that we could use to test.
I’m Abbie from AWS and I’m here with Kapil from Capital One. Thank him much for joining me. Before we start it Kapil can tell us something about what he does at Capital One and what Capital One does.
Capital One is a leading information based technology company bringing ingenuity, simplicity and humanity to banking and he is a developer.
They’re awesome. What we’re talking about today is some of the ways that the Capital One tests our AWS resources. Kapil could walk through a couple of the approaches that people could use to test and then he will dig deep in an example as following.
“When we’re testing against AWS, we’re testing that external connectivity. It is an API call to the AWS endpoints. There are a couple of different techniques which are common. One is mocking. Mocking is a generic technique in almost every language library and every language exploitation.
Instead of making that API call to inject a response in our test code, that’s a great way of doing things. One of the issues is that it ends up being a little bit brittle and it ends up testing our expectations of how the API call is. It is going to happen as opposed to testing what the actual real behavior is. I’ve definitely seen cases where people mock something and it works in tests but not in real.
Another technique which is good and easy to get started with is something called stubbing. Instead of having to inject the response, it is a little bit different from mocking. It is going to happen on every API called stepping. It builds up a mental model of how the underlying service works to allow us to use.
The API is naturally in our tests and internally it’ll figure out how to write the right response. A great stubbing library that I’d like to call out is something called moto. Moto is a Python. It is only a library but it is cool. It offers a rest endpoint and it covers a wide variety of services. Alas Ian and a couple of others in the AWS community have created something called local stack.
Moto and a few other best-of-breed stubs create a full implementation of over a dozen AWS services. You can get that by doing docker run. You have everything from API gateway to route 53 and everything in between.
These things seek the techniques. Stubs are great except for the fact that they are not necessarily going to test all and to be able to give you all of the different functionality underlying API. If there’s a new feature that comes ou, you have to wait for the stub to get updated and to reflect that. You get a bit of coverage gap to the actual thing.
As a result one of the ways that I like to test is using something called flight recording. A great library for that is placebo.
Let’s go into some unit tests using placebo. As a flight recorder this is the clock custodians unit test. Clock custodians which is an open source project from Capital One is used for fleet management at scale across all the AWS resources for security, compliance, cost optimization and lots of cool stuff.
We have over 700 unit tests with custodian that used this placebo flight recording technique. You can see that effectively all that we’re doing is that we’re setting up how we get our SDK session a little bit differently. We could do a replay if we want to. We could switch this out to record and it will go to the live AWS API endpoints.
We make API calls and restore all the responses on the disk. When we go into replay mode when we’re running our tests and it will be off line which won’t make any real API calls. It will replay all the stuff that was on disk. After we’ve set up our our placebo session, all we have to do is that we interact with the API effectively.
We’re creating a V PC. We’re deleting an act for the test is done. We’re describing Network ACLs and doing a couple of API calls. Finally we do or get our test assertions.
One of the nice things is that you can have a bit of flexibility in setting up with flight recording. You go in and manually create some stuff in the console or using a different tool. Then you can test against that in your unit test or you can do something where this unit test is self-contained.
In this unit test we’re creating a whole V PC. We’re creating all this stuff. That ends up being powerful because at least for us what we do is that we set up this little functional decorator on our test. When we say that a test is fully self-contained, we’re able to say we’re able to take this thing that we’re using as a unit test.
As we go to make a release for example, we’ll record it, run it and live against environment. Because we know that it’s completely self-contained. That ends up being a powerful way to test using flight recording to get full coverage on the all AWS APIs and to test actual behavior.”
It maybe get started with testing out our infrastructure instead of our code. It’s a couple of things to think about. Always use the right tool for the job and one way might be easier to crop an error. In some ways you keep up with the latest resources and services. Treat your infrastructure the same way as you treat your application code. So test often and test early. Thanks for reading this article.