“Quality is decided by the depth at which the work incorporates the alternatives within itself, and so masters them.” Theodor W. Adorno. In other words, to reliably build the API Gateway layer, an in-depth study is required to consider the different available alternatives.
After defining some of the main concepts in the API world in the previous article, I will talk about the different ways of deploying an API Gateway for the Machine Learning platform.
In this article, I will use the infrastructure and software layers designed in one of my previous articles. You may want to go through it to have a clearer view of the platform’s architecture before proceeding.
As a reminder, the scope of this series of articles is the model serving layer of the ML platform’s framework layer. In other words, its “API Gateway”.
“Don’t judge a book by its cover”…or so they say… Well in the real world, with a sacred production environment and deadlines, everyone is judging your platform by its cover, that is, its API…
After defining the framework layer of a custom machine learning (ML) platform on AWS, it’s time to talk in detail about some specific scopes of this layer.
The scope of this series of articles is the model serving layer of the framework. In other words, its “API Gateway”.
“Architecture is the reaching out for the truth” — Louis Kahn. The question is: Truth about what? Designing a platform is a continuous search for the truth. Nevertheless, this could be an endless process if the “about what” is not well defined.
This is the fourth chapter of my journey in building a Machine Learning Platform on AWS. This chapter is based on my work so far presented in the previous parts: the high-level overview of the ML Platform, the infrastructure & software layers, and the framework layer.
In this part, I am going to study the fourth layer of the Machine Learning Platform on AWS: Use cases layer.
1 | So, Truth about what?
When a new use case is selected by stakeholders, comes the ultimate goal of landing this use case on the machine learning platform. Achieving this goal is constrained by solving three riddles:
- What…
Architecting is all about how things fit together… How an object that performs a function can also be a work of art… That’s exactly what a framework is, a fascinating work of art!
This is the third piece of my journey in building a machine learning (ML) platform on AWS and a continuity of the high-level overview presented in the first article as well as the Infrastructure and Software layers demystified in the second one.
In this part, I am going to study the third layer of the ML platform: the framework layer.
1 | So, why a framework?
By definition, a framework is “an abstraction […] providing generic functionality.” ¹ In other words, it is a high-level layer that hides the tricky details of the platform’s software stack and exposes user-friendly functionalities.
Amazon understood this very well. That…
Properly preparing and passing the SAP-C01 exam gave me the answers… But what are the questions?… The job of an architect is to make sure everything he creates is designed to withstand the weight placed upon it. Such perfection is unreachable without asking the right questions.
This is the second slice of my journey in building a machine learning (ML) platform on AWS and a continuity of the high-level overview presented in the first article.
In this part, I am going to study in detail the first two layers of the ML platform: infrastructure and software layers.
1 | So, what are the questions?
A good design for the infrastructure and software layers should respond to many challenges:
- How to guarantee high availability?
- Is it possible to have a scalable design?
- How to solve the capacity planning riddle?
- How to insure data resiliency?
- What about the platform security?
Answering all these challenges is…
After getting my AWS Certified Solutions Architect — Professional certification, I was wondering: was it worth it? And does it give me enough knowledge to architect a platform?
So, I decided to put it to the test: I tried to build a Machine Learning Platform on AWS.
The result? This certification is definitely worth it: Properly preparing and passing it gave me the necessary tools to look at the big picture, see all the little moving parts. But it is not enough… In order to achieve this, I got inspired by what big companies with great expertise in this domain did, like Uber with their Michelangelo, Netflix, Comcast, and many others. …
Salah REKIK
Passionate Data Architect with progressive experience in building big data platform at @CreditAgricole and machine learning platform on AWS
