Over the week, I had the opportunity to do some hands-on research about one of the latest cloud services that Amazon has to offer. Combining several DevOps-services into one, Amazon tries to eliminate the hassle of combining several solutions and deliver a package service that perform one of the most common use cases when it comes to DevOps on AWS.
Traditional solution
Imagine that you want to build a brand-new CI/CD, you most likely use a git repository as a source for your code. Perhaps you are hosting it on GitHub, Bitbucket or GitLab. You then want to connect it to some kind of pipeline, be it Bamboo, CircleCI or even one of the Git-platforms you used to share code, such as GitLab for running build automation and testing. Then to deploy it using some other suitable tool. My point here is that for each tool you introduce there is need for knowledge, resources to perform maintenance and upgrades, reading documentation and version updates, integrating the tools and solving integration issues introduced on either end once upgraded.
Cloud native solution
If you instead would go the native cloud-solution you would of course then hook this up with AWS CodePipeline to set up your CI/CD pipeline structure, CodeCommit to act as your source code, CodeBuild to build it on an integrated build server and finally CloudFormation to handle the deployment in AWS.
This is where Amazon introduces AWS CodeCatalyst. CodeCatalyst is a PaaS tool to quickly deploy the above solution in AWS with few configurations. It also comes with its own web based platform in which developers, testers and stakeholders can join the workflow without direct access to the AWS infrastructure or management platform. Let me give you a quick tour of what this service looks like.
PaaS solution
Amazons CodeCatalyst is hosted on a new concept called space. This is where you can store all your projects that you want to use to manage your pipeline environments. After you connect a user and an Amazon account, you then have access to the AWS CodeCatalyst portal. In here you can start off using blueprints like most AWS services, or you start from scratch with a blank project. From this project you can then manage source code in repositories, establish scrum boards, run CI/CD workflows, handle secrets, collect test reports and collaborate with other invited members.
Running through the side panel you are first welcomed with a dashboard that shows an overview of all your repository along with the documentation from README.md with your source code. It then goes to show workflow runs, open pull requests and issues assigned to you. Unfortunately, none of these things in the dashboard are configurable.
Going over the issue board it is very similar to a normal scrum board, configurable columns representing status of an issue, creating issues with priorities, adding labels and fields and assigning the issues to members.
The code section holds your source repositories and any open pull requests. It is also possible to link with third-party repository providers if you fancy something else than the embedded one.
CI/CD is the heart of this platform. Here you find the workflows and any previous runs. You can inspect each run to get a visual representation and to follow the ongoing status of each step in that flow. Here you also find secrets management but notice that this or any of the other embedded services holds no references to services you would find from the management console such as AWS Secrets Manager. This means that you can’t set up secret rotations or encrypt it using your own customer managed key.
The reports show a run of all your test cases that you set up from code, their separate results and a pass rate. Though I can’t find how to set up the success criterias for my report. So I don’t know what statuses are available, if this is configurable or anything else about the test reports is configurable from this portal.
Under project settings I don’t find much besides an integration to Slack channels under notifications. It is interesting though that there are no settings how to handle other notifications that I assume goes out to the signed up members email address.
If you are familiar with Azure DevOps you will instantly feel a lot of resemblance. You have your pipelines, repositories, test reports and even issue tracking (no more Jira or Trello, right?). This is an obvious jab at Amazons strongest competition in which they compete to offer the same services but with different gift wrappings. In my opinion I find this attempt from AWS CodeCatalyst to fall a bit short. Yes, you have all the services you need and the ones you commonly use but all it does is just packing services you already have in AWS. There are no improvements that elevates the old way of doing DevOps in AWS besides the time to set up and integrate the services. Also, there are no improvements that make AWS CodeCatalyst to be a serious competition to Azure DevOps.
After some time with this tool I find myself greatly frustrated. Fields are sometimes not configurable in the issue board due to random connection issues, the orientation of pages is a labyrinth, layout of all the visuals won’t hold for a moment on my 13 inches screen and lack of documentation space is a miss. Also notice that this service is (as of now) only available in the region Oregon despite having status General Availability, which leaves a lot of geographical and compliance restrictions.
While I appreciate your effort, Amazon, I think you didn’t try hard enough.
Kind regards
Magnus Sjöstrand