ARTICLES

  • Setting Up a Local Development Environment With IntelliJ, DevContainers, and Amazon Linux 2023

    In modern software development, containerization offers an isolated and consistent environment, which is crucial for maintaining parity between development and production setups. This guide provides a comprehensive walkthrough on creating a local development environment using IntelliJ IDEA, DevContainers, and Amazon Linux 2023 for Java development. Why Use DevContainers? What Are DevContainers? DevContainers are a feature provided by Visual Studio Code and other IDEs like IntelliJ IDEA through extensions. They allow you to define a consistent and reproducible development environment using Docker containers. By encapsulating the development environment, you ensure that all team members work in an identical setup, avoiding the "it works on my machine" problem.

  • Setting Up a Local Development Environment With IntelliJ, DevContainers, and Amazon Linux 2023

    In modern software development, containerization offers an isolated and consistent environment, which is crucial for maintaining parity between development and production setups. This guide provides a comprehensive walkthrough on creating a local development environment using IntelliJ IDEA, DevContainers, and Amazon Linux 2023 for Java development. Why Use DevContainers? What Are DevContainers? DevContainers are a feature provided by Visual Studio Code and other IDEs like IntelliJ IDEA through extensions. They allow you to define a consistent and reproducible development environment using Docker containers. By encapsulating the development environment, you ensure that all team members work in an identical setup, avoiding the "it works on my machine" problem.

  • Microservice Logs Testing in the Cloud: Important but Often Ignored

    Logs of an application are the initial step to start debugging and analysis of issues, so they are quite an important part of the application. However, they are often ignored during the testing phase. As the world is moving to cloud-based microservices, gaining insights into any customer issue heavily relies on logs. If they are not properly structured or don’t contain enough information to analyze the issue, they can be a significant stumbling block for engineers. In this article, we’ll explore why testing microservice logs is crucial and how engineers can ensure logs are up to the mark. Why Logs Matter Logs are the backbone of debugging, monitoring, and security. They help engineers:

  • Deploy Your Data Pipelines With GitHub Actions

    Automate, customize and execute your software development workflows right in your repository with GitHub Actions. You can discover, create, and share actions to perform any job you'd like, including CI/CD, and combine actions in a completely customized workflow. GitHub Actions is a powerful tool for automating software development workflows, and it can also be used to automate data pipeline processes. In this post, we will walk through an example of using GitHub Actions to automate a data pipeline for a simple data analysis project.

  • Test Smells: Cleaning up E2E Tests

    In practical terms, knowing how not to write tests might be as important as knowing how to write them. There are some very helpful chapters on test smells in Gerard Meszaros's book about xUnit patterns — and more great stuff around the internet; however, it's always helpful to have practical examples for particular tech stacks. We've already shown how to clean up unit tests; this time, we'll do JUnit + Selenide end-to-end tests, at the top of the pyramid. We're assuming you're familiar with Selenide, but most stuff here is valid for other stacks, too.

  • AWS CDK: Infrastructure as Abstract Data Types, Part 3

    In this third part of our CDK series, the project cdk-quarkus-s3, in the same GIT repository, will be used to illustrate a couple of advanced Quarkus to AWS integration features, together with several tricks specific to RESTeasy which is, as everyone knows, the RedHat implementation of Jakarta REST specifications. Let's start by looking at the project's pom.xml file which drives the Maven build process. You'll see the following dependency:

  • AWS CDK: Infrastructure as Abstract Data Types, Part 3

    In this third part of our CDK series, the project cdk-quarkus-s3, in the same GIT repository, will be used to illustrate a couple of advanced Quarkus to AWS integration features, together with several tricks specific to RESTeasy which is, as everyone knows, the RedHat implementation of Jakarta REST specifications. Let's start by looking at the project's pom.xml file which drives the Maven build process. You'll see the following dependency:

  • Feature Flag Tools to Confidently Release New Features

    Feature flags offer an excellent way to quickly turn off and on product changes by enabling you to remove and add the code in the software quickly. Marketers or product managers can choose a time and moment to make a feature or function live to win that aha moment. The feature flags are helpful to various departments, including marketing, product, testing, CROs, and development. The number of feature flags can rise quickly as the team realizes their helpfulness and begins to utilize them. To avoid the mismanagement it may create, you need feature flag platforms. A comprehensive space where you can place all your feature flags and manage, modify, and delete them.

  • AWS CDK: Infrastructure as Abstract Data Types, Part 2

    If you're a Java software developer and you weren't living on the planet Mars during these last years, then you certainly know what Quarkus is. And just in case you don't, you may find it out here. With Quarkus, the field of enterprise cloud-native applications development has never been so comfortable and it never took advantage of such a friendly and professional working environment. The Internet abounds with posts and articles explaining why and how Quarkus is a must for the enterprise, cloud-native software developer. And of course, CDK applications aren't on the sidelines: on the opposite, they can greatly take advantage of the Quarkus features to become smaller, faster, and more aligned with requirements nowadays.

  • How To Change an Ansible Namespace With the FQCN Migration Tool

    Packaging Ansible Playbooks within a collection is the best way to distribute reusable automation content. To avoid naming conflicts, developers organize collections inside namespaces. Sometimes situations arise where you need to migrate a collection from one namespace to another, such as a personal or community collection graduating to a more well-known or certified namespace. Altering the namespace can be a tedious task. However, the Fully Qualified Collection Name (FQCN) migration tool simplifies this process by utilizing the fqcn_migration command. Employing a straightforward configuration file transforms an entire collection from one namespace to another. This article introduces the tool and demonstrates how to use it.

  • API Versioning in Microservices Architecture

    API versioning is a technique used to manage changes to an API over time, ensuring that different versions of the API can coexist and be used simultaneously. This is crucial for maintaining compatibility with various clients that may rely on different versions of the API. Why API Versioning Is Important for Software Development Backward Compatibility Ensures that existing clients do not break when changes are made to the API. Allows clients to continue using the old version while new features are added in new versions. Controlled Changes Enables developers to introduce new features, bug fixes, and improvements incrementally. Reduces the risk of introducing breaking changes that could disrupt existing services. Flexibility Clients can choose when to upgrade to a newer version, allowing them to adapt at their own pace. Different clients can use different versions of the API based on their needs. Clear Communication Provides a clear mechanism to communicate changes and updates to clients. Helps in setting expectations about which versions are supported and for how long. Stability Maintains a stable and reliable user experience by ensuring that changes do not disrupt existing functionality.

  • Datafaker Gen: Leveraging BigQuery Sink on Google Cloud Platform

    This is a continuation of the article Flexible Data Generation With Datafaker Gen about DataFaker Gen. In this section, we will explore the new BigQuery Sink feature for Google Cloud Platform, demonstrating how to utilize different field types based on the DataFaker schema. BigQuery is a fully managed and AI-ready data analytics platform available on Google Cloud Platform that gives anyone the capability to analyze terabytes of data.

  • Argo Rollout and Deployment Strategies

    Argo Rollouts is a Kubernetes controller and a part of the Argo project that provides advanced deployment capabilities for Kubernetes. It enhances the deployment process by offering advanced strategies such as blue-green deployments, canary releases, and experimentation. Argo Rollouts allows you to manage the release process of your applications more effectively, ensuring minimal downtime and better control over how updates are rolled out. Argo Rollouts (optionally) integrates with ingress controllers and service meshes, leveraging their traffic shaping abilities to gradually shift traffic to the new version during an update. Additionally, Rollouts can query and interpret metrics from various providers to verify key KPIs and drive automated promotion or rollback during an update.

  • Argo Rollout and Deployment Strategies

    Argo Rollouts is a Kubernetes controller and a part of the Argo project that provides advanced deployment capabilities for Kubernetes. It enhances the deployment process by offering advanced strategies such as blue-green deployments, canary releases, and experimentation. Argo Rollouts allows you to manage the release process of your applications more effectively, ensuring minimal downtime and better control over how updates are rolled out. Argo Rollouts (optionally) integrates with ingress controllers and service meshes, leveraging their traffic shaping abilities to gradually shift traffic to the new version during an update. Additionally, Rollouts can query and interpret metrics from various providers to verify key KPIs and drive automated promotion or rollback during an update.

  • Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

    With Spring Boot 3.2 and Spring Framework 6.1, we get support for Coordinated Restore at Checkpoint (CRaC), a mechanism that enables Java applications to start up faster. With Spring Boot, we can use CRaC in a simplified way, known as Automatic Checkpoint/Restore at startup. Even though not as powerful as the standard way of using CRaC, this blog post will show an example where the Spring Boot applications startup time is decreased by 90%. The sample applications are from chapter 6 in my book on building microservices with Spring Boot. Overview The blog post is divided into the following sections:

  • Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

    With Spring Boot 3.2 and Spring Framework 6.1, we get support for Coordinated Restore at Checkpoint (CRaC), a mechanism that enables Java applications to start up faster. With Spring Boot, we can use CRaC in a simplified way, known as Automatic Checkpoint/Restore at startup. Even though not as powerful as the standard way of using CRaC, this blog post will show an example where the Spring Boot applications startup time is decreased by 90%. The sample applications are from chapter 6 in my book on building microservices with Spring Boot. Overview The blog post is divided into the following sections:

  • Simplifying Developer Workflows: How Effective Backup Strategy Reduces Cognitive Load

    It’s easy to imagine the burden that you, as a developer, can feel rushing to perform your tasks quickly, sometimes forgetting about the amount of confusion you can feel by reading and producing the code fast. This confusion can cost both time and money and have an awful impact on the project you work on. This state of confusion, which takes place when a developer faces an overwhelming amount of information and multitasking, is not an imaginary sandcastle. This mental state is called cognitive overload. Increased forgetfulness, lack of focus, hampered creative thinking and innovation, and difficulties in learning new concepts are all symptoms of cognitive overload. What can you do to reduce it and have peace of mind?

  • 7 Essential Tips for a Production ClickHouse Cluster

    ClickHouse is the fastest, most resource-efficient OLAP database which can query billions of rows in milliseconds and is trusted by thousands of companies for real-time analytics. Here are seven tips to help you spin up a production ClickHouse cluster and avoid the most common mistakes.

  • 7 Essential Tips for a Production ClickHouse Cluster

    ClickHouse is the fastest, most resource-efficient OLAP database which can query billions of rows in milliseconds and is trusted by thousands of companies for real-time analytics. Here are seven tips to help you spin up a production ClickHouse cluster and avoid the most common mistakes.

  • API Implementation on AWS Serverless Architecture

    This article describes the implementation of RESTful API on AWS serverless architecture. It provides a detailed overview of the architecture, data flow, and AWS services that can be used. This article also describes the benefits of the serverless architecture over the traditional approach. What Is Serverless Architecture? Serverless architecture, also known as serverless computing or function as a service, is a software design approach that allows developers to build and run applications without managing the underlying infrastructure. A cloud service provider is responsible for managing and scaling the cloud infrastructure, including provisioning servers to run applications, databases, and storage.

  • API Implementation on AWS Serverless Architecture

    This article describes the implementation of RESTful API on AWS serverless architecture. It provides a detailed overview of the architecture, data flow, and AWS services that can be used. This article also describes the benefits of the serverless architecture over the traditional approach. What Is Serverless Architecture? Serverless architecture, also known as serverless computing or function as a service, is a software design approach that allows developers to build and run applications without managing the underlying infrastructure. A cloud service provider is responsible for managing and scaling the cloud infrastructure, including provisioning servers to run applications, databases, and storage.

  • A Hands-On Guide To OpenTelemetry: Intro to Observability

    Are you ready to start your journey on the road to collecting telemetry data from your applications? Great observability begins with great instrumentation! In this series, you'll explore how to adopt OpenTelemetry (OTel) and how to instrument an application to collect tracing telemetry. You'll learn how to leverage out-of-the-box automatic instrumentation tools and understand when it's necessary to explore more advanced manual instrumentation for your applications. By the end of this series, you'll have an understanding of how telemetry travels from your applications to the OpenTelemetry Collector, and be ready to bring OpenTelemetry to your future projects. Everything discussed here is supported by a hands-on, self-paced workshop authored by Paige Cruze.

  • Salesforce Bulk API 2.0: Streamlining Large-Scale Data Operations

    Have you ever faced the challenge of managing large data operations within Salesforce, such as updating, inserting, deleting, or querying records? These operations might arise from one-time data migration projects or ongoing data integration needs with external systems. In such scenarios, Salesforce Bulk API 2.0 is your solution. This robust API is designed to handle large-scale data operations efficiently, simplifying and streamlining the process. Salesforce Bulk API 2.0 is a REST-based API, that runs the requested operation asynchronously. It is designed to simplify bulk operations within Salesforce as well as to improve the performance and reliability of bulk data processing. The following are key features of Bulk API 2.0. 

  • Viking Enterprise Solutions: Empowering Modern Data Infrastructure

    In today's rapidly evolving technological landscape, developers, engineers, and architects face unprecedented challenges in managing, processing, and deriving value from vast amounts of data. Viking Enterprise Solutions (VES), a division of Sanmina Corporation, stands at the forefront of addressing these challenges with its innovative hardware and software solutions. This article explores how VES can help professionals in their roles, focusing on key areas of expertise and product offerings based on their recent presentation to the 56th IT Press Tour. Bridging Hardware and Software for Optimal Performance One of VES's core strengths lies in its ability to bridge the gap between hardware and software, creating solutions that maximize performance and efficiency. As a product division of Sanmina, a $9 billion public company, VES leverages decades of manufacturing expertise to deliver cutting-edge data center solutions.

  • How DevSecOps Can Combat Zero-Day Threats

    Zero-day threats are becoming more dangerous than ever. Recently, bad actors have taken over the TikTok accounts of celebrities and brands through a zero-day hack. In late May to early June, reports of high-profile TikTok users losing control over their accounts started to surface after opening a direct message. The malware used for the attack was able to infect devices without the users downloading or installing anything. TikTok appeared unaware of the extent of the damage. The company’s spokesperson, Alex Haurek, said that the number of accounts compromised was “very small,” but he also declined to provide a specific number. He said they have been working with the owners of the affected accounts to restore access and that they have implemented measures to make sure the problem does not happen again.

  • Mitigate the Security Challenges of Telecom 5G IoT Microservice Pods Architecture Using Istio

    Deploying microservices in a Kubernetes cluster is critical in 5G Telecom. However, it also introduces significant security risks. While firewall rules and proxies provide initial security, the default communication mechanisms within Kubernetes, such as unencrypted network traffic and lack of access control, are inherently insecure. This insecurity could compromise sensitive data. Therefore, implementing additional security measures within each microservice pod is not just a recommendation but a crucial step to ensure secure communication within the cluster. So, additional configuration inside each application is needed.  Istio provides a robust solution to these challenges by effectively managing communication between individual 5G telecom microservice pods. With its control plane, Istio automatically injects a sidecar proxy into individual microservices pods, ensuring secure and efficient communication.

  • Master AWS IAM Role Configuration With Terraform

    It's super important today to keep things secure and make sure everything is running as it should. AWS Identity and Access Management (IAM) helps with this by letting you manage who can get into what parts of your AWS account. One cool thing about IAM is that it lets you give permissions to different parts or people in your account without having to share sensitive info like passwords. Today, I'm going to talk about using Terraform, a tool that lets you set up infrastructure through code, to create and set up these IAM roles easily. Understanding AWS IAM Roles and Terraform Before we get into how to use Terraform for setting up IAM roles in AWS, it's key to grasp what AWS IAM roles and Terraform are all about. In your AWS account, you can create IAM roles, which are basically identities with certain permissions attached. These roles let you give specific rights to different parts of your AWS setup without any hassle. On the flip side, Terraform is a tool that lets you manage your infrastructure through code instead of doing everything manually. It’s made specifically for working smoothly with services such as those offered by AWS, thanks to the Terraform AWS provider.

  • Master AWS IAM Role Configuration With Terraform

    It's super important today to keep things secure and make sure everything is running as it should. AWS Identity and Access Management (IAM) helps with this by letting you manage who can get into what parts of your AWS account. One cool thing about IAM is that it lets you give permissions to different parts or people in your account without having to share sensitive info like passwords. Today, I'm going to talk about using Terraform, a tool that lets you set up infrastructure through code, to create and set up these IAM roles easily. Understanding AWS IAM Roles and Terraform Before we get into how to use Terraform for setting up IAM roles in AWS, it's key to grasp what AWS IAM roles and Terraform are all about. In your AWS account, you can create IAM roles, which are basically identities with certain permissions attached. These roles let you give specific rights to different parts of your AWS setup without any hassle. On the flip side, Terraform is a tool that lets you manage your infrastructure through code instead of doing everything manually. It’s made specifically for working smoothly with services such as those offered by AWS, thanks to the Terraform AWS provider.

  • Enhancing Security With ZTNA in Hybrid and Multi-Cloud Deployments

    Today’s network infrastructure is rapidly changing with the adoption of hybrid and multi-cloud architectures to leverage the benefits of flexibility, scalability, and redundancy. These advantages come with their own set of challenges, particularly in securing access to resources and users spread across cloud environments. Zero Trust Network Access (ZTNA) ensures secure connectivity in diverse and complex infrastructures. Understanding ZTNA in Hybrid and Multi-Cloud Environments The Zero Trust Network Access (ZTNA) marks a major shift from the traditional perimeter-based security which treats traffic as untrustworthy, irrespective of its source. Network segmentation is not the only reliable method of providing security. ZTNA checks every access request based on user identity, device security posture, and contextual factors before granting access to applications and data.

  • Mastering Serverless Debugging

    Serverless computing has emerged as a transformative approach to deploying and managing applications. The theory is that by abstracting away the underlying infrastructure, developers can focus solely on writing code. While the benefits are clear—scalability, cost efficiency, and performance—debugging serverless applications presents unique challenges. This post explores effective strategies for debugging serverless applications, particularly focusing on AWS Lambda. Before I proceed I think it's important to disclose a bias: I am personally not a huge fan of Serverless or PaaS after I was burned badly by PaaS in the past. However, some smart people like Adam swear by it so I should keep an open mind.

  • Mastering Serverless Debugging

    Serverless computing has emerged as a transformative approach to deploying and managing applications. The theory is that by abstracting away the underlying infrastructure, developers can focus solely on writing code. While the benefits are clear—scalability, cost efficiency, and performance—debugging serverless applications presents unique challenges. This post explores effective strategies for debugging serverless applications, particularly focusing on AWS Lambda. Before I proceed I think it's important to disclose a bias: I am personally not a huge fan of Serverless or PaaS after I was burned badly by PaaS in the past. However, some smart people like Adam swear by it so I should keep an open mind.

  • Writing a Simple Pulumi Provider for Airbyte

    In this quick tutorial, I will go over Pulumi by writing a very simple Provider for Airbyte. The simplified use case is as follows: I want to programmatically (Infrastructure as Code) create, update, and delete Sources, Destinations, and Connections in Airbyte instead of using the UI.

  • Writing a Simple Pulumi Provider for Airbyte

    In this quick tutorial, I will go over Pulumi by writing a very simple Provider for Airbyte. The simplified use case is as follows: I want to programmatically (Infrastructure as Code) create, update, and delete Sources, Destinations, and Connections in Airbyte instead of using the UI.

  • AWS: Metric Filter vs. Subscription Filter

    In this blog on AWS, let’s do a comparison study between two filter tools available with Amazon CloudWatch Logs — Metric Filter and Subscription Filter, which play a crucial role in log data management, allowing you to analyze, monitor, and act on log data effectively. Metric Filter Metric Filters define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. CloudWatch Logs uses these metric filters to turn log data into numerical CloudWatch metrics that you can graph or set an alarm on. For eg., you can create a metric to count the occurrences of the word “ERROR” in your logs & set an alarm if the count goes beyond a certain threshold.

  • AWS: Metric Filter vs. Subscription Filter

    In this blog on AWS, let’s do a comparison study between two filter tools available with Amazon CloudWatch Logs — Metric Filter and Subscription Filter, which play a crucial role in log data management, allowing you to analyze, monitor, and act on log data effectively. Metric Filter Metric Filters define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. CloudWatch Logs uses these metric filters to turn log data into numerical CloudWatch metrics that you can graph or set an alarm on. For eg., you can create a metric to count the occurrences of the word “ERROR” in your logs & set an alarm if the count goes beyond a certain threshold.

  • The Rise of Kubernetes: Reshaping the Future of Application Development

    Kubernetes has emerged as the de facto standard for container orchestration, revolutionizing how developers build, deploy, and manage applications. A recent report by Pure Storage's Portworx division reveals that 80% of respondents plan to build most of their new applications on cloud-native platforms in the next five years. This shift is not just a trend; it's a fundamental change in the way we approach software development and infrastructure management. The Kubernetes Advantage for Developers Kalyan Ramanathan, VP of Marketing at Portworx, emphasizes that Kubernetes is built with developers in mind. It offers three key advantages:

  • The Rise of Kubernetes: Reshaping the Future of Application Development

    Kubernetes has emerged as the de facto standard for container orchestration, revolutionizing how developers build, deploy, and manage applications. A recent report by Pure Storage's Portworx division reveals that 80% of respondents plan to build most of their new applications on cloud-native platforms in the next five years. This shift is not just a trend; it's a fundamental change in the way we approach software development and infrastructure management. The Kubernetes Advantage for Developers Kalyan Ramanathan, VP of Marketing at Portworx, emphasizes that Kubernetes is built with developers in mind. It offers three key advantages:

  • Build an Advanced RAG App: Query Rewriting

    In the last article, I established the basic architecture for a basic RAG app. In case you missed that, I recommend that you first read that article. That will set the base from which we can improve our RAG system. Also in that last article, I listed some common pitfalls that RAG applications tend to fail on. We will be tackling some of them with some advanced techniques in this article. To recap, a basic RAG app uses a separate knowledge base that aids the LLM in answering the user’s questions by providing it with more context. This is also called a retrieve-then-read approach.

  • Enhance IaC Security With Mend Scans

    Whether on the cloud or setting up your AIOps pipeline, automation has simplified the setup, configuration, and installation of your deployment. Infrastructure as Code(IaC) especially plays an important role in setting up the infrastructure. With IaC tools, you will be able to describe the configuration and state of your infrastructure that are desirable. The popular tools for IaC include Terraform, Pulumi, AWS CloudFormation, and Ansible; each of them has different possibilities for automating the deployment and management of infrastructure both in the cloud and on-premises. With the growing complexity of applications and heightened focus on security in software development, the tools SonarQube and Mend are more predisposed. As explained in my previous article, SonarQube is a code analysis tool aimed at helping developers have high-quality code by spotting bugs and vulnerabilities across several programming languages. SonarQube is very well integrated into pipelines of Continuous Integration/Continuous Deployment, producing continuous feedback while forcing enforcement of coding standards.

  • Enhance IaC Security With Mend Scans

    Whether on the cloud or setting up your AIOps pipeline, automation has simplified the setup, configuration, and installation of your deployment. Infrastructure as Code(IaC) especially plays an important role in setting up the infrastructure. With IaC tools, you will be able to describe the configuration and state of your infrastructure that are desirable. The popular tools for IaC include Terraform, Pulumi, AWS CloudFormation, and Ansible; each of them has different possibilities for automating the deployment and management of infrastructure both in the cloud and on-premises. With the growing complexity of applications and heightened focus on security in software development, the tools SonarQube and Mend are more predisposed. As explained in my previous article, SonarQube is a code analysis tool aimed at helping developers have high-quality code by spotting bugs and vulnerabilities across several programming languages. SonarQube is very well integrated into pipelines of Continuous Integration/Continuous Deployment, producing continuous feedback while forcing enforcement of coding standards.