I have seen articles discussing Apache Airflow and its many capabilities. It’s crucial to understand production-quality data pipelines meant to “handle” terabytes of daily data generated by the enterprise’s software-as-a-service (SaaS) applications. The article takes you beyond the basic introductory stuff and on to more advanced techniques and best practices for developing scalable, fault-tolerant, and […]
The Bottleneck Our systems are constantly generating high-volume transactional events. In our case, these events are funneled through Kafka and ingested into InfluxDB. Each event includes details such as timestamps, categories, and other metadata. Initially, this architecture supported our analytical needs well. We used InfluxDB to store these metrics and performed queries to generate category-wise […]
Like many of you, I have been playing around with Model Context Protocol (MCP). To dive in, I built a sample MCP server implementation for Azure Cosmos DB with Go. It uses the Go SDK, and mcp-go as the MCP Go implementation.
Cloud computing has reshaped how businesses operate, offering unmatched scalability, flexibility, and cost-efficiency. However, as organizations continue to shift critical operations to the cloud, they face escalating cybersecurity challenges. Traditional security systems often struggle to protect complex, interconnected cloud environments from increasingly sophisticated cyberattacks. Artificial Intelligence (AI) has emerged as the ultimate game-changer in cloud […]
This technical guide outlines the steps to set up data replication using Google Cloud Datastream. Specifically, it details the process of setting up data replication from an Oracle 19c database hosted on a Google Compute Engine virtual machine into Google BigQuery. The tutorial covers all necessary steps, including prerequisites—enabling APIs and configuring firewalls, setting up […]
This technical guide outlines the steps to set up data replication using Google Cloud Datastream. Specifically, it details the process of setting up data replication from an Oracle 19c database hosted on a Google Compute Engine virtual machine into Google BigQuery. The tutorial covers all necessary steps, including prerequisites—enabling APIs and configuring firewalls, setting up […]
In the field of big data, different database systems often use different SQL dialects. This is similar to people from different regions speaking different languages, which brings great trouble to data analysts and developers. When an enterprise needs to integrate multiple data sources for analysis, it may have to spend a great deal of time […]
As AI-powered applications like chatbots and virtual assistants become increasingly integrated into our daily lives, ensuring that they interact with users in a safe, respectful, and responsible manner is more important than ever. Unchecked user input or AI-generated content can lead to the spread of harmful language, including hate speech, sexually explicit content, or content […]
I’ve spent years building data pipelines and connecting project management to technical workflows. Disconnected systems lead to manual errors and delays, problems that Jira’s API helps solve. This tool lets code interact directly with project boards, automating tasks such as creating tickets when data checks fail or updating statuses after ETL.
Successful software development hinges on maintaining a balance between speed and quality. To stay ahead, many organizations are progressively adopting a shift-left approach. Rather than waiting until the end to catch bugs, this strategy emphasizes conducting quality checks and testing much earlier in the development process. One crucial aspect of this approach is input validation—ensuring […]