Mayank's Blog

Unit testing AWS Glue jobs presents challenges due to the complexities involved in replicating the Glue environment locally. Fortunately, AWS offers a solution in the form of Glue container images available at Glue container images . These images allow us to perform unit tests effectively, as outlined in detail in the official documentation here . In this blog post, we will delve into the process of running AWS Glue job unit tests within an Azure DevOps pipeline and discuss how to calculate and publish code coverage for these tests. To begin with, the Glue container image operates under a special user named GLUE_USER , which is referenced in the associated dockerfile . USER glue_user Assuming you have developed your Glue job in a Python script named myawesomegluejob.py , which is stored in an Azure DevOps (AzDO) Git repository, creating a pipeline for this purpose might initially seem straightforward. However, executing build steps directly within the Glue container is not feasible

Generative AI is a type of artificial intelligence that can create new content such as text, images, music, etc. in response to prompts. Generative AI models learn the patterns and structure of their input training data by applying neural network machine learning techniques, and then generate new data that has similar characteristics. They are all the rage these days. 😀 Some types of generative AI include: Foundation models , which are complex machine learning systems trained on vast quantities of data (text, images, audio or a mix of data types) on a massive scale. Foundation models can be adapted quickly for a wide range of downstream tasks without needing task-specific training. Examples of foundation models are GPT, LaMDA and Llama . Generative adversarial networks (GANs) , which are composed of two competing neural networks: a generator that creates fake data and a discriminator that tries to distinguish between real and fake data. The generator improves its ability to fool the d

 Azure Application Insights is frequently being used by workloads running on Azure. The most simple and common example of this is a Web Application calling a RESTful API. In most enterprise-y scenario, one would find it to be a single page app commonly written in React/Angular talking to a .NET/Java Web API. However a surprising number of developers don't use or haven't heard of telemetry correlation in Application Insights. Azure SDK provides a robust mechanism of enabling telemetry correlation for various languages and frameworks. For .NET Core, this is turned on by default and you don't have to do any thing except adding  Microsoft.ApplicationInsights.AspNetCore nuget package and including the following line in your code: builder.Services.AddApplicationInsightsTelemetry(); For other frameworks, it is not as simple. Here I am going to show an example for a React app. This app is bootstrapped using Create React App (CRA) tool. Next create a file AppInsights.js and add

For the past year and half, Covid-19 pandemic has forced most of the companies to allow their employees to work from home. However, now that vaccines are available, companies are looking for a way to get their employees back to office safely. Most of the organizations are creating some kind of app which will capture employee health information safely and allow them to get approve/deny an employee's return to office. But what if instead of rolling out your own platform, you could leverage the full power of Azure with almost no code and also ensure full compliance with health data regulations. Meet Azure Health Bot. Azure Health Bot is an Azure Marketplace offering by Microsoft built on top of Microsoft Bot Platform. It provides features like drag/drop editor, authentication with OAuth providers, generating reports and managing user information. It also integrates with existing chat apps like WhatsApp, Telegram, Facebook Messenger etc. so that users don't have to install a new ap

While working on a React based static website recently, there was a need to see exactly what was deployed in the Dev/Test environments to reduce confusion amongst teams. I wanted to show something like this: A quick look at the site's footer should show the Git Commit Hash and Build Number which was deployed and click through to actual commits and build results. Let's see how we achieved this using Azure DevOps. Git Commit Hash Azure DevOps exposes a variable called  $(Build.SourceVersion) which contains the hash of the commit. So I defined a variable in the Build Pipeline using it. Build Id and Build Number Azure DevOps also exposes two release time variables  $(Build.BuildId) and  $(Build.BuildNumber) which can be used to define custom variables in the pipeline. So we have a total of 3 variables defined: Next we use these variables in our React App. I created 3 global variables in index.html and assigned a token value to them. < script   type = "text/JavaScript&quo

Logging is one of the core pillars of application development. .NET Core has fantastic support for rich contextual logging which spans across distributed system using its Activity API . Azure Application Insights SDK offers extensive logging out of the box without writing a single line of code. Just wire up the SDK and you are good to go. However there may be situations where you may want more fine grained control over the logging experience. Fortunately, its as simple as writing: Activity activity = new Activity("Test Message"); var operation = telemetryClient.StartOperation<DependencyTelemetry>(activity); This TelemetryClient is injected in your application when you wire up Application Insights. services.AddApplicationInsightsTelemetry(options => options.InstrumentationKey = "YOUR_AI_KEY"); This works seamlessly till you hit the road block of Domain Driven Design and Entity Framework Core. In DDD, entities represent a real-world business object and perf