AWS Server Types

One of the key question for cloud computing - what is the right size of cloud server type for my business?

We can do any kind of reservation; but what is the right/efficient way to do?

Cloud server reservation should be with the right balance of cost, server usage, billing type, performance, etc.  To make it simple, I'm going to explain with hotel booking use case.

Amazon Cloud AWS has 4 major types of server reservation.  They are

1. On demand
2. Reserved
3. Spot
4. Dedicated

On demand

• if we want to come and stay in any hotel with full price, it fits here.  User doesn't bother about the deal for the booking.
• In AWS cloud, it fits for short-term uninterrupted critical work.

Reserved

• it refers to the well planned stay for a long time.  Here, user analyses lot of discount options to book in advance
• Technically, it fits for any enterprise with long term cloud usage.

Spot

• it is about the bidding for the room in last minute, can kick out the booking at any time with the competitive booking price
• In AWS, it fits for the tasks bound to resilence of failure

 Dedicated

• booking the entire hotel.  Assume the user wants to accommodate the entire family/friends circle for the wedding ceremony; ultimately books the entire hotel without allowing others to join on those days
• In AWS tech space, it fits to share the enterprise licenses like BYOL (Bring Your Own License) along with infra privacy act.

Each option has a purpose and associated cost.  Of course, Dedicated mode is expensive; but family/friends privacy is guaranteed.  In the same way, AWS Dedicated model is physically dedicated to one user account with full control till network.

Tech Ref: https://www.virtana.com/blog/demystifying-terminology-aws-instances/

Merry Christmas and Happy New Year 2021 !

FSx in AWS Gov Cloud

 Amazon FSx, is now available in the AWS GovCloud (US) Regions.  FSx is a fully managed service that makes it easy to launch and run feature-rich and highly-performant file systems.

Amazon FSx provides two file systems to choose from: Amazon FSx for Windows File Server and Amazon FSx for Lustre. 

Amazon FSx for Windows File Server provides fully managed, highly reliable file storage that is accessible over the industry-standard Server Message Block (SMB) protocol. 

Amazon FSx for Lustre provides a high-performance file system optimized for fast processing of workloads such as machine learning, high performance computing (HPC), video processing, financial modeling, and electronic design automation (EDA).  

Amazon Location Service

This week, AWS is making Amazon Location available in preview form.  Amazon Location Service gives you access to maps and location-based services from multiple providers on an economical, pay-as-you-go basis.

This new service can display maps, validate addresses, perform geocoding (turn an address into a location), track the movement of packages and devices, and much more.

It easily set up geofences and receive notifications when tracked items enter or leave a geofenced area. Also, it can even overlay your own data on the map while retaining full control.

 Ref: https://aws.amazon.com/location/ 

Google is down

 Google was continue to down for last few minutes across the world (mostly).  

It was a breaking news live at Times of India too! Ref: https://timesofindia.indiatimes.com/india/breaking-news-live-december-14/liveblog/79713778.cms?utm_campaign=andapp&utm_medium=referral&utm_source=native_share_tray

It reflected how tightly coupled with them.  Google suffered major outage, which disrupted most of google services like GMail, YouTube, GMeet, etc.  Google search seems stable.  The services were down, starting at 11:56 UTC for about 38 minutes; then it got recovered.

Last century, electricity is technology fuel for every day life; now internet and google are mandatory for everyone.  What a transformation in global technology!

AWS mac

 This week, it's an exciting news in AWS re:Invent- Week 2.

Mac instances enable customers to run on-demand macOS workloads in the cloud for the first time, extending the flexibility, scalability, and cost benefits of AWS to all Apple developers.

Customers who rely on the Xcode IDE for creating iPhone, iPad, Mac, Apple Watch, Apple TV, and Safari apps can now provision and access macOS environments within minutes with simple mouse clicks or API calls, dynamically scale capacity as needed, and benefit from AWS’s pay-as-you-go pricing.

Amazon EC2 Mac instances are built on Mac mini computers, and offer customers a choice of both the macOS Mojave (10.14) and macOS Catalina (10.15) versions.

AWS Proton

 AWS Proton is the first fully managed application deployment service for container and serverless applications. Platform engineering teams can use Proton to connect and coordinate all the different tools needed for infrastructure provisioning, code deployments, monitoring, and updates.

It solves this by giving platform teams the tools they need to manage this complexity and enforce consistent standards, while making it easy for developers to deploy their code using containers and serverless technologies.

450 mark

 

Today I reached a milestone worth celebrating!   On checking the blog scorecard, noticed I had hit 450! 

That's ~143k hits, 30+ followers, 320 accolades in last 10+ consistent years effort.

Three key moves to keep it going are

1. Passion for continuous learning and sharing
2. Self challenge to motivate every day experience
3. Consistency to work hard with your support

The next big milestone is 500+ mark.

As Gandhi said "Live as if you were to die tomorrow; Learn as if you were to live forever"

CloudFormation nested stack

This week, got a great tech info on release of the long awaited feature in CloudFormation nested stacks.

AWS CloudFormation is extending change sets to support applications modeled with nested stacks, enhancing the predictability of update operations. With this launch, you can now preview the changes to your application and infrastructure resources across the entire nested stack hierarchy and proceed with the update only when you confirm that all the changes are as intended.  

When you create a change set with this feature enabled, CloudFormation automatically creates a change set for each nested stack contained in the hierarchy. Each of these change sets contains changes to the resources contained in the stack as well as links to change sets of stacks nested within it. 

Using these links for navigation, you can view the changes across the entire hierarchy. After examining all the changes, you can either choose to proceed with the create or update operation or start over.

AWS Hyderabad

 

A notable exception to this policy is geographic expansion of AWS regions list.

Early this month, AWS per-announced a new region in Zurich, with an eye towards it launching in late 2022, and a second new region in Hyderabad coming in mid-2022.

AWS’s global infrastructure page exhibits the latest stack list. It consists of multiple physically separate availability zones, which are one or more data centers.

With few billions dollars, each region is built with massive investments in real estate, computer hardware, networking, and hiring. Their Hyderabad region apparently resembles a $2.8 billion investment.

Whenever AWS announces a region, they aren’t doing because they think it’d be neat to have a region in a given place.  It's purely driven by customer demand.

.NET 5

Microsoft's .NET 5 developer platform, key to the company's work to unify its different .NET flavors, is generally available as of yesterday, November 10.

In 2019, Microsoft announced that .NET 5 would be its first major deliverable on the road toward unifying .NET.  Ref: https://devblogs.microsoft.com/dotnet/introducing-net-5/

.NET 5 includes C# 9, F# 5, and the corresponding ASP.NET Core release. Microsoft is encouraging developers to use new application programming interfaces (APIs) in .NET 5 to integrate with other tools like its Power Platform low-code/no-code platform. It also provides native (not just emulated) support for Windows Arm64, as well as performance improvements in the .NET libraries and Garbage Collector; smaller container images; and single-file applications.

We knew that Microsoft adopted open source strategy for last 3+ years using .NET Core  .NET 5 is the next step forward with .NET Core. The project aims to improve .NET in a few key ways:

1. Produce a single .NET runtime and framework that can be used everywhere and that has uniform runtime behaviors and developer experiences.
2. Expand the capabilities of .NET by taking the best of .NET Core, .NET Framework, Xamarin and Mono.
3. Build that product out of a single code-base that developers (Microsoft and the community) can work on and expand together and that improves all scenarios.

Feel free to download and enjoy .NET 5 at https://dotnet.microsoft.com/download/dotnet/5.0

More fun to come with the attached product roadmap from Microsoft.  As of now, they're on track. Kudos to their great effort !

Metrics Explorer

 

Amazon CloudWatch launches Metrics Explorer – a tag-based dashboard tool that enables customers to filter, aggregate, and visualize operational health and performance metrics by tags.

On top of effective metric driven monitoring, the performance graphs will be useful to continuously monitor your application health and troubleshoot issues when they occur.

Metrics Explorer enables you to slice and dice metrics using existing AWS Resource Tags or using EC2 and Lambda resource properties as Tags. It aggregates the metrics by resource properties such as instance type and visualize one graph per Availability Zone or organize metrics on one or multiple graphs by splitting them based on tags.

default REST API

Amazon API Gateway now supports disabling the default, auto-generated REST API endpoint. The default REST API endpoint in API Gateway looks like https://{restapi_id}.execute-api.{region}.amazonaws.com. 

This feature is intended for customers who use custom domain names for REST APIs and want to ensure that all traffic to their API only goes through the custom domain name and not the default endpoint. This feature was already available for HTTP APIs. Now, it is available for REST APIs too.

Customers can configure a custom domain name endpoint to require different security than the default endpoint generated by API Gateway. For example, you can choose to configure a custom domain name with mutual TLS authentication, or require that traffic to a custom domain name use TLS 1.2. 

By disabling the default API endpoint, customers can require that clients access an API through only the custom domain name. Using this strategy, customers can ensure that clients are only accessing backend systems through approved security controls configured on the custom domain name.

Augmented Reality

 

30 years ago, I studied a paper titled "Artificial Intelligence" with the belief in reality by my grand kids time-frame.

You know what, now it's real around the world.  One of the classic event was IIT Madras 57th virtual convocation with Augmented Reality in real.

1 minute Video Ref: https://www.youtube.com/watch?v=xe79RbOzKnw&list=UUZgwOAjweG6NZKX5VWXLQLw&index=1

Victory comes from finding opportunities in problems.  Covid virus induces few innovative solutions in the world like this.

Amazing Artificial Intelligence at IIT Madras !

AWS Architecture Monthly

Each month, AWS Architecture Monthly team chooses a theme and curates the best content about architecting in the AWS Cloud. 

Their goal is to provide you with a broad selection of the best new technical content from AWS, from deep-dive tutorials and whitepapers to customer videos and trending articles. They also interview industry experts who provide unique perspectives about the month’s theme and its related AWS services and solutions.

How to access the magazine

1. View and download past issues as PDFs on the AWS Architecture Monthly webpage.
2. Readers in the US, UK, Germany, and France can subscribe to the Kindle version of the magazine at Kindle Newsstand.
3. Visit Flipboard, a personalized mobile magazine app that you can also read on your computer.

Happy reading & architecting !

AWS Solutions Constructs

AWS Solutions Constructs enable customers to build faster and more confidently with a
growing repository of vetted architecture patterns, spanning the most frequently used
combinations of services on the platform. Solutions Constructs are higher-level abstractions of AWS Cloud Development Kit (CDK) constructs, and have a multi-service focus to make assembling production-ready workloads possible in fewer steps. 

For example, a single Solutions Construct can be used to provision an AWS Lambda function, an Amazon DynamoDB table, and any permissions needed for the function to perform read/write operations on the table. This enables customers to cut down on configuring individual CDK constructs to achieve the same result.

While all Solutions Constructs are fully customizable, each comes pre-configured with best practice defaults to help customers ensure that their workload, as a whole, is wellarchitected. This gives customers the ultimate flexibility to use a particular Solutions Construct off-the-shelf, with zero configuration. Or, they can fine-tune it to meet the unique needs of their solution. Regardless of the decision, customers are given the confidence and power to build knowing that all of their bases are covered from the start.

Solutions Constructs are designed to be completely frictionless, and each Solutions
Construct can be used alongside others, as well as CDK constructs.

Inspire Ignite Leader

 

It is birth week for one of the greatest inspiration, clean politician, ex-president of India, rocket scientist, motivational speaker, etc. and so minor deviation on my regular technical track.

He is none other than Dr. APJ Abdul Kalam. Ref: https://en.wikipedia.org/wiki/A._P._J._Abdul_Kalam

He has tons of living leadership illustrations throughout his life, with lead by example model.  On his birthday occasion, wanted to share a sample to transform the life of his own driver.
 

As depicted, his driver (passed only 9th grade) finished Doctorate (PhD) and now an assistant professor.  Big salute to Dr. V. Kathiresan, who got inspired and ignited by Dr. Kalam  
 

xUnit in AWS

xUnit.net is a free and open-source unit testing tool for the .NET Framework, written by the original author of NUnit. It is licensed under Apache License 2.0 and the source code is available on GitHub. xUnit.net works with Xamarin, ReSharper, CodeRush, and TestDriven.NET.

The best practice is to write repeatable and automated tests to ensure that your functions stay correct as you make changes to them using frameworks like xUnit.net. The programming model for Lambda is simple enough that testing is generally no different from testing your class libraries. Most of the Lambda blueprints provided in Visual Studio have the option to create an accompanying xUnit.net test project.

There are times when quick ad-hoc testing is necessary. To help with ad-hoc testing, we recently released the AWS .NET Mock Lambda Test Tool. This tool isn’t a local Lambda runtime. It’s a tool to quickly load up your Lambda function code and test your application logic. It provides functionality similar to the quick F5 debugging experience.

Because the tool is a .NET Core application that loads your code into its process, any IDE that can debug .NET Core can attach to this tool and enable you to debug your Lambda functions. The latest version of the AWS Toolkit for Visual Studio takes care of configuring the Lambda test tool as the debug target when you open a .NET Core Lambda project.

Code Artifact

AWS CodeArtifact is a fully managed artifact repository service that makes it easy for organizations to securely store and share software packages used for application development. You can use CodeArtifact with popular build tools and package managers such as Maven, Gradle, npm, yarn, pip, and twine.

AWS CodeArtifact uses the following elements:

• Asset – An individual file stored in AWS CodeArtifact that is associated with a package version, such as an npm .tgz file or Maven POM and JAR files
• Package – A package is a bundle of software and the metadata that is required to resolve dependencies and install the software. AWS CodeArtifact supports npm, PyPI, and Maven package formats.
• Repository – A CodeArtifact repository contains a set of package versions, each of which maps to a set of assets. Repositories are polyglot—a single repository can contain packages of any supported type. Each repository exposes endpoints for fetching and publishing packages using tools like the npm CLI, the Maven CLI (mvn), and pip.
• Domain – Repositories are aggregated into a higher-level entity known as a domain. The domain allows organizational policy to be applied across multiple repositories. A domain deduplicates storage of the repositories packages.

Thiw week, AWS CodeArtifact supports AWS CloudFormation, enabling customers to create and manage CodeArtifact repositories with CloudFormation.

AWS Free Models

 

On deep insight, AWS free tier has three distinct models.  Let us review them

Free Tier 1 - Always Free

This section is a baseline level, which is always free for the underlying services. As an example, it is possible to invoke 1 million Lambda invocations for free, also get “up to 3.2 million seconds of compute time” per month. Also, one million free requests per month and 400,000 GB-seconds of compute time per month.

Ref: https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc

Free Tier 2 - One Year Limit

This free tier is a 12-month period starting the day you first create your account. During this period, roughly 36 services offer you free usage up to certain limits. Few snapshots are:

• 30GB a month of free EBS (disk volume) storage
• 750 hours a month of t2 or t3 micro instance usage
• 1GB of data transfer out to the internet for free each month in perpetuity
• free t2.micro in regions where they’re available
• free 1 Oracle RDS instance but only if you bring your own license
• one free load balancer but it’s only an ELB-classic or an Application Load Balancer

Free Tier 3 - Trials

It is a service specific limited-time trial period that begins from your first use of a service. This applies to any use of the service, by any user in the account.

As an example, on enabling Amazon Detective, it’ll be free for 30 days; then it will be charged.

Google AppSheet

 

AppSheet is a powerful business productivity solution that lets you build powerful mobile apps with your spreadsheet data.  Tech theme is "Data to Device"

Customers will be able to develop richer applications at scale that leverage Google technologies including Google Sheets, Google Forms, Android, Maps and Google Analytics and that integrate with cloud-hosted data sources including Salesforce, Dropbox, AWS DynamoDB and MySQL

It is absolutely zero coding with the below nine steps process:

1. Prepare your data.
2. Connect your data to AppSheet to create an app.
3. Refine how your app uses your data.
4. Refine how your app presents your data.
5. Add actions, workflows, and reports to your app.
6. Define your app's security settings.
7. Test your app with real users.
8. Deploy and share your app.
9. Improve your app and get feedback.

Ref: https://solutions.appsheet.com/how-to-create-an-app

Early 2020, Google acquired AppSheet, an eight-year-old no-code mobile-application-building platform.  With AppSheet,  Google  gets a simple way for companies to build mobile apps without having to write a line of code. It works by pulling data from a spreadsheet, database or form, and using the field or column names as the basis for building an app.

In an essence, AppSheet will be a game changing business model for anyone (including young student) to be a seamless entrepreneur with zero capital investment and rapid time to market.  R u ready!

Right Sizing Instances

Rightsizing starts with monitoring and analyzing your current services being used. In fact, Rightsizing tops the list of cost optimization four pillars

An observation period of at least two weeks or maybe even a month will give you sufficient information on the instance performance and usage patterns while also showing you the peak of your workload.

The metrics that define instance performance include:

1. vCPU utilization
2. Memory utilization
3. Network utilization
4. Disk usage

Right-sizing functionality is available as a native feature of AWS Compute Optimizer. Details here: https://aws.amazon.com/compute-optimizer/

Convertible Reserved Instance

 

Let’s say you’ve already purchased a Convertible RI and have since rightsized your instances. You can’t technically downsize a Convertible RI — the instance type that you convert to has to be of equal or greater value.

Trick is depicted in the given diagram.

First, convert your Convertible RI to t2.nano Linux RIs, which is the smallest and cheapest type of RI. It may mean that you’re converting your RI into dozens or even hundreds of t2.nano RIs.

Then from there, you can convert the t2.nano RIs to whatever RI type you wish.

Essentially, converting your larger RI to lots of small RIs is like converting a $100 bill into $1 bills, giving you the flexibility to trade up to a $10 bill, two $20 bills, and a $50 bill. You’re still spending the same amount (which gets around the Convertible RI limit). But you get to trade down to a smaller instance type and have leftover “change” to apply to another instance. 

 

SQS & CloudWatch

Amazon SQS (Simple Queue Service) is a managed queue component from AWS cloud service.  Amazon CloudWatch enables you to collect, access, and correlate this data on a single platform from across all your AWS resources, applications, and services.  User can view and analyze your queues' metrics from the Amazon SQS console, the CloudWatch console, using the AWS CLI, or using the CloudWatch API.

The combination of SQS and CloudWatch is powerful in terms of stability, scalability, resilency, performance, etc.  It is achievable by setting CloudWatch alarms for Amazon SQS metrics.

CloudWatch metrics for your Amazon SQS queues are automatically collected and pushed to CloudWatch at one-minute intervals. These metrics are gathered on all queues that meet the CloudWatch guidelines for being active. CloudWatch considers a queue to be active for up to six hours if it contains any messages or if any action accesses it.

CORS

 

Access-Control-Allow-Origin specifies either a single origin, which tells browsers to allow that origin to access the resource; or else — for requests without credentials — the " * " wildcard, to tell browsers to allow any origin to access the resource.

Cross-Origin Resource Sharing (CORS) is a mechanism that uses additional HTTP headers to tell browsers to give a web application running at one origin, access to selected resources from a different origin. A web application executes a cross-origin HTTP request when it requests a resource that has a different origin (domain, protocol, or port) from its own.

In AWS, S3 bucket is configured to allow cross-origin requests by adding CORS configuration to the bucket.

AWS XRay

“What gets measured, gets managed “ — Peter Drucker

AWS X-Ray is a service that collects data about requests of any underlying application. It also provides tools to view, filter, and gain insights into that data to identify issues and opportunities for optimization.

This week, my blog post is hosted in medium site at https://medium.com/trimble-maps-engineering-blog/x-ray-latency-measure-f73961b08959

Happy Blogging !

Everest survey succeeds with Trimble

 

Trimble is pioneer in GPS technology with market presence since mid 1980s.  As the industry leader, it generates a revenue of close to 4 billion USD per annum.

Trimble R10 GNSS System collects more accurate data faster and easier – no matter what the job or the environment.  As a proud testimony, Nepal government has completed fieldwork for measuring Everest’s height using Trimble's equipment.

Proud to be Trimble-ian !

Ref: https://www.gpsworld.com/everest-survey-succeeds-with-trimble-gnss/

AWS PO Preview

 

As part of the procure-to-pay process, AWS customers leverage Purchase Orders (“POs”) to procure AWS services and approve invoices for payment.  The current limitation is not able to track their PO information to ensure that valid POs are used for their invoices.  It can be maually achieved by connecting AWS Support service team.

Last week, AWS introduced Purchase order (preview) feature in AWS console app.  It allows the customers to configure multiple POs, define how they are mapped to invoices, and access invoices generated against their POs.  Now, the customer can manage PO status, track PO balance and expiration, and configure contacts to receive email notifications for PO expiration and balance depletion.

With the new PO Management capabilities, customers will be able to exercise greater control over their PO information, and leverage their procure-to-pay process more effectively with AWS.

Shift Left Security

In the current Covid situations, every activities (school, college, business, etc.) are online.  Risk is around web security breaches; it's too costly to resolve.

Shift Left Security is recent buzz word in the industry.  Objective is to build the pro-active security check from the development phase to the production move.

Bit-bucket blog ref: https://bitbucket.org/blog/shift-left-with-code-insights-for-bitbucket-cloud

Launch Template

AWS Launch Template is more advanced than Launch Configuration. 

Based on my work experience, it has been listed as below:

1. Version Management

It is possible to create one or more numbered launch template versions. Each version can have different launch parameters, as depicted in the above diagram.  But Launch configuration needs manual maintenance. 

2. Leverage the latest features

Launch template can leverage the latest features of Amazon EC2, such as T2 Unlimited instances.

3. EC2 Launch Model

Launch configuration creates EC3 instances using the given Auto Scaling group, where as launch templates enable you to store launch parameters.

4.Subnet collection

In terms of subnet configuration script, launch template specifies exactly one subnet, rather than a list used in launch configuration.

Graduation 2020

You are very unusual batch for last 100 years with pandemic and anxiety !! When you born, world defied 9/11.

Many countries have introduced support measures to protect jobs in the near-term in hard-hit sectors, but young workers in particular are vulnerable.

Life taught "Victory comes from finding opportunities in problems".

Be an Optimist. You have the power to stand for and fight for and vote for healthier conditions that will create a healthier society.

In spite of Covid-19 challenges, it is highly motivating and encouraging words to 2020 graduates from Oprah Winfrey, Sundar Pichai, Bill Gates, Malala Yousafzai, Barack Obama, Dr. Anthony Fauci, Megan Rapinoe, Tim Cook, LeBron James and more.

Dear 2020 graduating kids, time will make you to realize that the progress of one generation becomes the foundational premise for the next. My generation growth is due to high dedication of national political leaders.

I'm sure you will make the history for your next generation. God bless you, all !

Ref:
https://www.linkedin.com/pulse/graduation-2020-ganesan-senthilvel/