AWS ECS idempotency

Amazon Elastic Container Service (Amazon ECS) now supports idempotency for task launches, allowing you to safely retry task launches without side effects. This feature helps ensure that timeouts or connection errors do not result in the launch of more instances than you originally intended, saving you time and money.

Idempotent operations allow requests to be retried with no additional side effects. You can now ensure that at most the desired number of tasks are launched as part of a RunTask API request by adding the following input to the request —client-token abcd. 

With an idempotent request, after requested tasks are successfully launched, subsequent retries using the same client token “abcd” within the same Amazon ECS cluster will not launch any additional tasks. When you use the AWS SDK or the AWS Management Console, a client token is automatically generated and added to RunTask requests (and used in any subsequent retries) if you do not pass one explicitly. 

Amazon ECS now supports idempotency for task launches on AWS Fargate, Amazon EC2 instances, and Amazon ECS Anywhere in all AWS regions. To get started with adding idempotency to RunTask API requests, see the ECS documentation at https://docs.aws.amazon.com/AmazonECS/latest/APIReference/ECS_Idempotency.html

AWS MSK replicator

Last week, AWS introduced MSK Replicator, a new capability of Amazon MSK that makes it easier to reliably set up cross-Region and same-Region replication between MSK clusters, scaling automatically to handle your workload.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) provides a fully managed and highly available Apache Kafka service simplifying the way you process streaming data. When using Apache Kafka, a common architectural pattern is to replicate data from one cluster to another.

Cross-cluster replication is often used to implement business continuity and disaster recovery plans and increase application resilience across AWS Regions. Another use case, when building multi-Region applications, is to have copies of streaming data in multiple geographies stored closer to end consumers for lower latency access. It is possible to aggregate data from multiple clusters into one centralized cluster for analytics.

To address these needs, it is required to write custom code or install and manage open-source tools like MirrorMaker 2.0, available as part of Apache Kafka starting with version 2.4. However, these tools can be complex and time-consuming to set up for reliable replication, and require continuous monitoring and scaling.

Ref: https://www.infoq.com/news/2023/10/amazon-msk-replicator/ 

Amazon RDS Multi AZ

Amazon Relational Database Service (Amazon RDS) PostgreSQL Multi-AZ deployments with two readable standbys now supports major version upgrades. 

Starting this week, RDS PostgreSQL is possible to upgrade multi-AZ deployments with two readable standbys from major version 13.4 and above and 14.5 and above to 15.4 with just a few clicks on the AWS Management Console.

PostgreSQL 15.4 offers several performance and feature benefits that include SQL standard "MERGE" command for conditional SQL queries, performance improvements for both in-memory and disk-based sorting, and support for two-phase commit and row/column filtering for logical replication. 

PostgreSQL 15 release also adds support for new extension pg_walinspect, and server-side compression with Gzip, LZ4, or Zstandard (zstd) using pg_basebackup. 

Amazon Bedrock

Pinecone is now available (private preview) as a Knowledge Base for Amazon Bedrock, a fully managed service from Amazon Web Services (AWS) for building GenAI applications.

With this release, we can quickly and effortlessly integrate your enterprise data when building search and GenAI applications. This workflow is called Retrieval Augmented Generation (RAG), and with Pinecone, it aids in providing relevant, accurate, and fast responses from search or GenAI applications to end users. 

With Pinecone as a Knowledge Base for Amazon Bedrock, GenAI applications are ready to build with:

• Highly performant: Speed through data in milliseconds. Leverage metadata filters and sparse-dense index support for top-notch relevance, ensuring quick, accurate, and grounded results across diverse search tasks.
• Cost effective at scale: Start for free on the starter plan and seamlessly scale usage with transparent usage-based pricing. Add or remove resources to meet your desired capacity and performance, upwards of billions of embeddings.
• Enterprise ready: Launch, use, and scale your AI solution without needing to maintain infrastructure, monitor services, or troubleshoot algorithms. Pinecone meets the security and operational requirements of enterprises.

ImageBlind

Meta engineering team released a paper with the model named ImageBlind, which can bind 6 modalities. It outperforms prior specialist models trained for a particular modality.

It joins data from:

• - Images
• - Text
• - Audio
• - Depth
• - Thermal
• - IMU data

It uses large-scale vision-language models and extends its zero-shot capabilities to new modalities just by using their natural pairing with images, such as video audio and image-depth data, to learn a single joint embedding space.

The paper shows that not all combinations of paired data are required to train a joint embedding, but only image-paired data is sufficient to bind the modalities together.

Ref: https://lnkd.in/gXNndk63

Harvard AI/ML free course

 

6 free online courses by Harvard University, in ML, AI, and Data Science.

◼️ 𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐭𝐨 𝐀𝐫𝐭𝐢𝐟𝐢𝐜𝐢𝐚𝐥 𝐈𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 𝐰𝐢𝐭𝐡 𝐏𝐲𝐭𝐡𝐨𝐧

Link: https://lnkd.in/gygaeAcY

◼️ 𝐃𝐚𝐭𝐚 𝐒𝐜𝐢𝐞𝐧𝐜𝐞: 𝐌𝐚𝐜𝐡𝐢𝐧𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠

Link: https://lnkd.in/gUNVYgGB

◼️ 𝐇𝐢𝐠𝐡-𝐝𝐢𝐦𝐞𝐧𝐬𝐢𝐨𝐧𝐚𝐥 𝐝𝐚𝐭𝐚 𝐚𝐧𝐚𝐥𝐲𝐬𝐢𝐬

Link: https://lnkd.in/gv9RV9Zc

◼️ 𝐒𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐬 𝐚𝐧𝐝 𝐑

Link: https://lnkd.in/gUY3jd8v

◼️ 𝐂𝐨𝐦𝐩𝐮𝐭𝐞𝐫 𝐒𝐜𝐢𝐞𝐧𝐜𝐞 𝐟𝐨𝐫 𝐁𝐮𝐬𝐢𝐧𝐞𝐬𝐬 𝐏𝐫𝐨𝐟𝐞𝐬𝐬𝐢𝐨𝐧𝐚𝐥𝐬

Link: https://lnkd.in/g8gQ6N-H

◼️ 𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐭𝐨 𝐏𝐫𝐨𝐠𝐫𝐚𝐦𝐦𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐏𝐲𝐭𝐡𝐨𝐧

Link: https://lnkd.in/gAdyf6xR

From Introductory to Intermediate, great for beginners.

LangChain experimentation

Last month, LangChain experimentation was published at my company's blog at https://developer.trimblemaps.com/engineering-blog/experimenting-with-langchain-ai/