Cloud Logging

Last week, I noticed new buzz word 'Cloud Logging' at our architect Ambal desk. This blog is about that.

Trust is not only a cloud issue. Alan Murphy points out that to get the benefits of clouds, one has to trust the providers for certain things, but doing so continues a trend in technology

Clouds will need different audit models than do traditional data centers. The diagram shows physical servers onto which the virtual machine instances (VMI) may map. As each VMI generates a loggable event, typically using calls to syslog or snmp, the physical server inserts a time stamp from a trusted (i.e., cryptographically signed) network time protocol server, and then transmits multiple copies of the log events to distributed master logs within the provider infrastructure. At those locations, software and servers sort the log events by VMI and customer, and create viewable, secure logs that the customers can audit. This design is towards LaaS (Logging As A Service)

Alternatively, in a community cloud, independent auditors can apply suitable tests to the customer data extractors and certify them, perhaps for a given digitally signed version of the code. This design approach does assume that the VMI-hypervisor is trusted; there have been some experimental side channel attacks from one VMI to another. This is an area, especially when there are legal requirements to demonstrate due diligence, in which recognized expert help may be needed.

Apple iCloud

iCloud stores your music, photos, apps, calendars, documents, and more at cloud. And wirelessly

pushes them to all your devices — automatically. It’s the easiest way to manage your content.

iCloud is so much more than a hard drive in the sky. It’s the effortless way to access just about everything on all your devices. iCloud stores your content so it’s always accessible from your iPad, iPhone, iPod touch, Mac, or PC. It gives you instant access to your music, apps, latest photos, and more. And it keeps your email, contacts, and calendars up to date across all your devices. No syncing required. No management required. In fact, no anything required. iCloud does it all for you.

When you sign up for iCloud, you automatically get 5GB of free storage. And that’s plenty of room, because of the way iCloud stores your content. Your purchased music, apps, books, and TV shows, as well as your Photo Stream, don’t count against your free storage. Since your mail, documents, Camera Roll, account information, settings, and other app data don’t use as much space, you’ll find that 5GB goes a long way. And if you need more storage, you can easily purchase a storage upgrade right from your device.

Amazon Cloud Player

Interesting product from Amazon; itz nothing but Amazon Cloud Player. Amazon has made its entry into the music streaming world with Amazon Cloud Player. Rather than stream a library of predetermined music, Cloud Player lets you upload your existing music library and stream it from any computer or Android device.

Amazon Cloud Player is a browser based application that supports Mac and PC computers and iPad devices. Cloud Player is not optimized to run on some mobile phones or tablets including: iPhones, Blackberrys, and Windows Mobile devices. For Android phones and tablets, itz recommended to install the Amazon MP3 app for Android which includes Amazon Cloud Player for Android.

You can upload your existing music library to Amazon Cloud Drive so all of your music is stored in one place and accessible from anywhere. All Amazon accounts have access to 5GB of free Cloud Drive storage for uploading content. Additional storage is available for an annual fee.

Amazon has thrown down the gauntlet and set a high bar for cloud-based music streaming. Apple and Google, which are expected to launch their own cloud players sometime this year, will have to match Amazon on usability and price if they’re going to compete.

Amazon can’t rest on its laurels though; Apple will surely harness its control of the iPhone, iTunes and iOS to boost its own offering and give the shopping giant a run for its money.

Cloud Security

Cloud computing security (sometimes referred to simply as "cloud security") is an evolving sub-domain of computer security, network security, and, more broadly, information security. It refers to a broad set of policies, technologies, and controls deployed to protect data, applications, and the associated infrastructure of cloud computing. Cloud security is not to be confused with security software offerings that are "cloud-based" (a.k.a. security-as-a-service).

Many commercial software vendors have offerings such as cloud-based anti-virus or vulnerability management.Third-party cloud computing represents the promise of outsourcing as applied to computation. Services, such as Microsoft’s Azure and Amazon’s EC2, allow users to instantiate virtual machines (VMs) on demand and thus purchase precisely the capacity they require when they require it. In turn, the use of virtualization allows third-party cloud providers to maximize the utilization of their sunk capital costs by multiplexing many customer VMs across a shared physical infrastructure.

In the emerging cloud security space, Craig Balding laid the foundation for cloudsecurity.org (http://cloudsecurity.org/) This site is for people that want to learn more about cloud computing from an enterprise security perspective. Lot of interesting topic, discussion, blog are available and so the interested folks can fetch the benefit out of it

Cloud Types

So far, we had an overview of microsoft cloud storage and its types. Letz focus on 3 different types of Cloud. They are:

1. Public Cloud
2. Private Cloud
3. Hybrid Cloud

Public Cloud

Cloud storage means different things to different people depending on how it’s implemented. The most common implementation is a ‘public cloud’, which is essentially storage capacity accessed through the internet or a wide area network (WAN) connection, and purchased on an as-needed basis. Users can expand capacity almost without limit, by contacting the provider, which typically operates a highly scalable storage infrastructure, sometimes in physically dispersed locations. Itz from an off-site third-party provider who shares resources and bills on a fine-grained utility computing.

Private Cloud

Private cloud (also called internal cloud or corporate cloud) is a marketing term for a proprietary computing architecture that provides hosted services to a limited number of people behind a firewall. Advances in virtualization and distributed computing have allowed corporate network and datacenter administrators to effectively become service providers that meet the needs of their "customers" within the corporation. Marketing media that uses the words "private cloud" is designed to appeal to an organization that needs or wants more control over their data than they can get by using a third-party hosted service such as Amazon's Elastic Compute Cloud (EC2) or Simple Storage Service (S3) or Windows Azure.

Hybrid Cloud

A hybrid cloud environment consisting of multiple internal and/or external providers will be typical for most enterprises

Azure Queue

In general, Queue stores messages that may be read by any client who has access to the storage account. Azure queue is a FIFO queue, so when you peek/get a message, it returns the first message (earliest message by time) that was inserted into the queue, problem is this is not guaranteed, so you need to carefully plan your application.

When a new consumer requests a message in Azure queue, the next available visible message is sent to it. In case a consumer asks for a 30sec processing time, but fails to delete the message during this time, the message is marked visible again by the queue and is made available to the next consumer. If the consumer tries to delete a message after the timeout, the queue checks the ‘popreceipt’ to see if it indeed is the most recent requester of the message, if not an exception is thrown.

Azure queue can contain an unlimited number of messages, each of which can be up to 8 KB in size. Messages are generally added to the end of the queue and retrieved from the front of the queue, although first in, first out (FIFO) behavior is not guaranteed.

Windows Azure Queue provides a reliable message delivery mechanism. It provides a simple and asynchronous work dispatch mechanism, which can be used to connect different components of a cloud application. The Windows Azure Queues are highly available, durable and performance efficient. Its programming semantics ensure that a message can be processed at least once. Furthermore, Windows Azure Queue has a REST interface, so that applications can be written in any language and they can access the queue at anytime from anywhere in the Internet via the web.

Azure Table

Windows Azure Table storage causes a lot of confusion/query among developers becaz of their traditional RDBMS work experience. Most of their experience with data storage is with relational databases that have various tables, each containing a predefined set of columns, one or more of which are typically designated as identity keys

By design, Windows Azure Table services provides the potential to store enormous amounts of data, while enabling efficient access and persistence. A table doesn’t have a specified schema. It’s simply a structured container of rows/entities either stores one particular type or rows with varying structures in a single table, as shown in the attached image. Windows Azure Tables use keys that enable efficient querying and load balancing when the table service decides time to spread your table over multiple servers. How?

In Windows Azure Tables, the string PartitionKey and RowKey properties work together as an index for the table, so when defining them, you must consider how your data is queried. Each entity in a table must have a unique PartitionKey/RowKey combination, acting as a primary key for the row. PartitionKey is used to fetch the result from query. Also used for physically partitioning the tables, which provides for load balancing and scalability. Letz watch an illustration.

Letz considern EmployeeMaster and has PartitionKeys that correspond to job designation types, such as Director, Manager, Developer, etc. During DeveloperSummit, the rows in Developer partition might be very busy (hope of having niche developers!). The service can load balance EmployeeMaster table by moving the Developer partition to a different server for better handling the many requests made to that partition. If you anticipate more activity on that partition than a single server can handle, you should consider creating more-granular partitions such as DeveloperJunior and DeveloperSenior. This is because the unit of granularity for load balancing is the PartitionKey.

Isn't it Cool for load balancing and scalability? Cloud storage developers just deal with data, data, data.. BigData !